Convert BDV h5 to n5

Hi,
I cannot get the h5 to be read.
initialize_dataset('/g/emcf/Hamburg_XRay/20200515/20x/TAH01_parz.h5',"/t00000/s00/0/cells",ROOT,'test','X-Ray-r
    ...: aw',[0.1,0.1,0.1],DEFAULT_CHUNKS,scale_factors,is_default=0,target='local',max_jobs=16)
DEBUG: Checking if DownscalingWorkflow(tmp_folder=tmp_test, max_jobs=16, config_dir=tmp_test/configs, target=local, dependency=DummyTask, input_path=/g/emcf/Hamburg_XRay/20200515/20x/TAH01_parz.h5, input_key=./t00000/s00/0/cells, scale_factors=[[2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2]], halos=[[2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2]], metadata_format=bdv.n5, metadata_dict={"resolution": [0.1, 0.1, 0.1], "unit": "micrometer"}, output_path=./data/test/images/local/X-Ray-raw.n5, output_key_prefix=, force_copy=False, skip_existing_levels=False, scale_offset=0) is complete
/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/miniconda/envs/covid-em-dev/lib/python3.7/site-packages/luigi/parameter.py:279: UserWarning: Parameter "dtype" with value "None" is not of type string.
  warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value))
/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/miniconda/envs/covid-em-dev/lib/python3.7/site-packages/luigi/parameter.py:279: UserWarning: Parameter "scale_factor" with value "(2, 2, 2)" is not of type string.
  warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value))
DEBUG: Checking if WriteDownscalingMetadata(tmp_folder=tmp_test, output_path=./data/test/images/local/X-Ray-raw.n5, scale_factors=[[2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2]], dependency=DownscalingLocal, metadata_format=bdv.n5, metadata_dict={"resolution": [0.1, 0.1, 0.1], "unit": "micrometer"}, output_key_prefix=, scale_offset=0, prefix=downscaling) is complete
INFO: Informed scheduler that task   DownscalingWorkflow_tmp_test_configs_DummyTask_False_3134488336   has status   PENDING
DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_test, max_jobs=16, config_dir=tmp_test/configs, input_path=./data/test/images/local/X-Ray-raw.n5, input_key=setup0/timepoint0/s5, output_path=./data/test/images/local/X-Ray-raw.n5, output_key=setup0/timepoint0/s6, scale_factor=(2, 2, 2), scale_prefix=s6, halo=[2, 2, 2], effective_scale_factor=[64, 64, 64], dependency=DownscalingLocal) is complete
INFO: Informed scheduler that task   WriteDownscalingMetadata_DownscalingLocal___resolution_____bdv_n5_76f10fbef3   has status   PENDING
DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_test, max_jobs=16, config_dir=tmp_test/configs, input_path=./data/test/images/local/X-Ray-raw.n5, input_key=setup0/timepoint0/s4, output_path=./data/test/images/local/X-Ray-raw.n5, output_key=setup0/timepoint0/s5, scale_factor=(2, 2, 2), scale_prefix=s5, halo=[2, 2, 2], effective_scale_factor=[32, 32, 32], dependency=DownscalingLocal) is complete
INFO: Informed scheduler that task   DownscalingLocal_tmp_test_configs_DownscalingLocal__64__64__64__3d9aba8d28   has status   PENDING
DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_test, max_jobs=16, config_dir=tmp_test/configs, input_path=./data/test/images/local/X-Ray-raw.n5, input_key=setup0/timepoint0/s3, output_path=./data/test/images/local/X-Ray-raw.n5, output_key=setup0/timepoint0/s4, scale_factor=(2, 2, 2), scale_prefix=s4, halo=[2, 2, 2], effective_scale_factor=[16, 16, 16], dependency=DownscalingLocal) is complete
INFO: Informed scheduler that task   DownscalingLocal_tmp_test_configs_DownscalingLocal__32__32__32__d8449b3a29   has status   PENDING
DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_test, max_jobs=16, config_dir=tmp_test/configs, input_path=./data/test/images/local/X-Ray-raw.n5, input_key=setup0/timepoint0/s2, output_path=./data/test/images/local/X-Ray-raw.n5, output_key=setup0/timepoint0/s3, scale_factor=(2, 2, 2), scale_prefix=s3, halo=[2, 2, 2], effective_scale_factor=[8, 8, 8], dependency=DownscalingLocal) is complete
INFO: Informed scheduler that task   DownscalingLocal_tmp_test_configs_DownscalingLocal__16__16__16__8af7d887e6   has status   PENDING
DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_test, max_jobs=16, config_dir=tmp_test/configs, input_path=./data/test/images/local/X-Ray-raw.n5, input_key=setup0/timepoint0/s1, output_path=./data/test/images/local/X-Ray-raw.n5, output_key=setup0/timepoint0/s2, scale_factor=(2, 2, 2), scale_prefix=s2, halo=[2, 2, 2], effective_scale_factor=[4, 4, 4], dependency=DownscalingLocal) is complete
INFO: Informed scheduler that task   DownscalingLocal_tmp_test_configs_DownscalingLocal__8__8__8__af2167bc66   has status   PENDING
DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_test, max_jobs=16, config_dir=tmp_test/configs, input_path=./data/test/images/local/X-Ray-raw.n5, input_key=setup0/timepoint0/s0, output_path=./data/test/images/local/X-Ray-raw.n5, output_key=setup0/timepoint0/s1, scale_factor=(2, 2, 2), scale_prefix=s1, halo=[2, 2, 2], effective_scale_factor=[2, 2, 2], dependency=CopyVolumeLocal) is complete
INFO: Informed scheduler that task   DownscalingLocal_tmp_test_configs_DownscalingLocal__4__4__4__256b7017d5   has status   PENDING
DEBUG: Checking if CopyVolumeLocal(tmp_folder=tmp_test, max_jobs=16, config_dir=tmp_test/configs, input_path=/g/emcf/Hamburg_XRay/20200515/20x/TAH01_parz.h5, input_key=('t00000', 's00', '0'), output_path=./data/test/images/local/X-Ray-raw.n5, output_key=setup0/timepoint0/s0, prefix=initial_scale, dtype=None, fit_to_roi=False, effective_scale_factor=[], dependency=DummyTask) is complete
INFO: Informed scheduler that task   DownscalingLocal_tmp_test_configs_CopyVolumeLocal__2__2__2__e3414c594d   has status   PENDING
DEBUG: Checking if DummyTask() is complete
INFO: Informed scheduler that task   CopyVolumeLocal_tmp_test_configs_DummyTask_None_9670f53ac9   has status   PENDING
INFO: Informed scheduler that task   DummyTask__99914b932b   has status   DONE
INFO: Done scheduling tasks
INFO: Running Worker with 1 processes
DEBUG: Asking scheduler for work...
DEBUG: Pending tasks: 9
INFO: [pid 28537] Worker Worker(salt=836893348, workers=1, host=vm-schwab-02.embl.de, username=schorb, pid=28537) running   CopyVolumeLocal(tmp_folder=tmp_test, max_jobs=16, config_dir=tmp_test/configs, input_path=/g/emcf/Hamburg_XRay/20200515/20x/TAH01_parz.h5, input_key=('t00000', 's00', '0'), output_path=./data/test/images/local/X-Ray-raw.n5, output_key=setup0/timepoint0/s0, prefix=initial_scale, dtype=None, fit_to_roi=False, effective_scale_factor=[], dependency=DummyTask)
ERROR: [pid 28537] Worker Worker(salt=836893348, workers=1, host=vm-schwab-02.embl.de, username=schorb, pid=28537) failed    CopyVolumeLocal(tmp_folder=tmp_test, max_jobs=16, config_dir=tmp_test/configs, input_path=/g/emcf/Hamburg_XRay/20200515/20x/TAH01_parz.h5, input_key=('t00000', 's00', '0'), output_path=./data/test/images/local/X-Ray-raw.n5, output_key=setup0/timepoint0/s0, prefix=initial_scale, dtype=None, fit_to_roi=False, effective_scale_factor=[], dependency=DummyTask)
Traceback (most recent call last):
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/miniconda/envs/covid-em-dev/lib/python3.7/site-packages/luigi/worker.py", line 191, in run
    new_deps = self._run_get_new_deps()
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/miniconda/envs/covid-em-dev/lib/python3.7/site-packages/luigi/worker.py", line 133, in _run_get_new_deps
    task_gen = self.task.run()
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/cluster_tools/cluster_tools/cluster_tasks.py", line 95, in run
    raise e
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/cluster_tools/cluster_tools/cluster_tasks.py", line 81, in run
    self.run_impl()
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/cluster_tools/cluster_tools/copy_volume/copy_volume.py", line 64, in run_impl
    ds = f[self.input_key]
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/miniconda/envs/covid-em-dev/lib/python3.7/site-packages/h5py/_hl/group.py", line 264, in __getitem__
    oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/miniconda/envs/covid-em-dev/lib/python3.7/site-packages/h5py/_hl/base.py", line 137, in _e
    name = name.encode('ascii')
AttributeError: 'tuple' object has no attribute 'encode'
DEBUG: 1 running tasks, waiting for next task to finish
INFO: Informed scheduler that task   CopyVolumeLocal_tmp_test_configs_DummyTask_None_9670f53ac9   has status   FAILED
DEBUG: Asking scheduler for work...
 Shutting down Keep-Alive thread
INFO:
===== Luigi Execution Summary =====

Scheduled 10 tasks of which:
* 1 complete ones were encountered:
    - 1 DummyTask()
* 1 failed:
    - 1 CopyVolumeLocal(...)
* 8 were left pending, among these:
    * 8 had failed dependencies:
        - 6 DownscalingLocal(...)
        - 1 DownscalingWorkflow(...)
        - 1 WriteDownscalingMetadata(...)

This progress looks :( because there were failed tasks

===== Luigi Execution Summary =====

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-27-b2a49fb11710> in <module>
----> 1 initialize_dataset('/g/emcf/Hamburg_XRay/20200515/20x/TAH01_parz.h5',"./t00000/s00/0/cells",ROOT,'test','X-Ray-raw',[0.1,0.1,0.1],DEFAULT_CHUNKS,scale_factors,is_default=0,target='local',max_jobs=16)

/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/mobie-utils-python/mobie/initialization.py in initialize_dataset(input_path, input_key, root, dataset_name, raw_name, resolution, chunks, scale_factors, is_default, add_remote, tmp_folder, target, max_jobs, time_limit)
     70     import_raw_volume(input_path, input_key, data_path,
     71                       resolution, scale_factors, chunks,
---> 72                       tmp_folder=tmp_folder, target=target, max_jobs=max_jobs)
     73
     74     add_to_image_dict(dataset_folder, 'image', xml_path, add_remote=add_remote)

/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/mobie-utils-python/mobie/import_data/raw.py in import_raw_volume(in_path, in_key, out_path, resolution, scale_factors, chunks, tmp_folder, target, max_jobs, block_shape)
     55     ret = luigi.build([t], local_scheduler=True)
     56     if not ret:
---> 57         raise RuntimeError("Importing raw data failed")

RuntimeError: Importing raw data failed

Looks like I run into the same issue when trying to import segmentations (h5 as well...)
$ python add_seg_to_dataset.py /g/emcf/common/5792_Sars-Cov-2/Exp_070420/FIB-SEM/segmentation/upscale/20-04-23_S4_area2_Sam/seg_dmv_scale1.0/run_200518_00_dmv_full_dataset.h5 t00000/s00/0 Covid19-S4-Area2 S4-Area2_DMV
DEBUG: Checking if DownscalingWorkflow(tmp_folder=tmp_S4-Area2_DMV, max_jobs=16, config_dir=tmp_S4-Area2_DMV/configs, target=local, dependency=DummyTask, input_path=/g/emcf/common/5792_Sars-Cov-2/Exp_070420/FIB-SEM/segmentation/upscale/20-04-23_S4_area2_Sam/seg_dmv_scale1.0/run_200518_00_dmv_full_dataset.h5, input_key=t00000/s00/0, scale_factors=[[2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2]], halos=[[2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2]], metadata_format=bdv.n5, metadata_dict={"resolution": null, "unit": "micrometer"}, output_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, output_key_prefix=, force_copy=False, skip_existing_levels=False, scale_offset=0) is complete
/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/miniconda/envs/covid-em-dev/lib/python3.7/site-packages/luigi/parameter.py:279: UserWarning: Parameter "dtype" with value "None" is not of type string.
  warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value))
/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/miniconda/envs/covid-em-dev/lib/python3.7/site-packages/luigi/parameter.py:279: UserWarning: Parameter "scale_factor" with value "(2, 2, 2)" is not of type string.
  warnings.warn('Parameter "{}" with value "{}" is not of type string.'.format(param_name, param_value))
DEBUG: Checking if WriteDownscalingMetadata(tmp_folder=tmp_S4-Area2_DMV, output_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, scale_factors=[[2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2], [2, 2, 2]], dependency=DownscalingLocal, metadata_format=bdv.n5, metadata_dict={"resolution": null, "unit": "micrometer"}, output_key_prefix=, scale_offset=0, prefix=downscaling) is complete
INFO: Informed scheduler that task   DownscalingWorkflow_tmp_S4_Area2_DMV_DummyTask_False_0eeea888bf   has status   PENDING
DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_S4-Area2_DMV, max_jobs=16, config_dir=tmp_S4-Area2_DMV/configs, input_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, input_key=setup0/timepoint0/s5, output_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, output_key=setup0/timepoint0/s6, scale_factor=(2, 2, 2), scale_prefix=s6, halo=[2, 2, 2], effective_scale_factor=[64, 64, 64], dependency=DownscalingLocal) is complete
INFO: Informed scheduler that task   WriteDownscalingMetadata_DownscalingLocal___resolution___n_bdv_n5_28203bf66a   has status   PENDING
DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_S4-Area2_DMV, max_jobs=16, config_dir=tmp_S4-Area2_DMV/configs, input_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, input_key=setup0/timepoint0/s4, output_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, output_key=setup0/timepoint0/s5, scale_factor=(2, 2, 2), scale_prefix=s5, halo=[2, 2, 2], effective_scale_factor=[32, 32, 32], dependency=DownscalingLocal) is complete
INFO: Informed scheduler that task   DownscalingLocal_tmp_S4_Area2_DMV_DownscalingLocal__64__64__64__428a2b9e15   has status   PENDING
DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_S4-Area2_DMV, max_jobs=16, config_dir=tmp_S4-Area2_DMV/configs, input_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, input_key=setup0/timepoint0/s3, output_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, output_key=setup0/timepoint0/s4, scale_factor=(2, 2, 2), scale_prefix=s4, halo=[2, 2, 2], effective_scale_factor=[16, 16, 16], dependency=DownscalingLocal) is complete
INFO: Informed scheduler that task   DownscalingLocal_tmp_S4_Area2_DMV_DownscalingLocal__32__32__32__13c891021f   has status   PENDING
DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_S4-Area2_DMV, max_jobs=16, config_dir=tmp_S4-Area2_DMV/configs, input_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, input_key=setup0/timepoint0/s2, output_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, output_key=setup0/timepoint0/s3, scale_factor=(2, 2, 2), scale_prefix=s3, halo=[2, 2, 2], effective_scale_factor=[8, 8, 8], dependency=DownscalingLocal) is complete
INFO: Informed scheduler that task   DownscalingLocal_tmp_S4_Area2_DMV_DownscalingLocal__16__16__16__31d6160896   has status   PENDING
DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_S4-Area2_DMV, max_jobs=16, config_dir=tmp_S4-Area2_DMV/configs, input_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, input_key=setup0/timepoint0/s1, output_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, output_key=setup0/timepoint0/s2, scale_factor=(2, 2, 2), scale_prefix=s2, halo=[2, 2, 2], effective_scale_factor=[4, 4, 4], dependency=DownscalingLocal) is complete
INFO: Informed scheduler that task   DownscalingLocal_tmp_S4_Area2_DMV_DownscalingLocal__8__8__8__b441be779d   has status   PENDING
DEBUG: Checking if DownscalingLocal(tmp_folder=tmp_S4-Area2_DMV, max_jobs=16, config_dir=tmp_S4-Area2_DMV/configs, input_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, input_key=setup0/timepoint0/s0, output_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, output_key=setup0/timepoint0/s1, scale_factor=(2, 2, 2), scale_prefix=s1, halo=[2, 2, 2], effective_scale_factor=[2, 2, 2], dependency=CopyVolumeLocal) is complete
INFO: Informed scheduler that task   DownscalingLocal_tmp_S4_Area2_DMV_DownscalingLocal__4__4__4__5b67630ce9   has status   PENDING
DEBUG: Checking if CopyVolumeLocal(tmp_folder=tmp_S4-Area2_DMV, max_jobs=16, config_dir=tmp_S4-Area2_DMV/configs, input_path=/g/emcf/common/5792_Sars-Cov-2/Exp_070420/FIB-SEM/segmentation/upscale/20-04-23_S4_area2_Sam/seg_dmv_scale1.0/run_200518_00_dmv_full_dataset.h5, input_key=t00000/s00/0, output_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, output_key=setup0/timepoint0/s0, prefix=initial_scale, dtype=None, fit_to_roi=False, effective_scale_factor=[], dependency=DummyTask) is complete
INFO: Informed scheduler that task   DownscalingLocal_tmp_S4_Area2_DMV_CopyVolumeLocal__2__2__2__31b70a5591   has status   PENDING
DEBUG: Checking if DummyTask() is complete
INFO: Informed scheduler that task   CopyVolumeLocal_tmp_S4_Area2_DMV_DummyTask_None_b45bcd3dbf   has status   PENDING
INFO: Informed scheduler that task   DummyTask__99914b932b   has status   DONE
INFO: Done scheduling tasks
INFO: Running Worker with 1 processes
DEBUG: Asking scheduler for work...
DEBUG: Pending tasks: 9
INFO: [pid 39217] Worker Worker(salt=767320602, workers=1, host=vm-schwab-02.embl.de, username=schorb, pid=39217) running   CopyVolumeLocal(tmp_folder=tmp_S4-Area2_DMV, max_jobs=16, config_dir=tmp_S4-Area2_DMV/configs, input_path=/g/emcf/common/5792_Sars-Cov-2/Exp_070420/FIB-SEM/segmentation/upscale/20-04-23_S4_area2_Sam/seg_dmv_scale1.0/run_200518_00_dmv_full_dataset.h5, input_key=t00000/s00/0, output_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, output_key=setup0/timepoint0/s0, prefix=initial_scale, dtype=None, fit_to_roi=False, effective_scale_factor=[], dependency=DummyTask)
ERROR: [pid 39217] Worker Worker(salt=767320602, workers=1, host=vm-schwab-02.embl.de, username=schorb, pid=39217) failed    CopyVolumeLocal(tmp_folder=tmp_S4-Area2_DMV, max_jobs=16, config_dir=tmp_S4-Area2_DMV/configs, input_path=/g/emcf/common/5792_Sars-Cov-2/Exp_070420/FIB-SEM/segmentation/upscale/20-04-23_S4_area2_Sam/seg_dmv_scale1.0/run_200518_00_dmv_full_dataset.h5, input_key=t00000/s00/0, output_path=./data/Covid19-S4-Area2/images/local/S4-Area2_DMV.n5, output_key=setup0/timepoint0/s0, prefix=initial_scale, dtype=None, fit_to_roi=False, effective_scale_factor=[], dependency=DummyTask)
Traceback (most recent call last):
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/miniconda/envs/covid-em-dev/lib/python3.7/site-packages/luigi/worker.py", line 191, in run
    new_deps = self._run_get_new_deps()
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/miniconda/envs/covid-em-dev/lib/python3.7/site-packages/luigi/worker.py", line 133, in _run_get_new_deps
    task_gen = self.task.run()
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/cluster_tools/cluster_tools/cluster_tasks.py", line 95, in run
    raise e
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/cluster_tools/cluster_tools/cluster_tasks.py", line 81, in run
    self.run_impl()
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/cluster_tools/cluster_tools/copy_volume/copy_volume.py", line 64, in run_impl
    ds = f[self.input_key]
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/miniconda/envs/covid-em-dev/lib/python3.7/site-packages/h5py/_hl/group.py", line 264, in __getitem__
    oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5o.pyx", line 190, in h5py.h5o.open
KeyError: 'Unable to open object (component not found)'
DEBUG: 1 running tasks, waiting for next task to finish
INFO: Informed scheduler that task   CopyVolumeLocal_tmp_S4_Area2_DMV_DummyTask_None_b45bcd3dbf   has status   FAILED
DEBUG: Asking scheduler for work...
 Shutting down Keep-Alive thread
INFO:
===== Luigi Execution Summary =====

Scheduled 10 tasks of which:
* 1 complete ones were encountered:
    - 1 DummyTask()
* 1 failed:
    - 1 CopyVolumeLocal(...)
* 8 were left pending, among these:
    * 8 had failed dependencies:
        - 6 DownscalingLocal(...)
        - 1 DownscalingWorkflow(...)
        - 1 WriteDownscalingMetadata(...)

This progress looks :( because there were failed tasks

===== Luigi Execution Summary =====

Traceback (most recent call last):
  File "add_seg_to_dataset.py", line 44, in <module>
    args.resolution, args.chunks, args.target, args.max_jobs)
  File "add_seg_to_dataset.py", line 21, in add_seg_to_dataset
    add_default_table=True)
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/mobie-utils-python/mobie/segmentation.py", line 48, in add_segmentation
    max_jobs=max_jobs)
  File "/g/emcf/common/5792_Sars-Cov-2/covid-em-datasets/software/mobie-utils-python/mobie/import_data/segmentation.py", line 95, in import_segmentation
    raise RuntimeError("Importing segmentation failed")
RuntimeError: Importing segmentation failed

I cannot reproduce this, the following code works for me:
from mobie import initialize_dataset

ROOT = '.'
path = '/g/emcf/Hamburg_XRay/20200515/20x/TAH01_parz.h5'
key = "/t00000/s00/0/cells"

scale_factors = [[2, 2, 2]]

initialize_dataset(path, key, ROOT,
                   'test', 'X-Ray-raw',
                   resolution=[0.1, 0.1, 0.1],
                   chunks=(64, 64, 64),
                   scale_factors=scale_factors,
                   is_default=1, target='local', max_jobs=16)

Note that I had to fix something in the downscaling logic for this to work, so you will need to update cluster tools in your env:
https://github.com/constantinpape/cluster_tools

@martinschorb
maybe there is an issue with the h5py version.
In my env I have
h5py                      2.9.0           nompi_py37hf008753_1102    conda-forge

OK, runs now.
I just did a git pull in the main directory, but in order to get the proper cluster tools had to do it in that subdirectory as well...


fix: make event passive to fix sluggish vertical scroll

Thank you for your contribution to the KodaDot NFT gallery.
👇 _ Let's make a quick check before the contribution.
PR type

[x] Bugfix
[ ] Feature
[ ] Refactoring

What's new?

[x] PR closes #4160
[ ] Requires deployment <rubick/magick_issue>
[x] Fixed an issue that prevented users from scrolling smoothly over Carousels on Firefox

Before submitting Pull Request, please make sure:

[ ] My contribution builds clean without any errors or warnings
[ ] I've merged recent default branch -- main and I've no conflicts
[ ] I've tried to respect high code quality standards
[ ] I've didn't break any original functionality
[ ] I've posted a screenshot of demonstrated change in this PR

Optional

[ ] I've tested it at </rmrk/collection/26902bc2f7c20c546a-1FVG7>
[ ] I've tested PR on mobile and everything seems works
[ ] I found edge cases
[ ] I've written some unit tests 🧪

Had issue bounty label?

[x] Fill up your KSM address: Payout

Community participation

[ ] Are you at KodaDot Discord?

Screenshot

[ ] My fix has changed something on UI; a screenshot is best to understand changes for others.

pay 20

😍 Perfect, I’ve sent the payout
💵 $20 @ 34.22 USD/KSM ~ 0.584 $KSM
🧗 EqdyzrzVmeHwMdMwvPeCMnNdbuQDbD3YrjY93xq9Ln3jUGW
🔗 0xf48536a5a074378fa5bdc63286a6fc1307f04c768f50fcf521e3b1c1370119fd
🪅 Let’s grab another issue and get rewarded!
🪄 github.com/kodadot/nft-gallery/issues


{Network} az network public-ip prefix: Bump up api-version and add param  --ip-tags

Related command

Description

Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change
[Component Name 2] az command b: Add some customer-facing feature

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

[ ] I adhere to the Error Handling Guidelines.

Network


Prevent removal of unused import with comment

Does the source.unusedImports action support keeping an unused import tagged with a comment? If so what does it support? For example we have pytest fixtures that we import which are flagged as unused
from .mocks import unstub_and_verify_fixture  # pylint: disable=unused-import

If I do remove all unused imports that import gets removed.  I also tried using # noqa that some other linters support.

Pylance isn't a linter (like pylint or ruff). Pylance is built on a type checker called pyright. The types of conditions that linters and type checkers detect are mostly distinct, but there is some overlap, such as with unused imports.
For type checkers, there is a standard way (defined in PEP 484) to suppress an error on a line. You use the comment # type: ignore. Pyright also supports # pyright: ignore. And you can use the latter with a specific error code as well (# pyright: ignore[reportUnusedImport]).

# type: ignore ... # pyright: ignore ...# pyright: ignore[reportUnusedImport]

Of these three, only # pyright: ignore worked for me in this scenario.
If I'm in strict mode I get a diagnostic and they all work to suppress the diagnostic. But it looks like the "unused imports" action looks for DiagnosticCategory.UnusedCode hints rather than "true" diagnostics, and I'm guessing that's the reason for the difference.

The reportUnusedImport diagnostic check is enabled by default only in strict type checking mode. If type checking is disabled or in "basic" mode, you should never see a diagnostic for this condition.
You will see the symbol "grayed out", but that's expected and should occur regardless of the reportUnusedImport setting. If the grayed-out text really bothers you, you could add an explicit (dummy) reference to the symbol or use a redundant import form.

If the grayed-out text really bothers you, you could add an explicit (dummy) reference to the symbol or use a redundant import form.

The user's request was some way to prevent the "Remove all unused imports" action from removing certain imports that are used but not in a way that Pylance recognizes. Specifically, in pytest if a test function has an argument with the same name as a fixture, that fixture will be automatically passed in at test-time, but we don't see it as a reference.
Using # pyright: ignore was actually suggested as a solution for this in https://github.com/microsoft/pylance-release/discussions/4083.
@rchiodo, both this issue and  https://github.com/microsoft/pylance-release/discussions/4083 are about inadvertently removing pytest fixture imports. Maybe the pytest support in Pylance could be extended to prevent "Remove all unused imports" from removing these imports?

Maybe? That would likely require special code in the Remove unused imports action to special case this scenario.

The # pyright : ignore solution is fine with me

The # pyright : ignore solution is fine with me

This behavior is unintended and due to a bug that will be fixed in the next release. It's not a solution to this problem.

Moving this issue to discussion as an enhancement request for comments and upvotes.


Raspi 4 v0.99.6 disconnect every 30 mins or so

Hi,
I left speedify run overnight, and I notice it gets auto disconnected or so on every 30 mins or close to it.
I have two modem 5G hooks up to it, and they are online just fine; the reason I know is I set the speed test on my nas every 30 mins throughout the day, and when speedify auto disconnects, the speed test is still run from one of the two interfaces.
Here are the logs I got in the morning; I turned them on and 30 mins later disconnected again.
[REDACTED - irrelevant]

Try to connect to "Torrent friendly server" to see if Speedify is disconnecting due to P2P traffic.

To generate Speedify logs, do tar -zcvf /www/spdlog.tar.gz /usr/share/speedify/logs/
Visit http://<IP_ADDRESS>/spdlog.tar.gz to download and upload it here.

spdlog.tar.gz
here is it

Torrenting is detected.
You can either use a torrenting server, or another country that has the ability to block torrents as opposed to servers that disconnect, I'm not aware on how to segregate these.
You can also use VPN on top of VPN, it's preferred to set Speedify transport mode to UDP mode and use a UDP VPN.

probably I'm going to run the wireguard client on the nas server.
been testing alot lately with smoothwan, and one thing i notice, combining too wans do make speed slower.
For example, each wan individual when i test up to 200 to 300 Mbps but when run through smoothwan, i only get around 150ish max on both, regardless which mode i select or protocol.
I might just be running this through the flint instead just 1 wan and run wireguard client on it. Instead of doing speedify

On May 15, 2022, at 12:24 PM, Talal @.***> wrote:
https://user-images.githubusercontent.com/96490382/168483102-ca3d68fc-8152-4609-b51c-2a04f4491fec.png
Torrenting is detected.
You can either use a torrenting server, or another country that has the ability to block torrents as opposed to servers that disconnect, I'm not aware on how to segregate these.
You can also use VPN on top of VPN, it's preferred to set Speedify transport mode to UDP mode and use a UDP VPN.
—
Reply to this email directly, view it on GitHub https://github.com/TalalMash/SmoothWAN/issues/36#issuecomment-1126973574, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACDDGWFBT7CAG6SJH5ORZA3VKEQMZANCNFSM5V7IBC3A.
You are receiving this because you authored the thread.

I included the speeds in the README page for each device, 200-300 each is way too high for a Pi4 to handle, try disabling encryption although the difference will be minimal. The flint will be much slower and the R4S slightly faster.
x86 hardware may be the only option e.g Intel J3060 Mini PC (~$150+).

You can also discuss the performance for the NanoPi R4S in it's issue page, you can overclock to 2GHz (safe max) to check, there is one testing report left regarding AES hardware acceleration.

What about r4s?
I might het the intel j3060 mini pc to test today.

On May 15, 2022, at 12:46 PM, Talal @.***> wrote:

I included the speeds in the README page for each device, 200-300 each is way too high for a Pi4 to handle, try disabling encryption although the difference will be minimal. The flint will be much slower and the R4S slightly faster.
x86 hardware may be the only option e.g Intel J3060 Mini PC (~$150+).
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

Yes, i coundnt get it online. I posted in it issues, can you take a look. Want to test AES for you since all my 3 wans combined over 1G

On May 15, 2022, at 1:02 PM, Jimmy Le @.***> wrote:

What about r4s?
I might het the intel j3060 mini pc to test today.

On May 15, 2022, at 12:46 PM, Talal @.***> wrote:

I included the speeds in the README page for each device, 200-300 each is way too high for a Pi4 to handle, try disabling encryption although the difference will be minimal. The flint will be much slower and the R4S slightly faster.
x86 hardware may be the only option e.g Intel J3060 Mini PC (~$150+).
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

Intel J3060

To be on the safe side, I'd recommend something with more power than this baseline family, try to avoid Braswell series.

What about r4s? I might het the intel j3060 mini pc to test today.
…
On May 15, 2022, at 12:46 PM, Talal @.***> wrote:  I included the speeds in the README page for each device, 200-300 each is way too high for a Pi4 to handle, try disabling encryption although the difference will be minimal. The flint will be much slower and the R4S slightly faster. x86 hardware may be the only option e.g Intel J3060 Mini PC (~$150+). — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

To be on the safe side, I'd recommend something with more power than this baseline family, try to avoid Braswell series.
An i3 based Qotom should do well, includes multiple NICs as a bonus.


Set a precision for the float numbers in the geojson file

Summary
Fix a precision number for the creation lat-lon coordinates that are dumped into the geojson file.
This will avoid different values when running in different machines. In particular it will unblock #55
Proposal
Using 7 decimals we are targeting a precision of 0.0001 m (111.139  * 1e-7 =~0.0001 )
Originally posted by @francocipollone in https://github.com/maliput/maliput_xodr/issues/55#issuecomment-1798432010

I'm totally fine with the proposal since it's the simplest fix. Below are some random thoughts.

Auto-select the number of decimal places based on Maliput's inertial position tolerance. Choose the smallest number of decimal places to achieve a precision smaller than this tolerance.
Make the number of decimal places user-specifiable, with 7 decimal places as the default.


[DS-2707] Poor messaging when batch upload directory cannot be created

https://jira.duraspace.org/browse/DS-2707
Throw an exception if a working directory cannot be created, report it in UI.
Add lots of debug logging, including duplication of some console output that of course won't appear when running in a webapp.
Also realigned some nearby code that was hard to read.

I could take the debug logging out again.  Normally it won't produce any output.  Leaving it in would cost less if we switched to an slf4j Logger with parameterized messages.  The class really needs revamping to make it just throw things and/or provide messages to callback methods rather than writing to System.out, but that's a subject for another ticket.

+1 by review


RFP - madVR

madVR is a high quality video renderer (GPU assisted). Both 32bit and 64 bit version included.
Downloads: http://www.videohelp.com/software/madVR

https://github.com/chocolatey/chocolatey-package-requests/issues/40


How to enable Gzip

Hi,
I just can't find any information about how to enable gzip compression on zuul. Do you know how to do that?

The Netflix engineers have an example of GZipping responses if the requesting client asked for gzip, but for whatever reason the origin server didn't gzip the response body. This example filter is part of zuul-core in the 3.x rewrite, https://github.com/Netflix/zuul/blob/3.x/zuul-core/src/main/groovy/com/netflix/zuul/filters/GZipResponseFilter.groovy

I finally enabled it using this configuration in my application.properties:
server.compression.enabled=true
server.compression.mime-types=application/json,application/xml,text/html,text/xml,text/plain
Is there any diffference between using this solution and the filter solution suggested?

The gzipresponsefilter handles a bunch of cases of when to gzip and when not to. Sometimes downstream services gzip the content, so you don't want to double gzip, sometimes the client explicitly asks for gzip, and sometimes not. Because of this complexity, we wrote that gzipresponsefilter


Translation fails on empty line

The translation fails if an empty line is part of the translation.
ERROR: Auto translation from "en" to "de" failed: "Invalid request: Required Text", failed units: 0
We had this issue with HTML formatted like this:
<div i18n>
  text
</div>

This is not really recommended by ng xi18n so I am not sure if it is an issue - see:
https://github.com/angular/angular-cli/issues/6941
The state of the translation is reset from final to translated so it was a problem in our case.

Definitely there should not be an auto translation failure.
Will check it as soon as possible.
Also have a look at "allowIdChange"-flag (#65), which might be helpful.

I tried with the flag enabled - I think this flag would only trigger after the translation has been successful so it did not matter in this case.

Oh and thank you for the fast resonse :)


BREAKING CHANGE: consume endpoint while connecting

Motivation
Fix #450, makes it connect method consume endpoint to avoid lifetime issue, it makes this piece of code being able to compile
https://user-images.githubusercontent.com/33961674/92327716-f321f300-f05b-11ea-8c42-4e2fb6234fcf.png
Solution
change &self to self

@alce Hi, my purpose is to be able to keep the channel around and generate a grpc client while necessary. I'm actually not sure if we need to clone and keep &self because I don't need to store Endpoint, I need Channel though

Clients are cheap to clone. Maybe instead of trying to hold on to the channel future and connect whenever you need a client just clone the client itself?

Hi, I'm cloning Channel because it's even cheaper than Client and I can achieve at the same time multiplexing
I keep a shared channel and everytime when I need to call a service, I clone then the channel and build a client to do the call.  The health of the channel is checked every 10seconds
Am I doing the right thing?

Yeah it makes sense to do it that way if you have multiple clients and want to share the channel. The problem is, I think, the interaction between the actor implementation you are using and how tonic is implemented.
I do not know how you can make them play well together but I don't think the solution is to change tonic's Endpoint implementation as you propose here, but let's see what @LucioFranco thinks.

These borrow for a reaosn, you likely want to wrap the async connect call in your own async move {} block to allow it to be 'static.
Feel free to ping me on discord if the above solution doesn't work for you but I am going to close this PR since we don't want it to move like what @alce said. Thanks for the PR though :)


[enhancement] Support of GCP CloudSQL format

Hi
I'm back with the format of GCP CloudSQL Postgresql logs.
It's fairly normal PG log but encapsulated in JSON format.
{
  "textPayload": "[153735]: [10-1] db=cloudsqladmin,user=cloudsqladmin LOG:  duration: 11.446 ms  statement: SELECT datname, numbackends, xact_commit, xact_rollback, blks_read, blks_hit, tup_inserted, tup_updated, tup_deleted, deadlocks, blk_read_time, blk_write_time, stats_reset, current_timestamp FROM pg_catalog.pg_stat_database WHERE stats_reset IS NOT NULL ORDER BY datname ",
  "insertId": "s=31ab647413784ebda4ba4d0f0ef93d6b;i=cb0cc;b=ac15f05ee2554206b2aea5191234f431;m=15048af7ae7;t=5a2ce3c603f6e;x=3a11dd807086788-0@a1",
  "resource": {
    "type": "cloudsql_database",
    "labels": {
      "project_id": "test-dev",
      "region": "europe",
      "database_id": "test-dev:test-replica"
    }
  },
  "timestamp": "2020-04-08T21:32:23.456622Z",
  "severity": "INFO",
  "logName": "projects/test-dev/logs/cloudsql.googleapis.com%2Fpostgres.log",
  "receiveTimestamp": "2020-04-08T21:32:24.286915566Z"
}

In CloudSQL, we can't modify log_line_prefix but we can play with log_min_duration_statement, log_duration, log_connections, log_lock_waits, etc.
It would be cool if pgbadger could parse GCP JSON format !
Thanks !

Can you post a sample log file using pgbench for example, with the following configuration:
            log_checkpoints = on
            log_connections = on
            log_disconnections = on
            log_lock_waits = on
            log_temp_files = 0
            log_autovacuum_min_duration = 0

this will help covering all cases.

Hi
I set these options, I did the pgbench test and I will collect the log tomorrow.
Thanks

hi !
here is the log (GCP makes one file by hour)
https://send.firefox.com/download/2225cb134996dc60/#DPSmMnFnMgb4xJvmIvSz_w
I did one pgbench test with pgbench  -h <IP_ADDRESS> -U postgres -t 250  test, it ended at this timestamp : 2020-05-02T13:42:24
There is no SELECT lines in this file as I didn't set log_min_duration_statement.
Do you want another log with log_min_duration_statement = 0  ?
Thanks !

Commit 717a467 add support to GCP CloudSQL log file. Let me know if there is any issue.

Thanks ! I will test it.

Thanks @darold , it works :)

Hi
I'm back with the format of GCP CloudSQL Postgresql logs.
It's fairly normal PG log but encapsulated in JSON format.

Hello!
Can I ask you how do you export the cloudsql logs ?
and how do you consider the multiline subjects when big requests?
thx!

Hi !
you need to set up a log routing, AKA logsink :
https://cloud.google.com/logging/docs/export/configure_export_v2
For multiline, I don't know, you will need to test :)
Have a nice day
Le ven. 26 juin 2020 à 13:49, Thomas Leclaire<EMAIL_ADDRESS>a
écrit :

Hi
I'm back with the format of GCP CloudSQL Postgresql logs.
It's fairly normal PG log but encapsulated in JSON format.
Hello!
Can I ask you how do you export the cloudsql logs ?
and how do you consider the multiline subjects when big requests?
thx!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/darold/pgbadger/issues/579#issuecomment-650137391,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAXJYEBNRMNM6V2NIOU64ULRYSDLRANCNFSM4MEIIVUQ
.

sounds like multiline is correclty managed by sink export. (but not when using the google cli to read logs)
thx!

Hello,
I know that this si a closed topic but it's related to it.
I have downloaded version 11.3 and I am trying to analyze logs pgbadger -f jsonlog postgres.json but without success.
Here is an example of one entry
{
"textPayload": "2020-11-12 18:13:08.388 UTC [1824977]: [42-1] db=test-service,user=test-service LOG:  duration: 2.475 ms  parse : select user0_.id as id1_0_, user0_.active as active2_0_, user0_.language as language9_0_, user0_.last_name as last_na10_0_, user0_.mobile_number as mobile_11_0_,  user0_.role_id as role_id18_0_, user0_.sms_code as sms_cod14_0_, user0_.sms_code_expiration as sms_cod15_0_, user0_.uid as uid16_0_, user0_.zip_code as zip_cod17_0_ from app_user user0_ where user0_.uid=$1",
"insertId": "s=4027b358aec64ff0bf73af185b4cb473;i=dcead7;b=90d604700bc240409c1f8cfc9a47478f;m=5e266b4d20b;t=5b3ecdd342444;x=c6fd1262fc7fb735-0@a1",
"resource": {
"type": "cloudsql_database",
"labels": {
"project_id": "project-x",
"database_id": "project-x:test",
"region": "europe-west2"
}
},
"timestamp": "2020-11-12T18:13:08.388932Z",
"severity": "INFO",
"logName": "projects/compelling-cat-212513/logs/cloudsql.googleapis.com%2Fpostgres.log",
"receiveTimestamp": "2020-11-12T18:13:09.429173904Z"
}
The config on the database is:
log_min_duration_statement 1
log_duration on
log_checkpoints on
log_connections on
log_disconnections on
log_lock_waits on
log_temp_files 0
log_autovacuum_min_duration 0
Can you tell me if I am missing something?


make bias configurable

made biases configurable.
Most latest models dont train with biases (palm, llama).
Although ideally I'd like do disable layernorm bias also with the same config param

cc: @ptrendx @ksivaman appreciate if you can take a look, all tests pass

@timmoon10 thanks added your suggested changes

/te-ci

@ksivaman sorry didnt get your comment, both are already wrapped in the wrapper function.

/te-ci


Avoid flipping the state all the time

1.0.8 agent version of #6030

Beginning pylint and unit tests.
See https://cmssdt.cern.ch/jenkins/job/DMWM-PR-test/78/ for details

Seangchan, I just realized it's a broken patch, sorry! :(
I updated #6030, can you update this one please? Thanks

Thanks Alan, I updated this patch as well.

Beginning pylint and unit tests.
See https://cmssdt.cern.ch/jenkins/job/DMWM-PR-test/79/ for details

No changes to unit tests for pull request 6033


Cannot install gradio, tokenizers 0.14.1 requires huggingface-hub<0.18,>=0.16.4, but you have huggingface-hub 0.21.4 which is incompatible.

After installing TripoSR
Per the previous instructions #40
I could not install gradio to test the GUI

Building wheels for collected packages: ffmpy
  Building wheel for ffmpy (setup.py) ... done
  Created wheel for ffmpy: filename=ffmpy-0.3.2-py3-none-any.whl size=5606 sha256=69e1845f85b6407bcb398b06d44c762c677705c4c4bbb2d473baad23d1d4e878
  Stored in directory: c:\users\user\appdata\local\pip\cache\wheels\8a\63\a8\fad16a1c5c990569a478f40782ce543788ddac224288008635
Successfully built ffmpy
Installing collected packages: pytz, pydub, ffmpy, websockets, tzdata, toolz, tomlkit, sniffio, six, shellingham, semantic-version, ruff, python-multipart, pyparsing, pygments, pydantic-core, orjson, mdurl, kiwisolver, importlib-resources, h11, fonttools, cycler, contourpy, click, annotated-types, aiofiles, uvicorn, typer, python-dateutil, pydantic, markdown-it-py, huggingface-hub, httpcore, anyio, starlette, rich, pandas, matplotlib, httpx, gradio-client, fastapi, altair, gradio
  Attempting uninstall: huggingface-hub
    Found existing installation: huggingface-hub 0.17.3
    Uninstalling huggingface-hub-0.17.3:
      Successfully uninstalled huggingface-hub-0.17.3
**ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tokenizers 0.14.1 requires huggingface-hub<0.18,>=0.16.4, but you have huggingface-hub 0.21.4 which is incompatible.**
Successfully installed aiofiles-23.2.1 altair-5.2.0 annotated-types-0.6.0 anyio-4.3.0 click-8.1.7 contourpy-1.2.0 cycler-0.12.1 fastapi-0.110.0 ffmpy-0.3.2 fonttools-4.49.0 gradio-4.20.1 gradio-client-0.11.0 h11-0.14.0 httpcore-1.0.4 httpx-0.27.0 huggingface-hub-0.21.4 importlib-resources-6.1.3 kiwisolver-1.4.5 markdown-it-py-3.0.0 matplotlib-3.8.3 mdurl-0.1.2 orjson-3.9.15 pandas-2.2.1 pydantic-2.6.3 pydantic-core-2.16.3 pydub-0.25.1 pygments-2.17.2 pyparsing-3.1.2 python-dateutil-2.9.0.post0 python-multipart-0.0.9 pytz-2024.1 rich-13.7.1 ruff-0.3.1 semantic-version-2.10.0 shellingham-1.5.4 six-1.16.0 sniffio-1.3.1 starlette-0.36.3 tomlkit-0.12.0 toolz-0.12.1 typer-0.9.0 tzdata-2024.1 uvicorn-0.27.1 websockets-11.0.3

There is no installation error. You should be able to run gradio_app.py. What you're seeing is a dependency error caused by gradio upgrading huggingface-hub. It runs normally on my machine and has not resulted in any runtime errors.

Hi @shodanx2, can you check if the demo runs smoothly?

Sorry ,
I was mistaken, the gradio app does run.
I supposed the red error message was not fatal

The warning message should probably be yellow instead of red.
As far as I can tell, this works and this ticket can be closed.
Well, I tried running the TripoSR application and it failed with more errors messages.
But they are CUDA errors (which is not installed) and the gradio part of the application appears to work correctly.


add a size config for the datastore/ARCCache

This PR add a configuration option to change the size or disable the ARC cache in the datastore.
See https://github.com/ipfs/go-ipfs/issues/6556#issuecomment-521814663 for details.

Blocked on https://github.com/ipfs/go-ipfs/pull/6586#issuecomment-554606406.

FYSA go-ipfs-config got merged into go-ipfs repo: https://github.com/ipfs/go-ipfs/tree/master/config
If you feel this PR is still relevant, please reopen as a PR against go-ipfs


Add 'Send All' option on 'Send' tab

Useful to consolidate UTxOs.

Possibly either have send all to main, active or next address so consolidating is default behavior. It seems the more likely use case than sending all to some other address.

Makes sense, I need to design a good UX for that to avoid mistakes, some time ago, we had issues with users accidentally sending tokens to CoinEx with Yoroi.


Need Help with camera and few issues and commands

Jump camera option

We have an option to jump to a "Hero" ( "Select hero" ) with this command

alias "+tohero" "+dota_camera_follow;-dota_camera_follow;+dota_camera_follow"
bind "1" "+tohero"

I was wondering if there is an option to jump the camera to the location of "other Selected other Units"
( "Select other units" )

New commands added to cfg dont work unless i manually load it using the console or bind a button to load a cfg

is there a way i can  use the midas sound to make sure the cfg is automticaly loaded when i join a game or go into the game ( after the pick phase )

NOTE Im just using bits and parts of ur cfg since Im not much into modifiers. Thanks

You could try this bind 2 "dota_select_all_others;dota_select_all_others".

any idea about the other two issues ?

Well, I could probably help you, but github issues are such a slow way to communicate. Add me on steam: http://steamcommunity.com/id/dodekeract.

added you thanks

add me and drop me a message . Im gonna hit the bed . good night


Update cloudinary npm packages

@Lepozepo
cloudinary
cloudinary-jquery
I didn't find any problems updating the packages to the latest versions.
I used the npm package and want to replace it with your package (and want to make sure the cloudinary version is the same).


Make clear that memory usage is in Mb

Need to make it clearer in the UI that the resource metrics monitoring graph for the memory usage is in units of megabytes, not percentage of total memory used.
Also could update docs to make this clear

Moved to UI repo


Update acu.md

Sometimes customers open a support case to understand how to calculate the number of cores offered by each Azure VM size along with the respective virtual processors(vCPUs) as they are unable to check it following the available public documents.
Currently customers can only see the details like RAM, number of vCPUs, and disk related configurations like max IOPs, temp disks, etc, from the VM's "Size" section in Azure Portal as well as by referring to the MS Public documents available.
If customers need to have a specific number of cores for running their application with optimum efficiency on the Azure VM, they open a support case as they are unable to opt for specific VM size or series which will provide them the required number of cores along with the respective virtual processors(vCPUs).
Hence I have just added an example and also a note section on how to check the cores offered by an VM SKU in Azure.

@mimckitt
Can you review the proposed changes? Note that the validation warning must be resolved before merging.
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

Hello Everyone,
Can someone please provide any updates on this pull request?
Are there any blockers or edits required from my end to get it merged? Please feel free to let me know.


How to run the examples

When I try running the examples, I found out that I have to use  cargo run --bin, which then only is able to run bevy_yarnspinner_demo and generate_proto, because access_variables, custom_command, custom_function and hello_world are ambigous.

I'd expect to run the examples with cargo run --example hello_world, but no examples are available

If that is not possible because of the structure, could you add documentation on how to run the examples?

Good idea, that should be documented somewhere. I'm a bit confused though why you're getting duplicate examples :thinking:
Could you try running cargo run -p yarnspinner_examples --bin hello_world?

cargo run -p yarnspinner_examples --bin hello_world panics with Error in Yarn Spinner plugin: Failed to load Yarn file folder assets\dialogue. Help: Does the folder exist?
cargo run -p examples/bevy_yarnspinner --bin hello_world does output an error
error: package ID specification examples/bevy_yarnspinner looks like a file path, maybe try ...

@tigerplush Make sure you've downloaded the submodules as well as the main repo. git submodule update --init


docker version ?

would be awesome if there was a docker file for this

found one:
https://github.com/andrewferrier/misc-scripts/tree/master/draw.io-offline-docker

Dockerised it here https://github.com/kannangce/draw.io


Syncplay refuses to launch no matter what

I had just reinstalled windows, I had then downloaded Syncplay. However, Once I launched it, it gave me the error "Reinstalling the program may fix this problem. vcruntime140.dll is either not designed to run on Windows or it contains an error". My friend and I  tried a bunch of resolutions then eventually reinstalled that specific file. That fixed the error, However, Syncplay now refuses to launch no matter what, All versions included (Even Portable) The Syncplay server launches as intended however the Syncplay application itself does not.

Thank you so much! I installed so many different versions of the Microsoft Visual Program. but that's the one that works, TYSM!


Fix FollowLink when searching tags.

When FollowLink is called from the show_tags picker, it searches for the word that was under the cursor instead of the selected tag. This PR changes FollowLink so it instead searches for the selected tag.

Proposed change
Fix FollowLink so it searches for the selected tag instead of the word under the cursor when invoked from the show_tags picker.
Type of change

[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (thank you!)
[ ] Code quality improvements to existing code or addition of tests
[ ] Documentation update

Additional information

This PR fixes or closes issue: fixes #281

Checklist

[x] I am running the latest version of the plugin.
[x] The code change is tested and works locally.
[x] There is no commented out code in this PR.
[x] The code has been formatted using Stylua (a .stylua.toml file is provided)
[x] The code has been checked with luacheck (a .luacheckrc file is provided)
[x] The README.md has been updated according to this change.
[x] The doc/telekasten.txt helpfile has been updated according to this change.

Nice catch, thank you for the fix!


internal SetPassiveAsync can't be used

In the example of FirstUnsuccessfulResponseHealthPolicy, there are following issues need to be address with the code:

public void RequestProxied function was provided parameters with wrong order
SetPassiveAsync is internal function
IProxyErrorFeature cannot be found, should be IForwarderErrorFeature

/backport to main


FR: Submit many urls at once

Hi
I am not sure if this is possible but it would be great if one can just submit Urls on the commandline from a simple text file. For instance I have a text file with couple hundred Youtube channel ids, it would be nice if I could just submit those automatically and get individual rss links as atext file or in the terminal.
I am using the Docker compose method.
thanks

If you want to turn a page URL into a feed URL then the detect action is what you're after. If you want to automate it via curl on the command line you can for example do something like:
while read -r line; do
	curl -LGs \
		-o /dev/null \
		-w '%{url_effective}' \
		--data-urlencode 'action=detect' \
		--data-urlencode 'format=Atom' \
		--data-urlencode "url=$line" \
		'https://your.rss-bridge.com/'
done < 'urls.txt'

If https://your.rss-bridge.com/ is your rss-bridge instance, and you have page URLs on individual lines in url.txt that will print out feed URLs for you. The detect action needs to be implemented for each bridge individually though, and it doesn't look like it's available for the YouTube bridge yet, so that needs to be done for it to work.
If you have just the channel IDs themselves you could just build the feed URL on your own. If you have the IDs on individual lines in ids.txt you can do:
xargs -a ids.txt -I{} echo 'https://your.rss-bridge.com/?action=display&bridge=Youtube&context=By+channel+id&c={}&format=Atom'

Hope any of that helps!

Hi
Thanks for your help., this is great. Well it is almost there. Now how can I import this into an Rss reader? Do you know if there is a way to convert the converted urls into an opml file?

If you're using TT-RSS you can batch subscribe to manu URLs at once, but you have to go into the preferences. Here's where you find it:

And for the [Unknown] feed title I believe that's what TT-RSS shows when it hasn't fetched the feed yet, so just give it a bit until it does and hopefully it'll show up!
To force a feed update right away of a single feed you can use the feed debugger in TT-RSS. In the normal feed view select a feed, and press f then shift + d. It'll open a new page where you can check Force refetch and then press Continue. Give it a second to do its thing and when you refresh the TT-RSS page the feed should be updated and the title should be there!

Hi
Thanks for the help. This is great. I think this should be in the wikki so it is not buried in the issues.

I am using both services [Rss Bridge and TTRss] very often and to make things a little bit easier I created a 'plugin' / addition into TTRss to be able to generate Rss Feeds based on social media pages.
Of course you are able to create them with Rss Bridge abd copy/paste them into TTRss. With the API you are able to automate it too. But sometimes it is just helping me a lot to submit a social media link into TTRss [additional info will be asked if needed] which is being handled by an external Rss Bridge service and added to TTRss the moment the feed receives articles.
You may contact me if you like to use such a service too. No problem.


installation instructions

slight issue with installation instructions - if you copy/paste step 3 into terminal, it swallows the $@
exec swipl -q -t main -s ~/.local/marelle/marelle.pl "$@"
becomes
exec swipl -q -t main -s ~/.local/marelle/marelle.pl ""
Since I assume it's working for you, need something more complex than escaping the $.  So not sure
how to proceed, hence issue rather than pr.
8cD

Well picked up! It turns out this was escaped properly in the bootstrap scripts, but not in the README: https://github.com/larsyencken/marelle/blob/master/bootstrap_dev.sh#L118


Update dependencies and workflows

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (360dfdd) to head (93354e4).

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #727   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           75        75
  Lines         2103      2103
  Branches       261       261
=========================================
  Hits          2103      2103

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


add files (amen and js) and change link to run in local server

I fix to run your player in local machine.

copy amen chops from your website
copy bitcrusher.js from your website
change js.knob link to cdn

thanx, I should rework this project one day.


chore: secure hermetic-build docker image

This PR improves the Hermetic Build Docker image in a few different ways:

Use secure base images
Use a secure repository for Python dependencies. This was done by pointing to the Airlock Python registries, which requires a couple of special configuration files and a build-time secret.

The build-time secret must be configured before building the image locally (instructions added in the development guide). Cloud Build automatically injects the secrets and no special setup is needed.
A few dependencies were submitted for approval and they were automatically added to the staging repository. Once they become part of the "trusted" repository, we will use the "safest" trusted repository.

Use multiple stages where only the relevant binaries are copied into the final image

owl-bot-cli was compiled into a standalone binary
The library_generation python package and its dependencies are built in a separate stage

Use Alpine-based images

Since this kind of distro is not compatible with glibc-based binaries, a special compatibility enablement step was added
Since the UNIX tools are from FreeBSD, the -d flag in the rm command is not supported. This is why we removed its usage in the scripts

Reduction of image size to ~1 GB. Most space is used by:

OpenJDK: ~300MB
GLIBC compatibility: ~250MB
Standalone OwlBot binary: ~150MB

Reduction of build time to 150 seconds (parallel stage builds via buildkit).

Pending items

[ ]  Discuss a way to pass the Application Default Credentials to the now (now failing - see job output) hermetic_library_generation workflow in sdk-platform-java (other repos don't need to build the image). We can add a secret in the repo or migrate this action to Cloud Build.
[ ] Discuss a strategy for the integration tests now that we need a secret to passed it to the image building workflow. The current approach is to use Cloud Build instead of GitHub Actions since it has out-of-the box compatibility with APPLICATION_DEFAULT_CREDENTIAL secrets such as the one declared in the Dockerfile. In the meantime, I left a build workflow in .cloudbuild to replace the integration tests and removed the integration tests portion of the existing GitHub Action. Another possible approach is to have a repository secret and keep using the GitHub Action.

[ ] If approach is validated, create the Cloud Build trigger

I think this PR has done multiple things, change base image and optimize image building.
Can we modify the base image first and create a new PR for optimization?


Add .sh run script for macOS & linux, fix error on macs with low vram.

macOS users running RVC often experience issues due to the platform differences.
Therefore, splitting requirements.txt for macOS is necessary. For example: training feature index on macOS only doable with faiss-cpu 1.7.0 (#695), higher or lower version will cause an error and directly crash RVC. Splitting out requirements.txt for macOS users is obviously much better than continuing to use the old version package for all platform users.
Also it makes it easier to set up the entire environment by running .sh files for macOS and Linux users. This script install missing dependencies for example, and install python3.8 (bets for this project) automatically.

As #252 and #311, faiss-cpu1.7.0 fails to install (I also tried it in M1 environment, but no luck).
Did it work?

Yes :D
It worked for me, and feature index training worked without issue!

It was still not working in my environment. But it is a great PR and I want to merge it.
Can you tell me what version of swig you use?

Swig was not installed on my Mac?
naozumi@n218103174213 Retrieval-based-Voice-Conversion-WebUI-updated0618v2 % swig -version
zsh: command not found: swig
naozumi@n218103174213 Retrieval-based-Voice-Conversion-WebUI-updated0618v2 % which swig
swig not found
naozumi@n218103174213 Retrieval-based-Voice-Conversion-WebUI-updated0618v2 %

I installed faiss with pip btw.

I installed faiss-cpu in a virtual environment and the it installed without any problem?
naozumi@n218103174213 Retrieval-based-Voice-Conversion-WebUI-updated0618v2 % python3.8 -m venv faiss_test
naozumi@n218103174213 Retrieval-based-Voice-Conversion-WebUI-updated0618v2 % source faiss_test/bin/activate
(faiss_test) naozumi@n218103174213 Retrieval-based-Voice-Conversion-WebUI-updated0618v2 % pip install faiss-cpu==1.7.0
Collecting faiss-cpu==1.7.0
  Using cached faiss_cpu-1.7.0-cp38-cp38-macosx_10_14_x86_64.whl (1.9 MB)
Installing collected packages: faiss-cpu
Successfully installed faiss-cpu-1.7.0
WARNING: You are using pip version 21.1.1; however, version 23.1.2 is available.
You should consider upgrading via the '/Users/naozumi/Downloads/RVC/Retrieval-based-Voice-Conversion-WebUI-updated0618v2/faiss_test/bin/python3.8 -m pip install --upgrade pip' command.
(faiss_test) naozumi@n218103174213 Retrieval-based-Voice-Conversion-WebUI-updated0618v2 %

I see the reason. I'm using intel Macs and it seems<EMAIL_ADDRESS>have the pre-built wheel for x86_64 arch.
https://pypi.tuna.tsinghua.edu.cn/simple/faiss-cpu/
I checked the list and did not see there's arm version... D:

I've made changes to the .sh file and deleted requirements_macOS.txt . 👍

I almost got everything installed, but got stuck at this part: ImportError: cannot import name 'FFmpeg' from 'ffmpy' (/usr/local/lib/python3.8/site-packages/ffmpy/init.py) any help please?

same here:  ImportError: cannot import name 'FFprobe' from 'ffmpy'
i get this every time i try to use the "run.sh" script


chore(cli): bump data dependencies

Description of changes
Update data dependencies to consume 5.9.4/1.5.4 release.
Checklist

[x] PR description included
[x] yarn test passes

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Could you get passing E2E run on this PR?


Rewrite update lineage db script

Description
This is a complete rewrite of the lineage update script. The script used to do most its work in inline bash and the update delta was computed with unix tools. It was very hard to tell exactly what the unix commands were doing, and there was an issue where the lineage version of rows was set to NULL.
Now the script is in pure ruby, except mysqlimport, and the update delta is computed in SQL. The intermediate state is all stored as SQL tables for easy inspection. The final irreversible queries are executed only after a user prompt and they are wrapped in SQL transaction.
Notes

Even after our security talk, I didn't use SQL placeholders, because the values are hardcoded in the file, but I could refactor to use placeholders
See the log output below to see the sequence of events.
See the test data to understand the inputs.
I did not compare output of the old script to the new, but I could do so.
The current version is now defined as TaxonLineage.maximum('version_end'), and the next, +1. So the version must only increase in steps of 1.
After this goes live, it would be good to set the version columns to be NOT NULL
The started_at and ended_at date format is now the same YYYY-MM-DD as the NCBI date instead of the Time.now. I feel this represents better the intent of the script and I can't think of any downside.
The script is 4x faster to run locally than before, taking 15 min instead of an hour

Tests
I extracted some test data for this script then added some rows to force inserts and updates. By inspecting database, I saw each type of row was handled correctly. Comparing the 11 test rows to the existing rows from which they were randomly sampled:
Retire records...
1920336 records

Insert records...
1 records

Update records...
1 records

Keep same records...
9 records

TEST RUN

Starting import of s3://idseq-database/taxonomy/2019-09-17 ...

--------------
DROP TABLE IF EXISTS _new_names
--------------

--------------
CREATE TABLE _new_names(taxid text, name text, common_name text)
--------------

Connecting to db
Selecting database idseq_development
Loading data from LOCAL file: /app/tmp/taxonomy/2019-09-17/_new_names.csv into _new_names
idseq_development._new_names: Records: 50  Deleted: 0  Skipped: 0  Warnings: 0
Disconnecting from db
--------------
ALTER TABLE _new_names ADD UNIQUE idx(taxid(255))
--------------

--------------
DROP TABLE IF EXISTS _new_taxid_lineages
--------------

--------------
CREATE TABLE _new_taxid_lineages(taxid text, superkingdom_taxid text, kingdom_taxid text, phylum_taxid text, class_taxid text, order_taxid text, family_taxid text, genus_taxid text, species_taxid text)
--------------

Connecting to db
Selecting database idseq_development
Loading data from LOCAL file: /app/tmp/taxonomy/2019-09-17/_new_taxid_lineages.csv into _new_taxid_lineages
idseq_development._new_taxid_lineages: Records: 11  Deleted: 0  Skipped: 0  Warnings: 0
Disconnecting from db
--------------
DROP TABLE IF EXISTS _new_taxon_lineages
--------------

--------------
CREATE TABLE _new_taxon_lineages(taxid text, started_at text, version_start text, version_end text, superkingdom_taxid text, phylum_taxid text, class_taxid text, order_taxid text, family_taxid text, genus_taxid text, species_taxid text, superkingdom_name text, phylum_name text, class_name text, order_name text, family_name text, genus_name text, species_name text, superkingdom_common_name text, phylum_common_name text, class_common_name text, order_common_name text, family_common_name text, genus_common_name text, species_common_name text, kingdom_taxid text, kingdom_name text, kingdom_common_name text)
--------------

--------------
ALTER TABLE _new_taxon_lineages ADD UNIQUE idx(taxid(255))
--------------

--------------
INSERT INTO _new_taxon_lineages
      SELECT l.taxid,
      2019-09-17,
      5,
      5,
      `superkingdom`.taxid, `phylum`.taxid, `class`.taxid, `order`.taxid, `family`.taxid, `genus`.taxid, `species`.taxid, `superkingdom`.name, `phylum`.name, `class`.name, `order`.name, `family`.name, `genus`.name, `species`.name, `superkingdom`.common_name, `phylum`.common_name, `class`.common_name, `order`.common_name, `family`.common_name, `genus`.common_name, `species`.common_name, `kingdom`.taxid, `kingdom`.name, `kingdom`.common_name
      FROM _new_taxid_lineages l
      JOIN _new_names superkingdom ON l.superkingdom_taxid = superkingdom.taxid
      JOIN _new_names kingdom ON l.kingdom_taxid = kingdom.taxid
      JOIN _new_names phylum ON l.phylum_taxid = phylum.taxid
      JOIN _new_names class ON l.class_taxid = class.taxid
      JOIN _new_names `order` ON l.order_taxid = `order`.taxid
      JOIN _new_names family ON l.family_taxid = family.taxid
      JOIN _new_names genus ON l.genus_taxid = genus.taxid
      JOIN _new_names species ON l.species_taxid = species.taxid
--------------

Retire records...
1920336 records

Insert records...
1 records

Update records...
1 records

Keep same records...
9 records

Do you wish to execute the above irreversible changes? [y/N]

If you don't mind adding one more thing, I think it would be great to dump the old state of the table into a file and archive it somewhere in S3. That way if something goes wrong, we can just restore the previous state. It seems like an important addition for peace of mind.

I'll make a copy in mysql itself, then delete it in a month if there have been no issues.

If you don't mind adding one more thing, I think it would be great to dump the old state of the table into a file and archive it somewhere in S3. That way if something goes wrong, we can just restore the previous state. It seems like an important addition for peace of mind.

I'll make a copy in mysql itself, then delete it in a month if there have been no issues.

Ok that's fine. I meant in the update script itself in case someone else operates it in the future :)


Add Development Instructions

Add basic development instructions to readme.

Definitely agree. I've been trying to get it to a stable state but I'll tackle that next.


MessageEvent model does not include File objects

Hi,
I'm trying to listen to users posting a code snippet but I see the MessageEvent class does not include files, although the message is handled by the MessageEvent handler and as far as I could see the received json message does indeed have the shared snippet information under "files".
Is this a bug or should I listen to another event? I tried listening to MessageFileShareEvent but it's not fired since the received message does not have a subtype.
I'm currently using com.slack.api:slack-api-client:1.5.1 and com.slack.api:slack-api-model:1.5.1 (which is transitive).
Thanks in advance!

PS: It can be solved by extending the provided MessageEvent, here's an example:
@Data
@EqualsAndHashCode(callSuper = true)
public class CustomMessageEvent extends MessageEvent {

    private List<File> files;
}

(But I still think the library should cover this case)

PS: It can be solved by extending the provided MessageEvent, here's an example:
@Data
@EqualsAndHashCode(callSuper = true)
public class CustomMessageEvent extends MessageEvent {

    private List<File> files;
}

(But I still think the library should cover this case)

@amalagraba Thanks for sharing this. Did you surely receive files in MessageEvent (= no subtype payload)? If so, could you share an example payload (please redact IDs and any real values as much as possible)?
I have fixed a similar issue before and my understanding is that files can be included only when a payload is either file_share or message_changed subtype. If you are receiving message_changed when you have a file URL in a message, you can subscribe to MessageChangedEvent instead of MessageEvent.
See also:

https://github.com/slackapi/java-slack-sdk/issues/494
https://github.com/slackapi/java-slack-sdk/pull/495#issuecomment-647261142
https://github.com/slackapi/java-slack-sdk/pull/496

@amalagraba Thanks for sharing this. Did you surely receive files in MessageEvent (= no subtype payload)? If so, could you share an example payload (please redact IDs and any real values as much as possible)?
I have fixed a similar issue before and my understanding is that files can be included only when a payload is either file_share or message_changed subtype. If you are receiving message_changed when you have a file URL in a message, you can subscribe to MessageChangedEvent instead of MessageEvent.
See also:

https://github.com/slackapi/java-slack-sdk/issues/494
https://github.com/slackapi/java-slack-sdk/pull/495#issuecomment-647261142
https://github.com/slackapi/java-slack-sdk/pull/496

Hi @seratch, thanks for looking into this. This is the exact message I received (with redacted information ofc)
{
    "type": "message",
    "text": "send this to cvs in stage",
    "files": [
        {
            "id": "FILE_ID",
            "created":<PHONE_NUMBER>,
            "timestamp":<PHONE_NUMBER>,
            "name": "Sin título",
            "title": "Sin título",
            "mimetype": "text/plain",
            "filetype": "text",
            "pretty_type": "Texto simple",
            "user": "USER_ID",
            "editable": true,
            "size": 2133,
            "mode": "snippet",
            "is_external": false,
            "external_type": "",
            "is_public": false,
            "public_url_shared": false,
            "display_as_bot": false,
            "username": "",
            "url_private": "REDACTED",
            "url_private_download": "REDACTED",
            "permalink": "REDACTED",
            "permalink_public": "REDACTED",
            "edit_link": "REDACTED",
            "preview": "{\n  \"bookings\" : [ {\n    \"companyId\" : \"E10\",\n    \"officeId\" : 1,\n    \"valuationMode\" : \"STANDARD\",",
            "preview_highlight": "
\n
\n

{

\n

  "bookings" : [ {

\n

    "companyId" : "E10",

\n

    "officeId" : 1,

\n

    "valuationMode" : "STANDARD",

\n
\n
\n",
            "lines": 89,
            "lines_more": 84,
            "preview_is_truncated": true,
            "is_starred": false,
            "has_rich_preview": false
        }
    ],
    "upload": true,
    "blocks": [
        {
            "type": "rich_text",
            "block_id": "+j9Jb",
            "elements": [
                {
                    "type": "rich_text_section",
                    "elements": [
                        {
                            "type": "text",
                            "text": "send this to cvs in stage"
                        }
                    ]
                }
            ]
        }
    ],
    "user": "USER_ID",
    "display_as_bot": false,
    "team": "TEAM_ID",
    "client_msg_id": "CLIENT_MESSAGE_ID",
    "source_team": "TEAM_ID",
    "user_team": "TEAM_ID",
    "channel": "CHANNEL_ID",
    "event_ts": "1610968163.065700",
    "ts": "1610968163.065700"
}

As I stated in my previous comment, extending the event to add the files attribute works just fine. If you need more info let me know.

Hi @seratch, thanks for looking into this. This is the exact message I received (with redacted information ofc)
{
    "type": "message",
    "text": "send this to cvs in stage",
    "files": [
        {
            "id": "FILE_ID",
            "created":<PHONE_NUMBER>,
            "timestamp":<PHONE_NUMBER>,
            "name": "Sin título",
            "title": "Sin título",
            "mimetype": "text/plain",
            "filetype": "text",
            "pretty_type": "Texto simple",
            "user": "USER_ID",
            "editable": true,
            "size": 2133,
            "mode": "snippet",
            "is_external": false,
            "external_type": "",
            "is_public": false,
            "public_url_shared": false,
            "display_as_bot": false,
            "username": "",
            "url_private": "REDACTED",
            "url_private_download": "REDACTED",
            "permalink": "REDACTED",
            "permalink_public": "REDACTED",
            "edit_link": "REDACTED",
            "preview": "{\n  \"bookings\" : [ {\n    \"companyId\" : \"E10\",\n    \"officeId\" : 1,\n    \"valuationMode\" : \"STANDARD\",",
            "preview_highlight": "
\n
\n

{

\n

  "bookings" : [ {

\n

    "companyId" : "E10",

\n

    "officeId" : 1,

\n

    "valuationMode" : "STANDARD",

\n
\n
\n",
            "lines": 89,
            "lines_more": 84,
            "preview_is_truncated": true,
            "is_starred": false,
            "has_rich_preview": false
        }
    ],
    "upload": true,
    "blocks": [
        {
            "type": "rich_text",
            "block_id": "+j9Jb",
            "elements": [
                {
                    "type": "rich_text_section",
                    "elements": [
                        {
                            "type": "text",
                            "text": "send this to cvs in stage"
                        }
                    ]
                }
            ]
        }
    ],
    "user": "USER_ID",
    "display_as_bot": false,
    "team": "TEAM_ID",
    "client_msg_id": "CLIENT_MESSAGE_ID",
    "source_team": "TEAM_ID",
    "user_team": "TEAM_ID",
    "channel": "CHANNEL_ID",
    "event_ts": "1610968163.065700",
    "ts": "1610968163.065700"
}

As I stated in my previous comment, extending the event to add the files attribute works just fine. If you need more info let me know.

Thanks for sharing this. The message seems to be a text snippet creation. In this scenario, the payload always has the subtype "file_share". I am not sure why this pattern is different with your app yet.
I wanted to reproduce your situation before making changes to this SDK but it's okay now. I am sure you are encountering the situation for some reason.
If you don't mind, could you send a pull request to fix this (with a corresponding test here)? If you are busy, I will be quickly fix this tomorrow in my timezone (+09:00). In any case, I will release a new patch version for you shortly.

Thanks for sharing this. The message seems to be a text snippet creation. In this scenario, the payload always has the subtype "file_share". I am not sure why this pattern is different with your app yet.
I wanted to reproduce your situation before making changes to this SDK but it's okay now. I am sure you are encountering the situation for some reason.
If you don't mind, could you send a pull request to fix this (with a corresponding test here)? If you are busy, I will be quickly fix this tomorrow in my timezone (+09:00). In any case, I will release a new patch version for you shortly.

Sure thing, I'll submit a pull request adding the files attribute and the corresponding test.

Sure thing, I'll submit a pull request adding the files attribute and the corresponding test.

@amalagraba Thanks a lot for fixing the issue 👍 A new patch version (1.5.2) has been published.

@amalagraba Thanks a lot for fixing the issue 👍 A new patch version (1.5.2) has been published.


Removing xfails.

The pytest.mark.xfail should be used when the implementation is not correct, or similar. So eventually, this will get fixed.
XFAIL should not be used as a default when the test raises an exception because in that case, the implementation is behaving as expected and it should be a PASS.

Requesting so many reviewers just to validate my thoughts about XFAIL.


refactor(nexus): clean up label parsing code

dispensed with NexusChildLabel struct entirely
(which maintained a child reference and seemed to be causing deadlocks)
added extra sanity checking for valid labels
create labels correctly for disks of varying size
allow changing of disk GUID if creating nexus with different UUID

Resolves CAS-720

bors try

bors try

bors try

bors try

bors try

For the sake of testability and taking into account the future changes in Nexus/BlockDevices approach,  can we make the code that performs actual I/O (i.e. reads/writes GPT sectors) wore abstract over some generic interfaces, like read / write or so ?
This gives us possibility to smoothly switch to a new device model by just implementing a thin I/O adapter, rather than changing all label code + we can easily write tests that validate GPT using files as backend storage.

Yes I think this is definitely the way to proceed, but I think it would also be better to this as under a separate Jira (which could also encompass reading and writing to the MetaData partition)

bors try

bors try

bors merge

bors merge

bors merge

bors merge


The typography of Player1 is not correct

Describe the bug
The typography of Player1 is not correct
Steps to reproduce the behavior
1.Navigate to https://arcade.makecode.com/beta#editor
2.Click "View simulator keyboard shortcuts"
Expect behavior
Its typography should show like this:

Actual behavior
The typography of Player1 is not correct. Text "Right" should be next to "D",  not under it.

Additional context
1.OS: Windows(rs6)
2.arcade version: 1.5.27
3.Microsoft MakeCode version: 7.1.4

fixed in beta


Remove local collection logic.

NOTE: The test project for this is passing and this has been tested with 2 other projects with no issues.
This removes the local database, localClient, and updates _createUser to work with the default mongodb config.
This is a DRAFT PR because while we have discussed dropping local dbs we have not discussed dropping _createUser which previously used config.mongodb.local settings.
As rewritten it is possible the _createUser function might get called if a username and apassword are set, this could cause issues with production projects. If we decide to keep _createUser I believe we would need to test this branch in production with a username and password set.
EDIT: Also I believe that the user for mongodb is set either in a docker or kubernetes file now possibly via helm chart meaning that _createUser probably should be dropped.
new error:
Error:
The database "bedrock_mongodb_test" failed to open. Please ensure the database bedrock_mongodb_test exists, the primary user "adminGuy" exists and has sufficient access privileges, and that "connectOptions.authSource" admin2 is set correctly.
    at async.auto.check (/Users/andrewjones/Programs/javascript/digital_bazaar/bedrock-mongodb/test/node_modules/bedrock-mongodb/lib/index.js:455:13)
    at runTask (/Users/andrewjones/Programs/javascript/digital_bazaar/bedrock-mongodb/test/node_modules/bedrock-mongodb/node_modules/async/dist/async.js:1632:13)
    at /Users/andrewjones/Programs/javascript/digital_bazaar/bedrock-mongodb/test/node_modules/bedrock-mongodb/node_modules/async/dist/async.js:1572:13
    at processQueue (/Users/andrewjones/Programs/javascript/digital_bazaar/bedrock-mongodb/test/node_modules/bedrock-mongodb/node_modules/async/dist/async.js:1582:13)
    at taskComplete (/Users/andrewjones/Programs/javascript/digital_bazaar/bedrock-mongodb/test/node_modules/bedrock-mongodb/node_modules/async/dist/async.js:1601:9)
    at /Users/andrewjones/Programs/javascript/digital_bazaar/bedrock-mongodb/test/node_modules/bedrock-mongodb/node_modules/async/dist/async.js:1625:17
    at /Users/andrewjones/Programs/javascript/digital_bazaar/bedrock-mongodb/test/node_modules/bedrock-mongodb/node_modules/async/dist/async.js:969:16
    at /Users/andrewjones/Programs/javascript/digital_bazaar/bedrock-mongodb/test/node_modules/bedrock-mongodb/lib/index.js:437:18
    at /Users/andrewjones/Programs/javascript/digital_bazaar/bedrock-mongodb/test/node_modules/bedrock-mongodb/lib/index.js:553:13
    at /Users/andrewjones/Programs/javascript/digital_bazaar/bedrock-mongodb/test/node_modules/bedrock-mongodb/node_modules/async/dist/async.js:473:16
2020-08-25T23:41:35.305Z - critical: [bedrock/master] worker "8669" exited with code "1"

Also I believe that the user for mongodb is set either in a docker or kubernetes file now possibly via helm chart meaning that _createUser probably should be dropped.

Whatever we do here, we can't have things stop working for dev set ups that don't involve k8s/helm charts.

Also I believe that the user for mongodb is set either in a docker or kubernetes file now possibly via helm chart meaning that _createUser probably should be dropped.

Whatever we do here, we can't have things stop working for dev set ups that don't involve k8s/helm charts.

should not break any functionality for them, but they will need to create a user using the mongodb shell for their mongod.conf file. Should I add a readme section for that with links to the MongoDB documentation? Removing _createUser does remove some functionality, but I don't think it's unreasonable to assume a developer can create a mongodb user themself and also we were showing a command prompt, now we throw an error. I think the error is better in most situations such as AWS cloudwatch logs, docker logs, etc.

Add CHANGELOG entry with details.  Breaking, local db removed, maybe some user creating issues, etc.

but I don't think it's unreasonable to assume a developer can create a mongodb user themself

-1 if a developer didn't need to do this before, then it is unreasonable to add this friction to their process now. It is even more unreasonable if this has to be done for every application/database created. I'm very concerned when we say "it's not unreasonable to have developers do more work." No, it is. One of bedrock's core design goals is to reduce extra processes that create friction for developers in an effort to get to "zero config"/"zero boilerplate" as much as possible. Other frameworks are a pain to use because they have so much additional setup and documentation that must be read. The biggest problems with bedrock all revolve around extra documentation that must be read or doesn't exist -- and this direction would just create more of that.
If the above changes make it so we need extra docs or the developer has to do more things, we must not do it. We must find another way forward.

@davidlehn can you look here? I think everyone else has approved at this point.

@dlongley @mattcollier @davidlehn just so you know I tried this functionality in bedrock-mongodb ^7.0.0 and currently adminPrompt: true with a user and password specified causes bedrock-mongodb to fail. In other words we need to release this as a bug fix for the current version of bedrock-mongodb. While it does remove localCollections it does it in a way that should for the most part (there are removed options) be backwards compatible with the original feature.


Add Twitter logo button to footer

Added a new Twitter logo button to the footer component.
Description

Implemented a Twitter icon button in the footer component of the website.
Screenshots (if appropriate):

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

@prachi-kurmi great work

@MAVRICK-1 Thank you for being a supportive mentor.


Terms added to spellcheck dictionary cannot be removed

In GitLab by @nwalczak on Jun 26, 2019, 10:45
Once added to the spellcheck dictionary, there is no mechanism to clear those additions.  We should have some mechanism to display the added terms and to allow the user to remove them (or possibly even add new ones manually).
Similar to this, we may want to define a new default spellcheck dictionary for SD2, which contains common terms in SD2 that should be ignored (like adding Transcriptic, Ginkgo, etc.).

In GitLab by @jakebeal on Jul 1, 2019, 10:10
Two options:

If the user makes a manual addition that matches an ignored term or pair, then reactivate it.
Allow them to see and edit a dictionary.

Probably we want to go with 1, but be aware of 2 as a backup if 1 doesn't give the UI we want.

In GitLab by @jakebeal on Jul 1, 2019, 10:38
assigned to @nwalczak

In GitLab by @jakebeal on Oct 14, 2019, 10:17
removed assignee


Add FAQ entry on waiting for the browser

Intended to do that a long time ago:
https://github.com/akrennmair/newsbeuter/issues/582#issuecomment-317113498

The result can be viewed here.
@tsipinakis, @noctux, anyone else—can you review this, please? It's a FAQ; I want to make sure it's accessible and effective.

Coverage remained the same at 35.697% when pulling 974e939179156c4b1f31b1e236e28b673ca1e2b6 on feature/faq-waiting-for-browser into e38afd42353c49229fefcf40ded7433547e8d139 on master.

Coverage remained the same at 35.697% when pulling 7da3d753f7afa9b50b3adf237d406e5037abff2c on feature/faq-waiting-for-browser into e38afd42353c49229fefcf40ded7433547e8d139 on master.

Coverage remained the same at 35.697% when pulling 7da3d753f7afa9b50b3adf237d406e5037abff2c on feature/faq-waiting-for-browser into e38afd42353c49229fefcf40ded7433547e8d139 on master.

Coverage remained the same at 35.697% when pulling 307d37b68555af41f54f259ed80389b3cc42a6d2 on feature/faq-waiting-for-browser into e38afd42353c49229fefcf40ded7433547e8d139 on master.


Feature/code mirror theme exposed

First pass on exposing a codeMirrorTheme prop, plumbing it through, and docs on how to load/use css for the theme.

See discussion in https://github.com/graphql/graphiql/issues/129

Oh, forgot to mention: I applied some small fixes directly on the PR branch rather than incurring another round trip. If you do more work on this, be sure to pull those down.

The fact that the editor views are powered by CodeMirror is really just an implementation detail, and using codeMirror in the prop name leaks that out.

This is a great point, and something that I thought about in some capacity. Here's my thoughts:

theme is a bit broad in my opinion, as new adopters will likely assume that this theme would apply to the entirety of GraphiQL (which isn't the case).
Minor, but I think that at this point GraphiQL is pretty tied to the CodeMirror implementation (there's even sub-packages that were written to marry GraphiQL to CodeMirror). Not saying that won't change, but would be non-trivial work.

I think maybe syntaxTheme would suffice since it succinctly addresses the fact that this only applies to code portions of the UI, and is not a global UI-theme.
Would that be ok @wincent?

Or maybe editorTheme?
Agreed that for all practical purposes CodeMirror is likely to remain the implementation, but that doesn't mean we shouldn't try to practice good hygiene and abstract over it.

:boom: editorTheme is great. I'll update my PR tomorrow.

Looks good. Thanks for your work on this!

Thanks for catching that JSX error....again!


Better license errors

The error message when a license is bad doesn't contain relevant context to help the user debug. This PR adds that context.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

IMHO it's not a good idea to print the license to the log.

Even the file path?

Hi @pnovotnak you have not made the suggested change from @bradrydzewski. Thanks for your work but we are going to close this issue. If you still want this change, rebase and we can re-review it.
Thanks you for your patience


binary file source/sink - mmap

implement zero copy mmap based reader and writer. We should be able to pass around mapped pages with the buffer management api. It could be advantageous -- needs benchmarking as well.

Example: https://www.cs.purdue.edu/homes/fahmy/cs503/mmap.txt
Windows equivalent: CreateFileMapping + VirtualAlloc?

WIP branch: https://github.com/pothosware/PothosBlocks/tree/wip/mmap