[suggestion] supply zip file in parts

The data files with over hundreds of GB (e.g. potential_vorticity_1.40625deg.zip, 420GB) are very hard for rsync/https download (not easy/slow to recover from connection errors).
Could you provide these data in zip parts file (with each part file smaller than 4GB) to alleviate the download problem? Thanks.

Yes, I understand it's hard to download one huge file. However, it's not that straightforward for me to upload new data to the server.
Maybe you could try using the data downloading pipeline directly (https://github.com/pangeo-data/WeatherBench#data-processing). This takes a while but is fail-proof.

Thanks for the instructions, well ... may be you can supply zip parts file when adding new data next time...
BTW, could you also share command that generates the checksums.sha512 file? So we can easily check if any downloaded file corrupts.

Thanks for the instructions, well ... may be you can supply zip parts file when adding new data next time...
BTW, could you also share command that generates the checksums.sha512 file? So we can easily check if any downloaded file corrupts.

oops, I just found this sha512sum --quiet -c checksums.sha512p1 works. And I previously use awk 'NR%3==1' checksums.sha512 > checksums.sha512p2 to split it into parts to make use of parallelism.

Great, thanks for figuring this out! And I agree that providing smaller files would be useful. I will keep that in mind and try to change it at some future iteration of the dataset. For now I will close this issue.


ignored a few top level setup in test coverage

Coverage increased (+23.2%) to 100.0% when pulling 17f196aa753f89d742ce3f891a6cbc2a238d9ec7 on coverage into c1599a22be85ba983dce001859372eabf5e1507f on master.

Coverage increased (+23.2%) to 100.0% when pulling f6952274e990248d28f075110632019654725d2c on coverage into c1599a22be85ba983dce001859372eabf5e1507f on master.

Coverage increased (+23.2%) to 100.0% when pulling 41cc168376f5893510996152d32b0b36f257a649 on coverage into c1599a22be85ba983dce001859372eabf5e1507f on master.


Check clientData origin matches expected value

When a new key handle is registered with a u2f device, the origin is
encoded into the key handle. When a u2f device is authenticating it
verifies that the origin corresponds to the encoded origin in the
requested key handle, and puts the origin into the clientData in the
response.
To prevent a MITM who intermediated a registration we check that the
origin in the clientData corresponds with the expected value.

LGTM, thanks. (CI failures are unrelated, and some day may be fixed by opam-repository)


TOC in static export of the notebook

Describe the bug
I am unsure if I am doing something wrong when saving a static version of a marimo notebook.  Everything works fine, the HTML file is created and I can open it in any browser. However, the Table of Contents in the active version is not available.
Thanks
Environment
I am on a Windows 10 machine, using Python v3.12 and marimo v0.7.9.
Code to reproduce
No response

This is not a bug - we just dont show the sidebar when exporting as a static export

Is this something you would be open to contributing?

@mscolnick , Sorry about my wrong choice of words. Thanks for correcting the title.
Regarding my contribution, I do not know how to help beyond the feedback I have provided.  I am more of a user than a developer.  I use computing for a living in my teaching and research (I am an economist), and I have used Julia and Pluto extensively over the last 5 years (Matlab user in the old days). I am at ease using computation and a notebook, but I am not knowledgeable enough to enter the developer's territory.
To test Marimo, I have just finished replicating one of my Pluto notebooks. Unfortunately, notebooks are (or have to be) quite large for teaching (it is useless to write down a notebook with one/two exercises in this activity). It involves 13 exercises, 20 plots, reading several data frames, some simple linear algebra, and a lot of Markdown. I think I have a good idea about how Marimo compares with other notebooks by now.  If I can help with this, I will be glad to help.
Finally, let me congratulate you and Akshay on a beautiful piece of software. For some reason, I did not enjoy IPyflow (as well as some minor variations on the reactive camp of notebooks; I like Observable, but JS is not my lawn). However, when I tried Marimo, it seemed well thought out and implemented. Nevertheless, I believe certain aspects of the notebook's architecture deserve to be improved if there are expectations of marimo's widespread adoption in teaching activities. It looks like a wonderful piece of software for such a job.
Thanks.

No worries, it's all good. We are looking at marimo all day so makes sense why the table of contents is not there, while I can totally understand it's a bug.
If it's something you'd like (as opposed to just letting us know of it), happy to keep the issue open so maybe someone else can pick it up. I think it makes sense to have a tiny table of contents in static and read-only mode.

Just tested on 0.7.11, and the TOC is not included in HTML export.


Changing from SecKeyConvertible to SecKeyRepresentable

This change makes adding support for RSA keys even easier.  It seems like it might have been the original intent, and hope it helps

whoops, didn't take into account the retrieve methods

Okay, I found the issue that could be found. It's possible to store RSA keys, but not to retrieve them.
I extended .retrieve(...) the retrieval method to make it possible return not only EC keys (converted using init(x963Representation:)), but also RSA keys (using init(derRepresentation:)).
Updated logic follows the documentation for SecKeyCopyExternalRepresentation(::)


[android] Fixing clearTimeout and clearInterval doesn't when funId is greater than 127

Due the implementation of int auto-boxing and messageQueue, clearTimeout and clearTimeout won't work when funId is greater than 127. It works fine when funId is smaller than 127.

JIRA is down, BTW.
JIRA waring is irrelevant. I'm just fixing bug. NO document is needed.

https://issues.apache.org/jira/projects/WEEX/issues/WEEX-45?filter=allopenissues


Cherry-picks for the 3.2 branch (future 3.2.4) - 6th batch

The regression test project found a crash from 325b41248efe6890e4495436315825e159789ffb, nice CI!
ERROR: _get_tab_width: Index p_index = 0 is out of bounds (get_tab_count() = 0).
   At: scene/gui/tab_container.cpp:491.
ERROR: get: FATAL: Index p_index = 0 is out of bounds (size() = 0).
   At: ./core/cowdata.h:152.
handle_crash: Program crashed with signal 4
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[1] bin/godot.x11.tools.64s() [0x1456f46] (/home/runner/work/godot/godot/platform/x11/crash_handler_x11.cpp:54)
[2] /lib/x86_64-linux-gnu/libc.so.6(+0x46210) [0x7f80b32a0210] (??:0)
[3] CowData<Control*>::get(int) const (/home/runner/work/godot/godot/./core/cowdata.h:152 (discriminator 7))
[4] Vector<Control*>::operator[](int) const (/home/runner/work/godot/godot/./core/vector.h:85)
[5] TabContainer::_draw_tab(Ref<StyleBox>&, Color&, int, float) (/home/runner/work/godot/godot/scene/gui/tab_container.cpp:428)
[6] TabContainer::_notification(int) (/home/runner/work/godot/godot/scene/gui/tab_container.cpp:380)
[7] TabContainer::_notificationv(int, bool) (/home/runner/work/godot/godot/scene/gui/tab_container.h:38 (discriminator 14))
[8] Object::notification(int, bool) (/home/runner/work/godot/godot/core/object.cpp:934)
[9] CanvasItem::_update_callback() (/home/runner/work/godot/godot/scene/2d/canvas_item.cpp:460)
[10] MethodBind0::call(Object*, Variant const**, int, Variant::CallError&) (/home/runner/work/godot/godot/./core/method_bind.gen.inc:61 (discriminator 4))
[11] Object::call(StringName const&, Variant const**, int, Variant::CallError&) (/home/runner/work/godot/godot/core/object.cpp:922 (discriminator 1))
[12] MessageQueue::_call_function(Object*, StringName const&, Variant const*, int, bool) (/home/runner/work/godot/godot/core/message_queue.cpp:250)
[13] MessageQueue::flush() (/home/runner/work/godot/godot/core/message_queue.cpp:299)
[14] SceneTree::iteration(float) (/home/runner/work/godot/godot/scene/main/scene_tree.cpp:484 (discriminator 2))
[15] Main::iteration() (/home/runner/work/godot/godot/main/main.cpp:2086)
[16] OS_X11::run() (/home/runner/work/godot/godot/platform/x11/os_x11.cpp:3396)
[17] bin/godot.x11.tools.64s(main+0x334) [0x144dd3a] (/home/runner/work/godot/godot/platform/x11/godot_x11.cpp:57)
[18] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f80b32810b3] (??:0)
[19] bin/godot.x11.tools.64s(_start+0x2e) [0x144d94e] (??:?)

Poke @volzhs (might simply be that I messed up in my cherry-pick, I'll investigate too).

The regression test project found a crash from 325b412, nice CI!

Reverted in that PR.


Use without react

How we can use material-ui independent from react?

@KiarashS It is a question and not an issue. Ask on this forum: https://gitter.im/callemall/material-ui


Crash (Killed signal) after Epoch 1/200

Hi there,
Thanks for the super useful library. I can run almost all samples except this one when I tried to train my custom models. I tried on both my windows and mac machine but no luck.
Code: using the custom training example
from imageai.Detection.Custom import DetectionModelTrainer

trainer = DetectionModelTrainer()
trainer.setModelTypeAsYOLOv3()
trainer.setDataDirectory(data_directory="Baddth")
trainer.setTrainConfig(object_names_array=["Baddth"], batch_size=4, num_experiments=200, train_from_pretrained_model="../yolo.h5")
trainer.trainModel()

Behavior: Always received Killed signal (crash) after Epoch 1/200
I already tried installing tensorflow 1.13.1. But it doesn't seem to help.
My machine might not be using GPU right now tho (I'm using Windows 10 and Mac OSX). Not sure if that's related?
Suggestion on how to solve this issue would be appreciated.
Python 3.5.1
imageai              2.1.3
opencv-python        <IP_ADDRESS>
tensorboard          1.13.1
tensorflow           1.13.1
tensorflow-estimator 1.13.0
tensorflow-gpu       1.13.1
root@70c0ce95b9ba:/mydrive/custom-train# python ./custom_train.py
Using TensorFlow backend.
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:530: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:535: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Generating anchor boxes for training images and annotation...
Average IOU for 9 anchors: 0.79
Anchor Boxes generated.
Detection configuration saved in  Baddth/json/detection_config.json
Some labels have no annotations! Please revise the list of labels in your configuration.
Training on: 	None
Training with Batch Size:  4
Number of Experiments:  200
Traceback (most recent call last):
  File "./custom_train.py", line 9, in <module>
    trainer.trainModel()
  File "/usr/local/lib/python3.5/site-packages/imageai/Detection/Custom/__init__.py", line 227, in trainModel
    norm=normalize
  File "/usr/local/lib/python3.5/site-packages/imageai/Detection/Custom/generator.py", line 36, in __init__
    if shuffle: np.random.shuffle(self.instances)
  File "mtrand.pyx", line 4065, in numpy.random.mtrand.RandomState.shuffle
TypeError: object of type 'NoneType' has no len()
root@70c0ce95b9ba:/mydrive/custom-train# python ./custom_train.py
Using TensorFlow backend.
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:530: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:535: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Generating anchor boxes for training images and annotation...
Average IOU for 9 anchors: 0.81
Anchor Boxes generated.
Detection configuration saved in  Baddth/json/detection_config.json
Some labels have no annotations! Please revise the list of labels in your configuration.
Training on: 	None
Training with Batch Size:  4
Number of Experiments:  200
Traceback (most recent call last):
  File "./custom_train.py", line 9, in <module>
    trainer.trainModel()
  File "/usr/local/lib/python3.5/site-packages/imageai/Detection/Custom/__init__.py", line 227, in trainModel
    norm=normalize
  File "/usr/local/lib/python3.5/site-packages/imageai/Detection/Custom/generator.py", line 36, in __init__
    if shuffle: np.random.shuffle(self.instances)
  File "mtrand.pyx", line 4065, in numpy.random.mtrand.RandomState.shuffle
TypeError: object of type 'NoneType' has no len()
root@70c0ce95b9ba:/mydrive/custom-train# python ./custom_train.py
Using TensorFlow backend.
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:530: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:535: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Generating anchor boxes for training images and annotation...
Average IOU for 9 anchors: 0.81
Anchor Boxes generated.
Detection configuration saved in  Baddth/json/detection_config.json
Training on: 	['Baddth']
Training with Batch Size:  4
Number of Experiments:  200
WARNING:tensorflow:From /usr/local/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /usr/local/lib/python3.5/site-packages/imageai/Detection/Custom/yolo.py:24: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Training with transfer learning from pretrained Model
/usr/local/lib/python3.5/site-packages/keras/callbacks.py:1065: UserWarning: `epsilon` argument is deprecated and will be removed, use `min_delta` instead.
  warnings.warn('`epsilon` argument is deprecated and '
WARNING:tensorflow:From /usr/local/lib/python3.5/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Epoch 1/200
Killed

what mac you have? (just for curious) I thought I couldnt train on my macbook pro because i don't have nvidia

For this error,
 Some labels have no annotations! Please revise the list of labels in your configuration Training on: 	None Training with Batch Size:  4 Number of Experiments:  200
there is a chance you have missing data in your dataset

I will investigate further what might be the cause of the issue. However, have you tried training with Google Colab?
https://colab.research.google.com/

What is the memory usage on your MacBook? Please try to run on a lower batch_size, 2 for example, and see if it survives the first epoch.
I know the YoloV3 training is particularly hungry for memory. For example, training with a batch size of 6 almost completely uses up my 8GB of memory.

@Meulen92 do you have the code for training using cpu insteado of gpu?  Does this code still applies?
from imageai.Detection.Custom import DetectionModelTrainer
trainer = DetectionModelTrainer()
trainer.setModelTypeAsYOLOv3()
trainer.setDataDirectory(data_directory="/content/drive/My Drive/Colab Notebooks/bib2019-08-19")
trainer.setTrainConfig(object_names_array=["bibnumber"], batch_size=4, num_experiments=15, train_from_pretrained_model="/content/drive/My Drive/Colab Notebooks/bib2019-08-19/pretrained-yolov3.h5")
trainer.trainModel()
print ('finish')
as I'm having trouble on getting a gpu I'm thinking about putting 16gb on a linux vaio laptop I have at home and leave it running over a month for training a model with a 10k images dataset

Hi all, thanks everyone for being super supportive! I wanted to give an update on the finding and latest status here.

In short, training a new custom model is quite a resource consuming process. Without GPU, trainModel() seems to behave very abnormally -- It halted on Windows Docker and freeze on OSX Docker.  Not 100% verified but I might be able to reproduce this on small training dataset (with or without transfer-learning) as well. (First hope it'll be able to train something with smaller dataset)

Using Colab helps passing epoch 1/200. (with switching runtime mode to GPU and pip install tensorflow-gpu 1.13.1). However, if you run the code too much, CoLab'll stop giving GPU to the runtime. So, that's a little tricky part. In my case, it runs until epoch 4/200 then suddenly stopped running. Not sure if it finished successfully, or somethings happen in between.

I received a series of .h5 in the model folder tho. However, I tried using this in detectCustomObjectsFromVideo function but received the error below.  I wonder if this is because the training process failed. Or I didn't use detectCustomObjectsFromVideo correctly?
Colocations handled automatically by placer.
Traceback (most recent call last):
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\framework\ops.py", line 1659, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimension 0 in both shapes must be equal, but are 1 and 18. Shapes are [1,1,1024,255] and [18,1024,1,1]. for 'Assign_360' (op: 'Assign') with input shapes: [1,1,1024,255], [18,1024,1,1].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "video-obj-custom.py", line 10, in <module>
    detector.loadModel()
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\imageai\Detection\__init__.py", line 1055, in loadModel
    model.load_weights(self.modelPath)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\keras\engine\network.py", line 1166, in load_weights
    f, self.layers, reshape=reshape)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\keras\engine\saving.py", line 1058, in load_weights_from_hdf5_group
    K.batch_set_value(weight_value_tuples)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\keras\backend\tensorflow_backend.py", line 2465, in batch_set_value
    assign_op = x.assign(assign_placeholder)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\ops\variables.py", line 1762, in assign
    name=name)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\ops\state_ops.py", line 223, in assign
    validate_shape=validate_shape)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\ops\gen_state_ops.py", line 67, in assign
    use_locking=use_locking, name=name)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\framework\ops.py", line 3300, in create_op
    op_def=op_def)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\framework\ops.py", line 1823, in __init__
    control_input_ops)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\framework\ops.py", line 1662, in _create_c_op
    raise ValueError(str(e))
ValueError: Dimension 0 in both shapes must be equal, but are 1 and 18. Shapes are [1,1,1024,255] and [18,1024,1,1]. for 'Assign_360' (op: 'Assign') with input shapes: [1,1,1024,255], [18,1024,1,1].
```.

what mac you have? (just for curious) I thought I couldnt train on my macbook pro because i don't have nvidia

My Macbook is Pro 15" (touchbar model). I realized afterward that OSX might not be able to run tensorflow GPU at all. So, it's actually not the right option to run the training code.

@Meulen92 do you have the code for training using cpu insteado of gpu? Does this code still applies?
from imageai.Detection.Custom import DetectionModelTrainer
trainer = DetectionModelTrainer()
trainer.setModelTypeAsYOLOv3()
trainer.setDataDirectory(data_directory="/content/drive/My Drive/Colab Notebooks/bib2019-08-19")
trainer.setTrainConfig(object_names_array=["bibnumber"], batch_size=4, num_experiments=15, train_from_pretrained_model="/content/drive/My Drive/Colab Notebooks/bib2019-08-19/pretrained-yolov3.h5")
trainer.trainModel()
print ('finish')

as I'm having trouble on getting a gpu I'm thinking about putting 16gb on a linux vaio laptop I have at home and leave it running over a month for training a model with a 10k images dataset

As for the code to force it to run on CPU:
config = tf.ConfigProto(         device_count = {'GPU': 0}     ) session = tf.Session(config=config)

Hi all, thanks everyone for being super supportive! I wanted to give an update on the finding and latest status here.

In short, training a new custom model is quite a resource consuming process. Without GPU, trainModel() seems to behave very abnormally -- It halted on Windows Docker and freeze on OSX Docker.  Not 100% verified but I might be able to reproduce this on small training dataset (with or without transfer-learning). First hope it'll be able to train something with smaller dataset even without GPU. (So far, couldn't get the code to run with CPU version. Hopefully I didn't misunderstand anything)
Using Colab helps passing epoch 1/200. (with switching runtime mode to GPU and pip install tensorflow-gpu 1.13.1). However, if you run the code too much, CoLab'll stop giving GPU to the runtime. So, that's a little tricky part. In my case, it runs until epoch 4/200. Then it suddenly stopped running itself. Not sure if it finished successfully, or somethings happen in between.

I received a series of .h5 in the model folder tho. However, I tried using this in detectCustomObjectsFromVideo function but received the error below. I wonder if this is because the training process failed. Or I didn't use detectCustomObjectsFromVideo correctly?
detectCustomObjectsFromVideo :
from imageai.Detection import VideoObjectDetection
import os

execution_path = os.getcwd()

detector = VideoObjectDetection()
detector.setModelTypeAsYOLOv3()
detector.setModelPath( os.path.join(execution_path , "../detection_model-ex-11--loss-10.00.h5"))
#detector.setModelPath( os.path.join(execution_path , "../models/baddth/detection_model-ex-20--loss-4.88.h5"))
detector.loadModel()

custom = detector.CustomObjects(person=True)

video_path = detector.detectCustomObjectsFromVideo(custom_objects=custom, input_file_path=os.path.join(execution_path, "cake2.mp4"),
                                output_file_path=os.path.join(execution_path, "cake2_custom.mp4")
                                , frames_per_second=29, log_progress=True)
print(video_path)

ERROR:
Colocations handled automatically by placer.
Traceback (most recent call last):
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\framework\ops.py", line 1659, in _create_c_op
    c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimension 0 in both shapes must be equal, but are 1 and 18. Shapes are [1,1,1024,255] and [18,1024,1,1]. for 'Assign_360' (op: 'Assign') with input shapes: [1,1,1024,255], [18,1024,1,1].

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "video-obj-custom.py", line 10, in <module>
    detector.loadModel()
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\imageai\Detection\__init__.py", line 1055, in loadModel
    model.load_weights(self.modelPath)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\keras\engine\network.py", line 1166, in load_weights
    f, self.layers, reshape=reshape)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\keras\engine\saving.py", line 1058, in load_weights_from_hdf5_group
    K.batch_set_value(weight_value_tuples)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\keras\backend\tensorflow_backend.py", line 2465, in batch_set_value
    assign_op = x.assign(assign_placeholder)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\ops\variables.py", line 1762, in assign
    name=name)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\ops\state_ops.py", line 223, in assign
    validate_shape=validate_shape)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\ops\gen_state_ops.py", line 67, in assign
    use_locking=use_locking, name=name)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\framework\ops.py", line 3300, in create_op
    op_def=op_def)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\framework\ops.py", line 1823, in __init__
    control_input_ops)
  File "C:\Users\baddt\Desktop\docker\mydrive\imageai\env\lib\site-packages\tensorflow\python\framework\ops.py", line 1662, in _create_c_op
    raise ValueError(str(e))
ValueError: Dimension 0 in both shapes must be equal, but are 1 and 18. Shapes are [1,1,1024,255] and [18,1024,1,1]. for 'Assign_360' (op: 'Assign') with input shapes: [1,1,1024,255], [18,1024,1,1].
```.

@baddth Thank you for this detailed evaluation. Let me address the issues raised.

Training of custom models (prediction and detection) requires GPU acceleration, which for now is using NVIDIA GPUs. Google Colab provide a NVIDIA Tesla T4 GPU which is why it can run your model training many times faster than the CPU.

ImageAI primarily implements deep learning algorithms which require high compute (GPUs) for training, and same for real-time inference. You can read more on deep learning and GPUs here.

In your colab training, I could observe from your model file name that your training actually reached **20 experiments ** detection_model-ex-20-...., which means it was still training. If your Colab shows disconnected, all you need to do to see the latest state of your training is to click the Connect tab on the top right and the Notebook will update.

In your detection code, you were using the wrong class for your custom model. The class you need is the from imageai.Detection.Custom  import  CustomVideoObjectDetection. See the link below for the full code sample. Ensure you parse in your trained model path and the generated detection_config.json file path.

https://github.com/OlafenwaMoses/ImageAI/blob/master/imageai/Detection/Custom/CUSTOMVIDEODETECTION.md#videodetection


Add restart function to SnipSniper

Should be easy, just make sure to kill any windows cleanly and create a new SnipSniper().

i dont think we need this currently.


master

Hello, can you tell me the pre-processing steps for these datasets, I find it a bit different from the original UCI dataset

Unfortunately I don't have the answer to this since the initial preprocessing (and splitting) of these datasets from the raw UCI data was not done by myself but rather by the authors of this paper. I realize that this can make it hard to determine which feature is which if you really want to dig in; however, the datasets are still very useful for benchmarking against prior works.


Import source tree inside a Nickel package

Find a way to import any kind of source tree inside the Nickel part. The simplest example is a source file: in the current version of the future prototype, we have to pass it wrapped as a derivation from the Nix side:
The nickel package:
{
  inputs_spec = {
    bash = { input = "nixpkgs" },
    coreutils = { input = "nixpkgs" },
    gcc = { input = "nixpkgs" },
    hello = { input = "sources" },
  }

  , output = fun { inputs, nix, .. } =>
    {
      name = "hello",
      version = "0.1",
      build_command = {
        cmd = "%{inputs.bash.outputPath}/bin/bash",
        args = [ "-c", m%"
          %{inputs.gcc.outputPath}/bin/gcc %{inputs.hello.outputPath} -o hello
          %{inputs.coreutils.outputPath}/bin/mkdir -p $out/bin
          %{inputs.coreutils.outputPath}/bin/cp hello $out/bin/hello
        "%m, ]
      },
      env = {},
      dependencies = [ inputs.bash, inputs.coreutils, inputs.gcc, inputs.hello ],
    } | nix.builders.NickelPkg
}

The Nix wrapper:
        packages.default = importNcl ./hello.ncl (inputs // {
          sources.packages.${system}.hello = pkgs.runCommand "hello.c" {} "cp ${./hello.c} $out";
        });

Ideas

Use a special placeholder on the Nickel side, that the Nix part
interpret and substitute for actual store paths
Have them be just other inputs, and let the Nixel Nix library do the
work (determine which source trees are coming directly from Nix, and
which one are coming from other Nickel package definitions)

Here is one attempt: https://github.com/nickel-lang/nickel-nix/pull/10/commits/83a6e72e95897828e89c41cf9ee0ae671f49947a. We have to pass the base directory of the flake to importNcl, which is not very natural, but otherwise it looks rather natural.

Implemented with the nix.import_file function


Fix square bracket file name issue

Fixes an issue where file names surrounded by [] would not be stripped out properly in creating the file name. Also added some tests.
Note: This will likely be an issue for some files where brackets are legal like [id].jsx style file names.
Fixes #169

Fixes an issue where file names surrounded by [] would not be stripped out properly in creating the file name. Also added some tests.
Note: This will likely be an issue for some files where brackets are legal like [id].jsx style file names.
Fixes #169

may there can be some logic for particular file extensions which allow this type of file name convention, other wise apply your fix.

@jebarpg added some tests for the [id].jsx case and switched to a regex that checks for wrappings brackets.

These unstability issues that it works "most" of the time are still bugging me!
functions should solve this instability  (#94 )


postgresql: Remove static libs using host platform extension

Motivation for this change
Closes #50902. See LnL7's comment to understand what's going on.
This allows haskellPackages.postgresql-libpq to build on macOS.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[X] NixOS
[X] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Wow, that's a big rebuild number. I'll rebase this against staging.

@GrahamcOfBorg build postgresql

Success on aarch64-linux (full log)
Attempted: postgresql
Partial log (click to expand)
checking for references to /build/ in /nix/store/67rbmg39m5xpwqg38mvm59fff4nxhf64-postgresql-9.6.11-lib...
shrinking RPATHs of ELF executables and libraries in /nix/store/2rvjbm5rwrykhczypwq8gizq1dvhl6ra-postgresql-9.6.11-doc
strip is /nix/store/6yjvaipy9zffribmc4l13c7i8k19ama5-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/2rvjbm5rwrykhczypwq8gizq1dvhl6ra-postgresql-9.6.11-doc
checking for references to /build/ in /nix/store/2rvjbm5rwrykhczypwq8gizq1dvhl6ra-postgresql-9.6.11-doc...
shrinking RPATHs of ELF executables and libraries in /nix/store/cwmwr0mqd76c8rch1b1aqfjnailbcm11-postgresql-9.6.11-man
gzipping man pages under /nix/store/cwmwr0mqd76c8rch1b1aqfjnailbcm11-postgresql-9.6.11-man/share/man/
strip is /nix/store/6yjvaipy9zffribmc4l13c7i8k19ama5-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/cwmwr0mqd76c8rch1b1aqfjnailbcm11-postgresql-9.6.11-man
checking for references to /build/ in /nix/store/cwmwr0mqd76c8rch1b1aqfjnailbcm11-postgresql-9.6.11-man...

Timed out, unknown build status on x86_64-darwin (full log)
Attempted: postgresql
Partial log (click to expand)
cannot build derivation '/nix/store/hd3gk8z8acz3s5p1jyrdywsw51hj24vn-hook.drv': 5 dependencies couldn't be built
cannot build derivation '/nix/store/mp7wqryxzsylipwglqb3yshkcwx3sk0l-flex-2.6.4.drv': 6 dependencies couldn't be built
cannot build derivation '/nix/store/2my1js8z6bygcl8gismvax62mdicv8vl-bootstrap_cmds-dev-tools-7.0.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/4sfz6lg8j1af1yli1lmwq7hvdi9db2fx-xnu-osx-10.11.6.drv': 8 dependencies couldn't be built
cannot build derivation '/nix/store/vjk6b3nllciac6901sp4jx78xr2cdw2v-IOKit-osx-10.11.6.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/qym2i90pv3cilb41z0w6hzz6xn2kfyfq-configd-osx-10.8.5.drv': 8 dependencies couldn't be built
cannot build derivation '/nix/store/ffwgnx8qxa3jd44arm0kmb0avpc1d82f-python-2.7.15.drv': 8 dependencies couldn't be built
cannot build derivation '/nix/store/8lic1cab790dkpbyjvn3p0j8jndysrnr-libxml2-2.9.8.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/4rf6q1l27755mcrqjs4hrzn3cvwh57rs-postgresql-9.6.11.drv': 7 dependencies couldn't be built
error: build of '/nix/store/4rf6q1l27755mcrqjs4hrzn3cvwh57rs-postgresql-9.6.11.drv' failed

Timed out, unknown build status on x86_64-linux (full log)
Attempted: postgresql
Partial log (click to expand)
cannot build derivation '/nix/store/5q080wfp1cnrskmysa781z4yg7clq6dn-sqlite-3.25.3.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/47ix7k3df056gy3zkg36gyvhqp04hfhv-tzdata-2018e.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/9i511b2sfk1xzmh7ns9fszp95dq25v8w-hook.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/9s0p7d467n96n3pm821a607glyr7a27m-ncurses-6.1-20181027.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/bhk6vzw9f73ss6q7f8fj87indlqq44p6-openssl-1.0.2q.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/q56cz2vdpk2ixbnsq5a071ccaxgi28js-readline-6.3p08.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/vjzvg856fw71ij0r0g8hqn512za74vag-python-2.7.15.drv': 10 dependencies couldn't be built
cannot build derivation '/nix/store/d0l2zahr99nczia73bn8kaf934yljvw2-libxml2-2.9.8.drv': 4 dependencies couldn't be built
cannot build derivation '/nix/store/yzrn7yr8npclh551drpp97lw2ki5rpvz-postgresql-9.6.11.drv': 9 dependencies couldn't be built
error: build of '/nix/store/yzrn7yr8npclh551drpp97lw2ki5rpvz-postgresql-9.6.11.drv' failed


Uploading a blob to a OID field

How can I push a file to a field of type OID ? Is it possible ?
I tried pushing various JS datatypes. FormData, Blob and Object generate the same error:
sql: converting argument $2 type: unsupported type map[string]interface {}, a map
How should I encode my Blob object to upload it ?

I think I'm moving forward, but I'm not there yet. I've just discovered that I have to convert binary to base64 in order to pass incorporate into JSON. Silly me.
Note that I also changed the OID field to "bytea" because decode returns this type.
So, I wrote a SQL query to insert a line into the table. I tested the query directly with psql, it works. Then I moved the query to queries/data/upload.write.sql and changed it include my 3 fields:
INSERT INTO data(meta, texte, audio) VALUES
    ({{.meta}}, '{{.texte}}', decode({{.audio}}, 'base64'));

Then I ran this:
curl -XPOST "http://localhost:3000/_QUERIES/data/upload" \
    -d '{"meta":1, "texte":"pouet pouet", "audio":"W29iamVjdCBCbG9iXQ=="}'

And here is the result:
"error": "could not execute sql pq: syntax error at or near \"\u003c\",
INSERT INTO data(meta, texte, audio) VALUES\n
 (\u003cno value\u003e, '\u003cno value\u003e', decode(\u003cno value\u003e, 'base64'));"

Obviously the template engine didn't find the data to insert and replaced it with . What did I do wrong ?

OK, I think I found my answer: I simply cannot send a raw Blob through JSON as it is not a valid data type. So I must encode it in base64 and decode it back to binary server side.
I'm marking this as closed, but it raised another problem described here: #299


[Suggestion]: Section "Queueing a Series of State Updates": await in event handler cause immediate rerender

Summary
The documentation says here: "React waits until all code in the event handlers has run before processing your state updates"
This not the case when using await inside the handler!
Example
const [myNumber, setMyNumber] = useState(0);

async function handleClick(){
    setMyNumber(i => i + 1);
    await delay(100);
    setMyNumber(i => i + 1);
}

function delay(ms) {
    return new Promise(resolve => {
        setTimeout(resolve, ms);
    });
}

This will rerender the component twice!
Page
https://react.dev/learn/queueing-a-series-of-state-updates
Details
I suggest it should be mentioned as caveat/pitfall that if you call await inside an event handler,
all changes made to states before that await will then be flushed and trigger a rerender immediately, not waiting
for the event handler to finish completely.

Can i try this？

I try to add this


Removed ekf

Removed ekf code so users aren't confused about what localizer is running.  gnc/ekf, ekf_bag, ekf_video packages/directories are removed along with relevant matlab, autocode, and sim code.  Additionally, documentation is updated to change ekf nodes to loc nodes where necessary (TODO: how should I generate this to make sure everything looks ok?).  In the future ekf_video should be adapted to run with the graph_localizer as it's a pretty neat tool.

Yay!

Where does sim_wrapper depend on EKF?
Ideally all the matlab files with est_ can be removed. We would need someone with matlab (Marina??) to test and regenerate the autocode.
Did ekf_bag get replaced by graph_bag? If so we can get rid of it. Although it would be nice if you could take ekf_video and make a graph_video :) But we can delete these and you can go to an old version if needed.

Yay!

Where does sim_wrapper depend on EKF?
Ideally all the matlab files with est_ can be removed. We would need someone with matlab (Marina??) to test and regenerate the autocode.
Did ekf_bag get replaced by graph_bag? If so we can get rid of it. Although it would be nice if you could take ekf_video and make a graph_video :) But we can delete these and you can go to an old version if needed.

It depend on things like sim_wrapper/tools/test_sim.cc. Let me take a quick look at it :)

should compile now

Yay!

Where does sim_wrapper depend on EKF?
Ideally all the matlab files with est_ can be removed. We would need someone with matlab (Marina??) to test and regenerate the autocode.
Did ekf_bag get replaced by graph_bag? If so we can get rid of it. Although it would be nice if you could take ekf_video and make a graph_video :) But we can delete these and you can go to an old version if needed.

It depend on things like sim_wrapper/tools/test_sim.cc. Let me take a quick look at it :)

We can just delete test_sim.cc, I don't even know what that does and I (probably?) wrote it

Yay!

Where does sim_wrapper depend on EKF?
Ideally all the matlab files with est_ can be removed. We would need someone with matlab (Marina??) to test and regenerate the autocode.
Did ekf_bag get replaced by graph_bag? If so we can get rid of it. Although it would be nice if you could take ekf_video and make a graph_video :) But we can delete these and you can go to an old version if needed.

It depend on things like sim_wrapper/tools/test_sim.cc. Let me take a quick look at it :)

We can just delete test_sim.cc, I don't even know what that does and I (probably?) wrote it

I just deleted in the code the ekf test and left the sim test there

Is this branch now ready to merge? Any other suggestions for things to remove?

Is this branch now ready to merge? Any other suggestions for things to remove?

I'm going to check that the matlab code generation is still functional (should be done today), and then we can merge

I tried to run the matlab code generation, and getting some issues, I'm going to revert this back to draft until it is solved

@marinagmoreira this is still on your queue to work on?

yup!

Note that the streaming mapper in isaac by default uses /gnc/ekf. That does not run live, I think, but once all this work is the default, one should check how isaac is impacted.

Tested the code generation with Matlab2016b and worked great, no change in generated code besides date in header

Tested the code generation with Matlab2016b and worked great, no change in generated code besides date in header

Should I go ahead and merge?


Read only fleet adapter

This introduces a read-only fleet adapter that will send delay and replacement requests to the schedule based on rmf_fleet_msgs/FleetState data that comes from the fleet drivers.

@Yadunund will follow up later with some tests for this adapter.


⚠ Cioos National CKAN has degraded performance

In a828b97, Cioos National CKAN (https://catalogue.cioos.ca/) experienced degraded performance:

HTTP code: 200
Response time: 29967 ms

Resolved: Cioos National CKAN performance has improved in 2d152c2 after 5 days, 6 hours, 52 minutes.


Adds support for georeferenced coordinates.

New feature implementation
Implemented feature
Adds support for visualizing OSM maps and using Geographic coordinates (Lat/Lon) instead of just (x, y) coordinates.
To use:
Go to tools->Georeference->Set Reference.... Enter latitude and longitude of reference. Press Set Reference. You should see the maps loading.
You may also set reference via anchor or move an anchor to a specified latitude/longitude using Move To Lat/Lon
You can adjust the map resolution using the bottom slider.
To view an offset you can use the Tools->Georeference->View Reference as UTM.

Known Issues

We really should switch to Proj, but for now we use the utm crate because Proj's rust binding does not yet support WASM
Map tiles should ideally be drawn in web mercator frame, however this would require us to warp the units of our building graph to support web mercator and then have those warped back to UTM.
Changing resolution causes tiles to despawn and respawn. It would be better if the effect usses a fade in-fade out animation.
Currently we use OSM's main service. Ideally we should be able to switch tile providers in the future.

A few usability feedback points before diving too deep into the code, from trying to give it a spin:
It seems the Set Reference via Anchor menu button doesn't work if it's clicked on a fresh project, it only works after a reference has already been set. I'd suggest either making sure it works on a fresh project or greying it out (I suppose the first since it's meant to be a way to set a reference?)
The anchor reference, view reference / tile resolution windows cannot be closed. The last can maybe be kept but the first two sound a bit unnecessary to always keep?
I tried the move to lat lon with no anchor selected and got a panic:
thread 'main' panicked at 'index out of bounds: the len is 0 but the index is 0', rmf_site_editor/src/site/georeference.rs:135:33

If the Move to Lat/Lon button is clicked when the Set Reference window is open there will be glitches in egui, I believe due it might be due to reuse of widget names.
I tried to set the georeference of the office world and there are a few glitches between tiles, like black gaps and misalignments

When I tried to use Set reference via anchor in a world where a reference already exists, all the tiles disappeared and I got a bunch of warnings:
2023-06-16T03:22:07.336874Z  WARN bevy_ecs::world: error[B0003]: Could not despawn entity 878v0 because it doesn't exist in this World.
2023-06-16T03:22:07.336898Z  WARN bevy_ecs::world: error[B0003]: Could not despawn entity 879v0 because it doesn't exist in this World.
2023-06-16T03:22:07.336903Z  WARN bevy_ecs::world: error[B0003]: Could not despawn entity 880v0 because it doesn't exist in this World.
2023-06-16T03:22:07.336906Z  WARN bevy_ecs::world: error[B0003]: Could not despawn entity 881v0 because it doesn't exist in this World.

For nice to haves:
I wonder if we should have some sort of visualization for the reference anchor when it's used as a georeference, maybe adding an entity attached to it similar to how Locations work? It's basically impossible right now to find the anchor that was set as a reference, since all  that can be seen is its entity ID and clicking the button with its entity ID enters the "select anchor" mode.
It would be great if we can add parsing of legacy building projects that are in wgs84, like the rmf_demos campus world which currently fails importing:
Failed loading legacy building Message("unknown variant `wgs84`, expected `reference_image` or `cartesian_meters`", Some(Pos { marker: Marker { index: 19, line: 1, col: 19 }, path: "coordinate_system" }))

Imho it would also be nice if we can update the embedded demo map under assets/ to be georeferenced with the actual office location, to really showcase this feature

Thanks for breaking it... I've tried to break the issues down into a working list.

I tried the move to lat lon with no anchor selected and got a panic:
Fixed in 944fdd5

If the Move to Lat/Lon button is clicked when the Set Reference window is open there will be glitches in egui, I believe due it might be due to reuse of widget names.
Fixed in 944fdd5

I tried to set the georeference of the office world and there are a few glitches between tiles, like black gaps and misalignments
This an issue of web_mercator vs UTM. Its a whole can of worms that will need to be addressed. Our buildings are currently in meter units. However OSM maps are in "web mercator" pixels... There will need to be some deformation between the two. Ideally the tile sizes should also be in UTM.

When I tried to use Set reference via anchor in a world where a reference already exists, all the tiles disappeared and I got a bunch of warnings:
Currently investigating.

It would be great if we can add parsing of legacy building projects that are in wgs84, like the rmf_demos campus world which currently fails importing:
That's a separate PR. Hopefully will be less buggy.

Imho it would also be nice if we can update the embedded demo map under assets/ to be georeferenced with the actual office location, to really showcase this feature
Agreed.

When I tried to use Set reference via anchor in a world where a reference already exists, all the tiles disappeared and I got a bunch of warnings

For me if i re-check the View Map box, the tiles will reappear with the correct location
Also agreed on not being able to close widgets other than Set Geographic Reference. Maybe it would help if they are all grouped under a single widget too? Found myself having to move things around a little

I've reworked the implementation significantly for this iteration and significantly simplified the user experience. It is probably ready for another round of review. The gaps between tiles have been eliminated. There are several ways to use this feature.

By loading an existing world.

You may download an existing world here and open it. You should be able to see the world in the screen like so (notice no more gaps):

You should also be able to toggle visibility from the "view" checklist. To enable this I added an API that allows us to extend menus via the ECS like what was done in #163
Screencast from 2023-08-23 10-35-10.webm
You may also view the reference from the tools menu and change the tile reference via the settings. Note: There is a hard-coded tile limit setting the code (this is to prevent the cache size from exploding and us from hammering the OSM server). We should make this configurable in a future PR.
Screencast from 2023-08-23 10-38-32.webm

From a world without any existing reference.

We can use the example world. Open the Tool > Set reference panel. This should allow you to set an origin latitude and longitude.

You may also use the Inspect Anchor tool to move anchors to desired latitudes and longitudes.

We don't yet support using an anchor as a reference (that'll be a follow up PR) and we don't support lat/lon in theinspect Lane dialog (Also should be a follow up dialog).


Focus outline oddly rendered for some action icons when panel is on the side

VS Code Version: stable and from sources

@jeanp413 thanks a lot for catching this and for providing a PR 👏


Added JSON file w/ rule names and md5

https://github.com/mandiant/red_team_tool_countermeasures/pull/18#issue-767035933


isSoftDeleted description added

@ArvindHarinder1 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


way to apply functions to subplots when faceting

When plotting maps with cartopy, it is common to request plotting additional information over the map, e.g. coastlines using ax.coastlines().
When faceting maps (as in issue #1201), AFAICS there is no way to add coastlines to each of the faceted subplots. I do not know if or how this can be done, but in my view not being able to do it severely limits the interest of faceting when dealing with maps.

I have found in issue #604 a example of how to do this. I have submitted in issue #1203 a change to the docs.

closed via https://github.com/pydata/xarray/pull/1203


fix: fix errors when rollbackable is false

Context
Initial PR: #8
The PR #8 introduced a new option: rollbackable.
Unfortunately, this causes a few wrong behavior when it's set to false.

the amount of document stored in a bulk is wrongly computed

indeed, it was based on the amount of rollback documents created
in case of very fast bulk operation creation, this can lead to high memory consumption

for every bulk execution, we get the warning:

"The number of backup documents should be equal to the total updated documents"

Changes

Fix the amount of document stored in a bulk
Do not verify number of backup documents created when rollbackable is false.

@pp0rtal Hi, here are a few MBDM improvements. However, I'm not sure how to update tests to verify the new behavior...


Failed: Update autorest version to prerelease 3.0.0-alpha.20240710.5

Generated by Autorest Regen Preview build 20240710.5Triggered from pull request: https://github.com/Azure/autorest.csharp/pull/4908

API change check
API changes are not detected in this pull request.

API change check
API changes are not detected in this pull request.

API change check
API changes are not detected in this pull request.

API change check
API changes are not detected in this pull request.

API change check
API changes are not detected in this pull request.

API change check
API changes are not detected in this pull request.

No public interface change. We only need to fix the compilation errors.

SDK
Impact
Fix
Comments

Azure.Communication.CallAutomation
parameter of   internal clients changed
just fix the compilation error
no public interface change

Azure.Communication.CallingServer
parameter of   internal clients changed
no fix required
no public interface change

Azure.DigitalTwins.Core
parameter of   internal clients changed
no fix required
no public interface change, grouped by parameter

Azure.IoT.Hub.Service
parameter of   internal clients changed
no fix required
no public interface change

Azure.Maps.Rendering
parameter of   internal clients changed
no fix required
no public interface change, grouped by parameter

Azure.Maps.Search
parameter of   internal clients changed
just fix the compilation error
no public interface change

Azure.MixedReality.Authentication
parameter of   internal clients changed
no fix required
no public interface change, grouped by parameter

Azure.Search.Documents
parameter of   internal clients changed
no fix required
no public interface change

Azure.Storage.Blobs
parameter of   internal clients changed
just fix the compilation error
no public interface change

Azure.Storage.Files.Shares
parameter of   internal clients changed
just fix the compilation error
no public interface change

Azure.Storage.Queues
parameter of   internal clients changed
just fix the compilation error
no public interface change

close in favor of #45027


Revert "add default deny rule for wasmplugin & fix default network allow-all"

Reverts istio/istio#53363
This broke TestBadWasmRemoteLoad. Happy to merge a forward-fix if one is available before this merges.

@howardjohn  Sorry for breaking the telemetry integration test.
I will fix this in #53748 .

https://github.com/istio/istio/pull/53748

thanks!!


Fix performance test snapshot args

Noticed this failure in https://github.com/microsoft/LSKV/pull/128

This has been fixed on main recently so this PR is now a no-op.


fix for https://github.com/shogo4405/HaishinKit.dart/issues/27 taking…

Fix for https://github.com/shogo4405/HaishinKit.dart/issues/27
Added additional URI checks in the createAuthCommand command for RtmpAuthentificator.kt
Includes Wowza tests and removed the original + "?" that added breaking changes for RTMP mount validation.
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)

Can I see  this logs original code? Because there are additional formats for the RTMP path like rtmp://localhost/${appName}/${instanceName} , I need to consider whether it will work correctly.
    private fun createAuthQuery(uri: URI): String {
        Log.d(TAG, uri.toString())
        val query = uri.query ?: ""
        val user = uri.userInfo.split(":")[0]
        return "$uri" + if (query.isEmpty()) {
            "?"
        } else {
            "&"
        } + "authmod=adobe&user=$user"
    }

I can't reproduce... I tested it on Wowza(HaishinKit.kt with Wowza 4.8.23+2. I had 2 times createAuthQuery... But it seemes 3 times....
2023-06-08 22:41:51.911 11835-11973 RtmpAuthenticator       com.haishinkit.app                   D<EMAIL_ADDRESS>2023-06-08 22:41:51.979 11835-11975 RtmpAuthenticator       com.haishinkit.app                   D  rtmp://test:test@<IP_ADDRESS>/live/hello?authmod=adobe&user=test

First rtmpConnection.connect(url)
url =<EMAIL_ADDRESS>or<EMAIL_ADDRESS>How can I reproduce it?

I'm sorry, this is my bad, I've seen now that in your commit Refactoring you've already fixed this bug by editing this line from
- return uri.toString() + "/" + if (query.isEmpty()) {
to
+ return "$uri" + if (query.isEmpty()) {

thus removing the bug producing trailing slash "/".
I was on the wrong version, sorry.
This pull request should be rejected, as it is unnecesarry.
Thank you again for the help and for the fast response!

OK. Thank you.


Managed Identity Authentication

Hello,
I wonder to know if the azure-spring-boot-starter-servicebus-jms library supports Managed Identity Authentication?
I have tried the suggested connection string to enable Microsoft Documentation (Endpoint=sb://{{Fully qualified namespace name}}/;Authentication=Managed Identity), then the connection string has not been resolved.
Thank you in advance

@stliu could you please follow upw ith @alisharifi01 on the question

Hi @alisharifi01,
sorry, the azure-spring-boot-starter-servicebus-jms library doesn't supports Managed Identity Authentication.

Close the issue for there is no reply for 7 days.


UI language sometimes incorrect

I just started evaluating the hosted version of Miniflux. Today, one time when I loaded a page (a specific entry IIRC, on desktop), the UI was in French. Didn't think too much about it, but just now, on mobile, the UI was loaded in German.
So far, a page refresh has done the trick, but it seems a bit odd to me that it happened twice in a day, and with different languages at that. I haven't changed the UI language from English.

This is clearly a bug.


Retire ResolveConversationLocal and refactor chat service

I think this is ready for review now. This PR does a lot of refactoring stuff. GetInboxLocal returns a ConversationLocal now. The PR also retires ResolveConversationLocal and its friends. It's hitting GetInboxLocal anyway; using the new GetInboxLocal eliminates some redundant GetInboxRemote calls.
Sorry for the 400-pound PR! But I think this'll make future changes much easier.
r? @patrickxb @mmaxim @maxtaco @oconnor663 Thanks!

Thanks Mike!

@mmaxim @patrickxb PTAL Thanks!

The commits deefc73fe040...042955801897 introduce visual changes on linux. :mag_right: 1 changed

Changed: New_Proof_Post-Twitter_Error (before) (after)

I'm on board, but @patrickxb should probably approve as well.

Thanks guys!


Remove Database class from examples

Landed in https://github.com/metarhia/Example/commit/612687d92b8ead3f3214440090200daa05c215e5


Add pass to fold type transformations into function signature

As title.
This transformer gives users more control over where preprocessing steps may be located, and what input types their relay program can accept

@anwang2009 Please take a look at the CI problem.

@anwang2009 do you still have interest in merging this PR?

@AndrewZhaoLuo @anwang2009 are you guys doing this as part of #8296 or is that just related?

@anwang2009 any plans to continue working on this PR?

Unrelated to that issue linked. I plan to continue working on this, I just need to find the time :upside_down_face:
Sorry for the languishing, I'll try to get something in by start of next week.


Fix people showing as graduated from "Helsingin yliopisto" in trends graduated tab

vs. OODI:

caused by bunch of wrong organisation codes in sis studyrights. e.g. 014731475 studies in MH80 , so organisation should be H80, but is '01' (root organisation for whole university)

https://jira.it.helsinki.fi/browse/DOO-4477

fixed on newest sis data


⚠ Bofrost.at has degraded performance

In 6b28d04, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 2506 ms

Resolved: Bofrost.at performance has improved in 97c86a1.


feat: test runner

added runnables.scm to make it easier to run tests in IDE
closes: https://github.com/trbutler4/zed-cairo/issues/1

should this still be a draft?

yes, still a draft. should get back to this in 2 weeks.


Linguist - Removed Deprecated Code

Hey, I just made a Pull Request!
Removed support for what is known as the legacy backend, please use the New Backend System.
:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[x] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

Hi @BethGriggs, can you give this a quick look? Just need a sanity check, prefer not to approve and merge my own PRs.


Should not build slug if build_from fields are empty

If I create a user without a firstname and lastname (scope: fullname), it creates -1, -2, -3, -4, as slugs on the different users.
Have a look:

(of course above is fake data)

A fix for this is coming in the next release (see #162).  In the meantime, using dev-master should take care of it.


RTS-980,RTS-981,RTS-982: Recompile DDL

Store the DDL compiler version in DETS and at startup if the version does not match, recompile it.  Also update all pre-1.2 DETS entries to include the legacy version so they update cleanly as well.  Any DETS entries which were stuck in the compiling state when the node went down are now marked as retrying and recompiled.  The versions are registered as a riak_core_capability but are not used as of yet.
NOTE: This PR requires https://github.com/basho/riak_ql/pull/115

Skeptical of adding an unused capability since the system is designed to easily add one retroactively (any node which doesn't register it is centrally assigned the fallback value). Is there a good reason to add this one before we need it?

Having problems with the ts_simple_select test, the table isn't activating.
See this error:
2016-04-07 08:37:40.922 [error] <0.445.0>@riak_kv_ts_newtype:maybe_compile_ddl:140 Unknown DDL version type {ddl_v1,<<"GeoCheckin">>,[{riak_field_v1,<<"myfamily">>,1,varchar,false},{riak_field_v1,<<"myseries">>,2,varchar,false},{riak_field_v1,<<"time">>,3,timestamp,false},{riak_field_v1,<<"weather">>,4,varchar,false},{riak_field_v1,<<"temperature">>,5,double,true}],{key_v1,[{param_v1,[<<"myfamily">>]},{param_v1,[<<"myseries">>]},{hash_fn_v1,riak_ql_quanta,quantum,[{param_v1,[<<"time">>]},15,m],timestamp}]},{key_v1,[{param_v1,[<<"myfamily">>]},{param_v1,[<<"myseries">>]},{param_v1,[<<"time">>]}]}} for bucket type GeoCheckin (expecting ddl_v1 with old version notfound, new version 2)

Both compiler tables appear to be empty, so riak_kv_compile_tab:get_compiled_ddl_version returns notfound.

:+1: da7d02f


CLOUDP-48516: Add TLS support

With this PR we add support for TLS to the broker API server. Users can optionally pass a certificate and private key file with environment variables.

I've added documentation to the dev README for setting up TLS. I was considering adding samples for TLS setup but unfortunately the service catalog only accept a CA bundle literal. As it can't be included from a secret the user needs to update service-broker.yaml manually.


NoMethodError: undefined method `lastBuildDate' for #SimpleRSS:0x007facac46ecd0

Bug found in Jobs::PollWebsiteFeed when lastBuildDate not exist below
https://github.com/vinkashq/discourse-autobot/blob/c09adc461b6fb492e05a0470768daf6533330b78/app/jobs/regular/poll_website_feed.rb#L14

Hello @vinothkannans - is there a workaround or have you planned a fix? I'm having the same issue with your great plugin.

Hi @vinothkannans Do you have any plans to update this plugin? It seems really useful for RSS feeds.
Thanks!

Sorry! it's not in my roadmap.

Sorry! it's not in my roadmap.

OK, too bad.
Maybe you could recommend some other plugin for fetching RSS feeds into discourse?
Thanks!


Seems region leaders can lose after one tikv instance fail

Bug Report

What version of TiKV are you using?
TiKV
Release Version:   7.1.0-alpha
Edition:           Community
Git Commit Hash:   abb672b8218307e3811281c22643e0eb2e13cc2c
Git Commit Branch: heads/refs/tags/v7.1.0-alpha
UTC Build Time:    2023-04-08 14:33:07
Rust Version:      rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features:   pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile:           dist_release
What operating system and CPU are you using?
Steps to reproduce

Deploy a TiDB cluster with 4 TiKV instance.
Create 1000 sysbench tables with pre-split = 8, so that we get 256K Regions.

What did you expect?
Changefeed checkpoint lag may increase, but should decrease after a while.
What did happened?
Changefeed lag keeps increasing about 1 hour.
This may contains several issues. Let's focus on this one: seems some region leaders are lost in about 10min.

I think the root cause is when a TiKV node fails, other nodes can still dispatch Raft messages from it, after StoreUnreachable is reported. It will changes raft peers state from GroupState::Chaos to GroupState::Ordered incorrectly.
#14574  adds a test case for this. Here is its log:
test_store_disconnect_with_hibernate 2023/04/14 13:45:58.500 store.rs:2887: [INFO] broadcasting unreachable, unreachable_store_id: 1, store_id: 2
test_store_disconnect_with_hibernate 2023/04/14 13:45:58.522 store.rs:2887: [INFO] broadcasting unreachable, unreachable_store_id: 1, store_id: 3
test_store_disconnect_with_hibernate 2023/04/14 13:45:58.631 peer.rs:2446: [DEBG] handle raft message, to_peer_id: 3, from_peer_id: 1, message_type: MsgAppend, peer_id: 3, region_id: 1
test_store_disconnect_with_hibernate 2023/04/14 13:45:58.631 peer.rs:2446: [DEBG] handle raft message, to_peer_id: 2, from_peer_id: 1, message_type: MsgAppend, peer_id: 2, region_id: 1
test_store_disconnect_with_hibernate 2023/04/14 13:45:58.653 peer.rs:2175: [DEBG] stop ticking, election_elapsed: 1, peer_id: 2, region_id: 1, res: Some(CheckTickResult { leader: false, up_to_date: false, reason: "" })
test_store_disconnect_with_hibernate 2023/04/14 13:45:58.675 peer.rs:2175: [DEBG] stop ticking, election_elapsed: 1, peer_id: 3, region_id: 1, res: Some(CheckTickResult { leader: false, up_to_date: false, reason: "" })
test_store_disconnect_with_hibernate 2023/04/14 13:45:58.363 peer.rs:2136: [DEBG] follower hibernates, missing_ticks: 6, election_elapsed: 1, peer_id: 2, region_id: 1
test_store_disconnect_with_hibernate 2023/04/14 13:45:58.380 peer.rs:2136: [DEBG] follower hibernates, missing_ticks: 6, election_elapsed: 1, peer_id: 3, region_id: 1


[FLINK-26701] Revert FLINK-25927 apart from adding connector-base to flink-dist

With the introduction of FLINK-25927, every connector now relocates connector-base to better support connectors compatibility with different Flink versions. Unfortunately, not all classes in connector-base are only used by connector but some are supposed to be used inside the user jar directly i.e. DeliveryGuarantee, HybridSource...
Because of the relocation, the existing imports for those classes are broken.

@flinkbot run azure


Creation of PROPERTY_HISTORY Table with Repository and Tests

Design and implement a new table PROPERTY_HISTORY to log changes to properties in the PROPERTY table, including information about the user who made the change and the timestamp of the change.
This table will have a many-to-one (N:1) relationship with the PROPERTY table.
Additionally, develop the corresponding repository and unit tests to ensure the functionality is correctly implemented.

Not exactly, or at least not in this manner. However, in the long term, we should establish a mechanism to effectively track data changes. For now, I'll delete it.

Closing it


Update virtual-network-public-ip-address.md

remove BOM character

@meganbradley : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Thanks.


dry-run.rb Could not find gem 'vcr (= 6.0.0)' in locally installed gems. (Bundler::GemNotFound) in latest ghcr.io/dependabot/dependabot-core-development:latest DockerImage

Hi, I tried dry-run.rb following README.md, it raise an error below.
$ docker pull ghcr.io/dependabot/dependabot-core-development:latest
$ docker tag ghcr.io/dependabot/dependabot-core-development dependabot/dependabot-core-development
$ bin/docker-dev-shell
=> running docker development shell
[dependabot-core-dev] ~/dependabot-core $ bin/dry-run.rb go_modules rsc/quote

$ ruby bin/dry-run.rb go_modules zonedb/zonedb
Traceback (most recent call last):
        11: from bin/dry-run.rb:69:in `<main>'
        10: from /usr/lib/ruby/gems/2.7.0/gems/bundler-2.3.8/lib/bundler.rb:151:in `setup'
         9: from /usr/lib/ruby/gems/2.7.0/gems/bundler-2.3.8/lib/bundler/runtime.rb:18:in `setup'
         8: from /usr/lib/ruby/gems/2.7.0/gems/bundler-2.3.8/lib/bundler/definition.rb:238:in `specs_for'
         7: from /usr/lib/ruby/gems/2.7.0/gems/bundler-2.3.8/lib/bundler/definition.rb:190:in `specs'
         6: from /usr/lib/ruby/gems/2.7.0/gems/bundler-2.3.8/lib/bundler/definition.rb:468:in `materialize'
         5: from /usr/lib/ruby/gems/2.7.0/gems/bundler-2.3.8/lib/bundler/definition.rb:269:in `resolve'
         4: from /usr/lib/ruby/gems/2.7.0/gems/bundler-2.3.8/lib/bundler/resolver.rb:23:in `resolve'
         3: from /usr/lib/ruby/gems/2.7.0/gems/bundler-2.3.8/lib/bundler/resolver.rb:48:in `start'
         2: from /usr/lib/ruby/gems/2.7.0/gems/bundler-2.3.8/lib/bundler/resolver.rb:252:in `verify_gemfile_dependencies_are_found!'
         1: from /usr/lib/ruby/gems/2.7.0/gems/bundler-2.3.8/lib/bundler/resolver.rb:252:in `map!'
/usr/lib/ruby/gems/2.7.0/gems/bundler-2.3.8/lib/bundler/resolver.rb:269:in `block in verify_gemfile_dependencies_are_found!': Could not find gem 'vcr (= 6.0.0)' in locally installed gems. (Bundler::GemNotFound)

The source contains the following gems matching 'vcr':
  * vcr-6.1.0

It looks common/dependabot-common.gemspec version is invalid.
By changing package version below.(I also need fix faraday and gitlab versions)
It works fine.
-  spec.add_dependency "faraday", "1.7.0"
-  spec.add_dependency "gitlab", "4.17.0"
+  spec.add_dependency "faraday", "1.10.0"
+  spec.add_dependency "gitlab", "4.18.0"
-  spec.add_development_dependency "vcr", "6.0.0"
+  spec.add_development_dependency "vcr", "6.1.0"

Package ecosystem
docker

Package manager version
Language version
Manifest location and content prior to update
dependabot.yml content
Updated dependency

What you expected to see, versus what you actually saw
Can run dry-run without change, but it raise an error.
Native package manager behavior

Images of the diff or a link to the PR, issue or logs

🕹 Bonus points: Smallest manifest that reproduces the issue

I think you might need to rebuild your development image to get it up to date with https://github.com/dependabot/dependabot-core/commit/bad36d91e84194bb3d1dfc793c321cc98839f518 and other recent dependency changes.

A way to make this get in the middle less would be to not specify exact dependencies in the gemspec, which is best practice anyways.

@deivid-rodriguez
OK, thank you!


Compilation errors

I had trouble compiling, after running pod install within my project (lots of c++ errors). In case it can help others, I solved the issues by adding  "-lstdc++" to the "other linker flags" field of the build settings. Maybe that's something that can be fixed in the podspec file? (not sure).

Hi! Thanks for letting me know.
I think I have fixed the problem here:
https://github.com/256dpi/OSCKit/pull/7
Could you give it a try and give me feedback?

Successfully tested. Thanks for the fix!

Cool!


feat: update labels on update merge request

Hello,
I am working with your tool and I want to update a merge request with labels but this feature is not available today, only at the creation.
This merge request aims to fix this.
It follows this work: https://github.com/un-ts/changesets-gitlab/pull/180/files
Thanks for your work!

Thanks for your contribution!


qtads 2.1.6 (new formula)

QTads is a multimedia interpreter for TADS games built on Qt GUI.

@BrewTestBot test this please


Combine crop and resize

Hello,
Is it possible to resize an image before cropping it ?
For instance, I have a large image 2000x2000px. I want to get only the part of 100x200px located at coordinates (500px;500px), but in a smaller size like 50x100px. Is it possible to do something like this ?
Thanks

Nope, and this is intentional. It's almost always better to crop before resizing, and this is actually modelled after the Imgix API. You can still accomplish the same thing, you simply need to perform your crop operation at the larger size, which might require some basic calculations.
Often cropping libraries provide the ability to specifiy the cropping dimensions based on the original image as well. For example, jCrop allows you to do this using the Explicit Sizing Method.


Direct URL Input?

Can it be possible to manually download m3u8 video by simply inputting the url?
An alternative would be allowing the sniffer to detect video from an extension that allows native playback of m3u8

I wholeheartedly second @cdowning22's feature request! :+1:
The CWS contains many free "VPN" extensions, actually HTTPS proxy extensions, which can only be used within the context of the (Chromium-derived) browser; the "price" a user pays is probably ad insertion, tracking, the need to supply a valid e-mail address etc., but these concerns are OT here... :wink:
When such an extension is enabled, hls-downloader can use the browser's connection settings, i.e. fetch HLS fragments over said proxy connection (thus, geo-fenced HLS content can be dumped to disk) :+1: .
Quite often, I might use special URIs and/or apps (outside of hls-downloader) to extract geo-fenced master HLS playlists (.m3u8), but I can't use the proxy extension directly with a third party downloader (youtube-dl/ffmpeg, etc), so if hls-downloader could offer the ability to act as an in-browser downloader of a user supplied .m3u8, it could be a perfect answer for my usage scenario!

allowing the sniffer to detect video from an extension that allows native playback of m3u8

Yes, I too use Native MPEG-Dash + HLS Playback to watch geo-fenced .m3u8s inside the browser, over a "VPN" extension; sadly, no way to record the stream with hls-downloader; I don't know how feasible it would be for a Chrome extension to interact with another Chrome extension, chrome-extension://* URIs are special cases...

Part of v3.1.0
Download it from here: https://github.com/puemos/hls-downloader-web-extension/releases/tag/v3.1.0