Toco/TFLite_Convert for TFLite Problem

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: Pixel 1
TensorFlow installed from (source or binary): Source
TensorFlow version (use command below): 1.9.0 (commit r1.9)
Python version: N/A
Bazel version (if compiling from source): 0.16.1
GCC/Compiler version (if compiling from source): gcc (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609
CUDA/cuDNN version: N/A
GPU model and memory: N/A
Exact command to reproduce:

bazel run -c opt tensorflow/python/tools/optimize_for_inference -- --input=$ORIGINAL_PB  --output=$STRIPPED_PB --frozen_graph=True --input_names=Preprocessor/sub --output_names=concat,concat_1 --alsologtostderr

bazel run tensorflow/contrib/lite/toco:toco -- --input_file=$STRIPPED_PB --output_file=/absolute/path/to/tensorflow/tensorflow/contrib/lite/examples/android/assets/new_model.tflite  --input_format=TENSORFLOW_GRAPHDEF --output_format=TFLITE --input_shapes=1,300,300,3 --input_arrays=Preprocessor/sub --output_arrays=concat,concat_1 --inference_type=QUANTIZED_UINT8 --logtostderr --default_ranges_min=0 --default_ranges_max=5 --mean_values=128 --std_values=127 --allow_custom_opps
or
bazel run //tensorflow/contrib/lite/python:tflite_convert -- --graph_def_file=$STRIPPED_PB --output_file=/absolute/path/to/tensorflow/tensorflow/contrib/lite/examples/android/assets/tflite_convert_example.tflite --input_arrays=Preprocessor/sub --output_arrays=concat,concat_1 --output_format=TFLITE --input_shapes=1,300,300,3  --inference_type=QUANTIZED_UINT8 --default_ranges_min=0 --default_ranges_max=5 --mean_values=128 --std_dev_values=127 --allow_custom_opps
(which fails)
or
bazel run //tensorflow/contrib/lite/python:tflite_convert -- --graph_def_file=$STRIPPED_PB --output_file=/absolute/path/to/tensorflow/tensorflow/contrib/lite/examples/android/assets/tflite_convert_example.tflite --input_arrays=Preprocessor/sub --output_arrays=concat,concat_1 --input_shapes=1,300,300,3
(which succeeds)

Change TF_OD_API_MODEL_FILE and append new file to the assets list in BUILD

bazel build -c opt --cxxopt='--std=c++11' //tensorflow/contrib/lite/examples/android:tflite_demo

adb install -r -f bazel-bin/tensorflow/contrib/lite/examples/android/tflite_demo.apk

Open TFL Detect

Describe the problem
I'm attempting to import ssd_mobilenet_v1, ssd_mobilenet_v2 and ssdlite from the (model zoo)[https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md] into the TFLite Android example. Ultimately I'm aiming to retrain either the ssdlite or ssd_mobilenet_v2 models, but for right now all models I use trigger runtime errors. All of the errors imply that the models are changed by the optimize_for_inference and toco/tflite_convert commands in a way that makes them incompatible with r1.9.
Now, it's most likely that my command for toco/tflite_convert are to blame, but since these commands seem to be well formed I'm elevating this to github.
Source code / logs
Firstly, according to (the toco documentation)[https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/toco/g3doc/cmdline_examples.md] we're only supposed to use tflite_convert once we're in r1.9. When I try to actually specify all of the fields that the command has in the help (aka the tflite_convert command I put in above) I get the following log and no file is produced:
 WARNING: /home/bryan/.cache/bazel/_bazel_bryan/36e77cb69a80a4f75d1ba8b192f69b6d/external/grpc/BUILD:1960:1: in srcs attribute of cc_library rule @grpc//:grpc_nanopb: please do not import '@grpc//third_party/nanopb:pb_common.c' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'grpc_generate_one_off_targets', the error might have been caused by the macro implementation in /home/bryan/.cache/bazel/_bazel_bryan/36e77cb69a80a4f75d1ba8b192f69b6d/external/grpc/bazel/grpc_build_system.bzl:172:12
 WARNING: /home/bryan/.cache/bazel/_bazel_bryan/36e77cb69a80a4f75d1ba8b192f69b6d/external/grpc/BUILD:1960:1: in srcs attribute of cc_library rule @grpc//:grpc_nanopb: please do not import '@grpc//third_party/nanopb:pb_decode.c' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'grpc_generate_one_off_targets', the error might have been caused by the macro implementation in /home/bryan/.cache/bazel/_bazel_bryan/36e77cb69a80a4f75d1ba8b192f69b6d/external/grpc/bazel/grpc_build_system.bzl:172:12
 WARNING: /home/bryan/.cache/bazel/_bazel_bryan/36e77cb69a80a4f75d1ba8b192f69b6d/external/grpc/BUILD:1960:1: in srcs attribute of cc_library rule @grpc//:grpc_nanopb: please do not import '@grpc//third_party/nanopb:pb_encode.c' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'grpc_generate_one_off_targets', the error might have been caused by the macro implementation in /home/bryan/.cache/bazel/_bazel_bryan/36e77cb69a80a4f75d1ba8b192f69b6d/external/grpc/bazel/grpc_build_system.bzl:172:12
 INFO: Analysed target //tensorflow/contrib/lite/python:tflite_convert (0 packages loaded).
 INFO: Found 1 target...
 Target //tensorflow/contrib/lite/python:tflite_convert up-to-date:
   bazel-bin/tensorflow/contrib/lite/python/tflite_convert
 INFO: Elapsed time: 0.254s, Critical Path: 0.00s
 INFO: 0 processes.
 INFO: Build completed successfully, 1 total action
 INFO: Running command line: bazel-bin/tensorflow/contrib/lite/python/tflite_convert '--graph_def_file=/home/bryan/Downloads/ssd_mobilenet_v1_coco_2018_01_28/stripped' '--output_file=/home/bryan/Support/tensorflow/tensorflow/contrib/lite/examples/android/assets/tflite_convert_example.tflite' '--input_arrays=Preprocessor/sub' '--output_arrays=concat,concat_1' '--output_format=TFLITE' '--input_shapes=1,300,3INFO: Build completed successfully, 1 total action
 /home/bryan/.local/lib/python2.7/site-packages/scipy/__init__.py:114: UserWarning: Numpy 1.8.2 or above is recommended for this version of scipy (detected version 1.8.0)
   UserWarning)
 usage: tflite_convert.py [-h] --output_file OUTPUT_FILE
                          (--graph_def_file GRAPH_DEF_FILE | --saved_model_dir SAVED_MODEL_DIR)
                          [--output_format {TFLITE,GRAPHVIZ_DOT}]
                          [--inference_type {FLOAT,QUANTIZED_UINT8}]
                          [--inference_input_type {FLOAT,QUANTIZED_UINT8}]
                          [--input_arrays INPUT_ARRAYS]
                          [--input_shapes INPUT_SHAPES]
                          [--output_arrays OUTPUT_ARRAYS]
                          [--saved_model_tag_set SAVED_MODEL_TAG_SET]
                          [--saved_model_signature_key SAVED_MODEL_SIGNATURE_KEY]
                          [--std_dev_values STD_DEV_VALUES]
                          [--mean_values MEAN_VALUES]
                          [--default_ranges_min DEFAULT_RANGES_MIN]
                          [--default_ranges_max DEFAULT_RANGES_MAX]
                          [--drop_control_dependency DROP_CONTROL_DEPENDENCY]
                          [--reorder_across_fake_quant REORDER_ACROSS_FAKE_QUANT]
                          [--change_concat_input_ranges CHANGE_CONCAT_INPUT_RANGES]
                          [--allow_custom_ops ALLOW_CUSTOM_OPS]
 tflite_convert.py: error:

When I strip the tflite_convert params to just include the bare minimum (graph_def_file, output_file, input_arrays, output_arrays, input_shapes) it does create an unquantized tflite model. When I load an unquantized tflite model generated with either command, TFL Detect exits with the following log:
09-06 00:00:51.046 25024 25041 E AndroidRuntime: FATAL EXCEPTION: inference
09-06 00:00:51.046 25024 25041 E AndroidRuntime: Process: org.tensorflow.lite.demo, PID: 25024
09-06 00:00:51.046 25024 25041 E AndroidRuntime: java.lang.IllegalArgumentException: Output error: Shape of output target [1, 1917, 4] does not match with the shape of the Tensor [1, 1917, 1, 4].
09-06 00:00:51.046 25024 25041 E AndroidRuntime: 	at org.tensorflow.lite.Tensor.copyTo(Tensor.java:44)
09-06 00:00:51.046 25024 25041 E AndroidRuntime: 	at org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs(Interpreter.java:156)
09-06 00:00:51.046 25024 25041 E AndroidRuntime: 	at org.tensorflow.demo.TFLiteObjectDetectionAPIModel.recognizeImage(TFLiteObjectDetectionAPIModel.java:222)
09-06 00:00:51.046 25024 25041 E AndroidRuntime: 	at org.tensorflow.demo.DetectorActivity$3.run(DetectorActivity.java:242)
09-06 00:00:51.046 25024 25041 E AndroidRuntime: 	at android.os.Handler.handleCallback(Handler.java:873)
09-06 00:00:51.046 25024 25041 E AndroidRuntime: 	at android.os.Handler.dispatchMessage(Handler.java:99)
09-06 00:00:51.046 25024 25041 E AndroidRuntime: 	at android.os.Looper.loop(Looper.java:193)
09-06 00:00:51.046 25024 25041 E AndroidRuntime: 	at android.os.HandlerThread.run(HandlerThread.java:65)
09-06 00:00:51.049   914  2995 W ActivityManager:   Force finishing activity org.tensorflow.lite.demo/org.tensorflow.demo.DetectorActivity

The closest I've found as a solution is in (this stackoverflow page)[https://stackoverflow.com/questions/50388330/java-lang-illegalargumentexception-output-error-shape-of-output-target-1-191] which suggests modifying the TFLiteObjectDetectionAPIModel itself (which runs into problems when you get similar errors on the outputClassification array).
If we use a quantized model, it crashes with this error:
09-05 22:54:27.413 21650 21667 E AndroidRuntime: java.lang.IllegalArgumentException: Input error: DataType (1) of input data does not match with the DataType (3) of model inputs.
09-05 22:54:27.413 21650 21667 E AndroidRuntime: 	at org.tensorflow.lite.NativeInterpreterWrapper.run(Native Method)
09-05 22:54:27.413 21650 21667 E AndroidRuntime: 	at org.tensorflow.lite.NativeInterpreterWrapper.run(NativeInterpreterWrapper.java:123)
09-05 22:54:27.413 21650 21667 E AndroidRuntime: 	at org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs(Interpreter.java:144)
09-05 22:54:27.413 21650 21667 E AndroidRuntime: 	at org.tensorflow.demo.TFLiteObjectDetectionAPIModel.recognizeImage(TFLiteObjectDetectionAPIModel.java:222)
09-05 22:54:27.413 21650 21667 E AndroidRuntime: 	at org.tensorflow.demo.DetectorActivity$3.run(DetectorActivity.java:242)

Given similar error messages in (this test)[https://github.com/OAID/TensorFlow-HRT/blob/master/tensorflow/contrib/lite/java/src/test/java/org/tensorflow/lite/NativeInterpreterWrapperTest.java#L228] and how commits after r1.9 give the TFLiteObjectDetectionAPIModel class an isQuantized flag this makes me think that r1.9 may not support quantization. If so, is there a definitive source for this? There are several sources that are imperfect in different ways (for the most official sources (fixed point quantization)[https://www.tensorflow.org/performance/quantization] page seems geared towards classification instead of the object detection, the required output arrays in the (medium page)[https://medium.com/tensorflow/training-and-serving-a-realtime-mobile-object-detector-in-30-minutes-with-cloud-tpus-b78971cf1193] are not found when we run toco, and both of them are supposedly out of date because of (the toco documentation)[https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/toco/g3doc/cmdline_examples.md]).

I'm not sure how extensible the object detection harness is to using custom models. At the very least you'd need to modify the output tensor shape as dictated by outputLocations in TFLiteObjectDetectionAPIModel.
If you're mostly interested in classification, have you considered using the ImageClassifer sample as a template?

@jdduke Using a custom model is the next step - currently I'm just testing with the default models on the zoo.  Also good suggestion to look for alternate approaches, I just need to have the location of the object in the frame (so using just a classifier doesn't fit).

@BryanRansil The default models in the Detection Model Zoo will work. The tflite_convert in the latest nightly build corrected a bug that was on our end.

Please use the steps we detailed in this blog post

@achowdhery Good to know they should work - tried the toco command in the blog post on ssd_mobilenet_v2_coco_2018_03_29/frozen_inference_graph.pb and got the error

I then tried substituting the output array with concat,concat_1 and got the error
2018-09-06 12:04:52.844845: F tensorflow/contrib/lite/toco/tooling_util.cc:1613] Array Preprocessor/map/while/NextIteration, which is an input to the Merge operator producing the output array Preprocessor/map/while/Merge, is lacking min/max data, which is necessary for quantization. Either target a non-quantized output format, or change the input graph to contain min/max information, or pass --default_ranges_min= and --default_ranges_max= if you do not care about the accuracy of results.

This matches the error in https://github.com/tensorflow/tensorflow/issues/19014#issuecomment-405696749, but that thread is on custom graphs.
I then put in values for the two params, getting the following error
Unimplemented: this graph contains an operator of type (Unsupported TensorFlow op: Enter) for which the quantized form is not yet implemented. Sorry, and patches welcome (that's a relatively fun patch to write, mostly providing the actual quantized arithmetic code for this op).
Aborted (core dumped)

Again, this is all on r1.9. Is quantization unsupported on that release?

@BryanRansil
Can you please the latest checkpoints: http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18.tar.gz
http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18.tar.gz
?
We added native op for postprocessing that is available in r1.10. So, for your own model, you do need to go through export_tflite_ssd_graph.py scripts with steps we detailed in this blog post.

So I'm able to convert and run them. Unfortunately I get other runtime errors when I do so (at the end of this comment). When I attempt to use the script on ssd_mobilenet_v2_coco, I'm getting the error "object_detection.protos.SsdFeatureExtractor" has no field named "batch_norm_trainable", and I see in the logs that was deprecated and then removed. How is ssd_mobilenet_v2 supposed to be supported now? If not, A. Is there a plan to update the model zoo with this information and B. What's the best way of determining which models are supported?
Here's the error I'm getting with the newest checkpoints on master (so commit 4dcfddc5d12018a5a0fdca652b9221ed95e9eb23):
09-06 14:30:20.231 31110 31128 E AndroidRuntime: java.lang.ArrayIndexOutOfBoundsException: length=160; index=-2147483648
09-06 14:30:20.231 31110 31128 E AndroidRuntime:        at java.util.Vector.elementData(Vector.java:734)
09-06 14:30:20.231 31110 31128 E AndroidRuntime:        at java.util.Vector.get(Vector.java:750)
09-06 14:30:20.231 31110 31128 E AndroidRuntime:        at org.tensorflow.demo.TFLiteObjectDetectionAPIModel.recognizeImage(TFLiteObjectDetectionAPIModel.java:214)
09-06 14:30:20.231 31110 31128 E AndroidRuntime:        at org.tensorflow.demo.DetectorActivity$3.run(DetectorActivity.java:247)
09-06 14:30:20.231 31110 31128 E AndroidRuntime:        at android.os.Handler.handleCallback(Handler.java:873)
09-06 14:30:20.231 31110 31128 E AndroidRuntime:        at android.os.Handler.dispatchMessage(Handler.java:99)
09-06 14:30:20.231 31110 31128 E AndroidRuntime:        at android.os.Looper.loop(Looper.java:193)
09-06 14:30:20.231 31110 31128 E AndroidRuntime:        at android.os.HandlerThread.run(HandlerThread.java:65)

So the latest checkpoints are able to be loaded. Unfortunately they create the following error:
09-06 14:52:23.341  8812  8829 E AndroidRuntime: java.lang.ArrayIndexOutOfBoundsException: length=1280; index=-2147483647
09-06 14:52:23.341  8812  8829 E AndroidRuntime:        at java.util.Vector.elementData(Vector.java:734)
09-06 14:52:23.341  8812  8829 E AndroidRuntime:        at java.util.Vector.get(Vector.java:750)
09-06 14:52:23.341  8812  8829 E AndroidRuntime:        at org.tensorflow.demo.TFLiteObjectDetectionAPIModel.recognizeImage(TFLiteObjectDetectionAPIModel.java:214)
09-06 14:52:23.341  8812  8829 E AndroidRuntime:        at org.tensorflow.demo.DetectorActivity$3.run(DetectorActivity.java:247)
09-06 14:52:23.341  8812  8829 E AndroidRuntime:        at android.os.Handler.handleCallback(Handler.java:873)
09-06 14:52:23.341  8812  8829 E AndroidRuntime:        at android.os.Handler.dispatchMessage(Handler.java:99)
09-06 14:52:23.341  8812  8829 E AndroidRuntime:        at android.os.Looper.loop(Looper.java:193)
09-06 14:52:23.341  8812  8829 E AndroidRuntime:        at android.os.HandlerThread.run(HandlerThread.java:65)
09-06 14:52:23.345   914  1786 W ActivityManager:   Force finishing activity org.tensorflow.lite.demo/org.tensorflow.demo.DetectorActivity

I assume this is from using the incorrect labels file, when I use the pets label file in the blog post the index that triggers this error is smaller than in the error above.
However, I also updated tensorflow/models to master and ran export_tflite_ssd_graph.py on ssd_mobilenet_v2. This produced and error saying that batch_norm_trainable had been removed (which can be verified by looking at the logs). Therefore is ssd_mobilenet_v2 still supported? If it is, what was the most recent set of commits (for tensorflow/tensorflow and tensorflow/models) that it's been tested on? If not, A. is the model zoo going to be updated with this info and B. how do we determine which models are supported with which versions of TensorFlow?

@BryanRansil The app TFLiteObjectDetectionAPIModel.java references a COCO dataset pre-trained Mobilenet V1 SSD by default. Did the app build with pre-trained model?
In the blog post, we provided instructions to finetune weights for Pets dataset and provided the corresponding TF Lite files.
On your last question, we can double check if V2 SSD is working.

@achowdhery Thanks for checking V2 SSD! What's a reasonable timeline to expect for that checking to be done?
The app builds with the default model that's pointed to in the app, without crashing as I described in the last comment. Otherwise I'll work with the instructions on finetuning the latest models and post when that's successful.

Nagging Assignee @achowdhery: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @achowdhery: It has been 29 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

So as a follow up I was able to deal with this issue. For anyone wondering here's what I did:

I checked out r1.10 for both the models and tensorflow repos
Used bazel to clean the tensorflow repo, that way whenever I use bazel commands we'll use the r1.10 binaries. This by far is most likely what solved my problem, since I was trying different versions of tensorflow as I was dealing with this issue.
I trained using a command similar to this:

python ~/tensorflow/models/research/object_detection/model_main.py \
       --pipeline_config_path=${PIPELINE_CONFIG_PATH} \
       --model_dir=${MODEL_DIR} \
       --num_train_steps=${NUM_TRAIN_STEPS} \
       --num_eval_steps=${NUM_EVAL_STEPS} \
       --alsologtostderr

For specifically tflite I needed to use export_tflite_ssd_graph.py, not export_inference_graph. So the next command was something like:

python ~/tensorflow/models/research/object_detection/export_tflite_ssd_graph.py \
--pipeline_config_path=$CONFIG_FILE \
--trained_checkpoint_prefix=$CHECKPOINT_PATH \
--output_directory=$EXPORT_OUTPUT_DIR \
--add_postprocessing_op=true

Then we have the toco command. Similar to the blog, but I needed to add a few parameters:

./bazel-bin/tensorflow/contrib/lite/toco/toco \
  --input_file=$INPUT_PB_GRAPH \
  --output_file=$OUTPUT_TFLITE_FILE \
  --input_format=TENSORFLOW_GRAPHDEF --output_format=TFLITE \
  --inference_type=QUANTIZED_UINT8 \
  --input_shapes="1,300, 300,3" \
  --input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' \
  --std_values=128.0 --mean_values=128.0 \
  --allow_custom_ops --default_ranges_min=0 --default_ranges_max=6

Then when loading into the tflite example (so tensorflow/tensorflow/contrib/lite/examples/android) I needed some changes to compile or get past runtime errors and other behaviour:

git diff tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
diff --git a/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java b/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
index 9eb21de..2cfa7e0 100644
--- a/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
+++ b/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
@@ -208,17 +208,24 @@ public class TFLiteObjectDetectionAPIModel implements Classifier {
       // in label file and class labels start from 1 to number_of_classes+1,
       // while outputClasses correspond to class index from 0 to number_of_classes
       int labelOffset = 1;
-      recognitions.add(
-          new Recognition(
-              "" + i,
-              labels.get((int) outputClasses[0][i] + labelOffset),
-              outputScores[0][i],
-              detection));
+        final int classLabel = (int) outputClasses[0][i] + labelOffset;
+        if (inRange(classLabel, labels.size(), 0) && inRange(outputScores[0][i], 1, 0)) {
+            recognitions.add(
+                    new Recognition(
+                            "" + i,
+                            labels.get(classLabel),
+                            outputScores[0][i],
+                            detection));
+        }
     }
     Trace.endSection(); // "recognizeImage"
     return recognitions;
   }

+  private boolean inRange(float number, float max, float min) {
+    return number < max && number >= min;
+  }
+

And then I was able to run the tflite example on my phone! Thanks to @achowdhery and @jdduke for responding and the help!

I was getting an issue on
26829-27149/org.tensorflow.lite.demo W/System.err: java.lang.ArrayIndexOutOfBoundsException: length=10; index=-2147483648

i went into C:\Users\username\tensorflow\tensorflow\lite\examples\android in which i loaded it into android studio

I made the necessary changes in the DetectorActivity.java file=
2a) private static final String TF_OD_API_MODEL_FILE = "detect_1.tflite"; ## the model is converted into an tflite file
2b)   private static final String TF_OD_API_LABELS_FILE = "label.txt"; ##the label file for detection

3)clean the project in android studio

rebuild the project
run it in an android device by intsalling the apks

Here in the log console i am finding the java.lang.ArrayIndexOutOfBoundsException: length=10; index=-2147483648

I was getting an issue on
26829-27149/org.tensorflow.lite.demo W/System.err: java.lang.ArrayIndexOutOfBoundsException: length=10; index=-2147483648

@aravindchaluvadi This error is what step 6 above deals with. It comes from the lines

 recognitions.add(
     new Recognition(
         "" + i,
         labels.get((int) outputClasses[0][i] + labelOffset),
         outputScores[0][i],
         detection));

so I put the guards in (the whole "if (inRange...) {" part) and didn't get this error again.

Even if I prevent this error by using inRange(), the detection results are full of false positives. Any one can explain why this would produce weirdly out of bound indexes.

why do i get an error saying Specified output array “TFlite_Detection_PostProcess” is not produced by any op in this graph" even though it exist in graph eventhough it exist in my graph as specified by this?
 @achowdhery  @BryanRansil

why do i get an error saying Specified output array “TFlite_Detection_PostProcess” is not produced by any op in this graph" even though it exist in graph eventhough it exist in my graph as specified by this?
 @achowdhery @BryanRansil
I have exactly the same problem and I couldn't find any solution for that

While following the instructions here, I was getting this error when trying to run the bazel command: Specified output array "'TFLite_Detection_PostProcess'" is not produced by any op in this graph
I was able to resolve it by removing the ' characters around each of the TFLite_Detection_PostProcess entries in output_arrays command, like so:
output_arrays=TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3
I am doing this on Windows, so it may just be a weird way of how the command line interprets string characters.

@EdjeElectronics removing ' characters and, depending on which model I'm trying to convert via tflite, using --inference_type=FLOAT instead of --inference_type=QUANTIZED_UINT8 did the trick for me.

So as a follow up I was able to deal with this issue. For anyone wondering here's what I did:

I checked out r1.10 for both the models and tensorflow repos
Used bazel to clean the tensorflow repo, that way whenever I use bazel commands we'll use the r1.10 binaries. This by far is most likely what solved my problem, since I was trying different versions of tensorflow as I was dealing with this issue.
I trained using a command similar to this:

python ~/tensorflow/models/research/object_detection/model_main.py \
       --pipeline_config_path=${PIPELINE_CONFIG_PATH} \
       --model_dir=${MODEL_DIR} \
       --num_train_steps=${NUM_TRAIN_STEPS} \
       --num_eval_steps=${NUM_EVAL_STEPS} \
       --alsologtostderr

For specifically tflite I needed to use export_tflite_ssd_graph.py, not export_inference_graph. So the next command was something like:

python ~/tensorflow/models/research/object_detection/export_tflite_ssd_graph.py \
--pipeline_config_path=$CONFIG_FILE \
--trained_checkpoint_prefix=$CHECKPOINT_PATH \
--output_directory=$EXPORT_OUTPUT_DIR \
--add_postprocessing_op=true

Then we have the toco command. Similar to the blog, but I needed to add a few parameters:

./bazel-bin/tensorflow/contrib/lite/toco/toco \
  --input_file=$INPUT_PB_GRAPH \
  --output_file=$OUTPUT_TFLITE_FILE \
  --input_format=TENSORFLOW_GRAPHDEF --output_format=TFLITE \
  --inference_type=QUANTIZED_UINT8 \
  --input_shapes="1,300, 300,3" \
  --input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' \
  --std_values=128.0 --mean_values=128.0 \
  --allow_custom_ops --default_ranges_min=0 --default_ranges_max=6

Then when loading into the tflite example (so tensorflow/tensorflow/contrib/lite/examples/android) I needed some changes to compile or get past runtime errors and other behaviour:

git diff tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
diff --git a/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java b/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
index 9eb21de..2cfa7e0 100644
--- a/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
+++ b/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
@@ -208,17 +208,24 @@ public class TFLiteObjectDetectionAPIModel implements Classifier {
       // in label file and class labels start from 1 to number_of_classes+1,
       // while outputClasses correspond to class index from 0 to number_of_classes
       int labelOffset = 1;
-      recognitions.add(
-          new Recognition(
-              "" + i,
-              labels.get((int) outputClasses[0][i] + labelOffset),
-              outputScores[0][i],
-              detection));
+        final int classLabel = (int) outputClasses[0][i] + labelOffset;
+        if (inRange(classLabel, labels.size(), 0) && inRange(outputScores[0][i], 1, 0)) {
+            recognitions.add(
+                    new Recognition(
+                            "" + i,
+                            labels.get(classLabel),
+                            outputScores[0][i],
+                            detection));
+        }
     }
     Trace.endSection(); // "recognizeImage"
     return recognitions;
   }

+  private boolean inRange(float number, float max, float min) {
+    return number < max && number >= min;
+  }
+

And then I was able to run the tflite example on my phone! Thanks to @achowdhery and @jdduke for responding and the help!

Thank you Bryan! Spend me 2 days to find your post! I tried many solutions but just cannot get over the [1,10,4] input tensor of the Tensorflow lite Android OD example.
For the SSD_MobileNet_quatiazed_v2_coco model (that is my case), it should be ' int labelOffset = 0'.
And, good to see you here Evan. 💯 @EdjeElectronics

@innovimax  How to get this picture of the graph?

Even if I prevent this error by using inRange(), the detection results are full of false positives. Any one can explain why this would produce weirdly out of bound indexes.

From inspecting class index predictions coming out of MobileNet V2, it looks like the values start off as whole numbers (i.e. floats that are actually integers), and then turn into true floats. This makes me think that the prediction arrays are being initialized (or overwritten) with garbage, and then being filled back in with non-background detections, with unused predictions (since you have to specify a fixed number of predictions to return) being left as garbage floats. So, in addition to checking if the output class is in range, I also check if it is a whole number. This removed the noise for me.

@innovimax How to get this picture of the graph?

try Netron

These commands worked for me for ssd_mobilenet_v2_coco_2018_03_29 under Windows 10 (Tensorflow 1.13)

python export_tflite_ssd_graph.py --pipeline_config_path training/ssd_mobilenet_v2_coco.config --trained_checkpoint_prefix training/model.ckpt-XXXX --output_directory inference_graph_mobile --add_postprocessing_op=true
Make sure to replace model.ckpt-XXXX

tflite_convert --graph_def_file=inference_graph_mobile/tflite_graph.pb --output_file=tflite/detect.tflite --output_format=TFLITE --input_shapes=1,300,300,3 --input_arrays=normalized_input_image_tensor --output_arrays=TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3  --inference_type=FLOAT --allow_custom_ops

@HaFred
I'm also same problem while running segmentation model.
I don't have .txt file for this. Model is running good on IoS app but in android it gives exception
java.lang.ArrayIndexOutOfBoundsException: length=66049; index=66049
Any suggestion how I resolve this problem ?

@HaFred
I'm also same problem while running segmentation model.
I don't have .txt file for this. Model is running good on IoS app but in android it gives exception
java.lang.ArrayIndexOutOfBoundsException: length=66049; index=66049
Any suggestion how I resolve this problem ?

Seems like it is not the same issue we encountered here. The answer I quoted above is to solve shape does not match problem.

So as a follow up I was able to deal with this issue. For anyone wondering here's what I did:

I checked out r1.10 for both the models and tensorflow repos
Used bazel to clean the tensorflow repo, that way whenever I use bazel commands we'll use the r1.10 binaries. This by far is most likely what solved my problem, since I was trying different versions of tensorflow as I was dealing with this issue.
I trained using a command similar to this:

python ~/tensorflow/models/research/object_detection/model_main.py \
       --pipeline_config_path=${PIPELINE_CONFIG_PATH} \
       --model_dir=${MODEL_DIR} \
       --num_train_steps=${NUM_TRAIN_STEPS} \
       --num_eval_steps=${NUM_EVAL_STEPS} \
       --alsologtostderr

For specifically tflite I needed to use export_tflite_ssd_graph.py, not export_inference_graph. So the next command was something like:

python ~/tensorflow/models/research/object_detection/export_tflite_ssd_graph.py \
--pipeline_config_path=$CONFIG_FILE \
--trained_checkpoint_prefix=$CHECKPOINT_PATH \
--output_directory=$EXPORT_OUTPUT_DIR \
--add_postprocessing_op=true

Then we have the toco command. Similar to the blog, but I needed to add a few parameters:

./bazel-bin/tensorflow/contrib/lite/toco/toco \
  --input_file=$INPUT_PB_GRAPH \
  --output_file=$OUTPUT_TFLITE_FILE \
  --input_format=TENSORFLOW_GRAPHDEF --output_format=TFLITE \
  --inference_type=QUANTIZED_UINT8 \
  --input_shapes="1,300, 300,3" \
  --input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' \
  --std_values=128.0 --mean_values=128.0 \
  --allow_custom_ops --default_ranges_min=0 --default_ranges_max=6

Then when loading into the tflite example (so tensorflow/tensorflow/contrib/lite/examples/android) I needed some changes to compile or get past runtime errors and other behaviour:

git diff tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
diff --git a/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java b/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
index 9eb21de..2cfa7e0 100644
--- a/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
+++ b/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
@@ -208,17 +208,24 @@ public class TFLiteObjectDetectionAPIModel implements Classifier {
       // in label file and class labels start from 1 to number_of_classes+1,
       // while outputClasses correspond to class index from 0 to number_of_classes
       int labelOffset = 1;
-      recognitions.add(
-          new Recognition(
-              "" + i,
-              labels.get((int) outputClasses[0][i] + labelOffset),
-              outputScores[0][i],
-              detection));
+        final int classLabel = (int) outputClasses[0][i] + labelOffset;
+        if (inRange(classLabel, labels.size(), 0) && inRange(outputScores[0][i], 1, 0)) {
+            recognitions.add(
+                    new Recognition(
+                            "" + i,
+                            labels.get(classLabel),
+                            outputScores[0][i],
+                            detection));
+        }
     }
     Trace.endSection(); // "recognizeImage"
     return recognitions;
   }

+  private boolean inRange(float number, float max, float min) {
+    return number < max && number >= min;
+  }
+

And then I was able to run the tflite example on my phone! Thanks to @achowdhery and @jdduke for responding and the help!

Right but in other github issue someone suggested that Step 6 solve the problem statement java.lang.ArrayIndexOutOfBoundsException: length=66049; index=66049
And make sense because Step 6 solving in indexing problem which link with the label file. But in my case I don't have label file.
Code is working fine with other models and these models also have only .tflite file not .txt label file.
And the model which I'm using working fine with the IoS app.

I have not played with iOS development so there is nothing I could help on this. Maybe you could try doing the tensorflow lite Android flow from scartch
Frederick

From: Ehtasha<EMAIL_ADDRESS>Sent: Wednesday, February 26, 2020 8:11:29 PM
To: tensorflow/tensorflow<EMAIL_ADDRESS>Cc: Frederick HONG<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [tensorflow/tensorflow] Toco/TFLite_Convert for TFLite Problem (#22106)
So as a follow up I was able to deal with this issue. For anyone wondering here's what I did:

I checked out r1.10 for both the models and tensorflow repos
Used bazel to clean the tensorflow repo, that way whenever I use bazel commands we'll use the r1.10 binaries. This by far is most likely what solved my problem, since I was trying different versions of tensorflow as I was dealing with this issue.
I trained using a command similar to this:

python ~/tensorflow/models/research/object_detection/model_main.py
--pipeline_config_path=${PIPELINE_CONFIG_PATH}
--model_dir=${MODEL_DIR}
--num_train_steps=${NUM_TRAIN_STEPS}
--num_eval_steps=${NUM_EVAL_STEPS}
--alsologtostderr

For specifically tflite I needed to use export_tflite_ssd_graph.py, not export_inference_graph. So the next command was something like:

python ~/tensorflow/models/research/object_detection/export_tflite_ssd_graph.py
--pipeline_config_path=$CONFIG_FILE
--trained_checkpoint_prefix=$CHECKPOINT_PATH
--output_directory=$EXPORT_OUTPUT_DIR
--add_postprocessing_op=true

Then we have the toco command. Similar to the blog, but I needed to add a few parameters:

./bazel-bin/tensorflow/contrib/lite/toco/toco
--input_file=$INPUT_PB_GRAPH
--output_file=$OUTPUT_TFLITE_FILE
--input_format=TENSORFLOW_GRAPHDEF --output_format=TFLITE
--inference_type=QUANTIZED_UINT8
--input_shapes="1,300, 300,3"
--input_arrays=normalized_input_image_tensor
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3'
--std_values=128.0 --mean_values=128.0
--allow_custom_ops --default_ranges_min=0 --default_ranges_max=6

Then when loading into the tflite example (so tensorflow/tensorflow/contrib/lite/examples/android) I needed some changes to compile or get past runtime errors and other behaviour:

git diff tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
diff --git a/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java b/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
index 9eb21de..2cfa7e0 100644
--- a/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
+++ b/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
@@ -208,17 +208,24 @@ public class TFLiteObjectDetectionAPIModel implements Classifier {
// in label file and class labels start from 1 to number_of_classes+1,
// while outputClasses correspond to class index from 0 to number_of_classes
int labelOffset = 1;

 recognitions.add(

     new Recognition(

         "" + i,

         labels.get((int) outputClasses[0][i] + labelOffset),

         outputScores[0][i],

         detection));

   final int classLabel = (int) outputClasses[0][i] + labelOffset;

   if (inRange(classLabel, labels.size(), 0) && inRange(outputScores[0][i], 1, 0)) {

       recognitions.add(

               new Recognition(

                       "" + i,

                       labels.get(classLabel),

                       outputScores[0][i],

                       detection));

   }

}
Trace.endSection(); // "recognizeImage"
return recognitions;
}

private boolean inRange(float number, float max, float min) {

return number < max && number >= min;

}

And then I was able to run the tflite example on my phone! Thanks to @achowdheryhttps://github.com/achowdhery and @jddukehttps://github.com/jdduke for responding and the help!
Right but in other github issue someone suggested that Step 6 solve the problem statement java.lang.ArrayIndexOutOfBoundsException: length=66049; index=66049
And make sense because Step 6 solving in indexing problem which link with the label file. But in my case I don't have label file.
Code is working fine with other models and these models also have only .tflite file not .txt label file.
And the model which I'm using working fine with the IoS app.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/tensorflow/tensorflow/issues/22106?email_source=notifications&email_token=ADKX3ARF2FBWEBQELN22E5LREZL7DA5CNFSM4FTRNUE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM77PYI#issuecomment-591394785, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADKX3AWDEX645YO4GYDJNE3REZL7DANCNFSM4FTRNUEQ.

Okey

@BryanRansil I am trying to convert yolov3-tiny to tflite how we can achieve it.
I followed your commands for ssd model its working fine.
It will be helpful for all others as well, if we can successfully convert yolo model to tflite for android applications

So as a follow up I was able to deal with this issue. For anyone wondering here's what I did:

I checked out r1.10 for both the models and tensorflow repos
Used bazel to clean the tensorflow repo, that way whenever I use bazel commands we'll use the r1.10 binaries. This by far is most likely what solved my problem, since I was trying different versions of tensorflow as I was dealing with this issue.
I trained using a command similar to this:

python ~/tensorflow/models/research/object_detection/model_main.py \
       --pipeline_config_path=${PIPELINE_CONFIG_PATH} \
       --model_dir=${MODEL_DIR} \
       --num_train_steps=${NUM_TRAIN_STEPS} \
       --num_eval_steps=${NUM_EVAL_STEPS} \
       --alsologtostderr

For specifically tflite I needed to use export_tflite_ssd_graph.py, not export_inference_graph. So the next command was something like:

python ~/tensorflow/models/research/object_detection/export_tflite_ssd_graph.py \
--pipeline_config_path=$CONFIG_FILE \
--trained_checkpoint_prefix=$CHECKPOINT_PATH \
--output_directory=$EXPORT_OUTPUT_DIR \
--add_postprocessing_op=true

Then we have the toco command. Similar to the blog, but I needed to add a few parameters:

./bazel-bin/tensorflow/contrib/lite/toco/toco \
  --input_file=$INPUT_PB_GRAPH \
  --output_file=$OUTPUT_TFLITE_FILE \
  --input_format=TENSORFLOW_GRAPHDEF --output_format=TFLITE \
  --inference_type=QUANTIZED_UINT8 \
  --input_shapes="1,300, 300,3" \
  --input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' \
  --std_values=128.0 --mean_values=128.0 \
  --allow_custom_ops --default_ranges_min=0 --default_ranges_max=6

Then when loading into the tflite example (so tensorflow/tensorflow/contrib/lite/examples/android) I needed some changes to compile or get past runtime errors and other behaviour:

git diff tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
diff --git a/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java b/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
index 9eb21de..2cfa7e0 100644
--- a/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
+++ b/tensorflow/contrib/lite/examples/android/app/src/main/java/org/tensorflow/demo/TFLiteObjectDetectionAPIModel.java
@@ -208,17 +208,24 @@ public class TFLiteObjectDetectionAPIModel implements Classifier {
       // in label file and class labels start from 1 to number_of_classes+1,
       // while outputClasses correspond to class index from 0 to number_of_classes
       int labelOffset = 1;
-      recognitions.add(
-          new Recognition(
-              "" + i,
-              labels.get((int) outputClasses[0][i] + labelOffset),
-              outputScores[0][i],
-              detection));
+        final int classLabel = (int) outputClasses[0][i] + labelOffset;
+        if (inRange(classLabel, labels.size(), 0) && inRange(outputScores[0][i], 1, 0)) {
+            recognitions.add(
+                    new Recognition(
+                            "" + i,
+                            labels.get(classLabel),
+                            outputScores[0][i],
+                            detection));
+        }
     }
     Trace.endSection(); // "recognizeImage"
     return recognitions;
   }

+  private boolean inRange(float number, float max, float min) {
+    return number < max && number >= min;
+  }
+

And then I was able to run the tflite example on my phone! Thanks to @achowdhery and @jdduke for responding and the help!

Hi @BryanRansil  I used step 6 and change the TFLiteObjectDetectionAPIModel.java .But I still got the following error:

2020-08-05 16:09:30.781 28332-28442/org.tensorflow.lite.examples.detection E/AndroidRuntime: FATAL EXCEPTION: inference
Process: org.tensorflow.lite.examples.detection, PID: 28332
java.lang.IllegalArgumentException: Cannot copy to a TensorFlowLite tensor (serving_default_input_tensor:0) with 3 bytes from a Java Buffer with 1080000 bytes.
at org.tensorflow.lite.Tensor.throwIfSrcShapeIsIncompatible(Tensor.java:444)
at org.tensorflow.lite.Tensor.setTo(Tensor.java:189)
at org.tensorflow.lite.NativeInterpreterWrapper.run(NativeInterpreterWrapper.java:154)
at org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs(Interpreter.java:347)
at org.tensorflow.lite.examples.detection.tflite.TFLiteObjectDetectionAPIModel.recognizeImage(TFLiteObjectDetectionAPIModel.java:196)
at org.tensorflow.lite.examples.detection.DetectorActivity$2.run(DetectorActivity.java:181)
at android.os.Handler.handleCallback(Handler.java:883)
at android.os.Handler.dispatchMessage(Handler.java:100)
at android.os.Looper.loop(Looper.java:237)
at android.os.HandlerThread.run(HandlerThread.java:67)

Any help is appreciated.

Cannot copy to a TensorFlowLite tensor (serving_default_input_tensor:0) with 3 bytes from a Java Buffer with 1080000 bytes.

What is the shape of this tensor in your converted .tflite model? You might need to manually resize it (using Interpreter.resizeInput()). It looks like the buffer you're providing doesn't match the expected size for that tensor.


feat(prevent-no-label-execution): use notice message

Description

It is easier for users to understand what label to be added.
Reference: https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-a-notice-message
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[ ] I've confirmed the contribution guidelines.
[ ] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

Test result: https://github.com/autowarefoundation/autoware-github-actions/actions/runs/2909505149


Clicking on url throws an error

Steps to reproduce: While signed in to an Evernote account, create a note and add an url to it. Then double click on it and this error will appear
Oops, we encountered an error.
Access denied.
You have accessed a location on this server that is not available.
You may need to Sign in to your account to access this page.

@fsdias thank you for report.
Please, check issue #2

sorry for the duplicate, somehow I missed it even though it has two tags


re-install Dockerfile markers

squash with super secrets commit

@rhatdan @sallyom Somehow the super secrets commit removed the Dockerfile modifications that the distribution commit made, which was causing us to build using ubuntu. This nullifies those deletion hunks in the super secrets commit.

Fixed


Fix localization for Mac OSX

Oh, the great horror! The recent fix for localization doesn't appear to work as described in the readme on OSX.
$ ASFLAGS=-DSUBJECT=\\"Ola\\" make test
clang -o pad pad.c
nasm -DSUBJECT=\Ola\ -o boot-raw.bin boot.asm
nasm: error: more than one input file specified
type `nasm -h' for help
make: *** [boot-raw.bin] Error 1

This works, however:
$ ASFLAGS=-DSUBJECT=\\\"Ola\\\" make
clang -o pad pad.c
nasm -DSUBJECT=\"Ola\" -o boot-raw.bin boot.asm
./pad boot-raw.bin boot.bin

And it still works with no customization as well:
$ make
clang -o pad pad.c
nasm  -o boot-raw.bin boot.asm
./pad boot-raw.bin boot.bin

Oh, the woes of using inferior operating systems. Are you able to test that the extra escaping still works for you, @kusma? If so, we can simply fix the documentation, and all will be right with the world again.

Actually, that was a typo on my behalf. I also need three slashes, so yeah. Sorry about that!

np, thanks!


show custom track for v4 functions

As is now all the link to #track-custom-data are not showing data as the content its only available on v3
on this PR we are removing the v3 block for this information as this should still work as expected in v4 (at least is not mentioned in the migration guide)

@ggailey777
Can you review the proposed changes?
Important: When the changes are ready for publication, adding a #sign-off comment is the best way to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

#sign-off

@ggailey777 gently ping on this, let me know if you need anything from my side to make the review easier


build: fix python load paths

Fixed python load paths so that vppcfg will work installed as python
library and standalone from the source directory, fixing load
pathes for resources such as yaml files along the way.
Added a make target for pylint called 'make check-style', fixed a
number of minor pylint issues along the way.
Signed-off-by: Ray Kinsella<EMAIL_ADDRESS>

I patched it in and bumped the setup.py and debian/rules both to 0.0.2, then:

pip installed, pip upgraded, pip uninstalled and dpkg -i installed it as 0.0.2
schema is there, relative paths work, you win the internet

pim@hippo:~$ vppcfg dump -o foo.yaml
[INFO    ] vppcfg.vppapi.connect: VPP version is 22.10-rc0~33-gd6c3b1f1f
[INFO    ] vppcfg.vppapi.write: Wrote YAML config to foo.yaml
pim@hippo:~$ ls -la foo.yaml
-rw-rw-r-- 1 pim pim 510 Jul 12 15:18 foo.yaml

And:
pim@hippo:~$ vppcfg plan -c src/vpp/hippo.yaml -o foo.exec
[INFO    ] root.main: Loading configfile src/vpp/hippo.yaml
[INFO    ] vppcfg.config.valid_config: Configuration validated successfully
[INFO    ] root.main: Configuration is valid
[INFO    ] vppcfg.vppapi.connect: VPP version is 22.10-rc0~33-gd6c3b1f1f
[INFO    ] vppcfg.reconciler.write: Wrote 18 lines to foo.exec
[INFO    ] root.main: Planning succeeded
pim@hippo:~$ ls -la foo.exec
-rw-rw-r-- 1 pim pim 919 Jul 12 15:18 foo.exec

You win the internet for the day!


test/scripts: Disable network debug logging

We’ve eliminated most of the network instability, so this output is not useful.

LGTM


Fix unchecked user becomes checked again when searching

Details

The logic to add the saved selected user is re-triggers every time the search keyword is changed even though we only want to run it once.
Fixed Issues

$ https://github.com/Expensify/App/issues/36350
PROPOSAL: https://github.com/Expensify/App/issues/36350#issuecomment-1942208390
Tests

Same as QA Steps

[x] Verify that no errors appear in the JS console

Offline tests

Same as QA Steps
QA Steps

Open workspace invite page
Select any user and press Next
Go back
Unselect the user
Search for the user
Verify the user is unchecked

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

https://github.com/Expensify/App/assets/50919443/02ed67f2-567a-4a19-a2b0-725ecc9c25d8

Android: mWeb Chrome

https://github.com/Expensify/App/assets/50919443/4fbc94e8-ad61-4d4c-9212-c8c64ee2e75c

iOS: Native

https://github.com/Expensify/App/assets/50919443/5c129756-8632-408e-8cb2-9a793410ca19

iOS: mWeb Safari

https://github.com/Expensify/App/assets/50919443/6e097b44-4bda-4df9-9178-f795048d0de9

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/50919443/6bfc2ec9-7860-4c52-814b-7c326b828c23

MacOS: Desktop

https://github.com/Expensify/App/assets/50919443/9c3f2cfc-8edb-4ff0-913f-ad2fd7d33947

Reviewer Checklist

[x] I have verified the author checklist is complete (all boxes are checked off).
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the steps for local testing are in the Tests section
[x] I verified the steps for Staging and/or Production testing are in the QA steps section
[x] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I checked that screenshots or videos are included for tests on all platforms
[x] I included screenshots or videos for tests on all platforms
[x] I verified tests pass on all platforms & I tested again on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

Please merge main to fix TS error

Done

Oh snap, what happened here? I must have forgot to merge. Can we update with merge conflicts and then re-review @situchan ? Thanks.

@AndrewGable conflicts solved

Thanks, can you quickly re-review @situchan as this has been stale for so long? (my fault)

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to production by https://github.com/mountiny in version: 1.4.64-6 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Feature : Adjusting the Alignment of Homepage Components

Issue: Adjusting the Alignment of Homepage Components

Assessing Alignment: Reviewing the layout of homepage components.
Making Adjustments: Realigning elements for consistent spacing and balance.
Testing Responsiveness: Ensuring the layout looks good on various devices.
Maintaining Design Consistency: Aligning changes with design standards.
Documenting Changes: Capturing before-and-after screenshots for reference.

@Harshdev098
Pls assign me this issue, give me labels, level, etc.

@Harshdev098
Pls give me level3 to this issue.

Will give levels after opening of the PR, don't worry


Connecting cameras for homeKit remote access

Hello!
I want to connect a usb-wired and a wireless camera to my raspberry pi (3) and access these cameras remotely via the "Home" App from Apple. I have an Apple TV 4 (for remote access and automation).
For this project I have some questions:
a) Does Apple pass-through (unofficial) paired Homekit Devices (via Homebridge) and can I access these remotely (via my ATV)?
b) Which (simple, cheap, <= 1080 Pixels) cameras are compatible to raspberry and homebridge? (usb and wireless models) Do you have some amazon links?
c) Is https://www.npmjs.com/package/homebridge-camera-ffmpeg the best plugin for this approch?
d) It is possible, that i can get an notification (since iOS 10.2) if the remote camera detects "movement" ? Or must I use another device (motion sensor)?
Thanks a lot

a) Yes.
b) Any USB camera that support v4l2 should world.
c) It depends on your camera
d) This depends on the plugin implementation. HomeKit will show notification when a motion sensor detects motion in a room. Plugin can choose to implement camera movement as "motion sensor" then you get notification.

Thanks,
what about v4|2 - do you have some hyperlinks for me?

https://en.wikipedia.org/wiki/Video4Linux

b) the raspicam both regular and noir works fine now with latest updates to Jessie and motion 4.0.1 https://github.com/Motion-Project/motion/releases
I get 13fps with motion detection on with 1280x720 on a rpi 3b.
c)you should use that addon for pairing and setup, then switch to homebridge-camera-ffmpeg-omx if run on a rpi, it uses a codec with hw support. there are problems with the -omx plugin regarding aspects other than 16:9, and the pairing/setup. (it pairs but streaming does not work, unless it is paired first on the non -omx)
Im planing a patch for the -ffmpeg plugin with option to choose codec.
d)You have to decide if you want motion detection from image difference or PIR.
If you decide on motion I have a plugin for that https://github.com/MrsDelish/homebridge-motion-fifo

I am using motion on individual Raspberry Pis in the house. Each Raspberry has an USB camera attached. Each camera works using MotionEye and direction hit to the IP address.  Once I had that working then I used NodeJS to get the camera's working in HomeKit.  Specifically the CameraCore.js.
Here are a few screen shot of the cameras in the Home app.  I can tell you it is pretty awesome to see the video in the home app.  When I turn on motion detection MotionEye catches the movements, saves the photo/video, and sends me an email or text message.  Works great.

Hi,
I can't see my camera in Home but they work in MotionEye, can you help me ?

@Nelrod49 did you manage to solve this? I have similar issue of unable to show up in Home but able in MotionEye

@cherrythia No sorry I d'ont find how to do it. If you find let me know ;)

@ehusar How did you modify CameraCore.js to work with USB webcams?

@cherrythia try this sudo modprobe bcm2835-v4l2

@Nelrod49 @cherrythia Hi, Sorry to disturb, but did you manage to fix this? On motionEye everything is fine, but exactly same issue for me, Home app saying Camera not responding..
Hope to hear from any of you!

@Nelrod49 @cherrythia @RichrdJ and @ca13ra1 : Also have the same issue ... is some of you managed to resolve that issue ?

@thenexus7 @RichrdJ I currently don't have a Pi camera to test this. I broke mine adjusting the focus (dumb idea). I think it was having difficulties staying connected from what I remember, it would constantly disconnect. I'm looking into just getting a HomeKit compatible IP camera, save my time and money.

@Nelrod49 @cherrythia @RichrdJ  @ca13ra1 and thenexus7  same issue any fix??

Same issue. On motionEye everything is fine, but Home app saying Camera not responding. When i stop motionEye service, I can see my camera in Home app

@hoangtuit I have the exact same problem...I thought they can't run in parallel.
So I decided that I might be able to use home app to stream the cam. And that I use another pi to stream the same cam in motioneye...didn't succeed yet because I am not sure which ip:port and login I should fill in while adding in motioneye

Hi all,
I finally fixed the error. There where 2 problems to fix. First there was no task file in my HAP-NodeJS and second the lib/Camera.js was not updated during the installation. I think both error came from installing HAP-NodeJS in an unexpected folder.
After taking a look to the installer and running it step by step it came out obvious to me at the end:
sudo wget http://goo.gl/F2EB5M -O task
sudo chmod +x task
...
cd lib/
sudo mv Camera.js OriginalCamera.js.bak
sudo wget http://goo.gl/Maq3o7 -O Camera.js

This was supposed to be executed in the cd /home/pi/HAP-NodeJS folder, but since I have no such folder it was not working. I moved to the correct HAP-NodeJS folder and it started to work right away.
Hope it helps.

@cecheto Your solution resolved the issue! Thanks.


Clearer E2E stack traces

Which issue this PR addresses:
This is a small PR to improve the readability of E2E test output by using the GinkgoHelper(). I noticed this feature while doing https://issues.redhat.com/browse/ARO-5321. The details are here but the tl;dr is we get more informative/clearer stack traces if there is an error in the helpers.
What this PR does / why we need it:
To get access to the GinkgoHelper() I needed to bump Ginkgo to 2.8.0 (currently at 2.7.0) and while I was at it I went all the way to 2.14.0.
Note: 2.15.0 is out but requires Go 1.20.0.
Test plan for issue:
The current E2E tests.
Is there any documentation that needs to be updated for this PR?
Is there any documentation that needs to be updated for this PR?
No. I think having this in the existing helpers is enough documentation.

/azp run e2e

@mociarain I think you'll need to adjust your target branch to master to get e2e going on this. I assume the intention is to merge to master?

@mociarain I think you'll need to adjust your target branch to master to get e2e going on this. I assume the intention is to merge to master?

Nope. I wanted to try a workflow of breaking PR's in smaller pieces to help review but with our current Git setup I'm not sure it'll actually work. Contex here if you're curious.

@mociarain two cents - I think breaking up large changes into smaller PRs is always preferable, but I'd target master in both PRs to keep the convention that we're used to and logically separate the enhancements. It seems that the two PRs (https://github.com/Azure/ARO-RP/pull/3393 and https://github.com/Azure/ARO-RP/pull/3392) are related but are solving different problems, and approvals from this PR won't carry over to the other, so it's duplicated review and merge work in a way. I'd personally be good to LGTM this.

Closing in favour of: https://github.com/Azure/ARO-RP/pull/3452

Gingko has been bumped to a good enough version so rebasing and handling this isn't worth it


Fixes SIP layer to correctly indicate truncated SIP packets

Currently, if a SIP packet is truncated, the parser does not indicate this.  The fix calls SetTruncated() if this condition is detected.   SIP packets can be considered as truncated if each line does not end in a CR-LF

@googlebot I signed it!

Closes #744

Any chance of getting this pulled?  Do I need to do anything else?

There you go, sorry for the delay!


Release Mode

I would like to add a "Release" mode, intended for people to use when releasing code flow to npm.  I would like it to only add typechecks for arguments (not return types etc.).  I would also like to wrap all typechecks in:
if (process.env.NODE_ENV !== 'production') {
  // ...typechecks here...
}

That way, most people would get the benefit of helpful type errors in development, but not have the overhead in production, since you can compile away the process.env.NODE_ENV then do dead-code elimination.  Currently I have to publish two versions (dev and prod) of the same npm module to offer this flexibility.

@ForbesLindesay to accomplish a similar thing I've been creating two builds of my modules and then deciding which to include in an entry point, so my folder structure looks like this:
lib
lib-checked
src
index.js

and index.js looks like:
if (process.env.NODE_ENV === 'production') {
  module.exports = require('./lib');
}
else {
  module.exports = require('./lib-checked');
}

The issue with your suggestion is that most people don't process node_modules with babel and so DCE  won't happen. The VM can't do it because process.env.NODE_ENV can change at any time, so the overhead would still be there.

In this example though, for client side, you're shipping users (who don't use browserify) two complete copies of the code, which is surely worse?

I suppose more people use tools like envify/babel on their client side dependencies than for server side node_modules though, so maybe this is the right option.

I suppose more people use tools like envify/babel on their client side dependencies than for server side node_modules though

Exactly, people deploying to the client are much more likely to be using tools which can do this.


Type with overloaded method with a generic overload matches type with incompatible method

TypeScript Version:  3.9.0-dev.20200305
Search Terms:
generic type overload
Code
interface I1 {
    f<V extends 'a'>(value: V): void;
    f(value: 'b'): void;
}

interface I2 {
    f(value: 'c'): void;
}

function test(x: I1) {
    function u(y: I2) { }

    u(x);
}

Expected behavior:
Compilation error, u cannot accept x as I1 cannot handle f('c')
Actual behavior:
Compilation succeeds.
Note that all of the following fail, as expected:
interface I1 {
    // f<V extends 'a'>(value: V): void;
    f(value: 'b'): void;
}

interface I2 {
    f(value: 'c'): void;
}

function test(x: I1) {
    function u(y: I2) { }

    u(x);
}

interface I1 {
    f<V extends 'a'>(value: V): void;
    // f(value: 'b'): void;
}

interface I2 {
    f(value: 'c'): void;
}

function test(x: I1) {
    function u(y: I2) { }

    u(x);
}

interface I1 {
    f(value: 'a'): void;
    f(value: 'b'): void;
}

interface I2 {
    f(value: 'c'): void;
}

function test(x: I1) {
    function u(y: I2) { }

    u(x);
}

Playground Link:
https://www.typescriptlang.org/play/?ts=3.9.0-dev.20200305#code/JYOwLgpgTgZghgYwgAgJIEZkG8CwAoZQ5GAHgDVkIAPSEAEwGdkByOZgPgAoA3OAGwCuEAFzIyASlHcA9sDoBufEWI9+Q0cwBGzSchlzFeAL758oSLEQpUAJmxKiMVYJEsEOqbIX4TefDAEQBDBgaRBkSAYwTipRDHF7AkdA4NDwgU4ATzibBKxkXwdCDKpxQyMgA
Related Issues:
Not sure.

If I recall correctly, in the presence of multiple overloads, TypeScript relates against the erased version of each signature signature (where all type parameters become any). @ahejlsberg probably recalls why we kept this in place.
Do you have an example of where this came up as an issue in your code?

It's a bit obscure tbh, I am trying to do something like this
export function addEvents<THandlers>(target: {
    on<TKey extends keyof THandlers>(type: TKey, handler: THandlers[TKey]): void;
    off<TKey extends keyof THandlers>(type: TKey, handler: THandlers[TKey]): void;
}, handlers: THandlers) {
    for (const key in handlers) {
        target.on(key, handlers[key]);
    }

    return (): void => {
        for (const key in handlers) {
            target.off(key, handlers[key]);
        }
    };
}

and apply it to CodeMirror library object where one overload of on is generic.
It's kind of a workaround for #37204.


kotatogram: Add version 1.3.4

Forks of already appliations are not subject of addition to official bucket.

In fact, it's a 3-rd party client of telegram, which contains much of enhanced features. Just it's introduced as "telegram fork" by the author. The relationship between Kotatogram and telegram is just like chrome and chromium, we could add it to this bucket.


IPv6 addresses are case-sensitive in AAAA records with --diff

I have the following record in my zone file:
@ 3600 AAAA 2001:DB8::
If I run the script with --diff the output is:
DELETE @ 3600 AAAA 2001:db8::
ADD @ 3600 AAAA 2001:DB8::

Because the address returned via LIST is in lowercase, it does not match any of the records in my zone file, so it is deleted. Likewise, the record in my zone file doesn't match anything in LIST, so it is added.
Strictly speaking, IPv6 addresses are case-insensitive, however RFC 5952 section 4.3 states:

The characters "a", "b", "c", "d", "e", and "f" in an IPv6 address MUST be represented in lowercase.

RFC 5952 is mainly concerned with displaying textual representations of IPv6 addresses, but if we are going to pick a rule for consistency we should probably follow the RFC.
There are three ways we could address this problem:

Convert IPv6 addresses to lowercase before processing.
Use a case-insensitive comparison in --diff.
Fail validation on IPv6 addresses containing uppercase characters.

The first option is the simplest to implement, and should not break any existing zone files. The third option is perhaps better from a compliance perspective, because we are forcing users to follow the relevant RFCs and not silently adjusting their records before sending (even though this adjustment doesn't change the result). However, this may break existing zone files and so should only be included in a major version bump.

I am going to leave this issue open for a while in case anyone else has comments to add. I am minded to go with the third option and roll this change into the 2.0.0 release.

Regardless of which option is chosen, we'll need to add unit tests for both success and failure.

Going to close this for now - the workaround is to use lowercase IPv6 addresses (as per the RFC 5952) as that means your zone file and LIST will match.


Fix issue 24686: SumType stopped working with unmatched DeducedParame…

…terType template

This should probably go to stable CC @JohanEngelen @RazvanN7 @dkorpel

I took the freedom to create Industry label. This is blocking Weka on upgrading to the latest language version.

Buildkite error seems unrelated?

please retitle the commit message : "Fix issue 24686:...". also please target stable

please retitle the commit message : "Fix issue 24686:...". also please target stable

The commit message is like that. Did I miss something? I thought that was for the bot to grab.

Gah, sorry. `"Fix _Bugzilla 24686:...".

Gah, sorry. `"Fix Bugzilla issue 24686:...".

Got it, I certainly didn't contribute for a while, I didn't know about this change. Thanks!

Got it, I certainly didn't contribute for a while, I didn't know about this change. Thanks!

Well the bot's message does say that, but I didn't notice it the first few times. I only really look for the "⚠⚠⚠ Warnings ⚠⚠", and don't read the actual content :)

I rebased and retargeted this PR onto stable

Again an unrelated CI failure... please merge this.

I rebased and retargeted this PR onto stable

Thanks. Sorry, I'm not used to go to my personal github that often. I wish I could merge it to keep track of things more often. Need to change my workflow a bit to be aware of these kind of things.


Getting ABRecordID

I need the ABRecordID to able to send/share messages to WhatsApp. I think SwiftAddressBookPerson should have a getter for ABRecordID
Currently Im getting it with:
let recordID : ABRecordID = ABRecordGetRecordID(contact.internalRecord)

SwiftAddressBookPerson is a subclass of SwiftAddressBookRecord, which has the recordId property.

Ohh I was looking in the wrong place! Thanks!


Main to Stage sync up

Main to Stage sync up
Test URLs:

Before: https://main--express--adobecom.hlx.page/express/?lighthouse=on
After: https://stage--express--adobecom.hlx.page/express/?lighthouse=on

Codecov Report

Merging #201 (1cd2c9f) into stage (028306e) will increase coverage by 0.31%.
Report is 15 commits behind head on stage.
The diff coverage is 85.71%.

@@            Coverage Diff             @@
##            stage     #201      +/-   ##
==========================================
+ Coverage   35.37%   35.68%   +0.31%
==========================================
  Files          27       27
  Lines        9489     9465      -24
==========================================
+ Hits         3357     3378      +21
+ Misses       6132     6087      -45

Files Changed
Coverage Δ

express/scripts/all-templates-metadata.js
50.00% <0.00%> (+18.49%)
:arrow_up:

express/blocks/columns/columns.js
43.06% <100.00%> (+0.40%)
:arrow_up:

express/scripts/gnav.js
48.85% <100.00%> (+0.16%)
:arrow_up:

express/scripts/instrument.js
7.22% <100.00%> (ø)

... and 1 file with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


brid.gy "likes" end up as type="mention" in wp_commentmeta semantic_linkbacks_type

(maybe related to #54)
Hi,
I'd love to get a hint on where to look for the right place where the plugin's code decides which semantic_linkbacks_type is stored in wp_commentmeta table. It seems that in my install it'll fallback to the generic "mention" when it recives a webmention from brid.gy.
Here's  a log from brid.gy:
2016-05-26 10:07:08.268750 D Params: NestedMultiDict([('response_key', u'aglzfmJy...')])
2016-05-26 10:07:08.523300 I Source: webrocker (Twitter) webrocker, http://brid-gy.appspot.com/twitter/webrocker
2016-05-26 10:07:08.523920 I Created by this poll: http://brid-gy.appspot.com/log?start_time=1464257167&key=aglzfmJyaWQtZ3lyFgsSB1R3aXR0ZXIiCXdlYnJvY2tlcgw
2016-05-26 10:07:08.524730 I Starting Response like tag:twitter.com,2013:735747260508602368_favorited_by_205651658 https://twitter.com/webrocker/status/735747260508602368#favorited-by-205651658
2016-05-26 10:07:08.556630 I Webmention from https://brid-gy.appspot.com/like/twitter/webrocker/735747260508602368/205651658 to https://www.webrocker.de/?p=24638
2016-05-26 10:07:08.616720 I Using cached webmention endpoint u'W https webrocker.de': https://www.webrocker.de/blog/?webmention=endpoint
2016-05-26 10:07:08.617180 I Sending...
2016-05-26 10:07:11.340670 I Sent! {'body': u'array(19) {\n ["comment_ID"]=>\n string(4) "7788"\n ["comment_post_ID"]=>\n string(5) "24638"\n ["comment_author"]=>\n string(13) "Holger Bartel"\n ["comment_author_email"]=>\n string(0) ""\n ["comment_author_url"]=>\n string(79) "https://brid-gy.appspot.com/like/twitter/webrocker/735747260508602368/205651658"\n ["comment_author_IP"]=>\n string(15) "<IP_ADDRESS>"\n ["comment_date"]=>\n string(19) "2016-05-26 12:07:10"\n ["comment_date_gmt"]=>\n string(19) "2016-05-26 10:07:10"\n ["comment_content"]=>\n string(158) "This Article was mentioned on brid-gy.appspot.com"\n ["comment_karma"]=>\n string(1) "0"\n ["comment_approved"]=>\n string(1) "0"\n ["comment_agent"]=>\n string(102) "Bridgy (https://brid.gy/about) AppEngine-Google; (+http://code.google.com/appengine; appid: s~brid-gy)"\n ["comment_type"]=>\n string(10) "webmention"\n ["comment_parent"]=>\n string(1) "0"\n ["user_id"]=>\n string(1) "0"\n ["children"]=>\n NULL\n ["populated_children"]=>\n bool(false)\n ["post_fields"]=>\n array(21) {\n [0]=>\n string(11) "post_author"\n [1]=>\n string(9) "post_date"\n [2]=>\n string(13) "post_date_gmt"\n [3]=>\n string(12) "post_content"\n [4]=>\n string(10) "post_title"\n [5]=>\n string(12) "post_excerpt"\n [6]=>\n string(11) "post_status"\n [7]=>\n string(14) "comment_status"\n [8]=>\n string(11) "ping_status"\n [9]=>\n string(9) "post_name"\n [10]=>\n string(7) "to_ping"\n [11]=>\n string(6) "pinged"\n [12]=>\n string(13) "post_modified"\n [13]=>\n string(17) "post_modified_gmt"\n [14]=>\n string(21) "post_content_filtered"\n [15]=>\n string(11) "post_parent"\n [16]=>\n string(4) "guid"\n [17]=>\n string(10) "menu_order"\n [18]=>\n string(9) "post_type"\n [19]=>\n string(14) "post_mime_type"\n [20]=>\n string(13) "comment_count"\n }\n ["comment_subscribe"]=>\n string(1) "N"\n}\nhttps://twitter.com/webrocker/status/735747260508602368#favorited-by-205651658', 'request': u'POST https://www.webrocker.de/blog/?webmention=endpoint (with source=https://brid-gy.appspot.com/like/twitter/webrocker/735747260508602368/205651658, target=https://www.webrocker.de/?p=24638)', 'http_status': 200}
2016-05-26 10:07:11.434250 I Setting last_webmention_sent

The way I see this, there are two possible indicators that this is a "like":
The webmention source contains /like/ in the URL, or the fragment-Identifier "#favorited-by-205651658"
I'm looking through the sematic-linkbacks.php source, but I don't see/understand where the comment meta data for semantic_linkbacks_type first gets set - all I see are the instances where checks run against this meta data (and a fallback to "mention" inside the comment_text_excerpt function).
When I receive the mention from the log above, it gets stored as "mention". When I change this manually in the database to "like", the display is fine. But somehow the Twitter "likes" via brid.gy end up as "mention".
Last comment stored and displayes as 'mention':

Last comment stored and displayes as 'like', manually changed in db:

Is the semantic_linkbacks_commentdatafilter (line 141) the right place to hook into for that?
I'm also wondering where the commentdata['_type'] (line 149) is set, if any ?
So first and foremost thanks for the plugin and then thanks in advance for any hints :-)
Tom

A follow-up to the above… with the recent update (wp 4.7.2, semantic-linkback v3.3.0, webmention v3.1.0) my complete comment display layout crashed -- even old webmentions like the ones in the screenshot above are now displayed as generic "mentions", again. Need to have a deeper look, will report…

I got a notification of a comment/edit... is there something new, or did I miss something?

I initially posted an issue that I thought related to this older one here, but it turned out to be a separate problem, which I just posted as new issues. sorry for any confusion this may caused :-)

is this still an issue or can we close this?

Hm, I use a callback function for the display of comments, in which I added a re-check for the src-url stored in the comment_meta and that sets the type accordingly if the generic "mention" is set first… since now (part of) this callback has new problems (#78 #79) I think this here can be closed.


Schema validation Array issue

I cannot validate schemas that contain arrays, especially if the number of objects is different per request.
Example response:
{
"name": "value",
"id": "value",
"arrayField": []
},
"objects": [
{
"Field1": "value",
"Field2": false,
},
{
"Field1": "value",
"Field2": true,
}
]
}
Bare in mind, that another request may have more objects in the objects array. Also, I do need to check that the Field1 and Field2 are present.
In the feature file, I use the classic:

def expectedSchema = read('path to json')
match response == expectedSchema

Problem is, I have tried multiple solutions in the schema, but I always get the error:
$ | not equal | actual does not contain key - '#array' (MAP:MAP)
or something similar.
Schema version 1 which will fail if the number of objects is different:
{
"name": "#string",
"id": "#string",
"arrayField": "#array"
},
"objects": [
{
"Field1": "#string",
"Field2": "#boolean",
},
{
"Field1": "#string"",
"Field2": "#boolean",
}
]
}
Schema version 2 which will fail given the above error:
{
"name": "#string",
"id": "#string",
"arrayField": "#array"
},
"objects": "#array"
"#array": [
"Field1": "#string"",
"Field2": "#boolean",
]
}
Is there a way to validate the schema successfully?

Apologies


chore(deps): update golang.org/x/exp digest to f66d83c

This PR contains the following updates:

Package
Type
Update
Change

golang.org/x/exp
require
digest
9a3e603 -> f66d83c

Configuration
📅 Schedule: Branch creation - "every weekday" in timezone Europe/Paris, Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

⚠ Artifact update problem
Renovate failed to update an artifact related to this branch. You probably do not want to merge this PR as-is.
♻ Renovate will retry this branch, including artifacts, only when one of the following happens:

any of the package files in this branch needs updating, or
the branch becomes conflicted, or
you click the rebase/retry checkbox if found above, or
you rename this PR's title to start with "rebase!" to trigger it manually

The artifact failure details are included below:
File name: hello/dagger/go.sum
Command failed: go get -d -t ./...
go: -d flag is deprecated. -d=true is a no-op
go: dagger/hello imports
	dagger/hello/internal/dagger: package dagger/hello/internal/dagger is not in std (/opt/containerbase/tools/golang/1.23.2/src/dagger/hello/internal/dagger)


Add basic support for Ldftn opcode

In preparation tracking leaks occurring because of event handlers.

We should revisit the event handling stuff at some point for which this is related work. At that time we will loop it into the ldftn parser that I added.


Add bigint to primitive list

bigint is a primitive since is not assignable to object. Not sure if we want this here though since there is no description for newer types such as bigint or unknown in "Basic Types.md". bigint is introduced in<EMAIL_ADDRESS>page.
const foo: object = BigInt(100); // Type 'bigint' is not assignable to type 'object'.ts(2322)

https://mariusschulz.com/blog/the-object-type-in-typescript

Agree, thanks!


Rewards Signup

This is my first time on Github, pretty sure I'm doing this all wrong so sorry about that.
I just wanted to post a short video clip of the problem.
As I click on the notification it goes to the other page, but only for a split second as seen in video clip.
I signed up for Odysee a long time ago and have over 100LBC, mostly from watch rewards.
I don't know anything about code and I'm sure you guys are very busy so no need for a reply, I just wanted to share this.
Also, I have never been able to view a live feed.  I won't get into that, just thought I'd add it as I thought maybe it's related somehow.
Browser: Waterfox 78.12.0
OS: Windows 10
Video:  https://user-images.githubusercontent.com/35485882/121387405-e3478000-c907-11eb-803f-1b74b761cd41.mp4

Sorry to hear about the trouble - shoot us an email with your email so we can take a look into the account history. Should not be getting those notifications if you are reward approved. Thanks for this report!
You can also give us the details of the live streaming issues - also maybe try another browser like Chrome/Brave if you see a cookies warning.

Thanks tzarebczan, will do.


New distributed installation command is being missed by users

This seems to be coming up a lot now that we have a new doc structure: https://github.com/spinnaker/spinnaker/issues/2813

@lwander can hal config deploy edit --type distributed ... be step one, before linking to configuring provider? If they're following those links to configure K8s, that might be why they're missing step 2.

The problem is they need to know the name of the account, which they might not have picked until configuring their provider... Maybe we can make that clear somehow?


Aimbot Issues - Viridian#1613

Disable aimbot on things like mines, etc.? E.g. Engineer mines / Railgunner mine. For example, you wouldn't want to throw your mobility at enemies
Disable aimbot aiming towards enemies not in line of sight? E.g. when there's enemies behind rocks but not directly visible to players. I don't know how achievable this is, but the old aimbot in the old Aerolt seemed to do this just fine (might just be placebo)

Disable aimbot is done via holding interact, and that is all you get. Implementing it per skill would be annoying.

The aiming towards code is much better now, and i've added a slider to control bias towards distance or angle.


Join Domain Failed - Windows Virtual Desktop

Hi, I am doing a POC on windows virtual desktop. I have issues in creating a host pool. The error is saying failed to join the AD with the domain XYZ.
To be more specific I followed steps 1 2 3 in the documentation. In the third step, the host pool is failing to join the AD. I am giving UPN name as one of the users in the domain which has tenant creator and global admin role.

@AgassiJoeLVedala89 Could you please let us know the documentation link that you are using before facing the above problem? This would help us redirect you to the right team.

@AgassiJoeLVedala89  Could you please let us know the documentation link that you are using before facing the above problem? This would help us redirect you to the right team.

@AgassiJoeLVedala89 We will now proceed to close the issue. You can create a new issue against the document link used if you are still facing an issue.


Fix spelling in datamodel-MYSQL

Prisam to Prisma

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


"az ad sp create-for-rbac" does not support scope management group

Describe the bug
az ad sp create-for-rbac does not support scoping to management group and fails with error Subscription 'Microsoft.Management' not found.
To Reproduce
az ad sp create-for-rbac --name "<MANAGEMENT_GROUP_NAME>-Contributor" --role "Contributor" --scopes /providers/Microsoft.Management/managementGroups/<MANAGEMENT_GROUP_NAME> --sdk-auth --debug
After this Azure API Request...
PUT /providers/Microsoft.Management/managementGroups/<MANAGEMENT_GROUP_NAME>/providers/Microsoft.Authorization/roleAssignments/<ROLE_ASSIGNMENT_ID>?api-version=2020-04-01-preview HTTP/1.1
... you will see the following error message:
azure.cli.core.util.handle_exception is called with an exception:
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 215, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 654, in execute
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 718, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 709, in _run_job
    cmd_copy.exception_handler(ex)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/role/commands.py", line 69, in graph_err_handler
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 688, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 325, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 784, in default_command_handler
    return op(**command_args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/role/custom.py", line 1507, in create_service_principal_for_rbac
    app_id, password, cert_file)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 714, in get_sp_auth_info
    account = self.get_subscription(subscription_id)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 511, in get_subscription
    "Check the spelling and casing and try again.".format(subscription))
knack.util.CLIError: Subscription 'Microsoft.Management' not found. Check the spelling and casing and try again.

cli.azure.cli.core.azclierror : Subscription 'Microsoft.Management' not found. Check the spelling and casing and try again.
Subscription 'Microsoft.Management' not found. Check the spelling and casing and try again.

Site note: <MANAGEMENT_GROUP_NAME> and <ROLE_ASSIGNMENT_ID> masked for security concerns.
Expected behavior
Successful creation and role assignment of service principal to management group incl. output of the JSON with client ID and client secret - just like for subscriptions.
Environment summary

Azure Cloud Shell with Azure CLI 2.14.0

az ad

This issue is still open and I have come across it today with a customer.
Any plans to resolve this issue or is it still in the backlog ?