Remove local functionality

Companion PR to #5578. Removes local-only functionality, and replaces it with a dependency on HPXLocal. Work in progress...

I'd appreciate reviews on this now. I'll try to summarize the changes, but please ask if you think I haven't explained something in enough detail (or if it seems like I've forgotten something). If you see problems with the choices here, please let me know. Nothing is set in stone.

This removes the libs/core directory and the hpx_core target from this repository.
It adds an option HPX_WITH_FETCH_HPXLOCAL to automatically fetch HPXLocal. This is on by default to be as similar to the current setup with a single repo as possible.
Most CMake options have been separated to be either for this repo (still called HPX_WITH_...) or for HPXLocal (called HPXLocal_WITH_...).
When fetching HPXLocal HPX_WITH_... variables that were moved to HPXLocal will be forwarded to HPXLocal_WITH_... variables with a warning.
When HPXLocal is installed separately no forwarding or warnings will happen as HPXLocal is already built.
All C++ feature tests happen only in HPXLocal, and HPX reuses their results (this may need to be revisited if we think the two will have a need for significantly different feature tests).
The C++ standard is now set in HPXLocal with HPXLocal_CXX_STANDARD. This replaces the use of CMAKE_CXX_STANDARD and HPX_WITH_CXXAB. When I made the change to prefer CMAKE_CXX_STANDARD I did not take into account the possibility that we'd need CMAKE_CUDA_STANDARD (and possibly CMAKE_HIP_STANDARD in the future), so it makes more sense to have an HPX-specific variable for it.
The HPX_WITH_CXXAB variables forward to HPXLocal_CXX_STANDARD with a warning if HPXLocal is fetched.
Documentation lives entirely in this repository. The documentation will include API documentation for HPXLocal if HPXLocal is fetched.
I've removed the CUDA options HPX_WITH_COMPUTE_CUDA and HPX_WITH_ASYNC_CUDA since one of the modules is now in HPX and the other in HPXLocal. There's only HPXLocal_WITH_CUDA (and HPXLocal_WITH_HIP) now. If HPXLocal_WITH_CUDA=ON, HPX will also use CUDA.
Likewise, there's no HPX_WITH_ASYNC_MPI option anymore. There is HPXLocal_WITH_MPI, and if HPXLocal_WITH_MPI=ON HPX will automatically set HPX_WITH_PARCELPORT_MPI=ON.
There is no HPX_WITH_DISTRIBUTED_RUNTIME option anymore. HPXLocal only provides the local runtime entry point. Applications compiled with HPX can still choose to use the local-only runtime at compile- or run-time (with --hpx:local) as before.
CMake options affecting the build system (UNITY_BUILD, PRECOMPILED_HEADERS) are separate for HPX and HPXLocal.
Tests and examples for HPXLocal are off by default, and will not be built or run when fetching HPXLocal. They can be enabled and built through the same targets, however (i.e. tests, tests.unit, examples, etc.).
Until the next release HPXLocal is fixed to a specific tag/commit (currently the branch corresponding to https://github.com/STEllAR-GROUP/hpx-local/pull/1) and should be updated explicitly. HPXLocal is set to version 0.0.1 at the moment and the exact version is required by the CMake configuration.
After the next release I propose we still require an exact version, but only released versions of HPXLocal. Once things stabilize, we may require only a minimum version.

@hkaiser I've added an advanced CMake variable HPXLocal_WITH_BINARY_DIR (to HPXLocal) which is used as the root binary output directory. This was previously using PROJECT_BINARY_DIR directly as the root. I set it to default to HPX_BINARY_DIR when on Windows and HPX has set HPX_WITH_FETCH_HPXLOCAL=ON, and otherwise it defaults to the usual PROJECT_BINARY_DIR. I'm hoping this will make sure libraries are found correctly on Windows, but I'd appreciate if you could try it out as well. CI will likely tell me before that as well...
I've also removed the duplicate tests and examples that you removed in https://github.com/msimberg/hpx/pull/8. You should be able to drop that commit from https://github.com/msimberg/hpx/pull/8.

Performance test reportHPX PerformanceComparisonBENCHMARKNO-EXECUTORFuture Overhead - Create Thread Hierarchical - Latch=InfoPropertyBeforeAfterHPX Commit96a2e4bd1c76207dea8d5302784962c9278824065075ea7c386288492d8d8889d2f2cc78859f465aHPX Datetime2021-11-11T08:14:57+00:002021-12-08T10:20:23+00:00Compiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0ClusternamedaintdaintHostnamenid00006nid00478Datetime2021-11-11T09:28:13.071121+01:002021-12-08T11:46:40.791000+01:00EnvfileComparisonBENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATORStream Benchmark - Add---Stream Benchmark - Scale-(=)(=)Stream Benchmark - Triad---Stream Benchmark - Copy(=)(=)(=)InfoPropertyBeforeAfterHPX Commit71d8dbe40b737be02ea734f9d98fcc0b1e21f01a5075ea7c386288492d8d8889d2f2cc78859f465aHPX Datetime2021-11-10T19:14:21+00:002021-12-08T10:20:23+00:00Compiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0ClusternamedaintdaintHostnamenid00120nid00478Datetime2021-11-10T20:28:18.266961+01:002021-12-08T11:46:56.392336+01:00EnvfileExplanation of SymbolsSymbolMEANING=No performance change (confidence interval within ±1%)(=)Probably no performance change (confidence interval within ±2%)(+)/(-)Very small performance improvement/degradation (≤1%)+/-Small performance improvement/degradation (≤5%)++/--Large performance improvement/degradation (≤10%)+++/---Very large performance improvement/degradation (>10%)?Probably no change, but quite large uncertainty (confidence interval with ±5%)??Unclear result, very large uncertainty (±10%)???Something unexpected…

Performance test reportHPX PerformanceComparisonBENCHMARKNO-EXECUTORFuture Overhead - Create Thread Hierarchical - Latch-InfoPropertyBeforeAfterHPX Commit96a2e4bd1c76207dea8d5302784962c9278824062051403d54d98a8e47acda78800ad93384465df8HPX Datetime2021-11-11T08:14:57+00:002021-12-10T13:50:12+00:00Hostnamenid00006nid00111Datetime2021-11-11T09:28:13.071121+01:002021-12-10T15:38:35.629449+01:00ClusternamedaintdaintCompiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0EnvfileComparisonBENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATORStream Benchmark - Add(=)(=)=Stream Benchmark - Scale(=)=(=)Stream Benchmark - Triad(=)(=)(=)Stream Benchmark - Copy(=)=(=)InfoPropertyBeforeAfterHPX Commit71d8dbe40b737be02ea734f9d98fcc0b1e21f01a2051403d54d98a8e47acda78800ad93384465df8HPX Datetime2021-11-10T19:14:21+00:002021-12-10T13:50:12+00:00Hostnamenid00120nid00111Datetime2021-11-10T20:28:18.266961+01:002021-12-10T15:38:51.356384+01:00ClusternamedaintdaintCompiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0EnvfileExplanation of SymbolsSymbolMEANING=No performance change (confidence interval within ±1%)(=)Probably no performance change (confidence interval within ±2%)(+)/(-)Very small performance improvement/degradation (≤1%)+/-Small performance improvement/degradation (≤5%)++/--Large performance improvement/degradation (≤10%)+++/---Very large performance improvement/degradation (>10%)?Probably no change, but quite large uncertainty (confidence interval with ±5%)??Unclear result, very large uncertainty (±10%)???Something unexpected…

Performance test reportHPX PerformanceComparisonBENCHMARKNO-EXECUTORFuture Overhead - Create Thread Hierarchical - Latch-InfoPropertyBeforeAfterHPX Commit96a2e4bd1c76207dea8d5302784962c927882406adb8b3ab4fd133e849ab85b2781ee636e59d00bbHPX Datetime2021-11-11T08:14:57+00:002021-12-17T10:22:31+00:00ClusternamedaintdaintEnvfileCompiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0Datetime2021-11-11T09:28:13.071121+01:002021-12-17T11:39:59.544596+01:00Hostnamenid00006nid00162ComparisonBENCHMARKFORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATORPARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATORSCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATORStream Benchmark - Add(=)(=)(=)Stream Benchmark - Scale(=)(=)(=)Stream Benchmark - Triad(=)(=)(=)Stream Benchmark - Copy(=)=(=)InfoPropertyBeforeAfterHPX Commit71d8dbe40b737be02ea734f9d98fcc0b1e21f01aadb8b3ab4fd133e849ab85b2781ee636e59d00bbHPX Datetime2021-11-10T19:14:21+00:002021-12-17T10:22:31+00:00ClusternamedaintdaintEnvfileCompiler/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0Datetime2021-11-10T20:28:18.266961+01:002021-12-17T11:40:15.013282+01:00Hostnamenid00120nid00162Explanation of SymbolsSymbolMEANING=No performance change (confidence interval within ±1%)(=)Probably no performance change (confidence interval within ±2%)(+)/(-)Very small performance improvement/degradation (≤1%)+/-Small performance improvement/degradation (≤5%)++/--Large performance improvement/degradation (≤10%)+++/---Very large performance improvement/degradation (>10%)?Probably no change, but quite large uncertainty (confidence interval with ±5%)??Unclear result, very large uncertainty (±10%)???Something unexpected…

@hkaiser I've added an advanced CMake variable HPXLocal_WITH_BINARY_DIR (to HPXLocal) which is used as the root binary output directory. This was previously using PROJECT_BINARY_DIR directly as the root. I set it to default to HPX_BINARY_DIR when on Windows and HPX has set HPX_WITH_FETCH_HPXLOCAL=ON, and otherwise it defaults to the usual PROJECT_BINARY_DIR. I'm hoping this will make sure libraries are found correctly on Windows, but I'd appreciate if you could try it out as well. CI will likely tell me before that as well...

I missed this comment before. Thanks for applying this change. I think we should set the HPXLocal_WITH_BINARY_DIR variable in the distributed repository just before calling fetch_content,

I've also removed the duplicate tests and examples that you removed in msimberg#8. You should be able to drop that commit from msimberg#8.

Will do.

@hkaiser I've added an advanced CMake variable HPXLocal_WITH_BINARY_DIR (to HPXLocal) which is used as the root binary output directory. This was previously using PROJECT_BINARY_DIR directly as the root. I set it to default to HPX_BINARY_DIR when on Windows and HPX has set HPX_WITH_FETCH_HPXLOCAL=ON, and otherwise it defaults to the usual PROJECT_BINARY_DIR. I'm hoping this will make sure libraries are found correctly on Windows, but I'd appreciate if you could try it out as well. CI will likely tell me before that as well...

I missed this comment before. Thanks for applying this change. I think we should set the HPXLocal_WITH_BINARY_DIR variable in the distributed repository just before calling fetch_content,

It's now here: https://github.com/msimberg/hpx/blob/1698bc8d61c7210bdfeb897f64a20f80c590f5c9/cmake/HPX_SetupHPXLocal.cmake#L171-L177.

I've also removed the duplicate tests and examples that you removed in msimberg#8. You should be able to drop that commit from msimberg#8.

Will do.

Note, that was already done and merged ;)

bors merge


feat: add logs for builder validator registrations

Motivation
Closes https://github.com/ChainSafe/lodestar/issues/5337
Description
Adds logs for builder validator registrations. Especially on the validator client this should be logged to info as it helps to detect issues like https://github.com/rocket-pool/smartnode-install/pull/105 earlier. On the beacon node side, I opted to keep it as debug for now and only log errors.
Note: For users that want more visibility on the beacon node, it is possible to set --logLevelModule="api=debug" flag. This will print out the debug log Forwarded validator registrations ... to stdout instead of just the log file.
What are other clients doing?

Nimbus

vc : no log
bn: no log

Lighthouse

vc: INFO Published validator registrations to the builder network, count: 10, service: preparation
bn: INFO Forwarding register validator request to connected builder, count: 9

Teku

vc: INFO  - Validator   *** 10 out of 10 validator registration(s) were successfully sent to the builder network via the Beacon Node.
bn: no log

Prysm

vc: level=info msg="Submitted builder validator registration settings for custom builders" prefix=validator
bn: no log

:tada: This PR is included in v1.12.0 :tada:


Switch to go 1.23

What type of PR is this?
/kind dependency-change
What this PR does / why we need it:
Update to the latest golang and golangci-lint.
Which issue(s) this PR fixes:
None

Special notes for your reviewer:
None
Does this PR introduce a user-facing change?

None

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 47.56%. Comparing base (4e0f474) to head (870ab5d).
Report is 706 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #2403       +/-   ##
===========================================
+ Coverage   37.53%   47.56%   +10.02%
===========================================
  Files          15      121      +106
  Lines        1268     4007     +2739
  Branches      414      830      +416
===========================================
+ Hits          476     1906     +1430
- Misses        526     1590     +1064
- Partials      266      511      +245

@kwilczynski PTAL

/approve
/lgtm


Add 3iL from Rodez

@jade-panchaud Please provide us with:

the school official website URL, if different from the domain you are submitting (is it https://www.3il-ingenieurs.fr/3il-rodez/) ?
an URL of a page on the official website where an IT-related long-term (>1 year) course is offered by the school
an URL of a page or another proof (.pdf or a screenshot are OK) showing that the school recognizes the domain which you are submitting as an official email domain for the students.

Hi, sorry,
The URL website you have is the good one,  i attached a picture of the document we received for email (this isn't mine, because i lost it..), we don't have anything else, is it ok ?
Thank you and have a good day !

@jade-panchaud Pull request merged. Thank you!


Windows installation creates non-hidden .atom folder under user's home dir

I just installed Atom on Windows 7 today using the download link from https://atom.io/. After installing and running it, a non-hidden folder named .atom appeared in my user profile directory (C:\Users\janke). Windows doesn't have the convention that an initial . in a file name causes the folder to be hidden, so the folder needs to have its Hidden attribute set to keep it from showing up in Windows Explorer by default.
It might make more sense to store this folder under the user's AppData folder(s) instead, if you want this to seem like a native Windows application. That's the standard Windows location for application-managed data and settings, and it would let users take advantage of roaming profiles to share their settings between machines.

Closing in favor of #8281 - please subscribe there for updates.

Thanks. Chimed in over there.


Refactoring Modals (WIP)

Fixes  #146 .
Long way to go

Hey! @enzo-beltrami how is this going? Do you need some help or is there an update on this?


Code completion of Keras Module no longer work since 2.6

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 20H2
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): v2.4.0-49-g85c8b2a817f 2.4.1/
Python version: 3.8.9
Bazel version (if compiling from source): N/A
GCC/Compiler version (if compiling from source): N/A
CUDA/cuDNN version:  N/A
GPU model and memory: N/A

Describe the current behavior
Code completion is no longer work for tf.keras on the VSCode. since TensorFlow 2.6.

Describe the expected behavior
Code completion is work for tf.keras on the VSCode until TensorFlow 2.5.

A related issue found #52031.
The cause of this problem is lazy import of Keras.
A similar problem exists in TensorFlow Probability.
One solution is to prepare a stub file, but that is a lot of work.
Are you aware that this problem is a VSCode/Pylance problem?
Contributing

Do you want to contribute a PR? (yes/no): yes, but I don't have a solution.
Briefly describe your candidate solution(if contributing): N/A.

It also doesn't work in PyCharm, I opened issue here #53144
At first I was using Visual Studio to write tensorflow python code, but then I switched to PyCharm because code completion was broken and I thought it was Visual Studio issue. Now I see that writing tensorflow keras code is absolutely broken in every major IDE.
I can't believe so few people are complaining about this, what IDE are other people using to write tensorflow code, notepad++? :D

I can't believe so few people are complaining about this, what IDE are other people using to write tensorflow code, notepad++? :D
Vim :P

But coming back to the issue, I think this is due to the keras moving to a separate repository. Probably the solution would also come from there.

But coming back to the issue, I think this is due to the keras moving to a separate repository. Probably the solution would also come from there.

I think so.
And also currently Keras is not work code completion too.
This is very similarly with this issue.

@elda27 try my solution.
https://github.com/tensorflow/tensorflow/issues/53144#issuecomment-985179600

@cpuimage
Thanks.
A module completion such as from tensorflow.keras import ~ is works good.
But tf.keras.~ is not work code completions.
Can you work completion about tf.keras.~ on the PyCharm.
My VSCode is not work code completion after changes.
Maybe PyCharm is more intelligence.

@elda27  yes,it work.
after modify the  [site-packages\tensorflow_init_.py]
then write code like:
import tensorflow as tf
from tensorflow.keras import layers

tf.keras.~ # it work good.

@cpuimage Oh nice! maybe my work is something wrong.
I will read again your posts.
By the way, I found good feature in PEP563.
https://www.python.org/dev/peps/pep-0563/#runtime-annotation-resolution-and-type-checking
There are example code in the document.
import typing

if typing.TYPE_CHECKING:
    import expensive_mod

def a_func(arg: expensive_mod.SomeClass) -> None:
    a_var: expensive_mod.SomeClass = arg
    ...

This code is work good in the VSCode. (I'm not check on the PyCharm)
import typing
if typing.TYPE_CHECKING:
  from tensorflow import keras
else:
  _keras_module = "keras.api._v2.keras"
  keras = _LazyLoader("keras", globals(), _keras_module)
  _module_dir = _module_util.get_parent_dir_for_name(_keras_module)
  if _module_dir:
    _current_module.__path__ = [_module_dir] + _current_module.__path__
  setattr(_current_module, "keras", keras)

@cpuimage Oh nice! maybe my work is something wrong. I will read again your posts.
By the way, I found good feature in PEP563. https://www.python.org/dev/peps/pep-0563/#runtime-annotation-resolution-and-type-checking
There are example code in the document.
import typing

if typing.TYPE_CHECKING:
    import expensive_mod

def a_func(arg: expensive_mod.SomeClass) -> None:
    a_var: expensive_mod.SomeClass = arg
    ...

This code is work good in the VSCode. (I'm not check on the PyCharm)
import typing
if typing.TYPE_CHECKING:
  from tensorflow import keras
else:
  _keras_module = "keras.api._v2.keras"
  keras = _LazyLoader("keras", globals(), _keras_module)
  _module_dir = _module_util.get_parent_dir_for_name(_keras_module)
  if _module_dir:
    _current_module.__path__ = [_module_dir] + _current_module.__path__
  setattr(_current_module, "keras", keras)

good job. it work good in PyCharm.

@cpuimage Thanks! I haven't been successful your suggestion yet. Maybe VSCode has something difference with PyCharm.
@sanatmpa1 Can you review my solution and please tell us to any one is better for TensorFlow?

@elda27,
As @mihaimaruseac pointed out in this comment, it may be due to keras being moved to a different repo.
Can you please post this issue on keras-team/keras repo. Thanks!

@sanatmpa1 This change apply the tensorflow/__init__.py.
I guess the issue for tensorflow-team.

This is still happening with 2.8.0, 2.8.1, and 2.9.0.

This issue is solved in #54104 and released at least in tf-nightly and is also more or less a dublicate of #56231 and #53144 and could be closed as it is now hopefully fully resolved with #56231? @qlzh727

Let's just use #56231 to track the overall IDE auto-complete issue.

@elda27  No, the autocompletion mentioned in this issue is fixed in nightly, so probably in v2.10. You can currently fix it in your installation by doing what is done in #54104 locally.

@alkatar21 Thanks! I await next release.

@alkatar21 The issue still happened on v2.10.0

However tf.keras is OK.

So, why is that?

I try to modify tensorflow.init.py
before:
if _typing.TYPE_CHECKING:
from tensorflow_estimator.python.estimator.api._v2 import estimator as estimator
from keras.api._v2.keras import keras
from keras.api._v2.keras import losses
from keras.api._v2.keras import metrics
from keras.api._v2.keras import optimizers
from keras.api._v2.keras import initializers
now:
if _typing.TYPE_CHECKING:
from tensorflow_estimator.python.estimator.api._v2 import estimator as estimator
import keras.api._v2.keras as keras
from keras.api._v2.keras import losses
from keras.api._v2.keras import metrics
from keras.api._v2.keras import optimizers
from keras.api._v2.keras import initializers
And it's work for me.

So, why is that?
TF: 2.10.0
Python: 3.8.13 by miniconda
os: ubuntu20.04.5 LTS  (GNU/Linux 5.4.0-125-generic x86_64)
vscode:v1.71.0

TF's APIs should be used via tf.. So tf.keras.* is legitimate, keras.* not so much.

TF's APIs should be used via tf.. So tf.keras.* is legitimate, keras.* not so much.

@mihaimaruseac ,Thanks for you reply, So why it can run successful but dont support autocompletion.
And why tf tutorials show  a  code  from tensorflow.keras import Model ,from tensorflow.keras.layers import Dense, Flatten, Conv2Detc. as you know it's also has the same issue, but the tutorials insist do that.
https://tensorflow.google.cn/tutorials/quickstart/advanced

TF's APIs should be used via tf.. So tf.keras.* is legitimate, keras.* not so much.

And you can see the code from tensorflow.keras import * is on every official examples almost. Whatever tensorflow or keras.  like this.https://keras.io/examples/vision/image_classification_from_scratch/
So, via tf.* to use tensorflow or keras sounds not make sence.

Thanks for the information.
Based on my rough knowledge for auto completion, I think the issue probably caused by difference between package/directory/module.
Most of the IDE will try to auto complete by the package tree structure, so if there is a directory with init under tensorflow call "keras", then it will resolve correctly when you do "from tensorflow.keras.xxxx import yyy". This was working before 2.7 since Keras was part of the Tensorflow PIP package.
Note that the "import tensorflow as tf; tf.keras.xxx.yyy" is resolved differently. tf.keras is firstly resolved into the logic in the tensorflow/init.py, and then the keras becomes the "keras.apis._v2.keras", which resolved correctly to the keras PIP package.
Before TF 2.7, I think the we did some manual mimic for the keras APIs to tensorflow/keras folder from tensorflow/python/keras/apis, so that it the package level auto completion works, but that is not the case anymore after 2.7 since the keras is a standalone package.
As a quick test on my local, I manually created a symlink under venv/lib/python3.10/site-packages/tensorflow by ln -s ../keras/api/_v2/keras/ keras (assuming you have tensorflow and keras pip package installed properly), and then the IDE can resolve the package level auto completion correctly.
I am not sure if this fix can be applied to TF direclty, since the symlink is not well supported on windows, and might be in different location if the package is installed differently.

Having said that, all the import should always work, since the auto complete and python import works differently. Tensorflow has extra logic in the tensorflow/init.py to ensure all the public API are populated correctly via the _api folder, instead of directly using the directory structure.

Thanks for the information.
Based on my rough knowledge for auto completion, I think the issue probably caused by difference between package/directory/module.
Most of the IDE will try to auto complete by the package tree structure, so if there is a directory with init under tensorflow call "keras", then it will resolve correctly when you do "from tensorflow.keras.xxxx import yyy". This was working before 2.7 since Keras was part of the Tensorflow PIP package.
Note that the "import tensorflow as tf; tf.keras.xxx.yyy" is resolved differently. tf.keras is firstly resolved into the logic in the tensorflow/init.py, and then the keras becomes the "keras.apis._v2.keras", which resolved correctly to the keras PIP package.
Before TF 2.7, I think the we did some manual mimic for the keras APIs to tensorflow/keras folder from tensorflow/python/keras/apis, so that it the package level auto completion works, but that is not the case anymore after 2.7 since the keras is a standalone package.
As a quick test on my local, I manually created a symlink under venv/lib/python3.10/site-packages/tensorflow by ln -s ../keras/api/_v2/keras/ keras (assuming you have tensorflow and keras pip package installed properly), and then the IDE can resolve the package level auto completion correctly.
I am not sure if this fix can be applied to TF direclty, since the symlink is not well supported on windows, and might be in different location if the package is installed differently.

Thanks for you reply, It's really very useful for me, and this fix is work on my local. But I dont think this is perfect way to solve this issue(maybe now), cos it seems I need do that every time when I was upgrade or reinstall tensorflow. As I konw, tf.keras is the recommended api by TF2, but it is inconvenient to use, I think we shouldn't do any additional operation once we do pip install tensorflow. So, is there will have a better way to solve this issue in the future version?


GraphEngine: Save Cell

Save a cell with size larger than 1G in GraphEngine, it will crash, but the return value is "E_SUCCESS". it is not reasonable, it should be “cell too big” or some others like this.

Fixed in #53


Add Github Greetings Bot

Hey 🥳 Looks like an amazing project idea! I see this repo lacks a small enhancement Github Greetings bot that sends auto messages to greet new users when they raise a PR or an issue. Let me know if you would like this level1 set up! Please assign to me if interested! (GSSoC24-Ext Contribution)

@KiranBaliga you can start to work on this! <3

@KiranBaliga any updates?

@soham0005 Please review the PR and action accordingly. Thank you. Please add level1 and gssoc24-ext to the PR. Thanks again.


support latin1 encoding for channels in read_raw_nihon

When opening Nihon-Kohden files with read_raw_nihon I get the error
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc4 in position 1070: invalid continuation byte

raised in this line.
I guess it is similar to this issue which was already solved here, as replacing the above mentioned line with
with open(e_fname, 'r', encoding='latin1') as fid:

fixed the issue.
Maybe we could add a similar loop over different encodings in _read_21e_file?

No objection, can you make a PR?

I'll have to see how this works, but I'll try


Breaking changes with Numpy dependency update

It appears with NumPy's new major version release the target-bigquery loader has broken. I'm seeing ImportError: numpy.core.multiarray failed to import messages in all of my affiliated meltano taps and targets.
Additional information from logs:
A module that was compiled using NumPy 1.x cannot be run in NumPy 2.0.0 as it may crash. To support both 1.x and 2.x versions of NumPy, modules must be compiled with NumPy 2.0. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'. If you are a user of the module, the easiest solution will be to downgrade to 'numpy<2' or try to upgrade the affected module. We expect that some modules will need time to support NumPy 2.
Are you able to add a version control to either point to Numpy<2 or update things to work with the new version?

Adding a specific version of numpy to the requirements.txt resolves the issue
numpy~=1.24.1


[BUG] After rebooting all Harvester nodes concurrently some VMs can't be restarted

Describe the bug
After rebooting all Harvester nodes concurrently , a few existing VMs failed to start. Inspecting the state of the corresponding image backing volumes in LH, the volumes are infinitely stuck in detaching state with all three replicas in "failed" state.

Harvester 0.3-preview1
Logs
longhorn-csi-plugin-xxx
time="2021-09-23T09:00:17Z" level=debug msg="Polling volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 state for volume unpublished at 2021-09-23 09:00:17.70718131 +0000 UTC m=+2595.885092954"
time="2021-09-23T09:00:17Z" level=debug msg="Polling volume pvc-fd22289a-3617-4031-943c-5a7342d2c26b state for volume unpublished at 2021-09-23 09:00:17.708363988 +0000 UTC m=+2595.886275652"
time="2021-09-23T09:00:19Z" level=debug msg="Polling volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 state for volume unpublished at 2021-09-23 09:00:19.70720978 +0000 UTC m=+2597.885121464"
time="2021-09-23T09:00:19Z" level=debug msg="Polling volume pvc-fd22289a-3617-4031-943c-5a7342d2c26b state for volume unpublished at 2021-09-23 09:00:19.708352904 +0000 UTC m=+2597.886264588"
time="2021-09-23T09:00:21Z" level=warning msg="waitForVolumeState: timeout while waiting for volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 state volume unpublished"
time="2021-09-23T09:00:21Z" level=error msg="ControllerUnpublishVolume: err: rpc error: code = DeadlineExceeded desc = Failed to detach volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 from node pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370"
time="2021-09-23T09:00:21Z" level=warning msg="waitForVolumeState: timeout while waiting for volume pvc-fd22289a-3617-4031-943c-5a7342d2c26b state volume unpublished"
time="2021-09-23T09:00:21Z" level=error msg="ControllerUnpublishVolume: err: rpc error: code = DeadlineExceeded desc = Failed to detach volume pvc-fd22289a-3617-4031-943c-5a7342d2c26b from node pvc-fd22289a-3617-4031-943c-5a7342d2c26b"

longhorn-manager-xxy
time="2021-09-23T09:06:18Z" level=warning msg="Dropping Longhorn backing image longhorn-system/image-cql6j out of the queue: fail to sync backing image for longhorn-system/image-cql6j: failed to sync backing image file state: BUG: backing image recorded checksum 1050a8ccbaa72d552b72660ed9c43460fb75dc198b1802d0332e5ee6eb5cfb37dce9ac75a7af9b8b06021aad6e383051027d071e2809aff4111f8779185972d7 doesn't match the file checksum 40d883b86625e48ce0bab91f7f70935cca6d2d7f73be0e0b804d1d384dab98f41a676a1d787e966d11ac00876db6b240fa996310794bfa0d4d378b050b7bc4a0 in disk 3af17a6d-c8ea-4b21-8da0-adeeb09e568d"
time="2021-09-23T09:06:19Z" level=warning msg="Error syncing Longhorn backing image longhorn-system/image-cql6j: fail to sync backing image for longhorn-system/image-cql6j: failed to sync backing image file state: BUG: backing image recorded checksum 1050a8ccbaa72d552b72660ed9c43460fb75dc198b1802d0332e5ee6eb5cfb37dce9ac75a7af9b8b06021aad6e383051027d071e2809aff4111f8779185972d7 doesn't match the file checksum 40d883b86625e48ce0bab91f7f70935cca6d2d7f73be0e0b804d1d384dab98f41a676a1d787e966d11ac00876db6b240fa996310794bfa0d4d378b050b7bc4a0 in disk 3af17a6d-c8ea-4b21-8da0-adeeb09e568d"
time="2021-09-23T09:06:19Z" level=warning msg="Error syncing Longhorn backing image longhorn-system/image-cql6j: fail to sync backing image for longhorn-system/image-cql6j: failed to sync backing image file state: BUG: backing image recorded checksum 1050a8ccbaa72d552b72660ed9c43460fb75dc198b1802d0332e5ee6eb5cfb37dce9ac75a7af9b8b06021aad6e383051027d071e2809aff4111f8779185972d7 doesn't match the file checksum 40d883b86625e48ce0bab91f7f70935cca6d2d7f73be0e0b804d1d384dab98f41a676a1d787e966d11ac00876db6b240fa996310794bfa0d4d378b050b7bc4a0 in disk 3af17a6d-c8ea-4b21-8da0-adeeb09e568d"
E0923 09:06:19.104645       1 backing_image_controller.go:161] fail to sync backing image for longhorn-system/image-cql6j: failed to sync backing image file state: BUG: backing image recorded checksum 1050a8ccbaa72d552b72660ed9c43460fb75dc198b1802d0332e5ee6eb5cfb37dce9ac75a7af9b8b06021aad6e383051027d071e2809aff4111f8779185972d7 doesn't match the file checksum 40d883b86625e48ce0bab91f7f70935cca6d2d7f73be0e0b804d1d384dab98f41a676a1d787e966d11ac00876db6b240fa996310794bfa0d4d378b050b7bc4a0 in disk 3af17a6d-c8ea-4b21-8da0-adeeb09e568d
time="2021-09-23T09:06:19Z" level=warning msg="Dropping Longhorn backing image longhorn-system/image-cql6j out of the queue: fail to sync backing image for longhorn-system/image-cql6j: failed to sync backing image file state: BUG: backing image recorded checksum 1050a8ccbaa72d552b72660ed9c43460fb75dc198b1802d0332e5ee6eb5cfb37dce9ac75a7af9b8b06021aad6e383051027d071e2809aff4111f8779185972d7 doesn't match the file checksum 40d883b86625e48ce0bab91f7f70935cca6d2d7f73be0e0b804d1d384dab98f41a676a1d787e966d11ac00876db6b240fa996310794bfa0d4d378b050b7bc4a0 in disk 3af17a6d-c8ea-4b21-8da0-adeeb09e568d"
E0923 09:06:19.674295       1 backing_image_controller.go:161] fail to sync backing image for longhorn-system/image-cql6j: failed to sync backing image file state: BUG: backing image recorded checksum 1050a8ccbaa72d552b72660ed9c43460fb75dc198b1802d0332e5ee6eb5cfb37dce9ac75a7af9b8b06021aad6e383051027d071e2809aff4111f8779185972d7 doesn't match the file checksum 40d883b86625e48ce0bab91f7f70935cca6d2d7f73be0e0b804d1d384dab98f41a676a1d787e966d11ac00876db6b240fa996310794bfa0d4d378b050b7bc4a0 in disk 3af17a6d-c8ea-4b21-8da0-adeeb09e568d
time="2021-09-23T09:06:19Z" level=warning msg="Dropping Longhorn backing image longhorn-system/image-cql6j out of the queue: fail to sync backing image for longhorn-system/image-cql6j: failed to sync backing image file state: BUG: backing image recorded checksum 1050a8ccbaa72d552b72660ed9c43460fb75dc198b1802d0332e5ee6eb5cfb37dce9ac75a7af9b8b06021aad6e383051027d071e2809aff4111f8779185972d7 doesn't match the file checksum 40d883b86625e48ce0bab91f7f70935cca6d2d7f73be0e0b804d1d384dab98f41a676a1d787e966d11ac00876db6b240fa996310794bfa0d4d378b050b7bc4a0 in disk 3af17a6d-c8ea-4b21-8da0-adeeb09e568d"

longhorn-manager-xyz
E0923 09:10:40.970362       1 volume_controller.go:214] fail to sync longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370: BUG: multiple engines detected when volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 is detached
time="2021-09-23T09:10:40Z" level=warning msg="Dropping Longhorn volume longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 out of the queue" controller=longhorn-volume error="fail to sync longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370: BUG: multiple engines detected when volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 is detached" node=harvester-6k94p
E0923 09:11:10.940513       1 volume_controller.go:214] fail to sync longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370: BUG: multiple engines detected when volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 is detached
time="2021-09-23T09:11:10Z" level=warning msg="Dropping Longhorn volume longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 out of the queue" controller=longhorn-volume error="fail to sync longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370: BUG: multiple engines detected when volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 is detached" node=harvester-6k94p
time="2021-09-23T09:11:10Z" level=warning msg="Error syncing Longhorn volume longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370" controller=longhorn-volume error="fail to sync longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370: BUG: multiple engines detected when volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 is detached" node=harvester-6k94p
E0923 09:11:10.970371       1 volume_controller.go:214] fail to sync longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370: BUG: multiple engines detected when volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 is detached
time="2021-09-23T09:11:10Z" level=warning msg="Dropping Longhorn volume longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 out of the queue" controller=longhorn-volume error="fail to sync longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370: BUG: multiple engines detected when volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 is detached" node=harvester-6k94p
E0923 09:11:40.941027       1 volume_controller.go:214] fail to sync longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370: BUG: multiple engines detected when volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 is detached
time="2021-09-23T09:11:40Z" level=warning msg="Dropping Longhorn volume longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 out of the queue" controller=longhorn-volume error="fail to sync longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370: BUG: multiple engines detected when volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 is detached" node=harvester-6k94p
time="2021-09-23T09:11:40Z" level=warning msg="Error syncing Longhorn volume longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370" controller=longhorn-volume error="fail to sync longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370: BUG: multiple engines detected when volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 is detached" node=harvester-6k94p
E0923 09:11:40.970341       1 volume_controller.go:214] fail to sync longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370: BUG: multiple engines detected when volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 is detached
time="2021-09-23T09:11:40Z" level=warning msg="Dropping Longhorn volume longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 out of the queue" controller=longhorn-volume error="fail to sync longhorn-system/pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370: BUG: multiple engines detected when volume pvc-7bb9b250-ae3e-4482-be3d-93395d7c0370 is detached" node=harvester-6k94p

longhorn-backing-image-manager
time="2021-09-23T08:17:24Z" level=info msg="Backing image manager becomes state running" backingImageManager=backing-image-manager-c00e-6f92 controller=longhorn-backing-image-manager diskUUID=6f926337-c8ba-49a5-b05f-9274f727f46f node=harvester-94tfp nodeID=harvester-94tfp
time="2021-09-23T08:17:24Z" level=info msg="Event(v1.ObjectReference{Kind:\"BackingImageManager\", Namespace:\"longhorn-system\", Name:\"backing-image-manager-c00e-6f92\", UID:\"0bd747b5-99cd-431b-97ca-85103d074ba1\", APIVersion:\"longhorn.io/v1beta1\", ResourceVersion:\"1123759\", FieldPath:\"\"}): type: 'Normal' reason: 'Update' Backing image manager becomes state running"
time="2021-09-23T08:17:24Z" level=debug msg="Start monitoring" backingImageManager=backing-image-manager-c00e-6f92 controller=longhorn-backing-image-manager diskUUID=6f926337-c8ba-49a5-b05f-9274f727f46f node=harvester-94tfp nodeID=harvester-94tfp
time="2021-09-23T08:17:26Z" level=info msg="Event(v1.ObjectReference{Kind:\"Node\", Namespace:\"longhorn-system\", Name:\"harvester-d2pkg\", UID:\"a9ad34d3-8fbc-4c10-87c6-ec2f89c5c31e\", APIVersion:\"longhorn.io/v1beta1\", ResourceVersion:\"1123844\", FieldPath:\"\"}): type: 'Normal' reason: 'Ready' Node harvester-d2pkg is ready"
time="2021-09-23T08:17:26Z" level=debug msg="Requeue longhorn-system/harvester-d2pkg due to conflict: Operation cannot be fulfilled on nodes.longhorn.io \"harvester-d2pkg\": the object has been modified; please apply your changes to the latest version and try again"
time="2021-09-23T08:17:54Z" level=debug msg="Start to sync backing image" backingImage=image-cql6j backingImageManager=backing-image-manager-c00e-6f92 controller=longhorn-backing-image-manager diskUUID=6f926337-c8ba-49a5-b05f-9274f727f46f fromHost=<IP_ADDRESS> node=harvester-94tfp nodeID=harvester-94tfp size=566886400 toHost=<IP_ADDRESS>
time="2021-09-23T08:17:54Z" level=debug msg="Syncing backing image" backingImage=image-cql6j backingImageManager=backing-image-manager-c00e-6f92 controller=longhorn-backing-image-manager diskUUID=6f926337-c8ba-49a5-b05f-9274f727f46f fromHost=<IP_ADDRESS> node=harvester-94tfp nodeID=harvester-94tfp size=566886400 toHost=<IP_ADDRESS>
time="2021-09-23T08:17:54Z" level=info msg="Event(v1.ObjectReference{Kind:\"BackingImageManager\", Namespace:\"longhorn-system\", Name:\"backing-image-manager-c00e-6f92\", UID:\"0bd747b5-99cd-431b-97ca-85103d074ba1\", APIVersion:\"longhorn.io/v1beta1\", ResourceVersion:\"1123908\", FieldPath:\"\"}): type: 'Normal' reason: 'Syncing' Syncing backing image image-cql6j in disk 6f926337-c8ba-49a5-b05f-9274f727f46f on node harvester-94tfp from backing-image-manager-c00e-3af1(<IP_ADDRESS>)"
time="2021-09-23T08:17:54Z" level=info msg="Cannot re-fetch or re-sync backing image file image-cql6j immediately since it is still in the backoff window" backingImage=image-cql6j backingImageManager=backing-image-manager-c00e-6f92 controller=longhorn-backing-image-manager diskUUID=6f926337-c8ba-49a5-b05f-9274f727f46f node=harvester-94tfp nodeID=harvester-94tfp
time="2021-09-23T08:17:54Z" level=info msg="Cannot re-fetch or re-sync backing image file image-cql6j immediately since it is still in the backoff window" backingImage=image-cql6j backingImageManager=backing-image-manager-c00e-6f92 controller=longhorn-backing-image-manager diskUUID=6f926337-c8ba-49a5-b05f-9274f727f46f node=harvester-94tfp nodeID=harvester-94tfp

LH Support Bundle
longhorn-support-bundle_80d8d33e-947c-4d26-8d3f-ce8e8447cf97_2021-09-23T09-20-12Z.zip

Just from the log snippets there are 2 issues present. There might be additional issues in the support bundle, did not check yet.
Issue01 backing image checksum missmatch:
https://github.com/longhorn/longhorn-manager/blob/5118ea589f9840167652a5dc97d4245a4b3868af/controller/backing_image_controller.go#L731
failed to sync backing image file state: BUG: backing image recorded checksum 1050a8ccbaa72d552b72660ed9c43460fb75dc198b1802d0332e5ee6eb5cfb37dce9ac75a7af9b8b06021aad6e383051027d071e2809aff4111f8779185972d7 doesn't match the file checksum 40d883b86625e48ce0bab91f7f70935cca6d2d7f73be0e0b804d1d384dab98f41a676a1d787e966d11ac00876db6b240fa996310794bfa0d4d378b050b7bc4a0 in disk 3af17a6d-c8ea-4b21-8da0-adeeb09e568d

Issue02 multiple engines for a detached volume:
Which runs into this bug case. https://github.com/longhorn/longhorn-manager/blob/0596a3f7cc9ecb5101cff7e45d138c26854e9a56/datastore/longhorn.go#L588
I am guessing since the nodes probably did not restart all at the same time harvester initiated a migration, and then went offline. So once the nodes come back the volume is detaching/detached (status.currentNodeID == "") but the additional engines are still present. One more reason to do the volume-controller refactoring https://github.com/longhorn/longhorn/issues/2527
To fix the current attach/detach intermediate states.

The follow-up of the backing image checksum mismatch issue: https://github.com/longhorn/longhorn/issues/3061

@joshimoo We definitely need the volume controller refactor for 1.3.0. However due to the big amount of the change, I am not sure if we can backport it to v1.2.x, which will be the base for Harvester v1.0 GA release. Is there any other way we can patch it for v1.2.x regarding the multiple engine problem? Also, we need a separate issue for this fix and link it to the volume controller refactoring.

should be resolved by https://github.com/longhorn/longhorn/issues/3315 of bumping the LH to v1.2.3-rc1, can we please help to re-validate it thanks.
To Test:

After rebooting all Harvester nodes concurrently, all VMs should be restored correctly.
Inspecting the state of the corresponding image backing volumes in LH, the volumes originally are infinitely stuck in detaching state with all three replicas in a "failed" state, now it should be restored properly.


Atlas loads fonts insecurely.

When using atlas in an HTTPS site I get these warnings in the console.

Not in atlas in GC.


KeepAlive property is not cloned

When using WebResource clone function, keepAlive property is not cloned
https://github.com/Azure/ms-rest-js/blob/01cfc78e2fe58d376cf1398f758d6e34415864c8/lib/webResource.ts#L418

@joshgummersall could you possibly also include a fix for this in the 2 PRs you have out?  Should just be a matter of making sure keepAlive gets assigned in the clone function.

Included in both 2.x and 1.x


Yarn version update

This is a PR for issue #134.
I started to experience weird build issues during netlify deployment. Mostly with childImageSharp, basically the same error message as what on #133 and #128. This was running fine on my machine and on my machine i had yarn with  version 1.9.4. After updating the yarn version the deployment was succesfull.
Just as a sidenote i added Yarn as prerequisite, because we maintain the package dependencies with yarn and folks using only npm can experience issues, so just to avoid a big wave of questions in the future, i think we need make yarn a prerequisite

Thanks @ZoltanVeres!
We actually want to support npm as well - are you aware of any failures that are only affecting npm?

@erquhart Adding gatsby plugins is much more stable with yarn and running the starter after a renovate[bot] commit gives runtime error in multiple occasions, so running yarn is absolutely a must after a repo sync to me. There were other issues, but can't remember the specifics, using yarn fixed all my struggles. Let me know if you need any changes to this PR, i honestly think we need to bump the yarn version, i believe yarn 1.3.4 has security issues as well.

For sure on the update, but I'd rather not say Yarn is a requirement just yet, npm install should do fine too, maybe with a cache clean?
Also: added you as a collaborator so you can merge PR's - thanks for all of your help on this repo!


Support for For and While Stmt in PrintASTNodeSub

Now kASTForStmt is printed as an operator
   (op=<for> cond=(op=<<>:int reg: 1 L=(op=<i>:lvalue<int> reg: 1) R=(op=<n>:lvalue<int> reg: 2))),

done in 50ba54679d21f3b4ec7edf7426a591db87b435ba


Initial docs

add examples dir - from file extraction example
update README

:tada: This PR is included in version 1.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Simplify/clean up folders

Related to #188 , and as per #197 and suggested by @shiqin-liu , we should simplify and clean up folders (e.g. the old validation folder is specifically related to the 25-cities project, the analysis and maps folder likewise -- they aren't required)

A clarification: regarding the validation-related folders - while I did (re)move these in a recent push on a feature branch https://github.com/global-healthy-liveable-cities/global-indicators/commit/d5e0c4ce97dcdc4de1e72f8aa3ac4490efe27b00 and https://github.com/global-healthy-liveable-cities/global-indicators/commit/3b2c4707ce6881428106ec5560d9f227bf42e6c4, this wasn't intended as a remove per se.  The plan is to re-incorporate the local partner and GIS-based data validation in our framework.  However, the current code is no longer functional due to heavy re-factoring during the last 6-months, so these have been moved to archive folders on my local computer and can otherwise still be accessed from the Git version history.
It is one of our priorities to re-introduce these (e.g. as per #170).


Add History of Spaceflight RSS from SpaceDock

This pull request was automatically generated by SpaceDock on behalf of Morphisor, to add History of Spaceflight RSS to CKAN.
Please direct questions about this pull request to Morphisor.
Mod details:
name = /mod/2385/History%20of%20Spaceflight%20RSS
author = Morphisor
abstract = A contract pack recreating almost all real-life missions that were actually flown, currently up until 1967. This version is for users of RSS and fits well with RP-0/RP-1.
license = CC BY-NC-SA 4.0
Homepage = https://github.com/Frylovespi/History-of-Spaceflight/
description =
History of Spaceflight is a contract pack that aims to accurately reproduce historical missions as they happened. Currently this contract pack covers nearly 700 missions, covering nearly every real mission that was ever flown up until 1967.
The contracts progress on a chronological basis, starting with the early rocket tests, going into the very first satellites and to early interplanetary and manned missions.
All the major programs are there: Vanguard, Explorer, Sputnik, Kosmos, Keyhole, Vostok, Mercury, Gemini, Pioneer, Mariner, Proton... I could go on.
This is the RSS version of the contract pack, which is exclusively suitable for the RSS planet pack. It is made with RP-0/RP-1 in mind and should suit those who use those mods very well.
In terms of progression, the pack has been divided into 3 main sections: US missions, Soviet missions and International missions. Each can be played separately and you can even delete their respective maps from the mod folder if you don't wish to use them.
Further, within each of these trees the pack is structured by the actual programs involved, keeping progress mostly within each program where possible. This means that mostly you will have to do each program's missions if you want missions from further in that same program.
It is also possible to skip ahead with a little cheating however, by using the CC debug menu. When in the main KSC view, press alt-F10 (by default).
Navigate to the History of Spaceflight contracts and find the contract you want to do. It will have a CompleteContract requirement with a button next to it.
Simply press this button to disable the requirement, allowing the game to load the contract to be directly available. You may need to fastforward a bit for it to load.
Contract Configurator by Nightingale is required to be able to use this contract pack.
This contract pack should be compatible with any version that's supported by Contract Configurator, starting with 1.2.2.
Please report any issues in the KSP forum thread (TBD) or on the mod GitHub issues page: https://github.com/Frylovespi/History-of-Spaceflight/

Asked the author about the option of a depends/conflicts with RSS.
https://forum.kerbalspaceprogram.com/index.php?/topic/192880-12-19x-contract-pack-history-of-spaceflight/&do=findComment&comment=3765394


archive/tar: unbounded memory consumption when reading headers [go1.15 backport] - CVE-2022-2879

From Golang security announcement:
Reader.Read did not set a limit on the maximum size of file headers.
A maliciously crafted archive could cause Read to allocate unbounded
amounts of memory, potentially causing resource exhaustion or panics.
Reader.Read now limits the maximum size of header blocks to 1 MiB.
Thanks to Adam Korczynski (ADA Logics) and OSS-Fuzz for reporting this issue.
This is CVE-2022-2879 and Go issue https://go.dev/issue/54853.

Closed by #492


Use crypto drivers for Bluetooth

In addition to making the Bluetooth subsystem work with crypto drivers, this PR makes a few changes to accommodate it:

This adds all the primitives needed to support the Bluetooth subsystem, including new algorithms (PRNG, CMAC, and ECC-DH).
Implements the new algorithms and their associated modes in both the TinyCrypt and mbedTLS shims
Changes the Bluetooth subsystem to not require TinyCrypt, but rather declare its need for particular algorithms; the crypto backend can then be selected.  Declaring the need for a particular algorithm will enable/disable the features in either mbedTLS or TinyCrypt.  In the case of mbedTLS, however, only the default configuration file is aware of these declarations.

This has only been build-tested; I don't know enough of Bluetooth to carry on the necessary testing to ensure that this is working correctly, and I would appreciate help in getting this tested.
This is a step in closing #8081.

Codecov Report

Merging #8351 into master will increase coverage by 1.58%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #8351      +/-   ##
==========================================
+ Coverage   64.53%   66.12%   +1.58%
==========================================
  Files         422      409      -13
  Lines       40265    38942    -1323
  Branches     6797     6553     -244
==========================================
- Hits        25986    25749     -237
+ Misses      11157    10126    -1031
+ Partials     3122     3067      -55

Impacted Files
Coverage Δ

include/drivers/bluetooth/hci_driver.h

include/bluetooth/buf.h

include/bluetooth/bluetooth.h

tests/bluetooth/bluetooth/src/bluetooth.c

include/bluetooth/hci.h

subsys/bluetooth/common/log.c

subsys/bluetooth/host/uuid.c

include/net/buf.h
100% <0%> (+38.46%)
:arrow_up:

include/misc/byteorder.h
96.15% <0%> (+39.33%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 791daa7...e539b39. Read the comment docs.

@vakulgarg I've added more detailed size measurements in one of the commit messages; I'm copying it here so it's easier to find:

        Sample: mesh      Target: bbc_microbit

        Backend: tinycrypt
        Memory region         Used Size  Region Size  %age Used
                   FLASH:       98040 B       256 KB     37.40%
                    SRAM:       15760 B        16 KB     96.19%
                IDT_LIST:         132 B         2 KB      6.45%

        Backend: mbedTLS
        Memory region         Used Size  Region Size  %age Used
                   FLASH:      124200 B       256 KB     47.38%
                    SRAM:       15932 B        16 KB     97.24%
                IDT_LIST:         132 B         2 KB      6.45%

        Sample: mesh_demo     Target: bbc_microbit

        Backend: tinycrypt
        Memory region         Used Size  Region Size  %age Used
                   FLASH:       93904 B       256 KB     35.82%
                    SRAM:       14548 B        16 KB     88.79%
                IDT_LIST:         132 B         2 KB      6.45%

        Backend: mbedTLS
        Memory region         Used Size  Region Size  %age Used
                   FLASH:      113004 B       256 KB     43.11%
                    SRAM:       14720 B        16 KB     89.84%
                IDT_LIST:         132 B         2 KB      6.45%

what are the plans for this?

@lpereira the size different is big enough that it will cause trouble on our chips, so this is a problem.
CC @sjanc @MariuszSkamra

But the size difference between tiny and mbed where the application can choose which one it wants to use? Also, on the mbed, could this be that we are including stuff that isn't needed?

But the size difference between tiny and mbed where the application can choose which one it wants to use?

@dleach02 Not sure I understand this. I thought we'd replace TinyCrypt with mbedTLS, making the size increase essentially mandatory.

The first comment above indicates that the changes here are to define a standard crypto front end interface and to hook that up to either tinycrypt or mbedtls. I'm all for the standard interface and need this PR for the work I'm doing on KW41Z

@dleach02 apologies, I misread the purpose of the PR, thanks for the explanation!

I am curious though about the size differences. @lpereira, can you send me the two map files for this? (david.leach @ nxp.com)

@dleach02 You can find the files here.  Warning: they decompress to the current directory.

So the two things that get you when using mbedtls is the inclusion of bignum.c and des.c. These add 10K from .text and .rodata. The bignum.c states that it is required when doing ecc but I'm not sure why des.c is included?
Also, in general the mbedtls functions seem to be slightly bigger then tinycrypt.

I agree with the PR objectives. I was just curious about what the sizing comparison is. I think we should have a separate PR to further break down mbedTLS based on the application needs as that would be good thing to do.

@lpereira can you rebase this?

I'd like to get PTS tests run on this before merging, to make sure we're not breaking anything with pairing or mesh provisioning.

Rebased.

@vakulgarg mind taking another look?

@jhedberg I appreciate this being tested on real devices.  I lack the knowledge to perform those myself.  I'll remove the DNM tag once this is confirmed working.

@jhedberg @lpereira I wanted to test this yesterday but there were some compilation error, and I gave up.
https://pastebin.com/rpvWJZfF

@MariuszSkamra I've only test-built this for the bbc_microbit board; the mesh sample builds fine here.  Can you share which code you were building, board name, and Kconfig options different from the default for that board?

@lpereira To test this against PTS (Bluetooth qualification tool) I need to have tests/bluetooth/tester application built. I'm using nrf52840_pca10056  board. Kconfig options are defined in tests/bluetooth/tester/default.conf

@MariuszSkamra Thanks.  I've fixed the build and force-pushed.

As far as the size goes: I agree that maybe some changes could be made to the config file. In addition, it may be worth working with the upstream project if there are things in mbed TLS that could be made more configurable.

Fixed some issues found in CI.  Rebased, and repushed.

Excellent, @MariuszSkamra, thanks for testing!  Results are better than I expected, to be honest.  I don't know much about Bluetooth, so short of addressing the assertions, I would need help to get every PTS test to pass again.

@lpereira as just point the things I'll fix in this pr. I'll rebase it apply the fixes I mentioned here.

I've removed the DNM as this seems to be a generally accepted approach.  @ceolin is ironing out some issues in this patch and will send a newer version soon; I'll close this PR once that happens.

Just a heads-up: #9181 will likely cause conflicts and you'll want to update this PR to follow suit (or possibly the other way around - depending on which PR gets merged first)

@lpereira #9227

Closing in favor of #9227 (@ceolin is taking over the series from me)


Use input field as the scrolling container

Fixes #163
Fixes #162
Fixes #165

This change is

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Nit: several empty lines contain unnecessary whitespace.

Review status: 0 of 23 files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

Fixed

Comments from Reviewable

Reviewed 1 of 23 files at r2, 1 of 2 files at r3, 21 of 22 files at r4, 2 of 2 files at r5.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

Reviewed 1 of 2 files at r3, 21 of 22 files at r4, 2 of 2 files at r5, 1 of 1 files at r6.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Refactor app with json metadata as sources

This PR refactors the sample app to:

Have a structure that will enable the public developers site to pull samples data into the samples directory. As an MVP this involved:

An individual directory for each sample
Containing a README.metadata.json file and the sourecode
As part of this, I also updated any sample names (names only, not implementation) to match the common designs. And I removed one sample which is not part of the designs (serialize feature collection).

I have also made the Sample Viewer app use the metadata:

We generate a single json as a source of the metadata files - this means we don't need to add and load each individual json file as an asset - which when we get up to 100s of files would be a large overhead. This is loaded in main.dart and fed to the SampleListView.
Refactor the sample model class to utilize the metadata. We will want to utilize more of this as we build out the viewer app.
Refactor the sample viewer to use a Widget and simplifies by removing the need for an AppBar. We manually list these in samples_widget_list.dart, and access from the viewer using a json key when the sample is selected.
Added a SampleDetailPage widget which displays the above widget and which we will be able to more easily extend when we implement the ability to view the Readme and code tabs (per the full sample viewer design)

Other updates:

Made all the SampleState classes private
Renamed the main app to SampleViewerApp from MyApp
Updated launch.json
Updated README

removing the need for an AppBar

The app bar gave us a "back" button. And if I remember correctly, on a particular sample that used a drawer, the app bar had a drawer pop-em-out button (but that was mostly accidental). Does the new setup cover that?

The app bar gave us a "back" button. And if I remember correctly, on a particular sample that used a drawer, the app bar had a drawer pop-em-out button (but that was mostly accidental). Does the new setup cover that?

The AppBar is instead provided by SampleDetailPage, which provides the back navigation functionality, and displays the selected sample name.
For the samples themselves, AppBar was only being used to display the title. Find Address and Set Basemap have icons outside of the AppBar that load drawers - the drawer itself is defined on the Scaffold, so that behavior is the same.
Based on other SDK viewers, the app bar should be for the sample viewer application itself - e.g. it may eventually have an i icon that loads the readme/code views, and then any sample UI itself is a self contained widget. So nothing has changed / we haven't lost anything in that regard, but then in terms of the app bar being essentially a level higher, it leaves us better placed to extend the viewer application.

The AppBar on the "Find Address" sample gives us this weird extra bonus drawer button. If we can get rid of that, that would be preferred.

The AppBar on the "Find Address" sample gives us this weird extra bonus drawer button. If we can get rid of that, that would be preferred. ("Set Basemap" has it too.)

Yeah that's gone with the new architecture, so for Find Address there's just the icon next to the search bar. Set Basemap just has the basemap icon, bottom right.
I think Flutter automatically adds the hamburger if a drawer is created (either as the leading or trailing, depending where the drawer is configured start/end) and you have to overwrite in the AppBar properties. But since the AppBar is detached from the sample now it doesn't appear anymore anyway.


vlan tagged interfaces on ECW5211-L

Are VLAN tagged interfaces supported on the ECW5211-L?
After doing the following:
localhost ~ # ip l add link eth1 name vlan200 type vlan id 200
localhost ~ # ip l add link eth1 name vlan300 type vlan id 300
localhost ~ # ip l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br-lan qlen 1000
    link/ether 90:3c:b3:c6:98:08 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan qlen 1000
    link/ether 90:3c:b3:c6:98:09 brd ff:ff:ff:ff:ff:ff
4: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue qlen 1000
    link/ether 90:3c:b3:c6:98:08 brd ff:ff:ff:ff:ff:ff
5: wl24g: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 90:3c:b3:c6:98:0a brd ff:ff:ff:ff:ff:ff
6: wl5g: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 90:3c:b3:c6:98:0b brd ff:ff:ff:ff:ff:ff
15: vlan200@eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 90:3c:b3:c6:98:09 brd ff:ff:ff:ff:ff:ff
16: vlan300@eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 90:3c:b3:c6:98:09 brd ff:ff:ff:ff:ff:ff
localhost ~ # ip a add <IP_ADDRESS>/24 dev vlan200
localhost ~ # ip a add <IP_ADDRESS>/24 dev vlan300
localhost ~ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br-lan qlen 1000
    link/ether 90:3c:b3:c6:98:08 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan qlen 1000
    link/ether 90:3c:b3:c6:98:09 brd ff:ff:ff:ff:ff:ff
4: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue qlen 1000
    link/ether 90:3c:b3:c6:98:08 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global br-lan
       valid_lft forever preferred_lft forever
    inet6 fe80::923c:b3ff:fec6:9808/64 scope link
       valid_lft forever preferred_lft forever
5: wl24g: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 90:3c:b3:c6:98:0a brd ff:ff:ff:ff:ff:ff
6: wl5g: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 90:3c:b3:c6:98:0b brd ff:ff:ff:ff:ff:ff
15: vlan200@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue qlen 1000
    link/ether 90:3c:b3:c6:98:09 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 scope global vlan200
       valid_lft forever preferred_lft forever
    inet6 fe80::923c:b3ff:fec6:9809/64 scope link
       valid_lft forever preferred_lft forever
16: vlan300@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue qlen 1000
    link/ether 90:3c:b3:c6:98:09 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 scope global vlan300
       valid_lft forever preferred_lft forever
    inet6 fe80::923c:b3ff:fec6:9809/64 scope link
       valid_lft forever preferred_lft forever

Edit: Same results after ripping out default bridge br-lan:
localhost ~ # cat /etc/systemd/network/eth0.network
[Match]
Name=eth0

[Network]
Address=<IP_ADDRESS>/24
VLAN=vlan200
localhost ~ # cat /etc/systemd/network/eth1.network
[Match]
Name=eth1

[Network]
Address=<IP_ADDRESS>/24
localhost ~ # cat /etc/systemd/network/vlan200.netdev
[NetDev]
Name=vlan200
Kind=vlan

[VLAN]
Id=200
localhost ~ # cat /etc/systemd/network/vlan200.network
[Match]
Name=vlan200

[Network]
Address=<IP_ADDRESS>/24
localhost ~ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq qlen 1000
    link/ether 90:3c:b3:c6:98:38 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::923c:b3ff:fec6:9838/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq qlen 1000
    link/ether 90:3c:b3:c6:98:39 brd ff:ff:ff:ff:ff:ff
4: vlan200@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue qlen 1000
    link/ether 90:3c:b3:c6:98:38 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global vlan200
       valid_lft forever preferred_lft forever
    inet6 fe80::923c:b3ff:fec6:9838/64 scope link
       valid_lft forever preferred_lft forever
5: wl5g: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 90:3c:b3:c6:98:3b brd ff:ff:ff:ff:ff:ff
6: wl24g: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 90:3c:b3:c6:98:3a brd ff:ff:ff:ff:ff:ff

I'm unable to ping these addressed tagged interfaces from a connected system with a matching (verified working) configuration for 802.1q VLAN tagged interfaces on vlans 200 and 300. I also tried creating the tagged interfaces on the bridge br-lan but also was not successful pinging the tagged interfaces from the connected system. Seems like this does not work, at all.
Is this a known limitation of the current hardware support for the ECW5211-L?
cc: @storrgie

Hi Jon,
This is a limitation of the current driver, the DSA driver combo will remove those limits and allow normal HW offloaded VLAN filtering.
I plan to push the updated kernel with new drivers next week

@robimarko thanks for the quick response! I will be watching for the new kernel to arrive here.

@jmpolom Updated kernel that contains DSA has been pushed.
CI has also already finished building:
https://drone.sartura.hr/artifacts/main/ecw5211-l-t/

@robimarko thanks for getting this out quickly. I will give this a try.


TEP-0058: Graceful Termination - PipelineRunCancelled is deprecated

Changes

In TEP-0058: Graceful Termination, we deprecated PipelineRunCancelled and added:

StoppedRunFinally - To "stop" (i.e. let the tasks complete, then execute finally tasks) a Pipeline
CancelledRunFinally - To "cancel" (i.e. interrupt any executing non finally tasks, then execute finally tasks)
Cancelled - Same as today's PipelineRunCancelled - i.e. interrupt any executing tasks without running finally tasks

The above was released in v0.25 in June 2021.
This change adds the deprecated field - PipelineRunCancelled - to the deprecation table.
Issue: https://github.com/tektoncd/pipeline/issues/4611
Pull Requests:

https://github.com/tektoncd/pipeline/pull/3915
https://github.com/tektoncd/pipeline/pull/4246
https://github.com/tektoncd/community/pull/638

/kind cleanup
Submitter Checklist
As the author of this PR, please check off the items in this checklist:

[x] Docs included if any changes are user facing
[n/a] Tests included if any functionality added or changed
[x] Follows the commit message standard
[x] Meets the Tekton contributor standards (including
functionality, content, code)
[x] Release notes block below has been filled in or deleted (only if no user facing changes)

Release Notes

NONE

/lgtm
Thanks Jerop!

thank you @jerop, please add this announcement in the release notes 🙏

@pritidesai it was already announced in v0.25, just not added to the deprecations table - do you think we need to announce again?

thank you @jerop, its my misunderstanding, I thought we are removing the flag in the next release (0.34 - March 2022). This is just an update the the deprecation table. When are we removing the flag then? with V1? or after promoting graceful termination to beta?


Shapehelper infra used in Dim Analysis

This PR uses the ShapeHelper interface to return the shape helpers used for the analysis. We use to detect each op, and then call that op's shape helper. Now it is done via the interface.
There is another bit of cleanup: we had 2 lists of unsupported ops, one for shape inference and one for shape helper. This replication adds to the work needed to be done when adding new operators. There is now one instance: src/Dialect/ONNX/ONNXUnsupportedOps.hpp with a macro to generate the right content in the right context.
I also verified that each ONNX ops that has the shape helper interface did indeed have the code associated with it.

Please do not merge into main until the LLVM PR goes through (#1977).

Jenkins Linux s390x Build #9535 [push] Shapehelper infra used i... started at 09:15

Jenkins Linux amd64 Build #9516 [push] Shapehelper infra used i... started at 08:15

Jenkins Linux amd64 Build #9516 [push] Shapehelper infra used i... passed after 1 hr 34 min

Jenkins Linux s390x Build #9535 [push] Shapehelper infra used i... passed after 2 hr 6 min

Jenkins Linux ppc64le Build #8581 [push] Shapehelper infra used i... passed after 3 hr 3 min


Throttle not functioning consistently in all app states

Hi,
We are working on a new app and have redux and redux-saga at the core. Our app requires frequent location updates so we have a LOCATION_CHANGED action being dispatched every 500ms. We have API requests that we also want to make triggered by this action so in our saga we have.
yield throttle(3000, LOCATION_CHANGED, requestReverseGeocodeData);
This function in turn dispatches an API_REVERSE_GEOCODE_REQUEST action.
Now this works 99% of the time. Every 3 seconds we get 6 LOCATION_CHANGED actions and 1 API_REVERSE_GEOCODE_REQUEST action. However we have found an edge case. On an Android device when the app is placed in the background we continue to get the LOCATION_CHANGED actions from out React Native code however the API_REVERSE_GEOCODE_REQUEST action is never dispatched.
Changing the throttle to takeLatest works perfectly we get 1 API_REVERSE_GEOCODE_REQUEST for every LOCATION_CHANGED action.
Any help or guidance would be appreciated.

Are you sure this happens on a real device and not only on RN simulator? The latter (not sure about real devices) has known issues around timers and throttle has to use setTimeout under the hood.

Yes this is on a real device with the packager running on my local machine.

Doing some tests, it could very well be to do with setTimeout/setInterval being called properly with RN on android.
setInterval(() => console.log('test'), 1000); stops firing the second I put the app into the background.

Need to close this as not actionable on our part. I think there are some workarounds for this on RN, but you'd have to search for them on your own as I have no experience with that platform. Feel free to comment if you find the solution for this!


fix node size description

Description of your changes

Fixes #
I have:

[ x] Read and followed Upbound's contribution process.
[ x] Run make reviewable to ensure this PR is ready for review.
[ x] Added backport release-x.y labels to auto-backport this PR, as appropriate.

How has this code been tested

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thank you for this contribution @natereid72! Do you want to update the PR description now with some more details about how you tested this change during our walk through this morning? :)

thanks, will do


[SPARK-27269][SQL] File source v2 should validate data schema only

What changes were proposed in this pull request?
Currently, File source v2 allows each data source to specify the supported data types by implementing the method supportsDataType in FileScan and FileWriteBuilder.
However, in the read path, the validation checks all the data types in readSchema, which might contain partition columns. This is actually a regression. E.g. Text data source only supports String data type, while the partition columns can still contain Integer type since partition columns are processed by Spark.
This PR is to:

Refactor schema validation and check data schema only.
Filter the partition columns in data schema if user specified schema provided.

How was this patch tested?
Unit test

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9275/
Test PASSed.

@cloud-fan @dongjoon-hyun @HyukjinKwon

Test build #103913 has started for PR 24203 at commit 43743b3.

Test build #103913 has finished for PR 24203 at commit 43743b3.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103913/
Test FAILed.

Test build #103936 has started for PR 24203 at commit 214bd8b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9294/
Test PASSed.

Test build #103936 has finished for PR 24203 at commit 214bd8b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103936/
Test PASSed.

@dongjoon-hyun, do you mean the issue is that we don't check the schema in toBatch but lazily within Table? The data schema looks being checked via overridden schema at FileTable in read-path.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9323/
Test PASSed.

Test build #103966 has started for PR 24203 at commit d8b2638.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9324/
Test PASSed.

Test build #103967 has started for PR 24203 at commit 63466b1.

Test build #103967 has finished for PR 24203 at commit 63466b1.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103967/
Test FAILed.

Test build #103966 has finished for PR 24203 at commit d8b2638.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103966/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/9336/
Test PASSed.

Test build #103984 has started for PR 24203 at commit 4ca742d.

Test build #103984 has finished for PR 24203 at commit 4ca742d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/103984/
Test PASSed.

Merged to master.


ci(travis): enabled commitlint in Travis CI

Commitlint was originally turned off to prevent old commits from causing Travis CI to fail.
Closes #210

@ashblue I assume you want to push the magic button to merge?

@allancorbett Thanks. Yeah, go for it (once Travis has finished doing its thing)!


Missing instance for askLiterate

Not all common data types have AskInliterate instances. Non-exhaustive list

()
'Integer`
Maybe a
[a]

This makes it impossible to run cells returning these types without wrapping them into show

Yes, this is a problem of the inliterate library that we are using 😄

Paging @glutamate ^^


Finalize 2.0 Release

Description
We created the dev-2.0 branch to introduce a handful of major breaking changes to connexion.
https://github.com/zalando/connexion/milestone/10
At the time, the two major changes were:

#420 openapi 3 support
#406 removing vendored swagger-ui

Both of these changes are now in dev-2.0
Next Steps
What other things should be in the 2.0 release? Is it large enough as is? What's could be better?
It would be great to finalize the feature set for Connexion 2.0 and get it shipped! :ship:
Let me know what you think!

The PR for merging dev-2.0 -> master is #619

I successfully run this development connexion with a non-trivial yaml. Good work!
I now have some issues with
connexion/jsonref.py:from jsonschema import RefResolver

because it can't parse yaml schema files like https://opensource.zalando.com/problem/schema.yaml
I workaround'd it in the jsonschema module

+++ b/jsonschema/validators.py
@@ -2,6 +2,7 @@ from __future__ import division

 import contextlib
 import json
+import yaml
 import numbers

 try:
@@ -466,12 +467,15 @@ class RefResolver(object):
             # Requests has support for detecting the correct encoding of
             # json over http
             if callable(requests.Response.json):
-                result = requests.get(uri).json()
-            else:
-                result = requests.get(uri).json
+                try:
+                    result = requests.get(uri).json()
+                except:
+                    result = None
+            if not result:
+               result = yaml.load(requests.get(uri).content)
         else:
             # Otherwise, pass off to urllib and assume utf-8
-            result = json.loads(urlopen(uri).read().decode("utf-8"))
+           result = yaml.load(urlopen(uri).read().decode("utf-8"))

         if self.cache_remote:

But on https://github.com/Julian/jsonschema/issues/420 it seems yaml is currently out of scope.
Can we fix it in connexion subclassing RefResolver?

@dtkav thanks for the great work!

Thanks for the bug report @ioggstream ! I'll follow that up in a separate ticket. I'd like to keep this one a bit more of a high-level discussion on what features / breaking changes should make it into Connexion 2.0.
@hjacobs and @jmcs , I would love your thoughts especially on high-level next steps!
Thanks to everyone in the community who helped make these features possible :heart:

@dtkav I think the 2.0 release should not be bigger as the OpenAPI 3 support is already a monster change. Let's get some feedback from people trying it out and the releasing it with any additional blocker bug fixes we encounter.

I'm a quite new user of connexion, but I really like the project.
OpenAPI 3 uses JSON Schema Specification Wright Draft 00 (= Draft 5), which is mostly compatible to JSON-Schema Draft 4.
But it points out that "$ref" should only be used wherever a schema is expected. This allows "$ref" to be a valid key for the "properties" keyword. The new resolve_refs function breaks that feature.
Future versions of OpenAPI may use Draft 6 or later, which introduce more incompabilities, e.g. exclusiveMinimum/exclusiveMaximum being a number instead of a boolean.
I therefore want to discuss if it makes sense to have a validator_map per api (add parameter to add_api) instead of having it globally.
In our setup, we already have some extensions to Draft4Validator, regarding x-nullable and readOnly (the first one with a different implementation than here). I will open seperate issues/PRs for discussion about these, but that shouldn't block the 2.0 release.

Can you have a look at https://github.com/zalando/connexion/compare/dev-2.0...cziebuhr:nullable ? According to OpenAPI 3, nullable "allows sending a null value for the defined schema", so it should also work outside of properties.

Depending on using swagger 2 / openapi 3, ConnexionOptions sets swagger_ui_local_path to swagger_ui_2_path / swagger_ui_3_path. IMHO this is only the major version of swagger-ui, swagger-ui 3 supports both swagger 2 and openapi 3.

PR for nullable is at https://github.com/zalando/connexion/pull/684.
I changed it a bit, so I don't need to overwrite iter_errors.
And I have another suggestion, regarding changes in security handling: https://github.com/zalando/connexion/compare/dev-2.0...cziebuhr:security
It addresses some issues I had when trying to use connexion for a production-ready product.

Use token_info['sub'] instead of token_info['uid'] to better align with RFC7662 (oauth token introspection)
x-tokenInfoUrl is only kept for compability reasons. IMHO it's only useful for a small amount of people, because it hardcodes bearer authentication and the response format is not standardized. I can add an example later on how to use RFC7662 with x-tokenInfoFunc.
validate_token_info was not customizable and simply compared two sets of strings.
In our setup, we want to add more logic, e.g. scope 'ressource' is superior to 'ressource:read'. One can now set a custom scope validator.
If security definitons don't match the connexion implementation (only one definition supported), connexion silently ignors ALL definitions and allows full access to the api
Basic authentication had to be done manully as function decorator instead of using the security definitions from the swagger definition. Now one can define callbacks similar to x-tokenInfoFunc.
Bonus: If both, oauth and basic auth is specified for an operation, the new x-basicInfoFunc also receives the required oauth scopes to use them for authorization.
Also added support for apiKey for completeness, I don't use it myself

Sorry for "misusing" the dev-2.0 issue here, but I think it would be huge benefit for all connexion users and would fit into a major version bump.
It's not yet fully tested and documentation/examples are not updated yet, but I would like to get some general feedback on that approach.

Thank @cziebuhr - I'll try to take a look at #684 tonight.
Can you make a pull request for your other proposed changes as well? That's the best way to get specific feedback. We'll likely need some more in-depth review from Zalando folks because your proposal changes the security flow.

Speaking in the name of OpenAPI-generator, a community fork of swagger-codegen, we are very eager to see the v2 of connexion. Our current flask-connexion generator is broken because we can only output a v3 version of the spec in the templates (https://github.com/OpenAPITools/openapi-generator/issues/323).
So :100:  to release as soon as possible. Do you have an ETA for the release or for a RC ?

It's a big plus one for us as well! Looking forward to it :)

A bit of an update:
PR's merged:

#681 allows references to yaml files (thanks @ioggstream )
#683 adds x-openapi-router-controller for OpenAPI-generator compatibility. (thanks @wheelsandmetal)
#684 added (proper) support for jsonschema nullable, readOnly, and writeOnly (great work @cziebuhr )

PR's in need of review:

#686 changes the security flow, so probably needs proper review from someone at zalando (@jmcs or @hjacobs)

Issues in work:

#691 oneOf bug (thanks @czchen)
#692 setting base_url does not update the spec that is served to swagger-ui (thanks @czchen)

Based on this thread so far, it sounds like #692 is probably the last feature to make it in the merge window.
Thanks to everyone who has been testing the dev-2.0 branch and reporting issues.
Everyone please test the dev-2.0 branch if you can!
It's easy to install it with:
pip install<EMAIL_ADDRESS>
@cbornet and @Lawouach - thanks for the feedback! It's very hard to to give any concrete dates - It will depend heavily on the availability of the folks at Zalando.

I released version 2.0.0rc1 this can be installed with pip --pre ... (or pipenv install --pre ...).

FYI: Due to #707 if you want to install RC1 with pip install --pre, you'll need to prevent jsonschema from installing their alpha release as well:
pip install --pre connexion[swagger-ui] 'jsonschema<3.0.0'

I released version 2.0.0rc2 with the jsonschema version fix.

I released version 2.0.0rc3 with  https://github.com/zalando/connexion/pull/712

Feature request: it would be great to have JWT support in v2.0, which seems to have been asked many times before (#607, #389, #124).

FYI @lsorber , @krise3k put up this great PR (#732) for JWT support.

@jmcs has released 2.0.0rc5 :tada:
pip install --pre connexion[swagger-ui] to install it.

Big well done guys--sorry I haven't been active but I'm definitely keeping an eye. @dtkav Would you consider rc5 to be somewhat stable? I can start using it soon if so

@JuxhinDB yes, I think it's fairly stable. Please bang on it and see if you find any issues. I've recommended that we push RC5 as the 2.0 release (and put JWT support in 2.1, which could easily be soon after).


[Do not merge] Run rust tests only

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 89.64%. Comparing base (f8e1840) to head (f59347e).

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #961      +/-   ##
============================================
+ Coverage     86.29%   89.64%   +3.35%
============================================
  Files           168       86      -82
  Lines         19294    15499    -3795
  Branches       1086        0    -1086
============================================
- Hits          16649    13894    -2755
+ Misses         2315     1605     -710
+ Partials        330        0     -330

Flag
Coverage Δ

uitests
?

uitests-android
?

unittests
89.64% <ø> (+4.08%)
:arrow_up:

unittests-android
?

unittests-ios
?

unittests-rust
89.64% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Field 'None'  in .Net Enumeration is not supported

Prerequisites

[x] Are you running the latest version?
[x] Are you reporting to the correct repository?

Description
Hi,
Field None in .Net Enumeration is not supported, works fine in ipy2
Steps to Reproduce
import clr
import sys
import System

testEnum = System.IO.FileShare.None

Expected behavior:
No Error
Actual behavior:
Syntax Error

Versions
ipy 3.4.0

None is a keyword as part of the Python 3 language spec so you cannot use . to access the member. You'll have to use alternate notation, for example FileShare["None"]. See None is a keyword in the upgrade documentation.

Ah ok, I understand, thanks for the feedback.
Congratulations again to the team


DLA-34 model

Hi, do you have any plan to implement the DLA-34 (deep layer aggregation) network described in the paper? I think it provide a better speed/accuracy trade off than the hourglass model (3 times faster for 3 to 4 less AP on COCO dataset).

Hi @sonctmirasoft ,
yes, I looked into DLA models. The problem is that TensorFlow does not support deformable convolutions. As you read, all other networks need them. There are open source libraries for deformable convolutions, but they aren't optimized. I don't think that you get 3x speedup. The hourglass model is probably still faster.

Thanks for the head up, I guess we have to use the Pytorch version for now.

@see-- do you have the plan to release the training code？

You can upvote here for deformable convolution v2 https://github.com/tensorflow/addons/issues/179


Update FA embed rating colors

Yeah, pretty much that

Suggested:
Red - #992D22
Blue - #697CC1
Green - #79A977


[SuperEditor] Typing fast can place the caret at the wrong offset

Steps to reproduce on stable:

Open Demo app
On an empty paragraph, start typing fast

Expected: characters are inserted and caret continues at the end of the paragraph
Actual:
https://user-images.githubusercontent.com/31278849/211542114-6d9c27fa-592b-4875-a2c1-480a76ffb6cf.mov

@matthew-carroll


I can not get selected node after append

Hi.
I am trying to get the node after appending it, but I'm not getting.
My code:
`
let node = this.$refs.tree.append({ text: 'My Text', state: { selected: true }});
let selection = this.$refs.tree.selected();
alert(selection.length);  // <-------- this always stays zero
`
Can you help me solve this?
Thank you in advance.

Hi!
Yeah, it is a bug. I will fix it soon.
To solve this problem now you can use:
let selection = this.$refs.tree.find({ state: {selected: true}})

This works! Thanks.

Now your code will works:
let node = this.$refs.tree.append({ text: 'My Text', state: { selected: true }});
let selection = this.$refs.tree.selected();


Add temp fix for the failing knative installation

Add temp fix for the failing knative installation

Codecov Report

Merging #410 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master    #410   +/-   ##
======================================
  Coverage    82.6%   82.6%
======================================
  Files          29      29
  Lines         776     776
  Branches       54      54
======================================
  Hits          641     641
  Misses         95      95
  Partials       40      40

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1f128bc...b7389ad. Read the comment docs.


[WIP] Refactor work loop a bit

Besides the legendary code review just given by Dylan, I don't have anything to add.


Set subject_id & subject_type

Is there any way to set directly subject_id & subject_type so it will reduce some queries in some cases?

I think you can just set it, those or just attributes of an eloquent model.

I know it can be done via Model, but how can I do that by activity() helper function?

No you can't do it directly, only way is to chain on causedBy demonstrated in the readme.

Hello - Can I have further clarification on what you mean by chain on causedBy?
I am using the activity log to reference a page open. It is not active on a model directly although I'd like to reference the associated model using subject_id and subject type. Perhaps I should just use properties for that data instead for this type of activity?


CSOPS-1458: Refactor scooch centering

Images were missing on Firefox due to the way we handled centering when displaying fixed-width items. It used large negative margins to position the elements. Unfortunately, Firefox would incorrectly calculate the width of .m-scooch-inner because of margin-right: -30000px on the last .m-item. This PR changes how we handle centering fixed-width images, which corrects the bug in Firefox.
Status: Ready for review
Reviewers: @donnielrt @fractaltheory
GH Tickets:

https://github.com/mobify/scooch/issues/30

Changes

Remove all old styles pertaining to .m-center:not(.m-fluid)
Set the margin of the first item to 50%
For all items, transform: translate(-50%)

Notes

This is an alternative fix to https://github.com/mobify/scooch/pull/32

Hey, we’re looking to prune older abandoned PRs. If this PR is still relevant and you would like to see it merged in, please reopen the PR and we’ll add it to our backlog! Thanks!

This version of Scooch is now being used on the GLFS/T builds and seems to be working quite well: https://mobify.atlassian.net/browse/GLFS-130, https://mobify.atlassian.net/browse/GLFT-152

@donnielrt would be great if we could get this PR merged this week!

@cdegit @jnurse sure thing!

Tested this on Mac OS Chrome, Safari and Firefox browsers --- looks good. Works fine on my iPhone and if it's passed QA on a couple projects I'd say it gets a :+1:
@jnurse do you want to add a new release after merge? Are you familiar with the process?

@fractaltheory I'm not really familiar with the process. Also it's @cdegit's PR so she might want to add the new release.

:boom:

Thanks y'all :)


Profanities

Currently, alex mostly checks for words which may or may not be deemed “inconsiderate” in certain contexts. I added racial terms a month ago, which started moving away from possibly inconsiderate to clearly profane.
Now, I’m thinking (and others have requested this too) of adding other profanities.
The best point for this is that people (often not from the OSS community) expect alex to check these terms when they see the demo: “when something’s clearly insensitive, why does alex only warn about ‘he’” (paraphrased).
However, the downside of adding these aren’t used by people writing “real” text.
In fact, having these terms in the module discourages people from both using and working (source: private e-mail) on alex.
I’m not sure what to do; I’d like to hear your opinions :+1:

I think it should catch profanities, possibly under a flag to start with. If we want to warn about possibly insensitive writing slurs are part of it.

Thanks @yoshuawuyts!
Friendly ping: @sindresorhus, @pjf, @Charlotteis, would love your feedback on this :+1:

I mainly want this to make sure there's nothing insensitive in my source code written by other developers. Programmers are the more likely culprits than copywriters for this, and it's more easily obfuscated. (Related: #49)

Thanks @alexgleason :+1:

When you get a large project, contributed to by 10s-100s of developers you'll probably get something profane slipped in, maybe because they were frustrated at the time. See twitter.com/gitlost for the sheer amount developers will swear in git commit messages alone :smile:
Great point on the historical texts too.
I think @yoshuawuyts has the right idea with offering it under a flag, but I would also have no problem making it default behaviour. Maybe you could make it default but give a flag to turn off profanity warnings? Either way, at least it is in :)

Thanks for the points @Charlotteis! :ok_hand:

:+1: I think it's a good idea and should be on by default. Profanities are inconsiderate too. I want to ensure my docs to be profanity-free.

If profanity is a separate switch one can enable and disable, then I'm all for it. Features are great, and from the discussion thus far we'd have users that would find it useful.
 I definitely want alex to be able to find slurs and language that is offensive towards particular people, but neutral profanities are something I find much more acceptable, and a lot of people use them quite expressively. I hope to find people saying things are "ducking awesome" in my projects. :)
I'd be weakly inclined to have the option for alex to check for them off by default, as I feel they dilute alex's core purpose, which for me is to flag often unintentionally insensitive language. Profanities are almost always intentional, and neutral profanities especially so.

Okay, sounds good. Thanks all for the feedback!
I’ll work on creating retext-profanity (probably based on wooorm/profanities), externalised because it’s quite different from retext-equality.
Then, I’ll add support for a new flag to the CLI (-p, --profanities?) and an option to the API, to include profanities.

--profanities is good because it's explicit. But you could alias -p. Use --profanities in the docs, but let people know they can use -p too? :)

@Charlotteis Yeah I was indeed thinking of using both! :wink:


Image: 图片替换为 picture 标签 支持传入 source

Please make sure these boxes are checked before submitting your PR, thank you!

[ ] Make sure you follow Element's contributing guide (中文 | English | Español | Français).
[ ] Make sure you are merging your commits to dev branch.
[ ] Add some descriptions and refer relative issues for you PR.

I think it is not a good idea?