importing ray raise a AttributeError on sys.stdout()

Hello,
I'm trying to implement a simple generation of netflow from ugr16 dataset. I'm on linux and python3.6
My config.json is
{
    "global_config": {
        "original_data_file": "../traces/ugr16/raw.csv",
        "dataset_type": "netflow",
        "n_chunks": 10,
        "dp": false,
        "max_flow_len": false #to avoid the raising of an attribute error in netshare_pre_post_processor.py
    },
    "default": "netflow.json"
}

My driver.py is

from netshare import Generator
if __name__ == '__main__':
    # configuration file
    generator = Generator(config="netflow/config_example_netflow_nodp.json")

    # `work_folder` should not exist o/w an overwrite error will be thrown.
    # Please set the `worker_folder` as *absolute path*
    # if you are using Ray with multi-machine setup
    # since Ray has bugs when dealing with relative paths.

    #generator.visualize(work_folder='../results/vis_test')
    generator.train_and_generate(work_folder='../results/vis_test/')

After building all the chunk, the program starts working on the first chunk and raise an Attribute error linked to ray (wich is not use in the program)
The error Traceback is

Traceback (most recent call last):
  File "/srv/tempdd/aschoen/NetShare/netshare/pre_post_processors/pre_post_processor.py", line 29, in pre_process
    log_folder=log_folder)
  File "/srv/tempdd/aschoen/NetShare/netshare/pre_post_processors/netshare/netshare_pre_post_processor.py", line 406, in _pre_process
    flowkeys_chunkidx=flowkeys_chunkidx,
  File "/srv/tempdd/aschoen/NetShare/netshare/ray/remote.py", line 25, in remote
    import ray
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/__init__.py", line 169, in <module>
    from ray import autoscaler  # noqa:E402
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/__init__.py", line 1, in <module>
    from ray.autoscaler import sdk
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/sdk/__init__.py", line 1, in <module>
    from ray.autoscaler.sdk.sdk import (
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/sdk/sdk.py", line 9, in <module>
    from ray.autoscaler._private import commands
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/_private/commands.py", line 24, in <module>
    from ray.autoscaler._private import subprocess_output_util as cmd_output_util
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/_private/subprocess_output_util.py", line 8, in <module>
    from ray.autoscaler._private.cli_logger import cf, cli_logger
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/_private/cli_logger.py", line 61, in <module>
    import colorful as _cf
  File "/udd/aschoen/.conda/envs/NetShare-env/lib/python3.6/site-packages/colorful/__init__.py", line 133, in <module>
    sys.modules[__name__] = ColorfulModule(Colorful(), __name__)
  File "/udd/aschoen/.conda/envs/NetShare-env/lib/python3.6/site-packages/colorful/core.py", line 342, in __init__
    colormode = terminal.detect_color_support(env=os.environ)
  File "/udd/aschoen/.conda/envs/NetShare-env/lib/python3.6/site-packages/colorful/terminal.py", line 48, in detect_color_support
    if not sys.stdout.isatty():

I think it is the import of ray in the RemoteFunctionWrapper of remote.py which is causing this fault. Is there a way to ignore this wrapper in your code by giving a specific argument to the json file ?
If you have any advice, I would really appreciate.
Good luck for your big merge ;)
Adrien

Dear Adrien,
Thank you for opening this issue.
Blaming ray is interesting.
Can you please try to change the configuration to disable ray (it's in the driver.py file, something like ray.config.enabled = False) and check if the error still happens?
Moreover, can you please attach the entire error message?
Saar

Hello,
This is the global stdout :
(NetShare-env) aschoen@igrida14-01:/srv/tempdd/aschoen/NetShare/examples$ python3 driver.py
NetsharePrePostProcessor._pre_process
../traces/ugr16/raw.csv
Training word2vec model from scratch...
Training Word2Vec model from scratch...
Word2Vec model is saved at ../results/vis_test/pre_processed_data/word2vec_vecSize_10.model
Using fixed_time
Chunk_id: 0, # of pkts/records: 659
Chunk_id: 1, # of pkts/records: 1092
Chunk_id: 2, # of pkts/records: 102214
Chunk_id: 3, # of pkts/records: 125351
Chunk_id: 4, # of pkts/records: 127242
Chunk_id: 5, # of pkts/records: 131047
Chunk_id: 6, # of pkts/records: 133070
Chunk_id: 7, # of pkts/records: 130395
Chunk_id: 8, # of pkts/records: 123788
Chunk_id: 9, # of pkts/records: 125142
df_chunk_cnt_validation: 1000000
Chunk time: 134.2636 seconds
compute flowkey-chunk list from scratch...
processing chunk 1/10, # of flows: 653
processing chunk 2/10, # of flows: 1085
processing chunk 3/10, # of flows: 89308
processing chunk 4/10, # of flows: 109210
processing chunk 5/10, # of flows: 109674
processing chunk 6/10, # of flows: 114892
processing chunk 7/10, # of flows: 116132
processing chunk 8/10, # of flows: 113368
processing chunk 9/10, # of flows: 107063
processing chunk 10/10, # of flows: 108010
# of total flows: 793801
# of total flows (sanity check): 793801
# of flows cross chunk (of total flows): 48169 (6.068145542774574%)
# of non-continuous flows: 5153
chunk_id: 0, max_flow_len: 2
chunk_id: 1, max_flow_len: 3
chunk_id: 2, max_flow_len: 30
chunk_id: 3, max_flow_len: 16
chunk_id: 4, max_flow_len: 26
chunk_id: 5, max_flow_len: 24
chunk_id: 6, max_flow_len: 16
chunk_id: 7, max_flow_len: 12
chunk_id: 8, max_flow_len: 15
chunk_id: 9, max_flow_len: 34
global max flow len: 34
Top 10 per-chunk flow length: [18, 19, 24, 25, 25, 26, 26, 27, 30, 34]
0it [00:00, ?it/s]
Chunk_id: 0
0it [00:02, ?it/s]
Traceback (most recent call last):
  File "/srv/tempdd/aschoen/NetShare/netshare/pre_post_processors/pre_post_processor.py", line 29, in pre_process
    log_folder=log_folder)
  File "/srv/tempdd/aschoen/NetShare/netshare/pre_post_processors/netshare/netshare_pre_post_processor.py", line 406, in _pre_process
    flowkeys_chunkidx=flowkeys_chunkidx,
  File "/srv/tempdd/aschoen/NetShare/netshare/ray/remote.py", line 25, in remote
    import ray
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/__init__.py", line 169, in <module>
    from ray import autoscaler  # noqa:E402
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/__init__.py", line 1, in <module>
    from ray.autoscaler import sdk
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/sdk/__init__.py", line 1, in <module>
    from ray.autoscaler.sdk.sdk import (
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/sdk/sdk.py", line 9, in <module>
    from ray.autoscaler._private import commands
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/_private/commands.py", line 24, in <module>
    from ray.autoscaler._private import subprocess_output_util as cmd_output_util
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/_private/subprocess_output_util.py", line 8, in <module>
    from ray.autoscaler._private.cli_logger import cf, cli_logger
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/_private/cli_logger.py", line 61, in <module>
    import colorful as _cf
  File "/udd/aschoen/.conda/envs/NetShare-env/lib/python3.6/site-packages/colorful/__init__.py", line 133, in <module>
    sys.modules[__name__] = ColorfulModule(Colorful(), __name__)
  File "/udd/aschoen/.conda/envs/NetShare-env/lib/python3.6/site-packages/colorful/core.py", line 342, in __init__
    colormode = terminal.detect_color_support(env=os.environ)
  File "/udd/aschoen/.conda/envs/NetShare-env/lib/python3.6/site-packages/colorful/terminal.py", line 48, in detect_color_support
    if not sys.stdout.isatty():
AttributeError: 'DuplicateWriter' object has no attribute 'isatty'
Traceback (most recent call last):
  File "example_netflow.py", line 13, in <module>
    generator.train_and_generate(work_folder='../results/vis_test/')
  File "/srv/tempdd/aschoen/NetShare/netshare/generators/generator.py", line 203, in train_and_generate
    if not self.train(work_folder):
  File "/srv/tempdd/aschoen/NetShare/netshare/generators/generator.py", line 189, in train
    work_folder)):
  File "/srv/tempdd/aschoen/NetShare/netshare/generators/generator.py", line 100, in _pre_process
    log_folder=log_folder)
  File "/srv/tempdd/aschoen/NetShare/netshare/pre_post_processors/pre_post_processor.py", line 29, in pre_process
    log_folder=log_folder)
  File "/srv/tempdd/aschoen/NetShare/netshare/pre_post_processors/netshare/netshare_pre_post_processor.py", line 406, in _pre_process
    flowkeys_chunkidx=flowkeys_chunkidx,
  File "/srv/tempdd/aschoen/NetShare/netshare/ray/remote.py", line 25, in remote
    import ray
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/__init__.py", line 169, in <module>
    from ray import autoscaler  # noqa:E402
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/__init__.py", line 1, in <module>
    from ray.autoscaler import sdk
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/sdk/__init__.py", line 1, in <module>
    from ray.autoscaler.sdk.sdk import (
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/sdk/sdk.py", line 9, in <module>
    from ray.autoscaler._private import commands
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/_private/commands.py", line 24, in <module>
    from ray.autoscaler._private import subprocess_output_util as cmd_output_util
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/_private/subprocess_output_util.py", line 8, in <module>
    from ray.autoscaler._private.cli_logger import cf, cli_logger
  File "/udd/aschoen/.local/lib/python3.6/site-packages/ray/autoscaler/_private/cli_logger.py", line 61, in <module>
    import colorful as _cf
  File "/udd/aschoen/.conda/envs/NetShare-env/lib/python3.6/site-packages/colorful/__init__.py", line 133, in <module>
    sys.modules[__name__] = ColorfulModule(Colorful(), __name__)
  File "/udd/aschoen/.conda/envs/NetShare-env/lib/python3.6/site-packages/colorful/core.py", line 342, in __init__
    colormode = terminal.detect_color_support(env=os.environ)
  File "/udd/aschoen/.conda/envs/NetShare-env/lib/python3.6/site-packages/colorful/terminal.py", line 48, in detect_color_support
    if not sys.stdout.isatty():
AttributeError: 'DuplicateWriter' object has no attribute 'isatty'

Indeed, when indicating ray.config.enabled = False, I have the training execute correctly.
Thank you very much for your quick answer, and good luck for the future. You can close the issue.

Thank you very much, glad to hear that!


Adding PDB to es master and data

With this PDB, only one pod can be unavailable (but not if we request
manually a pod delete)

I'm not sure this is a good idea because it means that if a data node is rescheduling, the Masters can't move during that operation. This is not linked, and do not have sense to hard link them. What do you think ?

Makes sense. Can you add instructions to the README.md?

Hi,
Can you tell me what is blocking to get merged ?
Thanks

Thank you!


Is it posssible to make the tcp idle timeout and and TCP Reset configurable for the Ingress?

This issue is a: (mark with an x)

[x] bug report -> please search issues before submitting
[ ] documentation issue or request
[ ] regression (a behavior that used to work and stopped in a new release)

Issue description
We have Spring Boot application and deployed as ACA Apps. On the client side, we just call the private dns name of the ACA App and have exceptions probably due to the fact that the connections is closed on the server side (ACA app) without notice. Similar to this issue https://github.com/microsoft/azure-container-apps/issues/597, is it possible to make the idle timeout and TCP Reset configurable for the ingress? This is already possible in Azure Load Balancer. See https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-tcp-reset

@cachai2

Thank you for the feedback @pengxo. This isn't possible today, but we'll take idle timeouts and TCP reset configurations as feature requests and evaluate if we can get these added to the ingress proxy for ACA.

@cachai2 Thanks for your reply! I notice that service resiliency policy is currently in preview for ACA (https://learn.microsoft.com/en-us/azure/container-apps/service-discovery-resiliency?tabs=bicep). Could you tell me when the feature will be GA for ACA?

@cachai2 Thanks for your reply! I notice that service resiliency policy is currently in preview for ACA (https://learn.microsoft.com/en-us/azure/container-apps/service-discovery-resiliency?tabs=bicep). Could you tell me when the feature will be GA for ACA?

This works for me!


HomeBridge roborock s6 log errors after update to latest plugin version

It worked fine some time. Today I want to update some plugins and when I was checking log I found these errors for my roborock s6 after update to latest plugin version. Settings did not changed.
For now Im back on this version homebridge-xiaomi-roborock-vacuum v0.26.3 and it looks like its working again.
`
[16/10/2022, 14:31:22] [Vysavač] [Model=unknown] getSerialNumber | No vacuum cleaner is discovered yet.
[16/10/2022, 14:31:22] [Vysavač] [Model=unknown] getFirmware | No vacuum cleaner is discovered yet.
[16/10/2022, 14:31:22] [Vysavač] [Model=unknown] getCleaning | Failed getting the cleaning status. Error: Not connected yet
at DeviceManager.get device [as device] (/usr/lib/node_modules/homebridge-xiaomi-roborock-vacuum/src/services/device_manager.ts:79:13)
at DeviceManager.property (/usr/lib/node_modules/homebridge-xiaomi-roborock-vacuum/src/services/device_manager.ts:85:17)
at DeviceManager.get state [as state] (/usr/lib/node_modules/homebridge-xiaomi-roborock-vacuum/src/services/device_manager.ts:66:17)
at DeviceManager.get isCleaning [as isCleaning] (/usr/lib/node_modules/homebridge-xiaomi-roborock-vacuum/src/services/device_manager.ts:70:43)
at MainService.getCleaning (/usr/lib/node_modules/homebridge-xiaomi-roborock-vacuum/src/services/main_service.ts:108:45)
at On.getHandler (/usr/lib/node_modules/homebridge-xiaomi-roborock-vacuum/src/services/main_service.ts:38:25)
at On. (/usr/lib/node_modules/homebridge/node_modules/hap-nodejs/src/lib/Characteristic.ts:1528:32)
at step (/usr/lib/node_modules/homebridge/node_modules/tslib/tslib.js:144:27)
at Object.next (/usr/lib/node_modules/homebridge/node_modules/tslib/tslib.js:125:57)
at /usr/lib/node_modules/homebridge/node_modules/tslib/tslib.js:118:75
[16/10/2022, 14:31:22] [Vysavač] [Model=unknown] getSpeed | No vacuum cleaner is discovered yet.
[16/10/2022, 14:31:22] [Phoscon deConz] Philips Hue Go: set homekit supported transition configuration to {"configuration":[{"iid":11,"characteristic":1},{"iid":13,"characteristic":2}]}
[16/10/2022, 14:31:22] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'On': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
[16/10/2022, 14:31:22] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'Status Low Battery': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
[16/10/2022, 14:31:22] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'Battery Level': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
[16/10/2022, 14:31:22] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'Charging State': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
[16/10/2022, 14:31:22] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'Filter Change Indication': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
[16/10/2022, 14:31:22] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'Filter Life Level': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
[16/10/2022, 14:31:22] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'Filter Change Indication': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
[16/10/2022, 14:31:22] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'Filter Life Level': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
[16/10/2022, 14:31:22] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'Filter Change Indication': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
[16/10/2022, 14:31:22] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'Filter Life Level': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
[16/10/2022, 14:31:22] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'Filter Change Indication': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
[16/10/2022, 14:31:22] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'Filter Life Level': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
[16/10/2022, 14:31:23] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'Filter Change Indication': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
[16/10/2022, 14:31:23] [homebridge-xiaomi-roborock-vacuum] This plugin threw an error from the characteristic 'Filter Life Level': Unhandled error thrown inside read handler for characteristic: Not connected yet. See https://homebridge.io/w/JtMGR for more info.
`

@nseries100 those logs show that the plugin did not manage to connect to the vacuum yet. But it keeps trying. The same applies to older versions.
Did you get any sort of crash or anything that made you roll back to the previous version?

I rolled back to older version because I want to try if it will be work again. And It just works. Older version = no errors in log.

I am having the same problem. It worked when initially setup but now does not. I have a static IP for my roborock s6 and the token is definitely correct. Rolling back to 0.26.3 did not work for me.

FWIW, the reconnection logic has not changed in recent versions.
The described behavior seems familiar as it has randomly happened in the past. The plugin failed to connect on the first attempt and was not able to connect on the retries. Typically, restarting the plugin (sometimes a couple of times would fix it).
Probably that's why @nseries100 saw this resolved after rolling back. But @dehlen's scenario confirms it's not a regression of the latest version.
@nseries100 would you mind trying to upgrade again and confirm if you still see that issue?


jpegoptim: apply patch for CVE-2023-27781

Description of changes
See https://github.com/tjko/jpegoptim/issues/132

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review run on x86_64-linux 1

7 packages built:

akkoma-frontends.akkoma-fe
curtail
discourse
discourseAllPlugins
image_optim
jpegoptim
sacad

Result of nixpkgs-review pr 221829 run on x86_64-linux 1

7 packages built:

akkoma-frontends.akkoma-fe
curtail
discourse
discourseAllPlugins
image_optim
jpegoptim
sacad


Checkpoint for ensemble-based methods

Hi,
The paper has results for ensemble-based methods like MCDropout and DeepEnsemble. These methods require re-training the classifier. However, the released checkpoints do not contain these checkpoints. Could you please provide checkpoints for ensemble-based methods?
By the way, does scripts/uncertainty/ensemble indicate DeepEnsemble? If so, how to train the model used for testing good on cifar10?
Thank you.

I don't think we provide (or plan to provide) checkpoints for ensemble-based method. They should be pretty easy and fast to train especially on small datasets like CIFAR.
Correct me if I'm wrong, but I think DeepEnsemble just ensembles multiple independently trained models? Therefore you can just use the single-model training script to train several models and ensemble them for testing.


Implement the application tab in the ? pop-up

resolve #151

Thanks for the contribution!
I tested this branch and two things that stand out to me:

In the official game it shows a red border around a GameObject if the requirements are currently not met. However it doesn't seem to work correctly for me. Below an example, I cannot build Terraformer as shown by the facilities page, but it shows as a green border in the popup. I would expect the terraformer to be greyed out and have a red border.

Shipyard "?" popup does not show interplanetary missiles as requirement. In the official game the shipyard includes the interplanetary missiles as a requirement. But it's missing from the list for me. I'm not sure if this is a bug in the code or if a requirement is missing somewhere. I already noticed I made a few mistakes with the initial entry of the game objects, so might be my mistake as well. 😸

Oooh good catch, i fix before to have the green border, but maybe i break something, i will check again :)

some defense and station had the wrong requirement so i fix it, and i rework the method to check the requirement.

Nice, looks good. Just tested it and as far as I can see it works beautifully now. Awesome! 🏆


Write basic GH Action for tests

This is just the beginning.
There is scope for other actions: linters, formatters, other tests (notebooks and
documentation).

I think the way to approach this is to make separate branches (and eventually PRs) from here for each of the following:

flake8
isort
black
interrogate

Do we then wrap everything up in tox?

Hmmm... there may be clashes between the formatters so let's just do it in order but on this branch.


[Bug] [dolphinscheduler-master]  dolphinscheduler is unable to stop the workflow instance

Search before asking

[x] I had searched in the issues and found no similar issues.

What happened
https://github.com/apache/dolphinscheduler/issues/14222
https://github.com/apache/dolphinscheduler/issues/15503
我在dolphinscheduler中配置了seatunnel任务，并运行了实例，但我无法通过dolphinscheduler停止对应的seatunnel任务进程。尽管dolphinscheduler中显示任务状态为Kill，但服务器上的seatunnel任务还在运行。
这个问题非常严重，希望能尽快解决。

I configured the seatunnel task in dolphinscheduler and ran the instance, but I was unable to stop the seatunnel task process using dolphinscheduler. The seatunnel task is running on the server despite the task status being shown as Kill in dolphinscheduler.
This problem is very serious and I hope it can be solved soon.
What you expected to happen
pass
How to reproduce
pass
Anything else
pass
Version
3.2.x
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

@ruanwenjun @zhongjiajie please see

以下是我的测试代码 Demo.java
Here's my test code demo.java
import java.io.File;
import java.io.IOException;
import java.lang.reflect.Field;

public class Demo {
    public static void main(String[] args) throws IOException {
        ProcessBuilder processBuilder = new ProcessBuilder();
        processBuilder.directory(new File("/opt/test"));
        processBuilder.redirectErrorStream(true);
        processBuilder.command(args);
        Process p = processBuilder.start();
        int processId = getProcessId(p);
        System.out.println("pid:"+processId);
        System.out.println("isAlive:"+p.isAlive());
    }
    private static int getProcessId(Process process) {
        int processId = 0;

        try {
            Field f = process.getClass().getDeclaredField("pid");
            f.setAccessible(true);

            processId = f.getInt(process);
        } catch (Exception e) {
            System.out.println("Get task pid failed"+e.getMessage());
        }

        return processId;
    }
}

先编译java文件，并创建一个sh脚本文件
Compile the Java file first and create an sh script file
javac Demo.java
echo "java -jar server-0.0.1-SNAPSHOT.jar" > server-start.sh

执行sh脚本启动java程序，则获取到的pid是错误的。
Executing the sh script starts the Java program, and the pid is wrong.
java Demo sh server-start.sh
pid:35054
isAlive:true

jps
35080 Jps
35055 jar

直接执行java命令启动java程序，则能获取到正确的pid。
The direct execution of the Java command starts the Java program and gets the correct pid.
kill -9 35055
java Demo java -jar server-0.0.1-SNAPSHOT.jar
pid:35641
isAlive:true

jps
35669 Jps
35641 jar

以下是运行环境
The following is the operating environment
java -version
java version "1.8.0_261"
Java(TM) SE Runtime Environment (build 1.8.0_261-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.261-b12, mixed mode)

uname -a
Linux urc-esb-server2 3.10.0-514.el7.x86_64 #1 SMP Wed Oct 19 11:24:13 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux

我看到 #14652 解决了这个问题。
但我依然出现了异常，原因是我的Linux没有安装pstree。
将pstree安装之后问题得到解决。
I saw #14652 to solve the problem.
But I still get an exception, because my Linux doesn't have pstree installed.
Once pstree is installed, the issue is resolved.

这好像只是local模式的处理，cluster模式下，worker上只是client，杀进程并不能停止掉集群上的任务


Update the tracker used with batch execution

is that correct that you execute the circuit once for evaluating it and twice for the parameter shift rules?

Yep. :slightly_smiling_face:


MTR / tracepath don't work

It would be super helpful for Ops engineers if you could make mtr and tracepath tools work.
Running as 'admin' with 'root privileges:
root@XXXX:~# mtr <IP_ADDRESS>
mtr: unable to get raw sockets
root@XXXX:~# tracepath <IP_ADDRESS>
IP_MTU_DISCOVER: Invalid argument
Windows Build number: 1607

@amoiseiev - Thanks for trying out WSL and the feedback. Yes, we are aware that these tools currently don't work on WSL and we are looking into what we can enable. Meanwhile, please also help us prioritize by upvoting (or adding a new task) in the User Voice Page. It will be helpful if you can add specific comments about what you are looking for.

Got the same issue.

Got the same issue.

@netroby, @Cercaj - thanks for the feedback. As suggested previously, please head out to the user voice page and open a new ticket (upvote, if already exists) for mtr/tracepath support. We use the user voice page to prioritize future work.

its not fixed in latest bash build

@gofirst2220 - Thanks for the (re)post. Yes, we are aware that both mtr and 'tracepath` do not currently work on WSL and we are looking to enable them in WSL in future Insider builds. It requires some support in the native Windows networking stack and we are working with the respective team to enable those options. Stay tuned!

Would be awesome to get more networking stuff usable within WSL. So many ideas...

Still not working on build 1709 (16299.248)
But ifconfig, ping, nslookup, dig works fine.

Tried sudo (which is sometimes an issue on Linux boxes) - same issue.

Same issue here... it would be definitely handy for Ops engineers for this to work.

Same issue +Integer.MAX_VALUE

I am having this issue as well.

The feature to make this work (AF_PACKET) is now being tracked in #1349, so this issue might as well be closed as a duplicate of that.

This is sort of the landing zone for IP_MTU_DISCOVER, not AF_PACKET. The OP reported two things which is most unfortunate.

Sadly, until these issues - lack of ability to mtr/traceroute/nmap (to name but a few) - are resolved, there's really no point using WSL (for me) at all.
I guess this stuff is just too early days. Appreciate the efforts, will keep an eye and maybe look at using it again when this stuff is part of it.

It appears that then running Bash for windows normally Permission is denied from opening IPv4 / IPv6 Sockets even with sudo / su. Running Bash on Ubuntu on Windows as Administrator (Right click Ubuntu Icon ->More-> Run as Administrator) and MTR works fine.

In my case, MTR cannot display any ip(bash.exe run as admin). And traceroute does't work.

@wobblybobz what build of Windows 10? Under 1803, that takes care of the error messages but mtr from the mtr package does not display any pings.

hrrm very strange. I have Windows 10 1803 Build 17134.165 with Ubuntu 18.04
I right click on the Ubuntu Icon, Click More then Run as Administrator.
in the window i type "mtr hostname" and i get.
                                             My traceroute  [v0.92]

MYHOSTNAME (<IP_ADDRESS>)                                                                     2018-07-31T15:23:00+0800
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
Packets               Pings
Host                                                                        Loss%   Snt   Last   Avg  Best  Wrst StDev

<IP_ADDRESS>                                                            0.0%     8    1.3   1.3   1.0   1.4   0.1
<IP_ADDRESS>                                                              0.0%     8    3.6   4.6   3.6   9.7   2.1
<IP_ADDRESS>                                                            0.0%     7    6.8   7.7   6.8  11.2   1.6

So it appears to be working correctly for me.

I can use mtr with IPv4 if a run WSL with admin and run mtr with root, but I still can't mtr a IPv6 address.

@Menci Hi. For me, it shows nothing when tracing with IPv4 even though running with admin permission in Windows and root permission in WSL as long as the Windows Firewall is active. But non-privileged Windows-native program like tracert / winmtr do work. When you make it work with IPv4, did you disable Windows Firewall?

"Run as Admin" can be circumvented by avoiding setting IP_HDRINCL socket option, here is a tiny patch that works for me: https://github.com/Gowee/mtr/commit/c0e347cc8d7244fea01c89a2473b27b6b25a422c .
Note: To test, MTR_PACKET environment should be the path to mtr-packet, otherwise mtr will use the one given in PATH by default.
Root permission is not necessary as long as CAP_NET_RAW is activated for the program, that is:
# setcap cap_net_raw+ep ./pathto/mtr-packet && setcap cap_net_raw+ep ./pathto/mtr .
This cap should have been set by package managers on most distros when installing mtr.
But disabling Firewall is inevitable. The problem seems to be ICMP errors are not delivered to mtr properly (~a test: ping <IP_ADDRESS> -t 1 should have told TTL exceeded error, but it shows nothing instead~ very confusing: even with Firewall disabled, ping still does not tell TTL exceeded error).

really strange
bzanaj@leaf:~$ mtr <IP_ADDRESS>
Failure to open IPv4 sockets: Operation not permitted
Failure to open IPv6 sockets: Operation not permitted
mtr: Failure to start mtr-packet: Invalid argument
bzanaj@leaf:~$ ping <IP_ADDRESS> -c2
PING <IP_ADDRESS> (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=121 time=17.9 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=121 time=32.3 ms

--- <IP_ADDRESS> ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 17.994/25.177/32.360/7.183 ms

Just installed WSL with Kali Linux Distro,
Running as admin permission on Windows and Sudo MTR but this still doesnt work.
Windows version 10.0.17134 Build 17134

Same thing here,
Windows 10.0.18362 on Thinkpad x230;
Alpine Linux.

Windows 10.0.18363.535 Ubuntu still doesn't work.

I see you've added a label fixed-in-wsl2, but my converted ubuntu image still has this problem. Are there some additional steps needed?

still has this problem

If cat /proc/version returns "Linux version 4.19-microsoft-standard", and simultaneously exhibits this problem ("mtr <IP_ADDRESS> mtr: unable to get raw sockets"), that would be difficult to explain. There are no additional steps.

yes, that is the case

Appreciate the screencap, thanks. So... not this problem. You are getting an EPERM, which I am not able to reproduce. The OP couldn't get a raw socket (because WSL1 doesn't have 'em).
Spin up another issue, and include as many details about your Windows networking config (notably third party stuff). Also, for the data point, whether tracert.exe <IP_ADDRESS> succeeds from cmd.exe (as the same user that started WSL above).

So to be clear, if using WSL1 then mtr, etc. you are out of luck because it does not have and never will have raw sockets so use WSL2 if you want that functionality? That's what I think @therealkenc said.

at this point this issue can be closed as not fixable

at this point this issue can be closed as not fixable
you're wrong.
mtr works perfect if bash run 'Run as administrator".

It "works", but doesn't seem to get past localhost and you need to follow the following steps:

Run Terminal Preview or WSL as Administrator
sudo mtr:

3. sudo mtr google.ca:

For reference, I am running the latest Insider build:

I can confirm, that mtr at least opens in admin mode, but does not send any requests properly:
16:33:15 gettimeofday({tv_sec=1587479595, tv_usec=719169}, NULL) = 0
16:33:15 write(4, "33007 send-probe ip-4 <IP_ADDRESS> local-ip-4 <IP_ADDRESS> protocol icmp size 64 bit-pattern 0 tos 0 ttl 8 timeout 10\n", 113) = 113
16:33:15 select(8, [0 5 7], NULL, NULL, {tv_sec=0, tv_usec=100000}) = 0 (Timeout)
16:33:15 time(NULL)                     =<PHONE_NUMBER> (2020-04-21T16:33:15+0200)
16:33:15 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2335, ...}) = 0
16:33:15 rt_sigaction(SIGTSTP, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f5706e61730}, {sa_handler=0x7f5707058940, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f5706e61730}, 8) = 0
16:33:15 poll([{fd=0, events=POLLIN}], 1, 0) = 0 (Timeout)
16:33:15 poll([{fd=0, events=POLLIN}], 1, 0) = 0 (Timeout)
16:33:15 rt_sigaction(SIGTSTP, {sa_handler=0x7f5707058940, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f5706e61730}, NULL, 8) = 0
16:33:15 gettimeofday({tv_sec=1587479595, tv_usec=824349}, NULL) = 0
16:33:15 write(4, "33008 send-probe ip-4 <IP_ADDRESS> local-ip-4 <IP_ADDRESS> protocol icmp size 64 bit-pattern 0 tos 0 ttl 9 timeout 10\n", 113) = 113
16:33:15 select(8, [0 5 7], NULL, NULL, {tv_sec=0, tv_usec=100000}) = 0 (Timeout)
16:33:15 time(NULL)                     =<PHONE_NUMBER> (2020-04-21T16:33:15+0200)
16:33:15 stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2335, ...}) = 0
16:33:15 rt_sigaction(SIGTSTP, {sa_handler=SIG_IGN, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f5706e61730}, {sa_handler=0x7f5707058940, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7f5706e61730}, 8) = 0
1```

It looks as if mtr is not able to get a proper network socket from the related WSL "kernel", also running the newest build.

When trying to dig deeper (strace -f) it totally not works, looks like Microsoft is isolating the WSL to much from the native linux kernel and functionality.

@therealkenc
I request reopenning this issue as the issue reported by @Fabian123333 happened here too. WSL2 is not a option for me because NAT-IPv6. WSL1 cannot open the socket properly and this is a issue to be fixed.
Thanks

Just reinforcing, this issue cannot be considered fixed-in-wsl2 because wsl2 introduces a totally different network stack and its not suitable by any terms while wsl2 at least support bridge mode + ipv6 without any kind of nat. MTR while used to detect hop issues needs raw sockets since we use all 3 protocols to check connectivity issues and even ecmp and other kind of network propriety.

upvote to get working hping

Encountered this just now. mtr still not working. Here's my kernel version.
cat /proc/version
Linux version 4.4.0-18362-Microsoft<EMAIL_ADDRESS>(gcc version 5.4.0 (GCC) ) #1049-Microsoft Thu Aug 14 12:01:00 PST 2020
You can't close an issue by creating a different network stack.  That's not how it works ;)

Just reinforcing, this issue cannot be considered fixed-in-wsl2 because wsl2 introduces a totally different network stack and its not suitable by any terms while wsl2 at least support bridge mode + ipv6 without any kind of nat. MTR while used to detect hop issues needs raw sockets since we use all 3 protocols to check connectivity issues and even ecmp and other kind of network propriety.

linux is network system, it should as base support network tools.
i think ms do not need to develop linux stack, just wrapping network stack to linux network libs

tracert in powershell:
Tracing route to dns.google [<IP_ADDRESS>]
over a maximum of 30 hops:

  1     2 ms     1 ms     2 ms  <IP_ADDRESS>
  2    10 ms    10 ms     7 ms  cr-dui1-pwether10748.x-win.dfn.de [<IP_ADDRESS>]
  3    14 ms    13 ms    13 ms  cr-fra2-be16.x-win.dfn.de [<IP_ADDRESS>]
  4    11 ms    11 ms    14 ms  <IP_ADDRESS>
  5    17 ms    13 ms    14 ms  <IP_ADDRESS>
  6    19 ms    12 ms    11 ms

tracepath in wsl2 after resetting network, flushing dns and so on:
 1?: [LOCALHOST]                      pmtu 1500                                                                          1:  no reply                                                                                2:  ???                                                   3.669ms                                                       3:  ???                                                   9.725ms                                                       4:  ???                                                  51.344ms                                                       5:  no reply                                                                                6:  no reply                                                                                7:  no reply                                                                                8:  no reply                                                                                9:  no reply                                                                                10:  no reply                                                                                11:  no reply                                                                                12:  no reply                                                                                13:  no reply                                                                                14:  no reply                                                                                15:  no reply                                                                                16:  no reply                                                                                17:  no reply                                                                                18:  no reply                                                                                19:  no reply                                                                                20:  no reply                                                                                21:  no reply                                                                                22:  no reply                                                                                23:  no reply                                                                                24:  no reply                                                                                25:  no reply                                                                                26:  no reply                                                                                27:  no reply                                                                                28:  no reply                                                                                29:  no reply                                                                                30:  no reply                                                                                Too many hops: pmtu 1500                                                                                Resume: pmtu 1500

TLDR, WSL needs AF_PACKET - go here:
https://github.com/microsoft/WSL/issues/1349#issuecomment-440935043

There are two different permissions issues, and only one of them seems to have a workaround.
The first is that mtr in general doesn't run under wsl. The solution there is to start wsl w/ admin permissions (sudo wsl.exe in Windows) then run mtr, optionally w/ elevation ('sudo mtr <IP_ADDRESS>` now works)
The second issue is that even when run w/ elevated Windows and Linux privileges (as above), MTR still cannot get the permissions it needs if the target is an IPv6 host (sudo mtr 1:2:3::4 exits with "mtr: Permission denied").


Output descriptions updated

The description of output updated to fit actual output. Not really sure, why there are two different output formats.
Are they wrong in modbus-getter too?
What does PDU mean?

#299 for introduction / issue

PDU is a known term if you are familiar with the Modbus protocol and I don't want to remove it. I'll check if I could take some of your suggestion in code.

Closing it just like that?? I described the output as it is, so why not include my contribution?
The data arrays output are not the PDU. PDU is the request code and associated data. So that is not correct either.

I'm sorry, it is not personally, but it is the "data Array of the PDU" - I see it different and it was since 2016 no problem for the users. From that perspective I don't see a point to change it.

And I wrote: " I'll check if I could take some of your suggestion in code."

Well, you don't know if it has been a problem to users. Most people probably just figure it out like I did by looking at the data and everyone using modbus can figure that out too. So if you insist on not being open to contributions from users to improve the documentation and believe all users should waste time on wrong documentation, then be my guest.

As the list of pull requests shows, we are very much open to pure documentation improvements. Nevertheless, the group of developers decides whether it is an improvement we want to adopt. There are always several people deciding about a PR and we appreciate the work of the community very much. And again: " I'll check if I could take some of your suggestion in code."

The output descriptions are still wrong in all the getters.


chore: run tests on all supported versions of Node.js

This PR fixes the unit tests on previous versions of Node.js (< 18). These tests are currently no longer working because of a breaking change in a dependency.
Unit tests in this package use esmock as a dev dependency.

esmock < 2.4 works only on Node.js < 19.
esmock >= 2.4 works only on Node.js >= 18.6.

(These is not officially specified information, I determined it by testing)
In order to run unit tests on all versions of Node.js that we need to support, I have created a postinstall script that checks the version of Node.js and replaces the esmock folder in node_modules with a legacy release on Node.js < 19.

Thanks for the review @aladdin-add, I've added a commit to update the comments, including clarification about the 111. I'd be still open to switching to process.versions.node if that looks cleaner.


Fix bug: Allow passing a URL to --spec

#132 broke the ability to pass a URL to the --spec option. This PR adds a test and fixes the regression.

@tardypad would you mind reviewing this PR?

That looks fine :)
I indeed didn't notice that the spec file could be remote
(despite the line below being "normalize_url" :man_facepalming:)
Sorry about that, I don't use Swagger spec so I didn't test that part.


The LLVM man pages change is miserable

I'm trying to build a rust package right now on master and because sqlalchemy changed recently, I'm rebuilding sphinx, LLVM, rustc, and a few other things in between. Can we default man pages to being off until this gets handled properly?
What prompted this is that the sphinx build is failing for reasons I haven't figured out yet, so now I can't do my rust work...
Edit: I'm on Darwin, running against latest master at https://github.com/NixOS/nixpkgs/commit/5cbc6ca9bb63f939dab2c2c82a63d72f1f30d9b7
cc @dtzWill @LnL7 @vcunat

The man pages for LLVM 4 are not build on Darwin for the stdenv, but they are otherwise.
I would also prefer not building those man pages because there's a whole lot of tiny packages that may now cause a mass-rebuild.

so now I can't do my rust work...

Surely you can use a slightly older commit to work from.

Is it possible to build an output independent from the rest of the package?
llvm = stdenv.mkDerivation {
  man = stdenv.mkDerivation {
  };
}

Then a change to python would only rebuild the man pages, but not any other program using llvm.
Otherwise it could be a dedicated package: llvm-manpages

It would technically not be an output of the same derivation, but it is possible to add such attributes (though passthru to avoid the dependency).

Would it be pulled by buildEnv or systemPackages?

It won't work with extraOutputsToInstall, but you would be able to add clang.cc.man explicitly.

@edolstra would be kind of cool if we could pretend to be an output of a derivation so the existing machinery would work seamlessly with it (i.e., it would show up in outputs), even though it's a passthru attribute. Seems like it could be a cute Nix feature. Something like fakeOutputs = { man = mkDerivation ...; } and fakeOutputs would behave like passthru but show up in outputs.

Yeah that would be pretty nice.

Well, the outputs list is easy to overwrite.  I'm not sure if there's something else we would be missing in nixpkgs.


Update pip with master branch version

Or better - automate it with github actions (quite easy).

It is already automated with GH actions, but triggered only by tagging. Will update now.

Done. v0.0.5 available at https://pypi.org/project/sanskrit-tts/0.0.5/


Failing to authenticate on lockscreen

Please describe the issue in as much detail as possible, including any errors and traces
Setup: As the "streamer" package is not available in Debian testing I created a faux package using equivs which fulfills this dependency and then installed Howdy using the following command:
	HOWDY_NO_PROMPT=1 sudo -E dpkg -i howdy_2.5.1.deb

I then set the device_path variable and added a face model.
The installation was successful, and Howdy works correctly for sudo and executing programs which require elevated rights.
The problem occurs when attempting to unlock my device. I see my infrared cameras turn on for a moment, much shorter than when attempting to use sudo, and then the computer fails to unlock.
Running
	/usr/lib/x86_64-linux-gnu/libexec/kscreenlocker_greet --testing

Shows the following console messages:
	Data files have not been downloaded, please run the following commands:

	        cd /lib/security/howdy/dlib-data
	        sudo ./install.sh

	Traceback (most recent call last):
	  File "/lib/security/howdy/compare.py", line 219, in <module>
	    face_locations = face_detector(gsframe, 1)
	TypeError: 'NoneType' object is not callable
	Authentication failure

Running
	tail /var/log/auth.log

Shows
	kcheckpass[4363]: pam_unix(kde:auth): Couldn't open /etc/securetty: No such file or directory
	kcheckpass[4344]: pam_unix(kde:auth): authentication failure; logname=nicholas uid=1000 euid=1000 tty=/dev/pts/1 ruser= rhost=  user=nicholas

With the first line being repeated 2-6 times before authentication failure message.
I've run sudo ./install.sh in /lib/security/howdy/dlib-data, and it shows that the data files have been downloaded.

I've searched for similar issues already, and my issue has not been reported yet.
Linux distribution (if applicable): Netrunner (Debian Testing based distribution)
Howdy version: 2.5.1

The /etc/securetty issue should be fixed by touching an empty file there, but i don't think that's what's breaking Howdy.
The datafiles being there (while Howdy is warning that they aren't) might point to Howdy not having the right permissions to read them. If not, it could be that the PATH is wrong. You could try to print the PATH by adding a print statement at line 61 in compare.py.

Printing the path at line 61 returns "/lib/security/howdy".
What permissions should be present on the dlib-data folder? Currently, it's set with drwxr--r--

It did seem to be a permission issue. Running
chmod o+x dlib-data/
fixed the issue.

It did seem to be a permission issue. Running
chmod o+x dlib-data/
fixed the issue.

Thanks, resolved the issue for my sddm lock screen as well

I changed the permissions for the dlib-data folder but even then it seems like my camera just doesn't recognize that there's a face at all at the lock screen. When I go into sudo -i it does it very quick.

came here from neon 20.04, same issue

From https://github.com/boltgolt/howdy/issues/450 setting
sudo chmod a+x /lib/security/howdy/recorders/  solves the issue at the lock screen.

For my screensaver (mate-screensaver) under Ubuntu 20.04, with Howdy version 2.6.1 to work, I needed both
sudo chmod o+x /lib/security/howdy/{dlib-data,recorders}
(this is a summary of the separately mentioned hints from above - it took me a moment to realise that there are two different directories mentioned - and I needed both.)


does not create a utility meter assistant in HA, does not reset the utility meter

does not create a utility meter assistant in HA, does not reset _the

_ utility meter

It's super important to provide a time source. Without the time the component is unable to reset the counter at midnight:
https://github.com/syssi/esphome-jk-bms/blob/main/yaml-snippets/esp32-ble-energy-dashboard.yaml#L43-L44

time:

platform: sntp
id: my_time
timezone: Europe/Kiev

Please close this issue if it's resolved.

but it doesn't work like that

time:

platform: homeassistant
id: my_time

now wrote like this

This:
time:
  - platform: sntp

receives the time from NTP servers. This retrieves the time from Home Assistant:
time:
  - platform: homeassistant
    id: my_time

Does the bms0 charging energy today sensor reset on midnight now?

time:

platform: sntp
reset in 2 am

timezone: Europe/Kiev should fix this.

when I type it, it stops working, does not reset the counter

Did you resolve the issue? I'm unable to replicate it.

yes, having solved the task by flashing esp


Release 4.1.0

Resolves #706

Codecov Report

Merging #707 (a46e1cc) into master (322933d) will increase coverage by 10.94%.
The diff coverage is n/a.

:exclamation: Current head a46e1cc differs from pull request most recent head be041fe. Consider uploading reports for the commit be041fe to get more accurate results

@@             Coverage Diff             @@
##           master     #707       +/-   ##
===========================================
+ Coverage   59.07%   70.02%   +10.94%
===========================================
  Files         101       20       -81
  Lines        5229     1321     -3908
===========================================
- Hits         3089      925     -2164
+ Misses       2140      396     -1744

Impacted Files
Coverage Δ

...dditions/Source/Common/HashableRepresentible.swift

...Input/Photos/PhotosInputPermissionsRequester.swift

...dMessage/Content/MessageManualLayoutProvider.swift

...dditions/Source/Input/Text/TextChatInputItem.swift

...attoAdditions/Source/Common/CGRect+Additions.swift

...Photos/Camera/LiveCameraCellPresenterFactory.swift

...poundMessage/CompoundMessagePresenterBuilder.swift

...Source/Input/Photos/Camera/DeviceImagePicker.swift

ChattoAdditions/Source/Common/Observable.swift

...ompoundMessage/Views/CompoundBubbleViewStyle.swift

... and 71 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 322933d...be041fe. Read the comment docs.


Update README.md

What does UMD stand for?
What does it provide its customers?
Remove deprecated Makefile documentation

Some of the answers to high level goals of UMD can be found here: https://docs.google.com/drawings/d/1-m1azdsBqMA0A6ATYRMfkhyeuOJuGCEI62N5a96LXj0/edit


Update Composer dependencies (2021-10-15-00-04)

Visual regression test passed!
View the visual regression test report


Update psycopg2 to 2.9.1

https://github.com/heartexlabs/label-studio/issues/925#issuecomment-923072441

Tested if I could Docker Build off this branch on my M1 mac (in a venv) and it was successful.


feat: add GRPC server and JSON server

Description
front end needs to fetch the validator container data, withdrawal container data and also the proof data to build the tx that calls depositBeaconChainValidator or processWithdrawal, so that we need to add grpc server and json server to enable this feature

Summary by CodeRabbit

New Features

Introduced new configuration files for managing Protocol Buffers and dependencies.
Added structured JSON files for validator and withdrawal proofs, enhancing blockchain state verification.
Implemented a ProofServer for generating validator and withdrawal proofs, supporting Ethereum 2.0 functionalities.
Established a dual communication capability with a gRPC server and an HTTP server.

Bug Fixes

Improved error handling and logging during proof generation processes.

Documentation

Updated documentation to explain new configuration setups and features.

Tests

Added unit tests to validate new JSON structures and proof functionalities.

I deploy this on qa-test machine, which has an beacon node that exposes rpc locally and it works as expected:
curl -X POST -H "Content-Type: application/json" -d '{
>   "slot": 2300000,
>   "validator_index": 1647525
> }' http://localhost:8989/v1/validator-proof
{"stateRoot":"oad7h+JTJLob7kuKcKXEcmAABwn1cIUNJlJtfaS+uC4=", "stateRootProof":["hYFbKzjahiLeHOkQwwet7KldrbIryShnyQfEFMv3Iuo=", "Lu24jFQpF6iDssWpKu8MQzdMPDYkC5scqcAeMlx/X+4=", "KHr1fjYkIQFAp2XsXs4p7J1LHFyBo0vKm3aRwWsRRgk="], "validatorContainer":["0xed43e89d71901bd903479efa75ec36640777119e7c023675a8ed72b1d9ef82b8", "0x010000000000000000000000b87ccabe08275cc5293339589af7ac69f20c06b6", "0x0040597307000000000000000000000000000000000000000000000000000000", "0x0000000000000000000000000000000000000000000000000000000000000000", "0xada0000000000000000000000000000000000000000000000000000000000000", "0x67a8000000000000000000000000000000000000000000000000000000000000", "0xffffffffffffffff000000000000000000000000000000000000000000000000", "0xffffffffffffffff000000000000000000000000000000000000000000000000"], "validatorContainerProof":["+FpULyP7+gODjunLteZVL/6SUyzpaNw+H/4Bt+YwoC0=", "bJLkJEfveRz02vt3qYH5fQnxSoZ/5rN/dCP/GdRC0mc=", "pQFjlA9hySCftDkk/ZQqOLCkw1msmpdyGZ2sixBAuCo=", "3JcCg/2ieJgwQ5eQTl5CVY+vCAz7x/OCDswpeODDDQM=", "q6Ri4HkczkkEYfd3EfogZ9Pls4Aab6mgQReBRJth3oY=", "34hFS/CpLI8HQILko7FxHcb/WRQnG/yV6I78LhN1oMI=", "GrjVTLBvrrmgshFhrD3hl5SvcBNWWRhE5gPsGM/UioI=", "n4+DJKJfWNTK7wQe10x7YytjBHBMl0CRqnFXr11NyhI=", "roQbfk8gmuim1BY11Xbc4l5+ZF65tq4pxsMF0+rBn8w=", "xMMygUYPaikeBysvJiz+fenxgWphT9Vaq7o2Vubbt8A=", "1uDFC3fNqmfsR2J7d+qnYywC01+pccvo+NkEt1293hg=", "NkMJf3etA3sAvxis0DTXtm7RhoRIUKOGwS8pjSQSu24=", "n+aEhJvW67AwFCNYeDAeCrF2U02Gq24/ZTrHAJ2HF2k=", "uLnLhGV4kEZE8lU6joFmM7DAG/w9PnlMP2+wphNwpYI=", "JwbG1zGt8ltUN/u1m8I3LtQXPCqx6rlpXrMLe7zovd0=", "tG5lD1yw5tUKAzkPTL1GF+J1ZZez63qmqOICPmo0Gh0=", "mox1f1b2a42+t8/aZ8S2L6ZsJKrjhE2X3J9oG8FSzW0=", "j2zZbnUcyZgmv7qVwZgYYvo51JpMdefUuFAzWo8thJ8=", "le7IsuVBytTpHeODhfLgRmGfVElsI4LLbKzVuYwm9aQ=", "wCk0K9EzNdStYVAwg1s9S7zs6F5/ix1Bpylfk/CXlTU=", "VOX636KqcSFtSA+sACTCbyHDJGEl6P54LV59pnHhT+g=", "io1/46+MqghadjmoMgAUV9+5EoqAYRQq0DNWKf8j/5w=", "/rPDN9elGm+/ALnjTFLhyRlclpvU56C/1R1cW+2cEWc=", "5x8KqDzDLt+++p9NPgF0yoUYLuyfOgn2psDfY3elENc=", "MSBvqApQu2q+KQhQWPFiEiEqYO7I8En+y5LYyOCoS8A=", "ITUr/svt3emTg59hTD2sCj7jdUP5tBKxYZncFY4jtUQ=", "YZ4xJyS7bXwxU+2d55HXZKNms4mvE8WL+KjZBIGkZ2U=", "fN0phiaCUGKNDBDjhcWMYZHm++BRkbzATxM/LOpywcQ=", "hIkwvXuoysVGYQchE/sniGnge7hYf5E5KTM3TQF7y+E=", "iGn/LCKyjMEFENmFMpKAMyi+T7DoBJXou40nH1uIljY=", "tf4o558bhQ+GWCRs6bah57SfwG23FD6P4LTysMVSOlw=", "mF6Sn3CvKNC90akKgI+Xf1l8fHeMSJ6Y072JENMawPc=", "xvZ+Aubk4b3vuZTGCYlT80Y2uitsogpHIdKyaohnIv8=", "HJp+X/HPSLStFYLT9OShAE87INjForcTh6QlStkz68U=", "Lwda4ilka29q7Rml43LPKVCBQB64k/9Zmz+azAwNPn0=", "Mokh3rWWEgdoAejNYVkhB7XGfHm4RllcxjIMOVtGNiw=", "v7kJ/bI2rSQRtOSIOBCgdLhARkaJmGw/ioCRgn4Xwyc=", "Vdj7Noe6O6SfNCx39aH4m+yD2BFEbhpGcTkhPWQLanQ=", "9yENT45+EDl5Dnv076IHVVoQptsd1LldoxOqqIuI/nY=", "rSG1FsvGRf/jSrXeHIrvjNTn+NK1Ho4UVq3HVjzaIG8=", "eGYbAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=", "yXsFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=", "277JapZ+sf6MazcOIUeDAPaPLACbMhQciY0MQe0YdWA=", "qHPQ2HxhFgi7NxwmYl5vKcUU4nBpoKuJwiG34pYjMMM=", "1LpOSP5PQIZD/NFeWot7Uy7JWPeh8QVhLSnSZmgsPhw=", "uAsVW3QoNUTv8BQZKDwqS/jTWF7zETAhy+Z4qh/WQdA="]}

https://github.com/ExocoreNetwork/proofs-generation/pull/1/commits/cfb825f0342f5dea0aed9335d0090f7922ec9f47  Test passed.


chaining interactors

Hi,
I like interactors, but I'm not a big fan of organizers.
I'd like to code something usable on the fly, this way:
Interactor::Chain.new([ InteractorClass1, InteractorClass2 ]).call!

having a dedicated object like this could be useful to add other methods like:
Interactor::Chain.new([ InteractorClass1, InteractorClass2 ])
  .on_success {|context|  }
  .on_failure {|failure|    }

What do you think?

Some very repetitive stuff could be abstracted again with conventions, like:
Interactor::Chain.new
   .chain([ InteractorClass1, InteractorClass2 ])
   .when_falsy {|context| some_boolean_returned_here  }
      .halt { some_error_string_or_whatever }  # would add  the returning value of `some_error_string_or_whatever` in context.error
   .chain([ InteractorClass3, InteractorClass4 ])
   .on_failure {}
   .on_success {}

@apneadiving thanks for the idea. I think this is do able, but not sure if it is something we have considered in our current workflows.
Do you have an implementation of this already? We would be happy to take a look.

My point is to code it, for sure.
Do you agree with the suggested api?

I have some ideas brewing currently for adding to organizers, giving them the ability to specify per-interactor options:
class DoAllTheThings
  include Interactor::Organizer

  organize DoThingOne, ignore_failure: true
  organize DoThingTwo, async: true
end

There are several reasons for the addition of per-interactor options but my point is that a chainable interface should allow for these same options. Any chainable interface outside of an organizer should still adhere to an organizer's ability, if for no other reason that we can reuse code.
In your example, how would your chain know when to execute?
So far, I'm not sold on the chain interface but I'd like to see where how the API evolves and this issue is a good forum for that. You'd have the best luck being as organizer-y as possible but I can't guarantee that this is something we'll ultimately decide to pursue.
All the same, thank you!! :clap:

the api should be
Interactor::Chain.new(context_init)
   .chain { InteractorClass1 }
   .chain { [InteractorClass2, InteractorClass3] }
   .chain { |context|  some_regular_code }
   .on_failure {}

everything would be evaluated on the fly:

first { InteractorClass1 }, contains an interactor so the interactor is called
if interactor succeeds, next block is evaluated
if not, only on_failure blocks are evaluated

I feel like we could be able to keep your options, of course would be a more verbose syntax, but thats already the case in your example were the slim array of classes is now a few lines.
What lead me to chainable stuff is I have Interactors within Interactors: because some of my interactors have a very small scope, I need them in many places.
So ultimately I want to be able to do:
Interactor::Chain.new(context_init)
   .chain do |context|
      Interactor::Chain.new(context)
        .chain { InteractorClass }
   end
   .chain { SomeOtherInteractorClass } # SomeOtherInteractorClass could itself be a chain
   .on_failure {}

Maybe this is a better way to sell you this :)
Actually I already coded a first concept (without interactors) https://github.com/apneadiving/waterfall but to me, it makes sense to merge the concept with interactors.
Let me know :)

To sum this up, it would be like synchronous promises.

May I know your final word please? :)

@apneadiving We're not sold on the API yet and have talked about this a bit internally but have not come to any conclusions. In fact, I can't give you any guarantee that we'll have clarity for you on this in the immediate future. Please feel free to work on your own implementation of this but please understand that we can't promise inclusion of your implementation until we have more opportunity to discuss. Thank you!

ok thanks for letting me know :)


Move worker from silkworm_core to silkworm_db

Rationale: we want to compile silkworm_core into WebAssembly (Issue #111), which doesn't support multi-threading yet.

Just for my understanding... Does not support means doesn't build ? I mean a thread initialization instruction is an error despite not being used ?

Currently WebAssembly doesn't support threads, so there're no thread-related instructions. I'm using https://github.com/WebAssembly/wasi-sdk to compile into WebAssembly, and it does not provide facilities like <atomic>, so the compilation fails.


Potentially always incorrect warning?

This function:
func X(d []byte) bool {
	return d != nil && len(d)>0
}

Always generates this warning:

should omit nil check; len() for nil slices is defined as zero (S1009)

My thought is this, while the warning is true for d==nil && len(d)
I think the warning is erroneous in the case where you want to check for non-nil and non-zero slice status.
Thoughts?

Not sure what you mean here. Any slice that has a len
greater than zero is a non-nil slice. And the condition in the code you
provided is redundant, as len(nil) == 0.

You are absolutely right. len(x)>0 is sufficient for a non-nil check also.


Add Scala formatter

I was creating a few test files for checking vulnerabilities over Scala language, and found that scalafmt is very nice for formatting. It's also available on nixpkgs
So this would be very helpful, and also a good starting point for some guys in the enterprise world

@drestrepom Is this complete?

As discussed in #502, we will no longer support specific-purpose builtins.


Fix too permissive IAM policy for CA associated to controller nodes

Fix #903

Sweet, least permissive policies are the best

Codecov Report

Merging #904 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #904   +/-   ##
=======================================
  Coverage   35.41%   35.41%
=======================================
  Files          57       57
  Lines        3978     3978
=======================================
  Hits         1409     1409
  Misses       2406     2406
  Partials      163      163

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a4de36e...1f852b8. Read the comment docs.


New Crowdin updates

We don't do this manually, translations are controlled by Crowdin, not sure why it would throw those translations away other than if they change in the source language (which I'm pretty sure they didn't)

Ok, I didn't know


Try to publish to ghcr.io ("GitHub Container Registry")

Fixes #10

Hey @koppor, thanks for the quick implementation 👍🏻 maybe you already googled / fixed it. The build failure could be related to the feature beeing opt in only for now: https://github.com/docker/build-push-action/issues/205#issuecomment-728894085. Anyway great work,

Hey @koppor, thanks for the quick implementation 👍🏻 maybe you already googled / fixed it. The build failure could be related to the feature beeing opt in only for now: https://github.com/docker/build-push-action/issues/205#issuecomment-728894085. Anyway great work,

@splitt3r Thank you for the pointer. I also found it. I enabled it for my profile and for the dante-ev organization.
The last lines of the error message read as follows:
#19 pushing manifest for docker.io/danteev/texlive:latest
#19 pushing manifest for docker.io/danteev/texlive:latest 0.6s done
#19 pushing layers 0.3s done
#19 ERROR: unexpected status: 403 Forbidden

The image at dockerhub is publishes successfully (I checked https://hub.docker.com/repository/registry-1.docker.io/danteev/texlive/tags?ref=login&page=1&ordering=last_updated).
Need to trigger a build again; maybe it works.

@splitt3r Thank you for the pointer. I also found it. I enabled it for my profile and for the dante-ev organization.
The last lines of the error message read as follows:
#19 pushing manifest for docker.io/danteev/texlive:latest
#19 pushing manifest for docker.io/danteev/texlive:latest 0.6s done
#19 pushing layers 0.3s done
#19 ERROR: unexpected status: 403 Forbidden

The image at dockerhub is publishes successfully (I checked https://hub.docker.com/repository/registry-1.docker.io/danteev/texlive/tags?ref=login&page=1&ordering=last_updated).
Need to trigger a build again; maybe it works.

Found the issue for the authoentication. It is dante-ev, not danteev (as in DockerHub).

Found the issue for the authoentication. It is dante-ev, not danteev (as in DockerHub).

Now, I am blocked by https://github.com/docker/build-push-action/issues/251.
I will merge nevertheless and hope the transfer of large images will be fixed some day.

Now, I am blocked by https://github.com/docker/build-push-action/issues/251.
I will merge nevertheless and hope the transfer of large images will be fixed some day.

I think you still need to make the image public.

I think you still need to make the image public.

For reference: The workflow is:

Ensure correct orga settings
First publish package (before, it is not available in the package list https://github.com/orgs/dante-ev/packages/)
Then, the package is available (https://github.com/orgs/dante-ev/packages/container/texlive/settings)
Make the package public

Fixed.

For reference: The workflow is:

Ensure correct orga settings
First publish package (before, it is not available in the package list https://github.com/orgs/dante-ev/packages/)
Then, the package is available (https://github.com/orgs/dante-ev/packages/container/texlive/settings)
Make the package public

Fixed.


[MINOR] Fix build of Hudi website

What is the purpose of the pull request
The build of Hudi website is broken due to the following error from npm run build:
(asf-site)> npm run build

><EMAIL_ADDRESS>build
> docusaurus build

[INFO] Website will be built for all these locales:
- en
- cn
[INFO] [en] Creating an optimized production build...

✔ Client

✖ Server
  Compiled with some errors in 2.64m

[ERROR] Docusaurus Node/SSR could not render static page with path / because of following error:
Error: Minified React error #130; visit https://reactjs.org/docs/error-decoder.html?invariant=130&args[]=object&args[]= for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
    at a.b.render (main:115785:32)
    at a.b.read (main:115781:83)
    at Object.exports.renderToString (main:115792:138)
    at doRender (main:25801:356)
    at async serverEntry_render (main:25797:329)

Error: Server-side rendering fails due to the error above.
[ERROR] Unable to build website for locale en.
[ERROR] Error: Failed to compile with errors.
    at /Users/ethan/Work/repo/hudi-docs-8/website/node_modules/@docusaurus/core/lib/webpack/utils.js:207:24
    at /Users/ethan/Work/repo/hudi-docs-8/website/node_modules/webpack/lib/MultiCompiler.js:554:14
    at processQueueWorker (/Users/ethan/Work/repo/hudi-docs-8/website/node_modules/webpack/lib/MultiCompiler.js:491:6)
    at processTicksAndRejections (node:internal/process/task_queues:78:11)

The root cause is that docusaurus versions specified in website/package.json are not honored.  Looking at the website/package-lock.json generated, 2.0.0-beta.15 is actually used instead of ^2.0.0-beta.3 (up to 2.0.0-beta.3) specified.  Another evidence of higher version already used is that 2.0.0-beta.14 shows up in generated content:
./content/docs/next/clustering/index.html:<meta name="generator" content="Docusaurus v2.0.0-beta.14">

The build failure is likely due to recent new versions (2.0.0-beta.15, 2.0.0-beta.16) of docusaurus and related dependencies.
The fix is to bound the docusaurus version properly.
Note that the build failure can only be reproduced from a fresh clone of the branch from remote, with npm install and npm run build under website folder.  If there is previous successful build and package info is cached, such build failure may not show up.
Brief change log

Updates website/package.json to bound the docusaurus version properly.

Verify this pull request
The change is verified by a fresh build of the website.  The website can be successfully launched after npm start.
Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

CC @vingov


Problems with Custom Raster Data

I am a graduate student who is new to the development of native mobile 3D maps. I was fortunate to have contact with this project. I compiled this project under windows and wanted to modify it, and ran the Android demo. I want to implement Custom Raster Data in the demo, but after entering the module, it prompts me to contact the DJ, what should I do? Thanks for answering!

Hi. The online service that the sample uses no longer returns heightmap tiles, it returns bitmap tiles with the text you are seeing. Thus this is not a SDK issue.
But thanks for reporting this, I will check our samples and try to find a replacement service for heightmap tiles.

Thanks for your reply. If I want to build a 3D map model using elevation data and satellite images, can I learn and implement it through our project?

Hi. The online service that the sample uses no longer returns heightmap tiles, it returns bitmap tiles with the text you are seeing. Thus this is not a SDK issue.
But thanks for reporting this, I will check our samples and try to find a replacement service for heightmap tiles.

Thanks for your reply. If I want to build a 3D map model using elevation data and satellite images, can I learn and implement it through our project?

@clownHHu CARTO Mobile SDK has limited support for elevation data. Basically you could use HillShaderRasterTileLayer as an overlay on top of normal layers, but there is no support for real 3D height fields.

@clownHHu CARTO Mobile SDK has limited support for elevation data. Basically you could use HillShaderRasterTileLayer as an overlay on top of normal layers, but there is no support for real 3D height fields.

I understand, I will try to do some research on this, thanks for your reply.

@clownHHu CARTO Mobile SDK has limited support for elevation data. Basically you could use HillShaderRasterTileLayer as an overlay on top of normal layers, but there is no support for real 3D height fields.

Excuse me , would you please tell me how to get a HillShaderRasterTileLayer on the server , It seems that the url ("http://tiles.wmflabs.org/hillshading/{zoom}/{x}/{y}.png") in the sample of AdvancemapAndroid is an error, it is not displayed .

@clownHHu In order to use HillShadeRasterTileLayer you need heightmap tiles with special encoding. SDK supports MapBox defined encoding as described here: https://docs.mapbox.com/data/tilesets/guides/access-elevation-data/. If you register MapBox account, you could access the tiles using following url: https://api.mapbox.com/v4/mapbox.terrain-rgb/{zoom}/{x}/{y}.pngraw?access_token=YOUR_MAPBOX_ACCESS_TOKEN
The 'wmflabs' source you refer did not really encode height information and does not seem to work anymore anyway.

@mtehver @clownHHu you can also use the free to use https://s3.amazonaws.com/elevation-tiles-prod/terrarium/{z}/{x}/{y}.png.

@clownHHu In order to use HillShadeRasterTileLayer you need heightmap tiles with special encoding. SDK supports MapBox defined encoding as described here: https://docs.mapbox.com/data/tilesets/guides/access-elevation-data/. If you register MapBox account, you could access the tiles using following url: https://api.mapbox.com/v4/mapbox.terrain-rgb/{zoom}/{x}/{y}.pngraw?access_token=YOUR_MAPBOX_ACCESS_TOKEN
The 'wmflabs' source you refer did not really encode height information and does not seem to work anymore anyway.

Thank you so much.

@mtehver @clownHHu you can also use the free to use https://s3.amazonaws.com/elevation-tiles-prod/terrarium/{z}/{x}/{y}.png.

Thank you so much.


[move-vm] Removed most RCs from runtime

Migrating from other repo
Motivation

RCs are no longer necessary due to Move's reference safety checks
Some RCs are still needed to update clean/dirty status or hold onto external memory safely
I feel this greatly simplifies the layout of values in value_impl.rs
It does however make write_ref unsafe, as it could lead to memory issues if the reference safety checks are wrong or not run
I do not have a reference heavy bench mark to test if this makes any perf difference. My guess is it won't, but might help with memory usage

Test Plan

ran tests
I'm a bit nervous that our tests might not be exhaustive enough though....

I ported one of my comments on your original PR to issue  #40 . The other ones can be still found in the older PR. Basically, those comment were around my concern using unsafe APIs throughout our code (in the VM and native functions). Perhaps this can be avoided, and that is what #40 explores.


Implement table-based MAST

Closes #1217

Oh - and let's also update the changelog.

Given some info you listed in one of the previous comments, I'm expecting cycle count and execution time to double (because we are no longer merging basic blocks) - but would be good to confirm this.

It can get much worse than 2x. I think the worst case scenario is with repeat, where all iterations used to get combined into a single basic block, but not instead each iteration is a separate basic block (joined by a tree of JOINs).
Our generate_fibonacci_program() is exactly this. For a repeat.16 instantiation of the program,

before PR: 50 steps (64 steps, 21% padded)
after PR: 235 (256 steps, 8% padded)

A ~5x increase in pre-padding trace length.

@bitwalker was the purpose of this line (#[cfg(feature = "nope")]) to disable the tests? Currently, the code in there no longer compiles, as it is not tracked neither by the compiler nor rust-analyzer. Should I remove it?

I think all looks good here. There are a couple of small questions for @bitwalker (i.e., this one and this one) - but other than that, we should be good to merge.

@bitwalker was the purpose of this line (#[cfg(feature = "nope")]) to disable the tests? Currently, the code in there no longer compiles, as it is not tracked neither by the compiler nor rust-analyzer. Should I remove it?

@plafer I'm actually not 100% sure why that is still in there, I use that fake feature trick to disable a section of code temporarily, but never as a permanent thing. That said, I'm pretty sure the purpose was to remove that code entirely, because the serialization referred to there is basically DOA - we're switching to MAST, so serializing the Miden Assembly syntax tree isn't important. In fact, we probably should just remove all of the serialization-related code from the assembly crate once the MAST refactoring is complete, for now it is still used by the code that emits .masl files (which itself is going away, once the new package format is finalized).


make -j24 problem

collect2: error: ld returned 1 exit status
test/CMakeFiles/test_hiprand_cpp_wrapper.dir/build.make:467: recipe for target 'test/test_hiprand_cpp_wrapper' failed
make[2]: *** [test/test_hiprand_cpp_wrapper] Error 1
CMakeFiles/Makefile2:1174: recipe for target 'test/CMakeFiles/test_hiprand_cpp_wrapper.dir/all' failed
make[1]: *** [test/CMakeFiles/test_hiprand_cpp_wrapper.dir/all] Error 2

What is your device?
Please show hipconfig output.
Show the full make log, perhaps there is more information.

1.ubuntu16+cuda10+cudnn7.4
2.HIP version  : 1.5.18353
== hipconfig
HIP_PATH     : /opt/rocm
HIP_PLATFORM : nvcc
CPP_CONFIG   :  -D__HIP_PLATFORM_NVCC__=  -I/opt/rocm/include -I/usr/local/cuda/include
== nvcc
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:01_CDT_2018
Cuda compilation tools, release 10.0, V10.0.130
=== Environment Variables
PATH=/usr/local/cuda/bin:/home/lyl/bin:/home/lyl/.local/bin:/usr/local/cuda-9.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin:/opt/rocm/bin:/opt/rocm/profiler/bin:/opt/rocm/opencl/bin/x86_64:/home/lyl/bin
LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda-9.0/lib64:/opt/rocm/opencl/lib/x86_64:/opt/rocm/hsa/lib:
== Linux Kernel
Hostname     : lyl
Linux lyl 4.15.0-38-generic #41~16.04.1-Ubuntu SMP Wed Oct 10 20:16:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.5 LTS
Release:	16.04
Codename:	xenial
3.Please check attached file
make error.txt

I'm a bit confused. I assume that you have an NVidia GPU, your main environment is CUDA and you're trying to build pytorch, right?
But it seems strange to me that it requires rocRAND. I know nothing about pytorch but I think that it shouldn't require any ROCm/HIP libraries to be built on CUDA.
Are you sure that pytorch was configured correctly?

Seems the original problem is https://github.com/pytorch/pytorch/issues/14025 ?

@VincentSC yes, Sir. the original problem is about caffe2.


Wait timeout fix

Problem:
Some apps require wait and timeout flags to be passed to helm. CLI does not support these flags for apps.
Solution:
Accept flags for wait and timeout and then pass them to helm.
Depends on following PRs:
types - https://github.com/rancher/types/pull/911
rancher - https://github.com/rancher/rancher/pull/21604
intended merge order:
types -> rancher -> cli
Issue:
https://github.com/rancher/rancher/issues/17471

lgtm

lgtm


Support for bitwise operations

Support for bitwise operators in tensorflow frontend and NNVM top
Thanks for contributing to TVM!   Please refer to guideline https://docs.tvm.ai/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers.

@yzhliu could you please add reviewers?


Cargo init creates duplicates in .gitignore

Doing cargo init in an existing git repository will either create a new .gitignore or append to an existing one. However if we already have a .gitignore file with both /target and **/*.rs.bk ignored, cargo init will still add these to the existing file.
Steps

$ mkdir foo && cd foo
$ git init
$ echo "/target\n**/*.rs.bk" > .gitignore
$ cargo init

The new .gitignore will now contain
/target
**/*.rs.bk

/target
**/*.rs.bk

Possible Solution(s)

Before creating or adding to the existing .gitignore, we could perhaps check the contents and only add what is missing. Looks like the code handling this is https://github.com/rust-lang/cargo/blob/485670b3983b52289a2f353d589c57fae2f60f82/src/cargo/ops/cargo_new.rs#L460-L465, https://github.com/rust-lang/cargo/blob/485670b3983b52289a2f353d589c57fae2f60f82/src/cargo/ops/cargo_new.rs#L471-L476 and https://github.com/rust-lang/cargo/blob/485670b3983b52289a2f353d589c57fae2f60f82/src/cargo/ops/cargo_new.rs#L482-L487
Version: cargo 1.31.0-nightly (2d0863f65 2018-10-20)

I'd happily start working on that one as a first contribution. Seems to be quite straight forward.

Great! Go for it.

I'd like to reopen this issue; I'm still seeing this behavior on git when I run cargo init --lib .

@Lucretiel That's because this fix hasn't made it way into stable Rust yet.  It's in nightly and beta (try cargo +nightly init or cargo +beta init) and stable is releasing in next week Friday 1 March.

Oh, sorry, I didn't see the dates that this was so recent. Thank you!


fix {Open,Boring}SSL conflicts

uses objcopy to rename all exported BoringSSL symbols, prepending LK_ to them, so that they don't conflict with native OpenSSL on the Rust side.
previous PR description before figuring this out below.

PR for review.
trying a different approach than manually patching in the list (though that's still available & commented out), since even when it finally built (90 minute patch time!) it fixed nothing.
the BoringSSL build doc gives us an option to prefix all BSSL symbols with whatever we want, in our case LK. i've punched that into the BUILD.gn file, and it builds just fine in this diff, but the basic_room linking fails with a lot of the following error
/usr/bin/ld: openssl_adapter.cc:(.text+0x7f): undefined reference to `LK_i2d_X509' # and similar symbols

even after i've manually gone in and updated the files to use LK_-prefixed symbols, they just don't link. fresh building environment. i don't know exactly how it would compile by itself (clearly no undefined symbols) and then fail at the basic_room link phase.

Definitivelly awesome work!!!
And looks like it all works (also if I add extra isahc which uses curl+openssl).


No NBT data export 1.14.4

we need a update a we can export bow etc.

The button for this in now called "Exact Mode".

oh thanks but why that's really confusing


clustermgtd fails when I configure a custom partition for the Master node

Hi ParallelCluster Dev Team!
With previous versions of ParallelCluster (up to 2.8.1), we used to configure an additional partition in Slurm for our Master node.
We are currently moving to PC2.10.1 with multiple compute queues support.
After we add the custom Nodename and Partition in a dedicated file that we include at the end of the slurm.conf file, we get the following issues:

The supervidord+cfn_hup+clustermgtd do not start automatically anymore (I could see that both 3 processes are supposed to be started by supervisord).
If I start them manually (through supervisord), the clustermgtd_heartbeat file is not created
In the clustermgtd logs, the following error messages appear every 60 seconds:

"2021-01-20 15:13:42,304 - [slurm_plugin.clustermgtd:manage_cluster] - INFO - Retrieving nodes info from the scheduler
2021-01-20 15:13:43,372 - [slurm_plugin.clustermgtd:_get_node_info_from_partition] - ERROR - Failed when getting partition/node states from scheduler with exception
2021-01-20 15:13:43,373 - [slurm_plugin.clustermgtd:manage_cluster] - ERROR - Unable to get partition/node info from slurm, no other action can be performed. Sleeping..."
After investigation, the check fails because of the Master node. As soon as we remove the Master node and custom partition from the configuration and restart the slurm daemons, the heartbeat works fine again...
Can you help us find the appropriate way to have the Master Node in a custom partition AND maintain successful clustermgtd checks on the compute partitions?
Best regards,
Vincent.

Hi Vincent,
unfortunately there is no way to do it with the current version of clustermgtd. We would need to expose an option in the clustermgtd config file in order to exclude some nodes from the daemon management operations.
If you want to go ahead and patch the clustermgtd  to be able to manually add additional nodes to Slurm, the required code change should be minimal. From a very quick look it seems like that removing the head node from the active_nodes, inactive_nodes lists initialized here should be enough: https://github.com/aws/aws-parallelcluster-node/blob/develop/src/slurm_plugin/clustermgtd.py#L397.

Hi Vincent,
unfortunately there is no way to do it with the current version of clustermgtd. We would need to expose an option in the clustermgtd config file in order to exclude some nodes from the daemon management operations.
If you want to go ahead and patch the clustermgtd  to be able to manually add additional nodes to Slurm, the required code change should be minimal. From a very quick look it seems like that removing the head node from the active_nodes, inactive_nodes lists initialized here should be enough: https://github.com/aws/aws-parallelcluster-node/blob/develop/src/slurm_plugin/clustermgtd.py#L397.

As long as I set the master's node name to the parition-instancetype-[st/dy]-ordinal format, I was able to add my master node to slurm.conf. In my case, that's master-t3large-dy-1 (st didn't work for me actually, even though the node isn't actually dynamic). I also had to add this as a valid alias to /etc/hosts, and after that it worked as expected. Tested on 2.10.3.
It would be good though to have a simple option in the pcluster config to include the master node as its own partition.

Hi @chambm ,
Thank you for suggesting the workaround.
However, I would like to point out that this modification is NOT safe.
clustermgtd will attempt to replace any node configured in slurm that is in a problematic state, so if head node somehow becomes DOWN in slurm, or another problematic scenario happens, the head node instance will likely be terminated and this would break the cluster.
We would suggest not to configure head node in slurm at this time. If you want to make a workaround, please be sure that the head node is excluded from clustermgtd actions, so it wouldn't be terminated by the daemon. We will continue to evaluate this feature request.
Thank you!

Hmm. Several times while I was working on getting the head node to work on SLURM, it was getting set to DOWN by clustermgtd, but it was never terminated. I definitely would have remembered that. :) But I haven't had it set to DOWN since I got it working. Did you test this workaround and had it terminate the head node?

Hi @chambm ,
Sorry I was mistaken, current logic in clustermgtd only retrieves compute instances, so the head node instance cannot be terminated from clustermgtd.
However, because of this, if NodeAddr of the head node configured in slurm ever pointed to the real head node instance, clustermgtd will set the node to DOWN because it cannot retrieve the head node instance, and thinks that there is no actual instance backing the head node configured in slurm
There are still some issues with the workaround that I would like to call out:

Configuring the head node with SuspendExcNodes essentially makes it a static node because it's excluded from the normal Suspend logic. This will also interfere with any other static nodes in the system because slurm will just overwrite SuspendExcNodes and remove other static nodes from that setting
Configuring the node name as a dynamic node master-dy-$master_type-1, makes the clustermgtd not treat the node as a Static node either, which means there is no process that will terminate the node.
Since NodeAddr is not configured, the mapping between master-dy-$master_type-1 and the actual head node instance is never established. Maybe the node is able to get used by slurm as a Dynamic node, which will trigger the Resume program (SuspendExcNodes does not prevent resume from running). However, in this case Resume program will launch a new instance to back master-dy-$master_type-1. So essentially a new dynamic node instance is launched but the actual head node instance is never used by slurm

We would like to point out again that configuring head node in slurm not a supported path, and a workaround will most likely require some custom changes on the node package logic.
Thank you!


FRONTEND-10. Feature: implemented message search.

Codecov Report

Merging #229 (a765b3f) into develop (297e8bc) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #229   +/-   ##
========================================
  Coverage    17.50%   17.50%
========================================
  Files          226      226
  Lines        20309    20309
  Branches       107      107
========================================
  Hits          3555     3555
  Misses       16718    16718
  Partials        36       36

Flag
Coverage Δ

web
17.50% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 297e8bc...a765b3f. Read the comment docs.


AttributeError: 'Entry' object has no attribute 'qualified_name'

------------------------------------------------------------
        answer = 0
        previous_digit = 0
        current_build = 0

        for character in text:
       ^
------------------------------------------------------------

slu/normalizers/age_normalizer.py:185:8: Compiler crash in IterationTransform

ModuleNode.body = StatListNode(age_normalizer.py:1:0)
StatListNode.stats[6] = StatListNode(age_normalizer.py:164:0)
StatListNode.stats[0] = PyClassDefNode(age_normalizer.py:164:0,
    doc = '\n    xxxxxxxxxx\n    ',
    name = 'ChineseNumberParser')
PyClassDefNode.body = StatListNode(age_normalizer.py:165:4)
StatListNode.stats[2] = CompilerDirectivesNode(age_normalizer.py:176:4)
CompilerDirectivesNode.body = StatListNode(age_normalizer.py:176:4)
StatListNode.stats[0] = DefNode(age_normalizer.py:176:4,
    is_cyfunction = True,
    is_staticmethod = True,
    modifiers = [...]/0,
    name = 'parse',
    needs_closure = True,
    np_args_idx = [...]/0,
    num_required_args = 1,
    outer_attrs = [...]/2,
    py_wrapper_required = True,
    reqd_kw_flags_cname = '0',
    used = True)
DefNode.body = StatListNode(age_normalizer.py:177:8,
    is_terminator = True)
StatListNode.stats[5] = ForInStatNode(age_normalizer.py:185:8)

Compiler crash traceback from this point on:
  File "Cython/Compiler/Visitor.py", line 180, in Cython.Compiler.Visitor.TreeVisitor._visit
  File "lib/python3.6/site-packages/Cython/Compiler/Optimize.py", line 190, in visit_ForInStatNode
    return self._optimise_for_loop(node, node.iterator.sequence)
  File "lib/python3.6/site-packages/Cython/Compiler/Optimize.py", line 200, in _optimise_for_loop
    if annotation.entry and annotation.entry.qualified_name == 'typing.Dict':
AttributeError: 'Entry' object has no attribute 'qualified_name'
Traceback (most recent call last):
  File "setup.py", line 17, in <module>
    "annotation_typing": False
  File "lib/python3.6/site-packages/Cython/Build/Dependencies.py", line 1097, in cythonize
    cythonize_one(*args)
  File "python3.6/site-packages/Cython/Build/Dependencies.py", line 1220, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: slu/normalizers/age_normalizer.py

It seems that Class Entry dont have a attribute named "qualified_name" in Cython/Compiler/Symtab.py.
When add "qualified_name=None" in class defination, the error is gone.

@scoder this error happens if you translate python code with typings and annotation_typing turned off. An MCVE:
spam.py
# cython: language_level=3

def f(text: str):
    for c in text:
        print(c)

setup.py
from setuptools import setup
from Cython.Build import cythonize

setup(ext_modules=cythonize('spam.py', compiler_directives={'annotation_typing': False}))

Running build_ext yields:
Compiling spam.py because it changed.
[1/1] Cythonizing spam.py

Error compiling Cython file:
------------------------------------------------------------
...
# cython: language_level=3

def f(text: str):
    for c in text:
   ^
------------------------------------------------------------

spam.py:4:4: Compiler crash in IterationTransform

ModuleNode.body = StatListNode(spam.py:3:0)
StatListNode.stats[0] = DefNode(spam.py:3:0,
    is_cyfunction = True,
    modifiers = [...]/0,
    name = 'f',
    np_args_idx = [...]/0,
    num_required_args = 1,
    outer_attrs = [...]/2,
    py_wrapper_required = True,
    reqd_kw_flags_cname = '0',
    used = True)
DefNode.body = StatListNode(spam.py:4:4)
StatListNode.stats[0] = ForInStatNode(spam.py:4:4)

Compiler crash traceback from this point on:
  File "Cython/Compiler/Visitor.py", line 180, in Cython.Compiler.Visitor.TreeVisitor._visit
  File "/usr/lib64/python3.6/site-packages/Cython/Compiler/Optimize.py", line 190, in visit_ForInStatNode
    return self._optimise_for_loop(node, node.iterator.sequence)
  File "/usr/lib64/python3.6/site-packages/Cython/Compiler/Optimize.py", line 200, in _optimise_for_loop
    if annotation.entry and annotation.entry.qualified_name == 'typing.Dict':
AttributeError: 'Entry' object has no attribute 'qualified_name'
Traceback (most recent call last):
  File "setup.py", line 4, in <module>
    setup(ext_modules=cythonize('spam.py', compiler_directives={'annotation_typing': False}))
  File "/usr/lib64/python3.6/site-packages/Cython/Build/Dependencies.py", line 1097, in cythonize
    cythonize_one(*args)
  File "/usr/lib64/python3.6/site-packages/Cython/Build/Dependencies.py", line 1220, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: spam.py

Tested with Python 3.6.8, Cython 0.29.4.

If you change annotation_typing to True (or simply remove compiler directives), the module is translated correctly.

Thanks for the report and digging up the further hints.

Hmm, that wasn't enough, it seems. Looking into it.

Awesome! Thank you for all the good work and looking forward to the 3.0 release!

@scoder you tagged this with 0.29.7 but it looks like it only went into 3.0

@da-woods I changed my mind after the fix (2019-04-14), but didn't leave a message here why. Probably because it turned out that a backport wouldn't be easy enough due to some infrastructure dependencies in the code.