forgejo: add missing internal version ldflags, nixosTests.forgejo: test /api/forgejo/v1/version

Description of changes
PR is a bug fix in our nixpkgs packaging, prepares us for the next major release and adds a version test.

Those two ldlfags provide the version string for the /api/forgejo/v1/version API endpoint and the forgejo_sem_ver database table.
Prio to this change, /api/forgejo/v1/version returned "development" and the database table "1.0.0".
The proper version string, which we are now providing, is a combination of a hardcoded version compatibility string (like "gitea-1.21.8") and the forgejo semver version.
The forgejo semver version in forgejo v1.21.8-0 is technically hardcoded in the Makefile, but will use git describe starting with the next major release (which will also be the release when forgejo makes this semver user facing).
So in summary, this commit is a bug fix in our nixpkgs packaging and prepares us for the next major release.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Self-merging this, since it has an approval from a maintainer and forgejo v1.21.10-0 has just been released.


The timing of apply rewards/penalties

Issue
Follow up of https://github.com/ethereum/eth2.0-specs/issues/616

Not if we fix the base reward for each validator before starting the rewards section. So either apply rewards at end or make a base reward var at the beginning and use that to update. Either would result in the same

The current description of the spec says "If validator ...., gains ..." and "If validator ...., loses ...". It's unclear to know that do we apply the reward immediately or at the end.
Although our in-spec parameters setting won't let underflow happen (i.e., the validator won't get penalized that much before they get ejected.), it would be nice to define the procedure strictly.
Proposed solution
Ref: https://github.com/ethereum/trinity/pull/273

Set local map/dict variables rewards_received: Dict[ValidatorIndex -> Gwei] and penalties_received: Dict[ValidatorIndex -> Gwei] at the beginning of the "Rewards and penalties" section.
Use Python code to explicitly add rewards/penalties to the rewards_received map.
Add an "Apply rewards and penalties" section at the end of "Rewards and penalties" section.

Apply rewards_received to state.validator_balances.
Apply penalties_received to state.validator_balances; If underflow, set balance to 0.

closing via #711


Suggestion: Add an option to have tokens without any status be invisible to the tracker by default

Basically, i would like to be able to tell the tracker to only track tokens who have a specific status icon, instead of having to put a status on all the tokens i don't want to be seen on the tracker
Thanks for reading!

Why not, I'll try to take a look, sorr for the delays

@pplans Thank you! is this still in the plans?

Hey, yeah, I don't work on it often but if it becomes pressing I do ^^ I'll do add this today, expect it arriving in the next days :)

Hello, I added an option that inverts the token statuses, so that if you flag as usual a token as immobile it will actually be mobile and vice-et-versa, please let me know how it goes for you with that feature

@pplans This does exactly what i wanted, thank you very much!

Lemme know if you have issues with it, I'll close the ticket :) glad I helped


odb: fix OdbReader read return value

git_odb_stream_read return the number of bytes read.
This should be a typo, and will cuase Read.read_to_end to read for ever since it will continuously call  read to append more data to buf until read returns either Ok(0)
or an error of non-ErrorKind::Interrupted kind.

Thanks! I believe this is correct, but the libgit2 docs say it returns 0. I believe the docs are incorrect, so I opened https://github.com/libgit2/libgit2/pull/6837 just to verify.


Possible memory leak in shuffle-server

Background
I found the metric size of res  is too large when using the command of top to inspect the shuffle-server process, which has been excluded in the cluster and no app runs on it.
When I dump the jvm heap, I found the heap size is normal. But it has too many netty PooledUnsafeDirectByteBuf objects, it this normal?

Shuffle-server startup Environment
XMX=30g
top RES = 54g

We have made the shuffle-servers colocate with the Yarn nodemangers. So we hope the used memory of shuffle-server could be limited by using the JVM options -XX:MaxDirectMemorySize or other netty config.

Could u help check whether this is a problem? Or give me some ideas? @jerqi Thanks.

Maybe we could remove grpc.


merge_and_unload issue?

System Info
I am using the latest dev version of transformers, accelerate and peft (installed via !pip install -q -U git+) installed via Google Colab.
This worked a few days ago, but now when I attempt to merge an adapter back into the base model and then save to hub, the size is much smaller than that of the base model and it can't be loaded (generates error "Cannot copy out of meta tensor; no data!" when I attempt to do so).
The function I am using to merge the PeftModel back into the base model is:
(Code begins here)
def merge_adapter(lora_id, merged_id):
    config = PeftConfig.from_pretrained(lora_id)

    model_id = config.base_model_name_or_path
    model = AutoModelForCausalLM.from_pretrained(
        model_id,
        torch_dtype=dtype,
        device_map="auto",
        offload_folder="offload"
        )

    adapter = PeftModel.from_pretrained(
        model,
        lora_id,
        torch_dtype=dtype,
        device_map="auto",
        offload_folder="offload"
    )

    model = adapter.merge_and_unload(progressbar=True)
    tokenizer = AutoTokenizer.from_pretrained(model_id) #, use_fast=False)

    model.save_pretrained(
        merged_id,
        push_to_hub=True,
        repo_id=merged_id,
        private=True,
        #max_shard_size="4GB"
    #,
    )

    tokenizer.save_pretrained(
        merged_id,
        push_to_hub=True,
        repo_id=merged_id,
        private=True,
    )

The base model is 'meta-llama/Llama-2-7b-hf', which has two bin files, 1 @ 9.98GB and 1 @ 3.5GB. Previously when I ran this code, the merged model would be the same size. Now it only produces a single file @ 1.07GB.
This may be an error with the library, although I am not seeing any bug reports to indicate this. It may also be an error with the training code in the first place, my upload code, the HF library or anything else.
If anyone has any solutions, please let me know. Otherwise, if this is a bug, I guess this is my first bug report.
Who can help?
No response
Information

[ ] The official example scripts
[X] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder
[X] My own task or dataset (give details below)

Reproduction
This is the Colab I have been using. The data and LoRAs are private, but the two I am playing with are LLama 2 7B QLoRA fine-tuned on 1) a chat dataset and 2) a big hunk of raw text split into paragraphs (a piece of fanfiction).
https://colab.research.google.com/drive/15g2NU2wJ9fOvY3PJCCN5dVDYV8KSXbeS?usp=sharing
Expected behavior
The merged model should be created and be the same size as the base model, and I should be able to load it using AutoModelForCausalLM.from_pretrained as I was able to a few days ago.

i meet the same issue.
when i merge lora to llama model, and the save it .the model size is smaller than before

I think the issue is that the layers that are on "meta" device are not properly handled in this case. Their weights are not loaded and the lora weights are also just on meta device. The result you see probably only contains the weights that were actually loaded, missing all the meta device weights.
Honestly, I'm not completely sure how best to handle this situation. Maybe @pacman100 or @younesbelkada have a good idea.

Sorry, I'm really not sure what a solution would look like here. The reason why the weights are on meta device is that they needed to be offloaded for lack of memory, so it's not as simple as just loading everything. Hopefully one of the others can shine some light on this.

This issue may occur when the GPU is occupied by other processes. device="auto" may not be able to load models into GPUs, so try to ensure that device is free or set device="cpu" instead.

I've encountered this issue, the model was trained and merge with the same GPU(3090 24GB), VRAM is sufficient for lora training and inferencing, but after merged i got this issue

This issue may occur when the GPU is occupied by other processes. device="auto" may not be able to load models into GPUs, so try to ensure that device is free or set device="cpu" instead.

Setting device_map="cpu" from auto sovled the issue for me

I think that merge and unload is not supported for models that are offloaded into disk / cpu as accelerate puts the offloaded weights on the meta device. This is a bug we should look into

how to merge ptuning model?

Hi @xiaobai52HZ I think that sadly currently merging p-tuning models is not supported. cc @pacman100 @BenjaminBossan

Indeed.

Can we add boolean arguments to disable all off-loading?
I still want the benefits of using multiple GPUs but raise an error if the weights cannot fit with GPUs alone

Can we add boolean arguments to infer device map to disable all off-loading?

Offloading is not performed by PEFT. If you want to have this feature, please consider opening an issue in accelerate.

This issue may occur when the GPU is occupied by other processes. device="auto" may not be able to load models into GPUs, so try to ensure that device is free or set device="cpu" instead.

This fixed the issue for me as well

Closing this issue ! Feel free to re-open in case you think your concerns are not being addressed

Still facing the problem:
trainer.train()

# Save the adapter
trainer.save_model(saver_dir + '/adapter')

# Retrieve the model
model = trainer.model.base_model

# Loading the adapter
model = PeftModel.from_pretrained(model, saver_dir + '/adapter', torch_dtype=torch.float16, device_map="auto")

# Merge the base model and the adapter
model = model.merge_and_unload()

# Save the overall model
model.save_pretrained(saver_dir)

It seems that the base model will be saved without the LoRA adapter.
{
  "_name_or_path": "meta-llama/Llama-2-7b-hf",
  "architectures": [
    "LlamaModel"
  ],
  "attention_bias": false,
  "bos_token_id": 1,
  "eos_token_id": 2,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 11008,
  "max_position_embeddings": 4096,
  "model_type": "llama",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 32,
  "pad_token_id": 2,
  "pretraining_tp": 1,
  "rms_norm_eps": 1e-05,
  "rope_scaling": null,
  "rope_theta": 10000.0,
  "tie_word_embeddings": false,
  "torch_dtype": "float16",
  "transformers_version": "4.34.0",
  "use_cache": true,
  "vocab_size": 32001
}

This is expected @indiejoseph
If you call merge_and_unload() it will merge lora adapters into the base model, unload the lora layers and return the base transformers model.


Port Shell Lib Code to Go

This logic needs to be ported to Go for functionality in OpDom and other CLI utilities.

rke2_list_image_repos
rke2_list_images
rke2_charts_get_index
rancher_list_local_repos

Complete.


回车后和上一行对齐，比如上面一行首行前端有一个tab，那回车后也自动空出一个tab（比如editplus做的那样）

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

实现了：https://github.com/JasonStein/Notepads/commit/d359000cb5f63b8a30c6424749967a3d275dd016
v<IP_ADDRESS>会加入


Storybook MDX docs

As demoed earlier today, this is an experiment to find a way to be able to show multiple variations of a component and document stories using markdown whilst still retaining Haml partials.
New sidebar structure

Docs view

Usable code snippets
Structured so these could be copied directly into a rails app:

Haml usage notes written in markdown

Technical notes
Pros:

Avoids the need to use locals.rb
Allows multiple variants of each component
Makes it possible for designers to write documentation for components

Cons:

We are still working against storybook a little by using Haml partials so there's still a little bit of a dance to get some of the previews to work

👩💻 👨💻  Recommend checking this out and having a play for yourselves
🗒  I'll add explanatory comments for some of the structural changes
~⚠ I've included my initial work on OISC warnings in here as a working example. Looking for reviews on the storybook structure  at this stage, not this component. If we are happy with the approach I'll remove that work from this PR so we can merge this without shipping early work on the warnings component.~ Updated to remove OISC warnings from this PR to keep things simple.

Updated to remove OISC warnings from this PR to keep things simple.

This is a fantastic improvement to Storybook! Makes it much easier for non-devs to navigate and also see all the different formats for a component.
I have two questions:

Is it possible to have the "Docs" tab first? I think it will be useful to get an overview first, and then be able to explore an individual variant in more depth if desired. I'm also worried people won't notice the docs tab. It's not that big of a thing, we can always do demos to share the features.
I know you haven't gotten to the forms yet, but will this format allow us to show the different states (with or without hint, with an error, etc)? That would be super useful as well!

Either way, this is great. Thanks!

Is it possible to have the "Docs" tab first? I think it will be useful to get an overview first, and then be able to explore an individual variant in more depth if desired

I think so, let me double check

I know you haven't gotten to the forms yet, but will this format allow us to show the different states (with or without hint, with an error, etc)? That would be super useful as well!

Yep! (I did a throwaway experiment for exactly that when I was trying this new format out)


Resolve database deadlocks

Sentry issue #BACKEND-3Q4 - has only happened once but worth looking into
Error inserting records: deadlock detected
DETAIL:  Process 1946901 waits for ShareLock on transaction 18808124; blocked by process 1946903.
Process 1946903 waits for ShareLock on transaction 18808128; blocked by process 1946901.
HINT:  See server log for query details.
CONTEXT:  while inserting index tuple (40190,7) in relation "_hyper_1_13960_chunk"

happen again this morning - opening again (on NEM price inset)

Fixed using increased timeout, will verify with Sentry logs


Unit tests for check_action_value function

Add unit tests for check_action_value function to check what would be if we will pass wrong param type, or wrong param, we should be sure it's working as expected
please create new file in tests/ for it, ensure it starts from test_ so pytest will find it and fetch automatically

ImportError while importing test module 'C:\Users\boris\PycharmProjects\tg_chat_mate\bot\tests\test_unit.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
......\AppData\Local\Programs\Python\Python310\lib\importlib_init_.py:126: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
tests\test_unit.py:4: in
from support_bot.routes.utils import check_action_value
E   ModuleNotFoundError: No module named 'support_bot'

pipenv run pytest tests/test_unit.py -v -s

we can`t remove Magic because all imports will be broke


Mangle - Fix for classes and program scope

EDIT: DO NOT MERGE.. yet
Still doesn't cover some cases. #138
EDIT 1: Updated.

Updated!.

Looks good, thanks!


Postmaster fails on space-delimited FAM files

We should parse FAM files using whitespace and not require tab

Closing because postmaSTR is not part of trtools anymore


Add force parameter (small feedback)

@christonog I made tiny changes to get the tests green. Please merge into your branch, which I will then merge into my repo. Thanks for waiting and again thanks for your contribution.

awesome, thanks @jckuester! ❤


pagination handling error for BioThings APIs

From the BTE logs:
1|bte-trapi  | 2021-09-20T21:05:19.209Z bte:call-apis:query Failed to make to following query: {"url":"https://biothings.ncats.io/text_mining_co_occurrence_kp/query","params":{"fields":"object,association","q":"subject.MONDO:\"MONDO:0004335\" AND object.type:Gene","size":1000,"from":324000},"method":"get","timeout":50000}. The error is Error: Request failed with status code 400
1|bte-trapi  | 2021-09-20T21:05:19.209Z bte:call-apis:query The request failed with the following error response: {"code":400,"success":false,"error":"Bad Request","keyword":"from","max":10000,"num":324000,"alias":"skip"}

With BioThings API , from greater than 10000 will receive 400 error. Large query hits are not recommended to use from+size pagination, either we can stop by 10000 or switch to use fetch_all streaming feature of the Biothings APIs.

Also involved:

changing the x-bte annotation's specification of the sub-query to have a cut-off. then less records would match and be returned

Keep in mind:

BTE currently does not ingest this API (text mining co-occurrence) for its normal sub-querying. This query was likely done directly to the Text Mining co-occurrence API, either by someone on that team or the ARS

PR https://github.com/biothings/call-apis.js/pull/33 merged and deployed to prod.


all_layouts and all_parts options

Coverage decreased (-0.003%) to 83.965% when pulling 73ff88c99a4d7634ff5f6727fefcf906a58f12b3 on ShinjiTanimoto:search-all-layouts into 52610475ee1cfae4f5e1fa6aecde249511956010 on shirasagi:master.

Coverage decreased (-0.7%) to 83.239% when pulling 7852c9a8883515215635ed8062e39d31cd8801c9 on ShinjiTanimoto:search-all-layouts into 52610475ee1cfae4f5e1fa6aecde249511956010 on shirasagi:master.

Coverage decreased (-0.7%) to 83.226% when pulling f2fe33cc4a1cb7c2bf24bb229fcf44a53ad48553 on ShinjiTanimoto:search-all-layouts into 52610475ee1cfae4f5e1fa6aecde249511956010 on shirasagi:master.

scope で nil が返ります
https://github.com/ShinjiTanimoto/shirasagi/blob/9bef29c427cc0a993ac913a9562af4243f7298e1/app/models/concerns/cms/content.rb#L43

Coverage decreased (-0.7%) to 83.234% when pulling 9bef29c427cc0a993ac913a9562af4243f7298e1 on ShinjiTanimoto:search-all-layouts into 52610475ee1cfae4f5e1fa6aecde249511956010 on shirasagi:master.

Coverage decreased (-0.7%) to 83.234% when pulling d2c1ec56916663141c7b6c567167463acd19e2f5 on ShinjiTanimoto:search-all-layouts into 52610475ee1cfae4f5e1fa6aecde249511956010 on shirasagi:master.


SQL statement extension support in AT mode

Why you need it?
Is your feature request related to a problem? Please describe in details
Currently, the AT transaction mode supports a limited number of SQL statements. For details, see: https://seata.io/zh-cn/docs/user/sqlreference/sql-restrictions.html . SQL statement support needs to be further extended to meet application requirements.
目前，AT事务模式支持有限的SQL语句，具体参考：https://seata.io/zh-cn/docs/user/sqlreference/sql-restrictions.html 。需要进一步扩展SQL的语句支持来满足应用的需求。
Tasks are divided by SQL statement dimensions
任务将以SQL维度划分。
How it could be?
A clear and concise description of what you want to happen. You can explain more about input of the feature, and output of it.
Other related information
Add any other context or screenshots about the feature request here.

INSERT ALL INTO TABLE_NAME (...)VALUES(...)
https://github.com/seata/seata.github.io/issues/414#issue-1103177287

INSERT IGNORE INTO TABLE_NAME (...)VALUES(...)

ref https://github.com/seata/seata/issues/3845

INSERT IGNORE INTO TABLE_NAME (...)VALUES(...)

ref https://github.com/seata/seata/issues/3845

mysql: replace into

mysql: replace into

please assgin to me

INSERT ALL INTO TABLE_NAME (...)VALUES(...) seata/seata.github.io#414 (comment)

please assign it to me

mysql: insert into select #3965

INSERT ALL INTO TABLE_NAME (...)VALUES(...) seata/seata.github.io#414 (comment)

please assign it to me

已有pr提交参与解决了，#3292

update join语句支持 #4118

mysql: insert into select #3965

i can do it


Move SLC cloning to notebook

In the past SLC was cloned by Ansible task exasol/ds/sandbox/runtime/ansible/roles/script_languages/tasks/main.yml
In the future this should be done by the related SLC notbook in the DSS.
This will require changes to

user_guide.md
test/ansible_conflict/slc_setup.yml
test/test_ansible.py
test/ansible_conflict/slc_setup.yml
test/test_ci.py
test/test_install_dependencies.py
test/test_release_build.py
exasol/ds/sandbox/lib/config.py
exasol/ds/sandbox/lib/setup_ec2/run_install_dependencies.py

See also

#17
#19

Folder script_languages has been removed from exasol/ds/sandbox/runtime/ansible/roles/


Upgrade to Gradle 7

As of this writing I believe the only blocker to this is the gatling-gradle-plugin which only supports up to version 6.6.1. More information here:
https://gatling.io/docs/current/extensions/gradle_plugin/#compatibility
Once the upgrade can happen here's documentation on upgrading:
https://docs.gradle.org/current/userguide/upgrading_version_6.html#changes_7.0

This PR adds compatibility for gradle 7:
https://github.com/gatling/gatling-gradle-plugin/pull/32


Implement time-related types with standard Rust representation

#175 introduced serialization mechanisms for CQL types used to express time (timestamp, datetime, date), but they are represented in a raw form - an integer - which is not ideal for user experience. Instead, these types can be expressed with standard Rust mechanisms from https://doc.rust-lang.org/std/time/index.html .
Refs #104 #97

There is one bug I've found which might be automagically fixed when moving to a new timestamp representation. The following query:
session.execute("SELECT * FROM table WHERE timestamp_field = ?", &(&0,)).await?

fails with the the error:
ScyllaDB query error: Database returned an error: The query is syntatically correct but invalid, Error message: Exception while binding column starts_at: marshaling error: Validation failed for timestamp - got 1 bytes

Old issue, it's already implemented.


Testing with Logarex LK13BE606739 and RS485

Actually, it looks that this is exactly the piece of software I'd need to connect my new LK13BE606739 power meters (3 of them) via RS485 on a Raspi 2B with Home Assistant and I'd love to test it.
Unfortunately, I'm already failing with installation. poetry install takes forever (no progress for hours), obviously there is an issue with installing pyparsing :
admin@autoberry:~/Logarex/py-power-meter-monitor-main $ poetry install -vvv
Loading configuration file /home/admin/.config/pypoetry/config.toml
Using virtualenv: /home/admin/.cache/pypoetry/virtualenvs/py-power-meter-monitor-Oi_2Xx_V-py3.9
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 35 installs, 0 updates, 0 removals, 10 skipped

  • Installing pyparsing (2.4.7): Pending...
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
^C
^C
^CException ignored in: <module 'threading' from '/usr/lib/python3.9/threading.py'>
Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 1415, in _shutdown
    atexit_call()
  File "/usr/lib/python3.9/concurrent/futures/thread.py", line 31, in _python_exit
    t.join()
  File "/usr/lib/python3.9/threading.py", line 1033, in join
    self._wait_for_tstate_lock()
  File "/usr/lib/python3.9/threading.py", line 1049, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt:

My active coding time ended with Delphi a long time ago, so I'm not really familiar with modern concepts and languages like Python. Chances are high that I missed or did something wrong. But after some recherching it looks like there is a common issue poetry has with resolving of dependencies. Do you have a hint what I might be doing wrong or what to do?
Thanks a lot!

I now got it running, but there is an error with mqtt popping up, preventing further exeution:
py-power-meter-monitor --config-file ./logarex-config.toml
DEBUG:asyncio:Using selector: EpollSelector
DEBUG:py_power_meter_monitor.commands:Opened serial connection /dev/serial/by-id/usb-FTDI_FT232R_USB_UART_AB0P5R4Z-if00-port0 with 9600 baud.
Traceback (most recent call last):
  File "/home/admin/.local/lib/python3.9/site-packages/asyncio_mqtt/client.py", line 195, in connect
    await loop.run_in_executor(
  File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/admin/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 914, in connect
    return self.reconnect()
  File "/home/admin/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 1044, in reconnect
    sock = self._create_socket_connection()
  File "/home/admin/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 3685, in _create_socket_connection
    return socket.create_connection(addr, timeout=self._connect_timeout, source_address=source)
  File "/usr/lib/python3.9/socket.py", line 822, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/usr/lib/python3.9/socket.py", line 953, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
OSError: Int or String expected

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/admin/.local/bin/py-power-meter-monitor", line 8, in <module>
    sys.exit(app())
  File "/home/admin/.local/lib/python3.9/site-packages/typer/main.py", line 214, in __call__
    return get_command(self)(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3/dist-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/admin/.local/lib/python3.9/site-packages/typer/main.py", line 532, in wrapper
    return callback(**use_params)  # type: ignore
  File "/home/admin/.local/lib/python3.9/site-packages/py_power_meter_monitor/cli.py", line 31, in run
    asyncio.run(
  File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/admin/.local/lib/python3.9/site-packages/py_power_meter_monitor/commands/monitor_serial.py", line 45, in run_monitor_serial
    async with asyncio_mqtt.Client(
  File "/home/admin/.local/lib/python3.9/site-packages/asyncio_mqtt/client.py", line 535, in __aenter__
    await self.connect()
  File "/home/admin/.local/lib/python3.9/site-packages/asyncio_mqtt/client.py", line 206, in connect
    raise MqttError(str(error))
asyncio_mqtt.error.MqttError: Int or String expected

Setting mqtt_enabled to false in the config didn't help. Any idea? Thank you!
Her my config file (parity ="N"  quotes were necessary, they were missing in the demo config).
[logging]
level = 10 # error level (see built-in python logging levels)

[mqtt]
enabled = true
configuration_topic_template = "homeassistant/sensor/{entity_id}/config"
state_topic_template = "homeassistant/sensor/{entity_id}/state"

[mqtt.broker]
hostname = "192.168.XXX.YYY"
port = 1883

[mqtt.device]
id = "power-meter-0"
name = "Power Meter 0"
manufacturer = "Logarex"
model = "LK13BE606739"

[serial_port]
port_url = "/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_AB0P5R4Z-if00-port0"
baud_rate = 9600
byte_size = 8
parity = "N"

[obis]

[[obis.data_sets]]
id = [1, 0, 96, 1, 0, 255]
name = "Meter Id"
value_type = "string"

[[obis.data_sets]]
id = [1, 0, 0, 2, 0, 255]
name = "Meter Program Version"
value_type = "string"

[[obis.data_sets]]
id = [1, 0, 97, 97, 0, 255]
name = "Meter Error Flags"
value_type = "string"

[[obis.data_sets]]
id = [1, 0, 1, 8, 0, 255]
name = "Positive Active Power Integral Total"
value_type = "float"

[[obis.data_sets]]
id = [1, 0, 2, 8, 0, 255]
name = "Negative Active Power Integral Total"
value_type = "float"

[[obis.data_sets]]
id = [1, 0, 16, 7, 0, 255]
name = "Sum Active Power Instantaneous Total"
value_type = "integer"

[[obis.data_sets]]
id = [1, 0, 32, 7, 0, 255]
name = "Voltage Instantaneous L1"
value_type = "float"

[[obis.data_sets]]
id = [1, 0, 52, 7, 0, 255]
name = "Voltage Instantaneous L2"
value_type = "float"

[[obis.data_sets]]
id = [1, 0, 72, 7, 0, 255]
name = "Voltage Instantaneous L3"
value_type = "float"

[[obis.data_sets]]
id = [1, 0, 31, 7, 0, 255]
name = "Current Instantaneous L1"
value_type = "float"

[[obis.data_sets]]
id = [1, 0, 51, 7, 0, 255]
name = "Current Instantaneous L2"
value_type = "float"

[[obis.data_sets]]
id = [1, 0, 71, 7, 0, 255]
name = "Current Instantaneous L3"
value_type = "float"

[[obis.data_sets]]
id = [1, 0, 14, 7, 0, 255]
name = "Frequency Instantaneous"
value_type = "float"

[[obis.data_sets]]
id = [1, 0, 1, 8, 0, 96]
name = "Positive Active Power Integral Last Day"
value_type = "float"

[[obis.data_sets]]
id = [1, 0, 1, 8, 0, 97]
name = "Positive Active Power Integral Last Week"
value_type = "float"

[[obis.data_sets]]
id = [1, 0, 1, 8, 0, 98]
name = "Positive Active Power Integral Last Month"
value_type = "float"

[[obis.data_sets]]
id = [1, 0, 1, 8, 0, 99]
name = "Positive Active Power Integral Last Year"
value_type = "float"

[[obis.data_sets]]
id = [1, 0, 1, 8, 0, 100]
name = "Positive Active Power Integral Since Reset"
value_type = "float"


Duplicate flowlines sometimes returned by get_flowline_index when matching multiple points

Related to #348, I'm using get_flowline_index to match many thousands of point locations to NHDPlusv2 flowlines. I noticed that the function sometimes returns different flowline matches if you attempt to match multiple sites within a single call to get_flowline_index compared to matching a point individually. This behavior was unexpected to me, so documenting here for awareness.
Below I've included a small reprex with just two points. In the first case where we attempt to match both sites together, the flowline with COMID == 19718419 gets returned twice. A third flowline within the search radius, COMID == 19718425, is omitted from the returned matches, but does get returned if the point is matched individually.
library(nhdplusTools)
library(sf)
#> Linking to GEOS 3.10.2, GDAL 3.4.1, PROJ 7.2.1; sf_use_s2() is TRUE
library(dplyr)
#>
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#>
#>     filter, lag
#> The following objects are masked from 'package:base':
#>
#>     intersect, setdiff, setequal, union

# Define points
locs <- tibble(id = c(1, 2),
               lon = c(-83.87865, -83.87975),
               lat = c(35.60989, 35.60963))
points <- sf::st_as_sf(locs, coords = c("lon", "lat"), crs = 4326) %>%
  sf::st_transform(5070)
flines_aoi <- sf::st_buffer(points, dist = units::as_units(5000, "m")) %>%
  sf::st_bbox() %>%
  sf::st_as_sfc()

# Define objects that we'd pass to get_flowline_index
search_rad <- units::as_units(500, "m")
max_match <- 3
precision <- 10
flines <- nhdplusTools::get_nhdplus(AOI = flines_aoi, realization = "flowline")
#> Spherical geometry (s2) switched off
#> Spherical geometry (s2) switched on

# get_flowline_index returns different NHD flowline matches if you analyze all
# sites together versus one site at a time
## BOTH SITES
get_flowline_index(flines, points,
                   search_radius = search_rad,
                   max_matches = max_match,
                   precision = precision) |>
  filter(id == 2)
#>   id    COMID      REACHCODE REACH_meas   offset
#> 1  2 19718419<PHONE_NUMBER>0580     2.9661 15.30653
#> 2  2 19718419<PHONE_NUMBER>0580     3.4706 15.30653
#> 3  2 19718427<PHONE_NUMBER>0191    97.5895 15.91549

## ONE SITE
get_flowline_index(flines, filter(points, id == 2),
                   search_radius = search_rad,
                   max_matches = max_match,
                   precision = precision)
#>   id    COMID      REACHCODE REACH_meas   offset
#> 1  1 19718419<PHONE_NUMBER>0580     3.4706 15.30653
#> 2  1 19718427<PHONE_NUMBER>0191    97.5895 15.91549
#> 3  1 19718425<PHONE_NUMBER>0192     0.0000 23.18244

Created on 2023-08-09 by the reprex package (v2.0.1)
I traced this behavior to the step that returns the intersection of the flowlines with the buffered points. An intersecting fline is created for each point, resulting in very similar flowlines being included in the nn search if there are multiple points and any are relatively close to each other:
# Using same reprex as above.

# If similar flowlines are within search_radius, these count towards the "max_matches" limit
point_buffer <- sf::st_buffer(points, search_rad)
flines_sub <- sf::st_intersection(flines, point_buffer)
#> Warning: attribute variables are assumed to be spatially constant throughout all
#> geometries

nrow(flines_sub)
#> [1] 8
length(unique(flines_sub$comid))
#> [1] 4

# Compare with filtering flines based on whether they're within the search radius
# for ANY point. This is slower, includes the full nhd reach even the portion
# outside of the search radius, and we may actually want the flines by point -
# just illustrating for the purposes of this reprex
flines_sub2 <- sf::st_filter(flines, points,
                             .predicate = sf::st_is_within_distance,
                             dist = search_rad)

nrow(flines_sub2)
#> [1] 4
length(unique(flines_sub2$comid))
#> [1] 4

library(mapview)
mapviewOptions(fgb = FALSE)

mapview(filter(flines_sub, id == 1), color = "darkblue", layer.name = "intersection - site 1") +
  mapview(filter(flines_sub, id == 2), color = "orange", layer.name = "intersection - site 2") +
  mapview(points, zcol = "id", col.regions=list("darkblue","orange"))

Interesting. Do you think this is an artifact of using the RANN package the wrong way?
https://cran.r-project.org/web/packages/RANN/RANN.pdf
The nn2 call is buried down here.
https://github.com/DOI-USGS/hydroloom/blob/main/R/index_points_to_lines.R#L6
  matched <- RANN::nn2(data = coords[, 1:2],
                       query = matrix(points[, c("X", "Y")], ncol = 2),
                       k = ifelse(max_matches > 1, max_match_, 1),
                       searchtype = "radius",
                       radius = search_radius)

Right, I took a glance at that nn2 function but think this behavior is stemming from somewhere upstream - and is possibly an artifact of returning one set of intersected flines for each point.
So for the point with id == 2 in the figure above, the set of yellow linestrings is getting passed to nn2 in addition to the set of blue linestrings, which were generated when considering which flines intersect the buffered pt 1. I suspect COMID == 19718419 belongs to both sets, which is why is gets returned in duplicate.
What do you think, does that seem plausible?

hmmm... I see what you mean now. I'll have to step through the code and see where the duplication is coming in I guess.

The issue seems to be stemming from the intersection returning a mix of geometry types.

I think this is down to the fact that the result of the current code is splitting geometries into multi-geometries and mucking things up.


Inspector: Update only once while many system changes occurs

d9aeeab1193bf087478c501174c667e95a984d41

https://github.com/feenkcom/gtoolkit-utility/commit/d9aeeab1193bf087478c501174c667e95a984d41

When I compile a bunch of methods, I'm no longer getting an update for every method change.


Project Setup

Initial Technologies used: NodeJS, MongoDB, ExpressJS.
Code setup with basic Node architecture(MC).
MongoDB integration into the code.
Building proper servers.
Modularization.

Task completed.
MongoDB integrated and debugger(ndb) setup done. Initial setup is running successfully.


QSCAN command & travis.yml

Hi,
I am using this library for one of my projects and I have implemented the QSCAN command ( I will eventually also implement JSCAN ). I have also added .travis.yml, feel free to merge if you wish.
Thanks for the library
cheers
andrea

Hey, thank you! I'll review this as soon as I can.

:100: Thanks, I'm going to make a couple changes to the travis.yml to accommodate PEP8 checks.


Fix Typo in Cert Fetching Retry Code

Why
There was a typo
How
Removed the typo

:tada: This PR is included in version 1.7.3 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


SSH through tunneling bastian host

Hello,
I would like to use SSH Tunneling through Bastian host together with pdweb.
<IP_ADDRESS>+<IP_ADDRESS>+database
Would this work?

While the basic ssh proxy feature is available in pgweb, there's no way to configure a ssh jump-host (bastion) at this time. You would probably need to configure the local ssh tunnel through your bastion server using your local ssh configuration files. What's your current setup look like ?


scrypt: maxmem ignored when N >= 65536**r

Originally reported at https://github.com/nodejs/node/issues/29686 against openssl 1.1.1c.
EVP_PBE_scrypt() does not seem to respect the maxmem parameter under certain conditions. The bug report mentions N=2**18, p=8, r=1.
To my understanding, the canonical scrypt algorithm's memory usage is roughly 128*N*p*r. Thus, the parameters should work when e.g maxmem=300 * 2**20 because 128 * 2**18 * 8 * 1 equals 256 * 2**20.
Instead however they are rejected with EVP_R_MEMORY_LIMIT_EXCEEDED because of this check:
https://github.com/openssl/openssl/blob/7c2a981ff7ca721a85687dcd972d54361434806f/providers/default/kdfs/scrypt.c#L382-L387
I'm unsure whether this is intentional or an oversight, hence why I'm filing this issue. If this is intentional, a note in the man page might be good.
My apologies if it's been discussed before. I searched the bug tracker and openssl-users@ but came up empty-handed.

Oh, and if I can interject a side question: am I right that openssl uses 128*p*r + 128*(2+N)*r rather than the canonical equation?
I ask because that's the minimum number of bytes needed to compute e.g. N=2**18, p=8, r=2.

This isn't a bug.
According to the RFC:

The CPU/Memory cost parameter N ("costParameter") must be larger than 1, a power of 2, and less than 2^(128 * r / 8).

2^(128 * 1 / 8) = 65536.  Your N is larger than this, so the fail is mandated.


Fix: adding topology strategy to addons to be valid only for the yaml type.

Description of your changes
To achieve a completely white-box definition of cue type addons, adding topology strategy to addons to be valid only for the yaml type.
So when determining whether to attach the policy, to consider whether the addon was generated from a cue file.

Fixes #5932
I have:

[x] Read and followed KubeVela's contribution process.
[ ] Related Docs updated properly. In a new feature or configuration option, an update to the documentation is necessary.
[x] Run make reviewable to ensure this PR is ready for review.
[ ] Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested

Special notes for your reviewer

I think this check logic shoud put before the runtime check because the template type's priority is higher than the runtime addon.
also, there's no need to actually construct a template cue in the unit test, because we're just checking to see if it's data is empty and don't care about the contents of the file.
And I'm working on this issue(https://github.com/kubevela/kubevela/issues/5932) myself, so you don't need to follow up it.

I think this check logic shoud put before the runtime check because the template type's priority is higher than the runtime addon. also, there's no need to actually construct a template cue in the unit test, because we're just checking to see if it's data is empty and don't care about the contents of the file. And I'm working on this issue(#5932) myself, so you don't need to follow up it.

OK, I get it.


Added the NestedRelation

NestedRelation can be used for eager-loading of multi-level related entities.
Usage example for Post Entity:
namespace Entity;

use Spot\EntityInterface as Entity;
use Spot\MapperInterface as Mapper;

class Post extends \Spot\Entity
{
    protected static $table = 'posts';

    public static function fields()
    {
        return [
            'id'           => ['type' => 'integer', 'autoincrement' => true, 'primary' => true],
            'user_id'      => ['type' => 'integer', 'required' => true],
            'title'        => ['type' => 'string', 'required' => true],
            'body'         => ['type' => 'text', 'required' => true],
            'status'       => ['type' => 'integer', 'default' => 0, 'index' => true],
            'date_created' => ['type' => 'datetime', 'value' => new \DateTime()]
        ];
    }

    public static function relations(Mapper $mapper, Entity $entity)
    {
        $userRelation = $mapper->belongsTo($entity, 'Entity\User', 'user_id');
        $profileRelation =  User::relations($mapper, $entity)['profile'];
        return [
            'user' => $userRelation,
            'user.profile' => $mapper->nestedRelation($profileRelation, $userRelation);
        ];
    }
}

And then we can eager load user profiles with posts:
$posts = $posts->all()->with(['user', 'user.profile']);
NestedRelation can be used with any relations and can be multilevel. For example:
$posts = $posts->all()->with([
     'user',
     'user.profile'
     'user.profile.image',
     'comments',
     'comments.user',
     'comments.user.profile',
     'comments.user.profile.image'
]);

Hey guys. Please tell me what do you think about this idea. I use it for one of my projects to avoid lazy-loading of related entities. This can save a lot of db queries. I would like to contribute it to Spot2.
If you like the idea I can continue working on it and improve it further.

I like it! Somebody from the team will review it as soon as possible.

Ok, great. Thanks.

Hey guys,
I added some tests to the new relation.
Also I changed the way of adding nested relations to entities. So actually now there is no need to define them inside entities. Nested relations will be created automatically when added with 'with' method. So the only things to ensure is that all the parent levels are loaded and the target relation exists. For example:
$posts->all()->with(['user_comments', 'user_comments.user']);
For this case Post entity should have a 'user_comments' relation and UserComment entity should have 'user' relation. That will be enough for the above code to work.
I can improve it further if there is a need.
Thanks.


Deleted unnecessary code comments

Notes for Reviewers
This PR fixes #
Deleted unnecessary code comments
Signed commits

[✅] Yes, I signed my commits.

We're going to stay true to my word: https://github.com/meshery/meshery/pull/7217#issuecomment-1448260683

Sure!! @leecalcote... Didn't read the comment due to office work. I'll make sure to do open tasks.


cdot function notation fails to macro expand properly when there is ambiguous syntax

Description
If ambiguous notation appears inside a cdot function, then the function can end up with more arguments than expected since expandCDot? collects cdots from each alternative in choice nodes.
Context
Appeared on Zulip
Steps to Reproduce
In the following, the cdot function should have one argument, but it ends up with two:
def tag (_ : α) (y : α) := y
notation "f" x => tag 1 x
notation "f" x => tag "2" x
#check (String.length <| f ·)
-- fun x x_1 => (f x).length : (x : String) → ?m.4972 x → Nat

Versions
nightly 7/25/2024
Impact
Add :+1: to issues you consider important. If others are impacted by this issue, please ask them to add :+1: to it.

A way expandCDot? could work is to name the variables x_1, x_2, x_3, sequentially all in the same new macro scope, and then when coming across choice nodes, it saves the state, expands each alternative, and checks that each alternative creates the same number of arguments. The state could be reduced to a single Nat.
I think this would improve how (· + ·) looks when expanded, which would be fun x_1 x_2 => x_1 + x_2 rather than fun x x_1 => x + x_1.
Something that could simplify the implementation would be to do a pre-check to count arguments.

Draft implementation at #4833


CLI Create command may cause un-replaced keys contained in broker.xml

As HashMap does not supply predictable iteration order,  the variable
"HashMap filters = new HashMap()" defined in
org.apache.activemq.artemis.cli.commands.Create is dangerous, some keys
,let's say '${cluster-password},  may be put before '${cluster-security-settings}'.
as a result, '${cluster-password}' may
appear in broker.xml, without replacing the hash map key '${cluster-password}'
to user defined value.
I use LinkedHashMap to resolve this problem.
Please check it.

Can there be a unit test to a) prove the fix b) avoid future regression

umm.. I think it's hard for me to make a unit test, as it is related with OS and java version.
This problem occurs on my Cent OS server but do not occur on Windows 7.
@michaelandrepearce
Would you please tell me some tips for making such type of unit test?
thanks

Suggest trying to make a test that re-creates the issue first without the code change.
CI builds occur on linux boxes, so i think it can be assumed it would be verified if you got the test, that a regression hasn't occured.

I really want to understand the issue here... There shouldn't be any inter-dependency between the keys and values.. That is.. there shouldn't be any values being replaced as ${VARIABLES} at all..
is that something you're doing? like setting the password as another variable?
Can you provide some context on how you made it fail? I don't want to simply accept the PR without understanding what's going on.. as I could be hiding a worse issue.

One real issue that I see is that reading the security settings is not applying the filters:
filters.put("${cluster-security.settings}", readTextFile(ETC_CLUSTER_SECURITY_SETTINGS_TXT));

it should be:
filters.put("${cluster-security.settings}", applyFilters(readTextFile(ETC_CLUSTER_SECURITY_SETTINGS_TXT), filters);

Please, look at #1303... this should fix the real issue here.

@clebertsuconic

One real issue that I see is that reading the security settings is not applying the filters
yes, this is the real issue

:)


MAPL 2 with HPE MPT cannot run at NCCS

So, this is not fun. To wit, I was running two C720 jobs at NAS to compare MPT and HPC-X. The HPC-X job ran just fine, but the MPT job crashed:
 NOT using buffer I/O for file: tavg2d_aer_x.rcx
 NOT using buffer I/O for file: tavg3d_aer_p.rcx

 Freq: 060000  Dur: 060000  TM:   -1  Collection: geosgcm_prog
MPT ERROR: Rank 1535(g:1535) received signal SIGSEGV(11).
        Process ID: 60601, Host: r901i1n25, Program: /nobackupp2/mathomp4/Experiments/stock-gcm-MAPL20-2019Dec16-1day-c720/scratch/GEOSgcm.x
        MPT Version: HPE MPT 2.17  11/30/17 08:08:29

And it was reproducible. But, I found that if I edited gcm_run.j and changed the MPT environment variables from the "default":
   setenv MPI_COLL_REPRODUCIBLE
   setenv SLURM_DISTRIBUTION block

   setenv MPI_XPMEM_ENABLED no
   setenv SUPPRESS_XPMEM_TRIM_THRESH 1
   setenv MPI_NUM_MEMORY_REGIONS 0

   #setenv MPI_DISPLAY_SETTINGS 1
   #setenv MPI_VERBOSE 1

   setenv MPI_COMM_MAX  1024
   setenv MPI_GROUP_MAX 1024
   setenv MPI_BUFS_PER_PROC 256

   setenv MPI_IB_TIMEOUT 23

   unsetenv MPI_MEMMAP_OFF
   unsetenv MPI_NUM_MEMORY_REGIONS
   setenv MPI_XPMEM_ENABLED yes

to this:
   setenv MPI_COLL_REPRODUCIBLE
   setenv SLURM_DISTRIBUTION block

   unsetenv MPI_MEMMAP_OFF
   unsetenv MPI_NUM_MEMORY_REGIONS
   setenv MPI_XPMEM_ENABLED yes

it worked. From previous MPT MAPL 2 work, we found that the last three were needed with MPT to get one-sided communication to work.
But any MPT issue makes me afraid for NCCS. So on NCCS I ran two C720 jobs, one with the "default" options and one trimmed down.
Both failed.
The "default" C720 run died with:
 NOT using buffer I/O for file: tavg2d_aer_x.rcx
 NOT using buffer I/O for file: tavg3d_aer_p.rcx

 Freq: 060000  Dur: 060000  TM:   -1  Collection: geosgcm_prog
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image              PC                Routine            Line        Source
GEOSgcm.x<PHONE_NUMBER>61CB2D  Unknown               Unknown  Unknown
libpthread-2.11.3  00002AAAB4E59850  Unknown               Unknown  Unknown
libesmf_fullylink  00002AAAB15815D1  _ZN5ESMCI2DD12acc     Unknown  Unknown
libesmf_fullylink  00002AAAB157EA06  _ZN5ESMCI27sparse     Unknown  Unknown
libesmf_fullylink  00002AAAB157B23C  _ZN5ESMCI5Array18     Unknown  Unknown
libesmf_fullylink  00002AAAB157A1AA  _ZN5ESMCI5Array17     Unknown  Unknown
libesmf_fullylink  00002AAAB1558E1A  c_esmc_arraysmmst     Unknown  Unknown
libesmf_fullylink  00002AAAB1EC8665  esmf_arraymod_mp_     Unknown  Unknown
libesmf_fullylink  00002AAAB21E4D81  esmf_fieldsmmmod_     Unknown  Unknown
GEOSgcm.x<PHONE_NUMBER>218673  mapl_esmfregridde        1460  MAPL_EsmfRegridder.F90
GEOSgcm.x<PHONE_NUMBER>2198CB  mapl_esmfregridde        1308  MAPL_EsmfRegridder.F90
GEOSgcm.x<PHONE_NUMBER>FEB0E6  mapl_abstractregr         963  MAPL_AbstractRegridder.F90
GEOSgcm.x<PHONE_NUMBER>01FE91  mapl_regridderman         268  MAPL_RegridderManager.F90
GEOSgcm.x<PHONE_NUMBER>17BA61  mapl_newcfiomod_m         134  MAPL_newCFIO.F90
GEOSgcm.x<PHONE_NUMBER>1CA795  mapl_historygridc        2339  MAPL_HistoryGridComp.F90
libesmf_fullylink  00002AAAB1938D81  _ZN5ESMCI6FTable1     Unknown  Unknown
libesmf_fullylink  00002AAAB193C68F  ESMCI_FTableCallE     Unknown  Unknown
libesmf_fullylink  00002AAAB1E00215  _ZN5ESMCI2VM5ente     Unknown  Unknown

the first time and actually stalled the job (it took 2 hours of allocation). I then re-submitted and got:
 Freq: 060000  Dur: 060000  TM:   -1  Collection: geosgcm_prog
 create_route_handle        1463
 initialize_subclass        1309
pe=00649 FAIL at line=00964    MAPL_AbstractRegridder.F90               <status=545>
 MAPL_RegridderManager_private::make_regridder_from_grids         269
pe=00649 FAIL at line=00135    MAPL_newCFIO.F90                         <status=545>
pe=00649 FAIL at line=02340    MAPL_HistoryGridComp.F90                 <status=545>
pe=00649 FAIL at line=01802    MAPL_Generic.F90                         <needs informative message>
pe=00649 FAIL at line=00608    MAPL_CapGridComp.F90                     <status=1>
pe=00649 FAIL at line=00556    MAPL_CapGridComp.F90                     <status=1>
pe=00649 FAIL at line=00827    MAPL_CapGridComp.F90                     <status=1>
pe=00649 FAIL at line=00431    MAPL_Cap.F90                             <status=1>
pe=00649 FAIL at line=00307    MAPL_Cap.F90                             <status=1>
pe=00649 FAIL at line=00149    MAPL_Cap.F90                             <status=1>
pe=00649 FAIL at line=00127    MAPL_Cap.F90                             <status=1>
pe=00649 FAIL at line=00032    GEOSgcm.F90                              <status=1>
MPT ERROR: Rank 649(g:649) is aborting with error code 106949120.
        Process ID: 10112, Host: borgk024, Program: /gpfsm/dnb02/mathomp4/Experiments/stock-gcm-MAPL20-2019Dec17-1day-c720-MPT-AllOptions/scratch/GEOSgcm.x
        MPT Version: HPE MPT 2.17  11/30/17 07:45:32

Now the job with the trimmed down options. First run:
 Freq: 060000  Dur: 060000  TM:   -1  Collection: geosgcm_prog
 create_route_handle        1463
 initialize_subclass        1309
pe=00650 FAIL at line=00964    MAPL_AbstractRegridder.F90               <status=545>
 MAPL_RegridderManager_private::make_regridder_from_grids         269
pe=00650 FAIL at line=00135    MAPL_newCFIO.F90                         <status=545>
pe=00650 FAIL at line=02340    MAPL_HistoryGridComp.F90                 <status=545>
pe=00650 FAIL at line=01802    MAPL_Generic.F90                         <needs informative message>
pe=00650 FAIL at line=00608    MAPL_CapGridComp.F90                     <status=1>
pe=00650 FAIL at line=00556    MAPL_CapGridComp.F90                     <status=1>
pe=00650 FAIL at line=00827    MAPL_CapGridComp.F90                     <status=1>
pe=00650 FAIL at line=00431    MAPL_Cap.F90                             <status=1>
pe=00650 FAIL at line=00307    MAPL_Cap.F90                             <status=1>
pe=00650 FAIL at line=00149    MAPL_Cap.F90                             <status=1>
pe=00650 FAIL at line=00127    MAPL_Cap.F90                             <status=1>
pe=00650 FAIL at line=00032    GEOSgcm.F90                              <status=1>

and second run:
 NOT using buffer I/O for file: tavg2d_aer_x.rcx
 NOT using buffer I/O for file: tavg3d_aer_p.rcx

 Freq: 060000  Dur: 060000  TM:   -1  Collection: geosgcm_prog
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image              PC                Routine            Line        Source
GEOSgcm.x<PHONE_NUMBER>61CB2D  Unknown               Unknown  Unknown
libpthread-2.11.3  00002AAAB4E59850  Unknown               Unknown  Unknown
libesmf_fullylink  00002AAAB15815D1  _ZN5ESMCI2DD12acc     Unknown  Unknown
libesmf_fullylink  00002AAAB157EA06  _ZN5ESMCI27sparse     Unknown  Unknown
libesmf_fullylink  00002AAAB157B23C  _ZN5ESMCI5Array18     Unknown  Unknown
libesmf_fullylink  00002AAAB157A1AA  _ZN5ESMCI5Array17     Unknown  Unknown
libesmf_fullylink  00002AAAB1558E1A  c_esmc_arraysmmst     Unknown  Unknown
libesmf_fullylink  00002AAAB1EC8665  esmf_arraymod_mp_     Unknown  Unknown
libesmf_fullylink  00002AAAB21E4D81  esmf_fieldsmmmod_     Unknown  Unknown
GEOSgcm.x<PHONE_NUMBER>218673  mapl_esmfregridde        1460  MAPL_EsmfRegridder.F90
GEOSgcm.x<PHONE_NUMBER>2198CB  mapl_esmfregridde        1308  MAPL_EsmfRegridder.F90
GEOSgcm.x<PHONE_NUMBER>FEB0E6  mapl_abstractregr         963  MAPL_AbstractRegridder.F90
GEOSgcm.x<PHONE_NUMBER>01FE91  mapl_regridderman         268  MAPL_RegridderManager.F90
GEOSgcm.x<PHONE_NUMBER>17BA61  mapl_newcfiomod_m         134  MAPL_newCFIO.F90
GEOSgcm.x<PHONE_NUMBER>1CA795  mapl_historygridc        2339  MAPL_HistoryGridComp.F90
libesmf_fullylink  00002AAAB1938D81  _ZN5ESMCI6FTable1     Unknown  Unknown
libesmf_fullylink  00002AAAB193C68F  ESMCI_FTableCallE     Unknown  Unknown

Since both jobs both encountered both tracebacks, I'm forced to conclude there is a fundamental fault somewhere in MAPL. Hopefully our more knowledgeable @GEOS-ESM/mapl-team members can see if there is a commonality between the two crashes.

Found another odd case. C180 with MPT. Running with the oserver "on", I get a consistent crash:
 Using parallel IO for reading file: sst.data
 Using parallel IO for reading file: sst.data
 Integer*4 Resource Parameter: INTERPOLATE_OCEAN_ICE_CURRENTS:0
 AGCM Date: 2015/04/14  Time: 21:07:30  Throughput(days/day)[Avg Tot Run]:   41.1   41.1  106.0  TimeRemaining(Est) 000:34:51   25.9% Memory Committed
pe=00035 FAIL at line=03413    DynCore_GridCompMod.F90                  <needs informative message>
pe=00035 FAIL at line=01802    MAPL_Generic.F90                         <needs informative message>
SUPERDYNAMICSRun                               708
pe=00035 FAIL at line=01802    MAPL_Generic.F90                         <needs informative message>
AGCMRun                                       2150
pe=00035 FAIL at line=01802    MAPL_Generic.F90                         <needs informative message>
GCMRun                                        1460
pe=00035 FAIL at line=01802    MAPL_Generic.F90                         <needs informative message>
pe=00035 FAIL at line=01055    MAPL_CapGridComp.F90                     <status=1>
pe=00035 FAIL at line=01000    MAPL_CapGridComp.F90                     <status=1>
pe=00035 FAIL at line=00722    MAPL_CapGridComp.F90                     <status=1>
pe=00035 FAIL at line=00841    MAPL_CapGridComp.F90                     <status=1>
pe=00035 FAIL at line=00433    MAPL_Cap.F90                             <status=1>
pe=00035 FAIL at line=00307    MAPL_Cap.F90                             <status=1>
pe=00035 FAIL at line=00149    MAPL_Cap.F90                             <status=1>
pe=00035 FAIL at line=00127    MAPL_Cap.F90                             <status=1>
pe=00035 FAIL at line=00032    GEOSgcm.F90                              <status=1>
MPT ERROR: Rank 35(g:35) is aborting with error code 106949120.
	Process ID: 12701, Host: borgk092, Program: /gpfsm/dnb02/mathomp4/Experiments/stock-gcm-MAPL20-2019Dec17-1day-c180-MPT-AllOptions-Oserver/scratch/GEOSgcm.x
	MPT Version: HPE MPT 2.17  11/30/17 07:45:32

Running with the oserver "off" (i.e., no extra nodes), one time it ran successfully, one time it died with:
 Using parallel IO for reading file: sst.data
 Using parallel IO for reading file: sst.data
 Integer*4 Resource Parameter: INTERPOLATE_OCEAN_ICE_CURRENTS:0
 AGCM Date: 2015/04/14  Time: 21:07:30  Throughput(days/day)[Avg Tot Run]:   33.4   33.4   71.3  TimeRemaining(Est) 000:42:57   25.9% Memory Committed
pe=00035 FAIL at line=03413    DynCore_GridCompMod.F90                  <needs informative message>
pe=00035 FAIL at line=01802    MAPL_Generic.F90                         <needs informative message>
SUPERDYNAMICSRun                               708
pe=00035 FAIL at line=01802    MAPL_Generic.F90                         <needs informative message>
AGCMRun                                       2150
pe=00035 FAIL at line=01802    MAPL_Generic.F90                         <needs informative message>
GCMRun                                        1460
pe=00035 FAIL at line=01802    MAPL_Generic.F90                         <needs informative message>
pe=00035 FAIL at line=01055    MAPL_CapGridComp.F90                     <status=1>
pe=00035 FAIL at line=01000    MAPL_CapGridComp.F90                     <status=1>
pe=00035 FAIL at line=00722    MAPL_CapGridComp.F90                     <status=1>
pe=00035 FAIL at line=00841    MAPL_CapGridComp.F90                     <status=1>
pe=00035 FAIL at line=00433    MAPL_Cap.F90                             <status=1>
pe=00035 FAIL at line=00307    MAPL_Cap.F90                             <status=1>
pe=00035 FAIL at line=00149    MAPL_Cap.F90                             <status=1>
pe=00035 FAIL at line=00127    MAPL_Cap.F90                             <status=1>
pe=00035 FAIL at line=00032    GEOSgcm.F90                              <status=1>
MPT ERROR: Rank 35(g:35) is aborting with error code 106949120.

Same as above.

I can't seem to trigger anything at C48 or C90 with MPT.

I will setup and run a similar series of runs with Intel MPI: C48->C720 with and without oserver on separate node(s).

Intel MPI seems happy at all resolutions. Looks like something to do with MPT.

The C180 MPT error is a bit familiar:
https://github.com/GEOS-ESM/FVdycoreCubed_GridComp/blob/7f6b75b38b85a5e7a9f79c6fa797e2c0ac4343f2/DynCore_GridCompMod.F90#L3409-L3421
@bena-nasa was/is this the place the CTM was failing?

If anyone wishes to test, I am using this:
~mathomp4/GitG5Modules/SLES11/6.0.3/g5_modules.intel1805.mpt

as the g5_modules. It just swaps Intel MPI for MPT at NCCS.

I will discuss with @bena-nasa later this morning.    Unfortunately, the fact that it appears to work for one flavor of MPI gives little traction in terms of finding a bug in the code.     Just one more of a long-line of odd symptoms that suggest something odd in terms of memory corruption that is sensitive to MPI, NetcCDF, etc.