Concurrent appends to the same table using HiveCatalog lead to deadlock in the Metastore

I've noticed that, appending files the the same table from multiple threads takes a very large amount of time. The problem is more evident if the number of threads are more. From the logs it seems there's deadlock in the Metastore.
Below is the log I'm pasting from the test case.

[main] INFO org.apache.iceberg.BaseMetastoreTableOperations - Refreshing table metadata from new version: file:/var/folders/dg/xnsw71ls347dkft4qr284zr4000bt7/T/hive3179686150067127808/hivedb.db/tbl/metadata/00000-5001916d-81a9-408c-ad9e-9cdec89090e1.metadata.json
[main] INFO org.apache.iceberg.BaseMetastoreTableOperations - Refreshing table metadata from new version: file:/var/folders/dg/xnsw71ls347dkft4qr284zr4000bt7/T/hive3179686150067127808/hivedb.db/tbl/metadata/00000-5001916d-81a9-408c-ad9e-9cdec89090e1.metadata.json
Starting append 0
Starting append 1
Starting append 2
Starting append 3
Starting append 4
Starting append 5
Starting append 6
Starting append 7
Starting append 8
Starting append 9
2019-08-08 11:08:59,302 WARN  [pool-3-thread-2] txn.TxnHandler (TxnHandler.java:checkRetryable(982)) - Deadlock detected in checkLock(CheckLockRequest(lockid:7) ). Will wait 200ms try again up to 10 times.
2019-08-08 11:09:19,518 WARN  [pool-3-thread-3] txn.TxnHandler (TxnHandler.java:checkRetryable(982)) - Deadlock detected in checkLock(CheckLockRequest(lockid:6) ). Will wait 200ms try again up to 10 times.
2019-08-08 11:09:39,743 WARN  [pool-3-thread-3] txn.TxnHandler (TxnHandler.java:checkRetryable(982)) - Deadlock detected in checkLock(CheckLockRequest(lockid:6) ). Will wait 400ms try again up to 9 times.
[pool-6-thread-7] INFO org.apache.iceberg.SnapshotProducer - Committed snapshot<PHONE_NUMBER>032677660 (MergeAppend)

The testcase is published here: https://github.com/apache/incubator-iceberg/compare/master...rdsr:hivecatalog_deadlock?expand=1
Most likely not a bug in Iceberg, but creating ticket for tracking purposes

I'm not sure this is a deadlock. It might be that the concurrent appends keep failing and retrying and one is always holding the lock attempting to commit.

HMS has an internal mechanism for recovering in such cases, which might be slow in highly concurrent environments. One point that concerns me a bit is that we constantly send check lock requests without any time between requests, which puts quite some pressure on HMS (see HiveTablesOperations$acquireLock).
The following succeeds in 2-20 seconds after adding a sleep between check lock requests (the test is identical to what @rdsr had, except table properties):
    Table icebergTable = catalog.loadTable(TABLE_IDENTIFIER);

    icebergTable.updateProperties()
        .set(TableProperties.COMMIT_NUM_RETRIES, "10")
        .set(TableProperties.COMMIT_MIN_RETRY_WAIT_MS, "50")
        .set(TableProperties.COMMIT_MAX_RETRY_WAIT_MS, "10")
        .commit();

    String fileName = UUID.randomUUID().toString();
    DataFile file = DataFiles.builder(icebergTable.spec())
        .withPath(FileFormat.PARQUET.addExtension(fileName))
        .withRecordCount(2)
        .withFileSizeInBytes(0)
        .build();

    // If we reduce the number of threads, the problem is less evident.
    ExecutorService executorService = MoreExecutors.getExitingExecutorService(
        (ThreadPoolExecutor) Executors.newFixedThreadPool(10));

    Consumer<Integer> t = (idx) -> {
      long startTime = System.currentTimeMillis();
      System.out.println("Starting append " + idx);
      icebergTable.newAppend().appendFile(file).commit();
      System.out.println("Time taken for append " + idx + "in thread " + Thread.currentThread() + " = "
          + (System.currentTimeMillis() - startTime + "ms"));
    };

    for (int i = 0; i < 10; i++) {
      int finalI = i;
      executorService.submit(() -> t.accept(finalI));
    }

    executorService.shutdown();
    executorService.awaitTermination(1, TimeUnit.MINUTES);

Below is the part in HiveTableOperations I was talking about:
    while (state.equals(LockState.WAITING)) {
      lockResponse = metaClients.run(client -> client.checkLock(lockId));
      state = lockResponse.getState();
    }

@rdblue @rdsr what do you guys think about adding a timeout between check lock requests?

Yeah, definitely. Good catch.

Sounds like a blocker to me.

I'll follow up with a PR then


Sort localizable strings for English localization

This fixes the localicalization strings sorting order

@kbhawkey @shannonxtreme plzz review and merge this

/retitle Sort localizable strings for English localization

@kbhawkey @sftim @shannonxtreme review and merge please..i have done the recommendee changes

Duplicate of #39013.
/close


Form wont load when using widgets with own script media

It seems that if you use a widget such as a datetimepicker, select2 etc that loads its own scripts with the field then the form wont load at all. I know this is a common problem with modal content but  this seems to destroy the form completely. Any ideas/workaround?

@7wonders hello, could you give me a simple example so I could try it myself?

@FZambia I have tried with a couple of different widgets. One I use is bootstrap datetimepicker and another is a custom widget with Selective.js. Here is a simple example of the form with DateTimePicker but I believe its a problem with any widget that includes its own js files.
class TestForm(UserKwargModelFormMixin, ModelForm):
    created = DateTimeField(
        required=True,
        widget=DateTimePicker(options={
            "calendarWeeks": True,
            "showClear": True,
            "showClose": True,
            "format": "YYYY-MM-DD HH:mm:ss",
            "maxDate": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
            "pickTime": True,
        }))

    class Meta:
        model = Whatever

    def __init__(self, *args, **kwargs):
        super(TestForm, self).__init__(*args, **kwargs)
        self.helper = FormHelper()
        self.helper.form_id = 'test-form'
        self.helper.form_class = 'form-inline'
        self.helper.form_method = 'post'
        self.helper.layout = Layout(
            Field('created'),
        )

Removing the widget basically gets the form working straight away.

So I removed form.media from form.html and into the base template:
{% load crispy_forms_tags %}
<div id="form-container">
    <form role="form" class="form" method="POST" enctype="multipart/form-data" action="">
        {% if form.helper %}
            {% crispy form form.helper %}
        {% else %}
            {% crispy form %}
        {% endif %}
    </form>
</div>

and then it starts loading intermittently which is really strange. So with the form.media it doesn't load at all. Without it loads every few times and the form is rendered correctly but datetimepickers etc still won't work which is a common issue with modal. Now just need to figure out how to get it to load every time as expected!

Found it! Very simple fix. Pull coming your way.

basically any scripts that are included with a widget get evaluated when using html for dataType. Not a problem if its a nice small bit of script but causes a brain fart when widgets include src links to momentjs, selective, jquery etc etc. Using text instead is not a problem in this case because the scripts and html get included properly.

hm, I tried to reproduce it with bootstrap datetimepicker  - and everything worked without changes.
Created branch with demo project using  bootstrap_datetime widget - https://github.com/django-fm/django-fm/tree/bootstrap_datetime - it works with Django 1.7 and django-fm 0.2.4

Does the picker actually work also? I am trying now on django 1.10 and the form works which is an improvement from my setup but the picker and selectize widget im testing with does not.

Yes, picker also works - I can select date and it appears in input. Could you try to run that demo?

It seems there was some conflicting scripts loading on the page that were obviously bottling everything up during load. Strangely they were not throwing any errors in chrome console every time but then one popped up which helped me figure out the problem. Sorry for wasting your time!


Several IDP architecture changes

Part of the changes needed for #722. But I realized an entire PR for that would probably be too much at the sime time so already pushing some intermediate changes.
First commit: moves the logic for the hidden field in the reset password page to the html template instead of the server logic. This page was the only one that had a different logic for GET/POST requests, making it more difficult to have a general architecture change for the handlers. With this change it's the same as all the other ones in that no extra server logic is required for the GET request.
Second commit: The InitialInteractionHandler is the class that gets the request that get sent to /idp/. It then has to decide which actor has to handle the request based on the session parameters. With this change it does that by sending a 302 with the correct URL instead of linking to the relevant handler directly. This way its behaviour is more similar to the other handlers and hierarchically it fits more next to the other ones.

Okay with me, but the client-side code might make things more brittle than they need to be.

This page was the only one that had a different logic for GET/POST requests

How about we keep the same logic, but POST also looks for fields in the query string? Then we don't have to copy them over on the client.
(Note my remark about not having set action.)

I just realized that we are going to need some handlers for handling GET requests (see #827) so there's no real point anymore in doing this reset password commit. I might still keep it and use the no-action solution without query parameter since that still seems cleaner to me.


Represent shortref phrase "ISO 19115-1:2014"

See https://github.com/riboseinc/isodoc-models/issues/29

ISO is bibdata/contributor[role@type="publisher"]/organization/name = ISO. In our XML:
  <contributor>
    <role type="publisher"/>
    <organization>
      <name>ISO</name>
    </organization>
  </contributor>

ISO/IEC are represented by multiple instances of bibdata/contributor[role@type="publisher"]

For JSON serialisation, if you want to keep the full XML complexity, that will be somewhat tricky; but you could just hardcode contributor[role@type = publisher] as publisher.

JSON serialization we're going to use JSON-LD so maybe not so tough?
When you said "hardcode" that line is an XPATH, are you suggesting (1) we store XML (2) we use an XPATH on a method call to return the publisher object?

I merely meant we could shortcut the structure in Json, so that publisher contributors are encoded as publisher.
But really, up to you guys. I'll parse what you give me.


BDD tests failures in CI

This bug is a bit hard to reproduce on my local setup.. however, it's ocurring in CI relatively frequently.
See this build log: https://github.com/hyperledger/aries-framework-go/runs/1101967510
Notes:

The JS tests are using different fixtures. In particular, they're using mem storage
The Go test fixtures do not declare dependencies on other containers

I declared some dependencies on the fixtures (set dependency on the relevant storage container for each agent) and also increased the Go BDD test timeout to 30... still sometimes seeing this error:
  Scenario: did exchange e2e flow using controller api # features/didexchange_e2e_controller.feature:12
    Given "Alice" agent is running on "localhost" port "8081" with controller "https://localhost:8082" # features/didexchange_e2e_controller.feature:13
      Error: Get "https://localhost:8082": dial tcp <IP_ADDRESS>:8082: connect: connection refused

And yet:
$ docker ps | grep alice.agent.example.com
$
$ docker logs alice.agent.example.com
Error: No such container: alice.agent.example.com
$

Ah - the container does exist - I need to search with the container_name:
$ docker logs -f alice.aries.example.com
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
WARNING: ca-cert-ec-key.csr.pem does not contain exactly one certificate or CRL: skipping
WARNING: ca-cert-ec-cacert.srl.pem does not contain exactly one certificate or CRL: skipping
Error: failed to start aries agent rest on port [<IP_ADDRESS>:8082], failed to initialize framework :  default option initialization failed: can't initialize verifaible store : failed to open vc store: failed to create db: Unauthorized: You are not a server admin.
....

excerpt from couchdb logs:
View
[info] 2020-09-11T18:56:05.873319Z nonode@nohost <0.11.0> -------- Application mem3 started on node nonode@nohost
[info] 2020-09-11T18:56:05.873603Z nonode@nohost <0.11.0> -------- Application fabric started on node nonode@nohost
[info] 2020-09-11T18:56:05.924190Z nonode@nohost <0.11.0> -------- Application chttpd started on node nonode@nohost
[info] 2020-09-11T18:56:05.964180Z nonode@nohost <0.11.0> -------- Application couch_index started on node nonode@nohost
[info] 2020-09-11T18:56:05.964253Z nonode@nohost <0.11.0> -------- Application couch_mrview started on node nonode@nohost
[info] 2020-09-11T18:56:05.964320Z nonode@nohost <0.11.0> -------- Application couch_plugins started on node nonode@nohost
[notice] 2020-09-11T18:56:05.969539Z nonode@nohost <0.326.0> -------- chttpd_auth_cache changes listener died because the _users database does not exist. Create the database to silence this notice.
[error] 2020-09-11T18:56:05.969709Z nonode@nohost emulator -------- Error in process <0.327.0> with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,399}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,374}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,403}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,96}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,39}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,198}]},{chttpd_auth_cache,listen_for_changes,1,[{file,"src/chttpd_auth_cache.erl"},{line,145}]}]}

[notice] 2020-09-11T18:56:06.059788Z nonode@nohost <0.82.0> -------- config: [couch_httpd_auth] secret set to 44117e3e0a1d7b7b65002e9ab87903ae for reason nil
[notice] 2020-09-11T18:56:06.060396Z nonode@nohost <0.82.0> -------- config: [couch_httpd_auth] secret set to f244a16201dafa6b3cd27c39491fa629 for reason nil
[info] 2020-09-11T18:56:06.066711Z nonode@nohost <0.11.0> -------- Application couch_replicator started on node nonode@nohost
[info] 2020-09-11T18:56:06.094945Z nonode@nohost <0.11.0> -------- Application couch_peruser started on node nonode@nohost
[notice] 2020-09-11T18:56:06.130826Z nonode@nohost <0.308.0> 77e9f629ee couchdb.example.com:5984 <IP_ADDRESS> undefined POST /_session 200 ok 182
[notice] 2020-09-11T18:56:06.131231Z nonode@nohost <0.309.0> 1207332a92 couchdb.example.com:5984 <IP_ADDRESS> undefined POST /_session 200 ok 183
[info] 2020-09-11T18:56:06.137224Z nonode@nohost <0.11.0> -------- Application ddoc_cache started on node nonode@nohost
[notice] 2020-09-11T18:56:06.146340Z nonode@nohost <0.311.0> ed31e4f7df couchdb.example.com:5984 <IP_ADDRESS> admin HEAD /_up 200 ok 14
[notice] 2020-09-11T18:56:06.146544Z nonode@nohost <0.310.0> 9748e557b4 couchdb.example.com:5984 <IP_ADDRESS> undefined HEAD /_up 200 ok 14
[notice] 2020-09-11T18:56:06.148517Z nonode@nohost <0.310.0> fa6dc4068e couchdb.example.com:5984 <IP_ADDRESS> undefined PUT /alice_verifiable 401 ok 1
[error] 2020-09-11T18:56:06.170129Z nonode@nohost <0.311.0> 92629fa8f7 Request to create N=3 DB but only 1 node(s)
[info] 2020-09-11T18:56:06.175613Z nonode@nohost <0.11.0> -------- Application dreyfus started on node nonode@nohost
[notice] 2020-09-11T18:56:06.239619Z nonode@nohost <0.311.0> 92629fa8f7 couchdb.example.com:5984 <IP_ADDRESS> admin PUT /erin_verifiable 201 ok 92
[error] 2020-09-11T18:56:06.241762Z nonode@nohost <0.312.0> 7adb09d71a Request to create N=3 DB but only 1 node(s)
[info] 2020-09-11T18:56:06.251640Z nonode@nohost <0.11.0> -------- Application global_changes started on node nonode@nohost
[info] 2020-09-11T18:56:06.251694Z nonode@nohost <0.11.0> -------- Application jiffy started on node nonode@nohost
[notice] 2020-09-11T18:56:06.254550Z nonode@nohost <0.312.0> 7adb09d71a couchdb.example.com:5984 <IP_ADDRESS> admin PUT /erin_kmsdb 201 ok 14
[error] 2020-09-11T18:56:06.258825Z nonode@nohost <0.314.0> 76bea59fd2 Request to create N=3 DB but only 1 node(s)
[notice] 2020-09-11T18:56:06.273688Z nonode@nohost <0.314.0> 76bea59fd2 couchdb.example.com:5984 <IP_ADDRESS> admin PUT /erin_peer 201 ok 16
[error] 2020-09-11T18:56:06.276178Z nonode@nohost <0.315.0> 6355196f03 Request to create N=3 DB but only 1 node(s)
[info] 2020-09-11T18:56:06.282158Z nonode@nohost <0.11.0> -------- Application jwtf started on node nonode@nohost

There's the 401 while attempting to create Alice's verifiable store.

Calling the _session endpoint is not enough:
View
[notice] 2020-09-11T19:54:41.771458Z nonode@nohost <0.82.0> -------- config: [couch_httpd_auth] secret set to 9d259ff2b2c492fa0aedfe8c80db7c2e for reason nil
[notice] 2020-09-11T19:54:41.773335Z nonode@nohost <0.82.0> -------- config: [couch_httpd_auth] secret set to bd089cf84f1e713998c00f15d9be6883 for reason nil
[notice] 2020-09-11T19:54:41.773971Z nonode@nohost <0.82.0> -------- config: [couch_httpd_auth] secret set to c828f9478987308a4c76a143c935099f for reason nil
[notice] 2020-09-11T19:54:41.798154Z nonode@nohost <0.309.0> a11a309da8 couchdb.example.com:5984 <IP_ADDRESS> undefined POST /_session 200 ok 155
[notice] 2020-09-11T19:54:41.798547Z nonode@nohost <0.310.0> 053a4f97ca couchdb.example.com:5984 <IP_ADDRESS> undefined POST /_session 200 ok 121
[notice] 2020-09-11T19:54:41.798708Z nonode@nohost <0.308.0> 1e3d583e7f couchdb.example.com:5984 <IP_ADDRESS> undefined POST /_session 200 ok 161
[notice] 2020-09-11T19:54:41.811285Z nonode@nohost <0.311.0> 564ad78c61 couchdb.example.com:5984 <IP_ADDRESS> undefined GET /_session 200 ok 2
[notice] 2020-09-11T19:54:41.811295Z nonode@nohost <0.312.0> 5e4dbddfa8 couchdb.example.com:5984 <IP_ADDRESS> undefined GET /_session 200 ok 1
[notice] 2020-09-11T19:54:41.812173Z nonode@nohost <0.313.0> da82b484d0 couchdb.example.com:5984 <IP_ADDRESS> admin GET /_session 200 ok 2
[notice] 2020-09-11T19:54:41.813491Z nonode@nohost <0.311.0> a152c7ecdd couchdb.example.com:5984 <IP_ADDRESS> undefined PUT /alice_verifiable 401 ok 1
[notice] 2020-09-11T19:54:41.813927Z nonode@nohost <0.312.0> 1d9f8c94c1 couchdb.example.com:5984 <IP_ADDRESS> undefined PUT /erin_verifiable 401 ok 1
[error] 2020-09-11T19:54:41.844414Z nonode@nohost <0.313.0> a29f528afb Request to create N=3 DB but only 1 node(s)
[notice] 2020-09-11T19:54:41.912725Z nonode@nohost <0.313.0> a29f528afb couchdb.example.com:5984 <IP_ADDRESS> admin PUT /bob_verifiable 201 ok 99
[error] 2020-09-11T19:54:41.915884Z nonode@nohost <0.314.0> 54e584230f Request to create N=3 DB but only 1 node(s)
[notice] 2020-09-11T19:54:41.928788Z nonode@nohost <0.314.0> 54e584230f couchdb.example.com:5984 <IP_ADDRESS> admin PUT /bob_kmsdb 201 ok 15
[error] 2020-09-11T19:54:41.932135Z nonode@nohost <0.315.0> 3f0559dfc2 Request to create N=3 DB but only 1 node(s)
[notice] 2020-09-11T19:54:41.950262Z nonode@nohost <0.315.0> 3f0559dfc2 couchdb.example.com:5984 <IP_ADDRESS> admin PUT /bob_peer 201 ok 20
[error] 2020-09-11T19:54:41.959225Z nonode@nohost <0.316.0> 31a8c49d7c Request to create N=3 DB but only 1 node(s)

Querying _users was not enough... seems CouchDB sometimes can't create it?
View
[notice] 2020-09-11T20:13:02.846674Z nonode@nohost <0.324.0> -------- chttpd_auth_cache changes listener died because the _users database does not exist. Create the database to silence this notice.
[error] 2020-09-11T20:13:02.846869Z nonode@nohost emulator -------- Error in process <0.2644.0> with exit value:
{database_does_not_exist,[{mem3_shards,load_shards_from_db,"_users",[{file,"src/mem3_shards.erl"},{line,399}]},{mem3_shards,load_shards_from_disk,1,[{file,"src/mem3_shards.erl"},{line,374}]},{mem3_shards,load_shards_from_disk,2,[{file,"src/mem3_shards.erl"},{line,403}]},{mem3_shards,for_docid,3,[{file,"src/mem3_shards.erl"},{line,96}]},{fabric_doc_open,go,3,[{file,"src/fabric_doc_open.erl"},{line,39}]},{chttpd_auth_cache,ensure_auth_ddoc_exists,2,[{file,"src/chttpd_auth_cache.erl"},{line,198}]},{chttpd_auth_cache,listen_for_changes,1,[{file,"src/chttpd_auth_cache.erl"},{line,145}]}]}


docs(README): sync package docs

No Issue
Sync README and related documentation among main repositories
Author Checklist

[x] Ensure you provide a DCO sign-off for your commits using the --signoff option of git commit.
[x] Commits messages follow AP format
[x] Extend the documentation, if necessary
[x] Merging to master from fork:branchname

Coverage remained the same at 89.081% when pulling 994db320485e14e0850e629cd25bc9fa614f8259 on irmerk:irmerk/sync into bdf05d5ba4fa3cf8e07e799ef869dda1264df246 on accordproject:master.


Transformed to OO, logging, auto-detection of modules, single topleve…

The original problem that this PR solves is removing the requirement of multiple Terrafiles. One per sub folder per environment(DEV|STAGING|PROD), if you try and have a single Terrafile at the top level it downloads every module every time.
Using this solution you have one Terrafile at the top level of each environment. You run pterrafile as normal within each subfolder and it only downloads the modules required for that subfolder.
While I was doing the above I took took the opportunity to transform it to OO and add logging.
Its a bit of a break from previous version, a release note would need to be added to explain to existing users the changes they would require to upgrade, but these changes would be reducing all those Terrafiles down to a central one per environment.
If the above is too radical a change I can set up a separate repo with a new name and refer back to this repo as the originator.

Hi @TrickMcCarthy. I think the filtering logic is a bit too specific to how you use Terraform. Too opinionated as you say. This change would break all of our projects that are using python-terrafile.
When we use python-terraform (we've stopped using it in some newer projects) we tend to have a single modules directory with a Terrafile in it. We run python-terrafile once and it downloads all modules for all stacks/environments for the entire project. Then the various stacks/environments have source = "../../../modules/some-module".
Would that not work for you? The benefit of a single Terrafile is that you only download a module once, even when it's used by many different stacks/environments.
If that doesn't suit, I'd be interested to know more about how your project is structured and why.
As for changing it to object-oriented code... I'm sorry but I see this as a change for the worse, with no actual benefit. I prefer simple functions and generally only use classes when dealing with shared/persistent state or want to expose a particular API.

Hi @raymondbutcher ,
Thanks for getting back to me on the PR.  To give some replies to your questions.
Paraphasing here, why not a separate single common module location as the download and thus have single Terrafile?
Answer: This doesn't scale in a nice manner, its fine for small focused projects, but I'm involved in large infrastructure platform that supports 10s of small focused projects. Image 10s of subfolders within each environment(DEV,STAGING,QA,PROD,PERF,PREPROD, etc...) focused on particular technical areas such as vpc, security, monitoring, CI etc..., each sub folder containing subfolders that have specific terraform that contains multiple versioned modules.
When you want to apply a change to one of those subfolders, you do not want pterrafile downloading every single module from each subfolder across the whole environment. Thus with current implementation you have a Terrafile at each subfolder level, so it only refreshes the modules you are going to apply. In this PR you only require one Terrafile at the top of each environment that only downloads the modules required in your subfolder. It makes it easy to see what version each of your modules is currently at, so you know what changes to promote across your environments.
In the future when the underlying cloud vendors resources become more defensive in their ability to create and destroy(they need to destroy or create resources quickly), then all terraform will be auto applied. You would not want everything download under those circumstances either.
Hopefully this explains why I believe there is value in my proposed change.
OO being a change for the worse
Answer
It again comes to scaling. Procedural style for python scripts works fine until it doesn't. My opinion on this is its best to switch to OO style earlier than required rather than later. So I agree, it is early to switch, but perhaps not too early, as it will allow more controlled code structure to develop from this point, rather than a decision being made at a later date. I can see that this project may never grow in functionality to warrant the shift.

Hi again @TrickMcCarthy. You're welcome, and likewise thanks for explaining your use case.
If I understand correctly, your goal is to have one Terrafile that is used by multiple stacks, but you want python-terrafile to only download modules which are used by the stack that you're currently working on, to avoid downloading irrelevant modules. You think having a single Terrafile improves visibility of the module versions in that environment, as opposed to having separate Terrafiles in subdirectories. And not mentioned: having multiple Terrafiles could result in downloading modules multiple times if you switch between subdirectories that require the same modules.
Have I got that right? I'd be happy to explore solutions with you that don't break functionality for existing projects. But of course you're free to fork it if you'd prefer.
Regarding OO... I'm afraid we might have to agree to disagree. I don't think OO is required for larger scale or more complex projects. Functions can be moved into separate modules to maintain a sensible structure.
Let's take a look at one of the methods in your PR:
    def _get_repo_name_from_url(self, url: str) -> str:
        logger.debug("url {}".format(url))
        last_suffix_index = url.rfind(".git")
        last_slash_index = url.rfind("/",0,last_suffix_index)
        if last_suffix_index < 0:
            last_suffix_index = len(url)

        if last_slash_index < 0 or last_suffix_index <= last_slash_index:
            raise Exception("Badly formatted url {}".format(url))

        return url[last_slash_index + 1:last_suffix_index]

This function doesn't really have anything to do with a Terrafile. It doesn't use self. If you wanted to test this, you'd need to create an instance of the Terrafile class and then call this method. Or you could keep it as a function and call it directly. I don't see any reason for this to be part of the Terrafile class.
When using classes, I always use the @classmethod and @staticmethod decorators. They kind of highlight when you're using a class for API purposes only, as opposed to using a class to represent some kind of entity. In this case, most of your methods would use either of those decorators. The class doesn't really represent anything except the program itself (which I suppose is not necessarily a bad thing, it just seems unnecessary).

@raymondbutcher
In regards to OO,  many procedural projects can scale well, its your repo, so your opinion is the one that counts and it's not something worth getting stuck on. I am willing to remove the OO changes. Thanks for pointing out the classmethod, staticmethod decorators, I will utilize them in future for class methods not requiring self or separate external functionality as the case may present themselves.
So, let's discuss a you offered, how to make an acceptable change to meet the requirements outlined above.
if I add a 'parameter flag/config file needs to exist' to enable the opinionated version, so the default behaviour remains the same and existing projects would not be broken when they upgrade. I do not see that working too well. As its duality that would need to be maintained within the existing project. I think this could only work if the feature was called incubating and became default behaviour on next major version change, perhaps based upon user voting.  I think users accept that they need to make adjustments between major version increments.
If I just detect that the correct name matching between Terrafile module names and Terrafile yaml entries is matching, If even one mismatch occurred assume that the original behaviour is required. Thinking about this solution I do not like it, as its changing behaviour unbeknownst to the user, unless some message appeared asking them to confirm the usage of the new behaviour. Then this would become tiresome, so first option would seem better.
Any other thoughts on how it would be acceptably implemented?

I can see the benefit of this, it does seem useful, but I don't want to force users to change their projects. I think trying to force users to have a specific project structure would push most people away from using this tool. I know I would be less likely to use it.
I think it could be OK to put this behind a command line flag and/or have a reserved key in the Terrafile YAML that enables it. I think it could slot in here quite easily:
https://github.com/claranet/python-terrafile/blob/8c502112b00bebc010f0cd9d463252f2ec3fe7a4/terrafile/__init__.py#L93-L100
def update_modules(path):
    terrafile_path = get_terrafile_path(path)
    module_path = os.path.dirname(terrafile_path)
    module_path_name = os.path.basename(os.path.abspath(module_path))

    terrafile = read_terrafile(terrafile_path)

    if THIS_FEATURE_ENABLED:                                 # this
        found_modules = find_used_modules(os.getcwd())       # this
        terrafile = filter_modules(terrafile, found_modules) # this

    for name, repository_details in sorted(terrafile.items()):
        ...

Of course THIS_FEATURE_ENABLED would not be the real name, it probably needs to be a function argument rather than a global variable, etc. Maybe you wouldn't use os.getcwd() there. This is just a rough example to show how it could be made optional without being much of a maintenance burden.


Expose icosahedron faces, base cell mapping, and IJK orientation?

I've been thinking a lot about the various corner cases (no pun intended) the various algos have run into with pentagons where the triangles corners meet.
I think it would be a useful debugging tool (and a useful teaching aide, as well) if the 20 icosahedron faces could be exposed from the API with their coordinates and orientations (perhaps the orientation implicit from the coordinates, first pair being the furthest in the I direction, next furthest in the J, last furthest in the K?).
It would make it easier to see what coordinate orientation changes are required to transition "through" a pentagon and how there are a few different rules depending on which icosahedron you're transitioning from (since as I recall, 5 of the faces have all of their I directions meet at the same point near the north pole, another 5 have the I directions meet at the same point near the south pole, and an equatorial-like ring of 10 triangles have alternating orientations for the I direction, leading to 2 or three sets of icosahedron points when grouped by which orientations for each face are "pointed at" that point.
Functions like that could help with figuring out the h3Line issue, for instance (perhaps this only occurs at some pentagons and not others, and these things could help us figure out why), and could also help out with the occasional bugs that have cropped up across edges in the shared hexagons (should they have two icosahedron face parents)?
Its unclear to me if the H3Index should have a new mode to declare them (perhaps? h3ToParent on a base cell could return a value that way).
Something I've been thinking about since I feel half or more of the issues with the algos hitting pentagons is due to it not being clear in our minds how the coordinate orientations are being changed.

I think it would be a useful debugging tool (and a useful teaching aide, as well) if the 20 icosahedron faces could be exposed from the API with their coordinates and orientations ...

I agree this would be useful as a debugging, development, and teaching tool. I'm unsure of the purpose of exposing it as part of the "public API", since it seems it would not be useful to application developers.

Its unclear to me if the H3Index should have a new mode to declare them (perhaps? h3ToParent on a base cell could return a value that way).

This seems like trying to fit this into an unrelated structure/function because that structure/function already exists.

Something I've been thinking about since I feel half or more of the issues with the algos hitting pentagons is due to it not being clear in our minds how the coordinate orientations are being changed.

I agree orientation changes are one of the trickier parts we have to handle, I would also note that the deleted subsequence in the pentagon cells has come up quite a bit (and appears to be related to odd behavior with h3Line.)

I think it would be a useful debugging tool (and a useful teaching aide, as well) if the 20 icosahedron faces could be exposed from the API with their coordinates and orientations ...

I agree this would be useful as a debugging, development, and teaching tool. I'm unsure of the purpose of exposing it as part of the "public API", since it seems it would not be useful to application developers.

I think there could be some benefits for users that really care about the potential distortions. Because of the Gnomonic projection centered on each icosahedron face the distortions are greatest near the corners of each face and so knowing where on each face a particular hexagon is located can help with analysis that does want to keep data density analyses perfectly uniform.
It's true that the orientation doesn't matter from that perspective, which is why I proposed that part can just be implied by the geofence point order.

Its unclear to me if the H3Index should have a new mode to declare them (perhaps? h3ToParent on a base cell could return a value that way).

This seems like trying to fit this into an unrelated structure/function because that structure/function already exists.

I'm not entirely convinced it's unrelated: Each hexagon ID is an identifier for a uniformly-divided region of the planet. The icosahedron faces meet that definition. They are not hexagons, but we already bent the rule on that with the 12 pentagons per resolution also having "hexagon IDs."

Something I've been thinking about since I feel half or more of the issues with the algos hitting pentagons is due to it not being clear in our minds how the coordinate orientations are being changed.

I agree orientation changes are one of the trickier parts we have to handle, I would also note that the deleted subsequence in the pentagon cells has come up quite a bit (and appears to be related to odd behavior with h3Line.)

That's also true, and I bet it's interrelated with the orientation changes -- the deleted subsequence requires going into a different direction than usual, but those new directions also have multiple "answers" on how the IJK coordinates get rotated depending on which pentagon you're talking about due to the differences between the "polar" and "equatorial" pentagons.

My $0.02 here:

I don't mind exposing the icosahedron vertices, though it might be just as easy to do so with static data in KML (already in the repo) and GeoJSON (I have this and could add), and/or other formats.

I'm really not a fan of extending h3ToParent to support icosahedron faces - this would be a major version change, and having a new mode seems like overkill, since that mode could only have 20 valid values and the number of modes available are limited. Better to simply add h3ToFace, returning an integer index.

I suggest adding Face Numbers to the H3Index to reduce rotation and overage code.


dataclass_transform default init=False does not work

dataclass_transform(PEP 681) should be a more primitive usage than dataclass(PEP 557),
I think the default init=false should be valid,
but is actually ignored by pylance.
Is this a bug?
or am I missing something? This is as expected.
Example (metaclass behaves the same way)
from typing import Callable, TypeVar, Tuple

_T = TypeVar("_T")

def __dataclass_transform__(
    *,
    eq_default: bool = True,
    order_default: bool = False,
    kw_only_default: bool = False,
    init_default: bool = False,
    field_descriptors: Tuple[type, ...] = (()),
) -> Callable[[_T], _T]:
    return lambda a:a

@__dataclass_transform__()
class BaseModel():
    def __init_subclass__(
        cls,
        *,
        init: bool = False, # default does not add __init__
        frozen: bool = False,
        eq: bool = True,
        order: bool = False,
    ):
        super().__init_subclass__()

class Item(BaseModel):
    id: int

Item() # Argument missing for parameter "id"

By default, @dataclass adds an __init__ method. A base class or metaclass decorated with dataclass_transform does the same unless told otherwise. If you want your dataclass-like class to not add an __init__ method by default, you should call the dataclass transform decorator with the argument init_default=False
@__dataclass_transform__(init_default=False)

Ah yes, you're correct, init_default is not supported by the current proposal. That would need to be an extension to PEP 681, which is going through the ratification process currently. There still may be a window of opportunity to add this functionality to the standard, but that window is closing very quickly.
If this is something you are interested in, I recommend that you respond to this typing-sig email thread with a proposal and justification for why this should be added to the PEP.
cc @debonte, author of PEP 681

@mao-shonen, I'm closing this issue since it's a PEP 681 request rather than a Pylance problem. As Eric said, we're still accepting PEP 681 feedback on typing-sig. It would be helpful if you could point us to a library that would need this new dataclass_transform behavior.

Thank you for your answer and suggestions,
In short, this is just a draft for now,
I should participate in it in the email thread.
At the moment this is not a pylance bug or not yet implemented feature, I will close this issues


Type information is not working for ./Setup.hs with Stack

Cabal projects may have Setup.hs files in the folder where .cabal file reside.
Currently it doesn't show type information for its contents on Windows.
There are errors like:

haskell-ghc-mod warning: EXCEPTION: browse:
Could not find module `Main'
Use -v to see a list of the files searched for.
util.coffee:36 haskell-ghc-mod warning: EXCEPTION: browse:
Could not find module System.Process'            It is a member of the hidden package process-<IP_ADDRESS>'.
Perhaps you need to add `process' to the build-depends in your .cabal file.
Use -v to see a list of the files searched for.
util.coffee:36 haskell-ghc-mod warning: EXCEPTION: browse:
Could not find module System.FilePath'            It is a member of the hidden package filepath-<IP_ADDRESS>'.
Perhaps you need to add `filepath' to the build-depends in your .cabal file.
Use -v to see a list of the files searched for.

Setup.hs is not part of cabal project, but ghc-mod assumes it is. I suggest
you file an issue on ghc-mod repo.
6 июня 2016 г. 4:19 PM пользователь "varosi"<EMAIL_ADDRESS>написал:

Cabal projects may have Setup.hs files in the folder where .cabal file
reside.
Currently it doesn't show type information for its contents on Windows.
There are errors like:
haskell-ghc-mod warning: EXCEPTION: browse:
Could not find module `Main'
Use -v to see a list of the files searched for.
util.coffee:36 haskell-ghc-mod warning: EXCEPTION: browse:
Could not find module System.Process'
It is a member of the hidden packageprocess-<IP_ADDRESS>'.
Perhaps you need to add `process' to the build-depends in your .cabal file.
Use -v to see a list of the files searched for.
util.coffee:36 haskell-ghc-mod warning: EXCEPTION: browse:
Could not find module System.FilePath'
It is a member of the hidden packagefilepath-<IP_ADDRESS>'.
Perhaps you need to add `filepath' to the build-depends in your .cabal
file.
Use -v to see a list of the files searched for.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/atom-haskell/haskell-ghc-mod/issues/156, or mute the
thread
https://github.com/notifications/unsubscribe/AG8EZufOqmM-lXAn2Yf3XfcB8N-4yEP1ks5qJB5_gaJpZM4Iu4v4
.

That said, you could add dependencies to cabal file.
6 июня 2016 г. 4:23 PM пользователь "Nikolay Yakimov"<EMAIL_ADDRESS>написал:

Setup.hs is not part of cabal project, but ghc-mod assumes it is. I
suggest you file an issue on ghc-mod repo.
6 июня 2016 г. 4:19 PM пользователь "varosi"<EMAIL_ADDRESS>написал:

Cabal projects may have Setup.hs files in the folder where .cabal file
reside.
Currently it doesn't show type information for its contents on Windows.
There are errors like:
haskell-ghc-mod warning: EXCEPTION: browse:
Could not find module `Main'
Use -v to see a list of the files searched for.
util.coffee:36 haskell-ghc-mod warning: EXCEPTION: browse:
Could not find module System.Process'
It is a member of the hidden packageprocess-<IP_ADDRESS>'.
Perhaps you need to add `process' to the build-depends in your .cabal
file.
Use -v to see a list of the files searched for.
util.coffee:36 haskell-ghc-mod warning: EXCEPTION: browse:
Could not find module System.FilePath'
It is a member of the hidden packagefilepath-<IP_ADDRESS>'.
Perhaps you need to add `filepath' to the build-depends in your .cabal
file.
Use -v to see a list of the files searched for.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/atom-haskell/haskell-ghc-mod/issues/156, or mute
the thread
https://github.com/notifications/unsubscribe/AG8EZufOqmM-lXAn2Yf3XfcB8N-4yEP1ks5qJB5_gaJpZM4Iu4v4
.


Simple demo does not work

Steps:

yarn start:app
Load the flights-20k dataset https://github.com/leibatt/scalable-vega/blob/master/sample_data/flights_20k.json
Load the flights 20k sample spec https://github.com/leibatt/scalable-vega/blob/master/test/specs/vega_specs/flights_20k.json

The examples in the demo have the same issue.


CFFL

CFFL

Linter check found the following problems:
The following problems have been found:
ERROR: /tmp/abs_d01udk9tr9/clone/recipe/meta.yaml:15: pip_install_args: pip install should be run with --no-deps.
ERROR: /tmp/abs_d01udk9tr9/clone/recipe/meta.yaml:20: missing_wheel: For pypi packages, wheel should be present in the host section
ERROR: /tmp/abs_d01udk9tr9/clone/recipe/meta.yaml:46: http_url: http://globus-sdk-python.readthedocs.io/ is not https
Errors were found


Make BlockStorage.updateBlockMultiSig accepts the sig and bitfield

Instead of passing the whole block and serializing it,
just pass the fields to update along with the height
(to locate the block) and the hash (to check we have the right one).

So, this fails because the Height is variably-serialized.

So, this fails because the Height is variably-serialized.

Look like a compilation error in agora/tests/unit/BlockStorageMultiSig.d

Should be fixed now

Fixed


[Feature]: Media card button to launch media browser directly in a pop-up

Is your feature request related to a problem?
In relation to the new media card: Navigating to the media browser seems only possible by having the tap action navigating to "More Info" and then tapping  "Browse Media" and then selecting the source such as Spotify.
Describe the solution you'd like
Having the option to tap on the card itself or a button on the card and having the media dialog pop up directly would be nice, preferably to a chosen source such as an Spotify account if possible.
Describe alternatives you've considered
A proposed solution I found online and tried without success:
tap_action:
action: fire-dom-event
browser_mod:
command: popup
title: Media Browser
hide_header: false
large: false
card:
type: iframe
url: /media-browser
aspect_ratio: 200%
Additional context
No response

Agreed, this would be a great enhancement

+1 trying to get this too, assume it's a navigation call but not sure what the path is to get to it, would love to skip 'Browse Media' -> 'Spotify' or have the option to go one level deeper by going straight to 'Playlists'

I'd love this too.  I can use the following as a tap_action, but as the service name suggests it navigates away to the media browser rather than having a pop-up.
type: custom:mushroom-media-player-card
entity: media_player.living_room_tivo
media_controls:
  - on_off
  - play_pause_stop
collapsible_controls: true
tap_action:
  action: navigate
  navigation_path: /media-browser/media_player.living_room_tivo
hold_action:
  action: none
double_tap_action:
  action: none
use_media_info: true
icon_type: none
show_volume_level: false
card_mod:
  style: |
    ha-card {
      {% if not is_state("media_player.living_room_tivo", "off") %}
        background-blend-mode: overlay;
        background-color: rgba(var(--rgb-card-background-color), 0.5);
        background-image: url('{{ state_attr("media_player.living_room_tivo", "entity_picture") }}');
        background-position: right var(--spacing) top var(--spacing);
        background-repeat: no-repeat;
        transition: unset;
      {% endif %}
    }

I'd like to see this also - something similar to browse media button on the standard media card (which renders quite nicely with the Mushroom Square theme) or a tap action.


Support parsing Usage: where the command is on the next line

Support parsing multi-line usages.
$ podman image tag --help
Add an additional name to a local image

Description:
  Adds one or more additional names to locally-stored image.

Usage:
  podman image tag IMAGE TARGET_NAME [TARGET_NAME...]

Examples:
  podman image tag 0e3bbc2 fedora:latest
  podman image tag imageID:latest myNewImage:newTag
  podman image tag httpd myregistryhost:5000/fedora/httpd:v2

Implemented in 6d5ab5bda7422416143780cd58259cb0e0664c65.


Fix CI issues

I have made things!
I forgot to add pull_request to trigger workflow.. we only had on push. I see here that tests was failing on windows so I added --user to pip install commands.
Python 3.9 is failing on mypy issues:
mimesis/builtins/en.py:72: error: syntax error in type comment

Checklist

[X] I have read contributing guidelines
[X] I'm sure that I did not unrelated changes in this pull request
[X] I have created at least one test case for the changes I have made

Related issues

@lk-geimfari Sorry my fault I didn't setup correctly!

@michael0liver Thank you very much! Great job!

@michael0liver I'll fix issues not related to CI later.


Integrate racer's find-definition functionality

@edubkendo thanks for your work in integrating racer in Atom. I am using atom-racer for completion but I am relying on ctags to and the Atom symbols-view package in order to go to the definition of a given rust function/type. My hunch is that racer has a better Rust parser and it is also providing a find-definition. Here is the vim integration
https://github.com/phildawes/racer/blob/master/editors/racer.vim#L99
Would it make sense to integrate find-definition directly into atom-racer? If so what are next steps I could help with? I have written node code but I have no experience with Atom.

It would make sense. I don't know if Atom has an api for this or not, though. @alkama has been doing most of the recent work on atom-racer, and might have a better idea. (tbh, I'm using emacs more and more these days).

I would also appreciate on a find definition tool for atom-racer.

@winding-lines  @ruabmbua someone interested in adding this functionality could look at https://github.com/litgh/atom-godef for inspiration.

I am using vi more now, but will look a bit at the atom plugin over the
weekend.
On Tue, Mar 31, 2015 at 9:02 AM Eric West<EMAIL_ADDRESS>wrote:

@winding-lines https://github.com/winding-lines @ruabmbua
https://github.com/ruabmbua someone interested in adding this
functionality could look at https://github.com/litgh/atom-godef for
inspiration.
—
Reply to this email directly or view it on GitHub
https://github.com/edubkendo/atom-racer/issues/13#issuecomment-88143378.

racer is still missing scanning of project code. Without being able, to auto complete stuff inside the current project, definition finding will also be hard.

@ruabmbua AFAIK, we do add project code to the sauce ourself by making racer scan the rustc source code AND the current projects one :) So it "might" work.
See https://github.com/edubkendo/atom-racer/blob/master/lib/racer-client.coffee#L48  and https://github.com/edubkendo/atom-racer/blob/master/lib/racer-client.coffee#L50

Strange. I only get auto completion for the current open file, and all source code from the compiler source tree.
As example: I am in file foo.rs, an I am trying to import the struct bar::Bar with use bar::Bar;, which is located in bar.rs. Neither the module bar, nor the struct Bar gets auto completed in foo.rs.

I think, I know what the bug is.
atom.project.getPath(); returns "/path/to/my/project", but not "/path/to/my/project/src". I think the second path is, what racer actually needs.
Info: I am using a cargo generated project setup, and I do not open the src folder, because a lot of other stuff is not inside src, but one level above it.


Add shaking capability for UIIconBar widget

My first PR for Terasology and the application to GSOC!
To shake the health bar, set the health bar's AnimationMode enum to SHAKING. To turn it off set it back to NONE. This PR is in tandem with one to the HealthClientSystem in the Health module to shake the health bar when the player is low on health.

Might need to convert animationTimer to rely on some sort of Timer so that frame rate doesn't affect animation

https://github.com/MovingBlocks/TeraNUI/pull/56/commits/92c2fb9b5f9b286236baddb55fc43456466d6f4c

Revert "fix(ModuleTypeRegistry): don't skip merging reflections when possible (#52)"
This reverts commit 956f4db.

eh what? was there a problem with it?

As for the feature itself:
I am still learning my way around NUI, but it doesn't sound right to me that the shaking animation should go in the UIIconBar class.
Going down that path would lead to every application adding animation methods to the core UI library for each and every widget they want to animate with every animation style.
It would be much preferable to have some method of applying a transformation to any widget, probably something the application includes when it builds the scene.


Another module

While I use another module like JSQMessagesViewController,
and reuse JSQMessagesCollectionViewCellIncoming.nib
It will be error:Use of undeclared type 'JSQMessagesCollectionViewCellIncoming'

Would How to fix missing imports in the generated file? in the docs help you out here or is this another kind of problem?

Very thanks.


Fixes instant ramen

About the PR
Fixes #10060
Screenshots

Changelog

:cl:

fix: Instant ramen now works with cold water.

Just use a microwave. Seriously.

or just fix the description

1984

Just use a microwave. Seriously.


Update README.md

Hi, I would like to propose my indie app Feriapp: it is strongly afflicted by COVID-19 as it is about travels.
Thank you for opportunity,
Edoardo

Please keep the list in alphabetic order, like the README says 🙂

App already added


[LUNE] - Landing page - squashed

Hi @camelburrito
This is a new PR following the CLA issues we had on #518, containing all changed squashed in one commit to avoid the double author problem
Link to the staging can be found here:
https://gweb-amp-template-lune-review.firebaseapp.com/templates/lune/lune.amp
Thanks

Approved in https://github.com/ampproject/ampstart/pull/518


⚠ ClamAV has degraded performance

In 4b1947d, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 197 ms

Resolved: ClamAV performance has improved in 5047c25 after 23 minutes.


WIP: CI multi cluster

Description
Deploy 2 clusters on Packet to parallelize the test execution.
Motivation and Context

How Has This Been Tested?

[ ] Covered by existing integration testing
[ ] Added integration testing to cover
[ ] Tested locally
[ ] Have not tested

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

  You are creating two clusters, and have revamped the testing to use cluster1... but I don't see cluster2 used.   Am I reading things right? :)

Doh... you marked it WIP... I shouldn't code review before coffee :)
One other idea... perhaps we make the second cluster an IPv6 cluster?  Would give us some coverage on K8s configured for IPv6 :)

@edwarnicke I'll consider IPv6, makes sense.
Do you think a 3 node cluster is feasible too? It will allow us to enable 3 node scenarios. The first one that comes to mind (and I have tried it with kind already) is the VPN example. In it, we can run the client, firewall and VPN-GW on separate nodes. We don't have much of these today though.

@edwarnicke @fkautz this one is ready for review.


How Might One Specify Separate Working Directories for Start Parameter on Github Action

I'm trying to integrate Cypress Tests with a Phoenix application on a Github Actions workflow. In order to allow the tests to pass the it seems like the Phoenix backend needs to be up on port 4000. To support this, I'm attempting to use the start parameter to run mix phx.server (This brings up the Phoenix backend). However, due to the way Phoenix applications are structured  cypress is located in ./assets while the command to start the server mix phx.server depends on files in  .
I would like to check if there's a way to specify separate working directories for start and the cypress github action.
I initially tried setting the working-directory parameter but it would configure both the cypress run and the server to run from that directory(assets). Due to the way Phoenix projects are structured.
For added context, this question emerged from a discussion on the Cypress Gitter and I was directed to post the question here.
Here's what I posted:

Hey all,
Does anyone happen to know how I might link the backend startup step and the cypress start step so that cypress first starts the backend and then?I'm attempting to run a e2e test on Github Actions which depends on the backend being up(e.g. you have a command like cy.visit(localhost:4000) which depends on a backend like Django/Rails to be up before the container can run)
Context:
I’m trying to run a suite of end to end tests written with Cypress 6.8.0 on Github Actions. The suite of tests depends on a Phoenix server running on localhost:4000 and the relevant portion of the Github Action looks like this:
  - name: Install dependencies
    run: npm install --prefix assets
  - name: Start Phoenix server
    run: elixir --erl "-detached" -S mix phx.server
  - name: Cypress run
    uses: cypress-io/github-action@v2
    with:
      working-directory: assets
      wait-on: "http://localhost:4000"

I get the following error message when attempting to run a workflow: waiting on "http://localhost:4000" with timeout of 60 seconds. Am aware that this could be an X-Y and perhaps I should dig further into Github Actions to see how the networking is done and validate whether the backend is indeed running the background. However, I just wanted to check if there are alternative workarounds which I can get with Cypress(e.g perhaps coupling the two steps into one). I am able to run the suite of tests to completion locally so I doubt it is an issue with the test itself
Do let me know if you happen to have any suggestions/workarounds. Any help would be deeply appreciated.

Do let me know if you have any suggestions or if there's any additional information I should provide. Any help would be deeply appreciated. Thanks!

I'm facing the same issue, any updates on this? Or did you find a workaround @J0 ?

Hey @alvaromartmart ,
Thanks for reaching out! I unfortunately did not figure out an effective workaround. Let me know what you come up with or if there’s any way I can help.
Joel

We're also facing this issue I believe. Our cypress tests are in a sub/subdirectory of our monorepo, so the working directory needs to be that. but the package.json script that starts up the server is in the root directory.
I tried creating a shell script called start_app.sh in the subdir for our cypress tests that just does:
cd ../..
yarn start:app

But when I set the start command to be that shell command I get
Error: Unable to locate executable file: start_app.sh. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.

This is really putting a block on our ability to run cypress in a GitHub action. If anyone has any bright ideas, I'm all ears!

Alright so turns out we were able to get this to work just as I was typign this up by just running a yarn start:app as define din teh root package.json. I'm honestly surprised that worked. We've got other issues now, but this one seems to be at least starting up the app from what we can see.


when I shutdown the docker-compose and restart again the DB is refreshed and all data is lost

when I shutdown the docker-compose using docker-compose down and restart again using docker-compose up -d the DB is refreshed and all data is lost.
Is there any way I can keep my db consistent even if I restart.

The data in your docker compose database should persist. I've just confirmed that this works on my local install.  My test was running docker-compose up, creating a view, using ctrl-c to stop the container, re-running docker-compose up, and confirming that the view was still present. How are you storing your data in the db, and how are you starting/stopper docker?

Does pgml stores database outside docker container? Or maps some local directory to store the db?

Usually, if you run docker-compose down, the containers will get removed, and I believe the associated volumes, too.
Instead, if you stop the containers by using ctrl-c or docker-compose stop, the next time you spin them up, the database comes back with whatever was persisted in the volumes from the previous run.
This kind of behavior is common with other compose builds that include other db types, such as mysql, so I'm guessing it could be just the commands being used to stop the containers; docker-compose down usually clears the containers and volumes, and it's the expected behavior, as far as I know.

#523

I typically mount a volume in my docker-compose.yaml file so that docker compose down does not remove the data (docker compose down -v will remove the data)
Add this under services:postgres
    volumes:
      - db:/var/lib/postgresql/14

and this to the end of the file

volumes:
  db:


Try to understand CloudFront's forwarded-proto as well

This PR attempts to make the nginx-proxy support custom CloudFront headers, which should fix the infinite HTTPS redirect loop.
@pebble/webops

+1 makes sense


⚠ Jenkins Plugins Artifactory has degraded performance

In da1489d, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1084 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 01aaf6e after 12 minutes.


Support RTX

[ ] Enable RTX in SettingEngine
[ ] Support RTX in SDP layer
[ ] Support RTX in Track/PC layer

Closing for #9


ASC Continuous Export Policy Missing from Hub and Spoke

I noticed the ASC Continuous Export Policy is now omitted from the Hub and Spoke implementation. The policy was previously included with the VWAN implementation. Should this policy be included for Hub and Spoke as well?

Thanks for reaching out. This policy is now available as a built-in policy, and hence we will not continue to ship this as a custom policy.


3bot deployment: screen #11 success

NEW title: 3Bot Deployment Successful
|
Congratulations, you have just successfully deployed a 3Bot!
Please go to https://.....   To access your 3Bot.
Note:
~‘Next button’ should be ‘My 3Bot’ button: directing you to a page with a list of your deployed 3bot.
should be a home button too instead of back.~

"Next" ok for now because too hard,
after next redirect to the url itself

success message changed , but the finish button redirect to blank page with just page header

the redirection problem of finish button should be fixed by this PR #864 and it's verified now


system issues

how to fix
unable to resolve${workspace Folder}in configuration ("workspace Folder")  i have issues

@lilimastroeni You mean ${workspaceFolder}? That's without the space as you have it. Perhaps this article will help, 'Variables Reference'

/needsMoreInfo


use fully resolved path when push|run single file

if you push a file from a nested dir, it should untar into correct location

not sure why the test is failing now..

fixed the test.  I added the resolvedEntryPoint versus overwriting the entryPoint which was provided initially by user/cli.

or perhaps we should have the resolve of findProject add this new property and leave along the user provided entryPoint one?

we could also leave the opts alone, and just pass the project dict into tarBundle.  I feel changing the entryPoint in project was confusing a bit, which is why I chose resolvedEntryPoint.

I pulled this locally and reviewed it, ran tests and they passed, so I landed the patch. Just now I checked out to another branch to rebase, but tried t2 run nested/index.js -s before I did and it worked exactly as it should—without this patch.

the issue occurs when you try to push a file while in a nested directory.
the ignored path does not included the cwd portion, so it will default to
the relpath
On Fri, Sep 18, 2015 at 1:12 PM Rick Waldron<EMAIL_ADDRESS>wrote:

I pulled this locally and reviewed it, ran tests and they passed, so I
landed the patch. Just now I checked out to another branch to rebase, but
tried t2 run nested/index.js -s before I did and it worked exactly as it
should—without this patch.
—
Reply to this email directly or view it on GitHub
https://github.com/tessel/t2-cli/pull/316#issuecomment-141554595.

Output captured from branch that doesn't have this patch:
rwaldron at nova in ~/robotz/tessel-pir
$ t2 run nested/index.js -s
INFO Looking for your Tessel...
INFO Connected to bishop over USB
INFO Writing nested/index.js to RAM on bishop (2.56 kB)...
INFO Deployed.
INFO Running nested/index.js...
/tmp/remote-script/nested/index.js
1440599263655 1
1440599263726 1
1440599263733 1
1440599263737 1
1440599263741 1
INFO Stopping script...
rwaldron at nova in ~/robotz/tessel-pir
$ t2 run index.js -s
INFO Looking for your Tessel...
INFO Connected to bishop over USB
INFO Writing index.js to RAM on bishop (2.048 kB)...
INFO Deployed.
INFO Running index.js...
/tmp/remote-script/index.js
1440599279774 1
1440599279843 1
1440599279850 1
1440599279854 1
1440599279858 1
1440599279861 1
1440599279864 1
INFO Stopping script...

the issue occurs when you try to push a file while in a nested directory.
the ignored path does not included the cwd portion, so it will default to
the relpath

That doesn't appear to be the case in my output above.

cd nested; t2 run index.js -s works for you?

I used the same code, in two different files:

index.js
nested/index.js

var tessel = require('tessel');

console.log(process.argv[1]);

tessel.port.A.pin[7].input();

function read() {
  tessel.port.A.pin[7].read(function(error, data) {
    console.log(Date.now(), data);
    read();
  });
}

read();

setTimeout(function() {
  process.exit(0);
}, 200);

perhaps because the file is still there?

perhaps because the file is still there?

What file is still where? Look at the paths that are printed out.

cd nested; t2 run index.js -s works for you?

This is the difference I was looking for.


resource/cloudflare_bot_managment: add Bot Management resource

Cloudflare-Go has been updated to include bot_management, we should update terraform to also include this change.
Depends on https://github.com/cloudflare/cloudflare-go/pull/1363
Closes #693

Cloudflare-go was updated, decided to do some more testing. Had to update the PR, but I keep getting this error when I run terraform apply in my test setup locally:
cloudflare_bot_management.bm_bots_config: Creating...
╷
│ Error: Provider produced inconsistent result after apply
│
│ When applying changes to cloudflare_bot_management.bm_bots_config, provider "provider[\"registry.terraform.io/cloudflare/cloudflare\"]" produced an
│ unexpected new value: Root resource was present, but now absent.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

@jacobbednarz  thoughts?

@cwlowder that looks like it removed the resource due to it missing from the provider. i just bumped the Go library here so you should be able to re-run these tests now and have them work. otherwise, i can take a look shortly.

ah, i can see a couple of issues here:

your test cases are referencing cloudflare_api_shield
you're missing a d.SetId call which is used as a lookup anchor (tfproviderlint is also complaining about this) and why the test cannot find the resource despite it actually being created.

i'll push up a couple of commits addressing this and you can take a look if you'd like :)

@cwlowder i went ahead and pushed up those fixes. would you mind eyeballing the unentitled test case as i think i know what you were trying to achieve there but do want to be sure before we merge it that was the intention.
acceptance tests are passing
TF_ACC=1 go test ./internal/sdkv2provider -v -run "^TestAccCloudflareBotManagement_" -count 1 -timeout 120m -parallel 1
=== RUN   TestAccCloudflareBotManagement_SBFM
--- PASS: TestAccCloudflareBotManagement_SBFM (19.36s)
=== RUN   TestAccCloudflareBotManagement_Unentitled
--- PASS: TestAccCloudflareBotManagement_Unentitled (7.19s)
PASS
ok  	github.com/cloudflare/terraform-provider-cloudflare/internal/sdkv2provider	27.512s

My intention is that some zones, like a free zone, will not be able to turn on SBFM or disable auto updating of the ml model. When you attempt to do so, the API will return an error. I want to test that behavior.

perfect. that is the behaviour I accounted for here so I think we're good to merge.
thanks!


Can you help add this paper?

Unsupervised Open-Vocabulary Object Localization in Videos from ICCV 2023
Thanks!

Hi @JosephPai, thanks for the suggestion! I believe this work would be more suited for the awesome list https://github.com/Qinying-Liu/Awesome-Open-Vocabulary-Semantic-Segmentation. In ours we focus on fully-unsupervised methods which are mainly class-agnostic. Hope that makes sense,
Best,

Hi @osimeoni , thanks for the suggestion!
I think our work somehow lies in the middle ground between object localization and semantic segmentation.
If you look into the the detail, you will see that the localization part is actually achieved in an unsupervised way without category.
The labeling is an additional feature based on it.
And it turns out that the semantic labeling can further improve the localization.
But anyway, as stated in the title, our main target would still be object localization.
We are still some distance away from unsupervised semantic segmentation, which would be an exciting future work :)
So I insist on recommending that you add this paper.
As far as I know, there are not too many papers on object localization in videos.


RFE: Support async interceptors

As far as I see there's no way to asynchronously intercept resp. call the next interceptor which is sometimes inconvenient when e.g. authorization is lazily initialized.

It is possible, albeit inconvenient with a lot of boilerplate. Basically you must create and return your own [ClientStreaming|DuplexStreaming|ServerStreaming|Unary]Call (depending on the interceptor) and then manually resolve/reject the call's Deferreds. I outlined how one would create a retry interceptor for a unary call here.
Here's something similar for asynchronously getting/setting an authorization header:
import {
  Deferred,
  RpcError,
  RpcInterceptor,
  RpcMetadata,
  RpcStatus,
  UnaryCall,
} from '@protobuf-ts/runtime-rpc';

import { asyncGetAuthHeader } from '../somewhere';

const myAuthInterceptor: RpcInterceptor = {
  interceptUnary(next, method, input, options) {
    const defHeader = new Deferred<RpcMetadata>(),
      defMessage = new Deferred<object>(),
      defStatus = new Deferred<RpcStatus>(),
      defTrailer = new Deferred<RpcMetadata>();

    if (!options.meta) {
      options.meta = {};
    }

    void (async () => {
      try {
        // set auth request header
        options.meta.Authorization = await asyncGetAuthHeader();

        // perform the actual call
        const result = next(method, input, options);

        // resolve all our Deferreds
        defHeader.resolve(result.headers);
        defMessage.resolve(result.response);
        defStatus.resolve(result.status);
        defTrailer.resolve(result.trailers);
      } catch (err) {
        defHeader.rejectPending(err);
        defMessage.rejectPending(err);
        defStatus.rejectPending(err);
        defTrailer.rejectPending(err);
      }
    })();

    return new UnaryCall(
      method,
      options.meta,
      input,
      defHeader.promise,
      defMessage.promise,
      defStatus.promise,
      defTrailer.promise
    );
  },
};

Thanks for sharing this, but this is too much bit magic for me. Code is more read than written and my co-workers would just stumble upon it.
For the time being I am going with lazy-initialized transport explicitly setting up the authorization meta.
I still would appreciate a leaner way with less code.

@jcready Please can you provide an example how to handle unauthenticated request (expired access token) too? So we should request a new token with refresh token provided. I'm struggling with that stuff

@daudfauzy98 How are you sending the access token? Ideally you'd know that your access token was expired before issuing the request. If you were able to know it was expired (or about to expire) then the above should basically be all you need to do. Where asyncGetAuthHeader is where you both use your refresh token to get a new access token if it is expired and return access token.
If you aren't able to know that your access token is expired (or about to expire) then you could do something like this (perform initial request, if it fails then refresh access token, set new access token in the request headers, attempt the request again):
import {
  Deferred,
  RpcError,
  RpcInterceptor,
  RpcMetadata,
  RpcStatus,
  UnaryCall,
} from '@protobuf-ts/runtime-rpc';

import { getCurrentAccessToken, refreshAccessToken } from '../somewhere';

const myAuthInterceptor: RpcInterceptor = {
    interceptUnary(next, method, input, options) {
      const defHeader = new Deferred<RpcMetadata>(),
        defMessage = new Deferred<object>(),
        defStatus = new Deferred<RpcStatus>(),
        defTrailer = new Deferred<RpcMetadata>(),
        performRequest = async () => {
          const result = await next(method, input, options);
          defHeader.resolve(result.headers);
          defMessage.resolve(result.response);
          defStatus.resolve(result.status);
          defTrailer.resolve(result.trailers);
        },
        fail = (err: unknown) => {
          defHeader.rejectPending(err);
          defMessage.rejectPending(err);
          defStatus.rejectPending(err);
          defTrailer.rejectPending(err);
        };

      if (!options.meta) {
        options.meta = {};
      }

      void (async () => {
        try {
          await performRequest();
        } catch (err) {
          if (err instanceof RpcError && (
            err.code == GrpcStatus[GrpcStatus.PERMISSION_DENIED] ||
            err.code == GrpcStatus[GrpcStatus.UNAUTHENTICATED]
          )) {
            try {
              // Refresh the access token
              await refreshAccessToken();
              // update the request headers
              options.meta.Authorization = await getCurrentAccessToken();
              await performRequest();
            } catch (e) {
              return fail(e);
            }
          }
          fail(err);
        }
      })();

      return new UnaryCall(
        method,
        options.meta,
        input,
        defHeader.promise,
        defMessage.promise,
        defStatus.promise,
        defTrailer.promise
      );
    },
  };

Thanks for the example! Currently I did something like your given code. I got the session that contains access and refresh tokens when user has login, then store it to the session storage. So inside the interceptor, I take back the access token from the session storage, then provide it to option.meta.Authorization and finally send the request.
It runs fine as long if single request was performed. The problem is when there are multiple requests and the access token is expired, so it uses the same token and will got rejected by the server. Each request will perform refreshAccessToken() at the most same time, that makes a race between them. So, whoever get the new token first would be success and failed for the rest because the refresh token has changed by the first finished request. How do you handle such condition?

We're getting a little outside the scope of protobuf-ts here. Basically you need the refreshAccessToken function to be memoized (i.e. return the same promise to all callers based on if we're already refreshing the token) and we need to prevent sending new requests while we're waiting for the refreshed token. Something like:
// ../somewhere.ts

let lastRefreshPromise: Promise<void> | undefined;

export async function getCurrentAccessToken(): Promise<string> {
  await lastRefreshPromise;
  return sessionStorage.getItem('accessToken') ?? '';
}

export function refreshAccessToken(): Promise<void> {
  return lastRefreshPromise ?? (lastRefreshPromise = (async () => {
    try {
      // Actually retrieve a new access token somehow and store it in sessionStorage
      sessionStorage.setItem('accessToken', await magic());
    } finally {
      lastRefreshPromise = undefined;
    }
  })());
}

Then we need to modify the interceptor a bit:
      void (async () => {
        try {
          // set the request headers and prevent new requests while access token is refreshing
          options.meta.Authorization = await getCurrentAccessToken();
          await performRequest();
        } catch (err) {
          if (err instanceof RpcError && (
            err.code == GrpcStatus[GrpcStatus.PERMISSION_DENIED] ||
            err.code == GrpcStatus[GrpcStatus.UNAUTHENTICATED]
          )) {
            try {
              // Refresh the access token
              await refreshAccessToken();
              // update the request headers
              options.meta.Authorization = await getCurrentAccessToken();
              await performRequest();
            } catch (e) {
              return fail(e);
            }
          }
          fail(err);
        }
      })();

Again, you would be much better off if you could know the access token was about to expire and preemptively refresh the access token as it would avoid having to make requests we know are going to fail.

Just want to confirm that this is a reasonable implementation of both client/server streaming. In particular that I've covered both the successful and error cases of the RpcOutputStreamController
import {
  Deferred,
  type RpcInterceptor,
  type RpcMetadata,
  RpcOutputStreamController,
  type RpcStatus,
  ServerStreamingCall,
  UnaryCall,
} from "@protobuf-ts/runtime-rpc";
import invariant from "tiny-invariant";

// async interceptors require reimplementing quite a few of the internals of protobuf-ts
// see https://github.com/timostamm/protobuf-ts/blob/main/packages/grpcweb-transport/src/grpc-web-transport.ts
// and https://github.com/timostamm/protobuf-ts/issues/580
// basic strategy is:
// 1. create a Deferred for each of the properties of the RpcCall
// 2. create a Promise to run the actual RPC call
// 3. return a new RpcCall using the Deferreds

async function getAccessToken(): Promise<string> {
  // eslint-disable-next-line unicorn/no-useless-promise-resolve-reject
  return Promise.resolve("fake-access-token");
}

export const createAsyncInterceptor = (): RpcInterceptor => {
  return {
    interceptUnary(next, method, input, options) {
      const defHeaders = new Deferred<RpcMetadata>();
      const defResponse = new Deferred<object>();
      const defStatus = new Deferred<RpcStatus>();
      const defTrailer = new Deferred<RpcMetadata>();

      if (!options.meta) {
        options.meta = {};
      }

      const runRPC = async () => {
        try {
          // set the access token
          invariant(options.meta);
          options.meta.Authorization = `Bearer ${await getAccessToken()}`;

          // perform the actual call
          const result = next(method, input, options);

          // resolve all our Deferreds
          defHeaders.resolve(result.headers);
          defResponse.resolve(result.response);
          defStatus.resolve(result.status);
          defTrailer.resolve(result.trailers);
        } catch (error) {
          defHeaders.rejectPending(error);
          defResponse.rejectPending(error);
          defStatus.rejectPending(error);
          defTrailer.rejectPending(error);
        }
      };

      // eslint-disable-next-line @typescript-eslint/no-floating-promises
      runRPC();

      const call = new UnaryCall(
        method,
        options.meta,
        input,
        defHeaders.promise,
        defResponse.promise,
        defStatus.promise,
        defTrailer.promise
      );

      return call;
    },
    interceptServerStreaming(next, method, input, options) {
      const defHeaders = new Deferred<RpcMetadata>();
      const responseStream = new RpcOutputStreamController<object>();
      const defStatus = new Deferred<RpcStatus>();
      const defTrailer = new Deferred<RpcMetadata>();

      if (!options.meta) {
        options.meta = {};
      }

      const runRPC = async () => {
        try {
          // set the access token
          invariant(options.meta);
          options.meta.Authorization = `Bearer ${await getAccessToken()}`;

          // perform the actual call
          const result = next(method, input, options);

          // resolve all our Deferreds
          defHeaders.resolve(result.headers);
          result.responses.onMessage((message) => {
            responseStream.notifyMessage(message);
          });
          result.responses.onComplete(() => {
            responseStream.notifyComplete();
          });
          defStatus.resolve(result.status);
          defTrailer.resolve(result.trailers);
        } catch (error) {
          invariant(error instanceof Error);
          defHeaders.rejectPending(error);
          responseStream.notifyError(error);
          defStatus.rejectPending(error);
          defTrailer.rejectPending(error);
        }
      };

      // eslint-disable-next-line @typescript-eslint/no-floating-promises
      runRPC();

      const call = new ServerStreamingCall(
        method,
        options.meta,
        input,
        defHeaders.promise,
        responseStream,
        defStatus.promise,
        defTrailer.promise
      );

      return call;
    },
  };
};

What you have basically does nothing since you aren't actually doing anything in the error handling (you aren't retrying the request with a new access token, just re-throwing the error). You're not awaiting the next() call so your error handling would never be run anyhow. You also have invariant(error instanceof Error); inside a catch which would (if your invariant didn't hold) cause your call to never resolve or reject (you probably don't want that).


Onsave messge switcher

Disabled verbose on saving messages, set onSaveMessage to enable.
example:
//"out": "main.css", "onSaveMessage": true

@import "component.less";

html, body {
    height: 100%;
}

I'm sorry but the code hasn't been updated in a long time, since I am no longer using Atom, and therefore not maintaining the code anymore, or publishing any new versions.
Sorry about that. I am willing to transfer ownership of the code and plugin to anyone who wants it.


Implement automation ROI forecast functionality

To implement ability to forecast and visualize test automation ROI based on set of initial parameters
the very basic underlying formula is
$\frac{Mt - At}{At}$
where:

Mt - time saved on manual testing
At - time spent on automation

implemented in #10


bug : it's will failed to login when the user name is chinese !

https://github.com/YUbuntu0109/sms/blob/97f7b13c5d38b9cbe3d925bcd9bf9ec42dc97429/student_information_management_system/src/pers/huangyuhui/sms/servlet/LoginServlet.java#L64-L66

This project isn't maintained so please add new user info into the database when you want to login . . .

https://github.com/YUbuntu0109/sms/blob/97f7b13c5d38b9cbe3d925bcd9bf9ec42dc97429/student_information_management_system/src/pers/huangyuhui/sms/servlet/LoginServlet.java#L64-L66

This project isn't maintained so please add new user info into the database when you want to login . . .

The Solution : Please change the character encoding of database to utf-8 or others which support chinese. . .


Allow posting messages to channel ID

The method postMessageToChannel accepts channel name but the events like message provide only the channel id. It would be great if instead of:
bot.on('message', function ({ channel }) {
  bot.getChannelById(channel)
    .then(({ name }) => bot.postMessageToChannel(name, 'hi!'))
})

we could simply do:
bot.on('message', function ({ channel }) {
  bot.postMessageToChannelId(channel, 'hi!') // new method that receives the channel id!
})

I would be glad to contribute with a PR!

Ah, thank you for opening this issue; I was confused why my code wasn't working. Your workaround is slightly off; you'll want
bot.on('message', function ({ channel }) {
  bot.getChannelById(channel).then((chanInfo) => {
      bot.postMessageToChannel(chanInfo.name, 'hi!');
  });
});

Thanks @yamikuronue! I updated the issue description.

Thanks for reporting this issue. Try to use a postMessage method, it gets an ID as a first argument. It's common for posting to channels, users, groups.

Thanks! Will give it a try.


os/booting-on-ecs: config to allow IAM Task Roles

Added the base config/rules for IAM Task Roles to match latest AWS documentation for other distributions.
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-install.html
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html#enable_task_iam_roles

Do I need to add any more info on this?


Undefined const regression on master

It seems #11208 caused a regression in master, for example from nightly CI:
src/components/framework
  Showing last frame. Use --error-trace for full trace.

  In src/components/framework/spec/controllers/query_param_controller.cr:74:7

   74 | @[ATHR::Time::Format(format: "%Y--%m//%d  %T")]
          ^-----------------
  Error: Error: undefined constant ATHR::Time::Format

https://github.com/athena-framework/athena/actions/runs/8682447817/job/23806982620
Best I can tell its because the annotation is defined via a nested macro?
macro configuration_annotation(name, *args, &)
  annotation {{name.id}}; end
end

struct MyApp::ValueResolvers::Time
  macro configuration(name, *args)
    configuration_annotation ::{{@type}}::{{name.id}}
  end

  configuration Format
end

class MyClass
  @[MyApp::ValueResolvers::Time::Format(format: "%Y--%m//%d  %T")]
  def time
    {{pp @def.annotations}}
  end
end

pp MyClass.new.time

On 0efbf53cc2762431146eb028de95691cac7db039:
$ ccrystal test.cr
Using compiled compiler at .build/crystal
Showing last frame. Use --error-trace for full trace.

In test.cr:14:5

 14 | @[MyApp::ValueResolvers::Time::Format(format: "%Y--%m//%d  %T")]
        ^----------------------------------
Error: undefined constant MyApp::ValueResolvers::Time::Format

On commit before that:
$ crystal test.cr
[@[MyApp::ValueResolvers::Time::Format(format: "%Y--%m//%d  %T")]]
nil

The issue seems to be that name.id drops the global prefix.
macro foo(path)
  {% p path.id %} # => Foo
end

foo ::Foo

So I guess running athena CI against Crystal nightly has already payed off 🚀 😄


Publish package on packagist.org?

What about adding this package to https://packagist.org ?
That way, installing it in a composer-based installation would be as easy as running
composer require ounziw/ounziw_osm

Adding a package to packagist can be done with these simple steps:

go to https://packagist.org and click Sign in → Use GitHub to log in
click the Submit button and specify the URL of this repository ( https://github.com/ounziw/Free-Map )

Hello,
I registered packagist and submit my addon.
https://packagist.org/packages/ounziw/ounziw_osm

Great, thank you!
One more thing though: packagist (and composer) can't extract the package version from the Concrete controller.php file (it's a Concrete-specific stuff).
In order to tell packagist/composer the version of the package, we need to use git tags.
Also, we should switch to semver for versions... In summary:

versions have always 3 numbers (MajorVersion.MinorVersion.PatchVersion)
when we publish a new version that only fixes bugs, we increase PatchVersion
when we publish a new version that adds new features, we increase MinorVersion
when we publish a new version that introduces breaking changes, we increase MajorVersion

So, what about tagging the last published version (which is at commit b70708cd000b06f312206f97c7a5994c7bddfd7e)? It'd be as easy as:

creating a tag withgit tag 2.1.0 b70708cd000b06f312206f97c7a5994c7bddfd7e

push the tag to this GitHub repository withgit push --tags

I added a tag 2.1.0.
https://github.com/ounziw/Free-Map/releases/tag/2.1.0

Thanks!


Issue with MQTT energy value on ShellyPro4PM

Hi Alexander,
Today I discovered with MQTT Explorer that there is an issue with the Topic of homeassistant/sensor/shellypro4pm-XXXX-3/energy/config
the values are:
{   "name": "shellypro4pm-XXXX-Energy 3",   "stat_t": "~status/switch:3",   "frc_upd": "false",   "uniq_id": "shellypro4pm-XXXX-2-energy",   "qos": 0,   "~": "shellypro4pm-XXXX/",   "dev_cla": "energy",   "stat_cla": "total_increasing",   "unit_of_meas": "Wh",   "val_tpl": "{{ value_json['aenergy'].total }}",   "device": {     "identifiers": [       "XXXX-3"     ],     "manufacturer": "Shelly",     "model": "shellypro4pm",     "name": "shellypro4pm-XXXX-3"   } }
As you can see the value uniq_id refers to "shellypro4pm-XXXX-2-energy" which of course should be "shellypro4pm-XXXX-3-energy"
At first I thought it was an issue in the MQTT template (Gen2), So I fixed the value from 2 to 3.
After rebooting the shelly device the shellyteacher4domo didn't find the energy 3 device, so I used MQTT Explorer and found out that the issue is on the Shelly MQTT side.
So I created a support ticket for this at Shelly.Cloud company.
(Reaction time is 72 hours)
In advance you already might update the MQTT_Templates_gen2.txt file with this.

Hi Alexander,
Today I discovered with MQTT Explorer that there is an issue with the Topic of homeassistant/sensor/shellypro4pm-XXXX-3/energy/config
the values are: { "name": "shellypro4pm-XXXX-Energy 3", "stat_t": "~status/switch:3", "frc_upd": "false", "uniq_id": "shellypro4pm-XXXX-2-energy", "qos": 0, "~": "shellypro4pm-XXXX/", "dev_cla": "energy", "stat_cla": "total_increasing", "unit_of_meas": "Wh", "val_tpl": "{{ value_json['aenergy'].total }}", "device": { "identifiers": [ "XXXX-3" ], "manufacturer": "Shelly", "model": "shellypro4pm", "name": "shellypro4pm-XXXX-3" } }
As you can see the value uniq_id refers to "shellypro4pm-XXXX-2-energy" which of course should be "shellypro4pm-XXXX-3-energy"
At first I thought it was an issue in the MQTT template (Gen2), So I fixed the value from 2 to 3. After rebooting the shelly device the shellyteacher4domo didn't find the energy 3 device, so I used MQTT Explorer and found out that the issue is on the Shelly MQTT side.
So I created a support ticket for this at Shelly.Cloud company. (Reaction time is 72 hours)
In advance you already might update the MQTT_Templates_gen2.txt file with this.

You are right its my bad, fixed the template in actual ShellyTeacher release.
I think you should delete both energy-2 and energy-3 device from Domoticz and rerun the Teacher to reteach them.

I don't think it's your bad......
The payload itself is wrong, checked it in MQTT Explorer. I created a support ticket at Shelly, they are investigating this...
I will come back to u when I get the feedback of Shelly.

In MQTT Explorer I see the following values:
TOPIC:
shellypro4pm-XXX/online
VALUE:
false
TOPIC:
shellypro4pm-XXX/events/rpc
VALUE:
{"src":"shellypro4pm-XXX","dst":"shellypro4pm-XXX/events","method":"NotifyStatus","params":{"ts":1699032240.20,"switch:3":{"id":3,"aenergy":{"by_minute":[0.000,0.000,0.000],"minute_ts":1699032239,"total":0.000}}}}
TOPIC:
shellypro4pm-XXX/status/switch:0
VALUE:
{"id":0, "source":"loopback", "output":false, "apower":0.0, "voltage":231.0, "freq":50.0, "current":0.000, "pf":0.00, "aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1699032299},"temperature":{"tC":24.0, "tF":75.3}}
TOPIC:
shellypro4pm-XXX/status/switch:1
VALUE:
{"id":1, "source":"loopback", "output":false, "apower":0.0, "voltage":231.2, "freq":50.0, "current":0.000, "pf":0.00, "aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1699032359},"temperature":{"tC":23.9, "tF":75.1}}
TOPIC:
shellypro4pm-XXX/status/switch:2
VALUE:
{"id":2, "source":"loopback", "output":false, "apower":0.0, "voltage":231.4, "freq":50.0, "current":0.000, "pf":0.00, "aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1699032359},"temperature":{"tC":23.9, "tF":75.1}}
TOPIC:
shellypro4pm-XXX/status/switch:3
VALUE:
{"id":3, "source":"loopback", "output":false, "apower":0.0, "voltage":230.9, "freq":50.0, "current":0.000, "pf":0.00, "aenergy":{"total":0.000,"by_minute":[0.000,0.000,0.000],"minute_ts":1699032419},"temperature":{"tC":23.9, "tF":75.1}}
TOPIC:
shellies/shellypro4pm-XXX/announce
VALUE:
{"name":"Shelly 4PM Pro v2","id":"shellypro4pm-XXX","mac":"XXX","slot":0,"model":"SPSW-104PE16EU","gen":2,"fw_id":"20231031-152251/1.0.7-g5db02bd","ver":"1.0.7","app":"Pro4PM","auth_en":true,"auth_domain":"shellypro4pm-XXX"}
TOPIC:
shellies/shellypro4pm-XXX/relay/0/energy
VALUE:
0
TOPIC:
shellies/shellypro4pm-XXX/relay/0/power
VALUE:
2e-7
TOPIC:
shellies/shellypro4pm-XXX/relay/1/energy
VALUE:
0
TOPIC:
shellies/shellypro4pm-XXX/relay/1/power
VALUE:
2e-7
TOPIC:
shellies/shellypro4pm-XXX/relay/2/energy
VALUE:
0
TOPIC:
shellies/shellypro4pm-XXX/relay/2/power
VALUE:
2e-7
TOPIC:
shellies/shellypro4pm-XXX/relay/3/energy
VALUE:
0
TOPIC:
shellies/shellypro4pm-XXX/relay/3/power
VALUE:
2e-7

In MQTT Explorer I see the following values: TOPIC: shellypro4pm-XXX/online VALUE: false

You find the reason in the first value: "/online" value should be 'true' when the device is connected, and only true is accepted by ShellyTeacher. Other values are automatically discarded.
Please try to disable MQTT then enable again on this shelly unit. If this wont help, do a factory reset.

Thanks, disabled and revbooted Shelly and then enabled again and rebooted MQTT but no fix.
Still waiting on Shelly Support, Factory reset is last thing I will because if it is a issue in the firmware I need to reconfigure the ShellyPro4PM again.....
Please standby....I will contact again if Shelly support has contacted me.

I think they will also propose the factory reset but i will be curious as you unit behaves very peculiarly.

A new firmware 1.0.8 has been released, after upgrading MQTT became online again.
Based on your adjustment in "mqtt_templates_gen2.txt" the energy-3 became available now also after Shelly fixed this in the new 1.0.8 release.


display "server error" on click "Discover DeFi, ETH and DAI"

App Version: 2.12.0
Phone model: Huawei P20 Pro
Android version: 9
Reproduce:

Add contract "0x8C0Edb69eBF038BA0c7a4873E40fC09725064c2e "
click 1 AlphaWallet Discover(ALP)
click "Discover DeFi, ETH and DAI"

@JamesSmartCell this is solved with #879 right? has this been released?


swiftformat: 0.44.0 -> 0.44.2

Motivation for this change
Update for the following changes in the program:
0.44.2 (2020-02-02)

Fixed crash in line wrapping logic
Fixed several cases where braces were not correctly moved according to wrap rule
Prevented wrapping of image and color literals
Fixed bug with trailingClosures rule breaking unwrap operators

0.44.1 (2020-01-26)

Fixed spaceInsideComments rule mangling preformatted comments with multiple slashes
Fixed redundantFileprivate breaking code relying on type-inference for init
Reverted overly aggressive argument wrapping change in 0.44.0
Fixed redundantParens rule breaking #if statements without space before argument
Fixed // swiftformat:disable directives not affecting wrap or wrapArguments rules
Fixed bug where yodaConditions rule caused formatting to never terminate
Fixed crash in fileHeader rule

Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/100


Create catdrout.is-cool.dev.json

Requirements

[x] You have completed your website.
[x] The website is reachable.
[x] You have put a full stop (.) at the end of each of the CNAME, MX and NS records.
[x] The CNAME record doesn't contain https:// or /.
[x] There is sufficient information at the owner field.

Description
Hosting my hobby site
Link to Website
catdrout.is-cool.dev

ok

There are now only one cname record now

Make sure to start the repo. @xcatdrout


Document need for normal roughness texture conversion for CompositorEffects

Fixes https://github.com/godotengine/godot-docs/issues/9591.

I haven't personally confirmed this behavior in compute shaders - I have confirmed that this conversion function was needed in some cases in Spatial shaders before https://github.com/godotengine/godot/pull/94812 was merged, though.

Changes have been made, is this still waiting on anything else?

Thanks!