Prevent mongo conflating data & syntax

One of the most critical features of database usage is a clear separation of input data and query syntax. In the SQL world, this mandates the use of parametrized queries. NoSQL solutions have similar but different restrictions: in Rethink, the attack surface is very limited due to its clear separation in both the driver API and the wire protocol.
In Mongo, things are less clear - JSON syntax conflates with JSON data, with no separation between the two. This is incredibly problematic when one desires to... store JSON, in the JSON store :)
My current understanding is that there are only two entry points for Mongo syntax: keys beginning in $ or containing .. These can occur anywhere in the query, and can cause insecure or failed queries.
Some docs about this:

https://docs.mongodb.org/manual/core/document/#dot-notation
https://docs.mongodb.org/manual/faq/developers/#faq-dollar-sign-escaping

Because we are using JSON schemas for a large portion of our input data, the majority by volume of our endpoints won't see this problem. This applies to any endpoints not covered by schemas, and anywhere unstructured data is allowed (for example, container metadata).
What's very concerning to me is that sanitization seems to be ad-hoc and in-house to any given codebase, rather than internal to the database protocol or driver API. Further, Mongo seems to advocate swapping unicode code points for equivalently-rendered characters, in lieu of a proper escape sequence.
Many thanks to @aaltepet, who has more experience in mongo and offered this code sample:
def mongo_escape(fname):
    """
    Sanitize MongoDB keys by replacing reserved chars with full-width
    equivalents.

    MongoDB doesn't support using '.' and '$' as document keys, and suggests
    converting the chars to their unicode full-width equivalents.
    cf. http://docs.mongodb.org/manual/faq/developers/#faq-dollar-sign-escaping
    """
    return fname.replace('.', '\uFF0E').replace('$', '\uFF04')

def mongo_unescape(fname):
    """
    Desanities MongoDB keys by replacing full-width chars with normal
    ascii equivalents

    MongoDB doesn't support using '.' and '$' as document keys, and suggests
    converting the chars to their unicode full-width equivalents.
    cf. http://docs.mongodb.org/manual/faq/developers/#faq-dollar-sign-escaping
    """
    return fname.replace('\uFF0E', '.').replace('\uFF04', '$')

Note that this means data stored in the database is no longer valid - it's a translation that can't be sent anywhere else, so it has to be unconverted back on its way out.
Contrast @rentzso's utility function, which takes a map, rather than a single key:
def mongo_dict(d):
    def _mongo_list(d, pk=''):
        pk = pk and pk + '.'
        return sum([_mongo_list(v, pk+k) if isinstance(v, dict) else [(pk+k, v)] for k, v in d.iteritems()], [])
    return dict(_mongo_list(d))

The focus of this ticket, then, is threefold:

[ ] Identify the most-robust, least-custom method for sanitizing and transport
[ ] Ensure said method is universally applied
[ ] Put measures / tooling in place to prevent diverging away from 100% coverage.

For the first item, my off-the-cuff feeling is that I like @rentzso's approach of sanitizing a map all at once; maybe we continue with something like that, or use unicode-swapping to avoid collisions. Maybe this rolls into the same logic we use for escaping & unescaping the mongo ID not-quite-strings.
Of course, this logic shouldn't be within 30 feet of an individual CRUD API project; I had a look around expecting an implementation in pymongo or similar and came up empty. A library option would be ideal.
For the second action item: Also discussed with @aaltepet was the possibility of wrapping or augmenting pymongo. Presumably the best approach might be a raw(map) func that is always used when passing any map to mongo that isn't expected to contain syntax. An unraw(map) will be necessary on each query result if we do unicode-swapping.
For the third action item: open to ideas! In other languages I'd have several tooling suggestions; maybe pylint or similar can help us here.

Excellent analysis! Thank you, @kofalt. Looking forward to a PR.

What's the current use-case for needing keys containing '.' and '$' ? If this is 3rd party data that we are simply stewarding, and don't require querying or updating pieces, we should consider storing as a blob in the DB.
If we really need to persist it as a json document in mongo, and code on the app side of pymongo is required, I would prefer wrapping pymongo.
Curious if this son_manipulator class is applicible to the concern? Maybe it doesn't cover queries?https://api.mongodb.org/python/current/api/pymongo/son_manipulator.html
Also if we are transforming keys, we need to understand the impact to elasticsearch use-cases.

Basically this part of the ticket:

This applies to any endpoints not covered by schemas, and anywhere unstructured data is allowed (for example, container metadata).

The specific incident we ran into was storing dicom metadata that happened to include . characters in the keys. The container metadata field is, by design, completely unstructured and (should be) allowed to store largely-arbitrary key-value content.

Curious if this son_manipulator class is applicible to the concern?

I'm not sure. I really don't want to be mucking about with SON stuff if we can help it. That said, it mentions pymongo.database.Database.add_son_manipulator which implies we could do some translation transparently, which would be nice.

I would prefer wrapping pymongo

Me too, I think, except that due to the nature of mongo queries it's not possible to tell after-the-fact what's data and what's syntax, so I'm not sure of the best approach. And if there is a good approach, I don't want to write it; I want someone else to have done so in a library... Maybe a discussion on the pymongo tracker would be productive.

@kofalt
Can you post a sample of the offending dicom metadata json?

@ryansanford Excellent question!
"metadata" : {
    "LastScanLoc" : 36.0000000000000000,
    "FlipAngle" : 90.0000000000000000,
    "DB/dtPeakRateOfChangeOfGradientField" : 0.0000000000000000,

    "User_fill_map_MSW" : 0,

    "Horiz_FrameOfRef_" :<PHONE_NUMBER>000000000,
    "PhaseContrastFlowAxis" : 0,
    "Slop_int_6___Slop_int_9" : [
        0,
        1,
        0,
        0
    ],

    "User_fill_map_LSW" : 0,
}

This is a liberally-truncated example from our sample data set after classification.
The full set can be viewed here.
@rentzso added a workaround to this issue that swaps .s for _s, so I cannot state with certainty that these keys originally contained .s. Right now, the workaround does not resolve key collisions!

I learned from @kofalt that the above sample had already been sanitized.
@gsfr
I think changing the API in reaction to this pain-point (metadata gear) is a mistake. Once we persist transposed data in the database, we'll need to deal with the repercussions for a long time. If the direction changes in the future, updating database values in the field at sites can be a labor intensive endeavor.
For bali, I would much rather us change the gear to sanitize the json keys. This gives us time to make sure we appreciate the impact of sanitizing in the API.
The strings in the sample are not canonical, the group,element codes are. We should not expect to use those keys in a predictable way down the line without an actual dicom group,element to string dictionary that is enforced. Ideally, we would store the dicom group,element in the db so it could stand on its own without the dictionary.

Proposal: for the bali release, we keep what we have with the modification of adopting @aaltepet's two-directional translation for metadata. This would keep everything the same with the exception that our keys no longer collide and can be retrieved in their original (post-gear) form.
Keeping dicom metadata, and translations thereof, in a robust fashion, is a pretty large task, and I don't think we're there yet. But we can deliver a non-destructive variant of the current behaviour without too much trouble.

The container metadata field is, by design, completely unstructured and (should be) allowed to store largely-arbitrary key-value content.

Indexable, queryable content, that is.

Spoke with @gsfr and @kofalt
Conclusion is we stick with status quo for bali.01
Bali.02 we consider

add schema validation to reject metadata with '.' for '$' in key names.
Change classifier gear to sanitize it's metadata
Change gear spec to require metadata not have '.' or '$' in key names of output.

Post bali.02

Change classifier gear to store metadata keys as dicom group,element, implementing dictionary for API and/or frontend to map those to text strings.
Update gear spec to recommend 3rd party gear keys be as cannonical as possible, and provide guidance how dicom metadata should be key'd to leverage our dictionary and search capabilities.
Rerun classifier gear to make suer all dicom metadata now fits our new guidelines.

The longer it takes us to get to that last item, the more data will be in the field that will need to be re-processed.


Refactor/functional/flatten

Codecov Report

Merging #21 (9e5e3d3) into development (e4c3af6) will not change coverage.
The diff coverage is 100.00%.

:exclamation: Current head 9e5e3d3 differs from pull request most recent head e5cb53c. Consider uploading reports for the commit e5cb53c to get more accurate results

@@              Coverage Diff              @@
##           development       #21   +/-   ##
=============================================
  Coverage       100.00%   100.00%
=============================================
  Files               18        19    +1
  Lines             1175      1215   +40
=============================================
+ Hits              1175      1215   +40

Impacted Files
Coverage Δ

include/Simple-Utility/functional/base.hpp
100.00% <100.00%> (ø)

include/Simple-Utility/functional/predicate.hpp
100.00% <100.00%> (ø)

include/Simple-Utility/functional/transform.hpp
100.00% <100.00%> (ø)

tests/functional/base.cpp
100.00% <100.00%> (ø)

tests/functional/predicate.cpp
100.00% <100.00%> (ø)

tests/functional/transform.cpp
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e4c3af6...e5cb53c. Read the comment docs.


[Messages light] - new message GUI app

a new app that replaces the message app GUI.

Thanks! Let's wait until https://github.com/espruino/BangleApps/pull/2297 goes in and then we should be able to tweak and merge this and the module renaming won't be needed

yes yes, no problem!
since perhaps the work on the message could be long, I thought of starting to publish this (also for my order) and then, when the new methodology is completed, modify this app.
(I also uploaded it, so in splitting the message, there is a reference to what could be needed in a "custom" GUI message)

Yes, ok - makes sense. I'll merge this now and I think we can remove the extra bits pretty easily later
In future, it'd be handy if you could make your version numbering have 2 decimal places like other apps though? - even if you want to start with 1.xx instead of 0.xx is just seems less confusing for users.

oh ok, no problem!


No way to control the Bestway Saluspa Coronado WiFi

Version 1.3.0

Bestway device
Bestway Saluspa Coronado (WiFi model from Costco)
Describe the bug
I installed the integration through HACS and it installed fine.  I was able to successfully sign in to my Bestway account through the US and it saw my hot tub.  The problem is that all I see are 5 entities:
Bestway MCU Hardware Version
Bestway MCU Software Version
Bestway Protocol Version
Bestway Wi-Fi Hardware Version
Bestway Wi-Fi Software Version
I don't have any way of controlling the spa at all.  I've attached a screenshot of what I get when the device is added.

This is probably a new and unsupported model. There should be more information in the logs. If you can share that, it will help add support in the future.

Here are all the Bestway logs since it was installed (still is):
This error originated from a custom integration.

Logger: custom_components.bestway.bestway.api
Source: custom_components/bestway/bestway/api.py:249
Integration: Bestway (documentation, issues)
First occurred: November 3, 2023, 12:37:32 PM (14872 occurrences)
Last logged: 4:44:59 PM

Status for unknown device type 'Airjet_V01' returned: {"E19": 0, "E18": 0, "E32": 0, "E31": 0, "E30": 0, "E11": 0, "E10": 0, "E13": 0, "E12": 0, "E15": 0, "E14": 0, "E17": 0, "E16": 0, "word5": 0, "word4": 0, "word7": 40, "word6": 0, "word1": 0, "word0": 0, "E29": 0, "word2": 0, "option6": 10244, "ver": 1174, "E22": 0, "jet": 0, "E23": 0, "power": 1, "Tnow": 104, "option7": 10260, "option4": 0, "option5": 27184, "option2": 0, "option3": 30, "option0": 59940, "option1": 59940, "E24": 0, "E25": 0, "E26": 0, "E27": 0, "E20": 0, "E21": 0, "E08": 0, "E09": 0, "E06": 0, "E07": 0, "E04": 0, "E05": 0, "E02": 0, "E03": 0, "E01": 0, "bit7": 0, "heat": 0, "Tunit": 0, "wave": 0, "word3": 0, "Tset": 104, "filter": 0, "E28": 0, "bit6": 0, "bit5": 0, "bit4": 0, "bit3": 0, "bit2": 1}
Status for unknown device type 'Airjet_V01' returned: {"E19": 0, "E18": 0, "E32": 0, "E31": 0, "E30": 0, "E11": 0, "E10": 0, "E13": 0, "E12": 0, "E15": 0, "E14": 0, "E17": 0, "E16": 0, "word5": 0, "word4": 0, "word7": 40, "word6": 0, "word1": 0, "word0": 0, "E29": 0, "word2": 0, "option6": 10244, "ver": 1174, "E22": 0, "jet": 0, "E23": 0, "power": 1, "Tnow": 102, "option7": 10260, "option4": 0, "option5": 27184, "option2": 0, "option3": 30, "option0": 59940, "option1": 59940, "E24": 0, "E25": 0, "E26": 0, "E27": 0, "E20": 0, "E21": 0, "E08": 0, "E09": 0, "E06": 0, "E07": 0, "E04": 0, "E05": 0, "E02": 0, "E03": 0, "E01": 0, "bit7": 0, "heat": 0, "Tunit": 0, "wave": 0, "word3": 0, "Tset": 104, "filter": 0, "E28": 0, "bit6": 0, "bit5": 0, "bit4": 0, "bit3": 0, "bit2": 1}
Status for unknown device type 'Airjet_V01' returned: {"E19": 0, "E18": 0, "E32": 0, "E31": 0, "E30": 0, "E11": 0, "E10": 0, "E13": 0, "E12": 0, "E15": 0, "E14": 0, "E17": 0, "E16": 0, "word5": 0, "word4": 0, "word7": 39, "word6": 0, "word1": 0, "word0": 0, "E29": 0, "word2": 0, "option6": 10244, "ver": 1174, "E22": 0, "jet": 0, "E23": 0, "power": 1, "Tnow": 102, "option7": 10260, "option4": 0, "option5": 27184, "option2": 0, "option3": 30, "option0": 59940, "option1": 59940, "E24": 0, "E25": 0, "E26": 0, "E27": 0, "E20": 0, "E21": 0, "E08": 0, "E09": 0, "E06": 0, "E07": 0, "E04": 0, "E05": 0, "E02": 0, "E03": 0, "E01": 0, "bit7": 0, "heat": 0, "Tunit": 0, "wave": 0, "word3": 0, "Tset": 104, "filter": 0, "E28": 0, "bit6": 0, "bit5": 0, "bit4": 0, "bit3": 0, "bit2": 1}
Status for unknown device type 'Airjet_V01' returned: {"E19": 0, "E18": 0, "E32": 0, "E31": 0, "E30": 0, "E11": 0, "E10": 0, "E13": 0, "E12": 0, "E15": 0, "E14": 0, "E17": 0, "E16": 0, "word5": 0, "word4": 0, "word7": 38, "word6": 0, "word1": 0, "word0": 0, "E29": 0, "word2": 0, "option6": 10244, "ver": 1174, "E22": 0, "jet": 0, "E23": 0, "power": 1, "Tnow": 100, "option7": 10260, "option4": 0, "option5": 27184, "option2": 0, "option3": 30, "option0": 59940, "option1": 59940, "E24": 0, "E25": 0, "E26": 0, "E27": 0, "E20": 0, "E21": 0, "E08": 0, "E09": 0, "E06": 0, "E07": 0, "E04": 0, "E05": 0, "E02": 0, "E03": 0, "E01": 0, "bit7": 0, "heat": 0, "Tunit": 0, "wave": 0, "word3": 0, "Tset": 104, "filter": 0, "E28": 0, "bit6": 0, "bit5": 0, "bit4": 0, "bit3": 0, "bit2": 1}
Status for unknown device type 'Airjet_V01' returned: {"E19": 0, "E18": 0, "E32": 0, "E31": 0, "E30": 0, "E11": 0, "E10": 0, "E13": 0, "E12": 0, "E15": 0, "E14": 0, "E17": 0, "E16": 0, "word5": 0, "word4": 0, "word7": 38, "word6": 0, "word1": 0, "word0": 0, "E29": 0, "word2": 0, "option6": 10244, "ver": 1174, "E22": 0, "jet": 0, "E23": 0, "power": 1, "Tnow": 99, "option7": 10260, "option4": 0, "option5": 27184, "option2": 0, "option3": 30, "option0": 59940, "option1": 59940, "E24": 0, "E25": 0, "E26": 0, "E27": 0, "E20": 0, "E21": 0, "E08": 0, "E09": 0, "E06": 0, "E07": 0, "E04": 0, "E05": 0, "E02": 0, "E03": 0, "E01": 0, "bit7": 0, "heat": 0, "Tunit": 0, "wave": 0, "word3": 0, "Tset": 104, "filter": 0, "E28": 0, "bit6": 0, "bit5": 0, "bit4": 0, "bit3": 0, "bit2": 1}

Then:
This error originated from a custom integration.

Logger: custom_components.bestway.coordinator
Source: custom_components/bestway/bestway/api.py:418
Integration: Bestway (documentation, issues)
First occurred: November 4, 2023, 4:28:56 AM (25 occurrences)
Last logged: November 4, 2023, 4:47:39 AM

Data update failed
Traceback (most recent call last):
  File "/config/custom_components/bestway/coordinator.py", line 36, in _async_update_data
    await self.api.refresh_bindings()
  File "/config/custom_components/bestway/bestway/api.py", line 140, in refresh_bindings
    device.device_id: device for device in await self._get_devices()
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/bestway/bestway/api.py", line 147, in _get_devices
    api_data = await self._do_get(f"{self._api_root}/app/bindings", headers)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/bestway/bestway/api.py", line 418, in _do_get
    response.raise_for_status()
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1005, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 500, message='INTERNAL SERVER ERROR', url=URL('https://usapi.gizwits.com/app/bindings')

Then:
This error originated from a custom integration.

Logger: custom_components.bestway.coordinator
Source: custom_components/bestway/coordinator.py:35
Integration: Bestway (documentation, issues)
First occurred: November 6, 2023, 2:18:04 PM (1 occurrences)
Last logged: November 6, 2023, 2:18:04 PM

Data update failed
Traceback (most recent call last):
  File "/config/custom_components/bestway/coordinator.py", line 37, in _async_update_data
    return await self.api.fetch_data()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/bestway/bestway/api.py", line 166, in fetch_data
    latest_data = await self._do_get(
                  ^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/bestway/bestway/api.py", line 417, in _do_get
    response = await self._session.get(url, headers=headers)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 560, in _request
    await resp.start(conn)
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 899, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/streams.py", line 616, in read
    await self._waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/config/custom_components/bestway/coordinator.py", line 35, in _async_update_data
    async with async_timeout.timeout(10):
  File "/usr/local/lib/python3.11/site-packages/async_timeout/__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "/usr/local/lib/python3.11/site-packages/async_timeout/__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
TimeoutError

Then:
This error originated from a custom integration.

Logger: custom_components.bestway.coordinator
Source: custom_components/bestway/bestway/api.py:417
Integration: Bestway (documentation, issues)
First occurred: November 7, 2023, 3:41:51 AM (4 occurrences)
Last logged: November 7, 2023, 3:43:36 AM

Data update failed
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiohttp/connector.py", line 1155, in _create_direct_connection
    hosts = await asyncio.shield(host_resolved)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/connector.py", line 874, in _resolve_host
    addrs = await self._resolver.resolve(host, port, family=self._family)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/resolver.py", line 33, in resolve
    infos = await self._loop.getaddrinfo(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 867, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/socket.py", line 962, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -3] Try again

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/config/custom_components/bestway/coordinator.py", line 36, in _async_update_data
    await self.api.refresh_bindings()
  File "/config/custom_components/bestway/bestway/api.py", line 140, in refresh_bindings
    device.device_id: device for device in await self._get_devices()
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/bestway/bestway/api.py", line 147, in _get_devices
    api_data = await self._do_get(f"{self._api_root}/app/bindings", headers)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/bestway/bestway/api.py", line 417, in _do_get
    response = await self._session.get(url, headers=headers)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 536, in _request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/connector.py", line 540, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/connector.py", line 901, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/connector.py", line 1169, in _create_direct_connection
    raise ClientConnectorError(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host usapi.gizwits.com:443 ssl:default [Try again]

Then:
This error originated from a custom integration.

Logger: custom_components.bestway.coordinator
Source: custom_components/bestway/bestway/api.py:416
Integration: Bestway (documentation, issues)
First occurred: November 3, 2023, 4:58:06 PM (17 occurrences)
Last logged: November 7, 2023, 8:24:22 PM

Data update failed
Traceback (most recent call last):
  File "/config/custom_components/bestway/bestway/api.py", line 417, in _do_get
    response = await self._session.get(url, headers=headers)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 536, in _request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/connector.py", line 540, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/connector.py", line 901, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/connector.py", line 1178, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/connector.py", line 980, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)  # type: ignore[return-value]  # noqa
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1112, in create_connection
    transport, protocol = await self._create_connection_transport(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1145, in _create_connection_transport
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/config/custom_components/bestway/coordinator.py", line 36, in _async_update_data
    await self.api.refresh_bindings()
  File "/config/custom_components/bestway/bestway/api.py", line 140, in refresh_bindings
    device.device_id: device for device in await self._get_devices()
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/bestway/bestway/api.py", line 147, in _get_devices
    api_data = await self._do_get(f"{self._api_root}/app/bindings", headers)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/bestway/bestway/api.py", line 416, in _do_get
    async with async_timeout.timeout(_TIMEOUT):
  File "/usr/local/lib/python3.11/site-packages/async_timeout/__init__.py", line 141, in __aexit__
    self._do_exit(exc_type)
  File "/usr/local/lib/python3.11/site-packages/async_timeout/__init__.py", line 228, in _do_exit
    raise asyncio.TimeoutError
TimeoutError

Then:
This error originated from a custom integration.

Logger: custom_components.bestway.coordinator
Source: helpers/update_coordinator.py:235
Integration: Bestway (documentation, issues)
First occurred: November 3, 2023, 4:58:06 PM (11 occurrences)
Last logged: November 7, 2023, 8:24:22 PM

Error fetching Bestway API data: Error communicating with API:
Error fetching Bestway API data: Error communicating with API: 500, message='INTERNAL SERVER ERROR', url=URL('https://usapi.gizwits.com/app/bindings')

I just bought the same Coronado spa. I get the exact same results...

I'm happy to help w/ logs as well.  Just let us know what you need?

@cdpuk  any pointers??

I see in those logs that the device is identifying itself as "Airjet_V01", which is the same issue as #41. I'm going to close this one and track the issue over there. I think we've already got enough in the way of log messages to understand the device, but what's needed is a bit of time to code this all up properly. I've got some WIP but not available for use quite yet.


README: fix typo

Thank you!

Mark @zqb-all, in the xpack-develop branch there were some changes to the README-DEVELOPER file. I merged them into the xpack branch after applying your PR.
Could you review the new version, both for content and typos? Thank you!

Sure. I just read it over and don't see any problems.

Do you think that the instructions are correct and clear enough for first time readers, in order to successfully run a build on their machines? Did you do this? Was it successful?

I think it's clear enough，I ran into this problem the first time I compiled: https://github.com/xpack-dev-tools/openocd-xpack/issues/25
The problem was solved with your help and is works well now

I think it's clear enough，

Ok.

I ran into this problem the first time I compiled: https://github.com/xpack-dev-tools/openocd-xpack/issues/25
The problem was solved with your help and is works well now

Yes, but by that time the README-DEVELOPER was not available, and you used the more complicated instructions in the README-MAINTAINER file; plus that you faced some download failures, which were later fixed with some fixes in xpm.
Now things should be easier with the new instructions in README-DEVELOPER.
The question was in fact related to the new instructions.


Multiple tests in the grid client tests are failing

Description
Multiple tests in the grid client tests are failing with the following error.
 var logFunction = require('debug')(identifier);
                                      ^

TypeError: require(...) is not a function
    at Object.createBufferingLogger [as BufferingLogger] (/home/mohamed/Desktop/Threefold Repos/sdkts/tfgrid-sdk-ts/node_modules/websocket/lib/utils.js:24:39)
    at new WebSocketConnection (/home/mohamed/Desktop/Threefold Repos/sdkts/tfgrid-sdk-ts/node_modules/websocket/lib/WebSocketConnection.js:42:25)
    at WebSocketClient.Object.<anonymous>.WebSocketClient.succeedHandshake (/home/mohamed/Desktop/Threefold Repos/sdkts/tfgrid-sdk-ts/node_modules/websocket/lib/WebSocketClient.js:343:22)
    at WebSocketClient.Object.<anonymous>.WebSocketClient.validateHandshake (/home/mohamed/Desktop/Threefold Repos/sdkts/tfgrid-sdk-ts/node_modules/websocket/lib/WebSocketClient.js:332:10)
    at ClientRequest.handleRequestUpgrade (/home/mohamed/Desktop/Threefold Repos/sdkts/tfgrid-sdk-ts/node_modules/websocket/lib/WebSocketClient.js:261:14)
    at ClientRequest.emit (node:events:513:28)
    at TLSSocket.socketOnData (node:_http_client:582:11)
    at TLSSocket.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at TLSSocket.Readable.push (node:internal/streams/readable:234:10)
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)

Node.js v18.14.2
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Logs
Full test results from the latest workflow run. https://github.com/threefoldtech/tfgrid-sdk-ts/runs/16162097060
Steps to reproduce

Run grid client tests.

After running the tests locally they appear to be passing but the suite itself fails with the same error mentioned above.

found the original error on the workflow and it was
  ● Test suite failed to run

    TypeError: Cannot read properties of undefined (reading 'isConnected')

      58 |             if (connection && this.api.isConnected)
      59 |                 return;
    > 60 |             if (!connection && !this.api.isConnected)
         |                                          ^
      61 |                 return;
      62 |             await new Promise(f => setTimeout(f, 100));
      63 |         }

      at TFClient.isConnected [as wait] (../tfchain_client/dist/node/client.js:60:42)
      at TFClient.disconnect (../tfchain_client/dist/node/client.js:114:13)
      at GridClient.disconnect (src/client.ts:219:24)
      at Object.<anonymous> (tests/modules/kvstore.test.ts:63:10)

the issue is the client on disconnect doesn't drop the event listener, so the client delete the api object and try to create another one while disconnecting.
The Solution
the client should properly drop the event when the disconnect method is called

Verified, It pass now.
From the workflow
https://github.com/threefoldtech/tfgrid-sdk-ts/actions/runs/6006557514
And locally


git: respect $SSL_CERT_FILE

This allows git to work on systems without
/etc/ssl/certs/ca-certificates.crt, such as OS X, instead of failing
with "error setting certificate verify locations".
Things done

[x] Tested using sandboxing (nix-build --option build-use-chroot true or nix.useChroot on NixOS)
Built on platform(s)

[x] NixOS
[x] OS X
[x] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

I think it's a good idea but would like a second opinion on this. /cc @vcunat

Right now, nixpkgs.git is unusable on OS X by default without this.

Also, while we're at it, we could get rid of the custom env var in fetchgit perhaps:
diff --git a/pkgs/build-support/fetchgit/default.nix b/pkgs/build-support/fetchgit/default.nix
index 7f98c97..9310e34 100644
--- a/pkgs/build-support/fetchgit/default.nix
+++ b/pkgs/build-support/fetchgit/default.nix
@@ -54,7 +54,7 @@ stdenv.mkDerivation {

   inherit url rev leaveDotGit fetchSubmodules deepClone branchName;

-  GIT_SSL_CAINFO = "${cacert}/etc/ssl/certs/ca-bundle.crt";
+  SSL_CERT_FILE = "${cacert}/etc/ssl/certs/ca-bundle.crt";

   impureEnvVars = [
     # We borrow these environment variables from the caller to allow

Any concrete requests for changes to this patch?

Nope, all good :)


[hotfix] git add check was missing

fix(git-wrapper missing git add handler): added case where user did not git add
git add check was missing

not yet ready

I was thinking on an different approach, more like this one: https://github.com/mcwhittemore/staged-git-files
Showing the files that are ready to be committed and eventually commit. We should find out which one is the best

Would you like it better to also try and apply "adds" from within the script, having the user choose which files to add or * directly, based on choice?

Can you please close this PR? I should have resolved with the latest commits using https://git-scm.com/docs/git-status#Documentation/git-status.txt---porcelainltversiongt


Imperfect detect collision strategy used in freespace planner

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I'm convinced that this is not my fault but a bug.

Description

Planning succeeded.
First segment of trajectory traveled normally.
Part of second segment of trajectory is out of costmap when arriving at the end of first segment so the system will replan but it finally stuck because the goal point is out of costmap too.

https://user-images.githubusercontent.com/103234047/203288355-ca4593a1-2739-43df-b559-42ebdb42d039.mp4
Expected behavior
The vehicle arrives at the goal or the path is successfully re-planned.
Actual behavior
The parking process stuck.
Steps to reproduce

Start autoware planning simulator using default. (select astar algorithms)
Set start and goal position using the commands below.

ros2 topic pub --once /initialpose geometry_msgs/msg/PoseWithCovarianceStamped "{header: {stamp: {sec: 0.0, nanosec: 0.0}, frame_id: map}, pose: {pose: {position: {x: 3713.99, y: 73732.0, z: 0.664812}, orientation: {x: 0.0, y: 0.0, z: 0.850511, w: 0.525957}}, covariance: [1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 1.]}}"
ros2 topic pub --once /planning/mission_planning/goal geometry_msgs/msg/PoseStamped "{header: {stamp: {sec: 0.0, nanosec: 0.0}, frame_id: map}, pose: {position: {x: 3746.88, y: 73748.5, z: 0.0}, orientation: {x: 0.0, y: 0.0, z: 0.847164, w: 0.531332}}}"

Engage the vehicle.

Versions

OS: Ubuntu 20.04
ROS2: Galactic
Autoware: Autoware.universe:main

Possible causes
Demonstrated in issue description part.
Additional context
Expand the costmap size could temporarily solve the problem.
https://user-images.githubusercontent.com/103234047/203290907-6aa3dad7-7a51-47e6-addb-127b4027ba1b.mp4
No response

@TakaHoribe Could you check this issue?

I want know how can I using topic control "engage"

@NorahXiong Could you share the map used in the simulation?

@NorahXiong Could you share the map used in the simulation?
It is the Autoware sample map. I think you can find it in the tutorial website. Anyway, the vector map is pasted below for you.
lanelet2_map.zip


Haxe - Suggestion - Enum autocomplete and templating with all possible values.

Hi,
It would be very usefull to be able to leverage the Enum mechanism of haxe in flahdevelop by being able to pre fill a switch Template with ALL values of an Enum as cases, and also give autocomplete tooltip info with all the possible values.
Thanks

via #2466 #2507 #2508

hi your updates of switch autocomplete are great, but it is not exactly what was suggested.
i meant to pre fill all cases one under the other for the switch.
from
switch(varName) {}
to
switch(varName) {
case VAL1:
case VAL2:
case VAL3:
}
in one keyboard typing.
if there are a lot of them, you dont want to put them one at one by hand even with autocomplete.
thanks

You can try var$(EntryPoint)Name where $(EntryPoint) is cursor press shift+ctrl+1 or Refactor -> Code Generator then select and execute Generate swith ;)


feat(home): add Wallora logo to header

Fixes: #54
Logged in case:

Logged out case:

since Aryaman was assigned to the issue, let's allow him to send the PR if he can

I have assigned myself to the terms and privacy issue


changes to avk for cubemap texture support

Regarding failed checks:
msvc can't find Vulkan headers.
clang fails including a std header:
In file included from /home/runner/work/Auto-Vk/Auto-Vk/src/avk.cpp:3:
In file included from /home/runner/work/Auto-Vk/Auto-Vk/include/avk/avk.hpp:30:
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ranges:3392:19: error: missing 'typename' prior to dependent type name 'iterator_traits<iterator_t<_Base> >::iterator_category'


[1Password Extension] Add second [work/personal] account?

Extension
https://www.raycast.com/khasbilegt/1password
Description
I can't find the ability to add a second account, e.g., your work or personal account, for 1password in Raycast with this extension. This is essential for daily usage.
Who will benefit from this feature?
Every 1password user with > one 1password account.
Anything else?
No response

+1

this may help: https://developer.1password.com/docs/cli/use-multiple-accounts/


Document the status codes of pulumi updates

Hello!

Vote on this issue by adding a 👍 reaction
If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

When running an engine update Pulumi prints various status codes such as "update", "delete-replaced" etc. We should document what these mean under resource/concepts. This page could also talk about other related concepts to how updates work such as the different operations involved (Check, Diff, Update, Create, Delete).

One thing I would point out that is confusing to me, is that in a chageset, I see 2 create-replacement and only 1 delete-replaced.   1 was deleted and 2 were created?  😕
Maybe because these are AWS Policies and olicyAttachments?
Changes below, names changed in a programmatic way (same name same change)
Changes:

    Type                         Name                                           Operation
+   aws:s3:Bucket                bl-ds-aabbccdd-di-raw                          create
+   aws:s3:Bucket                bl-ds-aabbccdd-di                              create
+   aws:s3:Bucket                bl-ds-code-depot                               create
+   aws:s3:BucketPolicy          diRawBucketPolicy                              create
+   aws:glue:CatalogDatabase     di-sis                                         create
+   aws:glue:CatalogTable        di-sis/dilling                                 create
+   aws:glue:CatalogTable        di-sis/aabbinfo_outbound_request               create
+   aws:glue:CatalogTable        di-sis/aabbinfo_inbound_response               create
+   aws:glue:CatalogTable        di-sis/sanitized_response                      create
+   aws:glue:CatalogTable        di-sis/sanitized_request                       create
+   aws:iam:Role                 AWSGlueServiceRole-di                          create
++  aws:iam:RolePolicyAttachment qa/sisdiDoc                                    create-replacement
--  aws:iam:RolePolicyAttachment mular_drg/sisdiDoc                             delete-replaced
++  aws:iam:Policy               sisdiDoc                                       create-replacement
+   aws:glue:Job                 sis                                            create
+   aws:iam:UserPolicy           ZeToolService                                  create
+   aws:iam:RolePolicyAttachment AWSGlueServiceRole-di/AWSGlueServiceRole       create
~   aws:iam:Policy               di                                             update
+   aws:iam:RolePolicyAttachment mular_drg/di                                   create
-   aws:iam:RolePolicyAttachment AWSGlueServiceRole-sisdiDoc/AWSGlueServiceRole delete
-   aws:iam:Role                 AWSGlueServiceRole-sisdiDoc                    delete
-   aws:iam:UserPolicy           ZeToolSvc                                      delete

adding to this week's triage list


[MacOS] When having custom resolver configuration inside /etc/resolver/ that was added by a vpn configuration, cli cannot resolve the address and connect to jira instance

Describe the bug
I'm using openconnect with vpn-slice configured for split-horizon DNS resolvers (ie - domains inside vpn are configured to use specific dns settings via added domain configs to /etc/resolver/<domain name>). When trying to resolve such domain (a jira instance) the setup fails with an error. When DNS is set globally to use the vpn DNS, everything works fine.
Please provide following details

JiraCLI Version:
(Version="1.1.0", GitCommit="3b93e147eac468ad985bdece27469153b4bb2814", CommitDate="2022-08-14T08:10:40+00:00", GoVersion="go1.18.3", Compiler="gc", Platform="darwin/arm64")

Are you using Jira cloud or on-premise jira server? Also mention the version for on-premise installation.
on premise, don't think the version is needed here :)

What operating system are you using? Also mention version.
 macOS Monterey 12.5.1

What terminal are you using? Also mention version.
iterm2 Build 3.5.0beta6

To Reproduce
Steps to reproduce the behavior:

Create domain configuration in /etc/resolver/<domain> - the domain cannot be accessible while using your usual DNS configuration otherwise it will just resolve the domain correctly
Execute jira-init and try to setup jira instance that's only resolvable by dns configured in step 1
See error:

? Installation type: Local
? Link to Jira server: // secret //
? Login username: // secret //

✗ Unable to generate configuration: dial tcp: lookup // secret // on // secret //: no such host

Expected behavior
Configuration ended successfully
Screenshots

Additional context
I'm using openconnect with vpn-slice, but I think this can be reproduced by just adding a DNS configuration for a domain manually. When I set the DNS that sees the jira instance for the whole system (globally), everything works fine. The problem is only with custom resolver configuration.

@rymek looks like you need a way to set custom DNS resolver. Just wondering if it will work if the tool allow you to set a resolver IP (and protocol) for communication with the Jira API?

i think this might be a bug in some go library you're using. This is a os feature that you can add a resolver for a specific domain to resolver dir. Every other app I'm using on the system is working correctly. I'm suspecting w bug in this dial library but haven't got time to look though it yet and I don't really know go so it's kinda more difficult for me 🙂

This indeed seem to be an issue in the standard go library https://github.com/golang/go/issues/12524 and will wait for the upstream fix.
As a workaround suggested in the comments, you can try compiling locally with cgo.
CGO_ENABLED=1 GOARCH=arm64 go build

I installed jira-cli on my company-managed Macbook with your Homebrew tap, and ran into this issue. I was able to resolve it by running brew install --HEAD jira-cli, which built the formula locally rather than installing the binary version. I did not need to specify CGO_ENABLED. Perhaps the binaries need to be rebuilt with the latest version of Go?

Hi @MageJohn, the latest binary v1.3.0 is compiled with go1.19.4. what version did you use to compile locally?

Hi @ankitpokhrel,
Apologies for the late reply; I believe it was the latest go at the time of the build, go1.19.5.


Update nav labels for middleware

What kind of changes does this PR include?

New or updated content

Description

Closes #3510
What does this PR change? Removes "experimental" from sidebar titles for the middleware page.

Thank you @TheOtterlord! This really saves us time when we can just shout out a task and it's done!  🙌

Yours to merge, @TheOtterlord!  🥳


in Android O not showing view

Unable to add window android.view.ViewRootImpl$W@8d54593 -- permission denied for window type 2002

I'm having same issue. I solved using
Intent myIntent = new Intent(Settings.ACTION_MANAGE_OVERLAY_PERMISSION);
myIntent.setData(Uri.parse("package:" + getPackageName()));
startActivityForResult(myIntent, APP_PERMISSIONS);
But still stand out window not showing for oreo.

this library is not working in API26, API27.
how can i fix it in these versions?

I've fixed it by downloading the sources and changing the following:

Replace startService with startForegroundService for API 26+ in StandOutWindow.java, line 119
Replace TYPE_PHONE with TYPE_APPLICATION_OVERLAY for API 26+ in StandOutWindow.java, line 1813

And keep in mind that the persistent notifications needs a notification channel.

can't resolve the method startForegroundService any ideas ?

Only notification is working but the Window is not showing please help

@rubenmx  I've fixed the notification but Floating window is not showing startForegroundService is not working

Did you change TYPE_PHONE within the StandOutLayoutParams constructor to TYPE_APPLICATION_OVERLAY for API 26+? That was the reason it wasn't showing with my code.
TYPE_PHONE is no longer supported.

@rubenmx  public static void show(Context context, 			Class<? extends StandOutWindow> cls, int id) { 		if (Build.VERSION.SDK_INT >= 26) { 			context.startForegroundService(getShowIntent(context,cls,id)); 		} 		else { 			context.startService(getShowIntent(context, cls, id)); 		} 	}
And this is the constructor
super(200, 200, TYPE_SYSTEM_OVERLAY, 					StandOutLayoutParams.FLAG_NOT_TOUCH_MODAL 							| StandOutLayoutParams.FLAG_WATCH_OUTSIDE_TOUCH, 					PixelFormat.TRANSLUCENT);
no luck :(

@rubenmx my bad i was using TYPE_SYSTEM_OVERLAY instead of TYPE_APPLICATION_OVERLAY


Magento connect manager error install

Hello support,
When I update or install a exstension I get the follow error.
CONNECT ERROR: Package community/symmetrics_trustedrating 0.2.6: requires PHP version >= 5.2.0 and <= 6.0.0 current is: 7.0.4-1byte1
Best regards,
Michel

This isn't an issue with this module.
Using Magento Connect after you've upgraded to PHP 7.0 is very risky since many extensions will have these kinds of min/max PHP version restrictions in their package XML.
If in doubt you should attempt to upgrade on a dev site first.  Alternatively you can edit the package.xml and repackage & reinstall the archive.

Please close.

I think this is the right place to file that issue: https://github.com/symmetrics/trustedshops_trustedrating/issues


use gpg instead of gpg2

Hi there :wave:,
TLDR:
This pretty simple PR would use the gpg command instead of gpg2.
Reasoning:
Most, if not all distros have deprecated gpg v1 and the standard gpg package is now v2. gpg2 is currently a "dummy transitional package." and many distros do not install it by default, meaning extra user interaction is currently required to enable gpg signature validation of appimages.
Research:
I checked quite a few distros and even the older supported ones like ubuntu 18.04 and centos 7 are using gpg v2 as the standard installed package, meaning I don't see any reason to continue using gpg2.
This is somewhat in relation to https://github.com/AppImage/AppImageUpdate/issues/97

Thanks for pointing this out. There is a difference between gpg v2 and gpg2? Talk about confusing...

:+1: It is kinda confusing. Perhaps a better way to explain it:
Back in the day:
gpg was gpg v1
gpg2 was gpg v2
simple enough.
Then they upgraded the gpg package to gpg v2, deprecated and moved v1 to gpg1 and kept gpg2 as a transitional package so nothing broke.
Now:
gpg == gpg2 == gpg v2
since the gpg package is installed by default and gpg2 is not....
Ofc this PR could be a one-liner, I just thought perhaps for consistency & clarity to change all the references. :shrug:

Please add this as a fallback. gpg2 should be preferred if available, otherwise gpg can be used.

Since I'll have to rebase my PR anyway due to https://github.com/AppImage/AppImageUpdate/commit/eb3d09d2a21d84e8fbeacf830b357895d48e70ab, I'll merge this PR first.


Added Icons for Scripting API

[x] This contribution adheres to CONTRIBUTING.md.

What does this Pull Request accomplish?
Added Icons for the Scripting API VIs.
2 VIs are not modified in this PR because they are modified in other PRs.
The 2 VIs will be addressed in a future PR.
Why should this Pull Request be merged?
The VIs have default or mock Icons.
What testing has been done?
N/A

For all the polyVI's, I prefer having icons for the actual polyVI, instead of relying on each subVI's icon. I am ok with all the ones you created (good job!), but please create icons for each polyVI, as well.

Nice work. ;)

The Template, as I see it, is inspired from the VS Native API Template. I am not sure I fully agree with this, since it might be misleading for some users - they might think that our scripting is part of native VeriStand API as well, and it is not.

The same template was used for Routing and Faulting Scripting API, and that was already reviewed and released.

I would probably favor our Icons to follow the same style (or similar) to the ones used in the other libraries of the Instrument Addon - the Engine, Configuration and Shared libraries. As it is, it feels disconnected ... again, this is just an opinion - I prefer consistency between related API.
The API users will probably never check the other Instrument Addon libraries VIs as they only care having a fast way of using the addon. I agree to have consistency if it makes sense, but if we can do better, why won't we?

I would probably use more words in the Icons - sometimes they convey the meaning much better then glyphs. Typically, I am using glyphs when, due to lack of space, I cannot put words and I need a higher degree of abstraction.
As a preference I am more comfortable with glyph's over words. Glyph's look more like native LabVIEW implementations, and if done proper they can give a faster sense of what's happening than words can.

I think you missed the icons for the SubVIs "Create Instrument Addon.vi" and the "Create Instrument Addon from Configuration File.vi" ...
He skipped those 2 on purpose to not generate conflicts with your and Zoli's work.


Not working well with the Mentions extensions

Thank you for an awesome plugin, this is exactly what I needed. There is a weird interaction with the Mention plugin:
While dragging, it correctly shows that it will drag the block, as soon as I drop it, it only takes the actual mention element, and leaves the remaining text in the original position.
What I'm seeing is below:

I just tested it and it seems to work. Can you provide the code to reproduce your behaviour?

Hi,
Well, its pretty standard stuff, the only thing that may be different could be the way we set up the Mentions plugin:
          Mention.extend({
            name: 'substitution',
          }).configure({
            HTMLAttributes: {
              class: 'mention',
            },
            suggestion: makeSuggestionOptions(makeSubstitutions(substitutions)),
            renderHTML({ node }) {
              return ['span', { class: 'mention' }, `${node.attrs.label ?? node.attrs.id}`];
            },
            renderText({ node }) {
              return `${node.attrs.label ?? node.attrs.id}`;
            },
          })

Also of interest maybe would be that the following node, when dragged, "drags" only the mention item, not the block:
    {
      "type": "paragraph",
      "attrs": {
        "textAlign": "left"
      },
      "content": [
        {
          "type": "text",
          "text": "My "
        },
        {
          "type": "substitution",
          "attrs": {
            "id": "country",
            "label": "country"
          }
        },
        {
          "type": "text",
          "text": " "
        }
      ]
    },

The newest commit should fix this issue

@NiclasDev63 Just tested it out, fixes it for me, thank you!


Add .split(',') to new_ticket_cc().

The admin doc string indicates you can pass in a comma separated list.  I don't believe that mail_templated allows a string for recipient_list.  It should be a list of strings.  Adding .split(',') should be safe here.
Recipients takes a python list.
https://github.com/django-helpdesk/django-helpdesk/blob/5112f0dfd07527157157b73a8c20efffd92687fc/helpdesk/management/commands/get_email.py#L507

Since the ticket_cc part of the code has been significantly updated, I believe this issue has been resolved. I'm going to close the ticket since I didn't get a response and assume everything is ok, but please feel free to re-open or file a new ticket if there's a problem. Thanks.


Remove codecov reports

Fixes #4544

Proposed Changes

Remove broken Codacy tool

Readiness Checklist
Author/Contributor

[x] If documentation is needed for this change, has that been included in this pull request

Reviewing Maintainer

[x] Label as breaking if this is a large fundamental change
[x] Label as either automation, bug, documentation, enhancement, infrastructure, or performance

Yeah, this didn't gain traction :(


[#658] Support JPQL Treated paths for WHERE clause

JPA 2.1 introduced the TREAT operator for explicit casting. Initial support for TREAT in the FROM clause was added in PR #705. However, the specification also describes the notion of treated_subpath expressions (chapter <IP_ADDRESS>) that are allowed in the WHERE clause of a query.
The syntax is as follows:

treated_subpath ::= TREAT(general_subpath AS subtype)

single_valued_path_expression ::= qualified_identification_variable |
    TREAT(qualified_identification_variable AS subtype) |
    state_field_path_expression |
    single_valued_object_path_expression

And can be used as such:
SELECT e FROM Employee e JOIN e.projects p WHERE TREAT(p AS LargeProject).budget > 1000

(Example from chapter 4.4.9).
This pull request adds the support for treated subpaths in QueryDSL. It does so by introducing a new PathType and convenience method JPAExpressions.treat.
Fixes #658
A full list of treat support throughout JPA providers can be found at https://github.com/beikov/jpa-treat-variations

@idosal could you review this one? 😄

Sure, I'll get on it soon. Sorry for the delay, it kind of fell between the cracks.


Identifying best approach to automate

I’m automation tester. I’m trying to identify best test automation approach. My application is developed in C# (DB : SQL Server) and I need to write test automation scripts for UI, DB and REST services. I have below two options available.

Selenium + C#
Selenium + Java
What do I need to consider before going to decide one of the above?
If I go with Selenium + Java, I think I need to have below:
Maven, Java, IDE(Eclipse), Bamboo or Jenkins(For CI), Bit bucket(For source code), Cucumber, Junit
If I go with Selenium + C#, what do I need?

Thanks for your help in advance !!

Hi arkadiyala,
This is not the right place to ask a question like you have asked.
Those guys have already developed framework on C# and it is here for free. You either use it or you don't. The question for you is what you personally prefer: Java or C#.
Try to google it: "selenium java vs c#"
Answer for you question: "If I go with Selenium + C#, what do I need?" - is also personal preference. I use Visual Studio Community + Team Server or GitHub, +NUnit, +MSTest and few other things.
Good luck!


add infra folder

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 69.36%. Comparing base (8350d10) to head (13a1d91).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #272      +/-   ##
==========================================
- Coverage   77.16%   69.36%   -7.81%
==========================================
  Files          34       34
  Lines        2409     2409
==========================================
- Hits         1859     1671     -188
- Misses        550      738     +188

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Youtube OAuth Saving initial token.

Getting and saving the OAuth token using a json file

merged it and gonna fix the issues myself


Paperless NGX - Failed to install add-on

Error shows on Hass Addon Install

The command '/bin/sh -c grep -sril "Potential tampering with custom" /etc/cont-init.d /etc/s6-overlay/s6-rc.d | xargs -r rm && grep -sril "/config" /etc/cont-init.d /etc/s6-overlay/s6-rc.d | xargs grep -riL "/config/" | xargs -r rm && for file in $(grep -srl "PUID:-911" /etc/cont-init.d /etc/s6-overlay/s6-rc.d); do sed -i 's/bash/bashio/g' "$file" && sed -i 's/{PUID:-911}/(bashio::config "PUID")/g' "$file" && sed -i 's/{PGID:-911}/(bashio::config "PGID")/g' "$file"; done && sed -i "s|/config|$PAPERLESS_DATA_DIR|g" /etc/cont-init.d/10-adduser && sed -i "s|/config|$PAPERLESS_DATA_DIR|g" /etc/cont-init.d/50-config' returned a non-zero code: 2

Hi, thanks for reporting it was a global issue on all addons. It should install fine now (or at least once your repositories will have updated in your HA instance).

Hi, solved in new version :-) you should now be able to install, or wait until tomorrow and it will install fine


Does this code work in replit?

If it's really possible, I'll try it

What would be the point of putting it on replit?

If it's really possible, I'll try it

it achieved?

If it's really possible, I'll try it

it achieved?

its a python stealer don't use it or it will steal all of your passwords

its a python stealer don't use it or it will steal all of your passwords

WTF SERIOUSLY? I INSTALLED IT ON MY CELL PHONE , idc how I'll see if you got my password 💀


Update Moderation.md msg-id docs to have backwards support for 2.3.4 stable

In 4cbe7ec197fa0d26aaa508b2e20bc1a57d012e67 we moved our documentation to use msg-id on the grounds that msg.id is no longer used, while this is true, we still support msg.id in the current version. msg.id is also the only variation that is supported in stable 2.3.4.
This PR moves to revert back to msg.id and include a note stating that msg-id is supported since chatterino/chatterino2@9b9fd7d403a0b3bd047ba7134de158c4e2fecbc7

credit to @sando for noticing this inconstancy while helping a user having the issue.

Sucks to be on 2.3.4 I guess


Error Class "Js" not found

Class "Js" not found Error in Filament Phone Input Plugin
Description
When using the filament-phone-input plugin, the application encounters a Class "Js" not found error. The issue arises due to a missing import for the Js class in the Blade template located in resources/views/phone-input.blade.php.
It appears that the plugin assumes the Js class is globally accessible, but it does not include a namespace or reference the correct version of the Laravel Js class. As a result, the error occurs during rendering.
This issue seems to be related to a conflict between the Filament\Support\Assets\Js class and Laravel's native Illuminate\Support\Js class. In this case, the application needs the Laravel Illuminate\Support\Js class, but it is not imported in the Blade template where it is used.

Steps to Reproduce

Use PHP 8.3.14, Laravel 11.35.0, and FilamentPHP v3.2.130
Install and enable the filament-phone-input plugin.
Attempt to load a page where the phone-input component is used, such as:

use Ysfkaya\FilamentPhoneInput\Forms\PhoneInput;

$form->schema([
    PhoneInput::make('number')
        ->label(__('Phone Number'))
        ->required(),
])

The application throws the error: Class "Js" not found in: /vendor/ysfkaya/filament-phone-input/resources/views/phone-input.blade.php:25.

Current Behavior
The application fails to render the page and provides the given error.

Expected Behavior
The phone-input.blade.php template should correctly use the Laravel Illuminate\Support\Js class to compile the JavaScript variables, allowing the application to render without error.

Suggested Fix
The template file in the filament-phone-input plugin needs to include the correct namespace for the Js class. Specifically, it should explicitly use the Laravel Illuminate\Support\Js class by adding the proper import statement, like so:
// resources/views/phone-input.blade.php
@php
    use Filament\Support\Facades\FilamentView;
    use Illuminate\Support\Js; // This use statement should be added at line 3

    ...

This ensures that the correct Js class is used when compiling JavaScript variables, avoiding conflicts with other packages (e.g., Filament\Support\Assets\Js).

Environment Details

PHP Version: 8.3.14
Laravel Version: 11.35.0
Plugin Version: ysfkaya/filament-phone-input
Filament Version: 3.2.130

Additional Context
This error occurs in the following file:
vendor/ysfkaya/filament-phone-input/resources/views/phone-input.blade.php.
It appears that other Laravel developers may encounter the same issue if their environment requires Laravel's Js class instead of Filament's Js class.
Thank you for reviewing this issue. Please let me know if additional information is required!

Thank you for your detailed explanation and support. Merged.


iPad Database Settings

Change the iPad Database settings to allow the use of a designated name (“ls1”) or IP address.

What does this mean exactly? Instead of using an ip address use a host name?

Correct.  Would having the option to have both be possible?

You don't need to use the ip address. It can be either an ip address or a valid hostname (the iPad must point to a DNS server that knows about the hostname).

If it is a virtual host?

What do you mean virtual host? Running on a virtual machine?

I’m going to get more information from the person who requested the addition.  It doesn’t work on our end using the host name and I’m trying to nail down the possible issue.
Our server’s virtual hosting for the Admin Tool is setup with a virtualhost proxypass through Apache2 to the Node.js server.
From: Paulo Vitor Magacho<EMAIL_ADDRESS>Sent: Wednesday, November 23, 2016 2:40 PM
To: NASA-Tournament-Lab/NTL-ISS-Food-Intake-Tracker<EMAIL_ADDRESS>Cc: Flottorp, Cheryl (JSC-SA)[WYLE LABORATORIES, INC.]<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [NASA-Tournament-Lab/NTL-ISS-Food-Intake-Tracker] iPad Database Settings (#143)
What do you mean virtual host? Running on a virtual machine?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/NASA-Tournament-Lab/NTL-ISS-Food-Intake-Tracker/issues/143#issuecomment-262624787, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AQq6UdyjjwT51kPvPXkw0Fe5AS9DtgQwks5rBKSRgaJpZM4KqGrL.

@cflottorp the only way to use a name instead of the IP is that in the local network a DNS server is present to provide the IP address based on the host's name.

Hi @cflottorp, just checking in on this.

@rsial2 this can't be fixed. We need a dns server to allow hostname.

hi @cflottorp would NASA want to set that up, or use a pre-existing one?

This may be something that NASA would have to do so I’m trying to confirm with the server owners.  I will need a couple of days to get their response.
From: Rashid Sial<EMAIL_ADDRESS>Sent: Wednesday, December 07, 2016 9:34 AM
To: NASA-Tournament-Lab/NTL-ISS-Food-Intake-Tracker<EMAIL_ADDRESS>Cc: Flottorp, Cheryl (JSC-SA)[WYLE LABORATORIES, INC.]<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [NASA-Tournament-Lab/NTL-ISS-Food-Intake-Tracker] iPad Database Settings (#143)
hi @cflottorphttps://github.com/cflottorp would NASA want to set that up, or use a pre-existing one?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/NASA-Tournament-Lab/NTL-ISS-Food-Intake-Tracker/issues/143#issuecomment-265478648, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AQq6UTPxIrvwMzrADNDZhgLvttnPRfvMks5rFtHQgaJpZM4KqGrL.

no response from owners yet.

@cflottorp have you heard anything about this yet?

This change for the use of the hostname will need to be implemented by NASA if needed due to the architecture of the LS1 server and iPad interface, and with the way the permissions/access is set up in the network.

Thanks, I'll close this ticket.


Add support for custom rules

Hi, nice tool! Thanks.
I would like to add my own custom rules, similar to rules param object, declare custom rules in the config. Would you consider?

Thanks!
The topic of custom rules was raised before in issue #39 - please have a look at my comments there for more. Briefly, this is an eventual goal, but there are a number of questions I need to answer before starting work.
In the meantime, if you list out the rules you have in mind, they might be things I think are broadly applicable and worth adding to the core set of rules myself.

Hi @DavidAnson!
Basically, I would like to be able to fix some common issues when copy/pasting the content (for external checking for example) where the result ends up with double spaces, double dots, a missing space after point or comma, invalid carachters. etc. I would gladly help working on this rule for the core, if you find it useful.

Double spaces was already on my list. What you describe seems like a generalization of that. I will add that to my notes. If you want to create this rule, please begin by proposing a name, parameters, and behavior (in a new issue) so we can be sure we agree on the basics first. Thanks!

More details about a proposed "typography" rule are captured in #51. The idea of custom rules has been on my list, but needs more thought/planning (see #39). I'm going to close this issue since it seems the intent is captured elsewhere. Thanks again!

I have mixed feelings about posting this, but I badly needed a way to extend the existing rules. So I went on a JavaScript adventure and cooked up a hacky solution for it. I'm using webpack and am using the markdownlint in the browser.
The markdownlint.js file looks for ./rules, which contains the ruleset. We can tell webpack to resolve ./rules to whatever we like using aliases. So we'll tell it to resolve to a custom lint-rules.js file:
{
  resolve: {
    alias: {
      './rules': 'path/to-your-custom/lint-rules.js'
    }
  }
}

In our customlint-rules.js file, we can import and subsequently export the original rules like this:
// Import original ruleset
const rules = require('markdownlint/lib/rules');

// Export ruleset
module.exports = rules;

At this stage, the markdownlint library should pretty much just behave as it usually does. But hey! Know we have got an opportunity to modify the rules. So we can do stuff like this:
// Import original ruleset
const rules = require('markdownlint/lib/rules');

// Copied from markdownlint/lib/rules
function filterTokens(params, type, callback) {
  (params.tokenLists[type] || []).forEach(callback);
}

// Copied from markdownlint/lib/rules
function forEachInlineChild(params, type, callback) {
  filterTokens(params, 'inline', (token) => {
    token.children.forEach((child) => {
      if (child.type === type) {
        callback(child, token);
      }
    });
  });
}

const sneakyRegex = /sneaky/g;

rules.push({
  name: 'KG001',
  desc: 'Sneaky beaky rules',
  tags: ['links'],
  aliases: ['capitalize-internal-links'],
  regexp: null,
  func: function KG001(params, errors) {
    forEachInlineChild(params, 'text', (token) => {
      const match = sneakyRegex.exec(token.content);

      if (match) {
        errors.addDetail(token.lineNumber, match[0]);
      }
    });
  }
});

module.exports = rules;

Now we've effectively added a new rule (KG001) to the ruleset that the markdownlint library uses. (Note: I needed to copy a few of the helper methods from the original rules.js, so I could use them in my new custom rule).
Finally I needed to turn on the rule in the markdownlint options config.
config: {
  (...),
  KG001: true // Custom rule for link formatting
}

Warning: It goes without saying that this is not a viable solution and it will certainly not fit every scenario. Even worse, it relies on the internals of the library, so if these changes, this approach will most likely break. But it is a way to add custom rules.

This is awesome! Thank you for sharing. :)


Cannot return null for non-nullable field Organization.id

I am experiencing this issue after upgrading from Prisma 1.29.1 to Prisma 1.30.1
Shows this on Queries and Mutations, even when they are successful.
Here's my graphql query and mutation:
Mutation:
mutation createOrganization($args: OrganizationCreateInput!) {
    createOrganization(data: $args) {
      id
    }
  }

Query:
query getViewerOrganization {
        getViewerOrganization {
            id
        }
    }

Here's my datamodel:
type Organization {
    id: ID! @unique
    name: String!
    .......
}

Here's my Schema:
type Mutation {
      createOrganization(data: OrganizationCreateInput!): Organization!
}

type Query {
      getViewerOrganization(data: OrganizationWhereInput): Organization
}

type Viewer {
    me: User!
}

Here's my resolvers:
Mutation
// @ts-ignore
export const createOrganization = async (args, context) => {

    try {

        const id = getUserId(context);
        const user = await context.db.user({id});

        ...........

        const organization = await context.db.createOrganization({
            ...args.data,
            owner: {
                connect: {
                    id
                }
            },
            users: {
                connect: [
                    {email: user.email}
                ]
            }
        });

        return {
             organization
        }

    } catch (e) {
        return new ApolloError(e.message);
    }

};

Query:
// @ts-ignore
  getViewerOrganization: async (parent, args, context) => {

    try {

      const id = await getUserId(context);
      const user = await context.db.user({id});
      const organization = await context.db.organization({id});

      return {
        organization
      }

    } catch (e) {
      return new ApolloError(e.message);
    }

  },

Connector:MySQL
Prisma Server: 1.30.1
prisma CLI: prisma/1.30.1
OS: Windows 10

What could be the issue?

Can you please add a console.log and see what the organization object contains?
Also, with the provided information, I am unable to reproduce this issue. If you are still getting this, can you please make a small repository with a minimal reproduction.

Sorry should not you replace
return {              organization         }
by return organizatoin

I was able to fix it by using
return {...organization}
or
return organization


Configurable memory settings for cassandra-loader.sh

Hi!
Noticed that the default cassandra-loader.sh script runs with 8G memory settings. Perhaps it is worth making this configurable? In some cases we're using cassandra loader on small virtual machines with few gb of memory.
As a workaround we're currently running cassandra-loader with java -jar and that way we can control the memory settings.
Thanks

Hi, I am also facing memory issue.
This is the Error Message, I was getting.
There is insufficient memory for the Java Runtime Environment to continue.
Native memory allocation (malloc) failed to allocate<PHONE_NUMBER> bytes for committing reserved memory.
Let me know how you ran. What options you gave, I dont see any options for memory.
Thanks,
Chaitanya

@chaitu1124 You can run cassandra-loader standalone with java -jar cassandra-loader -Xmx...

I have lowered the default RAM to 1GB, but as @ajermakovics pointed out, cassandra-loader is also a jar, so you can run with java -Xmx... -jar cassandra-loader.


[Enhancement] support connection pool for JDBC external table

What type of PR is this：

[ ] bug
[ ] feature
[x] enhancement
[x] refactor
[ ] others

Which issues of this PR fixes ：
for #9748
Problem Summary(Required) ：

refactor JDBC scanner to avoid local reference leak
add connection pool to reduce connection overhead

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] I have added user document for my new feature or new function

create a JDBC connection is a very heavy operator it will cost about 200ms-400ms for a query.
we should support cache connection pool DataSource

run starrocks_be_unittest

run starrocks_admit_test

run starrocks_admit_test

run starrocks_admit_test

run starrocks_admit_test

run starrocks_admit_test

run all

run starrocks_be_build

run starrocks_be_unittest

run starrocks_admit_test


eslint-index

I have created an ESLint utility module that I thought you might find useful.
The module is called eslint-index and you can read the full documentation on npm.
eslint-index provides a great deal of functionality including:

List all available rules declared by ESLint and any plugins you have included
Colour code rules depending on their status:

omitted (not declared anywhere in you ESLint config file)
0|off (declared, but set to 0|off)
1|warn
2|error

Display links to the rule documentation page next to each rule
Filter/reject rules by their status and/or their group

status is as described above (omitted|off|warn|error)
group is eslint for the core ESLint rules or the name of any of your plugins like react|import|flowtype

Format the output as a number or a table to get an overview of your rule settings
Rules that have been marked as deprecated are removed from all outputs
All of the above filters and formatting can be combined, for example:

filter omitted and off rules, output them as a list and display the rule doc links alongside
filter eslint rules and display the rule setting counters in a table

I wrote this plugin to aid the development of my own ESLint config settings and found it incredibly useful for keeping track of everything. I hope you find this module useful and please do let me know if you have any ideas on how to improve it.

Thanks, will give this a try when I start to prepare the standard v9 release in the next few months.

Thanks, just gave this a shot and added it to standard's devDependencies. Super useful!


Volume settings

Would be nice with a possibility to increase volume of announcements from Google aswell as a more native way to insert an alarm sound for Sonos. In Ui for us nooby HA people.

is a duplicate of so many requests .. :D
@nielsfaber is still not sure how to manage it. but i guess is super easy.
-1 #752
-2 #362
"Browser_mod" or "LANnauncer" to play from Tablets. Sonos etc. is already populated as media-player.

How do you set the volume on your Sonos device outside of alarmo (which service)?
What limitation did you find in Alarmo?

How do you set the volume on your Sonos device outside of alarmo (which service)?
What limitation did you find in Alarmo?

Hi Niels
So I tried to change the volume from the actions tap, by going into the yaml mode and add service: media_player.volume_set but it’s not saving the parameter.
The dream feature for me in alarmo would be the ability to change volume in the actions UI tab.
If it make sense?

I struggle with this too! since a lot of times the alerts played on my google home speakers are not loud enough, the tts function is practicly unusable!

Would still be so nice to have

I still hope this feature request will get some love :)

Completed? Woah I have to check that out..


About the credentials used in AWS RDS Data API PrivateLink Sample App

I am trying the AWS RDS Data API PrivateLink Sample App to verify Private Link.
I thought I followed the instructions in rds-data/README.md, but it didn't work.
Please check the authentication information used for "Create a user-provided service".
What values should I enter for "accessKeyId" and "secretAccessKey"?

I understand that it is issued by an AWS IAM user.


pointerdown fired at same time as pointerup in iOS 13 Safari

Both events are fired at the same time - when you release your finger. This is fixed in Safari 14.
It would be helpful to mention this in the Notes section of https://caniuse.com/pointer

I think this may just be browserstack


Can I change name of the struct R?

Is there any way to change the name of the autogenerated struct?
I have 3 subprojects in my workspace and I want to build seperate R files for each of them. See the photo below:

As far as I see the struct name is hardly coded, any suggestions?
By the way my idea is building seperate R files for each project like:

R(for Bingo)
UIR(for BingoUI)
SharedR(for BingoShared)

If you have 3 R.generated.swift files, each under its own module, you should then be able to access each as Bingo.R., BingoUI.R and BingoShared.R.
If that's too verbose, you could create a typealias for each:
public typealias SharedR = BingoShared.R
public typealias UIR = BingoUI.R

As mentioned, you can import and use the R structs from each library separately.
See: https://github.com/mac-cain13/R.swift/blob/master/Documentation/QandA.md#can-i-use-rswift-in-a-library


Unsupported Argument...""... Is Not Expected Here

Description
When creating an autoscaling group with a launch template, scaling policies, and a warm pool, a number of exceptions are presented for attributes in the module.

[ X ] ✋ I have searched the open/closed issues and my issue is not listed.

Versions

Module version [Required]:
v7.1.0
Terraform version:

Terraform v1.5.0
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.5.0

Provider version(s):
provider registry.terraform.io/hashicorp/aws v5.5.0
Terragrunt version:
terragrunt version v0.48.0

Reproduction Code [Required]
locals {
  external_deps = read_terragrunt_config(find_in_parent_folders("external-deps.hcl"))
  common_vars   = read_terragrunt_config(find_in_parent_folders("common.hcl"))
  account_vars  = read_terragrunt_config(find_in_parent_folders("account.hcl"))
  region_vars   = read_terragrunt_config(find_in_parent_folders("region.hcl"))
  product_vars  = read_terragrunt_config(find_in_parent_folders("product.hcl"))
  env_vars      = read_terragrunt_config(find_in_parent_folders("env.hcl"))
  product       = local.product_vars.locals.product_name
  prefix        = local.product_vars.locals.prefix
  account       = local.account_vars.locals.account_id
  env           = local.env_vars.locals.env

  # Used to get latest Amazon ECS Optimized AMI
  ami_id                  = trimspace(run_cmd("sh", "-c", "aws ssm get-parameters --names /aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id --region us-east-1 --query 'Parameters[0].Value' --output text"))
  latest_amazon_linux_ami = replace(local.ami_id, "\"", "")
  spot_enabled            = false

  tags = merge(
    local.env_vars.locals.tags,
    local.additional_tags
  )
  additional_tags = {
  }
}

include {
  path = find_in_parent_folders()
}

dependency "vpc" {
  config_path = "../../vpc"
}

dependency "instance_profile" {
  config_path = "../../iam/roles/instance-profile"
}

dependency "ecs_security_group" {
  config_path = "../../security-group/ecs-primary"
}

terraform {
  source = "git::git@github.com:terraform-aws-modules/terraform-aws-autoscaling.git//.?ref=v7.1.0"
}

## ASG Inputs
inputs = {
  create                    = true
  force_delete              = true
  force_delete_warm_pool    = true
  name                      = "${local.product}-ecs-${local.env}"
  use_name_prefix           = false
  desired_capacity          = 2
  min_size                  = 2
  max_size                  = 40
  wait_for_capacity_timeout = 0
  default_instance_warmup   = 90
  health_check_type         = "EC2"
  vpc_zone_identifier       = dependency.vpc.outputs.private_subnets
  initial_lifecycle_hooks   = []

  instance_refresh = {
    strategy = "Rolling"
    preferences = {
      checkpoint_delay             = 600
      checkpoint_percentages       = [35, 70, 100]
      instance_warmup              = 60
      min_healthy_percentage       = 50
      auto_rollback                = true
      scale_in_protected_instances = "Refresh"
      standby_instances            = "Terminate"
    }
    triggers = ["tag"]
  }
  enabled_metrics = [
    "GroupDesiredCapacity",
    "GroupInServiceCapacity",
    "GroupPendingCapacity",
    "GroupMinSize",
    "GroupMaxSize",
    "GroupInServiceInstances",
    "GroupPendingInstances",
    "GroupStandbyInstances",
    "GroupStandbyCapacity",
    "GroupTerminatingCapacity",
    "GroupTerminatingInstances",
    "GroupTotalCapacity",
    "GroupTotalInstances"
  ]

  warm_pool = {
    pool_state                  = "Stopped"
    min_size                    = 1
    max_group_prepared_capacity = 2

    instance_reuse_policy = {
      reuse_on_scale_in = true
    }
  }

  scaling_policies = {
    request-count-per-target = {
      policy_type               = "TargetTrackingScaling"
      estimated_instance_warmup = 120
      target_tracking_configuration = {
        predefined_metric_specification = {
          predefined_metric_type = "ALBRequestCountPerTarget"
        }
        target_value = 2500
      }
    }
  }

  ## Launch Template Inputs
  launch_template_id          = "${local.product}-ecs-${local.env}"
  launch_template_description = "Launch template for ${local.product} ${local.env}"
  iam_instance_profile_name   = dependency.instance_profile.outputs.iam_instance_profile_name
  update_default_version      = true
  image_id                    = local.latest_amazon_linux_ami
  instance_type               = "m7g.large"
  ebs_optimized               = true
  enable_monitoring           = true
  user_data_base64 = yamldecode(base64encode(templatefile("${get_terragrunt_dir()}/user-data.yaml", {
    spot_enabled = local.spot_enabled
    cluster_name = "${local.product}-${local.env}"
  })))
  block_device_mappings = [
    {
      device_name = "/dev/xvda"
      no_device   = 0
      ebs = {
        delete_on_termination = true
        encrypted             = true
        volume_size           = 50
        volume_type           = "gp3"
      }
    }
  ]

  capacity_reservation_specification = {
    capacity_reservation_preference = "open"
  }

  metadata_options = {
    http_endpoint               = "enabled"
    http_tokens                 = "required"
    http_put_response_hop_limit = 32
  }

  lifecycle = {
    create_before_destroy = true
  }

  tags_as_map = [
    local.tags
  ]
}

Steps to reproduce the behavior:

Are you using workspaces?
No
Have you cleared the local cache (see Notice section above)?
Yes
List steps in order that led up to the issue you encountered

Update release tag to v7.1.0
Run terragrunt plan

Expected behavior

A Terraform plan should happen without error.

Actual behavior

The terraform plan results in a number of unsupported argument errors that seem unrelated to attributes being used as inputs.

Terminal Output Screenshot(s)
ami-0cc97762b1dbb2347

Initializing the backend...

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.5.0"...
- Installing hashicorp/aws v5.5.0...
- Installed hashicorp/aws v5.5.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
Acquiring state lock. This may take a few moments...
Releasing state lock. This may take a few moments...
╷
│ Error: Unsupported argument
│
│   on main.tf line 389, in resource "aws_autoscaling_group" "this":
│  389:   ignore_failed_scaling_activities = var.ignore_failed_scaling_activities
│
│ An argument named "ignore_failed_scaling_activities" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│   on main.tf line 418, in resource "aws_autoscaling_group" "this":
│  418:           scale_in_protected_instances = try(preferences.value.scale_in_protected_instances, null)
│
│ An argument named "scale_in_protected_instances" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│   on main.tf line 419, in resource "aws_autoscaling_group" "this":
│  419:           standby_instances            = try(preferences.value.standby_instances, null)
│
│ An argument named "standby_instances" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│   on main.tf line 656, in resource "aws_autoscaling_group" "idc":
│  656:   ignore_failed_scaling_activities = var.ignore_failed_scaling_activities
│
│ An argument named "ignore_failed_scaling_activities" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│   on main.tf line 685, in resource "aws_autoscaling_group" "idc":
│  685:           scale_in_protected_instances = try(preferences.value.scale_in_protected_instances, null)
│
│ An argument named "scale_in_protected_instances" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│   on main.tf line 686, in resource "aws_autoscaling_group" "idc":
│  686:           standby_instances            = try(preferences.value.standby_instances, null)
│
│ An argument named "standby_instances" is not expected here.
╵
ERRO[0028] Terraform invocation failed in /Projects/redacted-nonprod/us-east-1/nonprod/asg/.terragrunt-cache/wygDz1B1-48OPpdgbFVbW3bjSak/7ii-Ht_IxPnCZFWq71fjiqG-f_c  prefix=[/Projects/redacted-nonprod/us-east-1/nonprod/asg]
ERRO[0028] 1 error occurred:
        * [/Projects/redacted-nonprod/us-east-1/nonprod/asg/.terragrunt-cache/wygDz1B1-48OPpdgbFVbW3bjSak/7ii-Ht_IxPnCZFWq71fjiqG-f_c] exit status 1

Additional context

Changing the release tag to v7.0.0 results in a successful plan.

I don't know how you are getting a provider version of 5.5.0, but the minimum supported version for 7.1.0 of the module is 5.16.2 https://github.com/terraform-aws-modules/terraform-aws-autoscaling/blob/00aadef7247ba1b83c512207091b0fa03087be61/versions.tf#L7

I don't know how you are getting a provider version of 5.5.0, but the minimum supported version for 7.1.0 of the module is 5.16.2
https://github.com/terraform-aws-modules/terraform-aws-autoscaling/blob/00aadef7247ba1b83c512207091b0fa03087be61/versions.tf#L7

Thank you for pointing this out. We are pinning the provider our root terragrunt.hcl but with a pessimistic constraint that is preventing anything beyond a patch version update. I will close this issue.
.
..
...
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      # Latest patch version
      version = "~> 5.5.0"
    }
  }
}
PROVIDER_VERSIONS
}

User generated issue.


docs(readme): add how to use Protobuf

Just add doc for for how to use Protobuf 😃

Thanks a lot

Hi @gklijs hmm, this one got closed and didn't get merge.


Class Section needs to implement Countable. PHP >=7.2 breaks FormatHTML

With php 7.2, any argument to count must be "Countable".
lib/RESTfm/Format/FormatHtml.php calls count on line 88 using the $section variable created on line 84. $section is a Section object defined in lib/RESTfm/Message/Section.php which is not implementing Countable resulting in a warning being thrown.
Adding the following to lib/RESTfm/Message/Section.php should implement Countable and fix the warning.
insert at line 22 use Countable;
Change class declaration to class Section implements SectionInterface, Countable {...}
Add count method to the class similar to the following:
/**
 * Returns an integer count of the number of rows.
 */
public function count() {
  if ($this->getDimensions() == 1 && count($this->getRows()) > 0) {
    return 1;
  }
  if ($this->getDimensions() > 1) {
    return count($this->getRows());
  }
  return 0;
}

James, Thanks for this.  Any chance you can submit a change request here with these changes?

What version of RESTfm are you using JamesSCV?

I created a PR against this issue. See PR #27

@gav- I'm using version 5.2.6 and PHP 7.4, but the PHP change that caused this issue is from PHP 7.2 https://www.php.net/manual/en/migration72.incompatible.php#migration72.incompatible.warn-on-non-countable-types


Phala Ambassador Growth Program.md

Consider moving this file to the expected location under content/en-us


partclone: add support for various filesystems

Adds support for ext2/3/4, fat, hfs+, minix, nilfs, ntfs, and xfs
Closes #39079
Motivation for this change
As it stands, the partclone package does not provide support for efficient cloning of filesystems; The partclone.[fs_type] symbolic links are missing. This means that cloning is limited to partclone.dd, a bit-identical copy.
In contrast, this change adds support for some of the supported filesystems, allowing partclone to perform efficient cloning of said filesystems.
Missing filesystems
The following filesystems are not in this change because I was unable to get the configure script to detect the dependencies.

reiserfs
ufs
vmsf
jsf

In addition, I did not add FUSE support.
Things done

[X] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[X] NixOS
~[ ] macOS~
[ ] other Linux distributions

~[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)~
[X] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
[X] Fits CONTRIBUTING.md.

@GrahamcOfBorg build partclone


Feature Request: Advice on modules that could/should be combined

Is your feature request related to a problem? Please describe.
Sometimes when you modularize a codebase the final structure is not clear and you might "over-modularize", meaning things that should be in a single module are separated into multiple modules
Describe the solution you'd like
Would like the plugin to give advice on which module are good candidates for combining into a single module.
a couple of heuristics on when you might want to combine modules:

when any given module depends on A it also depends on module B XX% (configurable) of the time, and vice versa.
module A and module B apply the same (or mostly the same) plugins ("mostly same" should be configurable)
module A and module B have the same (or mostly the same) dependencies ("mostly same" should be configurable)
module A and module B are less than NNN (configurable) lines of code when combined (don't want to create huge modules)

if some combination of those things are true, then "combineModuleAdvice" should propose combining module A and B
Additional context
I generally like to start with smallest modules possible, see what structure emerges, and then combine modules that are similar.

Thanks for the issue. This is an interesting idea, and is why the primary task of the plugin is buildHealth and not findBadDependencies or something. I'm not sure when I might have time to work on it, but I like it conceptually.


Abstract workflow state, add JsonFileStateProvider

Workflow towards an abstract workflow state provider. Currently we use Metamist for that in a limited way, only creating Analysis entries for stages that specify analysis_type in the decorator, i.e. @stage(analysis_type='qc'). This PR adds a MetamistStateProvider along with JsonFileStateProvider, with the latter creating JSON files instead of communicating with Metamist.
Future work would include:

On metamist side, supporting more flexible output objects in Analysis entries; probably allowing dictionaries of strings in addition to single strings, to match Stage.expected_outputs type;
Write state of each stage, not only those where analysis_type is specified. Use custom type by default.
Implement read_state for each provider, where workflow loads its state using the specified provider.

@illusional, how do you feel about this vs enabling a lightweight Metamist deployment with an easy way to set up the Metamist DB through e.g. a JSON file? That would keep the dependency from the pipeline side uniform, but would instead require more integration on Metamist.


Improve importing modules

Priorize builtin modules over files
Fix loading from game.egg
Fix loading resources from macOS app bundle
./ on custom modules is optional

Note: This change breaks existing .egg files made with NEST (which uses the ./ prefix on files).

Note: This change breaks existing .egg files made with NEST (which uses the ./ prefix on files).

My initial test, providing a folder:
frarees@MBP ivokdev % nest -z -o game.egg -- gamedata
Bundling: ./gamedata/tilemap.png
Bundling: ./gamedata/tilemap.wren
Bundling: ./gamedata/main.wren
Created bundle game.egg.
frarees@MBP ivokdev % tar tvf game.egg
-rw-rw-r--  0 0      0        1459 Jan  1  1970 /tilemap.png
-rw-rw-r--  0 0      0         269 Jan  1  1970 /tilemap.wren
-rw-rw-r--  0 0      0         326 Jan  1  1970 /main.wren

A second test, given your results, providing just file paths:
frarees@MBP ivokdev % nest -z -o game.egg -- gamedata/*.wren
Bundling: ./gamedata/main.wren
Bundling: ./gamedata/tilemap.wren
Created bundle game.egg.
frarees@MBP ivokdev % tar tvf game.egg
-rw-rw-r--  0 0      0         322 Jan  1  1970 ./gamedata/main.wren
-rw-rw-r--  0 0      0         281 Jan  1  1970 ./gamedata/tilemap.wren

So there's clearly differences. Since microtar seems to need the exact string (I'm not very familiar with the library, but seems like it can't handle path conversions or expansions), I suggest to:

Make sure nest produces consistent paths, be it root-based /, rel-path based ./ or non-prefixed. Non-prefixed makes the most sense to me. If we need further prefixing as an implementation detail, we could prepend it in code.
Decide if/make sure we are able to load game.egg files generated before the changes. Since it seems game.egg files can be created with ./ and / prefixes (as demonstrated before), if we want to maintain compatibility we would need to call microtar with both / and ./ variants. Personally, I think it makes sense to not allow backwards compatibility, since nest is still in development and I guess we can afford users re-generating egg files (maybe add functionality on nest to sanitize egg files?)

There are still cases not covered by the PR, such as loading from a specified directory. I'll be looking at that too

So the current state (as of this comment) is nice (I appreciate the relative simplicity), but it will break existing game code which uses imports prefixed with ./, since those will no longer be found in egg files, and the check for prefixes has been removed.

it will break existing game code

Indeed. I plan on working on backwards compatibility, now that I have something cleaned up and working. That is, if it makes sense and we agree to go forward with all these dome+nest path related changes.

Given how game.egg can have both ./ and / prefixes, we can:

Try both variants when asking for a file on microtar. Con: overhead for "legacy eggs".
Include a tiny manifest in .egg where we store (at least) the egg version, so that we know what we're dealing with.

I guess 2 makes most sense for a robust and ongoing tool. Should we go with that?

Alternatively, don't allow '/' prefixes at all. DOME uses '/' prefixing to assume absolute paths, so maybe it shouldn't be using the .egg for those at all.
I do like the idea of a small manifest for future considerations, but as a safety/security thing, DOME shouldn't be able to accidentally load it as a wren module.

Alternatively, don't allow '/' prefixes at all

nest currently prepends / when you build a folder e.g. nest -z -o game.egg -- folder (see nest's main.c:44), so if we're ok with not supporting legacy eggs being built with folders (for the lack of a better term), point 1. would be:

Fetch file and (fallback to) ./file via microtar. Con: overhead for "legacy eggs".

Oh problematic (I haven't looked much at nest's implementation since I wrote it like a year ago).
If we have a framework for supporting the legacy egg formats, then we can handle both, although it does complicate things a bit in an already complicated area of code.
I think I need a better view of who is using the EGG file format currently. If it's mostly just me, then we can probably assume it to be totally "unstable" and start again.

I've cleaned up the PR so that everything is contained within one commit. The core functionality is there. I've tested the changes with every entry on the jam (and some more small tests toying with mainstream nest and https://github.com/avivbeeri/nest/pull/2).
Changes have been tested on macOS Catalina and RG350 (Opendingux, linux-based), haven't tried Window out.
We need to update the docs to reflect the changes, check code conventions for consistency, and I guess we're good to go.
In case we don't need backwards compatibility with previous egg files (and assuming we're OK with https://github.com/avivbeeri/nest/pull/2) I could get rid of the extra compares.

Found a small issue. Importing a builtin module (e.g. import "random" for Random) does log a misleading error:
Loading module random
Reading from bundle: random.wren
Error: There was a problem reading random.wren from the bundle.

Although it works.

Yeah, I spotted that also. The built in modules do make this slightly
awkward.
On Wed, 8 Apr 2020, 20:06 Francisco Requena<EMAIL_ADDRESS>wrote:

Found a small issue. Importing a builtin module (e.g. import "random" for
Random) does log a misleading error:
Loading module random
Reading from bundle: random.wren
Error: There was a problem reading random.wren from the bundle.
Although it works.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/avivbeeri/dome/pull/62#issuecomment-611137081, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AALBBL5P6G4CMDDHO7GQ6ILRLTDLXANCNFSM4LW2YJFA
.

While I was working on this PR, I've noticed the startup time has gotten bigger. I'm not sure if it's related specifically to the changes on this PR (if so... how? any clue?). I tested your rhythm jam entry (https://avivbeeri.itch.io/rhythmdrive) and could see a noticeable difference between my built (release) dome and the one bundled.

Tested this on Windows, seems sufficient

This seems to work for Windows, Mac, Linux, in all the crazy cases I can think of, so we are probably good to go! Squash it and I can merge :)