Parsing slightly ambigious grammar with LALR

First of all thanks for creating LARK. I have used bison in past life and to me it always felt challenging but it was an absolute joy when I first used Lark.
I have a slightly ambiguous grammar which works fine with early parser but fails with error when using LALR. The grammar is
parser = Lark(r"""
    expr: SIGNED_NUMBER
        | SIMPLE_STRING
        | name
        | FUNCTION_NAME "(" function_args ")"

    function_args : expr
                  | function_args "," expr

    name: NAME

    NAME : /[a-zA-Z0-9_]+/
    SIMPLE_STRING : /'[^']*'/
    FUNCTION_NAME : /[a-zA-Z0-9_]+/

    %import common.SIGNED_NUMBER
    %import common.WS
    %ignore WS
""", start="expr", parser="lalr")

A token without any context can be either NAME or FUNCTION_NAME. While using the default parser, it resolves these ambiguities correctly
tree = parser.parse("A(B(x, y, 5))")
print(tree.pretty())

produces
expr
  A
  function_args
    expr
      B
      function_args
        function_args
          function_args
            expr
              name	x
          expr
            name	y
        expr	5

But with lalr it fails to parse with following error
UnexpectedCharacters: No terminal defined for ',' at line 1 col 6

A(B(x, y, 5))
     ^

Expecting: {'LPAR'}

Previous tokens: Token('FUNCTION_NAME', 'x')

I am sure its a very n00b question. So if its recommended to read something (books etc.) for me to understand this, I would love some suggestions. Thanks!

I'm glad you're enjoying lark!
Regarding your question, it's a very simple matter of terminal collision. You made NAME and FUNCTION_NAME match the exact same thing, and the lexer has to choose one over the other "arbitrarily".
Just drop FUNCTION_NAME, and everythign should work:
| NAME "(" function_args ")"

As for a tip for next time, try to keep your terminals to a minimum, make sure there no collisions between regexps (regexp-string and string-string aren't a problem, Lark can detect those).
We're actually working a feature that would print a warning in this exact situation, but it's not ready yet.

Thanks! It works. As suggested I changed the grammar to
parser = Lark(r"""
    expr: SIGNED_NUMBER
        | SIMPLE_STRING
        | name
        | function_name "(" function_args ")"

    function_args : expr
                  | function_args "," expr

    function_name: NAME
    name: NAME

    NAME : /[a-zA-Z0-9_]+/
    SIMPLE_STRING : /'[^']*'/

    %import common.SIGNED_NUMBER
    %import common.WS
    %ignore WS
""", start="expr", parser="lalr")
tree = parser.parse("A(B(x, y, 5))")
print(tree.pretty())

and got the right parse tree
expr
  function_name	A
  function_args
    expr
      function_name	B
      function_args
        function_args
          function_args
            expr
              name	x
          expr
            name	y
        expr	5

However I tried a small variation in the input
tree = parser.parse("A(B(7x, y, 5))")

and I got the error
UnexpectedCharacters: No terminal defined for 'x' at line 1 col 6

A(B(7x, y, 5))
     ^

Expecting: {'COMMA', 'RPAR'}

Previous tokens: Token('SIGNED_NUMBER', '7')

I am curious why does 7x not match the NAME. I agree that 7 matches both NAME and SIGNED_NUMBER but how is the decision made? What determines the priority? Also it parses fine with early parser
expr
  function_name	A
  function_args
    expr
      function_name	B
      function_args
        function_args
          function_args
            expr
              name	7x
          expr
            name	y
        expr	5

Thanks.

I am curious why does 7x not match the NAME. I agree that 7 matches both NAME and SIGNED_NUMBER but how is the decision made? What determines the priority?

The longest matching terminal is used. SIGNED_NUMBER matches 7, but NAME matches 7x so 7x is matched.

@MegaIng But when using LALR it didn't match 7x as NAME. Instead it matched 7 as a number and then failed. See error above. Thanks!

@ashish01 In LALR mode, Lark's lexer stops when it finds its first match. Conceptually, it iterates over the entire set of terminals, and stops once it finds a match. Therefor the order of the terminals matter. Lark uses a bunch of rules to try to minimize collisions, but it won't always work. If you have two colliding regexps on purpose, it's better to give them an explcit priority, rather than rely on Lark.

Excellent! Thanks, adding a explicit priority solves the issue. For reference, here is how I changed the grammar
parser = Lark(r"""
    expr: SIGNED_NUMBER_R
        | SIMPLE_STRING
        | name
        | function_name "(" function_args ")"

    function_args : expr
                  | function_args "," expr

    function_name: NAME
    name: NAME

    NAME.2 : /[a-zA-Z0-9_]+/
    SIMPLE_STRING : /'[^']*'/
    SIGNED_NUMBER_R.1: SIGNED_NUMBER

    %import common.SIGNED_NUMBER
    %import common.WS
    %ignore WS
""", start="expr", parser="lalr")
tree = parser.parse("A(B(x, 7y, 5))")
print(tree.pretty())

which produces
expr
  function_name	A
  function_args
    expr
      function_name	B
      function_args
        function_args
          function_args
            expr
              name	x
          expr
            name	7y
        expr
          name	5


WIP: Refactor package

Still WIP.
Work is in this branch:
The goals are:

[x] Move codegen core to a package and don't use fs
[x] Move generators to separate package
[x] Refactor directories structure and use Lerna to manage package's dependencies
[x] Refactor GraphQL Schema template context builder and object
[x] Handle fields and types in a better way  (related to https://github.com/dotansimha/graphql-code-generator/issues/96 and https://github.com/dotansimha/graphql-code-generator/issues/84) - my idea is to use expose the schema template context the same way GraphQL treat it's types (separate arrays for enum, type, input, union, interface, scalar)
[x] Treat Query and Mutation just like any other type
[x] Fix CI
[x] Refactor operations builder with flatten feature
[x] Reimplement typescript-single-file
[x] Reimplement imports builder for multiple files
[x] Reimplement typescript-multi-file
[x] Move and refactor CLI util to a separate package (use fs only here)
[ ] Add support for input/output of a whole project (to generate apps)
[ ] Add support for external template config file
[ ] Update README

After merging to master:

[ ] Reimplement flow-single-file
[ ] Reimplement swift-single-file
[ ]  100% code coverage for core package
[ ] Blog post

Ideas:

[x] Consider implementing the flatten types map in a better way - maybe just add a reference inside the template context to the actual type along the type name.

@DxCx  @zephraph any thoughts?

Whew... this is a huge change. Okay, I'll comment as I can go through it...

@zephraph my goal is to simplify the schema handler first.
I am almost done, and you can find it here: https://github.com/dotansimha/graphql-code-generator/blob/refactor/all/packages/graphql-codegen-core/src/schema/schema-to-template-context.ts
Also, I changed the schema template context object to be: https://github.com/dotansimha/graphql-code-generator/blob/refactor/all/packages/graphql-codegen-core/src/types.ts#L55-L62
This is much simpler implementation for the Schema handling, the difference is that I'm creating an array for each GraphQL base type (scalar/enum/type/input...), instead of a single array for all of them together.
I also separated it to 3 packages:

core (GraphQLSchema and operations string => template context object)
generators (combines template and template context, also in charge of converting primitives to language primitives, eg. String => string and Int to number in case of TS)
cli (Find, read, introspect and execute generator + core)

It might be a good idea to generate a copy of dev-test from master (on all supported variations) and have a test just for comparing final generated outputs.

@zephraph I'm done with the scheme generation for typescript, and it looks like the result file is the same as the current implementation - so we are good so far :)

Cool! I'll run through the updates. There's a few things I want to test.

@zephraph do you have good use cases for complicated operations (fragment/mutation/query)? I want to use it to write tests before implementing.

I don't at the moment but we can put one together.

It's really starting to come together!

@zephraph almost there!! my next goal is to add the ability to generate a whole project with custom templates, something like this:
my-project/
    - index.js
    - router.js
    - graphql/
             - feed.graphql
    - wrappers/
              - operation.js.template
    - data-tables/
              - table.js.type.template

and you have a type called MyType on your GraphQLSchema.
The output will be:
    - index.js
    - router.js
    - graphql/
             - feed.graphql
    - wrappers/
              - operation.js.template
              - feed.query.js
    - data-tables/
              - table.js.type.template
              - mytype.table.js

This way, users can create a project and run the generator over a project's directory and generate anything they need that related to GraphQL (generate UI, HOC wrappers for React, ready-to-use gql wrappers or anything else :) )
What do you think?


修复rtmp不能监听ipv6的问题，增加了ipv6端口验证代码

管理者你好，我最近修复了rtmp监听ipv6端口的一个小问题，请查阅

问题来源：Issues 配置IPV6启动失败
问题：原来的问题主要是ipv4端口和ipv6端口的监听方式略微不同，ipv4只需要写入端口号就可以监听，而ipv6至少需要写[::]:port才能监听，因此验证的时候会产生错误
修改内容：增加了ipv6的验证代码，目前只能支持[::]:port这样的格式
使用及效果：修改设置，启动srs，经过测试可以正确监听ipv6的端口，且推流和拉流正常

Codecov Report

Merging #2157 (5d86143) into 3.0release (4f01340) will decrease coverage by 0.00%.
The diff coverage is 58.33%.

@@              Coverage Diff               @@
##           3.0release    #2157      +/-   ##
==============================================
- Coverage       52.91%   52.91%   -0.01%
==============================================
  Files              82       82
  Lines           26693    26703      +10
==============================================
+ Hits            14125    14130       +5
- Misses          12568    12573       +5

Impacted Files
Coverage Δ

trunk/src/app/srs_app_config.cpp
81.07% <58.33%> (-0.12%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4f01340...5d86143. Read the comment docs.

Codecov Report

Merging #2157 (5d86143) into 3.0release (4f01340) will decrease coverage by 0.00%.
The diff coverage is 58.33%.

@@              Coverage Diff               @@
##           3.0release    #2157      +/-   ##
==============================================
- Coverage       52.91%   52.91%   -0.01%
==============================================
  Files              82       82
  Lines           26693    26703      +10
==============================================
+ Hits            14125    14130       +5
- Misses          12568    12573       +5

Impacted Files
Coverage Δ

trunk/src/app/srs_app_config.cpp
81.07% <58.33%> (-0.12%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4f01340...5d86143. Read the comment docs.

#2581，已修复提到的问题i


⚠ Spryr has degraded performance

In 18974e9, Spryr (https://spryr.com) experienced degraded performance:

HTTP code: 200
Response time: 3670 ms

Resolved: Spryr performance has improved in e5b9e76 after 7 minutes.


error: metadata-generation-failed

from this issues:https://github.com/microsoft/DeepSpeed/issues/4658
According to your tips, I have set DS_BUILD_EVOFORMER_ATTN=0 and DS_BUILD_AIO=0, but now it have a new problem, please help ，thanks！
【my computer environment：windows10、GPU A4000、pytorch 2.1.1、VS2022】
the error：
Collecting deepspeed
Using cached deepspeed-0.12.4.tar.gz (1.2 MB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [35 lines of output]
test.c
LINK : fatal error LNK1181: 无法打开输入文件“aio.lib”
系统找不到指定的文件。
系统找不到指定的文件。
系统找不到指定的文件。
系统找不到指定的文件。
系统找不到指定的文件。
系统找不到指定的文件。
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "C:\Users\issuser\AppData\Local\Temp\pip-install-yjaclsor\deepspeed_6bc529f3cc1f4e6b8905635d592b92a3\setup.py", line 192, in
ext_modules.append(builder.builder())
File "C:\Users\issuser\AppData\Local\Temp\pip-install-yjaclsor\deepspeed_6bc529f3cc1f4e6b8905635d592b92a3\op_builder\builder.py", line 637, in builder
extra_link_args=self.strip_empty_entries(self.extra_ldflags()))
File "C:\Users\issuser\AppData\Local\Temp\pip-install-yjaclsor\deepspeed_6bc529f3cc1f4e6b8905635d592b92a3\op_builder\inference_cutlass_builder.py", line 71, in extra_ldflags
import dskernels
ModuleNotFoundError: No module named 'dskernels'
DS_BUILD_OPS=1
[WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
[WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING]  cpu_adam requires the 'lscpu' command, but it does not exist!
[WARNING]  cpu_adam attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution.
[WARNING]  cpu_adam requires the 'lscpu' command, but it does not exist!
[WARNING]  cpu_adam attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution.
[WARNING]  cpu_adagrad requires the 'lscpu' command, but it does not exist!
[WARNING]  cpu_adagrad attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution.
[WARNING]  cpu_adagrad requires the 'lscpu' command, but it does not exist!
[WARNING]  cpu_adagrad attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution.
[WARNING]  cpu_lion requires the 'lscpu' command, but it does not exist!
[WARNING]  cpu_lion attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution.
[WARNING]  cpu_lion requires the 'lscpu' command, but it does not exist!
[WARNING]  cpu_lion attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution.
[WARNING]  Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING]  Filtered compute capabilities ['6.0', '6.1', '7.0']
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Hi @gitshaozhong - looks like you are building on Windows - using AIO there isn't supported, but depending on your system and where things are installed, it can be harder to debug on Windows.  This has some steps you can try to resolve the AIO error.  Also if you're able to, WSL will allow you to use all the features of DeepSpeed and doesn't hit this AIO issue.


Change low_memory default in docstrings

Fix for https://github.com/lmcinnes/umap/issues/640. I'm not actually sure if changing the default of low_memory from False to True was an intended change but I've updated the docstrings to match the default. Here is the commit where the default was changed: https://github.com/lmcinnes/umap/commit/9c9a0945456e5ce849e0e7badd72bf3469842eb5

Coverage remained the same at 88.956% when pulling 9c4d8e020211d7520ade143ae97befdc00e72db2 on gclen:issue-640 into ae5255be571c4a90bd93611a07612b4565c7984d on lmcinnes:master.


Use HTTPRoute hostname as SNI match for HTTPS traffic

Description:
The current Gateway API Translator logic maps the the Gateway hostname to the Xds IR Listener Hostname and uses the Hostnames within the HTTPRoute spec to populate the Xds IR HeaderMatches
field.
The logic can be optimized to use the intersection between the gateway hostname and http route hostnames, computeHosts to populate the Xds IR Hostnames field
Relates to https://github.com/envoyproxy/gateway/pull/662#issuecomment-1297443066

I can try to fix this :)
/assign

@shivanshu1333 thanks for your willingness to take on this issue. Are you still planning to fix this issue for the v0.3.0-rc.1 milestone?

Hello, I have seen this issues take a long time and not be fixed, I want to contribute to it. Can I get assigned?

hey @oowl , thinking out loud, we have a simple mapping of 1:1 today

Listener Hostname maps to Server name match within Filter Chain with a xds Listener
Route Hostname maps to a :authority header match with the VirtualHost with xds Route Configuration object.

Im unsure if this will actually be a perf improvement or not because we use delta xds to push these resources, which will cause EG to update two resources (Listener and RouteConfiguration) instead of just 1, in the data plane, when the hostname in a  HTTPRoute is updated, if we go about making this change.
probably needs more analysis before making the change.

closed as not needed - we use the listener Hostname for SNI matching and route Hostname in virtual host matching


replace 'which' with 'command -v'

For my education, why is command -v preferable to which? I noticed that command -v is a built-in in bash, while which is not. (both are built-in commands in zsh).

For my education, why is command -v preferable to which? I noticed that command is a built-in in bash, while which is not. (both are built-in commands in zsh).

command is specified by POSIX, but which is not I think. This eliminates the shellcheck CI error.


[PR][CLOSED] [Documentation] Fixed typos in README.md

 Issue by geunsik-lim
Wednesday Jul 04, 2018 at 02:47 GMT
Originally opened as https://github.sec.samsung.net/STAR/TAOS-CI/pull/91

It's trivial. This PR is to fix incorrect statements in README.md
Signed-off-by: Geunsik Lim<EMAIL_ADDRESS>
geunsik-lim included the following code: https://github.sec.samsung.net/STAR/TAOS-CI/pull/91/commits

 Comment by git-bot-sec
Wednesday Jul 04, 2018 at 02:47 GMT

Thank you for submitting PR #91. Your PR has to pass all verificiation processes of cibot before getting a review process from reviewers. We recommend that you read instruction manual in . If you want to monitor the progress status of your PR in detail, visit http://aaci.mooo.com/.

:octocat: cibot: Thank you for posting issue #115. The person in charge will reply soon.


Support for virtual-dom's Widget feature

Described here: https://github.com/Matt-Esch/virtual-dom/blob/master/docs/widget.md
Widgets provide a way to hook existing JavaScript UI elements that are not virtual-dom aware into a virtual-dom based application. A Widget forms an "island" of sorts, that virtual-dom doesn't touch directly, instead it delegates the DOM creation, manipulation, and deletion to the widget itself, however the state that gets passed to the widget is still managed by virtual-dom, and you can hook the widget into the virtual DOM tree just like any other node.
Specifically, I would like to integrate Leaflet into a Halogen app, because I really really don't feel like reinventing that particular wheel myself, and Leaflet pretty much Just Works.
Is there anything in Halogen (or a third-party addon library) that would allow me to do this? If not, would it be possible to add?

For something like leaflet, it's easy enough to wrap as a component using refs and the normal component lifecycle events. Halogen-vdom supports a notion of Widgets, which we use to implement components, but it's not extensible from Halogen proper, though I don't think it's necessary for your use case.

Take a look at the Ace example (which is a component that wraps the Ace editor) https://github.com/slamdata/purescript-halogen/blob/master/examples/ace/src/AceComponent.purs

Thanks, that looks close enough. I'll see how far I get with this.

It should be noted, that example component does not need the Parent stuff, as it has no child components.

Makes sense.


Cannot save notebooks "Filename is not supported by Jupyter"

Issue Type: Bug

ctrl + p | Jupyter: Create new blank notebook
ctrl + s | Save dialog appears - file type is default selected as "Plain Text"
Change to "Jupyter". Save file as "test"
The following error appears

"Failed to save 'Untitled-1" File name "fileL///.../.../..//test.ipynb" is not supported by Jupyter. Please make sure the file name matches following patterns: "
(no patterns are provided).
This seems to only happen with the built in Notebooks editor. If I open an existing *.ipynb file, and (for some strange reason) manually load the deprecated Jupyter editor, then I can run/edit/save as usual.
Extension version: 2021.8.1236758218
VS Code version: Code 1.59.0 (379476f0e13988d90fab105c5c19e7abc8b1dea8, 2021-08-04T23:13:12.822Z)
OS version: Windows_NT x64 10.0.19042
Restricted Mode: No
Fetching remote diagnostics for 'SSH' failed: The process tried to write to a nonexistent pipe
Remote OS version: Linux x64 3.10.0-1062.9.1.el7.x86_64

System Info

Item
Value

CPUs
11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz (8 x 2995)

GPU Status
2d_canvas: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onrasterization: enabledskia_renderer: enabled_onvideo_decode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
31.71GB (3.83GB free)

Process Argv
--crash-reporter-id e312a6c6-7f7a-4620-8c0d-0ea31ee8a5b1

Screen Reader
no

VM
0%

Fetching remote diagnostics for 'SSH' failed: The process tried to write to a nonexistent pipe

Item
Value

Remote
SSH:

OS
Linux x64 3.10.0-1062.9.1.el7.x86_64

CPUs
Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz (72 x 2300)

Memory (System)
503.62GB (376.94GB free)

VM
0%

A/B Experiments
vsliv368:30146709
vsreu685:30147344
python383:30185418
pythonvspyt602:30300191
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstes627:30244334
pythonvspyt639:30300192
pythontb:30283811
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
pythondataviewer:30285071
pythonvsuse255:30340121
vscod805:30301674
pythonvspyt200:30340761
vscextlang:30333561
binariesv615:30325510
vsccppwt:30329788
pythonvssor306:30344512
bridge0708:30335490
vstre464:30350172
bridge0723:30353136
vsdyn420:30354109

I see you're on VS Code 1.59.0. There was a fix for a related issue in the 1.59.1 release--would you mind upgrading and letting us know whether the issue persists please?


GitHub: Invite member Matthias Fischer

Who should be invited to Catena-X ng
GitHub User:
https://github.com/dsmf
Vouching person:
ds-mkanal

HI @mkanal, just a reminder, @dsmf did not yet accept the invite. As far as i remember, invitations expire after 1 week

Closing the issue, since invitation is accepted.


Update Relational Schema for Organoid and Cell models

Relational model needs to be update to store the organoid and cell model data templates
This will be required to load Cell Model Passport data.
ACCEPTANCE CRITERIA

[x] schema is updated

@CsabaHalmagyi Done


Add state to cluster etcd backups page

UI should be updated to the new changes tracked in https://github.com/rancher/rancher/issues/18133

QA:
Set up a new cluster with default etcd backup options.
Performing a backup.
Navigate to cluster level backups page
Verify the state badge is now present on the page.

Version: Master 2.2 (2/20/19)
Confirmed issue.
The state badge now exists on the cluster etcd backups page.
Additional smoke testing:
Opened two windows. One to do backup from (clusters), one to watch backup page update with new entry.
Confirmed the window updates with new entry (websocket).
Restored from recent backup; ensure this is working as expected. Backups page updates indicating Cluster is currently Updating (websocket). Encountered no issues with any testing.


A small request to the tool creator

hey guys,
hiii
can you guys make a screencast/video series for the detailed usage of this tool..?
i know there is detailed documentation but still if we get a video series that would be really great..
sorry for my poor english..
cheers...

Hello,
Thank you for your interests in OpenFPGA. I believe that video tutorials can bring a great value to OpenFPGA and beginner-level users.  We are planning to create a few in near future. On the other sides, we have a few tutorials in the documentation, for instance:

A quick start on OpenFPGA design flow

A quick start on Customizing Architecture.
After trying these tutorials, I believe you have learned the fundamental knowledges about how to use the tools.
If you have any questions, feel free to ask.

Close the issue as answered. If you need further help, please comment (reopen this issue) or create a new issue.
Thank you for the interests in OpenFPGA.


drivers: can: Z_SYCALL_HANDLER should return int

can_handlers.c:65:1: warning: control reaches end of non-void function
[-Wreturn-type]
Signed-off-by: Flavio Ceolin<EMAIL_ADDRESS>

Codecov Report

Merging #13969 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #13969   +/-   ##
=======================================
  Coverage   52.25%   52.25%
=======================================
  Files         307      307
  Lines       45437    45437
  Branches    10508    10508
=======================================
  Hits        23741    23741
  Misses      16904    16904
  Partials     4792     4792

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fc3ca95...18df1b4. Read the comment docs.


p384_mlkem1024 hybrid added

We would like to use a NIST P curve + ML-KEM hybrid and compliance reasons would mandate using P-384 and ML-KEM-1024.
We found that P-384 is the most accepted P-curve. See CNSA 1.0.
From the ML-KEM variants, level 5 will be the most widely accepted. See CNSA 2.0.
Having a P-384 + ML-KEM-1024 hybrid could be useful.

We would also like a stable oid for this hybrid and we suggest our internal, under Tresorit. The oid is <IP_ADDRESS>.4.1.42235.6.
See Tresorit OID.

Hi @baentsch, as far as we know there is no draft for this combination. The reason I included our oid in the pull request is to provide a fixed oid, just to make sure it doesn't constantly change in oqs-provider. We intended for this oid to be permanent, until (and if) it is standardized. Of course any other oid would suffice.

We intended for this oid to be permanent, until (and if) it is standardized. Of course any other oid would suffice.

OK -- understood & makes sense. Please feel free to amend the PR by comment to state this (at the top of "generate.yml" or next to the OID). Otherwise, git blame will allow us to trace this in the future. A re-base to "main" would be nice to get CI to turn green, though.

Thanks for the contribution. Merged.


Renew TEST certificate (mid April)

Describe the task
Our TEST certificate will expire on June 1st, 2024. Ticket for renew we done last year for PROD https://github.com/bcgov/nr-fom/issues/421. Since the Certbot is not working anymore, we might not get notification, created this ticket to remind us. We need to send service desk a ticket to renew the certificate, and write Derek a ticket to help us install the new one.
Acceptance Criteria

[ ] create a service desk tickt
[ ] pass the certificate to Derek
[ ] third

Additional context

Add any other context about the task here.
Or here

oh this is a FOM ticket, close this one, will create one in FOM repo


Catch problems early (Shift quality left) pattern

Our engineering patterns are example solutions for addressing problems, meeting standards and living our engineering principles. They should make it easier for engineers to do their jobs. We don't have a standard way of writing patterns as we like to be a bit more flexible with their structure. Please provide us with some information on what you are proposing below
What is the pattern you are suggesting?
Catch problems early (shift quality left)
What would be the benefit to the Home Office of following this pattern
Why this is important
It's important to catch problems early for the following reasons:
Any problems can be addressed more quickly.
Tighter feedback loop. Allows problems with changes to be fixed while it is still fresh in the developer's head.
Reduces development and test effort to deliver a change, as issues can be caught before a formal test phase.
Problems can be caught before they can impact on your users, which will increase your users' confidence in your service.
Reduces the risk of adverse changes entering the product.
Overall; this allows for faster iteration, whilst maintaining the quality of your service.
How do people implement this pattern
You should:
Enable developers to test their changes in a local development environment
Developers should be able to spin up a copy of the environment on their local machine
Define staging environments, the testing to be done in those environments, and the progression criteria
Have staging environments that match the production environment as closely as possible
Test changes in a CI pipeline before they reach a staging environment (for example, run unit & functional tests)
Test changes in staging environments before they reach production (for example, run system-wide and integration tests)
Additional information
Add any other information you think would be useful here

[x] I have looked through our existing patterns and think this is not covered elsewhere
[x] I have checked through the open issues on the repository and this pattern has not already been suggested

Considerations:

wary of manual tasks going to the left as this can slow down delivery
shoud also consider moving other activities to the left, for example secure by design
moving qc to the left should not be considered to remove the need for observability in production - use shift right activities to catch problems quickly there too

Can also reference shifting security checks and tests left as per this article from Snyk https://snyk.io/learn/shift-left-security/


Minor fix to deal with multiple imports of the same Sass file

There appears to be a bug when a files is @import'ed multiple times.
Don't know why you would want to import a file multiple times, but I ran
into when attempting to compile Bootstrap Sass 3.3.2.

Awesome! 👍
Thanks for your contribution.


Only the first parametric tube cross-section gets generated for the connection file

When using "Create Connection from IOM" component then only the first parametric tube section appears in the connection file. All sections appear correctly through Checkbot (also in RFEM/SOFiSTiK). When I manually change the cross-sections in Excel to be manufactured and to specific profiles then everything works correctly again. So it must be something to do with generating multiple parametric tubes in succession for the Connection file.
SAF file and Grasshopper script is attached.
Karamba's tube section logic has changed. They used to appear as box type but now they appear as "tube" type:

All cross-sectionss appear in Deconstruct Cross Section:

Only the first cross-section appears in Connection:

KarambaTB_Profiletest.xlsx
Connection Generation from SAF.zip

Hi @tbyrk, please check the latest avaliable version 0.1.5 it should now work. There is something with our SAF conversion I need to double check. Yes, I asked Karamba to export tubes where possible so that we can easily get the geometry in Rhino. :)

Works correctly in 0.1.5. Thanks!


warning C4800 emitted by VS2019 when using sol::readonly inside new_usertype call

After upgrading from VS2017 to latest VS2019, the compiler now emits warning C4800 with the latest sol code:
struct Foo
{
	static void Register(sol::table &table)
	{
		table.new_usertype<Foo>("Foo", "id", sol::readonly(&Foo::foo));
	}
	int foo;
};

Compiling as Debug 64-bit. Result:
1>sol2\single\include\sol\sol.hpp(10713,1): warning C4800: Implicit conversion from 'T' to bool. Possible information loss
1>        with
1>        [
1>            T=int Foo::*
1>        ]

...with a very long cascading tail, of course.

That's a bit of a concerning warning, actually...!
Lemme dig into it.

I can't reproduce this error on latest VS2019 or VS2019 Preview anymore with /W4 and all that on. I think this one might've been fixed...!
... Somewhere. Somehow?

Pasted the same code into a new project and for some reason it doesn't trigger the warning there, although the compiler command lines are pretty much the same (modulo stuff that shouldn't matter). Still investigating.

Okay, that was simple: warning C4800 is not standard with /W4. Just add the following to the snippet and it will warn:
#pragma warning(default:4800)

@ThePhD could you reopen this issue based on my previous comment? Apparently I cannot do so.

This should totes be fixed I promise no really I swear pinky promise cross my heart hope to die stick a needle in my eye.

Fix confirmed, thanks!


Name convention issues in the ballot

Attribute in the custom datatype uses the wrong name convention. For example:

https://build.fhir.org/ig/hl7-eu/laboratory/branches/master/StructureDefinition-Address-eu.html: "Streetname" -> should be "streetName"
https://build.fhir.org/ig/hl7-eu/laboratory/branches/master/StructureDefinition-Ratio-eu-lab.html: "Uncertainty" -> should be "uncertainty".

Reported in Jira as FHIR-42992 Name convention issues in the ballot

Let's switch to Jira


fix(region): avoid sync azure china redis failed

What this PR does / why we need it

[x] Smoke testing completed
[ ] Unit test written

If yes, write the release branches name in the below format and submit the related cherry-pick PR:

release/3.7
release/3.6

/area region
/cc @zexi

Need cherry pick to release/3.6 and release/3.7 branch. @ioito


non-chief worker stuck in distributed SYNC mode for graph with two optimizers

Seems distributed tensorflow cannot train graph with two optimizers in sync mode (one for local update, the other for ps update)
There are three parts in my graph:

each worker has its own copy of vars as the ps server, but are defined with local variable collections=[tf.GraphKeys.LOCAL_VARIABLES], each worker has its own forward-backward loop based on the local vars and its own optimizer
(local variable - ps variable) as the gradient and apply to ps variable with SyncReplicasOptimizer.apply_gradients
broadcast the ps variable to the local variable

The three parts are run in this way: run subgraph 1 several times, then run subgraph 2 in distributed sync mode to update ps params and then run subgraph 3
Source code / logs
    if args.job_name == 'ps':
        server.join()
    elif args.job_name == 'worker':
        is_chief = (args.task_index == 0)
        num_gpus = len(worker_spec)

        ps_device = '/job:ps/cpu:0'
        worker_device = '/job:worker/task:%d/gpu:0' % args.task_index
        with tf.device(
                tf.train.replica_device_setter(cluster=cluster, ps_device=ps_device, worker_device=worker_device)):
            global_step = tf.Variable(args.start_step, name='global_step', trainable=False)

            print 'building ps params'
            ps_tparams = init_tparams()

            print 'building local params'
            with tf.device(worker_device):
                worker_tparams = init_tparams(is_local=True)  # define variable in collection tf.GraphKeys.LOCAL_VARIABLES

            print 'building graph'

            print '-- local update'
            x, x_mask, y, y_mask, cost = build_graph(worker_tparams, config)
            opt = tf.train.MomentumOptimizer(config.lr, config.mr)
            updates = worker_tparams
            grads = tf.gradients(cost, updates, colocate_gradients_with_ops=True)
            clipped_grads, _ = tf.clip_by_global_norm(grads, config.clip_grads)
            train_op = opt.apply_gradients(zip(clipped_grads, updates))

            print '-- reduce average'
            ps_updates = ps_tparams
            avg_grads = [tf.sub(var, ps_var) for var, ps_var in zip(updates, ps_updates)]
            bopt = tf.train.MomentumOptimizer(config.blr, config.bmr, use_nesterov=True)
            bopt = tf.train.SyncReplicasOptimizerV2(bopt, replicas_to_aggregate=num_gpus, total_num_replicas=num_gpus)
            update_op = bopt.apply_gradients(zip(avg_grads, ps_updates), global_step=global_step)

            print '-- broadcast'
            broadcast_ops = []
            for kk, pp in ps_tparams.items():
                broadcast_ops.append(worker_tparams[kk].assign(pp).op)

            # Others related to sync mode
            chief_queue_runner = bopt.get_chief_queue_runner()
            sync_init_op = bopt.get_init_tokens_op()

            sv = tf.train.Supervisor(
                is_chief=is_chief,
                logdir=config.ckp,
                init_op=tf.global_variables_initializer(),
                local_init_op=tf.local_variables_initializer(),
                global_step=global_step)

            if is_chief:
                print('Worker %d: Initializing session' % args.task_index)
            else:
                print('Worker %d: Waiting for session to be initialized' % args.task_index)

            sess_config = tf.ConfigProto(allow_soft_placement=True, log_device_placement=False,
                                         device_filters=['/job:ps', '/job:worker/task:%d' % args.task_index])
            with sv.managed_session(server.target, config=sess_config) as sess:
                print('Worker %d: Session initialization completed' % args.task_index)

                if is_chief:
                    # Chief worker will start the chief queue runner and call the init op
                    sess.run(sync_init_op)
                    sv.start_queue_runners(sess, [chief_queue_runner])

The non-chief works stuck at sv.managed_session and showing below message again and again:
I tensorflow/core/distributed_runtime/master_session.cc:993] Start master session a4de1cec7011a62d with config:
The code can run successfully when there is no local optimizer.
System information

Linux Ubuntu 14.04
CUDA 8.0
tf 0.12.0-rc1
GPU: GeForce GTX 1080

@mrry can you comment? Is this a TF bug?

I don't think it's a bug, although without a self-contained example I can't say for sure.
The most likely cause of sv.managed_session() hanging is because sv.prepare_or_wait_for_session()  is blocking inside SessionManager.wait_for_session(). The non-chief workers will block in here until the chief initializes all of the variables (and hence the ready_op returns True). It's possible that, because you have a slightly unusual variable configuration, the ready_op never returns True. You might need to customize the ready_op (or possibly the init_op) when constructing the tf.train.Supervisor to work for your configuration.

Thanks @mrry for your reply. What do you mean by "slightly unusual variable configuration"? Is there any reference of the ready_op (or init_op) you mentioned?

The "slightly unusual" part is this line:
            print 'building local params'
            with tf.device(worker_device):
                worker_tparams = init_tparams(is_local=True)  # define variable in collection tf.GraphKeys.LOCAL_VARIABLES

...which should work, but it might conflict with some of the assumptions in the tf.train.SyncReplicasOptimizer code.
I'd expect this log statement to provide useful information in the non-chief replicas. Does it print anything?

No, there is no "Waiting for model to be ready." info printed. Just "Start master session *** with config:" hang there over and over again.

@hellolovetiger I just ran into a similar as you - repeating "Start master session xxx with config:" messages for all non chief workers. I'm curious if you ever found a solution.
Btw I suspect the reason why your training loss increases after the change you posted above is that apply_gradients() flips the sign of the gradient because it assumes you will be minimizing a loss function. This tripped me up once.

@adamklec , the issue is gone after I replaced the optimizer for local update with my own implementation. This may not apply to your situation. You may want to read the details above.
Thanks for you kind reminder of the "loss increases" issue. Yes, you are right. I got the wrong direction when incorporating the gradients from the workers.

@hellolovetiger Actually I did just that last night! I replaced SyncReplicasOptimizer with my own gradient accumulator and the problem went away. Cheers!

Cool!

It has been 14 days with no activity and this issue has an assignee.Please update the label and/or status accordingly.

Nagging Assignee: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @mrry: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @mrry: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

I'm going to close this issue, because it seems like a workaround was found, and the eventual plan is to drop support for tf.train.SyncReplicasOptimizer, replacing it with a more robust implementation in DistributionStrategy.


Download from videomanifest URL of my University

There are some Recs of my University that have URL videomanifest like
https://westeurope1-mediap.svc.ms/transform/videomanifest?provider=spo&inputFormat=mp4&cs=fFNQTw&docid=https%3A%2F%2Fmyuniversity.sharepoint.com%3A443%2F_api%2Fv2.0%2Fdrives%2Fb!p2vMJwRMykGn6ItXgVEVJ9NMxRUjCBdCjzLx6T307npH-p6A4Cv8TYb8PIqo6-Oi%2Fitems%2F01BZB4AY6SWNJORZKIRNB3KK5X3P55DFWC%3Fversion%3DPublished&cTag="c%3A{E852B3D2-48E5-438B-B52B-B7DBFBD196C2}%2C2"&action=Access&part=index&format=dash&useScf=True&altTranscode=1&altManifestMetadata=eyJBdWR....xkIjotMSwiX01ham9yIjoxLCJfTWlub3IiOjAsIl9SZXZpc2lvbiI6LTF9LCJWaWRlb0NvZGVjSW5mbyI6ImgyNjQiLCJWaWRlb1ByZXRyYW5zY29kZURhdGEiOm51bGwsIlZpZGVvU3RyZWFtRHVyYXRpb24xMDBOYW5vIjo1NjM3MTg0MzAwMH0%3D&enhanceAudio=true&pretranscode=0&transcodeahead=0&enableCdn=1&psi=dbb5e671-488a-4924-86f6-b111d09bb3c4&ccat=1&correlationid=a826cd8b-7c45-4c9b-b8c8-ad4cddb561d1&hybridPlayback=true&access_token=eyJ0eXAiOiJKV1....Yz0
can Destreamer download them?
I tried to put this URL but it shows me the error "invalid URL".
Sharepoint is compatible with this program?

The current version of Destreamer doesn't support Stream on Sharepoint. You could  use ffmpeg  to download any HLS source as long as you have valid videomanifest and permissions to read/download stream chunks.
ffmpeg -i "https://theURLtoTheManifestYouCopiedHere" -codec copy path\to\downloaded\video.mp4


LAB_AK_04_Lab1_Ex1_KQL.md - KQL best practice

Contact Details
<EMAIL_ADDRESS>What should we change?
There are a few querying practices that bug me when reading these labs. I don't think they are really worh raising a bug ticket here but I did want to mention them.
Data types
We should always be mindful of the data type of a column.
Replace:
    EventID == "4624"

With:
    EventID == 4624

Quotes
I know that KQL doesn't care about which quote you use but it would be nice to have some consistency in the lab.
For example, Lab 4 uses single and double quotes at different points.
    EventID == "4624"
    EventID == '4624'

Combining where clauses
It would be great to explain why the lab uses
    | where TimeGenerated > ago(1h)
    | where ProcessName != "" and Process != ""

Instead of
    | where TimeGenerated > ago(1h) and ProcessName != "" and Process != ""

Or
    | where TimeGenerated > ago(1h)
    | where ProcessName != ""
    | where Process != ""

Semicolons
Is it best practice to end all KQL queries with semicolons?
Lab
Other
Relevant screenshots
paste here 😉

Fixed in latest PR.


An updated C# FSM

Hi @theRealUnd3rdog,
While exploring your repository's code to improve my first-person controller, I noticed you're using the C# state machine I authored for GodotParadise (now closed).
I wanted to commend you on the excellent work in your repository. It greatly improved my understanding of first-person movement complexity.
I'm sharing an updated version of the state machine with some bug fixes related to the stack and a new, simpler transition system.
You're welcome to take a look or use it to update your FSM, if you wish.
FSM-Updated
Cheers!

Thanks! @BananaHolograma
I want to thank you for making your C# state machine open source.
It was extremely easy to integrate into my existing movement system.
I will take a look at your current implementation and update whatever I have already.
Since your repository was taken down the last time, I couldn't credit your work. I will credit it this time.
Cheers!

The credit is all yours! Keep up the fantastic work. Discovering that something I created time ago has helped someone is truly rewarding and I don't need anything else in life.


metrics[6]: heatmap visualization with bounds [GSoC 2023 @ OpenVINO]

Description
This is part of Anomaly Segmentation Metrics for Anomalib — GSoC 2023 @ OpenVINO.
Changes
This will make it possible to have heat maps with normalization bounds given as an argument, which will used combined with the bounds from AUPImO.
It also modifies superimpose_anomaly_map to make the anomaly score map transparent wherever the scores are bellow the threshold lower bound (since they'd never be classified as 1).
Example (white means "above upper bound", meaning it is always classified as 1):

[ ] Bug fix (non-breaking change which fixes an issue)
[X] Refactor (non-breaking change which refactors the code base)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation update

Checklist

[ ] I have added a summary of my changes to the CHANGELOG (not for minor changes, docs and tests).
[X] I have performed a self-review of my code
[X] I have commented my code, particularly in hard-to-understand areas (If applicable)
[X] I have made corresponding changes to the documentation (If applicable)
[X] I have added tests that prove my fix is effective or that my feature works (If applicable)

replaced by https://github.com/jpcbertoldo/anomalib/pull/13


finish to write every part of the entity deletion system

It will be more of an archive system, the pure deletetion will be reserved for the superadmin probably and for later.


Negative house numbers in US

I ran across the behavior in Ohio where it's most egregious, but there are just shy of 35,000 addresses in that state with a negative house number.  These should not be considered valid and filtered out during import.

Whoops, I meant this for pelias/openaddresses!


Postman CollectionTypeError on empty examples object

If I use the following stripped-down OpenAPI file, I get an error during the conversion saying: Converting OpenAPI to Postman Collection Type Error: Cannot read properties of undefined (reading '$ref').
The problem is the empty examples: {} which is sometimes autogenerated by Stoplight Studio. This wasn't a problem with earlier versions of Portman, so I assume that it is related to some recent openapi-format updates. I'm not sure whether this can be fixed here, but for me it's most important that Portman fails on such errors when I use it in the CI. Otherwise, nobody will realise that the tests are not generated at all.
openapi: 3.0.1
info:
  title: Test
  version: 1.0.0
  description: Test
servers:
  - url: https://example.de/test
security:
  - bearerToken: []
tags:
  - name: Test API
paths:
  /test:
    get:
      summary: Get test
      tags:
        - Test API
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                type: object
                properties:
                  test-value:
                    type: string
              examples: {}
      operationId: get-test
      description: Test
  securitySchemes:
    bearerToken:
      type: http
      scheme: bearer

hi @miriamgreis
Looking into the matter.
Do you apply a filter on the OpenAPI file?

No, just running Portman with the local OpenAPI file. Nothing special.

Openapi-format is only triggered with there is a filterFile, so I'll look further in the matter.
Do you happen to know, in which version it still worked?

I just tried to figure it out as I had a very old version installed locally. The first version that I see the error message with is v1.29.0, but you might want to doublecheck.

@miriamgreis Found the issue: the openapi-to-postman "convertV2" fails. We introduced the V2 method, which has become the default by Postman, in v1.29.
I'm created an the issue in the openapi-to-postman github.

It should fail:
postmanObj = await oaToPostman.convert(oaToPostmanConfig).catch(err => {
        console.error('Postman Collection generation failed: ', err.toString())
        process.exit(1)
      })

Let me double check if the error trigger is not properly catched in the nesting.

hi @miriamgreis, an update: We just submitted a PR that should overcome the error in the openapi-to-postman library.

as possible work-arounds for now:

manually remove the empty example
If you would use any filter or only a filter with preserveEmptyObjects: false, it will sanitize the OpenAPI spec of empty objects.

Since openapi-to-postman does not throw an error and returns a postman obj, we cannot catch the error with proper exit 1 code. In the PR on openapi-to-postman, we added the throwing of an error.

Perfect, thanks. For now I anyway manually removed the empty example anyway. I saw your PR for openapi-to-postman, so seems that everything needs to be resolved on their side.

update: The PR for openapi-to-postman is merged in.
We will have to wait for the next release of openapi-to-postman to have it in Portman.


Update app code

use return instead of echo
replace $heroes with $dungeons

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 4 (0.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 88.727%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

app/Controllers/HeroController.php
0
1
0.0%

app/Controllers/DungeonController.php
0
3
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
244

Relevant Lines:
275

💛  - Coveralls

@paulbalandan Do you know why this error is reported?
 ------ --------------------------------------------------------------------
  Line   tests/database/FakerTest.php
 ------ --------------------------------------------------------------------
  49     Call to method PHPUnit\Framework\Assert::assertInstanceOf() with
         'App\\Entities\\Dungeon' and array<int, App\Entities\Dungeon> will
         always evaluate to false.
         💡 Because the type is coming from a PHPDoc, you can turn off this
            check by setting treatPhpDocTypesAsCertain: false in your
            phpstan.neon.dist.
 ------ --------------------------------------------------------------------

https://github.com/codeigniter4projects/playground/actions/runs/6680594906/job/18153864260?pr=218

Oops! I missed the pipeline failures. My guess is that somewhere the array return type of find() is taking precedence over the single-value ID lookup version. I think this is a core issue with the method, violating single-responsibility, but I don't see in the framework what would cause this so hopefully @paulbalandan knows the source in his library. In the meantime we could ignore this or change it to where(...)->first().


[Rest Server] Add secrets field in protocol

Add secrets field in protocol, implement #2729:

Update protocol schema
Parse and render secrets field in commands/auth
Support Docker login using secrets, remove credentials from set -x debug log
Mask secrets field when storing yaml

Coverage decreased (-0.5%) to 52.57% when pulling 6ee78e485b253e2948bd13b641a76ff72a4f3736 on xiongyf/secrets-render into cdc67d628140d1281ce5bc05ffcfcfc6032cd233 on xiongyf/protocol-secrets.

Coverage decreased (-0.5%) to 52.57% when pulling 6ee78e485b253e2948bd13b641a76ff72a4f3736 on xiongyf/secrets-render into cdc67d628140d1281ce5bc05ffcfcfc6032cd233 on xiongyf/protocol-secrets.

Coverage decreased (-0.5%) to 52.57% when pulling 6ee78e485b253e2948bd13b641a76ff72a4f3736 on xiongyf/secrets-render into cdc67d628140d1281ce5bc05ffcfcfc6032cd233 on xiongyf/protocol-secrets.


refactor(userspace): move falco logger under falco engine

What type of PR is this?
/kind bug
/kind cleanup
Any specific area of the project related to this PR?
/area engine
What this PR does / why we need it:
Since introducing the method evttype_index_ruleset::print_enabled_rules_falco_logger(), we created an implicit dependency between the falco_engine target and the falco logger, which instead lives under the falco_application target. The two targets don't depend on each other and the linking succeeds just because they're both included by the falco cmake executable target. Pedantic linker checks will have this failing instead, and is fundamentally wrong even if it works by luck.
There are many ways for overcome this, but my proposal is to just move the falco logger as part of the falco engine's code. There are many instances in which logging would be useful and it hasn't been done exactly because of this uncomfortable dependency, so moving the component seems like an easy and appropriate fix IMO.
Which issue(s) this PR fixes:
Special notes for your reviewer:
I'll let the reviewers decide on the right milestone. IMO this may be 0.38.0 material for better code quality.
Does this PR introduce a user-facing change?:
refactor(userspace): move falco logger under falco engine

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: jasondellaluce
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/unhold
False positive

Question, what about creating an entire new target falco_utils (or whatever) that will be then liked by both falco_application and falco_engine?
I mean, aren't we still creating a link between falco_engine and falco_application with your solution?
Also, i can see people struggling to find source code for the falco logger that imo does not seem to belong to the engine, at a first glance at least.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP, jasondellaluce
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP,jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: e6f05b0208f6c6cc688d175b94cb527e47d26ceb

Yep let's go with this!
/milestone 0.38.0

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP, jasondellaluce, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP,jasondellaluce,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


[TEMP] Doc changes, non-function menu-cli

Minor changes to docs, non-function menu cli placed

Minor changes to docs, non-function menu cli placed


[switch] Clear dialing queues when switch is shutting down

Currently when the switch is shutting down the dialing queues are not cleared. When the switch is shutting down we should clear the pending dials queue, and then abort all in progress dials. This will help prevent unwanted behavior when a node is shutting down / "stopped".
Depends on https://github.com/libp2p/interface-transport/issues/43, aborting in progress dials.
Relates to https://github.com/libp2p/js-libp2p-switch/issues/281

This should be resolved


Validate command argument for subprocess/remote-shell execution

A couple of examples are the FractalSSH.remove_folder method
https://github.com/fractal-analytics-platform/fractal-server/blob/05474dbf6b18848b893b994c50ed9d2e6777bba8/fractal_server/ssh/_fabric.py#L291-L308
or the following TaskCollectCustomV2 validator:
https://github.com/fractal-analytics-platform/fractal-server/blob/fd3503ccdcffd429391fffb4f728ae4fca0b9998/fractal_server/app/schemas/v2/task_collection.py#L146-L155
Rather than re-defining this validation layer in multiple places, we should have a single validation function similar to these examples, and apply it every time we call e.g. subprocess.run or fabric.Connection.run.

Ref https://www.oreilly.com/library/view/learning-the-bash/1565923472/ch01s09.html

The new function validate_cmd is to be placed in https://github.com/fractal-analytics-platform/fractal-server/blob/main/fractal_server/string_tools.py, and used in front of any subprocess.run or FractalSSH.run_command_over_ssh call.

If we move https://github.com/fractal-analytics-platform/fractal-server/blob/main/fractal_server/app/runner/run_subprocess.py up one level, we could include validate_cmd in this wrapper and then try to always use the wrapper rather than the raw function.

[ ] Define validate_cmd as described above (exceptions from the linked list: /, ", ', ~) - in string-tools
[ ] Define sanitize_cmd, with the same logic
[ ] Unit tests of both
[ ] Include validate_cmd explicitly before any call to subprocess.run and any fabric.Connection.run (e.g.  self._connection.run somewhere)


[Snyk] Upgrade colors-cli from 1.0.28 to 1.0.32

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade colors-cli from 1.0.28 to 1.0.32.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 4 versions ahead of your current version.
The recommended version was released 5 months ago, on 2023-03-09.

Release notes

Package name: colors-cli

1.0.32 - 2023-03-09
npm i<EMAIL_ADDRESS>v1.0.31...v1.0.32

🐞 fix: fix subpath issue. 677c3c3 @ jaywcjlove
🌟 feat: exports color-name/color-safe/styles-name. f48884e @ jaywcjlove
🌟 feat: exports color-name/color-safe/styles-name. 8db6078 @ jaywcjlove

1.0.31 - 2023-03-07
npm i<EMAIL_ADDRESS>v1.0.30...v1.0.31

🐞 fix: fix subpath exports issue. e62bea7 @ jaywcjlove

1.0.30 - 2023-03-07
npm i<EMAIL_ADDRESS>v1.0.29...v1.0.30

🐞 fix: subpath exports. (#9) a0ef34a @ jaywcjlove
💄 chore: update workflows config. 3d6be0d @ jaywcjlove
🐞 fix: subpath exports. (#9) b46f997 @ jaywcjlove

1.0.29 - 2022-12-15
npm i<EMAIL_ADDRESS>v1.0.28...v1.0.29

💄 chore: Update .github/workflows/ci.yml b4a4666 @ jaywcjlove
📖 doc: Update README.md a34fdc2 @ jaywcjlove
📖 doc: Update README.md f659328 @ jaywcjlove
💄 chore: update .github/workflows/ci.yml 9842e53 @ jaywcjlove
💄 chore: update .github/workflows/ci.yml 887fde0 @ jaywcjlove
💄 chore: Add homepage feild in package.json 586bfe1 @ jaywcjlove
🆎 type: fix type error. #8 4105b25 @ jaywcjlove
💄 chore: update workflows config. fa35b5f @ jaywcjlove

1.0.28 - 2021-09-11
npm i<EMAIL_ADDRESS>v1.0.17...v1.0.28

📄 Update .npmignore b8c4bac
📄 Add type declaration file. #2 e96c2b9
📄 Fix #2 type declaration error. d42e0ac
📄 Add test case. c90fcb5
📄 Update screenshot image. 6274e17
📄 Update screenshot image. 0984350
📄 Modify typing. 336eee4
📄 Remove dependencies. fd0a9c7
📄 Create FUNDING.yml 34c7e90
📄 Modify typing. 9b3fa8e
📄 Merge branch 'master' of github.com:jaywcjlove/colors-cli 990eccb
📄 Modify package.json 8fbf667
📄 Add typing file. cef7b12
📄 Add renovate.json (#3) 77ac683 @ renovate[bot]
📄 [Security]Fix ReDoS (#7) 78ee15b @ ready-research
📖 doc: Delete .travis.yml a787e15
💄 chore: Add .github/workflows/ci.yml d9ebfd3
💄 chore: update .github/workflows/ci.yml dde389a
💄 chore: update .github/workflows/ci.yml 37d7780

    from colors-cli GitHub release notes

Commit messages

Package name: colors-cli

c90d257 released v1.0.32
8db6078 feat: exports color-name/color-safe/styles-name.
f48884e feat: exports color-name/color-safe/styles-name.
677c3c3 fix: fix subpath issue.
cc44f25 released v1.0.31
e62bea7 fix: fix subpath exports issue.
3ab78ce released v1.0.30 (#9)
b46f997 fix: subpath exports. (#9)
3d6be0d chore: update workflows config.
a0ef34a fix: subpath exports. (#9)
d4cecef released v1.0.29 #8
fa35b5f chore: update workflows config.
4105b25 type: fix type error. #8
586bfe1 chore: Add homepage feild in package.json
887fde0 chore: update .github/workflows/ci.yml
9842e53 chore: update .github/workflows/ci.yml
f659328 doc: Update README.md
a34fdc2 doc: Update README.md
b4a4666 chore: Update .github/workflows/ci.yml

Compare

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Codecov Report

Merging #9 (8c4d649) into master (65d94a9) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##            master        #9   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            4         4
  Lines          162       162
  Branches        25        25
=========================================
  Hits           162       162


Suppress alarmist Tomcat DataSource messages from the logs

Since "driverClassName" property is optional, most projects omit it. Still this absolutely correct configuration causes the following ugly message in the logs, which we need to suppress:
Dec 01, 2016 3:34:05 PM org.apache.tomcat.jdbc.pool.PooledConnection connectUsingDriver
WARNING: Not loading a JDBC driver as driverClassName property is null.

The fix only works partially, namely when SLF intercepts JUL (e.g. under bootique-logback and such). Ideally we should submit a patch to Tomcat to reduce this log level down...


Use context everywhere

Building on #1, this modifies the API so all methods on SqlExecutor (and their corresponding methods on DbMap and Transaction) take a context as their first argument.
This removes the internal ctx field on DbMap and context, and removes WithContext from both.
This also renames Exec, Prepare, Query, and QueryRow on those structs to add Context at the end of the names, matching database/sql.

Integration tests are failing because I didn't update them yet - didn't realize I needed to pass the special tag locally to build them. Working on it.

Alright, tests pass!

@jsha Can you rebase on main please?


Need unique enum values in IPhyIoType

to handle any controller which imports both the config and the info message definitions.
@bharani-zededa should fix your issue.

LGTM

LGTM