Deltacrons with NSP3 breakpoint

Thank you for developing this incredible tool! I'm testing it out by reproducing some of the recombinant clades described in pango-designation. So far it's going really well, but I'm struggling with detecting/interpreting Deltacrons with an NSP3 breakpoint. Some context:

So far, BA.1/BA.2 recombination is straightforward to detect and interpret (regardless of breakpoint) (ex. cov-lineages/pango-designation#448)
python3 search_recombinants.py pango-designation_issue_448.fasta

With some parameter tweaking, I can also recover Deltacrons when the breakpoint involves the S gene (ex. cov-lineages/pango-designation#444). So many intermissions though!
python3 search_recombinants.py pango-designation_issue_444.fasta \
  --parents 2-4 \
  --breakpoints 1-4 \
  --max-intermission-count 50 \
  --max-intermission-length 3

But I can't reliably detect Deltacrons that do not involve the S gene, such as at NSP3. (ex. cov-lineages/pango-designation#446).
python3 search_recombinants.py pango-designation_issue_446.fasta \
  --parents 2-4 \
  --breakpoints 0-10 \
  --unique 1

Part 1:

Part 2:

RIPPLES also struggles with these. It identifies the 21I & 21K samples as recombinants (Part 1), and not the 21I & 21J & 21K (Part 2).
My interpretation is that these are "Potential Deltacrons", but there is insufficient nucleotide information to resolve it one way or another. Would you agree with this classification/interpretation?

I will take a closer look on this later… Until then, two quick observations:
First picture
The many intermissions might be caused by the definition of the parent lineages, which might contain too many mutations which only occur rarely. This is discussed in more detail in #4 and is partly fixed in the current version. I can't see which version you are using (also, there are no real "versions" of this tool for now, just irregular updates to the main branch), but I can see that it's not the most current one. Later, there will be a full fix, that lets you select a threshold for the mutation prevalence, so that more or less mutations will be taken as required.
Third picture
There are some very faint grey letters on white ground - poor color choice by me and general issues with the colors on different systems, as noted in #2. But the actual point is: your sequences begin with TT..T.T..G...... which clearly isn't Omicron, and resembles 21I much more than 21J. While G4181T hints at 21J, I'd think that a single mutation is very little evidence for a three-parent-recombination-event. I would suspect that without --parents 2-4 and --unique 1 these would be classified as 21I / BA.1 recombinants and a much clearer picture would be the result.

Oh, and thanks for testing it out!
So far, I could only test very few of the sequences from the pango-designation proposals, because I still don't have access to GISAID and can only test those issues that contain actual fasta-downloads. I love to see how it performs - or doesn't - on input that I could not test.

First picture
The many intermissions might be caused by the definition of the parent lineages, which might contain too many mutations which only occur rarely. This is discussed in more detail in #4 and is partly fixed in the current version. I can't see which version you are using (also, there are no real "versions" of this tool for now, just irregular updates to the main branch), but I can see that it's not the most current one. Later, there will be a full fix, that lets you select a threshold for the mutation prevalence, so that more or less mutations will be taken as required.

This feature is now available. By default, mutations which occur in at least 75% of sequences on cov-spectrum are considered as required (was 70% in earlier versions). But this can be adjusted by using --mutation-threshold NUM where num is between 0.05 and 1.00.
For my BA.1 / BA.2 sequences from Germany, 75% was the best value (everything between 65% and 80% worked fine), but for your sequences the optimum might be something else.

PS: I guess  meant "second picture" whenever I wrote "first picture".

It's tempting to think of 4181T as "21J only" and 5584G as "21I only".  However, with <30,000 bases to mutate, untold millions of cases, and many millions of sequenced Delta genomes, there are chances for these clade-defining mutations to join each other by recurrent mutations or recombination and show up in some genome sequences.  This cov-spectrum query finds a cluster in AY.44 (21J) that has both 4181T and 5584G, as well as mutations 3505T and 4015G shared by the sequences in cov-lineages/pango-designation#446:
https://cov-spectrum.org/explore/World/AllSamples/Past6M/variants?variantQuery=3505T+%26+4015G+%26+5584G+%26+4181T
Genomes with those 4 mutations were found mostly in Nevada, Utah, Arizona and other western states, well into Dec. 2021.  So I think the puzzling combo of 4181 and 5584 happened in the AY.44 parent of this recombinant.  And I believe the recombination with Omicron happened at some position < 7124.
Before it occurred to me to look for the Delta parent in cov-spectrum, I was really struggling with how to explain 21I, 21J and Omicron all at once.  For the record, here's how far I got with that:

For cov-lineages/pango-designation#446 I think it would be best for someone with more sequencing experience to look at the raw read data for at least some of the sequences, especially the ones with 4181T.
I think that SARS-CoV-2 genomes unfortunately have a significant level of contamination & assembly errors, and I think the prior likelihood is low for the same genome to have Omicron and 21I and 21J mutations -- that means either a) a triple-infection, multiple recombination events and forward transmission, or b) a recombination and transmission event and a spontaneous mutation that just happens to be a 21J mutation.  4181 also seems to blink "off and on" within the UT-UPHL sequences in the UCSC/UShER tree, not a good sign that it's reliably detected there.
On the other hand, five different labs (CDCBI, UPHL, CDC-STM, CDC-FG and CDC-QDX) reported seeing 210T, 4181T, 5584G, followed by 8393A and a bunch of other Omicron mutations (minus Spike amplicon dropouts here and there, no surprise for Omicron), and 3505T and 4015G.  The probability of that happening by the same combination of error modes in five labs is hopefully vanishingly small.  It's almost six labs: UNM reported a sequence, USA/NM-UNM-ED_02539/2022|EPI_ISL_11010751|2022-02-21, that is the same except for an N at 4181.  So I believe in all the mutations except for a lingering doubt about 4181.
Regarding reference-allele-for-21J sites near the beginning -- most could be explained as artefacts of #4 that Lena has fixed with --mutation-threshold -- but not 7124T, that should be shared by pretty much all 21Js.  (But 21I's 5584G preceeds that anyway; by that point it could be either 21I or Omicron, there's a big region of uncertainty between 5584 and 8393.)

BTW this:

looks to me like Spike amplicon dropout, unfortunately very common in Omicron, combined with insufficient filtering out of stray reads from contamination (or just back-filling reference genome sequence?), apparently pretty common in genome assembly pipelines used for SARS-CoV-2 (and the bane of SARS-CoV-2 phylogenetics).

I might be too tired today to really grasp all of what you said, but let me add a few noted and observations:
21I and 21J
You mentioned 21I and 21J and I just noticed that in the current version, both are included in mappings.csv and as such the tool pretends to know them, but in virus_properties.json they are just empty. This is my fault, as I started with a version from nextstrain/nextclade_data (probably this one) and later switched to cov-spectrum. It was easy to map each clade to a pango lineage, except 21I and 21J which both correspond to several AY's each. I'm not sure what exactly to do about it, but it surely can't stay this way. I'll open an Issue for that shortly.
Amplicon dropouts
Distinguishing actual recombination from amplicon dropouts / co-infections / contamination seems to be the hardest and most important part to me. I recently built some features to help with that, by showing which amplicons cover which region, and where the respective primers are. Obviously, this depends on knowlege about the used primer sets. Do you have that information about those USA/UT-UPHL sequences?
To give you an idea what this feature does, here are several versions of ARTIC and EasySeq primers shown with some suspicious German sequences (just for illustration purposes, obviously they have nothing to do with the Delta/Omicron Recombinants this issue is about). In this case, I did not know which primers where used and just made some educated guesses. And I definitely need to write some documentation on how to use and interpret that, probably tomorrow.

Do you think this is a useful feature that could help answer those questions?

21I and 21J

That totally makes sense, yes, it is too bad that there is no direct Pango analog for 21I and 21J.

co-infections / contamination

For distinguishing those from recombinants I think it's best to have access to raw reads (or really, trimmed aligned reads distilled to allele frequencies) so that an an approach like the one in this twitter thread can be applied with plots showing mixture frequencies for co-infection / contam:

vs. all-or-nothing frequencies for transmitted recombinant:

In the absence of within-sample allele frequencies, however, having a lot of genomes with the same mutations, and one or two clear breakpoints, can give us pretty good confidence that a recombinant is real IMO.

Amplicon dropouts

That is a really cool new feature!

Obviously, this depends on knowlege about the used primer sets. Do you have that information about those USA/UT-UPHL sequences?

I don't have that info about UT-UPHL sequences -- @erinyoung? (specifcally USA/UT-UPHL-220202309634/2022, USA/UT-UPHL-220204125669/2022, UT-UPHL-220204444755/2022, UT-UPHL-220204978864/2022)

By the way, I just added the mindnight primers to the repository. From what I could find quickly, there's only one version of them, unlike ARTIC and EasySeq?

I think Midnight had to change only one primer for Omicron, pretty impressive: https://twitter.com/freed_nikki/status/1489015462570586114
This page has a lot of great info about primers and how they're affected by various lineages: https://primer-monitor.neb.com/lineages

@lenaschimmel Let me look those sequences up

The consensus fastas for the following sequences were generated were generated from single-end Illumina reads with the dragen pipeline.
submission_id,sra_id
UT-UPHL-220202309634,SRR17916009
UT-UPHL-220204125669,SRR17977717
UT-UPHL-220204444755,SRR17977317
UT-UPHL-220204978864,SRR17977192

I'm off to find their amplicon files.

Thanks a lot, @erinyoung! I'm not competent to interpret those VIRUS DETECTION METRICS, but ARTIC v4 and a spike from BA.2 for me seems to be a good indicator of amplicon 76 dropout, which is the only one that covers this region:

@ktmeaton, or really anyone with access to the Pango #446 sequences, you could run
python3 search_recombinants.py pango-designation_issue_446.fasta \
  --parents 2-4 \
  --breakpoints 0-10 \
  --unique 1
  --primers primers/artic_v4.bed
  --primer-intervals -8400 22882

to get a clearer picture of the primers / amplicons affected, maybe even with --show-private-mutations. I'm not sure if this will deliver any new insight, or only be a fun little exercise to show off my new feature and illustrate what has already been said by me and the others, though :)

Also, I will have a look at #10 (21I / 21J / AY.*) soon. I'm very confident that I can provide a solution within the next few hours.

There's an update on #10 which is also relevant to this issue. See my comment here.

I just wanted to say thank you for this super detailed discussion! :) I'm still thinking and testing it out, but this has helped a lot!


factor out handling of generators that require a model_name

this is currently handled by list in garak/cli.py and garak/generators/__init__.py
would be better to have just one list in one place
might be best to have this specified in the generator classdef

@jmartin-tech hey, what do you know, there's already an issue for this!


Upgrade graph API

Feature Request
As a operator developer that's had many operator releases, I would like capture what my upgrade DAG (Directed Acyclic Graph) looks like because it's hard for me to visualize it from all the releases I've had.  It would be nice for OLM to expose an API, so I can do a POST request with the name of a CSV that OLM is aware of and OLM returns to me a payload with the DAG.

a operator developer that's had many operator releases, I would like capture what my upgrade DAG (Directed Acyclic Graph) looks like because it's hard for me to visualize it from all the releases I've had. It would be nice for OLM to expose an API, so I can do a POST request with the name of a CSV that OLM is aware of and OLM returns to me a payload with the DAG.


[Suggestion] Item Whitelisting

Suggestion from: https://www.spigotmc.org/threads/safedrop-safely-drop-items.402996/#post-4652514
Hello great developers
I would like to provide a suggestion for this plugin,
Add settings to the configuration file to customize project notifications.

example:
#use [] to confirm all drops, use upper case for all items
whitelisted_items:
-BOW
-CROSSBOW
- DIAMOND_SWORD
- GOLD_SWORD
-IRON_SWORD
- STONE_SWORD
- WOOD_SWORD
- NETHERITE_SWORD
-TRIDENT
-IRON_PICKAXE
- DIAMOND_PICKAXE
- NETHERITE_PICKAXE
- GOLDEN_PICKAXE
- FISHING_ROD
-WOODEN_AXE
-STONE_AXE
-IRON_AXE
- DIAMOND_AXE
- GOLDEN_AXE
-NETHERITE_AXE
- IRON_SHOVEL
- DIAMOND_SHOVEL
- GOLDEN_SHOVEL
- NETHERITE_SHOVEL
- DIAMOND
- EMERALD

If you do the above, this plug-in will be even better
Thank you for developing this great plugin!

Done


Invalid content /en/shelley/get-started/creating-a-stake-pool/

Where does the issue occur?
https://testnets.cardano.org/en/shelley/get-started/creating-a-stake-pool/
Describe the issue
Quite a few broken links.
Expected changes
Correct links.
Additional context
Places where the links are:
Operational certificates
...
...
... See an example of an operational certificate generation here. https://github.com/input-output-hk/cardano-tutorials/blob/master/node-setup/node-op-cert.md
...You can see an example of a transaction containing certificates here. https://github.com/input-output-hk/cardano-tutorials/blob/master/node-setup/pool.md
Stake pool metadata
...
The registration certificate https://github.com/input-output-hk/cardano-tutorials/blob/master/node-setup/pool.md#registering-a-stake-pool  contains all the necessary information for the execution of the pro

Thank you for your input, the PR #446 provides updated links, the changes will be visible soon

Thank you for your input, the PR #446 provides updated links, the changes will be visible soon


[HOLD] Navigating to the same conversation will auto scroll to the recent messages

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Expected Result:
User should be taken to the last place in the conversation were it was left off
Actual Result:
User is taken to the recent messages and has to manually scroll back to the place were he/she left off.
Action Performed:

Open chat with user A
Scroll up the chat with user A to an old chat
Open chat with User B
Return to chat with user A

Workaround:
User has to manually scroll back to the place he/she previously left off.
Platform:
Where is this issue occurring?
Web ✔
iOS
Android
Desktop App
Mobile Web
Version Number: 1.0.55-0
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos:
https://user-images.githubusercontent.com/44479856/119872787-d8e1bb00-bef1-11eb-9415-2a91932ac8fe.mp4
Expensify/Expensify Issue URL:
View all open jobs on Upwork

Triggered auto assignment to @Luke9389 (Engineering), see https://stackoverflow.com/c/expensify/questions/4319 for more details.

I think this may be the intended behavior. Gonna go ask and come back.

I think that this could be a nice, but it's a new feature not a bug. The ability to save scroll position and scroll directly to that position in a chat is pretty closely tied to comment linking, and there are some substantial hurdles to overcome before this can be a reality. Mainly, we need to implement bidirectional pagination and scrolling, so we can load the middle of a report, and then use pagination to load newer or older report actions. We think that we'll potentially need to implement our own custom implementation of FlatList, but it's being actively investigated.
So long-story-short let's put this on HOLD until https://github.com/Expensify/Expensify/issues/147480 is complete.

@isagoico, this Monthly task hasn't been acted upon in 6 weeks; closing.
If you disagree, feel encouraged to reopen it -- but pick your least important issue to close instead.


Feedbach on ui

API moved from footer to header
Replaced download icon for new sharper one
Changed plan "huber" to "basic"

@Mikanebu can we delete the branch if this is merged?


libzetta needs libzfs_core to build

Crate name: libzetta
Build failure link: https://docs.rs/crate/libzetta/0.2.0/builds/226022
Additional details: I'm confused why is it building docs for all the dependencies even with --no-deps specified. I don't want this native dependency to be added to the builder because it's heavy.

@andoriyu rustdoc needs the dependency because it generates hyperlinks based on the type signature of a function. If the dependencies aren't built, it can't do type checking and so can't generate the hyperlinks.
@GuillaumeGomez might know more about why the dependencies are necessary.

@jyn514 I see that makes sense. Reason I opened it as a bug is because libzfs_core is PITA to build and install. Plus it pretty big.
So the only way to fix is to add libzfs_core?

Yes, unfortunately. How big is big? We have sandbox limits (https://docs.rs/about) that we can raise if necessary, and also we block network access.

@jyn514 on public pool in azure pipelines it takes ~5 minutes to install from packages, but it pulls in entire ZOL stack https://dev.azure.com/andoriyu/libpandemonium/_build/results?buildId=308&view=logs&j=fbfd68f3-c45d-5150-490a-ea46a8fff233&t=4d880c2f-7af5-59d0-cc8c-6b0059fc3b5e
I guess since builder is ubuntu based it can install the same packages.

Oh that's not bad at all. We preinstall all packages so this would only count the time to build the bindings, not the time to run apt.

@jyn514 I didn't realize that docs builder is very similar to what i use for CI, so I thought it might be PITA to add. If it's ubuntu than it's easy.

Great! If you're interested in adding it yourself, there are instructions at https://forge.rust-lang.org/docs-rs/add-dependencies.html. Otherwise I might get around to it this weekend.
Once it's been added to crates-build-env I can schedule a rebuild so you don't have to publish a new point release.

Great! If you're interested in adding it yourself, there are instructions at https://forge.rust-lang.org/docs-rs/add-dependencies.html. Otherwise I might get around to it this weekend.
Once it's been added to crates-build-env I can schedule a rebuild so you don't have to publish a new point release.

Any update on this? I don't any macOS or Linux installed anywhere to test it myself.

You don't need Mac or Linux to test it, just docker-compose.

You don't need Mac or Linux to test it, just docker-compose.

Well, I do need docker though. Anyone, I setup the VM with docker.
Documentation says how to add packages from official repos, but in my case it's from 3rd party PPA.
I tried at least building the docs dirty way. I built the image, but building actual docs isn't working for me:

docker-compose run -e DOCS_RS_LOCAL_IMAGE=build-env \
                     -v /opt/libzetta-rs:/opt/rustwide/workdir \
    web build crate --local /opt/rustwide/workdir

Starting docsrs_db_1 ... done
Starting docsrs_s3_1 ... done

export CRATESFYI_PREFIX=/opt/docsrs/prefix
export DOCS_RS_DOCKER=true
export RUST_LOG=cratesfyi,rustwide=info
export PATH="$PATH:/build/target/release"

# Try migrating the database multiple times if it fails
# This avoids the docker container crashing the first time it's started with
# docker-compose, as PostgreSQL needs some time to initialize.
set +e
failed=0
while true; do
    if ! cratesfyi database migrate; then
        ((failed=failed + 1))
        if [ "${failed}" -eq 5 ]; then
            exit 1
        fi
        echo "failed to migrate the database"
        echo "waiting 1 second..."
        sleep 1
    else
        break
    fi
done
set -e

cratesfyi database update-search-index
cratesfyi database update-release-activity

if ! [ -d "${CRATESFYI_PREFIX}/crates.io-index/.git" ]; then
    git clone https://github.com/rust-lang/crates.io-index "${CRATESFYI_PREFIX}/crates.io-index"
    # Prevent new crates built before the container creation to be built
    git --git-dir="$CRATESFYI_PREFIX/crates.io-index/.git" branch crates-index-diff_last-seen
fi

cratesfyi build update-toolchain --only-first-time
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', src/bin/cratesfyi.rs:205:31
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

send help

What are the permissions on /opt/libzetta-rs directory on the host? If you can run with RUST_BACKTRACE=1 on the host that would also be helpful.
In the meantime I'll see if I can make that PR to crates-build-env for you.

I haven't been able to build this, even installing every package you install on CI. I have the following added:

zfs-dkms zfsutils-linux libnvpair1linux
libblkid-dev libattr1-dev libcurl4-openssl-dev libelf-dev libdw-dev cmake gcc binutils-dev libiberty-dev zlib1g-dev libssl-dev
libzfslinux-dev

but the builder is still telling me libzfs_core not found (I'm building zfs-core-sys 0.1.0 because it doesn't take as long):
2020/03/24 06:55:00 [INFO] rustwide::cmd: [stderr] thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Failure { command: "\"pkg-config\" \"--libs\" \"--cflags\" \"libzfs_core\"", output: Output { status: ExitStatus(ExitStatus(256)), stdout: "", stderr: "Package libzfs_core was not found in the pkg-config search path.\nPerhaps you should add the directory containing `libzfs_core.pc\'\nto the PKG_CONFIG_PATH environment variable\nNo package \'libzfs_core\' found\n" } }', build.rs:8:9

Oh whoops, our docs on forge are out of date, that should say DOCS_RS_LOCAL_DOCKER_IMAGE. I'll update it in a second.


Add i18n (internationalization and localization)

GPGit should be translatable. I've read some tutorials but I'd like to have some help on that. Especially I need some setuptools scripts that I never used before. contact me pls
Also I need volunteer translators and those who review the translations.

English - Default
German - NicoHood

I will close this as it seems not to be essentially required for gpgit as command line tool.


[FEATURE REQUEST] Add arm64 support to 'postgresql' component

Is your feature request related to a problem? Please describe.
Currently we only support the x86_64 architectures but we need to add support for arm64.
Describe the solution you'd like

We want the ability to install the postgresql on arm64 machines running CentOS.
We only want to support the single master node configuration. Any additional configuration (replication, Pgpool etc) should not be available for arm64 for now.
Only the latest versions we support: 10
Any requirements missing or not added in #2033 will be added for the effected roles.
postgres-exporter, node-exporter, filebeatcan be skipped at this point.

Affected Ansible roles:

postgresql
applications
firewall

Skip the following Ansible roles for now by using a custom feature-mapping config:

postgres-exporter
node-exporter
filebeat

Describe alternatives you've considered
None.
Additional context
No.
Extra info
Design document and resources:
https://github.com/epiphany-platform/epiphany/tree/develop/docs/design-docs/arm
Terraform scripts for ARM hardware on AWS:
https://github.com/mkyc/aws-arm-simple-setup
YAML for testing:
TODO

DoD checklist

[ ] Changelog updated
[ ] COMPONENTS.md updated / doesn't need to be updated
[ ] Feature has automated tests
[ ] Automated tests passed (QA pipelines)

[ ] apply
[ ] upgrade

[ ] Idempotency tested
[ ] Documentation added / updated / doesn't need to be updated
[ ] All conversations in PR resolved
[ ] Solution meets requirements and is done according to design doc
[ ] Usage compliant with license

✅ Tested, works fine, db is available and I was able to execute (manually) tests from our code in https://github.com/epiphany-platform/epiphany/tree/ARM/core/src/epicli/data/common/tests/spec

In addition need to verify that it's possible to install only supported PostgreSQL extensions (pgaudit). It's validated in preflight checks.

Assertion was added in preflight check to fail if there is PgBouncer extension or PostgreSQL replication with RepManager enabled.
:heavy_check_mark: Tested on aarch64 instances.
2021-04-26T13:30:31.2657197Z [38;21m13:30:31 INFO cli.engine.ansible.AnsibleCommand - TASK [preflight : Check if PostgreSQL extensions are supported for ARM architecture] ***
2021-04-26T13:30:31.3655809Z [31;21m13:30:31 ERROR cli.engine.ansible.AnsibleCommand - fatal: [ec2-1-1-1-1.eu-west-1.compute.amazonaws.com]: FAILED! => {
2021-04-26T13:30:31.3657908Z [38;21m13:30:31 INFO cli.engine.ansible.AnsibleCommand -     "assertion": "not (postgresql.specification.extensions.pgbouncer.enabled | default(false) or postgresql.specification.extensions.replication.enabled | default(false))",
2021-04-26T13:30:31.3662985Z [38;21m13:30:31 INFO cli.engine.ansible.AnsibleCommand -     "changed": false,
2021-04-26T13:30:31.3664740Z [38;21m13:30:31 INFO cli.engine.ansible.AnsibleCommand -     "evaluated_to": false,
2021-04-26T13:30:31.3666061Z [38;21m13:30:31 INFO cli.engine.ansible.AnsibleCommand -     "msg": "PgBouncer and Repmgr extensions are not supported for ARM and cannot be installed"
2021-04-26T13:30:31.3669860Z [38;21m13:30:31 INFO cli.engine.ansible.AnsibleCommand - }

:heavy_check_mark:  Tested also on x64 (the assertion is skipped).
2021-04-26T15:29:26.8195295Z [38;21m15:29:26 INFO cli.engine.ansible.AnsibleCommand - TASK [preflight : Check if PostgreSQL extensions are supported for ARM architecture] ***
2021-04-26T15:29:26.8709069Z [38;21m15:29:26 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-1-1-1-1.eu-west-3.compute.amazonaws.com]


Allow Istio Ingress to use ELB to do SSL termination

Describe the feature request
I would like to use a cert from AWS cert manager on an ELB to do the SSL Termination
Describe alternatives you've considered
Just use a Kubernetes LoadBalancer Service
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:us-west-2:754210483472:certificate/0c9ccc7b-27f2-43c6-a93f-4efb8a3e6a46"
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"

@mabushey this already works fine. Keep in mind that you need to change the protocol of https port in you gateway configuration to HTTP.

@ibelikov Thanjk, awesome! Unfortunately I can;t make this work / have no idea how to do this.
This is what I have in my gateway.yaml:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: ecample.com-gateway
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*.example.com"
  - port:
      number: 443
      name: https
      protocol: HTTP
    hosts:
    - "*.example.com"

@mabushey well, similar config actually works for me.
A couple of things that you could check:

If you applied annotations to Ingress Gateway service after the ELB was created, check that they actually are applied (e. g. ELB listeners in AWS are HTTP/HTTPS, not TCP and the right certificate is actually set to be served). Trying with freshly created ELB also works.
Change port names in Gateway to something unique (but keeping the protocol prefix), I stick to "http-gatewayname-namespace" and "https-gatewayname-namespace" style naming. It's not that obvious through Istio docs, but a lot of issues with multiple Gateways come from using the same port names.

@ibelikov Thanks for the tip on the gateway name. I did not apply any annotations to the gateway, I'm doing everything in default as namespaces are not compatible with istio Ingress.

the right certificate is actually set to be served
No, it's not. It does not appear that Istio can work with cert manager. Kubernetes supports this, but not Istio.

I was able to manually hack the LB to get it to work. I deleted the listener for 443 HTTPS to 31390 HTTP and recreated it using the SSL cert in Cert Manager.

Hello All,
Curious if there's a way to configure/deploy Istio Ingress with L7 Term on the ELB using the ARN for a certificate (in AWS IAM)  similar to how it is done for kubernetes ingress-nginx?:
Ref: Kubernetes ingress-nginx
Kubernetes ingress-nginx uses an Elastic Load Balancer (ELB) to expose the NGINX Ingress controller behind a Service of Type=LoadBalancer. Since Kubernetes v1.9.0 it is possible to use a classic load balancer (ELB) or network load balancer (NLB).
Which looks something like this for a legacy ELB:
kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:iam::0123456789012:server-certificate/my-company-com-crt"
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
spec:
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  ports:
    - name: http
      port: 80
      targetPort: http
    - name: https
      port: 443
      targetPort: http
---

I've read the Istio docs noted below, but being new to Istio I could be missing something.
configuring-ingress-using-an-istio-gateway
secure ingress
On this note, seeing an open issue for Istio Ingress TLS key management use ACM #6566
Thanks


limit in subquery

my environment running ArangoDB
I'm using the latest ArangoDB of the respective release series:

[ ] 2.8
[ ] 3.0
[ ] 3.1
[x] 3.2
[ ] self-compiled devel branch

Mode:

[ ] Cluster
[x] Single-Server

Storage-Engine:

[ ] mmfiles
[x] rocksdb

On this operating system:

[ ] DCOS on

[ ] AWS
[ ] Azure
[ ] own infrastructure

[ ] Linux

[ ] Debian .deb
[ ] Ubuntu .deb
[ ] SUSE   .rpm
[ ] RedHat .rpm
[ ] Fedora .rpm
[ ] Gentoo
[ ] docker - official docker library
[ ] other:

[x] Windows, version:
[ ] MacOS, version:

this is an AQL-related issue:
I'm issuing AQL via:

[ ] web interface with this browser:     running on this OS:
[ ] arangosh
[x] this Driver: arangojs

I've run db._explain("<my aql query>") and it didn't shed more light on this.
The AQL query in question is:
FOR outer IN [1, 2, 3]
  limit 3
  let subresult = (for p in ["a", "b", "c"] limit 4 return p)
  for inner in subresult
    return [outer, inner]

my expected result is:
[ 1, 'a' ]
[ 1, 'b' ]
[ 1, 'c' ]
[ 2, 'a' ]
[ 2, 'b' ]
[ 2, 'c' ]
[ 3, 'a' ]
[ 3, 'b' ]
[ 3, 'c' ]

my actual result is:
[ 1, 'a' ]
[ 1, 'b' ]
[ 1, 'c' ]
[ 2, 'a' ]

query explain is:
Execution plan:
 Id   NodeType            Est.   Comment
  1   SingletonNode          1   * ROOT
  2   CalculationNode        1     - LET #6 = [ 1, 2, 3 ]   /* json expression */   /* const assignment */
  6   CalculationNode        1     - LET #13 = [ "a", "b", "c" ]   /* json expression */   /* const assignment */
  3   EnumerateListNode      3     - FOR outer IN #6   /* list iteration */
  4   LimitNode              3       - LIMIT 0, 3
  7   EnumerateListNode      9       - FOR inner IN #13   /* list iteration */
  8   LimitNode              4         - LIMIT 0, 4
 12   CalculationNode        4         - LET #10 = [ outer, inner ]   /* simple expression */
 13   ReturnNode             4         - RETURN #10

It seems to be a wrong optimization AFAIK

I subscribe to your view. This optimization is incorrect and must be prevented.

Hi,
Thanks again for opening this ticket. Version 3.2.10 has been released yesterday
I am now closing this ticket. Please feel free to comment if the problem persists
Many thanks,


METRON-1114: Add group by capabilities to search REST endpoint

Contributor Comments
This PR adds a group by feature to the search REST endpoint.  This PR is fairly complex so I don't think it should be merged before a thorough discussion and review.
The following is an intentionally simple example of how group by works.  This PR exposes a new REST endpoint ("/api/v1/search/group") that allows a user to group results based on a field or list of fields.  For example, grouping results by a field called "fieldA" would be done by POSTing a GroupRequest object that looks like:
{
  "groups": [
    {
      "field": "fieldA",
      "order": {
        "groupOrderType": "term or count",
        "sortOrder": "desc or asc"
      }
    }
  ],
  "indices": [
    "index to search"
  ],
  "query": "lucene query",
  "scoreField": "score"
}

Grouping by multiple fields would just be a matter of adding another group object:
{
  "groups": [
    {
      "field": "fieldA",
      "order": {
        "groupOrderType": "term or count",
        "sortOrder": "desc or asc"
      }
    },
    {
      "field": "fieldB"
    }
  ],
  "indices": [
    "index to search"
  ],
  "query": "lucene query"
}

The "order" property defaults to count descending so it is optional.  The groups would then be nested based on the order they appear in the list:
{
  "groupedBy": "fieldA",
  "groupResults": [
    {
      "key": "fieldA value1",
      "total": 100,
      "groupedBy": "fieldB",
      "groups": [
        {
          "key": "fieldB value1",
          "total": 200
        },
        {
          "key": "fieldB value2",
          "total": 150
        }
      ]
    }
  ]
}

The "scoreField" property is also optional and can be used to calculate the sum of this field within each group.  This has been verified and tested in full dev.  For example, this search query:
{
  "groups": [
    {
      "field": "ip_src_addr"
    },
    {
      "field": "ip_src_port",
      "order": {
        "groupOrderType": "term",
        "sortOrder": "asc"
      }
    }
  ],
  "indices": [
    "bro"
  ],
  "query": "*"
}

should return search results properly grouped by those fields.  The result should like something like:
{
  "groupedBy": "ip_src_addr",
  "groupResults": [
    {
      "key": "<IP_ADDRESS>",
      "total": 5140,
      "groupedBy": "ip_src_port",
      "groups": [
        {
          "key": "49184",
          "total": 121
        },
        {
          "key": "49185",
          "total": 113
        },
        {
          "key": "49186",
          "total": 121
        },
        ...
      ]
    },
    {
      "key": "<IP_ADDRESS>",
      "total": 2160,
      "groupedBy": "ip_src_port",
      "groups": [
        {
          "key": "5353",
          "total": 670
        },
        {
          "key": "50451",
          "total": 1490
        }
      ]
    }
  ]
}

There are a couple of outstanding issues that we need to work through before this gets merged:

I also did a fair amount of refactoring in an attempt to make this class easier to read.  If that is not the case and the refactoring made it worse, let me know.

It has been requested in the past and I think we definitely need more detailed documentation around this particular endpoint.  I am planning on adding some extensive documentation to this PR once we settle on an API design approach.  The alerts UI isn't quite ready for this feature so there's no rush in getting this merged.

Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
For all changes:

[x] Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
[x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

[x] Have you included steps to reproduce the behavior or problem that is being changed or addressed?

[x] Have you included steps or a guide to how the change may be verified and tested manually?

[x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
mvn -q clean integration-test install && build_utils/verify_licenses.sh

[x] Have you written or updated unit tests and or integration tests to verify your changes?

[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

[x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

For documentation related changes:

[x] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:
cd site-book
mvn site

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.

@merrimanr Does the group by support pagination?

No, the "from" parameter is not supported.

The latest commit moves this group by function to it's own REST endpoint.  Combining groups and search results became too awkward once I started looking into adding sorting and size constraints to each group.  I like this better.  The original description has been updated to match.

The latest commit adds the ability to sum a field within each group with the "scoreField".

This is really good, thanks for the contribution!  Are we intending (as a follow-on activity), to enable something like top hits aggregation in here so we can get the documents in the buckets back, or are we just expecting follow on queries to be the norm?
At least for using the API, it seems like it would be nice to at least have something rudimentary, but again, that's not a this PR thing.

The latest commit should address the feedback so far.  I don't feel like the top hits aggregation is necessary at this point but it could be in the future.  I would prefer to wait and keep it simple until we have a need for it.

Updates look good.
I'm +1.  Thanks again for contributing this.


Make half budget optional

This makes the "half-budget" alert optional, as mentioned in #2

Thanks for your contribution!


Features/presidio

This pull request includes several significant changes to the content-loading module, primarily focusing on the introduction of a new PII detection service and the refactoring of existing PII detection logic. The most important changes include the addition of the Presidio PII detection service, renaming and refactoring of existing classes, and updates to configuration and error handling.
Introduction of Presidio PII Detection Service:

content-loading/PresidioReaderFilter.py: Added a new filter class PresidioReaderFilter for PII detection using Presidio, including methods for loading data and detecting PII entities.
content-loading/requirements.txt: Added presidio-analyzer to the requirements file to support the new Presidio PII detection service.

Renaming and Refactoring:

content-loading/PIIServiceReaderFilter.py renamed to content-loading/CognitiveServicesPIIReaderFilter.py: Renamed the class to CognitiveServicesPIIReaderFilter and refactored methods to improve PII detection and error handling. [1] [2] [3] [4]
content-loading/PIIDetection.py: Added new classes PIIDetectedEntity and PIIDetectionError to handle PII detection errors more effectively.

Configuration Updates:

.vscode/settings.json: Updated file associations to include *.SAMPLE.json files as jsonc.
content-loading/local.settings.SAMPLE.json: Updated configuration to include PIIDetectionSource and relevant settings for both Azure Cognitive Services and Presidio.

Error Handling and Document Status Updates:

content-loading/function_app.py: Updated error handling to use the new PIIDetectionError class and modified document status updates to reflect PII detection results. [1] [2] [3]

On second thought, this isn't ready.  Need to figure out a way to get spaCy download in Azure Function.  Or perhaps run in a container.


Various K4A Viewer fixes

Fixes #419 #424 #453 #504
Description of the changes:

Fix IMU graph label being delayed #504
Fix point cloud fragment shader deprecated usage #419
Fix playback of PASSIVE_IR recordings #424
Add better error checking for failed window creation #453

Before submitting a Pull Request:

[x] I reviewed CONTRIBUTING.md
[x] I built my changes locally
[ ] I ran the unit tests
[ ] I ran the functional tests with a device
[ ] I ran the performance tests with a device

I tested changes on:

[x] Windows
[ ] Linux

Thanks for doing this :)

I just finished testing this on linux. Unfortunately there are further issues preventing the point cloud viewer from running on linux.
I will open up a new issue after this is merged, since it's not related to k4aviewer.


convert google sheet to Jason

convert data google sheet to Jason file

I have created Tools for convert Google Sheets to JSON file here.
https://github.com/iampz/c60-gs-data-extract
We can test our site with sample data for now.
For completed data, it will be converted again after data is ready.


Provide guidelines for proposing design review topics

The current guidelines for design reviews provides two ways for sharing topics for discussion in a design review:

a public design doc (shared as a public Google Document)
an I2I

In practice we have also allowed a wide variety of other mechanism, including:

a non-public design doc (generally a Google-internal doc, usually with no Google-internal information)
a non-I2I GitHub issue
a simple comment in the design review issue ("I want to discuss X")

We should provide more clear guidelines on how people can and should bring topics to a design review so that the guidelines match our actual practices.
My proposal is to:

Encourage

explainers (e.g. https://github.com/ampproject/wg-bento/pull/1) for major topics
non-I2I GItHub issues for smaller topics that don't warrant a full explainer, design doc or I2I, where the issue has enough context for anyone in the community to participate in the discussion

Allow

public design docs
I2Is

Disallow

comments in the design review issue not described by a separate GitHub issue
non-public docs

Topics not proposed using one of the Encouraged or Allowed formats would not be discussed at the design review.
/cc @ampproject/tsc @ampproject/reviewers-amphtml @ampproject/collaborators-amphtml

+1 on this proposal. I also support @dvoytenko's work to make the process transparent/public by the nature of the process (avoiding inadvertent under-sharing).

+1. Supportive of this.

OK; if there are no objections I'll go ahead and update the docs to reflect this proposal.


[TOMEE-4053] Dependency properties cleanup, step 2 on branch 8.x

this one has some properties renamed, only when they are used within the same root pom.
these name changes may be tested against ci tests if needed.

CI for this one is running. Let's see what will happen next ;)

my ide returned no other files impacted, unless there are string concatenation, which i believe would be bad.
expect step3 and step4 to impact microprofile :)


basic.ipynb

Issue Type: Bug
I am stuck while running cells of jupyter notebook it is showing error:Timed out waiting to get a heartbeat from kernel process.


Compatible with angular.js 1.6.4?

I like to update angular-vertxbus to version 6.2.0 and angular to version 1.6.4. I am not sure if the wrapper is compatible with angular 1.6.4 because the bower dependency seems to be broken:
"angular": ">=1.2 <=1.6"
When I enter "bower update" it tells me that the best option for angular would be version 1.5.11. What is about the latest version 1.6.4?

Sigh.. I don't know why this not working (any more). As >=1.2 <=1.6 is describing: angular-vertxbus should work with AngularJS 1.6 including.
In another project I'm using < 1.7, so I will update this.
@leolux So long prepare a fixed resolution.

6.2.0 adds official support for AJS 1.6 -- will say I have added an additional test scope without failures.


Don't send back empty proofs if light request fails

At the moment, if the node receives a so-called "light client request" (asking for a storage proof or a call proof), and this request cannot be served, we send back a message indicating that everything is ok, except that it contains an empty proof.
The reason why a request cannot be served is typically that the block has been pruned.
This PR modifies the protocol so that we send back a message without any proof instead.
To do an analogy, right now we more or less send back Some(Vec::new()), while this PR modifies this to send back None.
This makes it clearer from the point of view of the request sender that something bad has happened.

I am wondering a bit about versioning, but will probably leave it like this.

As far as I know, only smoldot sends these requests at the moment, and I'm ok with my own change.

What about the Header and Changes responses (currently both still just return HandleRequestError::BadRequest("Not supported."))?

bot merge


Allow specification of when temp files are deleted

Is your feature request related to a problem? Please describe.
If a file may be used after a checkpoint, but is not in the initial DAG, it is deleted once the last rule requiring the file in the original DAG completes. The file then needs to be recreated after the checkpoint. If the pipeline then fails or the DAG recalculated again, the pipleine must start from the beginning because the modification date on that file is newer than the intermediate files. Additionally, the rule creating the temp file must be executed multiple times.
Describe the solution you'd like
Add an option to the temp() function allowing the files to be deleted once the whole pipeline completes, rather than once the file is no longer required by another rule.
Describe alternatives you've considered
One alternative is not use the temp() function, but this is problematic for large files. Another option is to require the file in the last rule to run, but that adds a fictional dependency to the DAG.

I think I am having a similar issue, where a temp file is needed before and after a checkpoint.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job      count
-----  -------
a            1
all          1
b1           1
b2           1
total        4

Select jobs to execute...
Execute 1 jobs...

[Wed Mar  6 10:04:25 2024]
localrule a:
    output: A.txt
    jobid: 3
    reason: Forced execution
    wildcards: name=A
    resources: tmpdir=/tmp

[Wed Mar  6 10:04:25 2024]
Finished job 3.
1 of 4 steps (25%) done
Select jobs to execute...
Execute 1 jobs...

[Wed Mar  6 10:04:25 2024]
localrule b1:
    input: A.txt
    output: A.b1.out
    jobid: 2
    reason: Forced execution
    resources: tmpdir=/tmp

[Wed Mar  6 10:04:25 2024]
Finished job 2.
2 of 4 steps (50%) done
Removing temporary output A.txt.
Select jobs to execute...
Execute 1 jobs...

[Wed Mar  6 10:04:25 2024]
localcheckpoint b2:
    input: A.b1.out
    output: A.b2.out
    jobid: 1
    reason: Forced execution
    resources: tmpdir=/tmp
DAG of jobs will be updated after completion.

[Wed Mar  6 10:04:25 2024]
Finished job 1.
3 of 4 steps (75%) done
Select jobs to execute...
Execute 1 jobs...

[Wed Mar  6 10:04:25 2024]
localrule a1:
    input: A.txt
    output: A.a1.out
    jobid: 6
    reason: Forced execution
    wildcards: name=A
    resources: tmpdir=/tmp

Waiting at most 5 seconds for missing files.
[...]
    raise IOError(
OSError: Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
A.txt

The problem seems to be that temp file A.txt is remove after rule b1, but it is still needed by rule a1.


Update validaton for instanceURL and dbname

The change omits the validation in the required attribute for instance URL.
The lack of dbname is given as a warning.
Testing:
Manually tested that Dermio connector can be packaged.
Also added test for validating dermio connector

Hi, @saugatt! Thanks for opening this PR!

In the file connector-packager/connector_packager/xsd_validator.py:

Line 297:1: W293 blank line contains whitespace

In the file connector-packager/tests/test_xsd_validator.py:

Line 150:121: E501 line too long (146 > 120 characters)
Line 139:9: E265 block comment should start with '# '

For reference: Error Code Reference, PEP8, Python code style


$select warning is generated when request URL is URL-encoded

GET https://graph.microsoft.com/v1.0/me?%24select=displayName,id

[ REQUEST ]    GET https://graph.microsoft.com/v1.0/me?%24select=displayName,id
[   API   ]  ┌ Passed through
             └ GET https://graph.microsoft.com/v1.0/me?%24select=displayName,id
[ WARNING ]  ┌ To improve performance of your application, use the $select parameter.
             │ More info at https://learn.microsoft.com/graph/query-parameters#select-parameter
             └ GET https://graph.microsoft.com/v1.0/me?%24select=displayName,id

Thank you for reporting the issue @jasonjoh. This clearly shouldn't happen. We'll get on it.


Heroku setup does not run in app's directory if apps already exist on Heroku

When I run suspenders -H shorthanded, I get the following output:
         run   heroku config:add RACK_ENV=staging RAILS_ENV=staging --remote staging from "."
 !    No app specified.
 !    Run this command from an app folder or specify which app to use with --app APP.
      append  bin/setup
         run   heroku config:add SECRET_KEY_BASE=efcbbd7e2038086f52b0323e5c3ac7515efc4ea4b5a6381f7dc142643ce16a9fd3926ae0d0530f8c4f7c5aea9b955c6d77cd9fbd5732368e1f909d549b6875ab --remote staging from "."
 !    No app specified.
 !    Run this command from an app folder or specify which app to use with --app APP.
         run   heroku config:add SECRET_KEY_BASE=b4c97832a558a26405db99e11bb6dff906e1c2abbecd1ba7c10368d5114ee1b37b927116e0b2bc72f37b3b40c3dafc6a1b5529b74d7856ba564bb8c120f84022 --remote production from "."
 !    No app specified.
 !    Run this command from an app folder or specify which app to use with --app APP.
         run   heroku config:add NEW_RELIC_AGGRESSIVE_KEEPALIVE=1 --remote staging from "."
 !    No app specified.
 !    Run this command from an app folder or specify which app to use with --app APP.
         run   heroku config:add NEW_RELIC_AGGRESSIVE_KEEPALIVE=1 --remote production from "."
 !    No app specified.
 !    Run this command from an app folder or specify which app to use with --app APP.

This suggests that heroku is running in the directory that I'm running suspenders shorthanded from, not the actual shorthanded directory.

Hmm, running it in the app directory doesn't work either, same error. Looks like the Git remotes aren't being added if the app already exists:
$ cd shorthanded
$ git remote -v
$

Further up in the backlog, it says:
         run   heroku create shorthanded-production  --remote production from "."
 !    Name is already taken
         run   heroku create shorthanded-staging  --remote staging from "."
 !    Name is already taken

It's unfortunate that there's a lot of output before and after so these lines get buried. But, I think the error message from the heroku client is clear. From Suspenders' perspective, I'm thinking we shouldn't add extra code to guard against it, as the user can re-run the command with a different name.

I've seen a few cases (Heroku name being taken, gems failing to compile) where Suspenders fails somewhere and then attempts to continue. Should we try and get Suspenders to loudly fail and abort when something goes wrong?

Like set -e for Rails generators?

On Jan 19, 2015, at 8:11 AM, Joe Ferris<EMAIL_ADDRESS>wrote:
I've seen a few cases (Heroku name being taken, gems failing to compile) where Suspenders fails somewhere and then attempts to continue. Should we try and get Suspenders to loudly fail and abort when something goes wrong?
—
Reply to this email directly or view it on GitHub.

Like set -e for Rails generators?

Exactly.


feat(remix-eslint-config): extend from typescript-eslint/recommended

Wipes out most of the existing typescript.js lint rules, in favor of directly extending plugin:@typescript-eslint/recommended. Remaining rule configurations are grouped with comments:

Rules that Remix explicitly configures differently from the default
Rules enabled by rules/core.js that aren't needed for TypeScript code
Rules that enforce a style point not agreed upon by Remix ✳
Ones that could eventually be investigated for enabling

Addresses: #6126

[ ] Docs - ... where would I document this? Should I continue to hold off until this is reviewed?
~[ ] Tests~: yarn lint still passes, yes?

Testing Strategy: I'd encourage folks to try this out on any large Remix repository written with TypeScript. And let me know what those are so I can test them out too? 😄
Note that this does not (yet?) tackle two good next steps:

Enable the ever more powerful rules from plugin:@typescript-eslint/recommended-requiring-type-checking

I would highly recommend this, either as an addition to this PR or as a next step.

✳typescript-eslint@v6, currently in beta, separates out any remaining stylistic rules from the recommended configs. So moving to that new major version would make the stylistic disables section unnecessary.

Any code changes are commented inline.

@JoshuaKGoldberg What would it take to enable plugin:@typescript-eslint/recommended-requiring-type-checking? I attempted to do that in #6325, but was closed in favor of this pull request. There doesn't seem to be a extensive conflict between the two changes, so could be cribbed for adding to this pull request.

Yeah enabling plugin:@typescript-eslint/recommended-requiring-type-checking would be a very good next step after this. If the powers-that-be are particularly interested, it could certainly be done either as a followup or as part of this PR.
The main reason I didn't add type checked rules in this PR was type aware rule performance.

time yarn lint on main: 11.86s user
time yarn lint on #6325: 58.61s user

Sorry for taking a couple weeks to respond to your comments @brophdawg11! I lost track of this while traveling 😅. But I'm glad you're looking at it!
I responded to each thread inline - for each, let me know and I'm happy to make the change.

@JoshuaKGoldberg It seems like you missed something that causes an ESLint error
https://github.com/remix-run/remix/actions/runs/5227221637/jobs/9438647396?pr=6248#step:7:9
Do you want to fix that one please?
If you don't have time, I'm happy to do so as well

Aha, introduced merging from main - just pushed a fix now. Thanks!

This should go to dev since it changes a deployed package, so going to squash-merge to my branch and then will rebase onto dev there and get it in.  Thanks!


Remove the unused MSQ Controller context reference for Worker task

Description
The MSQ Controller context reference field for Worker task was added in this recent PR https://github.com/apache/druid/pull/16041
After further discussion, this field should be removed for now.

Key changed/added classes in this PR

ControllerImpl.java remove the new controllerCtx reference field for Worker task

This PR has:

[ ] been self-reviewed.

[ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[ ] added documentation for new or modified features or behaviors.
[ ] a release note entry in the PR description.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[ ] added integration tests.
[ ] been tested in a test Druid cluster.

Relevant comments:
https://github.com/apache/druid/pull/16041#discussion_r1562224219
https://github.com/apache/druid/pull/16041#discussion_r1554524797


(FACT-1737) FreeBSD disks and partitions facts

Managing FreeBSD storage devices is usually done through a modular disk transformation framework, GEOM.  The system ships with libgeom, a library allowing userland interaction with this system.
The commit adds a bunch of facter::util::freebsd::geom_* classes, wrapping the API exposed by FreeBSD for an easy use from Facter's code, and use them to build two missing facts on FreeBSD: disks and partitions.

CLA signed by all contributors.

Can one of the admins verify this patch?

I rebased this PR on top of master.  No conflicts and still working as expected.


[WIP] Add RPS threshold for average latency load-shedding

This PR enables a default RPS threshold for enforcing average latency load-shedding for Mixer.
/hold
At this point, it is exploratory.
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ X ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

@douglas-reid: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

pilot-e2e-envoyv2-v1alpha3_istio
dc4bdb85b48a0cc42fa54935fffa933111d0fbf0
link
/test pilot-e2e-envoyv2-v1alpha3_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@geeknoid seems like codeowners didn't apply here!

@howardjohn should be fixed by https://github.com/istio/istio/pull/17003

Taking a look. Can you also add a "number of messages" dropped metric along with "number of rpcs" dropped?

@bianpengyuan thanks!! I think you solved the mystery of why the owners randomly don't apply sometimes we have been running into

/cherrypick release-1.2
/cherrypick release-1.3

@douglas-reid: new pull request created: #18752

In response to this:

/cherrypick release-1.2
/cherrypick release-1.3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/cherrypick release-1.3

@douglas-reid: new pull request created: #18753

In response to this:

/cherrypick release-1.3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


Skip TLS verification for localhost

Signed-off-by: Aditi Sharma<EMAIL_ADDRESS>https://github.com/containerd/containerd/issues/4826

not related testing this job
/test pull-containerd-node-e2e

/ok-to-test

/test pull-containerd-node-e2e

/test pull-containerd-node-e2e

@mikebrow Updated!!!


Rename the repo to PowerEdu.jl

Go to Settings and the first thing you're gonna see is the option to rename our repo. Here's an example from one of my own repos:

Change made


setting up FastField

@briannakcalkins , @jenajordan , @jess-shue , I've added this .md file to document FastField setup. (Note: .md is similar to .Rmd, but a simpler version for GitHub that doesn't require knitting in R.)
I got the app on my phone, and it's working/ looking great. @briannakcalkins , it may be helpful for you to try it on a personal device before you have access to the lab iPads.

@jess-shue , I have two questions:
1- I may just be confused, but it seems the list of quads in Section 1 don't match available choices when you go to enter quad for specific trees
2- Should each person create their own FastField account? Or can we all share the lab account?

@briannakcalkins , I just discovered that there's a web app, so you can review there as you work on the protocol guide (#9 ).

@teixeirak
1 - It looks like the form wasn't assigned the correct column from the data; it should show the quadrat numbers correctly now.
2 - No, for now everyone should use the lab account (we are charged per user). The form should work for multiple people on the same account, but if it creates problems we can add users. We had used a lab account for our entire census without issue.

Thanks, @jess-shue !

I downloaded FastField app to my phone. No rush for me to log in - I still have plenty to learn here (even learning how to post a response here!). But it sounds like eventually @briannakcalkins & I will use the lab account log in.

@jess-shue , I think it would be valuable to add the surveyor name to the FastField form. We've been keeping track of that in all our previous censuses.

@jess-shue  & @ValentineHerr , we seem to have a problem with the data output into Fastfield, including nonsense lines that sometimes seem to break up individual records:

@ValentineHerr , would the above be messing up the checks?

Yes, probably.

@jess-shue , do you understand what's going on here?
@jenajordan, @briannakcalkins , @troger4, please disregard the error reports until we get this sorted out.

Actually, the only thing it will mess up is saying that the values that are moved to the empty line are missing from the original record (I assume the line above), even though they have been recorded. (my code first deletes the rows with empty quadrat).

This means it actually shouldn't have generated any erroneous reports, as we haven't yet coded any of the EAB tests.
I wonder if those empty lines correspond to photos (???). It would be best to fix this in the app if @jess-shue knows how; otherwise I suppose we just need to shift filled cells in those lines up. I'd like to be sure we understand what causes it, though.

Thanks, @jess-shue . Yes, I believe he extra lines or misaligned rows were created when exporting all of the submissions from FastField Forms into a single Excel file, and that these were not manipulated by R. @ValentineHerr  can verify.

correct. This excel file was imply extracted from the zip file downloaded from FFF, (after selecting all the forms at once)

Question - is there any easy way in FF to, once a tree's data is entered, to have that tree identifier # removed from the list of remaining trees left in that quad? That would prevent the possibility of entering data twice for the same tree.

@jenajordan Sorry Jen, I've tried to explain this before but that is the biggest issue with FFF's - and albeit not a little one. It only sees the data we've given it in the static file and it does not recognize once you've entered data for a tree. With the newest version of the form you'll see 'Stem Count' field on the landing page. When you select the quadrat the stem count will show. As you enter data for stems, a number also appears on that main page next to 'Mortality'. The 'Stem Count' number and the 'Mortality' number should match to show you've collected all the stems. That doesn't prevent double entry, or eliminate the list for you, but it is a small check.

Okay. And I saw that stem count # - it is very helpful!

@teixeirak @ValentineHerr OK, there must be different ways to download the data - when I download all the files it exports an excel file with separate sheets and does not combine the EAB data onto one sheet with the other columns. The misaligned rows appear to be those from the extra data recorded for ash trees.
Downloading the data with separate sheets would eliminate the row error, but we then need to compile the sheets into one file. Let me see if I can figure this out in R!

@jess-shue , maybe wait until @ValentineHerr gets a chance to respond on this, as the R code would need to be worked into her system. It may be more efficient for her to write it.

@teixeirak OK, I'll hold off for now. Do you want me to send the file I downloaded from FastField Forms? I'm not 100% sure why we're seeing different formatting.
I think part of the problem is caused by the 'section_2' worksheet that contains the extra ash data. I'm going to edit the form to include the tag and stem tag of the tree automatically. This won't correct the data collected in the field so far, but will help combine the worksheets going forward.

@jess-shue, sorry for the delayed response here.

Downloading the data with separate sheets would eliminate the row error, but we then need to compile the sheets into one file.

I am not a huge fan of downloading each sheet separately as it is a lot more steps, but if that avoids those misaligned rows, maybe it is worth it.
If I understand well, it sounds like you need to update FFF so that the 'section_2" also contains the tags so the merging is easy?
If yes and if you happen to redownload the data while you figure that out, yes, it would be great if you can put it in the raw_data/FFF_excel folder. If not, I'll go and download them myself, just let me know.
It is easy to work any existing script into the continuous integration system, so if you had worked on a way to merge the sheet together it would have been fine. But I don't mind working on a script once new format is in.

@ValentineHerr No, I didn't download each file individually but as one workbook with separate sheets. When you downloaded the data from FFF's, did it create an excel workbook with multiple sheets?
From the screenshot shared above all of the data are on one sheet with misaligned rows for the EAB data columns. This is not what I receive when downloading and why I asked if it had been post-processed somehow in R. I am able to download one file of all of the submissions, but they include separate sheets:  Root, Form Meta Data, multiphoto_picker1, section_1, section_2, and subform_1

Oh ok,! Let me try to download again and see if I get something different than last time.

I am getting the same file...
@jess-shue,  do you have this interface when ready to download?

I click on Excel (tried CSV and it does not give what we need) .

@ValentineHerr OK, it looks like the new version of the website is causing the problem. I still have access to what they call the 'Legacy Portal'. Using that version of the website, I can download an excel file that I've added to the raw_data/FFF_excel folder.
CSV does not export all of the data. The issue becomes connecting the EAB data to the correct row when combining the sheets into one file.

thanks for uploading the data @jess-shue.
Here is what I see/think:

section_1 has 1737 rows and section_2 has 1738 --> no one-to-on relationship
section_2  Tag and StemTag are empty --> no easy match up.
if any intern or myself proceeds to the download, we will have the other forms with misaligned data

So maybe it is easier to just re-align the data, assuming the misaligned line belongs to the previous line.

@ValentineHerr the Submission ID and row of the data match for the two sections from a quick assessment.

I can combine sheets in R, but I don't know how to designate that the submission ID and row of section_1 should align with the same from section_2
Going forward, new submissions will have the Tag and StemTag filled in and will make this much easier to match the data.
Certainly can align the data assuming that is correct and check!

@ValentineHerr the Submission ID and row of the data match for the two sections from a quick assessment.

It does until row 1722 where there is one extra row from submission 5e8cb8c4-be05-4b6c-9296-a745e7224723 in section_2 than in section_1.
If they were aligning exactly, we would just need cbind to put the data together.

@ValentineHerr Ah, so close! In your script can we remove that extra line and then cbind the data? This would not impact the raw data but allow us to move forward, correct? There are no EAB data after that row either and all those columns should be blank for the remaining rows/trees.

Actually, it does not work...
Look at row 1673 in section_2. It should be from a fram since there is data in there.

But then row 1673 in section_1 is a litu, the fram is right above on row 1672.

Who is going to be downloading the forms in the future (@teixeirak)? if not you @jess-shue and your 'Legacy Portal' we might be better off finding a solution for the other format (the one with misaligned info), which, I think I now trust better as, for the same tree mentioned above, the data is on next row:

Who is going to be downloading the forms in the future (@teixeirak)?

I'd like the interns to download/ push to GitHub after each day's work.

@teixeirak and @ValentineHerr Update from FFF's:  I reached out to FastField Forms about the data exporting/formatting issue and they are still working on fixing some bugs with the new portal's processing. When you login to the FFF's account, do you see these menu options on the left? The "Legacy Portal" would export the data into separate sheets that would then need to be consolidated. Without the tree tags in the ash section to start, this would still be a problem, but if there are issues with exporting from the new portal this is the other option.

Yes, I see the legacy portal.
We can use this to get a file in the needed format, correct? It would be good to get at least a temporary solution here because the field team is making rapid progress, and we want to be able to check those records sooner rather than later.

@teixeirak This at least provides the data in separate sheets - the main issue will be linking the ash data to the specific stem. @ValentineHerr took a look at a spreadsheet I had downloaded previously, so she can comment on aligning the data from multiple sheets as it may not work as I had thought - with the submissionID and row of the data matching.
Has everyone updated their iPads to be on the latest version of the form? It appears submissions are still on version 10 and the tag/stemtag are not populating for the ash section. Everyone should update their form to be on version 11 so new submissions will include those tag/stemtag numbers to help merge the multiple sheets appropriately.

Has everyone updated their iPads to be on the latest version of the form? It appears submissions are still on version 10 and the tag/stemtag are not populating for the ash section. Everyone should update their form to be on version 11 so new submissions will include those tag/stemtag numbers to help merge the multiple sheets appropriately.

@briannakcalkins , @jenajordan , could you please make sure the iPads are updated with the latest version of the FastField form?

@teixeirak This at least provides the data in separate sheets - the main issue will be linking the ash data to the specific stem. @ValentineHerr took a look at a spreadsheet I had downloaded previously, so she can comment on aligning the data from multiple sheets as it may not work as I had thought - with the submissionID and row of the data matching.

@ValentineHerr , could you please update me on where we are with this? (I'm sorry, I'm confused as to what's the next step here...)

Sorry I wasn't able to work on this sooner.
I ended up correcting the misalignment from the bulk Excel import, where all the data comes into one excel file and is located in only one tab.
I updated everything with this commit.
I think it worked but tests failed and I need to figure out why the % complete did not change.
So this is the protocol to download the data.

Thanks @ValentineHerr!  From what I see,  % complete updated (34%).

Oh right, great! it took a few minutes to show up for me.

I think we can close this, then? I'll start a new issue for putting this to routine use.

Yes, I think we can close.


Build Pygloo Error: pygloo/hiredis/include/hiredis is a dangling symbolic link

When I was building pygloo from source, dangling symbolic link error occurs. And I didn't find any file indecating the problem of pygloo/hiredis/include/hiredis. Here is my log.
(base) ➜  pygloo-main python setup.py build
running build
running build_py
running egg_info
writing pygloo.egg-info/PKG-INFO
writing dependency_links to pygloo.egg-info/dependency_links.txt
writing top-level names to pygloo.egg-info/top_level.txt
reading manifest file 'pygloo.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'pygloo.egg-info/SOURCES.txt'
running build_ext
INFO: Analyzed 5 targets (81 packages loaded, 7510 targets configured).
ERROR: /home/cyc/hpc/pygloo-test/pygloo-main/pygloo/BUILD:19:5: Error while validating output TreeArtifact File:[[<execution_root>]bazel-out/k8-fastbuild/bin]pygloo/hiredis/include/hiredis : Child read.h of tree artifact /home/cyc/.cache/bazel/_bazel_cyc/58be26f938b2b5ddf0a0de3d0c123a12/execroot/_main/bazel-out/k8-fastbuild/bin/pygloo/hiredis/include/hiredis is a dangling symbolic link
ERROR: /home/cyc/hpc/pygloo-test/pygloo-main/pygloo/BUILD:19:5: CcGnumakeMakeRule pygloo/hiredis/include/hiredis failed: not all outputs were created or valid

I'm using Ubuntu 20.04LTS, bazel 7.1.1 and Python 3.9.13. Thanks for reviewing.
By the way, I'm wondering whether this repo is still under maintenance? I didnt' find wheels supporting python3.9 and little examples of pygloo on ray were found.

@TRSWNCA Is this issue still there?


host:3000/?q=%s syntax doesn't appear to work

Describe the bug
The Using as a search engine entry indicates that browsers can be configured to use perplexica as a search engine.  However, while using this feature, perplexica opens but never returns a result.  Typing the same search query into the perplexica text entry area does return a result.
To Reproduce
Steps to reproduce the behavior:

open a browser tab
type 'localhost:3000/?q=stuff' in to the address bar and press enter
wait forever

Expected behavior
Expected result is that an answer page is generated.  Instead, perplexica returns no results.
Screenshots
typing "stuff" in to the perplexica text box:
https://github.com/rick-github/Perplexica/blob/use-as-search-engine/.assets/Screenshot from 2024-06-17 01-56-56.png?raw=true
using 'localhost:3000/?q=stuff' :
https://github.com/rick-github/Perplexica/blob/use-as-search-engine/.assets/Screenshot from 2024-06-17 01-57-26.png?raw=true
[Tried adding screenshots but they didn't seem to upload so added links]

Fixed in the latest release.


Fail to start on ubuntu 14.04.5 LTS

Hello,
I just installed, resolved all missing dependencies and run, attached is my logfile.
I am using nodejs 6.3.1 and npm 3.10.3
Thanks

0 info it worked if it ends with ok
1 verbose cli [ '/home/ubuntu/.nvm/versions/node/v6.3.1/bin/node',
1 verbose cli   '/home/ubuntu/pokemon-journal/node_modules/.bin/npm',
1 verbose cli   'run',
1 verbose cli   'start-hot' ]
2 info using<EMAIL_ADDRESS>3 info using<EMAIL_ADDRESS>4 verbose run-script [ 'prestart-hot', 'start-hot', 'poststart-hot' ]
5 info lifecycle<EMAIL_ADDRESS><EMAIL_ADDRESS>6 silly lifecycle<EMAIL_ADDRESS>no script for prestart-hot, continuing
7 info lifecycle<EMAIL_ADDRESS><EMAIL_ADDRESS>8 verbose lifecycle<EMAIL_ADDRESS>unsafe-perm in lifecycle true
9 verbose lifecycle<EMAIL_ADDRESS>PATH: /home/ubuntu/pokemon-journal/node_modules/npm/bin/node-gyp-bin:/home/ubuntu/pokemon-journal/node_modules/.bin:/home/ubuntu/.nvm/versions/node/$
10 verbose lifecycle<EMAIL_ADDRESS>CWD: /home/ubuntu/pokemon-journal
11 silly lifecycle<EMAIL_ADDRESS>Args: [ '-c',
11 silly lifecycle   'cross-env HOT=1 NODE_ENV=development electron -r babel-register -r babel-polyfill ./main.development' ]
12 silly lifecycle<EMAIL_ADDRESS>Returned: code: 1  signal: null
13 info lifecycle<EMAIL_ADDRESS>Failed to exec start-hot script
14 verbose stack Error<EMAIL_ADDRESS>start-hot: cross-env HOT=1 NODE_ENV=development electron -r babel-register -r babel-polyfill ./main.development
14 verbose stack Exit status 1
14 verbose stack     at EventEmitter. (/home/ubuntu/pokemon-journal/node_modules/npm/lib/utils/lifecycle.js:242:16)
14 verbose stack     at emitTwo (events.js:106:13)
14 verbose stack     at EventEmitter.emit (events.js:191:7)
14 verbose stack     at ChildProcess. (/home/ubuntu/pokemon-journal/node_modules/npm/lib/utils/spawn.js:40:14)
14 verbose stack     at emitTwo (events.js:106:13)
14 verbose stack     at ChildProcess.emit (events.js:191:7)
14 verbose stack     at maybeClose (internal/child_process.js:852:16)
14 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:215:5)
15 verbose pkgid<EMAIL_ADDRESS>16 verbose cwd /home/ubuntu/pokemon-journal
17 error Linux 3.13.0-91-generic
18 error argv "/home/ubuntu/.nvm/versions/node/v6.3.1/bin/node" "/home/ubuntu/pokemon-journal/node_modules/.bin/npm" "run" "start-hot"
19 error node v6.3.1
20 error npm  v3.10.5
21 error code ELIFECYCLE
22 error<EMAIL_ADDRESS>start-hot: cross-env HOT=1 NODE_ENV=development electron -r babel-register -r babel-polyfill ./main.development
22 error Exit status 1
23 error Failed at the<EMAIL_ADDRESS>start-hot script 'cross-env HOT=1 NODE_ENV=development electron -r babel-register -r babel-polyfill ./main.development'.
23 error Make sure you have the latest version of node.js and npm installed.
23 error If you do, this is most likely a problem with the pokemon-journal package,
23 error not with npm itself.
23 error Tell the author that this fails on your system:
23 error     cross-env HOT=1 NODE_ENV=development electron -r babel-register -r babel-polyfill ./main.development
23 error You can get information on how to open an issue for this project with:
23 error     npm bugs pokemon-journal
23 error Or if that isn't available, you can get their info via:
23 error     npm owner ls pokemon-journal
23 error There is likely additional logging output above.
24 verbose exit [ 1, true ]

If you are just trying to run it, please download from the releases

Thanks, I managed to run it.


Single stock prediction not working when using turbulence threshold

Hi,
Stocks models, including ensemble method, don't work with single stock in StockTradingEnv.
It fails when running prediction and using turbulence_threshold:
e_trade_gym = StockTradingEnv(df = trade, turbulence_threshold = 70, risk_indicator_col='vix', **env_kwargs)
df_account_value, df_actions = DRLAgent.DRL_prediction(     model=trained_agent,      environment = e_trade_gym)

AttributeError                            Traceback (most recent call last)
 in ()
1 df_account_value, df_actions = DRLAgent.DRL_prediction(
2     model=trained_agent,
----> 3     environment = e_trade_gym)
3 frames
/usr/local/lib/python3.7/dist-packages/finrl/neo_finrl/env_stock_trading/env_stocktrading.py in step(self, actions)
250             self.data = self.df.loc[self.day,:]
251             if self.turbulence_threshold is not None:
--> 252                 self.turbulence = self.data[self.risk_indicator_col].values[0]
253             self.state =  self._update_state()
254
AttributeError: 'numpy.float64' object has no attribute 'values'

Yes, it is expecting a data frame and it is possible with more than one stock. Probably you can navigate to /usr/local/lib/python3.7/dist-packages/finrl/neo_finrl/env_stock_trading/env_stocktrading.py and substitute this
if self.turbulence_threshold is not None:
if len(self.df.tic.unique()) == 1:
self.turbulence = self.data[self.risk_indicator_col]
elif len(self.df.tic.unique()) > 1:
self.turbulence = self.data[self.risk_indicator_col].values[0]
self.state =  self._update_state()

Thank you


Add exits over the air

Use the list merging with serde

The concept here is that we hit updates.altheamesh.com and get a list of exits via https from there we can merge that with our local liset.
Questions

do we just trust https? or do we add a sig
do we allow updating existing exits or just adding new ones (problems with merging)
how often and when do we check?


V3.0.3 Support

Hey guys,
Great work here! I'd like to help out with adding support for v3.0.3 however my pybind11 isn't great. For instance in collections.cpp, resolveMultiMatches (and regularexpressions) now take a Variables::KeyExclusions, however, try as I might to bring that namespace in with PyBind11 it seems to not recognize that. Any ideas?
  virtual void resolveMultiMatches(const std::string& var,
        std::string compartment, std::vector<const VariableValue *> *l,
        Variables::KeyExclusions &ke) {

Hi,
Same problem here, I ended up commenting those Collection functions, you can checkout the branch "feature/modsecurity-v3.0.3" from "https://github.com/j0k2r/pymodsecurity"

Thanks for the headstart @j0k2r If you don't mind I will use your branch as a start point for the migration.
I'm having the same issues with that namespace. In modsecurity source, that class is located at src/variables/variable.h although I cannot find that file in my compiled/installed headers bundle. Apparently it is intended to be used only in intern modules, but still it is exposed in the public api. I will check with them about it. In the mean time, I will publish a version with @j0k2r fixes and tests fixed.
Thanks for the help, guys

I've tested with modsecurity v3.0.4 in the master branch, so I think this can be closed :D


DateTimeColumn vs sorting

Hi people,
can't get my table to sort based on datetime column code i tried:
 $dataTable->add('created_at', DateTimeColumn::class,             [                 "label" => "Created at",                 'format' => 'Y-m-d H:i:s', 'orderable' => true]);
thank you

sorry, it works for me now,  the root cause was my own adpater with different method for sorting...


CMakeLists.txt: use CMAKE_CURRENT_LIST_DIR for module path

Fix #301

Hi! Thank you for your contribution! There are 4 places using CMAKE_CURRENT_SOURCE_DIR in the main CMakeLists.txt and the VcMacros.cmake file. I would be surprised if you do not need to change all 4 of them. Could you please also change the other instances?
Using CMAKE_CURRENT_SOURCE_DIR in the examples and tests is probably fine, since you don't care about them when using Vc as a submodule.

I had a quick look, and I think the remaining places can actually stay as CMAKE_CURRENT_SOURCE_DIR. We can merge this.


Any possible add how to use "learn" cammand and produce structure file in README ?

I try read the source file but can not figure it out.
Is any possible add how to use "learn" cammand and produce structure file in README.md ?

Normally, it should be enough to just use "learn" as a command and any asset or asset bundle file as second parameter. Keep in mind that the files need to be from a browser game.

Hey ata4 could you please reply again to my issue please? Thanks man :)


feat(metrics): Add cumulative transaction counters to improve observability

Closes #12211

hi @onbjerg please help me review it

See https://github.com/paradigmxyz/reth/tree/main/etc#grafana

i just updated the metrics, @onbjerg please help me review, thanks


Add support for miniupnp API v14+

API version 14
miniupnpc.h
  add ttl argument to upnpDiscover() upnpDiscoverAll() upnpDiscoverDevice()
  upnpDiscoverDevices()

https://github.com/miniupnp/miniupnp/blob/master/miniupnpc/Changelog.txt

I think that changelog link is broken.

Sorry, I forgot Github's flavor of Markdown is slightly different from what I'm used to. Fixed.

Thanks; merged.


Remove xerces from JMS starter transitive dependencies

Fixes #6

Pull Request Test Coverage Report for Build 71

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 43.174%

Totals

Change from base Build 69:
0.0%

Covered Lines:
370

Relevant Lines:
857

💛  - Coveralls


How to implement nested Include in beta 3

Hi,
To my understanding you implemented some way to nest multiple level of include (according to past issues).
I have this situation :
public ClassA {
    int id {get; set;}
    List<ClassB> myB {get; set;}
}

public ClassB {
    int idB {get; set;}
    List<ClassC> myC {get; set;}
}

public ClassC {
   int idC  {get; set;}
   ClassD myD  {get; set;}
}

public ClassD {
    int idD  {get; set;}
    string name  {get; set;}
}

I need to select all elements in classA but i need to get the name stored in ClassD. How would I do that?

I'm still working on this syntax on dev branch;
col.Include(x => x.myB[0].myC[0].myD)

ok, thanks for the info. For now i'm using for loop to get my data.

Does col.Include("myB.myC.myD") currently (3.1.0) work?


Update BUMP yaml keys

Companion PR of https://github.com/JCSDA-internal/saber/pull/389
To update all the yamls:
cd ${SOCA}/ewok/defaults
for file in `ls *.yaml`; do ${SABER}/tools/update_yaml.py $file; done
cd ${SOCA}/test/testinput
for file in `ls *.yml`; do ${SABER}/tools/update_yaml.py $file; done
cd ${SOCA}/tutorial/config
for file in `ls *.yaml`; do ${SABER}/tools/update_yaml.py $file; done

This PR can be merged now.


Something appearing in my Redis cache

This is appearing in my cache storage but I don't know why. This should be only saved in the database.
“set” “avl:LERN::Tylercd100\\LERN\\Components\\Recorder::Illuminate\\Validation\\ValidationException” “O:13:\“Carbon\\Carbon\“:3:{s:4:\“date\“;s:26:\“2019-11-09 12:55:58.510027\“;s:13:\“timezone_type\“;i:3;s:8:\“timezone\“;s:14:\“America/Bogota\“;}”

Oh, sorry! This is explained in the config/lern.php file.


Moving sidebar to the right overlays scroll bar

Version: 1.59.0-insider
Commit: c3b4df8f5d9c14aa4cfddb587231d0d52e72e5a2
Date: 2021-07-27T15:10:39.610Z
Electron: 13.1.7
Chrome: 91.0.4472.124
Node.js: 14.16.0
V8: 9.1.269.36-electron.0
OS: Darwin x64 18.7.0
Steps to Reproduce:

Open any repo in VS Code and open a preview of the readme
From menu, click on View > Appearance > Move sidebar right
Notice the scroll bar for the readme is overlayed onto the sidebar

duplicate of #129120

Hi Guys,
I'm ready to help,
If any one could give a little push on where exactly can find the relevant code and any other recommendations, I'm ready to gooo.
Best of luck !

duplicate of https://github.com/microsoft/vscode/issues/129120 .

Yes same root cause as #129120 but keeping open so I can test and verify it separately


Can move text in input field out beyond input field

Open Gallery, open Expansion Panels, open Trip Name, tap into Trip name text field.
Then, put finger at right side of text, and then move your finger to the left.
Notice how you can push the text off the left, beyond the boundaries of the text input field.

cc @mpcomplete @dragostis

I'm seeing similar behavior in the text input demo, only it's worse here.

Thanks for checking.

Update: Didn't check. Thought this was the old issue and someone had deleted my comment. 😆

Checked now. It's only happening in this particular demo.

I think this is a bug in expansion panels, or at least, layout of text inputs inside expansion panels. The Input's layout is not being performed until you edit the text (try typing before scrolling - it scrolls correctly). This means its Scrollable's scrollBehavior doesn't get its proper bounds. In particular, _handlePaintOffsetUpdateNeeded is not being called.

@mpcomplete The problem seems to be the fact that the input is inside a ScrollableViewport. I can reproduce it with this:
new ScrollableViewport(
        child: new Container(
          margin: const EdgeInsets.all(24.0),
          child: new Form(
            child: new Input(
              hintText: 'item.hint',
              labelText: 'item.name',
              value: new InputValue(text: 'item.value'),
              formField: new FormField<String>(
                setter: (String val) { }
              ),
            ),
          ),
        )
      )

One strange discovery: this only repros when the test case @dragostis provided above is run from within the Gallery app (replace the _ExpansionPanelsDemoState builder with the above code). Running it standalone (e.g. with a runApp(new ExpasionPanelsDemo()) call) does not repro the bug.

This is the same as https://github.com/flutter/flutter/issues/6329 . Fixed by https://github.com/flutter/flutter/pull/6328 .


ci:  init semantic-release

:tada: This PR is included in version 0.0.14 :tada:
The release is available on:

npm package (@npm-release dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


RejectedExecutionException in client.dispatcher().enqueue

Hi,
Getting the following exception after computer returns from sleep mode:
Exception in thread "EventThread" java.util.concurrent.RejectedExecutionException: Task okhttp3.RealCall$AsyncCall@17c8d3 rejected from java.util.concurrent.ThreadPoolExecutor@7822a9d2[Shutting down, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 25] 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) 	at okhttp3.Dispatcher.enqueue(Dispatcher.java:132) 	at okhttp3.RealCall.enqueue(RealCall.java:100) 	at okhttp3.internal.ws.RealWebSocket.connect(RealWebSocket.java:192) 	at okhttp3.OkHttpClient.newWebSocket(OkHttpClient.java:438)
Any ideas?
Much obliged.

Anyone ?

Duplicate of https://github.com/square/okhttp/issues/3424


DUPLICATE, please delete! Can't enable SSH access: "Running SSH server is not permitted" #2632

Describe the problem
I tried to enable SSH Access for a few of my peers (Linux & macOS) but it doesn't work
To Reproduce

Enable SSH Access for a peer in dashboard
client.log on the peer shows: WARN client/internal/engine.go:554: running SSH server is not permitted
try to connect connect:

Error: dial tcp <IP_ADDRESS>:44338: i/o timeout
Couldn't connect. Please check the connection status or if the ssh server is enabled on the other peer

Expected behavior

Enable SSH access

Are you using NetBird Cloud?
Self hosted!
NetBird version
0.29.3
NetBird status -dA output:
OS: linux/amd64
Daemon version: 0.29.3
CLI version: 0.29.3
Management: Connected to https://netbird.anon-ZsVFN.domain:33073
Signal: Connected to http://netbird.anon-ZsVFN.domain:10000
Relays:
[stun:netbird.anon-ZsVFN.domain:3478] is Available
[turn:netbird.anon-ZsVFN.domain:3478?transport=udp] is Available
Nameservers:
FQDN: anon-poKed.domain
NetBird IP: <IP_ADDRESS>/16
Interface type: Kernel
Quantum resistance: false
Routes: -
Peers count: 5/11 Connected
Screenshots

Please Delete, thanks.
I'm sorry -  I don't know how I created the same issue twice...


Storing/checking the checksum over Secret contents

The issue is related to Secret support in Instance input parameters, and possible to storing Binding credentials in Secret as well.
Apart from calculating a checksum based on the secret name/key, we also need to hash its contents to correctly handle the case when it gets updated - to propagate these changes further to brokers through OSB and/or trigger reprocessing loop in Service Catalog itself.

@pmorie's comment on the topic from https://github.com/kubernetes-incubator/service-catalog/pull/1028#discussion_r127391333:

I think technically you have to calculate both the checksum based on the spec and the checksum based on the values of the parameters. The tricky part is that you have to calculate the checksum of the parameters based on the value of the parameters you set. I think that implies a separate checksum field that the controller has to set that is based on the parameter values.

Note that if any of these values change due to user update, there's a bug in the controller because we do not implement update :) Someone should go and do that.

I've looked at the origin of adding checksum in the first place, and it seems to be https://github.com/kubernetes-incubator/service-catalog/issues/573 (and pull request https://github.com/kubernetes-incubator/service-catalog/pull/582).

For example, we noticed that using the s3 broker, a lot of buckets get created (the result of a provision) even though a single Instance was submitted and exists in the system.

This bug also manifested in multiple bindings being issued to the broker even though a single Binding resource was created

Both issues are bugs in s3 broker, and it's incorrect implementation of the OSB spec:

Instance duplicate create should be a no-op in broker if parameters have not changed.

200 OK
MUST be returned if the service instance already exists, is fully provisioned, and the requested parameters are identical to the existing service instance.

Binding duplicate create should be a no-op in broker with returning 200 OK if parameters have not changed, according to the spec:

200 OK
MUST be returned if the binding already exists and the requested parameters are identical to the existing binding.

Instance update request should be safe as well, and for the Binding we can invoke create (which could succeed or fail).
Therefore, reconciliation loop should be safe and leads to a no-op on the broker side in both cases, if it implements OSB spec correctly.
Hence the question: "Do we actually need to store the checksum?"
Would like to discuss this at one of the design calls.
/cc @arschles @pmorie @ash2k

We do need the checksum of the spec and probably a separate checksum of the payload we actually sent for a couple of reasons:

We need to be able to distinguish between a provision and an update for instances
We should not continually call provision or bind, even if it is a no-op; this is likely to be disruptive to brokers and to interfere with eventual rate-limiting per-broker
We need to be able to tell when the payload we would send to the broker has changed (so that we can do a parameter update)


fomcon version problem

I have a motor control block only use on matlab 2012a to control it. Is there have a  lowest version to cover the 2012a matlab?

In other words, you need the fod_v2.mdl library for an older version of MATLAB? Find a version for R2012b attached. Hopefully, it will work for you.
fod_v2_R2012b.zip
General comment: unfortunately, Mathworks have never really cared about backwards compatibility. As a developer, it is not feasible for me to keep a stack of MATLAB releases in my development environment. I typically use a MATLAB version that is about 2 years old and gradually upgrade to keep at least +/- 2 years of compatibility with my releases. So, the functionality/compatibility of FOMCON toolbox also follows that trend.


create url/link anchors that filter the team list

On the bottom of the https://www.ockam.io/mission page, there is a roster of whos on the team. When the user selects a group (ie all, full time, board of directors, advisors) in the left menu, then the roster is filtered for that group.
We need to have the following anchors created so that the page scrolls down to the team section, and selects the group associated with that anchor.
/mission**#all**
/mission**#full_time**
/mission**#board**
/mission**#advisors**

Here is PR
https://github.com/build-trust/ockam-website/pull/633

@lukasztucholski when these issues land, can you close the issue?


fix: get_payments where clause does query wrong paymentson certain arguments

i discovered that when testing lndhub, failed payments where still shown is pending. since we introduced the failed state we need now check for success and pending if complete and pending and for the else case, not complete and not pending we should query for failed

This PR requires a unit test.

failing tests here https://github.com/lnbits/lnbits/pull/2844


Multilanguage support

Hi,
First of all, I want to thank you for your work, this bundle saves me a lot of work.
Can you tell me what is the best way to use the bundle for a multilangage blog?
I wanted to put a property $locale in articles, categories and tags entities but if I do that, I have to change repositories. For example ArticleRepository functions to get  articles collection in a given language, and a lot of other things in templates, such as the sortableTh macro, in wich I have to put the locale parameter  for the given route and all the routes parameters for list and create action, because the controller needs to know the locale.
What is the best way to do what I want?
Thank you

@ciro-webcom I have similar problem, can you tell us how did you resolve the problem?

Hi,
I don't remember well, but I think the best way is to fork the repo and add a column locale on articles and categories tables, and change the queries in the repositories including the locale parameter. I also didin't use the original templates, so I didn't had to change the twig files

@ciro-webcom  thanks !


zaupshop, uconomy and zaupshopessencial

Doesen't work since the new version of the game or of rocket.

Update the plugins... they all work on my server in their latest version.