In-memory Engine: entered unreachable code in next_to_match

Bug Report

TiKV panic when preparing 1000 tpcc warehouses in a 4-node cluster.
[2024/06/11 09:06:26.437 +00:00] [FATAL] [lib.rs:477] ["internal error: entered unreachable code"] [backtrace="   0: tikv_util::set_panic_hook::{{closure}}
             at home/stn/tikv/components/tikv_util/src/lib.rs:476:18
   1: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:2029:9
      std::panicking::rust_panic_with_hook
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:783:13
   2: std::panicking::begin_panic_handler::{{closure}}
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:649:13
   3: std::sys_common::backtrace::__rust_end_short_backtrace
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:171:18
   4: rust_begin_unwind
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:645:5
   5: core::panicking::panic_fmt
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panicking.rs:72:14
   6: core::panicking::panic
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panicking.rs:144:5
   7: region_cache_memory_engine::background::next_to_match
             at home/stn/tikv/components/region_cache_memory_engine/src/background.rs:1001:13
   8: <region_cache_memory_engine::background::BackgroundRunner as tikv_util::worker::pool::Runnable>::run::{{closure}}
             at home/stn/tikv/components/region_cache_memory_engine/src/background.rs:1143:33
      yatp::task::future::RawTask<F>::poll
             at home/stn/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/task/future.rs:59:9
   9: yatp::task::future::TaskCell::poll
             at home/stn/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/task/future.rs:103:9
      <yatp::task::future::Runner as yatp::pool::runner::Runner>::handle
             at home/stn/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/task/future.rs:387:20
  10: <tikv_util::yatp_pool::YatpPoolRunner<T> as yatp::pool::runner::Runner>::handle
             at home/stn/tikv/components/tikv_util/src/yatp_pool/mod.rs:199:24
  11: yatp::pool::worker::WorkerThread<T,R>::run
             at home/stn/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/pool/worker.rs:48:13
  12: yatp::pool::builder::LazyBuilder<T>::build::{{closure}}
             at home/stn/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/pool/builder.rs:114:25
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:155:18
  13: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/mod.rs:529:17
      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:272:9
      std::panicking::try::do_call
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:552:40
      std::panicking::try
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:516:19
      std::panic::catch_unwind
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:142:14
      std::thread::Builder::spawn_unchecked_::{{closure}}
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/mod.rs:528:30
      core::ops::function::FnOnce::call_once{{vtable.shim}}
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
  14: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:2015:9
      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:2015:9
      std::sys::unix::thread::Thread::new::thread_start
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys/unix/thread.rs:108:17
  15: start_thread
  16: __clone3
"] [location=components/region_cache_memory_engine/src/background.rs:1001] [thread_name=audit-worker-0] [thread_id=32]

What version of TiKV are you using?

Nightly, built on https://github.com/tikv/tikv/pull/16868 83a419d
What operating system and CPU are you using?

Steps to reproduce

TiKV Config
[range-cache-engine]
  enabled = true
  hard-limit-threshold = "8GB"
  soft-limit-threshold = "5GB"

TPCC commands
/root/.tiup/components/bench/v1.12.0/go-tpc \
    tpcc prepare \
    -H $HOST -P $PORT \
    -D tpcc1000 --warehouses 1000 -T 50

What did you expect?
No such error.

There would be a "next inconsistent, disk iterator next failed" before the panic.

Found a different panic, it's about "audit range".
[2024/06/11 22:09:47.654 +00:00] [INFO] [background.rs:1103] ["audit range"] [range="CacheRange { tag: \"[region_id=162870]\", range_start: 7A7480000000000001FF885F720380000000FF000000D603800000FF0000000004038000FF0000000009610000FD, range_end: 7A7480000000000001FF885F720380000000FF000000EA03800000FF0000000005038000FF000000000DCA0000FD }"] [thread_id=37]
[2024/06/11 22:09:48.090 +00:00] [INFO] [background.rs:1163] ["audit range done"] [range="CacheRange { tag: \"[region_id=162870]\", range_start: 7A7480000000000001FF885F720380000000FF000000D603800000FF0000000004038000FF0000000009610000FD, range_end: 7A7480000000000001FF885F720380000000FF000000EA03800000FF0000000005038000FF000000000DCA0000FD }"] [thread_id=37]
[2024/06/11 22:09:48.090 +00:00] [INFO] [background.rs:1103] ["audit range"] [range="CacheRange { tag: \"[region_id=156798]\", range_start: 7A7480000000000001FF885F720380000000FF000000F403800000FF0000000007038000FF000000000EA20000FD, range_end: 7A7480000000000001FF885F720380000000FF0000010403800000FF0000000003038000FF000000000AFD0000FD }"] [thread_id=37]
[2024/06/11 22:09:48.090 +00:00] [ERROR] [background.rs:1123] ["seek_to_first result not equal"] [cf="\"lock\""] [seqno=20939126398] [cache_key=7A7480000000000001FF885F720380000000FF000000F503800000FF0000000004038000FF00000000139D0000FD] [upper=7A7480000000000001FF885F720380000000FF0000010403800000FF0000000003038000FF000000000AFD0000FD] [lower=7A7480000000000001FF885F720380000000FF000000F403800000FF0000000007038000FF000000000EA20000FD] [thread_id=37]
[2024/06/11 22:09:48.127 +00:00] [FATAL] [lib.rs:477] ["internal error: entered unreachable code"] [backtrace="   0: tikv_util::set_panic_hook::{{closure}}
             at home/stn/tikv/components/tikv_util/src/lib.rs:476:18
   1: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:2029:9
      std::panicking::rust_panic_with_hook
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:783:13
   2: std::panicking::begin_panic_handler::{{closure}}
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:649:13
   3: std::sys_common::backtrace::__rust_end_short_backtrace
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:171:18
   4: rust_begin_unwind
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:645:5
   5: core::panicking::panic_fmt
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panicking.rs:72:14
   6: core::panicking::panic
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panicking.rs:144:5
   7: <region_cache_memory_engine::background::BackgroundRunner as tikv_util::worker::pool::Runnable>::run::{{closure}}
             at home/stn/tikv/components/region_cache_memory_engine/src/background.rs:1131:33
      yatp::task::future::RawTask<F>::poll
             at home/stn/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/task/future.rs:59:9
   8: yatp::task::future::TaskCell::poll
             at home/stn/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/task/future.rs:103:9
      <yatp::task::future::Runner as yatp::pool::runner::Runner>::handle
             at home/stn/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/task/future.rs:387:20
   9: <tikv_util::yatp_pool::YatpPoolRunner<T> as yatp::pool::runner::Runner>::handle
             at home/stn/tikv/components/tikv_util/src/yatp_pool/mod.rs:199:24
  10: yatp::pool::worker::WorkerThread<T,R>::run
             at home/stn/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/pool/worker.rs:48:13
  11: yatp::pool::builder::LazyBuilder<T>::build::{{closure}}
             at home/stn/.cargo/git/checkouts/yatp-e704b73c3ee279b6/793be4d/src/pool/builder.rs:114:25
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:155:18
  12: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/mod.rs:529:17
      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:272:9
      std::panicking::try::do_call
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:552:40
      std::panicking::try
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:516:19
   std::panic::catch_unwind
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:142:14
      std::thread::Builder::spawn_unchecked_::{{closure}}
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/mod.rs:528:30
      core::ops::function::FnOnce::call_once{{vtable.shim}}
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
  13: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:2015:9
      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:2015:9
      std::sys::unix::thread::Thread::new::thread_start
             at home/stn/.rustup/toolchains/nightly-2023-12-28-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys/unix/thread.rs:108:17
  14: start_thread
  15: __clone3
"] [location=components/region_cache_memory_engine/src/background.rs:1131] [thread_name=audit-worker-0] [thread_id=37]

al error: entered unreachable code"] [backtrace="   0: tikv_util::set_panic_hook::{{closure}}
at home/stn/tikv/

It's in prepare or normal run?

What's the status on these? should the two panics be separate issues?

What's the status on these? should the two panics be separate issues?

We suspect that they share by the same root cause. The unreachable panic means it detects data inconsistency between in-memory engine and rocksdb.
We think they might be caused by DROP TABLE as in-memory engine does not handle UnsafeDestroyRange (for GC dropped table) properly.
https://github.com/tikv/tikv/blob/8903f2a5d9b7b1c7d9f6ab453accab90a5088bf3/src/server/gc_worker/gc_worker.rs#L1023-L1040

Since the first report, we've done several refactors, and the issue hasn't been reproduced for months.


Add emaillabs.io

Hi, thanks for the contribution.
Could you please add the service in alphabetical order and also the link in the table of contents ?
And do not forget to add a link to the main table of contents in readme.md.
Thank you !

@255kb , fixed. Do you can review?

perfect thanks 👍


help(Snackbar): Change z-index of snackbar without affecting Dialogs

Hi, I need to change the z-index of the snackbar but im not able to do it.
I found that if I have the following class in styles.scss, I can indeed change the z-index but it also affects the Material Dialogs.
.cdk-overlay-container {
z-index:<PHONE_NUMBER>9999;
}
Any way I can only affect the Snackbars?
Thanks in advanced!

You can pass a panelClass to the snack bar config and only target that class with your z-index.

You can pass a panelClass to the snack bar config and only target that class with your z-index.

I have already tried that, but the z-index affects the content of the snackbar, not the container

I see. Usually when we have a panelClass option it sets the class on the overlay wrapper, but for some reason the one of the snack bar sets it on the container.

I'm facing the same problem with MatDialog and MatSelect. I want to reduce the z-index of MatDialog but keep the MatSelect one intact. Is that possible, even with a workaround? I'm stucked at this for a long :cry:

I am dealing with the same issue. MatDialog has a backdropClass property that allow us to fiddle with the outer elements. But Snackbar doesn't.

Has anyone already found a solution to this?!

I found a workaround by using CSS's :has
.cdk-global-overlay-wrapper:has(> .cdk-overlay-pane > mat-snack-bar-container) {
  z-index: 1200;
}


only works on last added element

Seems that only the last element I've added reacts to the shortcut. I added 2 elements first and tested it and only second element reacted to the shortcut, then I added the third element and only that element reacted to the shortcut.
Is this the way it should be or am I doing something strange with my components :)
I am using this inside file component which I then use to create elements with v-for.
I am also using vuex, not sure if that could somehow interfere with shortcut directive?
Here is how i am using it:
<div v-shortkey.once="['ctrl', 'alt', 'o']" @shortkey="showHeader=!showHeader" v-show="showHeader" class="content-block-header">

Basically, I want to hide the header div of every component that I've added to the list and that is being displayed on the page.

@iwebnaut Could you please give us a reproductible test case, using http://jsfiddle.net/ for instance?

Closing as there is no details.

Add two v-shortkey maps on a page, only one will work.


Initial support for DWARF parsing (listing uprobe args)

This PR introduces support for parsing DWARF information from traced binaries. More on this topic can be found in #1901. I didn't implement all the mentioned features at once, rather I wanted to create an initial version of a DWARF parser to see if this is a correct way to go. Therefore, this PR only introduces a very basic feature - listing uprobe arguments (along with their types).
For example:
# bpftrace -lv 'uprobe:tests/testprogs/uprobe_test:function*'
uprobe:tests/testprogs/uprobe_test:function1
    int* n
uprobe:tests/testprogs/uprobe_test:function2
    struct Foo* foo1
    struct Foo* foo2

The parsing is implemented on top of the libdw library from elfutils. If the library is not present on the system, bpftrace will still compile, only some features won't work (similarly to when BTF is not available).
So far, there are only runtime tests as I haven't come with a reasonable way to mock DWARF data for GTest, yet.
I'd prefer merging this as it is and then start implementing various features on the top of it as separate PRs. If you feel that this PR should already provide more functionality, let me know.
Checklist

[ ] Language changes are updated in man/adoc/bpftrace.adoc and if needed in docs/reference_guide.md
[ ] User-visible and non-trivial changes updated in CHANGELOG.md
[x] The new behaviour is covered by tests

Looks like a great start! I don't know much about DWARF so I only skimmed those bits. Seemed reasonable to me. The only thought I had was if the code should use "???" or something for all unknown data. Given that DWARF supports many languages, I wouldn't be surprised if users bumped up against all kinds of weird stuff. Would be nice to have uniform "unknown" markers.

I used <unknown type> here, not sure if we deal with potentially unknown data somewhere else. If you prefer using "???" instead, I have no problem with it.

And one other thing: I think libdw-dev should be added to all the dockerfiles (for CI consistency).

Right, I added it to Dockerfiles for Ubuntu and Fedora (no need to add it to Alpine, see below).
There's a problem with static builds though. Linking against static libdw is not recommended (see here) as it requires also linking against libebl.a. That one is usually included in the same package, so it should be doable, but I wanted to check your opinion first - do you prefer including libdw in static builds even for the price of having to add one more library to CMake?

BTW, wasn't there mention of libdw being included in elfutils? So I thought we didn't need to install more packages.

Yeah, it's part of elfutils, but some distributions (Fedora, Ubuntu) ship in separately while others (Alpine) ship all elfutils in a single package.

I used  here, not sure if we deal with potentially unknown data somewhere else. If you prefer using "???" instead, I have no problem with it.

Let's keep <unknown type> for now -- better to be verbose I guess.


KeyError during AssessCheckm2Bins

PB-HiFi-Mag-Pipeline
I am running some Hifiasm-meta assemblies through the pipeline, and when I get to AssessCheckm2Bins, I seem to get a KeyError for one of the contig names. I have put hifiasm-meta assemblies through before and have not had this problem, so it may be file specific.
~/HiFi-MAG-Pipeline_V2$ cat logs/7283_metagenome_hifiasm.AssessCheckm2Bins.log
make_checkm_df: Making checkm2 dataframe.
make_depth_dict: Making depth dictionary.
add_contig_numbers_and_status: Adding contig numbers and assessing Pass/Fail filtering status.
Traceback (most recent call last):
File "scripts/Filter-Checkm2-Bins.py", line 158, in
main()
File "scripts/Filter-Checkm2-Bins.py", line 151, in main
add_contig_numbers_and_status(df, depth_dict, args.bin_dir, args.min_completeness, args.max_contamination, args.max_contigs)
File "scripts/Filter-Checkm2-Bins.py", line 86, in add_contig_numbers_and_status
depths.append(", ".join([depth_dict[rec.id] for rec in SeqIO.parse(filename, 'fasta')]))
File "scripts/Filter-Checkm2-Bins.py", line 86, in
depths.append(", ".join([depth_dict[rec.id] for rec in SeqIO.parse(filename, 'fasta')]))
KeyError: 's3.ctg000004l'

Hi @Htrivett,
Thanks for your patience, we've started our summer conference season and have been traveling.
I have not encountered this issue. During this step, the contigs within each bin fasta file are being looked at. There is a step immediately before that uses the same logic, so it is puzzling as to why this error occurs here (and not in previous calls).
If you try deleting the checkm2 results for this step, can you re-run without an error?

Hi @Htrivett,
Do you have any updates on this issue?

Hi @dportik
I ended up reassembling again and going through the pipeline again. Seemed to be a file error during assembly step.
Thanks


[feature] Table -> InlineTable

Hello! I'm trying to turn
[site]
bucket = ""
entry-point = "workers-site"

into an inline table using toml_edit. Oddly enough, when I parse my toml, I get
Table(Table { items: {"bucket": TableKeyValue { key: Repr { decor: Decor { prefix: "", suffix: " " }, raw_value: "bucket" }, value: Value(String(Formatted { value: "", repr: Repr { decor: Decor { prefix: " ", suffix: "" }, raw_value: "\"\"" } })) }, "entry-point": TableKeyValue { key: Repr { decor: Decor { prefix: "", suffix: " " }, raw_value: "entry-point" }, value: Value(String(Formatted { value: "workers-site", repr: Repr { decor: Decor { prefix: " ", suffix: "" }, raw_value: "\"workers-site\"" } })) }}, decor: Decor { prefix: "\n", suffix: "" }, implicit: false, position: Some(1) })

Type Table(Table... was not I expected to see above, and I suspect it's why calling .as_inline_table() fails on the table provided above. (instead, calling .as_inline_table() returns None).
Any pieces of advice? :)

hey @gabbifish! This is the expected output. Inline tables are something different: https://github.com/toml-lang/toml#user-content-inline-table.
For example,
site = { bucket = "", entry-point = "workers-site" }

This would be an equivalent inline table in your case. Does that make sense?

Ah, I see you're trying to convert a Table into an InlineTable and expecting as_inline_table to do the conversion. But that's not what it does. It performs a dynamic check and returns Some(InlineTable) if it's actually is an InlineTable.
Currently, the functionality you asking is missing: https://github.com/ordian/toml_edit/blob/b679b5e610b9a19f48fa0268f9174062e9c43962/src/table.rs#L8
PRs are welcome :)

Here is my snippet which converts Table into InlineTable:
fn make_inline_tbls(paths_arr: &mut ArrayOfTables) {
    for x in (0..paths_arr.len()) {
        let sub = paths_arr.get_mut(x).expect("");
        let tgt_raw = sub.entry("target");

        tgt_raw.as_inline_table_mut().map(|e| e.fmt());
        let tgt_table = tgt_raw.as_table().expect("");
        let mut tgt_inline = InlineTable::default();
        tgt_table.iter().for_each(|i| {
            let key = i.0;
            let value = i.1.as_value().expect("").to_owned();
            tgt_inline.get_or_insert(key, value);
        });
        tgt_inline.fmt();

        let mut tgt_val = Value::InlineTable(tgt_inline);
        let tgt_item = Item::Value(tgt_val);
        *tgt_raw = tgt_item;

    }
}

I'm not a rustacean, so, probably, this is a piece of crap, though it works. Would really like to see a built-in API.

The next release includes a item.into_value() which will turn a table into an inline table.  We don't have an in-place version yet, so you'll have to remove, convert, and insert.


BsReactNative.AnimatedRe.calculated

Was updating my project to the latest commit and ran into problems with the new type AnimatedRe.calculated. I don't get why the change to AnimatedRe.calculated and AnimatedRe.regular, since React native just worries about if the variable is animated or not?
I would like a way to change a AnimatedRe.calculated to AnimatedRe.regular or have this as the same type.

Unfortunately, it is not the same type and it's never been. In the past, there used to be Value.t and Interpolation.t.
We changed it to use phantom types to better align with Javascript API of React Native.
Please provide a sample of what cannot be achieved with current API and we will see whether it's our limitation or something that shouldn't be possible due to React Native API.
Here's some old discussion: https://github.com/reasonml-community/bs-react-native/issues/167

Here is an example:
Transform.makeAnimated(
  ~translateY=
    AnimatedRe.Value.add(
      state.floatingPageControlTopPosition,
      AnimatedRe.Value.multiply(
        Animated.Value.create(-1.),
        state.verticalScrollViewYPosition,
      ),
    ),
  (),
),

In this example both state.floatingPageControlTopPosition and state.verticalScrollViewYPosition are of type BsReactNative.AnimatedRe.Value.t.
This was working before the update to animatedRe in bs-react-native.
The error I get is that it is expecting AnimatedRe.regular but the add function returns Animated.calculated.

Thanks for reporting. We forgot to update the Transform module to accept new API. Try using Transform.makeInterpolated for now to accept value(calculated).

@Rikkiabb AFAIK, the interface hasn't changed here yet and it's consistent with the behaviour before @grabbou's change. Anyway, if you'd like to contribute, there's an easy fix:
Replace Animated.Value.t in Transform.re with Animated.value('a). Thanks to that, Transform.makeAnimated will accept both interpolated and normal values.


Trying To Copy an empty string doesn't trigger success or failure

Awesome project!!
I just spent some time debugging your code because the success trigger, not the error trigger was firing when the text property return an empty string.  I didn't realise the problem was the empty string (duh), but letting the success or failure trigger fire in this case might be nice to help users debug their code.

Thanks for reporting @pavlism! Feel free to send a pull request ;)


  I'm not convinced that 'token' is actually the right name for this at all (bikeshedding welcome).
We can probably also remove token_sealed_unsealed_alloc.  This has to return one result indirectly and so introduces some pain in the ABI.  It was added because token_obj_unseal was a second cross-compartment call, but now it is a (fast) call into the token library and so it may be simpler to just return the sealed object and rely on the caller to unseal it.

Whatever names we end up with, we should probably also rename SKeyStruct/SObjStruct/SKey/SObj.


Implement memory util

@jmcook1186 I have tuned both macos and linux memory utilization scripts. linux one can be checked with sar -r 1 2


Finding ipynb-meta files fails if filename contains '.'

File discovery will fail when .ipynb-meta files contains . in the name.
I think the fix is to replace filename.split('.')[0] with os.path.splitext(filename)[0]
I was having issued when the file was named "04.09.09.post.ipynb", but it works after this fix.

Thanks!


[documentation] Add scaling tests

How does pympipool compare to ThreadPoolExecutor and ProcessPoolExecutor ? It would be great to design benchmark studies. In addition, pympipool can be combined with the ProcessPoolExecutor when specifying the number of threads and running one pympipool process per node.

This topic was initially brought up by @ligerzero-ai in https://github.com/orgs/pyiron/discussions/211#discussioncomment-8034046

The benchmark was added in https://github.com/pyiron/pympipool/pull/264


How to get binary data instead of writing to filesystem

Hello, I would like to store very small resized images to a DB as binary data. What can I do?

Hello,
You can use the Encode function (imaging.Encode or jpeg.Encode or png.Encode etc) to write the resized image to a bytes.Buffer and store buffer.Bytes() to the database.


Luis.ai picks up unwanted entities

I have an intent that I specify 2 required action parameters for (neither of which is a datetime). When I try an utterance such as this: "post to biology looking for a lab partner this week - any takers?" I need Biology to be a class entity (works fine) and the "looking for a lab partner this week - any takers?" to be a message entity.
The message entity gets thrown off by the "this week" part it seems. "this week" is highlighted as a datetime entity, but I never expect a datetime entity with this intent -- that's why I specified the above mentioned action parameters. Is the purpose of action parameters not to limit the entities that get picked up? If not, what is it for?

This feature is meant to help you collect the important information related to this intent. But is doesn't limit the entities that get predicted. Anyway this feature is getting deprecated so please don't take dependency on it.


bs and bs.obj ppx apply deeply

When using [@bs] to uncurry a function it is applied deeply. Probably it's better to apply it only on the most shallow level:
[@bs] uncurriedFun({
  normalFun(1)
});

In this case the output of the ppx is that normalFun is also called uncurried.
A similar thing happens with<EMAIL_ADDRESS>All fields (even if it's part of the field expression) are converted to objects.
example:
[%bs.obj]
{
  test: {actually_a_record: 1}
}

cc @jfrolich
moved from https://github.com/BuckleScript/bucklescript/issues/4306

note the bs attribute looks good to me.
let u =(.f0,f1)=>
f0(. {
  f1(1)
});

The inferred type is
let u: (. (. 'a) => 'b, int => 'a) => 'b;


RFC: utility macros to minimize boilerplate

Hello,
I made a utility crate for rstml-component that helps me with making sites using rstml-component, for now its only on github (not yet on crates.io) because i think it could become part of rstml-component with a feature flag for example.
what it does
its an attribute proc macro that turns functions into components (an example of what the macro expansion might look like in the readme)
Additionally if it were to go upstream i could also make another macro that eliminates write_html!(formatter, ...) while keeping your ide helpers. Making a macro for it tricky and would lead to a lot of code duplication (same code in rstml-component-macro and my crate)
Would you be interested in heaving the utility crate be upstreamed into here or should i just publish it to crates.io  as a separate crate?
I'm willing to implement things from my crate into here and making a pr, i just don't want to waste time integrating it and then getting rejected in the pr.

That's kinda funny, cause rstml-component initially started with the following syntax:
rstml_component! {
  pub struct MyComp(prop_a: String, prop_b: impl HtmlContent) {
    <div />
  }
}

The issue I had was how to deal with branches and adding "normal" rust in top of the function, before you start with the html.
I believe most of the code to make this should already be available in this crate, and I'm definitely open to expanding it to make it more user friendly :).
What I don't really like about your syntax is the fact that make a function, but end up with a struct (with a different name no less). I find that somewhat surprising that if I write a function fn foo(...) {}, it doesn't exist, and can't be called just because you declared it an component. A better solution for this (I think) is to keep the function and also generate the struct. That means one of the impls can just call the other.

What I don't really like about your syntax is the fact that make a function, but end up with a struct (with a different name no less).

The reason why i made it that way is because of lint errors. Leptos uses the function ident and then it complains about non snake case functions. But i can see how that would be somewhat unintuitive so ill try and figure out a better solution.

I'm definitely open to expanding it to make it more user friendly :).

Awesome, ill get started on integrating it and making a pr.


Fix Final Video Challenge Completion

Pre-Submission Checklist

[x] Your pull request targets the staging branch of FreeCodeCamp.
[x] Branch starts with either fix/, feature/, or translate/ (e.g. fix/signin-issue)
[x] You have only one commit (if not, squash them into one commit).
[x] All new and existing tests pass the command npm test. Use git commit --amend to amend any fixes.

Type of Change

[x] Small bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds new functionality)
[ ] Breaking change (fix or feature that would change existing functionality)
[ ] Add new translation (feature adding new translations)

Checklist:

[x] Tested changes locally.
[x] Closes currently open issue (replace XXXX with an issue no): Closes #10089

Description

Redirect camper to /map if the next challenge is locked.

Tested and LGTM 👍 . Is there a way to keep the toast displayed until it's exited? It disappears pretty swiftly.

@dhcodes this is not possible with react-notification at the minute, I tried playing with v6.3.0 which should allow it, but I could not get it to work. I can extend the dismissAfter prop to something like 10s, it is currently at 6s.

Works for me. Much better to have the extended time for the toast. Sorry for taking so long to get back to this.


HIVE-27862: Map propertyContent to a wrong column in package.jdo

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

Is the change a dependency upgrade?

How was this patch tested?

Ideally the column name in METASTORE_DB_PROPERTIES should have been PROPERTY_CONTENT like other columns, but I think we have to live with it now, due to compat issues...
There is no test which was checking the output?

There are some tests HMSDirectTest which are testing out the fetch or save of  PROPERTY_CONTENT on derby, but they get through the test, not sure why.

hmm, do we need to catch someone from HMS side to figure out why those tests aren't working? I think it isn't an ideal situation for us, these bugs will keep on coming if those tests aren't working the way they should


Add basic authentication support for http4s

This is an idea, how authentication could be implemented. It's enabled automatically based on presents of global securitySchemes and at least one security requirement for a handler's route. Check sample-http4s/authentication for an example of generated code and Http4sAuthenticationTest for a possible usage example

Please pardon the initial silence here -- I didn't have the time this weekend, I'll try to get to it sometime this week

If we are to attempt to actually implement authentication in a general way, I think it is important that we follow the spec, as avoiding generated API churn is an explicit goal of this project.
While you are correct, supplying an additional type parameter is likely the correct direction to go, the similarities between authenticationMiddleware and mapRequest is so great, it is actually possible to implement this feature without a change to guardrail at all by changing the supplied F from IO to Kleisli[IO, Option[AuthContext], *].
As I commented elsewhere in this review, I think the primary concerns that need to be addressed are:

Instead of enabling/disabling authentication via Boolean, we need to explicitly thread through which kind of authentication is supplied, per route, with failover

If at all possible, source compatibility with http4s' existing authentication middleware would be very useful, as it would permit code reuse between existing and new implementations, as well as compatibility with already-published http4s middleware providers.

If this also means switching from HttpRoutes.of... to AuthedRoutes.of and composing the two implementations together via <+>, that may help keep reimplementation down, and utilize some of the infrastructure already provided to us by http4s.

Thank you for your initial effort here, it's great to see interest for first-class authentication support in guardrail, at Twilio this was implemented in a bespoke way for our Dropwizard generators, targeting internal Twilio authentication provider libraries. What are your thoughts to my above observations?

Thank you for the review, I‘ll come back after the holidays

If we are to attempt to actually implement authentication in a general way, I think it is important that we follow the spec, as avoiding generated API churn is an explicit goal of this project.
While you are correct, supplying an additional type parameter is likely the correct direction to go, the similarities between authenticationMiddleware and mapRequest is so great, it is actually possible to implement this feature without a change to guardrail at all by changing the supplied F from IO to Kleisli[IO, Option[AuthContext], *].

yes, it is similar, but it's not the same. mapRequest applies to all routes and to distinguish them a user needs to use String argument which a source of bugs. Using Option[AuthContext] doesn't guarantee a user has Some on routes, which were authenticated and has the context, this could lead having .get in user code (leaks authentication info to user code, but we should keep it on our side). Moreover, mapRequest doesn't have auth kinds and scopes information.

As I commented elsewhere in this review, I think the primary concerns that need to be addressed are:

Instead of enabling/disabling authentication via Boolean, we need to explicitly thread through which kind of authentication is supplied, per route, with failover

I agree, but we don't need to use securitySchemes for that, because any OAS route contains a list of auth kinds with scopes.

If at all possible, source compatibility with http4s' existing authentication middleware would be very useful, as it would permit code reuse between existing and new implementations, as well as compatibility with already-published http4s middleware providers.

I think it's possible to reuse, but it's better not to, because the http4s auth works on completely another level and doesn't have an information about auth kinds and scopes. Probably it could be implemented grouping routes but routes could have several kinds of auth at the same time. It would be hard to group such routes:

route A with auth kinds [basic]
route B with auth kinds [basic, oauth]
route C with auth kinds [oauth]
we could have different groups for routes A and C to authenticate them, but it's very hacky to wrap route B correctly reusing middlewares for basic and oauth, the only way I see is to implement another middleware which combines them special way.

I've got an idea while writing this comment, I'll try to experiment with it. ☺

If this also means switching from HttpRoutes.of... to AuthedRoutes.of and composing the two implementations together via <+>, that may help keep reimplementation down, and utilize some of the infrastructure already provided to us by http4s.

I wouldn't move it to a user code, maybe we can do it internally if it helps

Thank you for your initial effort here, it's great to see interest for first-class authentication support in guardrail, at Twilio this was implemented in a bespoke way for our Dropwizard generators, targeting internal Twilio authentication provider libraries. What are your thoughts to my above observations?

Thank you for the review, I'll try to push it forward whenever I have time

Using Option[AuthContext] doesn't guarantee a user has Some on routes, which were authenticated and has the context, this could lead having .get in user code

When I've done this in the past, I've done something along the lines of
ctx.fold(respond.NotAuthorized) { auth =>
  respond.Ok(auth.username)
}

though I appreciate the observation that this isn't always obvious to the user. It does raise a consideration though, which is that if the route only specifies 200/404, presumably auth failures would introduce an otherwise unspecified response code with an unspecified schema. Perhaps a UserError if we intend to fail with an error code that isn't mentioned in the specification (though that doesn't solve the body problem, I think it's a step in the right direction).

I agree, but we don't need to use securitySchemes for that, because any OAS route contains a list of auth kinds with scopes.

Good -- So long as we agree that Boolean is suboptimal, I don't have an opinion on how it is implemented, so long as it's implemented in a flexible and clear way. Looking at the spec, it seemed as though there was at best a loose correlation between the defined schemes at the root of the specification and the usage of those schemes in individual routes.
One of the objections I have to Boolean is that it removes our ability to verify and offer detailed error messages if we see something that doesn't make sense, to guide the user towards a resolution to their issue. If we agree, and you take that into account in your implementation, then I'm comfortable moving forward with it 😁

I wouldn't move it to a user code, maybe we can do it internally if it helps

I agree with this -- any careless modification could change fully functional production code to only partially implemented code without any compiler errors, which would be a huge detriment to users. Thanks for calling it out!

Hi again! I've found some time to get back.
I've experimented a bit with http4s's middlewares without success. It's made for another use case and doesn't fit into OAS definition of security handling. The main problem is that OAS allows to define pretty complex authentication checks like:
security:    # (A AND B) OR (C AND D)
  - A
    B
  - C
    D

Another potential problem here is caching, for example in case (A AND B) OR (A AND C) we probably want to cache A result and, if B fails, reuse it to continue evaluating directly to C. Also http4s's middlewares hardcode response types and don't expose their decision to user code which makes inconsistency with the scheme (as you pointed on your comment).
So, in the latest implementation all decisions should be made on the used code side: it decides what to do with security schemes and scopes, implements caching and whatever optimisations make sense (like always evaluate basic before oauth). As a result it returns an Option (maybe we should replace it by Either to be able to provide more details about an error) which is passed directly to the handler and should be proceed on the used code side to chose correct response type and generate valid response body.

Playing around a little bit more, in my example I've discovered that
schemes.collectFirstSomeM(...)

and
lookupAuthedScopes(...)

almost mesh with the type of AuthMiddleware from http4s, which would be a good entrypoint for reuse of existing libraries, or for users trying to migrate existing codebases.
If Kleisli[OptionT[F, *], Request[F], A] can be escorted carefully through NonEmptyList[NonEmptyMap[Scheme, Scopes]], there may be a good amount of code reuse and interoperation with the ecosystem there.

This looks really good. I did a few stylistic tweaks:

avoiding unsafe* methods
preferring A => A function composition instead of maintaining structurally similar branches
reducing case (Some(_), Some(_)) into a single Option[(A, B)]

The one sticking point for me is the ergonomics around authMiddleware itself. Looking at a fairly common case of being able to answer a challenge via one or more A => IO[B] functions, as a user I'd expect to be able to (straightforwardly):

Attempt to execute auth challenges in parallel
Collect and enforce the "and" and "or" constraints

Current implementation moves responsibility of processing "and" and "or" to the user side which increases complexity and reduces ergonomics, but from the other hand it gives user more freedom to optimise the check. Do you think it would be better to move parallelisation and running logic to the guardrail side and only request from the user a bunch of authentication functions ((List[String], Request[IO]) => IO[Option[B]])?

I'm sure I've gotten lost somewhere here, but I tried to do a mock implementation that would match the type signature. The following is what I ended up with:
  def lookupAuthedScopes(req: Request[IO], scopes: List[String]): IO[List[String]] =
    IO.pure(
      req.headers.get(CIString("dummy-scopes"))
        .map(_.head.value.split(",").intersect(scopes))
        .toList
        .flatten
    )

  def authMiddleware(schemes: NonEmptyList[NonEmptyMap[Resource.AuthSchemes, List[String]]], req: Request[IO]): IO[Option[List[String]]] = {
    println("Schemes:")
    schemes.toList.foreach(_.toSortedMap.foreach({ case (scheme, scopes) => println(s"  ${scheme.name} -> ${scopes.mkString(", ")}") }))
    schemes.collectFirstSomeM[IO, List[String]](_.toSortedMap.toList.flatTraverse({
      case (Resource.AuthSchemes.Basic1, scopes) =>
        lookupAuthedScopes(req, scopes)
      case (Resource.AuthSchemes.Basic2, scopes) =>
        lookupAuthedScopes(req, scopes)
    }).map(Option(_).filter(_.nonEmpty)))
  }

This additionally relies on the implementation agreeing with what the specification says in #!/components/securitySchemes, everything's kinda open-ended in that regard.
Do you have any guidance of how you were intending these functions be implemented? The tests are extremely narrow and immediately move away from the NonEmptyList/NonEmptyMap types, in favor of direct equality against known test values, which are not sufficient inspiration for how a user may adapt this to their environment.

I've reimplemented your example to how I see it could work in a real case:
    final case class User(id: Long, name: String, scopes: List[String])
    def getUserFromDB(id: Long, password: String): IO[Option[User]] =
      IO.pure(Some(User(id, "John Smith", List("admin"))))

    def base64Decode(in: String): Option[String] = ???

    def extractCredentials(req: Request[IO]): Option[(Long, String)] =
      req.headers
        .get[Authorization]
        .collect { case Authorization(Credentials.Token(AuthScheme.Basic, token)) => token }
        .flatMap(base64Decode)
        .map(_.split(":").toList)
        .collect { case id :: password :: Nil => (id.toLong, password) } // unsafe throw

    def authMiddleware(schemes: NonEmptyList[NonEmptyMap[Resource.AuthSchemes, List[String]]], req: Request[IO]): IO[Option[User]] = {
      println("Schemes:")
      schemes.toList.foreach(_.toSortedMap.foreach({ case (scheme, scopes) => println(s"  ${scheme.name} -> ${scopes.mkString(", ")}") }))
      schemes
        .collectFirstSomeM[IO, User](
          _.toSortedMap.toList
            .map {
              case (Resource.AuthSchemes.Basic1, scopes) =>
                extractCredentials(req).flatTraverse { case (id, pass) => getUserFromDB(id, pass).map(_.filter(_.scopes.toSet == scopes.toSet)) }
              case (Resource.AuthSchemes.Basic2, scopes) =>
                extractCredentials(req).flatTraverse { case (id, pass) => getUserFromDB(id, pass).map(_.filter(_.scopes.toSet == scopes.toSet)) }
            }
            .reduce[IO[Option[User]]] {
              case (l, r) =>
                l.flatMap {
                  case Some(_)  => r
                  case None       => IO.pure(None)
                }
            }
        )
    }

Some notes after writing it:

it doesn't support parallelisation
it doesn't support caching: it will invoke getUserFromDB twice
it doesn't check that in the first check user id is the same as in the second (but in a real case different checks could use different user sources with different id's)
I think scopes should be Set instead of List
I think we should return Either[AuthError, A] instead of Option[A] to be able to distinguish different errors on the handler (in the user's code) and map them to different responses. It would be also helpful for logging and debugging. If it's redundant for a user, it's always easy to convert an Either to an Option on the user side.

While I wrote this I've realised that the implementation looks a bit complicated, especially when a real use case has only one authentication type. I would suggest to introduce a config parameter like "custom authentication" which would enable current 'complicated' implementation for a complicated case (with "and"s and "or"s, parallelisation requirement and etc.). But replace the current implementation by simplified (from user point of view) one: the code you wrote in the authMiddleware we can execute on the guardrail side and require from a user only a list of functions (List[String], Request[IO]) => IO[Option[B]] depending how many authentication types are defined in a spec.
Even in the simplified case the http4s AuthMiddleware wouldn't be applicable because it cuts out the user code returning hardcoded error responses directly from it, which makes it harder to customise bodies and could create an inconsistency with a spec.

Current implementation moves responsibility of processing "and" and "or" to the user side which increases complexity and reduces ergonomics, but from the other hand it gives user more freedom to optimise the check. Do you think it would be better [...]

to move parallelisation and running logic to the guardrail side [...]

Enforcing parallelized calls against some user code seems like it would be easier for the user to implement, which is good, at the expense of some (possibly nonexistent?) case where a user does not want that parallelism.

and only request from the user a bunch of authentication functions ((List[String], Request[IO]) => IO[Option[B]])?

I wonder if a better strategy wouldn't be to offer some guardrail-auth library that conforms to the expected type you've specified here, with adapters to known-working implementations of various authentication schemes (I really like bare-bones-digest, as an example. Zero dependency digest auth implementation).

As for ways to improve, I'm interested in your thought about what the user experience of Either[E, A] instead of Option[A] for the auth context would look like, I think it's also useful to communicate No Auth vs Failed Auth states.
I was experimenting with whether I liked the idea of reducing the NEL[NEM[Scheme, Scopes]] => F[A] to just (Scheme, Scopes) => F[A], and while it does make the code simpler (see the example at the end of this post), it would make guardrail-auth much more cumbersome to manage for the user (choosing different objects or methods that would otherwise be nearly identical but for the number of elements in a list in a specification file).
Thank you for the dialogue around what end-user code would look like, your implementation clears up a fair amount.

Idea for what the singleton scheme code may look like, though to reiterate, I don't believe this is worth the cost of an explosion of complexity in the to-be-designed guardrail-auth library.
    final case class User(id: Long, name: String, scopes: Set[String])
    def getUserFromDB(id: Long, password: String): IO[Option[User]] =
      IO.pure(Some(User(id, "John Smith", Set("admin"))))

    def base64Decode(in: String): Option[String] = ???

    def extractCredentials(req: Request[IO]): Option[(Long, String)] =
      req.headers
        .get[Authorization]
        .collect { case Authorization(Credentials.Token(AuthScheme.Basic, token)) => token }
        .flatMap(base64Decode)
        .map(_.split(":").toList)
        .collect { case id :: password :: Nil => (id.toLong, password) } // unsafe throw

    val authMiddleware: ((Resource.AuthSchemes, Set[String]), Request[IO]) => IO[Option[User]] = {
      case ((Resource.AuthSchemes.Basic, scopes), req) =>
        extractCredentials(req)
          .flatTraverse { case (id, pass) =>
            getUserFromDB(id, pass)
              .map(_.filter(u => scopes.diff(u.scopes).isEmpty)) // If the user contains at least the scopes requested
          }
    }

I like the latest code example, I think it has enough simplicity and we should consider it as the main solution. For users who have special cases we can provide NEL[NEM[Scheme, Scopes]] => F[A] option through config. Also I'm thinking about a possibility to completely switch off the authentication on the guardrail side (at least to be compatible with the old implementations and don't force users to migrate). It's a good idea to provide a guardrail-auth library, but I think it should be an addition not a mandatory part.
The next steps I'll apply:

replace Option[A] result by Either[E, A]
introduce a config parameter with disable, simple and custom options considering the current implementation (with NEL and NEM) as a custom
implement simple one based on the latest code example

I just merged #1359, which will likely resolve the code coverage issue you're running into in this PR. I'm fine ignoring it for now, since we've been dancing around our coverage target for a while, pushed into the red by the http4s 0.22 and 0.23 tests being split in two until we drop 0.22 support. Feel free to rebase off master if you want to see green, there should be no merge conflicts.

Hi again, I'm still here)
It seems I've finalised the whole change:

Option result has been replaced by Either, but only for simple authentication. For custom authentication we just do nothing 😜 because we move the responsibility to a user and the user could define any model he/she wants through AuthContext, we just pass it from an authentication middleware to a handler.
x-security-optional type has been changed to boolean (it wasn't used anyway) to use it as a workaround for the "Optional Oauth2 Security". The underlying swagger library we use doesn't support it and the x-security-optional comes to help
also fixed a possibility to disable global security requirements for particular paths

The underlying swagger library we use doesn't support it

Before moving forward with the review, I did a small experiment with:
diff --git a/modules/sample/src/main/resources/issues/issue1218.yaml b/modules/sample/src/main/resources/issues/issue1218.yaml
index 08a65de4..05751565 100644
--- a/modules/sample/src/main/resources/issues/issue1218.yaml
+++ b/modules/sample/src/main/resources/issues/issue1218.yaml
@@ -11,6 +11,10 @@ paths:
       responses:
         '204':
           description: No response
+      security:
+        - basic:
+          - "bar:basic"
+        - {}
       parameters:
         - name: refvec
           in: query
@@ -125,6 +129,10 @@ paths:
               type: integer
               format: int64
 components:
+  securitySchemes:
+    basic:
+      type: http
+      scheme: basic
   schemas:
     refvec:
       type: array

combined with
diff --git a/modules/core/src/main/scala/dev/guardrail/Common.scala b/modules/core/src/main/scala/dev/guardrail/Common.scala
index 47b5c90b..6445a953 100644
--- a/modules/core/src/main/scala/dev/guardrail/Common.scala
+++ b/modules/core/src/main/scala/dev/guardrail/Common.scala
@@ -72,6 +72,15 @@ object Common {
         .filter(_ != "/")

       paths = swagger.downField("paths", _.getPaths)
+      () = {
+        import scala.jdk.CollectionConverters._
+        swagger.unwrapTracker.getPaths.asScala.values.toList.flatMap(_.readOperations().asScala.toList).flatMap(_.getSecurity.asScala.toList).foreach(println)
+      }
       globalSecurityRequirements = NonEmptyList
         .fromList(swagger.downField("security", _.getSecurity).unwrapTracker)
         .flatMap(SecurityRequirements(_, SecurityOptional(swagger), SecurityRequirements.Global))

which, while not null safe at all and quite unreadable, it does show the expected states:
[info] class SecurityRequirement {
[info]     {basic=[bar:basic]}
[info] }
[info] class SecurityRequirement {
[info]     {}
[info] }

What is not supported? Is this some interaction with the automatic type-conversion in Tracker? If so, I can help -- that structure is still somewhat difficult to work with.
Since the case of - {} is actually in the documentation for the OpenAPI specification itself, I'd really like to get that case working, instead of adding a workaround that extends the specification.
Additionally, if x-security-optional is currently not used today (perhaps it was used for something internally at Twilio in the past, unsure), we can likely drop it, simplifying the user experience.

oh, I definitely misunderstood some concepts, thanks for noticing. I replaced the hack with x-security-optional to normal optional security requirements definition and deleted the x-security-optional flag support. Other changes are left the same

Just force pushed to fix the last comment. Provided the tests pass, I'll release the branch back to you

Hi! It's been a while... Finally I've found some time to implement the 'native' authentication for http4s. It ignores OpenAPI spec security definitions and create AuthedRoutes to be provided to the AuthMiddleware.

I've tested native and simple in a test project, this looks excellent.
Thanks so much for seeing this through to the end! I'm convinced on functionality, I'll do a few more passes to feel comfortable I didn't miss anything and merge.

The last concern I had was whether these parameters made sense across Scala/Java, and for all the different supported frameworks. I'm alright with proceeding.

Thanks for your patience, @zeal18, life has been getting in the way of getting 100% behind where we ended up by the end here.

The last concern I had was whether these parameters made sense across Scala/Java, and for all the different supported frameworks. I'm alright with proceeding.

I thought about it, especially about the native. I think it become clearer over the time.
I've took a look at some of your improvements but had no chance to check them all, it ok for me by the way) It's nice that it's merged, that means I'll check how the idea works in our project very soon 🤤

Thanks for your patience, @zeal18, life has been getting in the way of getting 100% behind where we ended up by the end here.

thanks, it was nice to work on it!


Going emeritus, remove myself from WGs and SIGs

This PR removes myself as an approver/reviewer in SIG Docs.
It's been a wonderful four years...and now it's time for something different. ❤
Related PRs:

https://github.com/kubernetes/community/pull/5733
https://github.com/kubernetes/org/pull/2649

👋 @zacharysarah .
Good luck with your next adventure!
/lgtm
Hold for review 😃
/hold

/approve

Farewell @zacharysarah / thanks so much!

@zacharysarah you carved a welcoming path for new k8s contributors, myself included, that enabled many future successes. From leadership to documentation architecture, your dedication and love for docs is admirable.
Best of luck in future endeavors! Cheers! 🎉 💙

/approve
/pony
/honk
/woof
/meow
/this-is-not-fine


Fixes #14 -- Added Float Field

@millerthegorilla -- here's a draft implementation for floating fields. Do you want to take a look?
The test shows how this would be used. Is this the right way of doing this? 🤔

@millerthegorilla -- anything else to add here?
If not I'll write up a small documentation note -- probably to the readme -- and look to merge this.

I will take a look first thing tomorrow, but I still haven't got round to switching my project to crispy bs5 yet.

Perfect! Looking forward to merge this.

@eriktelepovsky Have you given this a run? Do you think the way this would be used in forms is right?

Hi, sorry I got snowed under.  I just checked this and it looks really good, and works well, at first glance.  Looking forward to it being merged.

Just one thing I notice straight away, probably me not getting it straight, but there are a few floating fields in a form, that have the following below them - display name
I guess this is from help_text_and_errors, but I don't have help text defined for those floating fields that have this hint text, and not all the floating fields are displaying a hint.
I have added help_text to the model field that is displayed and cannot see the help text in the form using 'floatingfield'.  However, I've been away from django for a month, and am just warming up to it again.  I will return first thing tomorrow, and take a closer look at where I might be going wrong.

Ok, let's merge this. Thanks all for your input.

The only issue that I can see is that when I have help_text defined in the model rather than the form, it is not showing in the rendered html.

This sounds like a separate issue, I'm not sure without taking a closer look

I haven't tested selects, but it also works fine with textareas which is nice.

Hey @smithdc1 ,  when is the merge with main likely to happen?  I am hoping to roll out the bootstrap5 branch of my site.  Thanks,

@smithdc1 is there any reason this can't be merged?  I am hoping to roll out my crispy forms bootstrap5 site, with the funky new floatfields!

Ok, I added additional tests for failing fields, text area and selects. All seem to look ok to me. 🙂


Really need example | How to play mp3 file

Dear Tristan.
Can you give an example with comments? How to play mp3 file? (osx)
I know about the example( https://godoc.org/github.com/tcolgate/mp3#ex-Decoder-Decode),
but it absolutely did not help me.
Please do not close the 	question, if you do not have time to give an example, maybe someone else will help.
Thank you very much!

 Thank you.
But can you then tell me what I'd better use to solve my problem?
I just can not find it on the Internet.
(And after, of course, you(or me) can remove this issue.) Please?

I'm not aware of a pure Go mp3 decoder yet, (I may add it to this package one day, but not soon).
I've had some discussion about this with another user before, they ended up using libmpg123 and Cgo...
https://github.com/angadn/empeggo/blob/master/mpg123/mpg123.go


Visual Samples List

Q
A

Bug fix?
no

New feature?
yes

New sample?
no

Related issues?

What's in this Pull Request?
Adds a table containing a list of every sample, description, preview and drop.
The file is auto-generated (I wrote the script while on the SIG call) and therefore easy to maintain.
I would have preferred to use a markdown "include" instead of changing the readme.md, but couldn't figure it out. If anyone knows how to do this, I can re-submit with the sample table in a separate markdown file.
I hope this helps. I can also submit a similar file for the other repos, if you'd like.

Let's see what kind of feedback we get from this... looks really nice, but no point spending too much time on it if it's not used. One alternative in future would be to host some sort of a website, where we list all of these... or wait until we get the SP dev portal in proper place for this kind of experience.


Listeners do not all agree on what order events happened in when some synchronously trigger.

Because of the call stack, if a listener (or filter) synchronously triggers 2, upon reciept of 1, listeners that were attached before that one, and spies, will see [1,2], while listeners attached later will see [2,1]..
It seems highly bad to have listeners ever disagree on the order of events!
Delay only returned synchronous observables by a tick  () => after(0, () => miniBus.trigger(2)). Listeners like  () => {   miniBus.trigger(2) } would be unsupported, maybe warned about in dev mode.

        const heardViaEarlyListen = [];
        const heardViaLateListen = [];
        const seenViaSpy = [];

        miniBus.spy((e) => {
          seenViaSpy.push(e);
        });

        miniBus.listen(
          () => true,
          (e) => {
            heardViaEarlyListen.push(e);
          }
        );

        const sub = miniBus.listen(
          (e) => e === 1,
          () => {
            miniBus.trigger(2);
          }
        );
        miniBus.listen(
          () => true,
          (e) => {
            heardViaLateListen.push(e);
          }
        );

        miniBus.trigger(1);
        expect(seenViaSpy).toEqual([1, 2]);
        expect(heardViaEarlyListen).toEqual([1, 2]);
        expect(heardViaLateListen).toEqual([2, 1]);
        sub.unsubscribe();

A possible solution here: https://github.com/deanrad/omnibus-rxjs/tree/queue-handling-evaluations
It will intentionally break the ability to DFS through synchronous recursion so that handlers always agree on what order events were sent.


TypeError: 'NoneType' object is not subscriptable

Getting TypeError: 'NoneType' object is not subscriptable when using the example usage on README:
from ycecream import y

def add2(i):
    return i + 2

y(add2(1000))

Full error message is:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mylocaluser/test-ycecream/venv/lib/python3.8/site-packages/ycecream/ycecream.py", line 220, in __call__
    this_line = frame.code_context[0].strip()
TypeError: 'NoneType' object is not subscriptable

Using new isolated venv with the following packages and their versions:
Package    Version
---------- -------
pip        21.0.1
setuptools 49.2.1
ycecream   1.1.4

Strange.
When I run it, everything seems to be fine.
Can you tell me how you run this code?
What operating system, what IDE (if any), etc.

I'm on OSX with detailed OS ver as follows:
 uname -na
Darwin MYMAC 20.3.0 Darwin Kernel Version 20.3.0: Thu Jan 21 00:07:06 PST 2021; root:xnu-7195.81.3~1/RELEASE_X86_64 x86_64

and here are the complete steps from creation of venv to the error:
cd ~
mkdir test-ycecream
cd test-ycecream
python3 -m venv venv
source venv/bin/activate
pip install pip --upgrade
pip install ycecream
python

and then I enter the sample code by copy/paste from README page:
from ycecream import y

def add2(i):
    return i + 2

y(add2(1000))

Are you running this in a REPL?
If so, that is not supported. The error message should be cleared, though. I will fix that in the next release.
If you are running this from a .py file, I have to look at it.

Yip, using this in the standard Python REPL and you are correct, when I put the sample in a test-y.py file and run it with "python test-y" then it works.
Recommend you add a disclaimer that ycecream is only supported in .py files and definitely make the error message more clear 😅
Thanks for this handy tool and definitely improves on my print() statements, especially the decorator to quickly check function performance when changing from one version to another in API calls 😄

Yes, I will make the error more clear and add somewhere that you can't use ycecream from a REPL.
Thanks for pointing out this behaviour (bug?).
 I want to do a little demo of ycecream tomorrow during the meetup. If you feel like it, please join (details on the forum).

Thanks Ruud and yup, on PythonistaCafé and will do my best to join meetup today. Would be good if you could record the ycecream demo part of the meetup for replay later 🤗


useLongPress should be cancelled on multiple touches

Provide a general summary of the feature here
Hi there!
I just received an accessibility issue from a user (see: https://github.com/aeharding/voyager/issues/1254).

3-finger double tap cause [long press menu] to open. This is commonly used as an accessibility shortcut. I use it for full screen zoom. Ii can't do it with [App] because it causes the [long press menu] to open in waiting to the zoom.

Essentially, the native iOS feature is conflicting with long press, because long press does not cancel when multiple touches are detected.
🤔 Expected Behavior?
Long press should cancel when multiple touches are detected - at very least, as an option.
This is the standard behavior for native iOS apps (for example, the Mail app).
😯 Current Behavior
Holding screen with two fingers, or pressing one finger and then another before the LONG_PRESS_THRESHOLD, causes the long press callback to be triggered.
💁 Possible Solution
New option: LongPressProps["cancelOnTouches"]?: boolean
🔦 Context
Please see accessibility concern from user, where they cannot use the native iOS zoom function with 3-finger tap because it conflicts with useLongPress.
https://github.com/aeharding/voyager/issues/1254
💻 Examples
No response
🧢 Your Company/Team
Voyager for Lemmy
🕷 Tracking Issue
https://github.com/aeharding/voyager/issues/1254

Cancelling the long press seem like the correct behavior in this case, and matches native iOS and Android behavior.


Configure TCPMTU probe instead of iptable rules

This enables TCP MTU probing by default with a base mss of 1024.
Signed-off-by: Aswin Surayanarayanan<EMAIL_ADDRESS>

Need to wait for the decision on https://github.com/submariner-io/submariner/pull/1480, before merge.


Navbar logo and items get displayed in new lines.

I just moved to bootstrap 4 and I am having a problem with the navbar.
All the items in navbar come in a new line
Here's my html :-

        Logo

                Link 1

                Link 2

                Link 3

The logo and items are getting broken into new lines.
Is there something wrong with my html or it an actual issue I'm not sure.
I have also attached a screenshot with the inspector so you can check better.

I have no css written at my end just the bootstrap.min.css is all thats included.
Here's the bootstrap version :-
    /*!
     * Bootstrap v4.0.0-alpha.6 (https://getbootstrap.com)
     * Copyright 2011-2017 The Bootstrap Authors
     * Copyright 2011-2017 Twitter, Inc.
     * Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE)
     *//*! normalize.css v5.0.0 | MIT License | github.com/necolas/normalize.css */

The HTML and CSS for the navbar in v4 is much different than that of v3's. Be sure to update to the latest HTML alongside the latest CSS.


enable autodiscovery of plugins; closes #99

new behavior: if no plugins are specified, then they are automatically discovered.  if plugins are specified, no autodiscovery happens
a plugin is defined as a module whose name begins with the prefix probot-
move plugin loading/resolving routines into module
add tests

Testing was a bit awkward here, as a user must save a plugin to package.json for it to be discovered.  This can be revisited in/after #101.

the lint problem is addressed by a forthcoming PR

also I need to rip out sinon, since I didn't realize expect came w/ spies

I'm now using expect for spies, and have placed a temporary directive in my test file to avoid build failures (see #124).

Awesome, looks great!


(fix) account for single order level in linear distribution

Before submitting this PR, please make sure:

[X] Your code builds clean without any errors or warnings
[X] You are using approved title ("feat/", "fix/", "docs/", "refactor/")

A description of the changes proposed in the pull request:

accounts for case where n_levels equals 1 in linear distribution

@fengtality the solution is good and we should test if has to be replicated in other distributions, but you need to add a test to cover that line that you are adding


Feature/fit generator

Codecov Report

Merging #284 into 2.0 will decrease coverage by 0.15%.
The diff coverage is 50%.

@@           Coverage Diff            @@
##            2.0     #284      +/-   ##
========================================
- Coverage   100%   99.84%   -0.16%
========================================
  Files        25       25
  Lines       635      637       +2
========================================
+ Hits        635      636       +1
- Misses        0        1       +1

Impacted Files
Coverage Δ

matchzoo/engine/base_model.py
98.76% <50%> (-1.24%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5affda0...2141d88. Read the comment docs.