Move examples to rust projects (remove skeptic)

The current approach to writing examples is to write the code directly in the markdown files an then combine and use skeptic for testing.
The downside of this is that it's more difficult to edit the code using the standard rust tooling (rust-analyzer, rustfmt, clippy, etc.)
A better approach (@jamesgraves did this in https://github.com/jamesgraves/rust-cookbook/) is to move all the code into actual projects and then import from files into mdbook (see https://rust-lang.github.io/mdBook/format/mdbook.html#including-files)
Doing this means that in general you can write code, lint it and format it, and then view the code snippet in the book directly. It also means that each snippet can be supported by the relevant cargo manifest necessary just for that snippet rather than a manifest for the entire cookbook, and it makes it possible to run each example directly when you clone the repo.
Implementation
We use this approach successfully in the Ratatui website. This started as an mdbook site and then we moved to using Starlight (a doc site template for Astro). To do this we added the same idea of importing code from files to the stock code block element, and then went a little further to reduce the amount of // ANCHOR comments needed for importing rust functions using tree sitter to parse out just functions of interest. This might be possible to backport into mdbook.
Minor side note (code organization)
I'm not 100% in agreement with the arrangement of where the rust projects end up in the tree in @jamesgraves version. They feel a bit jumbled together which is difficult to navigate. In Ratatui, we had a second root code folder with just the crates / code. This observation is a minor one however, as I think it's much better to have this functionality than to bikeshed on exactly how it's organized. Perhaps there's some hybrid where each crate has a src folder with a docs folder alongside it. That would make it feel less jumbled (I'm not whether it's possible with mdbook to have the file location and url not fully aligned like this)

Here's what motivated me to make the changes to how the cookbook works, in a bit more detail than what was in the CHANGES.md file.
At one point in 2022, the project would not pass cargo test because the skeptic crate was using the wrong version of a dependency for one of the tests. There was more than one version of a dependent crate, and it picked the wrong one to link into a particular example. I wasn't sure how to fix that with the skeptic crate. After learning about cargo workspaces, and how some other projects use code examples, I thought it would be easier to just rip out skeptic, and put all the code into .rs files.
Another benefit was the ability to test and run examples that have OS dependencies, or that interact with the filesystem, such as with the SQLite 3 database section.
And this ties into how I thought the cookbook ought to be used: rather than just looking at the examples and running some of them on the Rust Playground, instead the developer should clone the repository and try any the examples locally. So it is more code-focused rather than documentation-focused, in a sense.  That's why I also like having more of the comments in the code, rather than being separate from it, though I understand the other point of view.
While it is nice to have the code and commentary in a single file, to be edited together, I usually just edit both files (the .md and the .rs) in a split window, so not having them in the same file is not a significant downside to me.

As far as the file organization in the cookbook, I'm definitely open to ideas about how to organize the directory structure. I haven't put much thought into that since the implementation of workspaces.  If the boilerplate needed for cargo test could be consolidated, that would be convenient. I'd want to retain the ability to run cargo test inside a subdirectory, and have it just test the examples in that workspace.

On the organization side I can see how it makes sense after playing with it a bit more. Adding a /doc folder under each crate might be possible with a small mdbook preprocesser that modifies the location of each file. Then you'd have something like:
algorithms
├── randomness
│   ├── doc
│   │   ├── rand-choose.md
│   │   ├── rand-custom.md
│   │   ├── rand-dist.md
│   │   ├── rand-passwd.md
│   │   ├── rand-range.md
│   │   └── rand.md
│   ├── examples
│   │   ├── basic_usage.rs
│   │   ├── gen_range.rs
│   │   ├── generate_password.rs
│   │   ├── normal_distribution.rs
│   │   ├── random_string.rs
│   │   ├── tuple_distribution.rs
│   │   └── uniform_distribution.rs
│   ├── src
│   │   └── lib.rs
│   ├── Cargo.toml
│   └── randomness.md
└── sorting
    ├── doc
    │   ├── sort-float.md
    │   ├── sort-struct.md
    │   └── sort.md
    ├── examples
    │   ├── sort-basic.rs
    │   ├── sort-float.rs
    │   └── sort-struct.rs
    ├── src
    │   └── lib.rs
    ├── Cargo.toml
    └── sorting.md

Something that I would suggest is collapsing the structure to two levels to keep the TOC and folder arrangement a bit simpler (it goes to 3 in some places)
Another thing that might also be worth doing is automating the various lists of recipes with a custom preprocesser. This would keep them in sync pretty easily rather than having a lot of manual docs to do it and reduce the maintenance burden.
On the testing side it could be worth looking at something like trycmd or insta  to actually fully test the input / output. Maybe cli output could possibly even be generated and imported directly from each example rather than having to copy paste it in.

Have a look on how examples are organized in https://github.com/john-cd/rust_howto/tree/main/deps/tests

Example code have been moved to a separate project ('deps')
They are written as tests calling a main(), so that (1) the code can be copied-pasted and (ii) fully tested in CI
Examples / tests are grouped via modules, meaning  cargo nextest / test only have to compile a few executables and test exec is fast

Sounds like there's 3 versions of the same thing now. Perhaps it could be worth doing a audit of how these crossover and consolidating them rather than duplicating effort.

While I like how skeptic made the testing unobtrusive, it left a real gap with maintenance.  Patching skeptic is hard, and the explicit testing strategy is much easier to reason, despite having to maintain test files.  Unless @djc objects, I am in favor of manually testing all the recipes, despite the expensive overhaul.  I'm also in favor of only fixing recipes that haven't already been fixed by others, coordinating the remaining effort, and delivering a collaborative product.

if the Cargo.toml leverages workspaces if the Playground could read it

Yeah it will be fine.

Proposed leveraging modules for recipes to make the cookbook more of a banquet than a single course you reheat.  I've started working on web.  The important part is web is crate you can download and just update the types for your own api and pick and choose the modules you need.

thoughts?

I'd suggest going one step further and putting all the code in the rust file (including the main.rs), not just the lib call.


📈 SSW Rules - Do you use the right syntax to solve nullability errors?

null! vs default!

I've found some info from the EF Docs on this.  MS recommend using required , and in the scenarios where this can't be use null!.

Working with nullable reference types - EF Core | Microsoft Learn


fix(curriculum): wrap logo in a code block

…sed to be.
Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Closes #47015

You have forgotten to create a branch, other than that.
✨ 🎉 AWESOME! 🎉 ✨
Hi @DylanDevelops, thanks for this pull request and for contributing to the code-base for the first time. We are looking forward to more contributions from you in the future.
Cheers & happy contributing!

When I create a branch do I just name it anything? Like could I name my branch, DylanDevs-patches or something like that or how exactly do I work with that. Branches are still a bit confusing to me.

It is recommended to make changes on a separate branch, than the main branch. See more about it in point 3 at https://contribute.freecodecamp.org/#/how-to-setup-freecodecamp-locally?id=making-changes-locally.


Refactor the TCP_IP module

Description

Test Steps

Related Issue

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@AniruddhaKanhere , thanks again for this refactoring!
I have tried the TCP protocol by running FTP sessions. That uses many features of the TCP library.
I tried both active and passive TCP connections.
There were also zero-byte connections, in case an empty file is being copied with FTP.
Also interesting is sending the last bytes along with a FIN flag for a quick closure using the socket option FREERTOS_SO_CLOSE_AFTER_SEND.
It passed all tests and with the same great performance as usual.
There were 3 problems while compiling:
xTCPCheckNewClient not declared in FreeRTOS_IP.c
Maybe "FreeRTOS_TCP_IP.h" must be included?

eTCP_LISTEN is undefined in FreeRTOS_Sockets.c
Maybe "FreeRTOS_TCP_IP.h" must be included?

vTCPStateChange() is undefined in FreeRTOS_Sockets.c
Shouldn't "FreeRTOS_TCP_IP_utils.h" also be included?

As you know, I adapted my Makefile(s) as follows to include the new files, only when present:
ifeq (,$(wildcard $(PLUS_TCP_PATH)/FreeRTOS_TCP_IP_Reception.c))
    # old sources before split-up
else
	C_SRCS += \
		$(PLUS_TCP_PATH)/FreeRTOS_TCP_IP_Reception.c \
		$(PLUS_TCP_PATH)/FreeRTOS_TCP_IP_StateHandling.c \
		$(PLUS_TCP_PATH)/FreeRTOS_TCP_IP_TimerWork.c \
		$(PLUS_TCP_PATH)/FreeRTOS_TCP_IP_utils.c
endif

That makes it possible to easily switch between software releases, only $(PLUS_TCP_PATH) will change.

Hi Aniruddha,
Could you add to FreeRTOS_IP.c:
#if( ipconfigUSE_TCP == 1 )
    #include "FreeRTOS_TCP_IP.h"
#endif

and from FreeRTOS_Sockets.c :
#if( ipconfigUSE_TCP == 1 )
    #include "FreeRTOS_TCP_IP.h"
    #include "FreeRTOS_TCP_IP_utils.h"
#endif

so that the two modules can compile?

Hi Aniruddha,
Could you add to FreeRTOS_IP.c:
#if( ipconfigUSE_TCP == 1 )
    #include "FreeRTOS_TCP_IP.h"
#endif

and from FreeRTOS_Sockets.c :
#if( ipconfigUSE_TCP == 1 )
    #include "FreeRTOS_TCP_IP.h"
    #include "FreeRTOS_TCP_IP_utils.h"
#endif

so that the two modules can compile?

Hello Hein!
Of course. This is still changing and being modified with unit-tests etc. But yes, I'll add the header files to the source files you mentioned. Thank you for figuring that out!
Also, we need to come up with a folder structure which would be beneficial to our user/customer. I am looking at other TCP stacks to figure out a good way to arrange all these new files in a good manner.


Possible to set IRC client to OPER?

I am trying to give permitted IRC users OPER role so that I can set configure a custom discord bot that will relay IRC commands through discord-irc (https://github.com/reactiflux/discord-irc)
discord-irc connects and does most of what I need.
However, my livestreams are frequented by trolls and I wish to be able to send a KICK, KICKBAN, QUIET etc command from discord to mute the webhooks into discord (which would allow those users to remain in communication on the network chat directly -- just not flooding discord)
However, whenever I send a command my permitted IRC users are unable to pass any args to the IRC server because they are not an Operator.
The only operator I see is named root and I cannot figure out to connect as root to grant OPER status to select clients. I don't see in the irc/server.lua or elsewhere that would lead me to root's credentials/password or any definition that would allow to adjust who would be granted operator on IRC startup.
I may try to create a new bot command but I am just learning Lua since using this project.
What is the best way to grant OPER status to clients?

So the built-in IRC server is operating with the bare minimum needed to work with IRC clients. There's no real "operator" concept, the code to respond to topic changes, kicks, etc is mostly there to meet the minimum specs to meet the IRC RFC. I'm actually pretty amazed that discord-irc works!
I would recommend banning trolls on Twitch/YouTube/etc.


ReferenceError: Promise is not defined

/var/www/rainloop-webmail/node_modules/postcss/lib/lazy-result.js:157
        this.processing = new Promise(function (resolve, reject) {
                              ^
ReferenceError: Promise is not defined
    at LazyResult.async (/var/www/rainloop-webmail/node_modules/postcss/lib/lazy-result.js:157:31)
    at LazyResult.then (/var/www/rainloop-webmail/node_modules/postcss/lib/lazy-result.js:79:21)
    at DestroyableTransform._transform (/var/www/rainloop-webmail/node_modules/gulp-autoprefixer/index.js:24:6)
    at DestroyableTransform.Transform._read (/var/www/rainloop-webmail/node_modules/readable-stream/lib/_stream_transform.js:159:10)
    at DestroyableTransform.Transform._write (/var/www/rainloop-webmail/node_modules/readable-stream/lib/_stream_transform.js:147:83)
    at doWrite (/var/www/rainloop-webmail/node_modules/readable-stream/lib/_stream_writable.js:313:64)
    at writeOrBuffer (/var/www/rainloop-webmail/node_modules/readable-stream/lib/_stream_writable.js:302:5)
    at DestroyableTransform.Writable.write (/var/www/rainloop-webmail/node_modules/readable-stream/lib/_stream_writable.js:241:11)
    at DestroyableTransform.ondata (/var/www/rainloop-webmail/node_modules/readable-stream/lib/_stream_readable.js:531:20)
    at DestroyableTransform.emit (events.js:95:17)

using 4f2a8b5b9b694f88d427aa8d7f0739a6e7fd7498

Try to update your node.js version

@RainLoop
what version is best? i went with nodejs stable
(node:21104) fs: re-evaluating native module sources is not supported. If you are using the graceful-fs module, please update it to a more recent version.
[23:31:57] Using gulpfile /var/www/rainloop-webmail/gulpfile.js
[23:31:57] Starting 'js:libs'...
[23:31:57] Starting 'js:openpgp'...
[23:31:57] Starting 'js:openpgpworker'...
[23:31:57] Starting 'js:webpack'...
[23:31:57] Starting 'js:lint'...
[23:31:58] Starting 'js:eslint'...
[23:31:58] Starting 'less:main'...
(node:21104) fs: re-evaluating native module sources is not supported. If you are using the graceful-fs module, please update it to a more recent version.
[23:31:58] Starting 'ckeditor:clear'...
[23:31:58] Starting 'fontastic-fonts:clear'...
[23:31:58] Starting 'fontastic-svg:clear'...
[23:31:58] Finished 'fontastic-fonts:clear' after 41 ms
[23:31:58] Starting 'fontastic-fonts:copy'...
[23:31:58] Finished 'fontastic-svg:clear' after 43 ms
[23:31:58] Starting 'fontastic-svg:copy'...
[23:31:58] Finished 'fontastic-svg:copy' after 9.81 ms
[23:31:58] Finished 'ckeditor:clear' after 93 ms
[23:31:58] Starting 'ckeditor:copy'...
[23:31:59] Finished 'js:openpgpworker' after 1.78 s
[23:31:59] Finished 'js:openpgp' after 2.27 s
[23:31:59] Starting 'js:pgp'...
[23:31:59] Finished 'js:pgp' after 7.75 μs
[23:32:00] Finished 'less:main' after 2.5 s
[23:32:00] Starting 'css:main-begin'...
[23:32:02] Finished 'fontastic-fonts:copy' after 3.78 s
[23:32:02] Starting 'fontastic'...
[23:32:02] Finished 'fontastic' after 6.6 μs
[23:32:10] Finished 'css:main-begin' after 9.66 s
[23:32:10] Starting 'css:clear-less'...
[23:32:10] Finished 'css:clear-less' after 112 ms
[23:32:10] Starting 'css:main'...
[23:32:10] Finished 'css:main' after 4.87 μs
[23:32:10] Starting 'css:main:min'...
[23:32:12] Finished 'js:webpack' after 15 s
[23:32:12] Starting 'js:app'...
[23:32:12] Starting 'js:admin'...
[23:32:12] Starting 'js:chunks'...
[23:32:12] Finished 'js:chunks' after 23 ms
[23:32:12] Finished 'css:main:min' after 2.23 s
[23:32:12] Finished 'js:admin' after 620 ms
[23:32:12] Finished 'js:app' after 622 ms
[23:32:13] Finished 'js:libs' after 16 s
[23:32:14] Finished 'js:eslint' after 16 s
[23:32:16] Finished 'js:lint' after 19 s
[23:32:16] Starting 'js:validate'...
[23:32:16] Finished 'js:validate' after 5.26 μs
[23:32:16] Starting 'js:min'...
[23:32:24] Finished 'js:min' after 7.19 s
[23:32:24] Finished 'ckeditor:copy' after 26 s
[23:32:24] Starting 'ckeditor:copy-plugins'...
[23:32:24] Finished 'ckeditor:copy-plugins' after 12 ms
[23:32:24] Starting 'ckeditor'...
[23:32:24] gulp-stripbom: build-config.js ✔
[23:32:24] gulp-stripbom: ckeditor.js ✔
[23:32:24] gulp-stripbom: config.js ✔
[23:32:24] gulp-stripbom: styles.js ✔
[23:32:24] Finished 'ckeditor' after 55 ms
[23:32:24] Starting 'default'...
[23:32:24] Finished 'default' after 4.62 μs

Your log looks great. I do not see any error (except graceful-fs, but it does not matter)

@RainLoop
btw
found out this
npm WARN optional Skipping failed optional dependency /chokidar/fsevents:
npm WARN notsup Not compatible with your operating system or architecture<EMAIL_ADDRESS>
on npm install
if fsevents important? im using centos 7 and npm 3.9.3


[Fixes #41] Refresh the session even when its not accessed in current request

@Tratcher @muratg
#41

Per #41, @Tratcher recommends this.  Based on that, I'll add a :shipit: :)


Dependency spdlog missing

code-server version: 1d8da2161f4330e2cf26fd28005729e3b6e21286
OS Version: Darwin Kernel Version 17.7.0

Steps to Reproduce

yarn; yarn start;, then throw TS Compile Error
Missing dependency spdlog: https://github.com/codercom/code-server/blob/master/packages/protocol/src/common/helpers.ts#L1

Hmm. Did all sub-packages install correctly? spdlog is required by packages/protocol.

Lost one step in yarn postinstall script, the dependency microsoft/vscode is not installed.

Interesting. Did you run yarn in the root of the repository?

Yeap, i runned in root path.
And i don't think it is necessary to clone the entire repository. You can just clone the tag/1.13.0 branch of vscode. I will give you a PR.

Ah this makes sense. You'll need to manually run yarn inside the cloned vscode for that reference to exist. Will fix!

Closing in favor of discussion in PR


create nightly test run for Jenkins

We need a nightly run that runs some of the more resource-heavy tests that are currently ran as part of the build (specifically L7 stress tests).

[ ] create a Jenkins job that runs nightly using HEAD of master.
[ ] L7 stress tests should only run during this nightly run, not on merges.
[ ] establish a failure condition for the stress tests (TODO - does this exist in the current tests?)

Could be useful : https://hopstorawpointers.blogspot.com/2016/10/performing-nightly-build-steps-with.html

Related: #824

Suggested by @jrfastab for these nightly runs: perf tests for kproxy

Would also be great to capture and track performance data for super_netperf tests TCP_STREAM / TCP_RR for i) node local endpoints, ii) endpoints on different nodes. For the latter, would be good to have tests for direct routing and for each supported overlay mode (vxlan, geneve) to track potential performance regressions of kernel/bpf data path.

Lowered the priority as the base infrastructure is ready

@eloycoto will check in on this to see if we can close this and / or split it up into smaller, more up-to-date issues.


Moved content to a static subdomain

Files are now stored at scd.thingsima.de instead of img.thingsima.de.
Instead of images, it also houses the default stylesheet and any future external front end scripts.
scd = static, cookieless domain

Oh yeah, stylesheets are also up to date now (not that I imagine there's much visible change)

Oh yeah, this closes #24


Bugfix: Properly bucket future runs in flow run histogram

Related to: https://github.com/PrefectHQ/prefect/issues/11977
When viewing timespans in the future, the flow run history graph was incorrectly bucketing runs. This is due to two factors:

When requesting runs we request them sorted by START_TIME_DESC, but runs in the future don't have a start time, they only have an expected_start_time. So this PR will sort the runs accordingly if looking at future time spans.
The bucketing function was designed to bucket right to left, working on the latest run to the earliest. Timespans in the future need to bucket from the left to the right, earliest to latest. So the increment direction is now conditional based on if looking at future time spans.

Before:
https://github.com/PrefectHQ/prefect-ui-library/assets/6776415/250dfaef-6833-47d0-83a5-1682fdad507b
After:
https://github.com/PrefectHQ/prefect-ui-library/assets/6776415/1ae1be88-dab5-47c6-880d-1074196e7cd6

Nice, yep this makes sense. Does this effect PrefectHQ/prefect#11977 any?

@pleek91 It kinda doesn't really. I think the user was trying to see all runs within a certain time frame, but the graph is designed to only show recent runs. I'm writing them now to explain. They probably want to use the flow runs page where there's more filtering, or something.


[Feature]  In a Paimon primary key table, using ORC offers significantly higher efficiency for point lookups based on the primary key compared to Parquet.

Search before asking

[X] I searched in the issues and found nothing similar.

Motivation
Basic Information
table
{
  "version" : 2,
  "id" : 0,
  "fields" : [ {
    "id" : 0,
    "name" : "f0",
    "type" : "INT NOT NULL"
  }, {
    "id" : 1,
    "name" : "f1",
    "type" : "STRING"
  }, {
    "id" : 2,
    "name" : "f2",
    "type" : "STRING"
  } ],
  "highestFieldId" : 2,
  "partitionKeys" : [ ],
  "primaryKeys" : [ "f0" ],
  "options" : {
    "bucket" : "1",
    "file.format" : "parquet/orc",
    "manifest.compression" : "null"
  },
  "timeMillis" :<PHONE_NUMBER>602
}

RowCount  : 400w
bucket : 1
FileNumber : 3
Ordered Write

The data sample ：
1, -963544071, fdad7f1c-e5cc-45e6-886b-901bb35ffd01
2, 153336978, 850f0c9a-e5bb-4c90-b2df-b84798be3ca8
3,<PHONE_NUMBER>, 1f53d460-3327-43c5-aaeb-cf2cbbc7220e
4,<PHONE_NUMBER>, aabef5a8-9d4b-4a96-b6ad-d687bbc7ffa0

The Write Example
Table table = TableUtil.getTable();                // PrimaryKeyFileStoreTable
BatchWriteBuilder writeBuilder = table.newBatchWriteBuilder();
BatchTableWrite write = writeBuilder.newWrite();  // TableWriteImpl

Random random = new Random();

long startTime = System.currentTimeMillis();

for(int i = 0; i < 4000000; i++){

  GenericRow genericRow = GenericRow.of(
          i,
          BinaryString.fromString(String.valueOf(random.nextInt())),
          BinaryString.fromString(UUID.randomUUID().toString())
  );
  write.write(genericRow);
}

List<CommitMessage> messages = write.prepareCommit();

BatchTableCommit commit = writeBuilder.newCommit();

commit.commit(messages);

long stopTime = System.currentTimeMillis();

System.out.println("time : " + (stopTime - startTime));

The Read Example
random 30 for read
Table table = TableUtil.getTable();   // PrimaryKeyFileStoreTable
  PredicateBuilder builder = new PredicateBuilder(
          RowType.of(DataTypes.INT(),
                  DataTypes.STRING(),
                  DataTypes.STRING()));

  int[] projection = new int[] {0, 1, 2};

  ReadBuilder readBuilder = table.newReadBuilder()
                  .withProjection(projection);

  List<Split> splits = readBuilder.newScan()
          .plan().splits();

  Random random = new Random();

  long startTime = System.currentTimeMillis();

  for(int i = 0 ; i < 30 ; i ++){
    InnerTableRead read = (InnerTableRead)readBuilder.newRead();

    int key = random.nextInt(4000000);

    Predicate keyFilter = builder.equal(0, key);
    read.withFilter(keyFilter).executeFilter();
    RecordReader<InternalRow> reader = read.createReader(splits);

    reader.forEachRemaining(internalRow -> {

      int f0 = internalRow.getInt(0);
      String f1 = internalRow.getString(1).toString();
      String f2 = internalRow.getString(2).toString();
      System.out.println(String.format("%d - {%d, %s, %s}",key, f0, f1, f2));
    });

  }
  long stopTime = System.currentTimeMillis();
  System.out.println("time : " + (stopTime - startTime));

Time Consumption in ORC/Parquet

PARQUET reader  : 17982ms
ORC  reader          : 1096ms

Root Cause Analysis
Under the current query predicate pushdown, in ORC, it can be pushed down to the column index level, whereas in Parquet, it is only pushed down to the row group level.
Solution
No response
Anything else?
No response
Are you willing to submit a PR?

[X] I'm willing to submit a PR!

Can you turn on these two parameters and try again? @Aiden-Dong
是否可以打开这两个参数再试试？paimon默认并没有做orc下推。
'orc.reader.filter.use.selected'='true',
'orc.reader.sarg.to.filter'='true'

see https://github.com/apache/paimon/pull/4231

In addition, you can turn off executeFilter, because turning it on will cause reading to slow down. In addition, orc pushdown will ensure that only the required data is returned in most cases.
另外你可以关闭executeFilter，因为开启它会导致读变慢，另外orc下推大多数情况下会保证只返回需要的数据，

In addition, you can turn off executeFilter, because turning it on will cause reading to slow down. In addition, orc pushdown will ensure that only the required data is returned in most cases. 另外你可以关闭executeFilter，因为开启它会导致读变慢，另外orc下推大多数情况下会保证只返回需要的数据，

是否是我这个表述存在问题， 我实际使用过程中发现 ORC 谓词下推能力更强， parquet 的谓词下推没有推到 column page 级别， 我最近这两天正在研究修复这个问题， 打算将 读取 parquet 的过滤谓词推到 page 级别， 减少不必要的数据扫描

In addition, you can turn off executeFilter, because turning it on will cause reading to slow down. In addition, orc pushdown will ensure that only the required data is returned in most cases. 另外你可以关闭executeFilter，因为开启它会导致读变慢，另外orc下推大多数情况下会保证只返回需要的数据，

是否是我这个表述存在问题， 我实际使用过程中发现 ORC 谓词下推能力更强， parquet 的谓词下推没有推到 column page 级别， 我最近这两天正在研究修复这个问题， 打算将 读取 parquet 的过滤谓词推到 page 级别， 减少不必要的数据扫描
Is there an issue with my statement? In my practical usage, I've found that ORC's predicate pushdown capability is stronger. Parquet's predicate pushdown hasn't reached the column page level. I've been working on a solution to this problem for the past couple of days, aiming to push down the filtering predicates of Parquet reads to the page level to reduce unnecessary data scans.

I haven't done much research on the related implementation of Parquet. All our optimizations are centered around this ORC. I have tested that the default Paimon table configuration will not actually push the filter conditions to ORC for execution, so I raised PR #4231 to fix this problem.
I'm not sure if Parquet also successfully pushed down, if not you can try to fix it.
I see that your test takes 1 second to read the ORC cycle 30 times. I wonder if your disk is an SSD? After turning on the above two parameters, it will theoretically be faster, only about 100ms? I don't have time to test it yet, but I can test it when I have time.
Parquet的相关实现我没有做过多研究，我们的全部优化都是围绕这ORC展开的。我测试过，默认的Paimon表配置是不会真正的把filter条件下推给ORC执行的，因此我提了pr #4231修复了这个问题。
我不确定Parquet是否也成功下推了，如果没有你也可尝试修复一下。
我看你的测试读取ORC循环30次需要1s，不知道你的磁盘是ssd吗？开启了上面的两个参数后理论上要更快一些，大约只需要100ms以内？我暂时没有时间测试，有时间了我也可以测试一下。

I haven't done much research on the related implementation of Parquet. All our optimizations are centered around this ORC. I have tested that the default Paimon table configuration will not actually push the filter conditions to ORC for execution, so I raised PR #4231 to fix this problem. I'm not sure if Parquet also successfully pushed down, if not you can try to fix it. I see that your test takes 1 second to read the ORC cycle 30 times. I wonder if your disk is an SSD? After turning on the above two parameters, it will theoretically be faster, only about 100ms? I don't have time to test it yet, but I can test it when I have time.
Parquet的相关实现我没有做过多研究，我们的全部优化都是围绕这ORC展开的。我测试过，默认的Paimon表配置是不会真正的把filter条件下推给ORC执行的，因此我提了pr #4231修复了这个问题。 我不确定Parquet是否也成功下推了，如果没有你也可尝试修复一下。 我看你的测试读取ORC循环30次需要1s，不知道你的磁盘是ssd吗？开启了上面的两个参数后理论上要更快一些，大约只需要100ms以内？我暂时没有时间测试，有时间了我也可以测试一下。

Thank you for your suggestion. We will carefully consider it and try to implement it.
感谢您的意见，我们这边尝试一下。

Writing this way ensures that the data filtered by orc is equivalent to the given filter condition. But it also includes the plan time, so the comparison with the above is not accurate.
这样写可以确保orc过滤的数据和给定的filter条件等价。但是又包含了plan的时间，所以和上面对比不准确。
        Table table = TableUtil.getTable();   // PrimaryKeyFileStoreTable
        PredicateBuilder builder = new PredicateBuilder(
                RowType.of(DataTypes.INT(),
                        DataTypes.STRING(),
                        DataTypes.STRING()));

        int[] projection = new int[] {0, 1, 2};

        ReadBuilder readBuilder = table.newReadBuilder()
                .withProjection(projection);

        Random random = new Random();

        long startTime = System.currentTimeMillis();

        for(int i = 0 ; i < 30 ; i ++){
            InnerTableRead read = (InnerTableRead)readBuilder.newRead();
            int key = random.nextInt(4000000);

            Predicate keyFilter = builder.equal(0, key);

            InnerTableScan tableScan = (InnerTableScan) readBuilder
                    .withFilter(keyFilter)
                    .newScan();
            InnerTableScan innerTableScan = tableScan.withFilter(keyFilter);
            TableScan.Plan plan = innerTableScan.plan();
            List<Split> splits = plan.splits();

            read.withFilter(keyFilter);//.executeFilter();
            RecordReader<InternalRow> reader = read.createReader(splits);

            reader.forEachRemaining(internalRow -> {

                int f0 = internalRow.getInt(0);
                String f1 = internalRow.getString(1).toString();
                String f2 = internalRow.getString(2).toString();
                System.out.println(String.format("%d - {%d, %s, %s}",key, f0, f1, f2));
            });
        }
        long stopTime = System.currentTimeMillis();
        System.out.println("time : " + (stopTime - startTime));

Writing this way ensures that the data filtered by orc is equivalent to the given filter condition. But it also includes the plan time, so the comparison with the above is not accurate.
这样写可以确保orc过滤的数据和给定的filter条件等价。但是又包含了plan的时间，所以和上面对比不准确。
        Table table = TableUtil.getTable();   // PrimaryKeyFileStoreTable
        PredicateBuilder builder = new PredicateBuilder(
                RowType.of(DataTypes.INT(),
                        DataTypes.STRING(),
                        DataTypes.STRING()));

        int[] projection = new int[] {0, 1, 2};

        ReadBuilder readBuilder = table.newReadBuilder()
                .withProjection(projection);

        Random random = new Random();

        long startTime = System.currentTimeMillis();

        for(int i = 0 ; i < 30 ; i ++){
            InnerTableRead read = (InnerTableRead)readBuilder.newRead();
            int key = random.nextInt(4000000);

            Predicate keyFilter = builder.equal(0, key);

            InnerTableScan tableScan = (InnerTableScan) readBuilder
                    .withFilter(keyFilter)
                    .newScan();
            InnerTableScan innerTableScan = tableScan.withFilter(keyFilter);
            TableScan.Plan plan = innerTableScan.plan();
            List<Split> splits = plan.splits();

            read.withFilter(keyFilter);//.executeFilter();
            RecordReader<InternalRow> reader = read.createReader(splits);

            reader.forEachRemaining(internalRow -> {

                int f0 = internalRow.getInt(0);
                String f1 = internalRow.getString(1).toString();
                String f2 = internalRow.getString(2).toString();
                System.out.println(String.format("%d - {%d, %s, %s}",key, f0, f1, f2));
            });
        }
        long stopTime = System.currentTimeMillis();
        System.out.println("time : " + (stopTime - startTime));

可以获取您的联系方式吗？ 咨询个问题

The test result I used with the new code is that, including the planning time, the Orc format took 1.77s and the Parquet format took 12.95s.
我用新的代码测试结果是，在包含plan时间的情况下，Orc格式耗时1.77s，Parquet的耗时12.95s。
Orc table as
{
  "version" : 2,
  "id" : 0,
  "fields" : [ {
    "id" : 0,
    "name" : "f0",
    "type" : "INT NOT NULL"
  }, {
    "id" : 1,
    "name" : "f1",
    "type" : "STRING"
  }, {
    "id" : 2,
    "name" : "f2",
    "type" : "STRING"
  } ],
  "highestFieldId" : 2,
  "partitionKeys" : [ ],
  "primaryKeys" : [ "f0" ],
  "options" : {
    "bucket" : "1",
    "file.format" : "orc",
    "manifest.compression" : "null",
    "orc.reader.filter.use.selected":"true",
    "orc.reader.sarg.to.filter":"true"
  },
  "timeMillis" :<PHONE_NUMBER>602
}

Parquet
{
  "version" : 2,
  "id" : 0,
  "fields" : [ {
    "id" : 0,
    "name" : "f0",
    "type" : "INT NOT NULL"
  }, {
    "id" : 1,
    "name" : "f1",
    "type" : "STRING"
  }, {
    "id" : 2,
    "name" : "f2",
    "type" : "STRING"
  } ],
  "highestFieldId" : 2,
  "partitionKeys" : [ ],
  "primaryKeys" : [ "f0" ],
  "options" : {
    "bucket" : "1",
    "file.format" : "parquet",
    "manifest.compression" : "null"
  },
  "timeMillis" :<PHONE_NUMBER>602
}

After checking, after the filter is passed to orc, each query orc returns only 1 or 0 pieces of data, while basically all data is returned in Parquet format.
This should be the root cause of Parquet's slow implementation.
经过检查，filter传递给orc后，每个查询orc只返回1条或者0条数据，而Parquet格式下基本把所有数据都返回了。
这应该是Parquet实现慢的根本原因。
Reader code as this :
        Table table = TableUtil.getTable();   // PrimaryKeyFileStoreTable
        PredicateBuilder builder = new PredicateBuilder(
                RowType.of(DataTypes.INT(),
                        DataTypes.STRING(),
                        DataTypes.STRING()));

        int[] projection = new int[] {0, 1, 2};

        ReadBuilder readBuilder = table.newReadBuilder()
                .withProjection(projection);

        Random random = new Random();

        long startTime = System.currentTimeMillis();

        for(int i = 0 ; i < 30 ; i ++){
            InnerTableRead read = (InnerTableRead)readBuilder.newRead();
            int key = random.nextInt(4000000);

            Predicate keyFilter = builder.equal(0, key);

            InnerTableScan tableScan = (InnerTableScan) readBuilder
                    .withFilter(keyFilter)
                    .newScan();
            InnerTableScan innerTableScan = tableScan.withFilter(keyFilter);
            TableScan.Plan plan = innerTableScan.plan();
            List<Split> splits = plan.splits();

            read.withFilter(keyFilter);//.executeFilter();
            RecordReader<InternalRow> reader = read.createReader(splits);

            reader.forEachRemaining(internalRow -> {

                int f0 = internalRow.getInt(0);
                String f1 = internalRow.getString(1).toString();
                String f2 = internalRow.getString(2).toString();
                System.out.println(String.format("%d - {%d, %s, %s}",key, f0, f1, f2));
            });
        }
        long stopTime = System.currentTimeMillis();
        System.out.println("time : " + (stopTime - startTime));

@Aiden-Dong

@ranxianglei
parquet 当前版本过滤时只将过滤谓词下推到了 rowgroup 级别，我最近正在优化这块 :
https://github.com/apache/paimon/pull/4608
The current version of Parquet only pushes filter predicates down to the RowGroup level during filtering. I’m currently working on optimizing this area.

@Aiden-Dong 有最新的测试吗？不知道效果如何 Are there any latest tests? Don't know how effective it is

@Aiden-Dong 有最新的测试吗？不知道效果如何 Are there any latest tests? Don't know how effective it is

我刚刚做了个本地测试，将测试结果补充到PR下面去了
I just ran a local test and added the results to the PR.


fixed linting errors

Wanna tag with patch?

sure, how ? :)

:(
remote: Permission to schubergphilis/awsapilib.git denied to nikoizs.

Added you as maintainer, it should work now, I thought you were already one, sorry!

You need to accept the invitation, check your email

Added nikoizs too, you might want to clear things up with your profiles. Please try again and tomorrow we figure out the profile part.


(PUP-5055) parent attributes are set from metadata too early in static_compiler

In the static compiler we are setting the mode, owner and group
attributes on the parent too early. This is causing child
resources to inherit the same values.
Child resources should only inherit attribute values if they are
explicitly set on the parent resource. In other cases overriding
the child group, mode and owner attributes should depend on the
value of source_permissions.
This commit attempts to fix this by postponing the call to set the
parent's attributes until after all children have inherited.
It also prevents mode, owner and group from being set if
source_permissions is not set to 'use'.

Looks like this causes another issue with the following case:
  concat { '/tmp/concatenated.txt':
      owner => 'root',
      group => 'root',
      mode  => '0644',
    }
    concat::fragment { 'concatenated_test_header':
      target => '/tmp/concatenated.txt',
      source => 'puppet:///modules/role/f5/opt/test.txt',
      order  => 01,
    }

2015-08-21 16:27:13 +0200 Puppet (err): Parameter alias failed on File[/var/lib/puppet/concat/_tmp_concatenated.txt/fragments/01_concatenated_test_header]: /var/lib/puppet/concat/_tmp_concatenated.txt/fragments/01_concatenated_test_header can not create alias concat_fragment_concatenated_test_header: object already exists at /etc/puppet/environments/production/modules/concat/manifests/fragment.pp:132
/usr/lib/ruby/vendor_ruby/puppet/util/errors.rb:106:in fail' /usr/lib/ruby/vendor_ruby/puppet/type.rb:1403:in block (3 levels) in class:Type'
/usr/lib/ruby/vendor_ruby/puppet/type.rb:1400:in each' /usr/lib/ruby/vendor_ruby/puppet/type.rb:1400:in block (2 levels) in class:Type'
/usr/lib/ruby/vendor_ruby/puppet/parameter.rb:432:in munge' /usr/lib/ruby/vendor_ruby/puppet/parameter.rb:502:in value='
/usr/lib/ruby/vendor_ruby/puppet/type.rb:654:in []=' /usr/lib/ruby/vendor_ruby/puppet/type.rb:2272:in block in set_parameters'
/usr/lib/ruby/vendor_ruby/puppet/type.rb:2266:in each' /usr/lib/ruby/vendor_ruby/puppet/type.rb:2266:in set_parameters'
/usr/lib/ruby/vendor_ruby/puppet/type.rb:2222:in initialize' /usr/lib/ruby/vendor_ruby/puppet/type/file.rb:481:in initialize'
/usr/lib/ruby/vendor_ruby/puppet/resource.rb:349:in new' /usr/lib/ruby/vendor_ruby/puppet/resource.rb:349:in to_ral'
/home/tdevelioglu/git/forks/puppet/lib/puppet/indirector/catalog/static_compiler.rb:95:in replace_metadata' /home/tdevelioglu/git/forks/puppet/lib/puppet/indirector/catalog/static_compiler.rb:82:in find_and_replace_metadata'
/home/tdevelioglu/git/forks/puppet/lib/puppet/indirector/catalog/static_compiler.rb:54:in block in find' /home/tdevelioglu/git/forks/puppet/lib/puppet/indirector/catalog/static_compiler.rb:43:in each'
/home/tdevelioglu/git/forks/puppet/lib/puppet/indirector/catalog/static_compiler.rb:43:in find' /usr/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:201:in find'
/usr/lib/ruby/vendor_ruby/puppet/network/http/api/v1.rb:105:in do_find' /usr/lib/ruby/vendor_ruby/puppet/network/http/api/v1.rb:50:in block in call'
/usr/lib/ruby/vendor_ruby/puppet/context.rb:64:in override' /usr/lib/ruby/vendor_ruby/puppet.rb:244:in override'
/usr/lib/ruby/vendor_ruby/puppet/network/http/api/v1.rb:49:in call' /usr/lib/ruby/vendor_ruby/puppet/network/http/route.rb:82:in block in process'
/usr/lib/ruby/vendor_ruby/puppet/network/http/route.rb:81:in each' /usr/lib/ruby/vendor_ruby/puppet/network/http/route.rb:81:in process'
/usr/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:63:in block in process' /usr/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in profile' /usr/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:61:in process'
/usr/lib/ruby/vendor_ruby/puppet/network/http/webrick/rest.rb:37:in block in service' /usr/lib/ruby/vendor_ruby/puppet/network/http/webrick/rest.rb:36:in synchronize'
/usr/lib/ruby/vendor_ruby/puppet/network/http/webrick/rest.rb:36:in service' /usr/lib/ruby/2.1.0/webrick/httpserver.rb:138:in service'
/usr/lib/ruby/2.1.0/webrick/httpserver.rb:94:in run' /usr/lib/ruby/vendor_ruby/puppet/network/http/webrick.rb:33:in block (2 levels) in listen'
/usr/lib/ruby/2.1.0/webrick/server.rb:295:in call' /usr/lib/ruby/2.1.0/webrick/server.rb:295:in block in start_thread'

We need to have further design/discussion around the source_permission logic. If you pull that out and narrow the scope of this to just fixing the way permissions are propagated to child resources it'd be :+1: for merge.
It also looks like this needs a rebase

@branan
done and done  :+1:


Ugly fix to apply theme on neoboard component

:heavy_check_mark: Checklist

[ ] A changeset describing the change and affected packages (more info).
[ ] Added or updated documentation.
[ ] Tests for new functionality and regression tests for bug fixes.
[ ] Screenshots or videos attached (for UI changes).
[X] All your commits have a Signed-off-by line in the message (more info).

Buttons should have a white background

Also the dashboard still has the wrong background colour

Closing this as another (better) solution has been found.


Items all selected by default

The multi-select box is working great, but it seems that when a page is reloaded, all of the options are selected by default. Is there a way to make it so by default none are selected?

The format for adding the options is ,
multiSelect: [
{value: false, label: "aaa", id: 1},
{value: false, label: "bbb", id: 2}
]
Value: true/false - is the selection criteria.

It would be helpful if you can mention the type in the readme file.

How can I get the default selected values list from the options?

How can I get the default selected values list from the options?

Change the "value" attribute to "true" for the options you want to show selected by default.
multiSelect: [
{value: true, label: "aaa", id: 1},
{value: false, label: "bbb", id: 2}
]


Create what-is-mining.md

Short and simple introduction on what is mining.

Thanks @LavRadis , added this to https://lbry.io/faq/block-rewards and moved it to the top of the list under the mining category!


Help request: how to do infinite tables

I have been trying to rework the hacker news example code to use tables, and am failing to get it to work.  Is there any example I can look at, or is infinite scroll not suited to tables?  Thanks,

Could you provide a Svelte REPL with your current implementation? Maybe I can try to get it working.

Certainly, thanks.  This is a total mess though, sorry
https://svelte.dev/repl/e02305d6ddf341f5ae011d031a11f579?version=3.44.1

Sorry I couldn't respond yesterday, was busy.
Anyways:
Here is your fixed REPL: https://svelte.dev/repl/70aa77f9aff048bebedef20ad68e4eae?version=3.44.2
I think you main problem was using list.push(...) instead of having an assignment like list = [...list, newItem]. See here for more info.
Then, for some reason, putting the <InfinteLoading /> component inside a <tr> doesn't work. I've changed it to <div> and then it actually appeared. Probably an issue that could be solved with CSS - or just use a <div>.
Then, always add let:style {style} to the element with slot="item". This makes sure the items are rendered correctly.
There are some other improvements, like putting <tbody> outside the virtual list, and adding width: 100% to the .tall class. But these aren't completely essential.
 But I'm too lazy to make it better and also moved on to some other projects 😅
Sometimes, tho, the documentation of the original Vue.js library might help: vue-infinite-loading
I hope that helps!

Wow, this is unbelievable, thanks!  Works like a charm

Happy to help! 😊


Make state persistent

Right now, when starting all projections are stopped. We should make the state persistent so it knows what to do on start.
This way we dont have to hardcode "start", causing different problems (like starting projections which shouldnt be started)
So we need to persist the state on change, and read from it when starting.

Blocking for BRV-109

https://github.com/Informatievlaanderen/projector/commit/867a9ca90fb6d7843488db4fe25d8c45557be27c


No module found huggingface_hub.snapshot_download in 0.11.0

Describe the bug
We have the sentence_transformer version: 2.22.0 and it is by default downloading the huggingface_hub version 0.11.0
But it is giving error.
We tried using huggingface_hub==0.10.0 and it is working fine.
Reproduction
install sentence-transformers==2.22.0
Logs
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/sentence_transformers/__init__.py", line 3, in <module>
    from .datasets import SentencesDataset, ParallelSentencesDataset
  File "/usr/local/lib/python3.8/site-packages/sentence_transformers/datasets/__init__.py", line 3, in <module>
    from .ParallelSentencesDataset import ParallelSentencesDataset
  File "/usr/local/lib/python3.8/site-packages/sentence_transformers/datasets/ParallelSentencesDataset.py", line 4, in <module>
    from .. import SentenceTransformer
  File "/usr/local/lib/python3.8/site-packages/sentence_transformers/SentenceTransformer.py", line 25, in <module>
    from .evaluation import SentenceEvaluator
  File "/usr/local/lib/python3.8/site-packages/sentence_transformers/evaluation/__init__.py", line 5, in <module>
    from .InformationRetrievalEvaluator import InformationRetrievalEvaluator
  File "/usr/local/lib/python3.8/site-packages/sentence_transformers/evaluation/InformationRetrievalEvaluator.py", line 6, in <module>
    from ..util import cos_sim, dot_score
  File "/usr/local/lib/python3.8/site-packages/sentence_transformers/util.py", line 407, in <module>
    from huggingface_hub.snapshot_download import REPO_ID_SEPARATOR
ModuleNotFoundError: No module named 'huggingface_hub.snapshot_download'

System info
huggingface_hub version: 0.11.0
sentence_transformers version: 2.22.0

Hi @josephsachdeva, sorry for the inconvenience. snapshot_download module has been removed from huggingface_hub in favor of a private _snapshot_download module. A deprecation warning should have been triggered since a few versions now to warn about this.
The workaround here is to import REPO_ID_SEPARATOR from the constants module:
from huggingface_hub.constants import REPO_ID_SEPARATOR

Otherwise, you can temporarily pin huggingface_hub version to ==0.10 in a hot-fix, while you update the code in parallel. For the record, huggingface_hub==0.11.0 has been released on Wednesday (16/11/2022).

Hi @Wauplin thanks for replying!
Actually we are not using huggingface_hub directly, it is being used in sentence_transformers. And i think the workaround you suggested will be done by sentence_transformers devs.
But yes, we have downgraded huggingface_hub to 0.10.0 and it is working fine for us.
Thanks.

@josephsachdeva actually the problem is already fixed in sentence_transformers. If possible for you, I'd advice to update to latest version. See related issue https://github.com/UKPLab/sentence-transformers/issues/1599#issuecomment-1164371279.


MAAS-4459 | Get project exec path via env var

Injected alamo-worker executable path via environment variable. This is required to start a subprocess when the main process is run via supervisord.
Added new env vars to the Dockerfile.

Coverage remained the same at ?% when pulling ffd6fc6c354d97a11268c6ddd1f0acc3ba3a15ec on MAAS-4459-health-checks into 83bfd254b7dfb78dfea06a2e70ef0c0e806e8064 on develop.


Add graffiti API

Add keymanager API to manage graffiti per validator.
The rationale for this is similar as for gas limit (https://github.com/ethereum/keymanager-APIs/pull/39), the graffiti is already part of proposer config file and it should be quite simple to implement (https://github.com/ChainSafe/lodestar/pull/6083).

It would be awesome to have this! In Dappnode we could add the feature of editing the graffiti individually from the Web3Signer package UI!

It would be awesome to have this! In Dappnode we could add the feature of editing the graffiti individually from the Web3Signer package UI!

Given web3signer is only signing (not creating blocks) it won't be able to control graffiti, you'd still need to have the CL with keymanager and control the graffiti through that...

Any further comments? Looks like at least Prysm/teku happy... @michaelsproul  or @jimmygchen , @dapplion, @nflaig  ?


Similarity score

Thank you for sharing this repo!
I am trying to use this to do Facial Recognition by comparing similarity scores. For each frame, do I use
https://github.com/mk-minchul/caface/blob/e94ad0206d0a04a098594747138e4da5435f5ec7/demo/visualization.py#L20
or
https://github.com/mk-minchul/caface/blob/e94ad0206d0a04a098594747138e4da5435f5ec7/demo/visualization.py#L21
between a detected face and reference images?

should be indv similarity, fused similarity just gives the average input of all the fused similarity scores, it is used as an average benchmark to compare what's w CAFace and what's w/o CAFace as shown in the paper

should be fused_similarity


Palatino Font Package error

Background
Used Quarto in RStudio to render the example.qmd quarto document after installing tinytext but do not have MikText or MacTeX installed.
Problem Description
an error pops up that says RStudio can not find the Palatino Font package.
Replication Steps
opened eample.qmd and hit render
Possible Solutions
Went to an openscapes coworking session and someone with MikText with the qualitype package ran the same code and it worked for her. Our next step will be to ask IT to download MikTex or Tex Live. Having no experience with LaTeX at all, is one of those programs better to download then the other?
Thanks for your help!

I just fixed this this afternoon. The font is called "Palatino Linotype" on Windows while it is called "Palatino" on Mac. I just cut out the Palatino font reference altogether.
To update to the newest version of quarto_titlepages run from a terminal in your project. So if your RStudio project is at C:/Users/MM/Document/MyProject you want to open a terminal window and cd to that directory.
quarto install extension nmfs-opensci/quarto_titlepages

@mmccorml Regarding what TeX to install, you do not want MikTeX or TeX Live! For making PDFs with R Markdown or Quarto (in RStudio) you want tinytex. You should be able to install if right from RStudio, however before you do that, you may have tinytex already since Quarto may bundle it.  Update the quarto_titlepages and try to render again.
If it says it doesn't have TeX, then run the following from the R command line:
install.packages('tinytex')
tinytex::install_tinytex()

Okay great, thank you for your help!


pre-commit hooks for Cookbooks?

I'm confused about the current state of our reliance on pre-commit for code quality.
In Foundations we have pre-commit.ci running on every PR. This service runs as a GitHub App linked to the Foundations repo.
I don't think we've ever turned this on for any Cookbook repos.
However as of https://github.com/ProjectPythia/cookbook-template/pull/103 the Template repo has a .pre-commit-config.yaml, and pre-commit was run locally once on the repo.
All subsequent copies of the Template have the .pre-commit-config.yaml file included, but we're using it for anything.
It would be great to get this cleaned up for consistency. I see three options:

Get rid of the .precommit-config.yaml file, don't enforce any code style
Add guidance to our Contributor's Guide on how to install and run pre-commit locally on a Cookbook repo
Switch on the pre-commit.ci service for all existing Cookbook repos, and add guidance on how to do this for new Cookbooks.

Option 3 results in the strictest adherence to consistent code quality and formatting, with some potential for user confusion.
Option 1 is the easiest.
Option 2 is a wishy-washy bad compromise in my opinion.
@ProjectPythia/infrastructure thoughts?

Maybe there's a better compromise:

Switch on pre-commit.ci only for the Template repo. This will at least enforce some style guidance on future changes to the Template, which will avoid propagating poor style into future new Cookbooks -- but spare new Cookbook authors from dealing with pre-commit at all.

I think this was the intent with #103 but we never followed through and didn't switch on the service.

I think cookbook template pre-commit is failing:
An unexpected error has occurred: CalledProcessError: command: ('/Users/ahuang/.cache/pre-commit/repogb5xyvpd/py_env-python3.11/bin/python', '-mpip', 'install', '.')
return code: 1
stdout:
    Processing /Users/ahuang/.cache/pre-commit/repogb5xyvpd
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
      Preparing metadata (pyproject.toml): started
      Preparing metadata (pyproject.toml): finished with status 'error'
stderr:
      error: subprocess-exited-with-error

      × Preparing metadata (pyproject.toml) did not run successfully.
      │ exit code: 1
      ╰─> [17 lines of output]
          Traceback (most recent call last):
            File "/Users/ahuang/.cache/pre-commit/repogb5xyvpd/py_env-python3.11/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
              main()
            File "/Users/ahuang/.cache/pre-commit/repogb5xyvpd/py_env-python3.11/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
              json_out['return_val'] = hook(**hook_input['kwargs'])
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            File "/Users/ahuang/.cache/pre-commit/repogb5xyvpd/py_env-python3.11/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
              return hook(metadata_directory, config_settings)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            File "/private/var/folders/8h/mk8mz7qn3_325ztm7pgb4ld00000gp/T/pip-build-env-ptgrtmca/overlay/lib/python3.11/site-packages/poetry/core/masonry/api.py", line 41, in prepare_metadata_for_build_wheel
              poetry = Factory().create_poetry(Path(".").resolve(), with_groups=False)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            File "/private/var/folders/8h/mk8mz7qn3_325ztm7pgb4ld00000gp/T/pip-build-env-ptgrtmca/overlay/lib/python3.11/site-packages/poetry/core/factory.py", line 58, in create_poetry
              raise RuntimeError("The Poetry configuration is invalid:\n" + message)
          RuntimeError: The Poetry configuration is invalid:
            - [extras.pipfile_deprecated_finder.2] 'pip-shims<=0.3.4' does not match '^[a-zA-Z-_.0-9]+$'

          [end of output]

      note: This error originates from a subprocess, and is likely not a problem with pip.
    error: metadata-generation-failed

    × Encountered error while generating package metadata.
    ╰─> See above for output.

    note: This is an issue with the package mentioned above, not pip.
    hint: See above for details.
Check the log at /Users/ahuang/.cache/pre-commit/pre-commit.log

I got that same error locally. It was resolved by updating some package versions in the .pre-commit-config.yaml file in #121.

For now I've chosen to go with option 4. pre-commit.ci is now running on this repo. Some of the cookbook repos have an out-of-date .pre-commit-config.yaml that should be updated. Turning pre-commit.ci on for those repos would mean that auto-update PRs get generated, but may also cause confusion.
We can come back to this later!

We discussed some pros and cons of running pre-commit on cookbook repos at today's IWG.
The main pro argument is for ease of maintenance. Sticking to a consistent code style should lower the barrier for other folks aside from the original author to help with maintenance of notebooks in cookbooks.
The main con is the additional complexity and consequent need to write more guidance for cookbook authors.
For now we are taking no action but leaving the above compromise (option 4) in place.


elm-package.json does not allow BitBucket repositories

Error reading file elm-package.json:
    Problem with the `repository` field.

The given domain does not start with <https://github.com/>

Is there something about Elm that requires Github?  I use Github and Bitbucket interchangeably for different reasons and as far as I can tell they are nearly 100% compatible.

Also, if I can't nominate a Bitbucket repo, I'd rather just leave this out but it's a required field:
Error reading file elm-package.json:
    Missing field "repository", a link to the project's GitHub repo.
    Check out an example elm-package.json file here:
    <https://raw.githubusercontent.com/evancz/elm-html/master/elm-package.json>

@proehlen, please close this issue. It's a duplicate. See #59 and #71.

Apologies @jvoigtlaender - I didn't see 71.


Errors: unimplemented OBJ format statement 's' on line 's 1' AND convert failed: 6 columns passed, passed data had 3 columns

I've exported the OBJ file from Blender, it uses vertex colors and has no MTL file.
wtest2.txt <---change the extension on this file to obj.

@jkerman2000
Hi 👋
According to blender manual, colored .obj file which exported from blender must includes .mtl file.
Please check your blender setting!🥺

what do you mean im getting the same error and i have a mtl file in the same directory as the obj

untitled.zip
heres my obj and my mtl

okay so i fixed the problem with unimplemented OBJ format statement 's' on line 's 1', but now i am getting this
convert failed:  mapping file does not exists
Traceback (most recent call last):
File "Obj2SchemticConverter.py", line 241, in
args.obj_file, args.output_dir, args.h_max, args.w_max)
File "Obj2SchemticConverter.py", line 38, in init
self.df = self._create_obj_df(obj_path)
File "Obj2SchemticConverter.py", line 46, in _create_obj_df
vertices = self._analyze_mtl(obj_path, scene)
File "Obj2SchemticConverter.py", line 62, in _analyze_mtl
raise Exception('mapping file does not exists')
Exception: mapping file does not exists

@jajaradebela
You should add texture image path to your map_Kd line.
(.mtl)
# Blender MTL File: 'None'
# Material Count: 1

newmtl Default_OBJ
Ns 323.999994
Ka 1.000000 1.000000 1.000000
Kd 0.800000 0.800000 0.800000
Ks 0.500000 0.500000 0.500000
Ke 0.000000 0.000000 0.000000
Ni 1.000000
d 1.000000
map_Kd ./your_texture_image.png
illum 2


local type issues

Pull Request Test Coverage Report for Build 4172
Details

0 of 0   changed or added relevant lines in 0 files are covered.
25 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.5%) to 80.367%

Files with Coverage Reduction
New Missed Lines
%

result/result.mbt
1
75.86%

rational/rational.mbt
24
54.55%

Totals

Change from base Build 4170:
-0.5%

Covered Lines:
4466

Relevant Lines:
5557

💛  - Coveralls


Notifications Not Properly Working

I am running 1.8.69 using the PHP 8.2 version (:latest).
I have the Mentions module.  These are the only notifications that I receive.
Under my profile (/users/notifications/), I have all Browser Notifications turned on except for the following:

A conversation is assigned to someone else
To an unassigned conversation (both)
To a conversation owned by someone else (both)

I have not once received a notification of a new conversation or that a conversation has been assigned to me.  All users report the same.

This might be a good issue for the main Freescout repo. Otherwise, you could try running the PHP 8.1 or 8.0 variant of the image and see if it happens still. You can see here how I name the other versions: https://hub.docker.com/repository/docker/tiredofit/freescout/tags?page=1&ordering=last_updated


Profiling report

Test Pypy vs cPython
Measured using werkzeug.middleware.profiler
Calling is_alive and then mint function

first call after the init is the slowest, consequential calls become faster until a time of 1-2s on average
async primitive fill thread pool and with each thread aquire thread lock which has the longest time
potential python GIL lock limitation
didn't notice a performance boost by pypy at all, sometimes performed even worse
Replacing Flask with Quart and WSGI with ASGI produces huge performance loss for this test

To swith to pypy first install it on you machine than do poetry env use pypy
devent-pypy-report.txt
devent-cpy-report.txt

cPython cProfiler
First call /is_alive then consequential /mint run
devenet-profile-mint.txt
Pofiler Files, can be visualized with 'snakeviz'
mint-run-1.TXT
mint-run-2.TXT

pyinstrument cPithon
Profile from start of mint call to its end.
profile-mint-call.TXT
Profile from App start to end of mint call:
devnet-first-call.TXT
Rename files to .html and open to see results

Related: #52

Minting is relatively light operation. Have you tried using something more computationally intensive?

didn't notice a performance boost by pypy at all, sometimes performed even worse
Minting is a relatively light operation. Have you tried using something more computationally intensive? Team working on Cleopatra VM posts benchmarks often and CPython is much slower than PyPy there.


Unable to deploy log analytics workspace together with rg when restricting location based off resource group

Bicep version
Bicep CLI version 0.4.1272 (a69022dd2c)
Describe the bug
When deploying a Bicep template with a resource group and a log analytics workspace, if a policy is assigned which checks if a resources location matches it's parent resource groups location, the policy assignment will deny deployment.
This policy assignment supports all manner of other types of resources created in the same deployment into newly created resource groups (app service, sql server and db, storage account, key vaults, vnet, and more).
Just not a LAW.
Additional context
main.bicep
targetScope = 'subscription'
resource rgOps 'Microsoft.Resources/resourceGroups@2021-01-01' = {
  name: 'rg'
  location: deployment().location
}
module auditWorkspace './auditWorkspace.bicep' = {
  name: 'AuditWorkspace'
  scope: rgOps
  params: {
    location: location
  }
}

auditWorkspace.bicep
param location string = resourceGroup().location

resource auditLogWorkspace 'Microsoft.OperationalInsights/workspaces@2020-08-01' = {
  name: 'audit-log'
  location: location
  properties: {
    sku: {
      name: 'PerGB2018'
    }
    retentionInDays: 730
  }
}

policy definition - already assigned (or created earlier during the same deployment)
resource ensureLocation 'Microsoft.Authorization/policyDefinitions@2020-09-01' = {
  name: 'Ensure resource location matches....'
  properties: {
    displayName: 'display name'
    policyType: 'Custom'
    mode: 'Indexed'
    description: 'description'
    metadata: {
      category: 'Location'
    }

    policyRule: {
      if: {
        allOf: [
          {
            field: 'location'
            notEquals: '[resourcegroup().location]'
          }
          {
            field: 'location'
            notEquals: 'global'
          }
        ]
      }
      then: {
        effect: 'deny'
      }
    }
  }
}

This does not look like a bicep issue, it looks like a policy issue. There may be someone here who can help, but it is not something everyone here has expertise in. You may want to open a support case, so you can be directed to more policy expertise.


Error during WebSocket handshake: Unexpected response code: 204

I'm attempting to use SignalR on my local machine, but it appears that IIS/Express on Windows 7 isn't supported.  What are my options for local machine developing?

The WebSockets transport will not work if the server hosting SignalR is running behind IIS on Windows 7 or Windows Server 2008 R2, due to limitations in IIS

In case I'm missing something and Express is supported here is the error I'm getting:

VM46:164 WebSocket connection to 'ws://localhost:60609/chat?id=9d69e64f-eed1-4b28-af25-a34b8a963ec9' failed: Error during WebSocket handshake: Unexpected response code: 204
WrappedWebSocket @ VM46:164
(anonymous) @ signalr-clientES5-1.0.0-alpha2-final.min.js:1
t @ signalr-clientES5-1.0.0-alpha2-final.min.js:1
value @ signalr-clientES5-1.0.0-alpha2-final.min.js:1
(anonymous) @ signalr-clientES5-1.0.0-alpha2-final.min.js:1
r @ signalr-clientES5-1.0.0-alpha2-final.min.js:1
(anonymous) @ signalr-clientES5-1.0.0-alpha2-final.min.js:1
e.(anonymous function) @ signalr-clientES5-1.0.0-alpha2-final.min.js:1
a @ signalr-clientES5-1.0.0-alpha2-final.min.js:1
signalr-clientES5-1.0.0-alpha2-final.min.js:1 Error: Failed to start the connection. undefined
home:1 Uncaught (in promise) undefined

I have deployed to azure and the code works fine.  What are my options to get this running locally?

Only WebSockets transport is not supported if you use IIS. You can either use other transports (ServerSentEvents, LongPolling) or skip IIS and use Kestrel directly in which case WebSockets should work.

I apologize for editing the title but I believe there is a bigger problem going on.  Even though web sockets are not supported with IIS Express, Signal R does not fall back to a different transport.  Even specifying the transport such as :

connection.start({ transport: ['longPolling', 'foreverFrame', 'serverSentEvents'] })
.then(() => connection.invoke('send', 'Hello'));

still causes the error to occur.  The code works fine in azure but I cannot get it working on the local machine.  I've tried this on 3 different machines (using windows 7) reproducing the same error.

There is no transport fallback in SignalR Core although @moozzyk has a sample making fallback work https://github.com/aspnet/SignalR-samples/pull/8
Also connection.start doesn't take any parameters. If you want to pass a transport to use it needs to be in the constructor:
let connection = new HubConnection("someurl", { transport: 'longPolling' });
connection.start().then(() => {});

And lastly, foreverFrame isn't a thing anymore.

Thank you!


Design Review - Resize wombat image on gameplay status bar

The wombat image should be resized so that the full image is shown. (Currently the wombat ears are cut off)

Looks good to me, moving to design review.


Flickering transition when upgrading to 4.2.0

Describe the bug
There is a brief flickering, where the  element associated with a transition is made visible, right before the transition takes place.
My use case is a bit more complex due to the use of MUI's useScrollTrigger hook, but managed to simplify it in the Codesandbox below. This started to happening when upgrading to 4.2.0, Using 4.1.0 seems ok.
To Reproduce
Steps to reproduce the behavior:

Open this CodeSandbox;
Scroll from top to bottom slowly until the "ANIMATED HEADER" is displayed;
It will flicker briefly before the transition happens.

Expected behavior
It should not flicker.
Desktop (please complete the following information):

OS: Win 10
Browser Chrome
Version 4.2.0

Hi, please update to the latest version (v4.2.3 at the time of writing this).
However, I see you are using the useScrollTrigger hook from "@mui/material" and then you conditionally render the animated header. This causes a layout shift and it is not necessary – if you want to control how much the element should be in the viewport before triggering the animation, you can use the fraction prop.

Hey @morellodev, thanks for the feedback. Using 4.2.3 still causes the same effect. I don't think fraction prop is applicable here as the component is being conditionally rendered due to a requirement. Eitherway, given that conditional rendering is messing up with the revealing animation I will just use the regular MUI transition for this specific case.


Need way to suppress "Trying ../justfile" output when fallback occurs

Ideally, this output would be suppressed unless the --verbose flag is given, as it is only useful for debugging.

I think that's reasonable. Now that it requires a setting, the user probably doesn't need to be notified that fallback is occurring.

Fixed in #1510!

Wow! You're fast. Thanks for the fix.


403 error for uploadFiles

in uploadFiles, "$apiUrlv6/uploadFiles" is working well but I got 403 error in this line.
how can I solve it?
it agent issue please help me

Fixed. The error was due to missing "X-Amz-Security-Token" in the request field.


Enhance Search, Filter, and Sort Functionality on Public Cloud Request Page

Description
Enhance the search, filter, and sort functionalities on the public cloud request page.
Technical Details

Search: Supports searching by licencePlate, name, description, and member name.

Filters (multi-select):

request types: default to all
decision status: default to pending
ministry: default to all
provider: default to all

Sort By: Name, description, licencePlate, status, created date, updated date, decision date, provisioned date

Each field supports both ASC and DESC sorting options.
Default: Updated date (DESC).

Definition of Done (DOD)

The search, filter, and sort functionalities operate as expected.

Search and filtering works as expected on the Public Cloud Requests tab


fix: add nodePort 31032 for postgresql

Codecov Report

Merging #153 (3ba2044) into master (feb59cb) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #153   +/-   ##
=======================================
  Coverage   95.45%   95.45%
=======================================
  Files           8        8
  Lines         506      506
=======================================
  Hits          483      483
  Misses         23       23

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update feb59cb...3ba2044. Read the comment docs.


[Radarr] SQP 2-4

Updated: SQP-2
NEW: SQP 3-4
If you are interested in these Special Quality Profiles join the #access-to-sqp on Discord

All looks good. Only question @TRaSH- is this a naming typo !radarr-unwanted.md or did you want the ! in the name?

Not a typo, it's going to replace the other one, but that won't happen before I changed it in the main guide.


Use dev build instead of daily builds

Daily builds are no longer published
Also we don't have gencheck on this repo so some random changes are
included that are generated.

@howardjohn: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

noauth_installer
e69f1a46595366206f81b45f96aa95cda0b15402
link
/test noauth_installer

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest


xcode 13.3 doesn't like the use of the keyword final when 'virtual` is not defined on the function

Bug Report
xcode 13.3 doesn't like the use of the keyword final when 'virtual` is not defined on the function. This exposed a small chain reaction of other small polymorphic issues.

closed by removing the prototype multiMediaReference h and cpp files.


added logo

I have added the favicon
please label under gssoc and hacktoberfest

@jayanththalla great work buddy

@muruga21 thank you can u please label the hacktoberfest accepted


[k8s] Add conf to allow setting environment variables on the driver

closed in favor of https://github.com/apache-spark-on-k8s/spark/issues/299


fix: don't pin the wrangler version in package.json

Closes #1127

I've reverted this since it was causing issues, causing extremely old versions of wrangler to be installed. I dunno why, but I'm reverting this to fix the build right now.


Wrong minimal Laravel version in package.json

Describe the bug
Trait Spatie\Activitylog\Traits\LogsActivity on line 371 tries to call getStorableEnumValue method on an Eloquent model. package.json file of the spatie/laravel-activitylog repository requires Laravel version ^8.0 || ^9.0 || ^10.0, but for example in Laravel 9.26.0 there was no getStorableEnumValue method implemented yet. It was added later. For example it is present in Laravel 9.52.15.
To Reproduce
Create a new Laravel 9.0.0 app (not newer version!) Then install Spatie Laravel Activity Log. Then reproduce steps here: https://spatie.be/docs/laravel-activitylog/v4/advanced-usage/logging-model-events
You will end up with the Call to undefined method App\\Models\\SomeModel::getStorableEnumValue() exception.
Expected behavior
It should automatically upgrade my Laravel app to proper version. 9.0.0 is not new enough. You should change the minimal Laravel version in package.json.
Versions (please complete the following information)

PHP: 8.1
Laravel: 9.26.0
Package: spatie/laravel-activitylog

Additional context
I faced this issue in Laravel 9.26.0. Then I simply upgraded my app with $ composer update. Then my Laravel version raised to 9.52.15 and the problem disappeared. So I suppose that the Laravel developers added getStorableEnumValue somewhere between version 9.26.0 and 9.52.15.

Besides the fact that we are talking about composer.json: go ahead and provide a PR fixing it. In best case with links to the PRs/Commits that introduced that method.

Hi @mspiderv , I've been looking at this issue and found that the bug was introduced in commit 1ad6f9840ea8166ce2b24b3d0d6da0f8728daef2 and then published in version 4.7.0 of the package. However, I also found that it was fixed in commit a92fde9270b5a26f78e0364bfc91d2ae2fcff249 and released in version 4.7.2 of the package. Therefore, it appears that you might have been using version 4.7.0 or 4.7.1.
@Gummibeer, you may want to take a look at the changes made to the file src\Traits\LogsActivity.php in commit a92fde9270b5a26f78e0364bfc91d2ae2fcff24, where this issue has already been resolved.


[Request] Add support for attaching TemplateType.CLI with input parameters

Description
In current implementation TemplateType.FEATURE allows attach template with parameterized values by adding device_specific_vars keyword. TemplateType.CLI should have the same option, but now it returns:

self.vmanage_session.api.templates.attach(template_name, machine_vmanage_obj, device_specific_vars=device_specific_vars)
File "/home/tester/.local/lib/python3.8/site-packages/vmngclient/api/template_api.py", line 92, in attach
return self._attach_cli(name, device, **kwargs)
TypeError: _attach_cli() got an unexpected keyword argument 'device_specific_vars'

This issue was migrated from an old repository: https://github.com/CiscoDevNet/catalystwan/issues/272


[Automation] Generate Fluent Lite from Swagger oracle#package-2023-09-01

[Automation] Generate Fluent Lite from Swagger oracle#package-2023-09-01

API change check
APIView has identified API level changes in this PR and created following API reviews.
com.azure.resourcemanager:azure-resourcemanager-oracledatabase

Close due to service definition changed after this PR.


typo: detele instead of delete

https://github.com/phpreel/phpreel/blob/9dcd852d8a268c6b396fc64db1ece75d148cdc0f/app/Helpers/Translation/TranslationHandler.php#L50

Thank you for opening up this issue. Fixed.


Error using RedPen (No output generated, even though output is produced)

Hi,
after installing redpen, and opening nvim, with ALE, we immediately get an error

Error detected while processing function 98_NeoVimCallback[29]..7[1]..94_Ex
itCallback[28]..93_HandleExit[39]..ale#handlers#redpen#HandleRedpenOutput:
line    3:
E474: Attempt to decode a blank string

*ALEInfo at this link.
This issue seems twofold. One one hand it is related to  this issue, where empty output is not handled properly. But the real problem is, that even though ALEInfo says that the command gave no output

(finished - exit code 255) ['/usr/bin/zsh', '-c', 'redpen -f markdown -r json ''/tmp/nvimW0LVr5/6/Project_report1stEdit.md''']
NO OUTPUT RETURNED

But if one runs from the same buffer

:!redpen -f markdown -r json %

one gets returned with plenty output. Could you please direct me how to try and fix this? Somehow ALE gets no output, when it should have.

To anybody experiencing the same Issue. The problem is in the install from the snap. When one uses the release from their git, everything works.


docs(vue-component-meta): updated README

As per our discord chat I have updated vue-component-meta readme according to the bast of my knowledge.
I hope you like it.

I haven't had a chance to use vue-component-meta yet but I'm assuming it's able to handle cases where prop is of Object type and more specific type is provided through JSDoc's @type? Would it be worth including an example for that?

It will be great for completion, but first, let me check if it's working fine in Anu as I don't have any playground to test it 😉
Will be back soon.
Thanks for your suggestions 👍🏻

Just an addendum to that... I meant an example using @type as that is what one would use in JS code. If inside TS code then it would be done with as Foo instead, I presume. You seem to be already assuming TS as you have one example using as. It might be beneficial to include examples for both TS and JS.

Just an addendum to that... I meant an example using @type as that is what one would use in JS code. If inside TS code then it would be done with as Foo instead, I presume. You seem to be already assuming TS as you have one example using as. It might be beneficial to include examples for both TS and JS.

You mean like this:
props: {
  /*
   *
   * Array of texts to render as a message
   * @type {string[]}
  */
  texts: {
    type: Array,
    required: false,
  }
}

Is this enough?
Sorry if I didn't get your idea.

You mean like this:

Yes, something like this although I believe the correct type to use would be be:
props: {
  /**
   * Array of texts to render as a message
   * @type {import('vue').PropOptions<string[]>}
   */
  texts: {
    type: Array,
    required: false,
  }
}

Thanks! Update the md.

@rchl Done 😇

Great work, thank you!


some data doubled when combining data sets with overlapping values

If I combine two data sets which have partial overlap, it seems that the non-overlapping intensities get doubled, though their times and monitor rates don't change. I don't have this problem if the data sets don't overlap.
import numpy as np
import neutronpy

Data = neutronpy.Data

h = np.asarray([0., 0., 0., 0.])
k = np.asarray([0., 0., 0., 0.])
l = np.asarray([0., 0., 0., 0.])
e = np.asarray([2., 2.5, 3., 4.])
temp = np.asarray([10.,10.,10.,10.])
detector = np.asarray([49., 41., 30., 20.])
monitor = np.asarray([100000., 100000., 100000., 100000.])
time = np.asarray([1., 1., 1., 1.])
data1 = Data(h=h, k=k, l=l, e=e, temp=temp, detector=detector, monitor=monitor,
        time=time)

h = np.asarray([0., 0., 0.])
k = np.asarray([0., 0., 0.])
l = np.asarray([0., 0., 0.])
e = np.asarray([2.5, 3., 4.])
temp = np.asarray([10., 10., 10.])
detector = np.asarray([44., 33., 22.])
monitor = np.asarray([100000., 100000., 100000.])
time = np.asarray([1., 1., 1.,])
data2 = Data(h=h, k=k, l=l, e=e, temp=temp, detector=detector, monitor=monitor,
        time=time)

data12 = data1 + data2

print(data1.intensity)
print('    ' + str(data2.intensity))
print(data12.intensity)

This gets printed:
[ 49.  41.  30.  20.]
    [ 44.  33.  22.]
[ 98.  85.  63.  42.]

This is a result of the (automatic) change in m0 (monitor used to normalize to) when you add the two datasets together. The error is where the additional statistics help though. So when you do this, the error for the non-overlapping points remains the same (in this case, simply doubled to account for the scaling of the intensity), but the error for the overlapping points is less, illustrating the increase in statistics. As far as I can tell this is behaving as expected, since the raw detector counts and monitor rates remain correct.


fix NPE in case if the Signup policy is disabled

fixed #109

Thanks! As for CLA, I think this can be classified as an obvious fix, so we don't need a signature. https://developer.okta.com/cla/


Make it possible to configure the folder of imported dashboards

Dashboards are currently always imported to the root folder in Grafana. I'd like to change that.

We are in the process of removing support for file provisioning of dashboards in this role and focusing on grafana_dashboards ansible plugin.

Ok, I'll move the provisioning to my add-on role.


test case for filtering null and not null on relation

As discussed, I've written a new test case for checking $null on relations, and changed the existing test case for checking properly on $not:$null on relations. The first test case is failing as expected/described within the issue. Since this is my first time working on this repo (and submitting a PR to an open source project in general), let me know if this is fine.
Issue https://github.com/ppetzold/nestjs-paginate/issues/538

Oh I see. I think the issue is simply that we always LEFT JOIN. So nulled relations will be listed as well. I suppose it must be INNER JOIN to achieve what you expect.
I am currently only working on a sequelize project. Sequelize has the require property on joins. Looking over TypeORM's docs I cannot find the equivalent. Do you know what TypeORMs notation for this is? If so, we can try to imitate.

The problem is that inside of createQueryBuilderRelations we always left join, even on non-terminal relations. This makes it that the home.street relation left joins home and left joins street, while it ought to inner join home and left join street so that we can then filter out the null street ones, without including the null home ones. I'll give that solution a try.

@ppetzold I fixed the null queries on relationships by storing that when we use a relationship in a filter that it should be inner joined. On top of that I allow the user to configure the default join method, and the join method per column, since this all converged on the same approach I needed to extend for building the relationships either way.


contiv-crd: Vagrantfile refactor to support contiv-crd

Modified Vagrantfile to support deployments with and without the use of Kubernetes CRDs.

Coverage decreased (-0.1%) to 74.991% when pulling d7a644cdcd051decdaadcdd8ab1f813817b08270 on brecode:node-crd into b3a7e00a150ac59ed5e28530564e7454be003104 on contiv:dev.


Fix: report msg not generated at the end of run

At the end of "leapp upgrade" run we do not print message
about generated reports, in contrast to "leapp preupgrade"
where we do.

@drehak @bocekm please take a look...

Can one of the admins verify this patch?

Tested, LGMT.

@examon @vojtechsokol Are these linter issues false positives or not?

When running leapp upgrade --debug, I now get:
============================================================
                           REPORT
============================================================

A report has been generated at /var/log/leapp/leapp-report.json
A report has been generated at /var/log/leapp/leapp-report.txt
A report has been generated at /var/log/leapp/leapp-upgrade.log

============================================================
                       END OF REPORT
============================================================

The /var/log/leapp/leapp-upgrade.log shouldn't be there. That's not a report.

If you run "leapp preupgrade --debug", what do you see?

@bocekm how do we move forward with this PR?

The /var/log/leapp/leapp-upgrade.log shouldn't be there. That's not a report. The debug log should not be of any interest to users by default.
@Rezney would like to have a bigger quorum, so @shaded-enmity, @pirat89, do you agree with removing it?

Agree with Michal. log file should not be there.

I don't mind the log file if we show it differently:
============================================================
                           REPORT
============================================================

A report has been generated at /var/log/leapp/leapp-report.json
A report has been generated at /var/log/leapp/leapp-report.txt

============================================================
                       END OF REPORT
============================================================

Debug output written to /var/log/leapp/leapp-upgrade.log


A problem about error: "inplace operation"

Hi, Chiaraplizz,
I'd like to consult a problem encountered in running the code.
When I start training process, following error has occured:

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [16, 512, 75, 25]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

My device config is set as [0, 1, 2, 3] and use dataParallel for multi-processor calculating, which is the same as your configuration in the responsitory. Did youmeet the same issue? I can't appreciate more if you can help me solve this problem.

Me too, is there a way?

Hi, Iywinaaa,
I faced the same problem before. I fixed it by changing the pytorch to a lower version. Hope it is helpful.

好的，非常感谢！
Kuroshika @.***> 于2023年4月28日周五 10:23写道：

Hi, Iywinaaa,
I faced the same problem before. I fixed it by changing the pytorch to a
lower version. Hope it is helpful.
—
Reply to this email directly, view it on GitHub
https://github.com/Chiaraplizz/ST-TR/issues/36#issuecomment-1526887602,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A42XPTPR6JJCB7WB6JSZ3YTXDMSYZANCNFSM6AAAAAASY76MME
.
You are receiving this because you commented.Message ID:
@.***>

Hi, Iywinaaa, I faced the same problem before. I fixed it by changing the pytorch to a lower version. Hope it is helpful.

Hello, may I ask which version of Python did you downgrade to without reporting errors?

My pytorch version is 1.5.1 and the cudatookit version is 10.1, while my torchvision version is 0.6.1. I will be glad if this may help you!
我用的pytorch版本是1.5.1，cudatookit是10.1，torchvision版本是0.6.1，不太建议在30系显卡上运行，因为我没在3090上复现成功:(，但是在一台titan v的电脑上复现成功了。

Hi, Chiaraplizz, I'd like to consult a problem encountered in running the code. When I start training process, following error has occured:

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [16, 512, 75, 25]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

My device config is set as [0, 1, 2, 3] and use dataParallel for multi-processor calculating, which is the same as your configuration in the responsitory. Did youmeet the same issue? I can't appreciate more if you can help me solve this problem.

Hello
can you tell me how you solved this issue?

Hi, Chiaraplizz, I'd like to consult a problem encountered in running the code. When I start training process, following error has occured:

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [16, 512, 75, 25]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

My device config is set as [0, 1, 2, 3] and use dataParallel for multi-processor calculating, which is the same as your configuration in the responsitory. Did youmeet the same issue? I can't appreciate more if you can help me solve this problem.

Hello can you tell me how you solved this issue?

Hello @AnainaM have you been able to resolve this issue? I recently came across this issue and my cuda version is 11.6
A response would be helpful.

Hello @AnainaM have you been able to resolve this issue? I recently came across this issue and my cuda version is 11.6
A response would be helpful.

@olayinkaajayi
Actually I was not able to fix this till now. If you get the solution please share it with me as well. Thank you.

All right then, I'll be happy to share if I figure it out. By the way are you doing a PhD research related to skeleton-based action recognition?


Move your website to HTTPS

Hi,
I'm a big fan of the new website design, it looks very good! I was wondering why you haven't moved your website to HTTPS? I know that you're not doing any sensitive stuff but still....it's something you can get for free with CloudFlare.
Keep up the great work.

Hi,
Because I’m lazy and Github Pages does not provide HTTPS for custom domains out of the box. I could move it to Netlify like I did for Sass Guidelines. :)

I know...I just moved to HTTPS using CloudFlare and it was really easy, it shouldn't take much of your time.
I learnt a lot about structuring a Jekyll project using your blog. Thanks.


[Question]: Need help first example

Ask your question
Hello, i´m a rooki in the wrapper theme. I´ve installed the eufy security ws as a docker. It works.
Bit what are the first steps? Can you explain a simple example how to access the docker station?
When i tried: <IP_ADDRESS>:3000 i get the message connection refused
Should came a web frontend? Sorry for the rookie question...

Hi,
You can find some examples with easy python scripts on my post here (in french sorry, maybe you can use google translate) : https://community.jeedom.com/t/integration-de-materiel-eufy/76603/1


Issue with authentication in GKE for simple smoke test with spark:2.4.4 image

TL;DR:
Image gcr.io/spark-operator/spark:2.4.4 fails simple smoke test whereas google/cloud-sdk:261.0.0 "just works"
Detail:
As part of testing, I reduced the GKE authorization failure to a test case shown in the attached script.
My expectation is that an application will use the default service account in the GKE cluster to authorize to services.
This does appear to work with the gcloud image but not the spark image - should it?
This test just uses the spark image to run the seq2bigtable jar file to upload a row into bigtable.
To run it, you need the default service account in the test cluster granted admin/access rights to bigtable.
Perhaps some dependencies are missing that could be added?
See the attached files for the script to run it and the output from running it.
Any help is much appreciated.
testauthorization-bigtable.txt
testauthorization-bigtable-out.txt

My understanding is that the compute engine default service account (used by GKE) doesn't have write access to Bigtable. So you would need to use a service account with the write permission.

@liyinan926 thank you for your suggestion but that wouldn't explain why one image worked and the other didn't.  I made sure the service account had access within the project.
After looking carefully yesterday at what images worked with k8s and which did not, I tracked the difference to the alpine-based java distribution and then found this unreleased spark patch set that fixed this issue which I verified by building my own image.
It would be good for spark-operator to put out a 2.4.5 preview with this patch - though note the one-line fix to command pass-through that is desirable:

https://issues.apache.org/jira/browse/SPARK-28938
https://github.com/apache/spark/pull/26046
https://github.com/apache/spark/pull/26046#issuecomment-547207060

Interesting. Yes, I can definitely build and push a new image with the patch.

And did you include https://github.com/apache/spark/pull/26296 when you built your own image?

Yes, I did.  And it has been merged to the 2.4 branch now.

Cool. I will build and push spark:2.4.5-SNAPSHOT shortly.

Just built and pushed gcr.io/spark-operator/spark:2.4.5-SNAPSHOT and gcr.io/spark-operator/spark-py:2.4.5-SNAPSHOT.

@liyinan926 These worked, thank you.
Can you tell me exactly how you invoked the build image tools?
FYI, I created a .gitlab-ci.yml file in this branch and would like to make it conform to how you build.

@liyinan926 ping

@jkleckner the Spark build command I used is the following:
./dev/make-distribution.sh --name custom-spark --pip --tgz -Phadoop-2.7 -Pkubernetes

Thank you, @liyinan926

Closing this and feel free to reopen if you have more questions.

@liyinan926 could there be some other step you are performing that accounts for this difference?
If I build the spark image the example jars are not in the same place so test code using spark-pi
to test spark-operator needs to find the jar file.
See the pipeline build here

https://gitlab.com/jkleckner/spark/pipelines/101854341

that uses your build options:

https://gitlab.com/jkleckner/spark/blob/b12427eff165918da4c13db6c583b4b5aca54b8a/.gitlab-ci.yml#L34-36
https://gitlab.com/jkleckner/spark/tree/cphy-kubernetes

yet the output of this script gives a difference in the examples directory:
#!/bin/bash

set -x

for im in\
    gcr.io/spark-operator/spark:2.4.5-SNAPSHOT \
    registry.gitlab.com/jkleckner/spark/spark:v2.4.5-k4.6.1-cphy2-43-gb12427e \
    ; do
  docker run -i --rm $im ls -l '/opt/spark/examples'
done

Note in this edited output from that script, that the example jars are in jars or target.
$ ./compare-versions.sh
+ for im in gcr.io/spark-operator/spark:2.4.5-SNAPSHOT registry.gitlab.com/jkleckner/spark/spark:v2.4.5-k4.6.1-cphy2-43-gb12427e
+ docker run -i --rm gcr.io/spark-operator/spark:2.4.5-SNAPSHOT ls -l /opt/spark/examples
+ SPARK_K8S_CMD=ls
Non-spark-on-k8s command provided, proceeding in pass-through mode...
+ case "$SPARK_K8S_CMD" in
+ echo 'Non-spark-on-k8s command provided, proceeding in pass-through mode...'
+ exec /usr/bin/tini -s -- ls -l /opt/spark/examples

total 8
drwxr-xr-x 2 root root 4096 Oct 29 19:36 jars
drwxr-xr-x 3 root root 4096 Oct 29 19:36 src

+ for im in gcr.io/spark-operator/spark:2.4.5-SNAPSHOT registry.gitlab.com/jkleckner/spark/spark:v2.4.5-k4.6.1-cphy2-43-gb12427e
+ docker run -i --rm registry.gitlab.com/jkleckner/spark/spark:v2.4.5-k4.6.1-cphy2-43-gb12427e ls -l /opt/spark/examples
+ SPARK_K8S_CMD=ls
+ case "$SPARK_K8S_CMD" in
+ echo 'Non-spark-on-k8s command provided, proceeding in pass-through mode...'
+ exec /usr/bin/tini -s -- ls -l /opt/spark/examples

total 16
-rw-rw-rw- 1 root root 5768 Dec 10 01:16 pom.xml
drwxrwxrwx 3 root root 4096 Dec 10 01:16 src
drwxr-xr-x 8 root root 4096 Dec 10 01:41 target
+ exit 0

Closing this and feel free to reopen if you have more questions.

It is not possible for me to reopen by permission.  It may make sense for this to remain open until a spark 2.4.5 gets released.

@liyinan926  When I run further tests, I found that the spark-operator image also needs to be updated as well to move from openjdk:8-alpine to openjdk:8-jdk-slim.
Apparently, the driver and executor pods cannot communicate properly (with service accounts) as seen above until the spark-operator image is updated.  I hacked a version to use that does and it then worked.
docker build --build-arg SPARK_IMAGE=gcr.io/spark-operator/spark:2.4.5-SNAPSHOT -t gcr.io/project/spark-operator:v2.4.5-SNAPSHOT .
To use the ubuntu image, the tini executable needed to be added:
FROM ${SPARK_IMAGE}
COPY --from=builder /usr/bin/spark-operator /usr/bin/
RUN apk add --no-cache openssl curl tini
COPY hack/gencerts.sh /usr/bin/
USER root
ENV TINI_VERSION v0.18.0
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini /tini
RUN chmod +x /tini
RUN mv /tini /sbin/tini

I would really like to see a gitlab-ci.yml pipeline defined for building the spark-operator image, but I suppose that is a different issue :-)

@jkleckner can you create a PR for the changes to the operator image you made?

@liyinan926 Ok, I cleaned up the mess in my repo and created #723.  But it isn't obvious how far to push all the SNAPSHOT versions into doc files etc.  Note that the 2.4.5-SNAPSHOT image version you published is missing the leading v in the tag v2.4.5-SNAPSHOT present in other tags so I preserved that difference in the operator image tag.  Perhaps another --build-arg makes sense for the destination image tag to prevent useless churn in the repo?  We probably want to leave out those changes from this patch set but I included it for completeness.  Easy to remove them.
I would also appreciate you addressing the repeatable build problem mentioned in this comment: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/issues/669#issuecomment-564346219
Having repeatable builds goes a long way toward helping people to develop with this tool.
I would also be happy to have a hangouts meet if you are interested.

I would also appreciate you addressing the repeatable build problem mentioned in this comment: #669 (comment)

I would like to upstream that gitlab-ci.yml file in a PR to spark but I need to figure out the mismatch with your builds before I do.
https://gitlab.com/jkleckner/spark/commit/b12427eff165918da4c13db6c583b4b5aca54b8a

I'm going to rebuild the image from the latest branch-2.4, and correct the tag used.

Do you see any mismatch with the build commands in that gitlab-ci.yml file?
It builds just fine but puts the example jars into a target subdirectory.

@liyinan926 My apologies, I just discovered that my spark repo in gitlab was accidentally private so that must have been quite confusing!  It is now public.
Here is a branch with the gitlab-ci.yml file that builds the spark snapshot relying on a possibly lightweight tag for the version specification:

https://gitlab.com/jkleckner/spark/tree/add-gitlab-ci-yml

Note that this build has the example jars in /opt/spark/examples/target/ whereas gcr.io/spark-operator/spark:v2.4.5-SNAPSHOT has the example jars in /opt/spark/examples/jars.  It would be good if we can trace why they are in different locations.
I have created a jira story for adding the build script here:

https://issues.apache.org/jira/browse/SPARK-30275

Note that the use of git describe for both the builds of spark and spark-operator results in unique image tags even when snapshots are being created.
I built spark with the script above and it produced version v2.4.4-143-g7de7e16e97.
Putting that in as the SPARK_VERSION in the build for the spark-operator produced a snapshot build with the tag v1beta2-26-g86fb3bd_v2.4.4-143-g7de7e16e97.
This is a unique image tag in both the operator and the underlying spark versions which means that image pull policy won't matter:

registry.gitlab.com/jkleckner/spark-on-k8s-operator/spark-operator:v1beta2-26-g86fb3bd_v2.4.4-143-g7de7e16e97

I noticed an inconsistency with the release images you produce where the underscore should be a dash that separates the versions.  Perhaps this one should change to match.

A new release based on Spark 2.4.5 has been published. Closing this issue now.


remove tcmalloc from CMakeList because it cause pointer error on NUMA…

As stated in #5 , in current implementation, Sparkle fails with attempt to free invalid pointer error on NUMA environment.
This error seems to be caused by using tcmalloc in JNI, so I remove tcmalloc from target_link_libraries in CMakeLists.
The reference why we suspected tcmalloc : TCMalloc : Thread-Caching Malloc

👍