Fake os.walk does not resolve symlinks properly

Describe the bug
os.walk does not reolve the directory before checking if it is a symlink.
    def do_walk(top_dir, top_most=False):
        if not top_most and not followlinks and filesystem.islink(top_dir):
            return
        try:
            top_contents = _classify_directory_contents(filesystem, top_dir)
        except OSError as exc:
            top_contents = None
            if onerror is not None:
                onerror(exc)

        if top_contents is not None:
            if topdown:
                yield top_contents

            for directory in top_contents[1]:
                if not followlinks and filesystem.islink(directory): # here should check filesystem.joinpaths(top_dir, directory)
                    continue
                for contents in do_walk(filesystem.joinpaths(top_dir,
                                                             directory)):
                    yield contents

            if not topdown:
                yield top_contents

How To Reproduce
import os
from pathlib import Path
from pyfakefs.fake_filesystem_unittest import Patcher
def bug():
    with Patcher() as patcher:
        filesystem = patcher.fs
        filesystem.is_windows_fs = filesystem.is_macos = False
        filesystem.reset()

        filesystem.add_real_file(__file__)

        dir_path = Path('/test1/test2')
        dir_path.mkdir(parents=True)

        test_file = dir_path / 'test3'
        test_file.touch()

        print(list(os.walk('/test1')))

        Path('/test2').symlink_to(test_file)

        print(list(os.walk('/test1')))

bug()

Output is:
[('/test1', ['test2'], []), ('/test1\\test2', [], ['test3'])]
[('/test1', ['test2'], [])]

Your enviroment
Please run the following and paste the output.
python -c "import platform; print(platform.platform())"
python -c "import sys; print('Python', sys.version)"
python -c "from pyfakefs.fake_filesystem import __version__; print('pyfakefs', __version__)"

Output is:
C:\Users\gilad\Documents\my_module>python -c "import platform; print(platform.platform())"
Windows-10-10.0.19041-SP0

C:\Users\gilad\Documents\my_module>python -c "import sys; print('Python', sys.version)"
Python 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:37:02) [MSC v.1924 64 bit (AMD64)]

C:\Users\gilad\Documents\my_module>python -c "from pyfakefs.fake_filesystem import __version__; print('pyfakefs', __version__)"
pyfakefs 4.1.0

Good catch! And your proposed fix is also correct.
As an aside--I noticed the inconsistent path separators in your example, and added a fix and a more convenient way to set the file system OS, so thanks for that.
I would appreciate if you could test the master after I add the fix. I will wait for further issues and probably make a new release if none will appear for some time.

And done.

@giladreti - will you still be testing this? Otherwise, I would make a new release with the changes.

I couldn't find any more bugs, so it is fine by me :)


Respect per-file-ignores when checking noqa

RUF100 does not take into account a rule ignored for a file via a per-file-ignores configuration. To see this, try the following pyproject.toml:
[tool.ruff.per-file-ignores]
"test.py" = ["F401"]

and this test.py file:
import itertools  # noqa: F401

Running ruff --extend-select RUF100 test.py, we should expect to get this error:
test.py:1:19: RUF100 Unused `noqa` directive (unused: `F401`)

The issue is that the per-file-ignores diagnostics are filtered out after the noqa checks, rather than before.

Ah, good catch!

Looking again, I think I may have put the filtering in the wrong spot - it should probably come after this block https://github.com/charliermarsh/ruff/pull/2309/files#diff-9f39c861d0ab3e1bd158b77ee65246fe5c438ab6f2ddd112684cc6cef6185f01L136-L150


Add TheRock Trading support

Hi Team
Is there any plans to add support for TheRock Trading exchange?

The description of this PR does not match the changes.


ERRORS when I run Desktop Helloworld

I completed the requirement configuration according to the instructions of Installing on Debian and Ubuntu
but When running the last command:
$ export GLOG_logtostderr=1
# if you are running on Linux desktop with CPU only
$ bazel run --define MEDIAPIPE_DISABLE_GPU=1 \
    mediapipe/examples/desktop/hello_world:hello_world

Encountered the following error:
Starting local Bazel server and connecting to it...
INFO: SHA256 (https://github.com/bazelbuild/rules_cc/archive/master.zip) = a07fc8b90c22e42962142504438a84b31567976adcd9ee1a4c9f75d2e3396f8d
DEBUG: Rule 'rules_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "a07fc8b90c22e42962142504438a84b31567976adcd9ee1a4c9f75d2e3396f8d"
DEBUG: Repository rules_cc instantiated at:
  no stack (--record_rule_instantiation_callstack not enabled)
Repository rule http_archive defined at:
  /home/zcw/.cache/bazel/_bazel_zcw/43c4a13803b0469a6bd3b3fd6f82c078/external/bazel_tools/tools/build_defs/repo/http.bzl:336:16: in <toplevel>
INFO: Repository local_execution_config_python instantiated at:
  no stack (--record_rule_instantiation_callstack not enabled)
Repository rule local_python_configure defined at:
  /home/zcw/.cache/bazel/_bazel_zcw/43c4a13803b0469a6bd3b3fd6f82c078/external/org_tensorflow/third_party/py/python_configure.bzl:275:26: in <toplevel>
ERROR: An error occurred during the fetch of repository 'local_execution_config_python':
   Traceback (most recent call last):
	File "/home/zcw/.cache/bazel/_bazel_zcw/43c4a13803b0469a6bd3b3fd6f82c078/external/org_tensorflow/third_party/py/python_configure.bzl", line 208
		get_python_bin(repository_ctx)
	File "/home/zcw/.cache/bazel/_bazel_zcw/43c4a13803b0469a6bd3b3fd6f82c078/external/org_tensorflow/third_party/remote_config/common.bzl", line 44, in get_python_bin
		which(repository_ctx, "python")
	File "/home/zcw/.cache/bazel/_bazel_zcw/43c4a13803b0469a6bd3b3fd6f82c078/external/org_tensorflow/third_party/remote_config/common.bzl", line 29, in which
		execute(repository_ctx, <1 more arguments>)
	File "/home/zcw/.cache/bazel/_bazel_zcw/43c4a13803b0469a6bd3b3fd6f82c078/external/org_tensorflow/third_party/remote_config/common.bzl", line 208, in execute
		fail(<1 more arguments>)
Repository command failed

ERROR: Analysis of target '//mediapipe/examples/desktop/hello_world:hello_world' failed; build aborted: Traceback (most recent call last):
	File "/home/zcw/.cache/bazel/_bazel_zcw/43c4a13803b0469a6bd3b3fd6f82c078/external/org_tensorflow/third_party/py/python_configure.bzl", line 208
		get_python_bin(repository_ctx)
	File "/home/zcw/.cache/bazel/_bazel_zcw/43c4a13803b0469a6bd3b3fd6f82c078/external/org_tensorflow/third_party/remote_config/common.bzl", line 44, in get_python_bin
		which(repository_ctx, "python")
	File "/home/zcw/.cache/bazel/_bazel_zcw/43c4a13803b0469a6bd3b3fd6f82c078/external/org_tensorflow/third_party/remote_config/common.bzl", line 29, in which
		execute(repository_ctx, <1 more arguments>)
	File "/home/zcw/.cache/bazel/_bazel_zcw/43c4a13803b0469a6bd3b3fd6f82c078/external/org_tensorflow/third_party/remote_config/common.bzl", line 208, in execute
		fail(<1 more arguments>)
Repository command failed

INFO: Elapsed time: 13.957s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (9 packages loaded, 11 targets configured)
FAILED: Build did NOT complete successfully (9 packages loaded, 11 targets configured)
    currently loading: @bazel_tools//tools/jdk

Before asking the question, I searched for related issues, but still failed to solve it, so I hope to seek some help :)

The issue is Bazel doesn't find the local python binary. You can run $ which python to find where the python binary is and then add --action_env PYTHON_BIN_PATH=<path to python binary> to the Bazel command like the following
bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 --action_env PYTHON_BIN_PATH="/path/to/python" mediapipe/examples/desktop/hello_world

Thank you very much.I have followed your method to solve the previous problem.And get the following content:
Starting local Bazel server and connecting to it...
INFO: SHA256 (https://github.com/bazelbuild/rules_cc/archive/master.zip) = a07fc8b90c22e42962142504438a84b31567976adcd9ee1a4c9f75d2e3396f8d
DEBUG: Rule 'rules_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "a07fc8b90c22e42962142504438a84b31567976adcd9ee1a4c9f75d2e3396f8d"
DEBUG: Repository rules_cc instantiated at:
  no stack (--record_rule_instantiation_callstack not enabled)
Repository rule http_archive defined at:
  /home/zcw/.cache/bazel/_bazel_zcw/43c4a13803b0469a6bd3b3fd6f82c078/external/bazel_tools/tools/build_defs/repo/http.bzl:336:16: in <toplevel>
INFO: Analyzed target //mediapipe/examples/desktop/hello_world:hello_world (52 packages loaded, 1246 targets configured).
INFO: Found 1 target...
INFO: From ProtoCompile mediapipe/calculators/internal/callback_packet_calculator.pb.h:
bazel-out/k8-opt/bin/external/com_google_protobuf/src: warning: directory does not exist.
INFO: From ProtoCompile mediapipe/framework/tool/calculator_graph_template.pb.h:
bazel-out/k8-opt/bin/external/com_google_protobuf/src: warning: directory does not exist.
INFO: From ProtoCompile mediapipe/framework/calculator.pb.h:
bazel-out/k8-opt/bin/external/com_google_protobuf/src: warning: directory does not exist.
INFO: From ProtoCompile mediapipe/framework/calculator_profile.pb.h:
bazel-out/k8-opt/bin/external/com_google_protobuf/src: warning: directory does not exist.
Target //mediapipe/examples/desktop/hello_world:hello_world up-to-date:
  bazel-bin/mediapipe/examples/desktop/hello_world/hello_world
INFO: Elapsed time: 172.539s, Critical Path: 125.83s
INFO: 455 processes: 455 linux-sandbox.
INFO: Build completed successfully, 471 total actions

Am I successful,but did not print "Hello World" as expected @_@

You successfully built bazel-bin/mediapipe/examples/desktop/hello_world/hello_world. Then, you need to run it ;)
Try the following:
$ export GLOG_logtostderr=1
$ bazel-bin/mediapipe/examples/desktop/hello_world/hello_world

I make it！Thank you very much for your patience！


Bump CI to macOS 14 and build visionOS debug simulator

Platforms Impacted

[ ] iOS
[ ] macOS

Description of changes

Updating our workflows to use macOS 14 runners, which are ARM64 devices by default
Using these new runners, verify visionOS as part of the CI pipeline

Binary change
n/a -- only pipeline changes
Verification
Hopefully CI passes :)
Pull request checklist
This PR has considered:

[ ] Light and Dark appearances
[ ] iOS supported versions (all major versions greater than or equal current target deployment version)
[ ] VoiceOver and Keyboard Accessibility
[ ] Internationalization and Right to Left layouts
[ ] Different resolutions (1x, 2x, 3x)
[ ] Size classes and window sizes (iPhone vs iPad, notched devices, multitasking, different window sizes, etc)
[ ] iPad Pointer interaction
[ ] SwiftUI consumption (validation or new demo scenarios needed)
[ ] Objective-C exposure (provide it only if needed)

Microsoft Reviewers: Open in CodeFlow

do we want device too or is simulator enough

I was wondering about that too. In your experience, have there been any interesting differences between device and simulator that we should consider? I assumed that for most cases (catching deprecated API usage, for example), one should be sufficient.

do we want device too or is simulator enough

I was wondering about that too. In your experience, have there been any interesting differences between device and simulator that we should consider? I assumed that for most cases (catching deprecated API usage, for example), one should be sufficient.

nope!


Issue with Future.then() chain example on dart.dev documentation website

In the section about Errors in the middle of a long chain in the article on Futures and Error handling there is a code example which is not implemented in a straight forward way.
Future<String> one() => Future.value('from one');
Future<String> two() => Future.error('error from two');
Future<String> three() => Future.value('from three');
Future<String> four() => Future.value('from four');

void main() {
  one() // Future completes with "from one".
      .then((_) => two()) // Future completes with two()'s error.
      .then((_) => three()) // Future completes with two()'s error.
      .then((_) => four()) // Future completes with two()'s error.
      .then((value) => value.length) // Future completes with two()'s error.
      .catchError((e) {
    print('Got error: $e'); // Finally, callback fires.
    return 42; // Future completes with 42.
  }).then((value) {
    print('The value is $value');
  });
}

The then() callbacks can be improved by making them single layered and the same can be done to one() like follows.
Future<String> one = Future.value('from one');
Future<String> two(_) => Future.error('error from two');
Future<String> three(_) => Future.value('from three');
Future<String> four(_) => Future.value('from four');
int five(_) => _.length;

void main() {
  one // Future completes with "from one".
      .then(two) // Future completes with two()'s error.
      .then(three) // Future completes with two()'s error.
      .then(four) // Future completes with two()'s error.
      .then(five) // Future completes with two()'s error.
      .catchError(catchError)
      .then(finalCallback);
}
int catchError(e) {
  print('Got error: $e'); // Finally, callback fires.
  return 42; // Future completes with 42.
}
void finalCallback(int value) {
  print('The value is $value');
}

For more symmtery, int five(_) => _.length; can be changed to
Future<int> five(_) => Future.value(_.length);

The code is there to illustrate how then handlers work, the suggested rewrite does not actually materially change anything in this example. On the contrary, I think the example becomes more confusing.
/cc @lrhn

I think the formatting makes it look weirder than it is.
The example here is intending to show a sequence of asynchronous and synchronous computations, and then a catchError to handle them all.
Abstracting the one-five functions out is to suggest that they are some external async computations.
Maybe if we hard-formatted it as below, and maybe used async for the helper functions:
Future<String> one() async => 'one';
Future<String> two() async => throw 'error from two';
Future<String> three() async => throw "Never executed";
Future<String> four() async => 'four'; // Also never executed.

void main() {
  one() // Future completes with "from one".
      .then((_) => two()) // Future completes with two()'s error.
      .then((_) => three()) // Future completes with two()'s error.
      .then((_) => four()) // Future completes with two()'s error.
      .then((value) => value.length) // Future completes with two()'s error.
      .catchError((e) { // Does run.
        print('Got error: $e'); // Prints "error from two"
        return 42;
      }) // Future completes with 42.
      .then((value) {
        print('The value is $value'); // Prints "The value is 42"
      });
}

It's not what the formatter would do, but what the formatter does is not readable.


Added few updates to the code.

Description
Please include a summary of the change and specify which issue is being addressed. Additionally, provide relevant motivation and context.
Fixes # (issue number)
Type of change
Please delete options that are not relevant.

[x] Documentation (typos, code examples, or any documentation updates)
[ ] Bug fix (a non-breaking change that resolves an issue)
[ ] New feature (a non-breaking change that adds functionality)
[ ] Breaking change (a fix or feature that would disrupt existing functionality)
[ ] This change requires a documentation update

Checklist

[ ] My code adheres to the style guidelines of this project (scripts/lint.sh shows no errors)
[x] I have conducted a self-review of my own code
[x] I have made the necessary changes to the documentation
[x] My changes do not generate any new warnings
[ ] I have added tests to validate the effectiveness of my fix or the functionality of my new feature
[ ] Both new and existing unit tests pass successfully on my local environment by running scripts/test-cov.sh
[ ] I have ensured that static analysis tests are passing by running scripts/static-analysis.sh
[ ] I have included code examples to illustrate the modifications

I don't think, that such code comments are required, but not sure about documentation rewords
@kumaranvpl can you check the PR, please?

@Lancetnik Went through the changes. We can ignore and close this PR. The changes are extremely minor.
This is clearly generated with ChatGPT or some other LLM. Generating code or docs using LLM is not wrong. But these sorta PRs adds no value to the repo and wastes maintainers' time. We shouldn't be encouraging these kinda PRs.

@Lancetnik I am closing this PR. If you think it is not right to close the PR, please reopen it.


if all refs are removed from a BOM line, the line is not removed

we should remove lines that have Qnty set to zero when removing parts.

fixed in v0.0.12, #28


nextBlocks rollback in Jormungandr

Context
Using the global unstable blocks state (#648), we can do a simple non-optimal rollback in the Jormungandr network layer when an alternate chain is detected.
Decision
Adjust the nextBlocks return type so that rolling back is an option.
data NextBlocksResult
  = AwaitReply
  | RollForward [Block (Tx t)]
  | RollBackward BlockHeader

data NetworkLayer t m = NetworkLayer
    { nextBlocks
        :: BlockHeader
        -> ExceptT ErrGetBlock m NextBlocksResult
    , ...
    }

nextBlocks should be modified to detect a rollback condition:

It should first call networkTip to update the global state.
It compares the SlotId of the requested BlockHeader to those in
the unstable block headers.

If the SlotId is before the deepest unstable block header, return
blocks as normal under the RollForward constructor.
If the SlotId is within the unstable block headers and the parent
hash matches, return blocks in RollForward. If there are no
blocks, return AwaitReply.
Otherwise, return RollBackward to the block header of depth k.

Acceptance Criteria

nextBlocks must return RollBackward if the requested start point is not on the node's chain.
You may be able to develop a more optimal solution than rolling back the full k, but don't sweat over it too much. (For this to work, each wallet would need its own UnstableBlocks state, which is intersected with the global UnstableBlocks state.)
Because jormungandr is a concurrent process accessed through a stateless API, there should be testing of concurrency issues such as what happens if the network tip rolls back while fetching blocks.

Development Plan

Need to adjust the nextBlocks type a bit to carry around the state for calculating the intersection.
Property tests for intersection calculation.
Unit tests for rollback, probably by mocking JormungandrLayer.
Integration tests with the actual jormungandr node would be pointless, unless it can fork and roll back.

PR

Number
Base

#755
master

#800
master

QA

As part of #755, we've implemented a rather extensive unit tests to stress a bit the networking layer and see how chain following behave. The test defines a mock Jörmungandr client (which implements the JormungandrClient interface that we use for Jörmungandr and produces mock blocks. We plug that mock Jörmungandr client into the actual network layer implementation and we then try to follow the chain produced by this client using the networking layer. The mock has its own internal state where it maintain a valid chain. After each API call to the mock, the mock does one of the following mutation to itself (see https://github.com/input-output-hk/cardano-wallet/blob/master/lib/jormungandr/test/unit/Cardano/Wallet/Jormungandr/NetworkSpec.hs#L427-L432):

Add one or more blocks on its local chain

Rewind its local chain to a point in the past (but keep old blocks in its "database")

Garbage collect blocks in its database that are no longer on the chain

This allows for creating all sort of interesting scenario where our network layer will try to follow a chain that is evolving quite arbitrarily and, creating a lot of race conditions and concurrency problems between API calls. The quickcheck  check for 2 things:

For any generated mock node, for all mutations, it always produce a valid chain (in the sense of blocks following each other in sequence starting from genesis, there can be gaps in the slot numbers).

For any generated mock node, for any value of k > 1, we can define a consumer that can follow the chain using the network layer and eventually be in sync with the node.

So, the main spec does follow the node as it's evolving and until the node runs out of mutations to apply. Then, we make sure that the network layer can eventually catch up in a finite and capped number of steps.
Once the consumer has fully synced, we also label cases of interest and to make sure that we go into some specific scenarios. For example:
Follow a chain with forks
      +++ OK, passed 1000000 tests:
      56.1946% advanced more than k blocks
      53.1837% started with an empty chain
      40.6301% started with more than k blocks
      15.3164% switched to a shorter chain
      10.8648% switched to a longer chain
       9.7370% rewinded without switch
       1.4335% rolled back full k
       0.4092% recovered from genesis

Also, in case of failure, we display a full error log of the operation that happened on the consumer, the node and the mutation that happened on the network so that we get a very nice counter example in order to understand what's going on:
         Applied blocks:  0001 _ _ _ 0002 0005 _ _ 0006 0007 _ 0008 _ 0009 000a 000b 000c 000d
         Node chain:      0001 _ _ _ 0002 0003 0004 0005 _ _ 0006 0007 _ 0008 _ 0009 000a 000b 000c 000d
         Logs:
         [CONSUMER] nextBlocks SlotId {epochNumber = 0, slotNumber = 0}
         [PRODUCER] getTipId
             ↳  Hash {getHash = "genesis"}
         [NETWORK ] NodeAddBlocks [MockBlock {mockBlockId = Hash {getHash = "0000"}, mockBlockPrev = Hash {getHash = "genesis"}, mockBlockSlot = SlotId {epochNumber = 0, slotNumber = 0}, mockBlockContent = 0},MockBlock {mockBlockId = Hash {getHash = "0001"}, mockBlockPrev = Hash {getHash = "0000"}, mockBlockSlot = SlotId {epochNumber = 0, slotNumber = 1}, mockBlockContent = 1},MockBlock {mockBlockId = Hash {getHash = "0002"}, mockBlockPrev = Hash {getHash = "0001"}, mockBlockSlot = SlotId {epochNumber = 0, slotNumber = 5}, mockBlockContent = 2}]
         [PRODUCER] getBlock Hash {getHash = "genesis"}
             ↳  Left (ErrGetBlockNotFound (Hash {getHash = "genesis"}))
         [CONSUMER] AwaitReply
         [CONSUMER] nextBlocks SlotId {epochNumber = 0, slotNumber = 0}

LGTM :+1:


AttributeError: 'NoneType' object has no attribute 'release'

I was trying to run "python3 yolo.py --video-path='/media/lamp-moving.mpg".
(I am using 4.1.0 version of cv2)
And then the following error comes up:
Traceback (most recent call last):
File "yolo.py", line 146, in
writer.release()
AttributeError: 'NoneType' object has no attribute 'release'
Thank you

I met the same error! I want to know whether you have solved the error or not. Thank you.


Could not compile sample app

Hello,
This is more or less a question, I was able to compile the sample app for iOS but not for android, perhaps I am missing something. The build on android throws this error - Could not resolve all dependencies for configuration ':_debugCompile'. Could not find any version that matches com.google.android.gms:play-services-location:+.
Please what do you think?

Update: Solved my own problems by installing/updating Google Play services and Google Repository via SDK Manager. Thanks

@tolu360 That's actually good remark. I'll add it into README.md. Thanks

@mauron85 Thank you for sharing your plugin.


Grab bag 2 apr 24

Some random small things

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://660c386c80cdfc4cafe71889--harness-developer.netlify.app


Update symbol task with updated packaing-common nuget

Task name: PublishSymbolsV2
Description: Updates "azure-pipelines-tasks-packaging-common" version to 3.236.0 and Task version to 2.237.0
Documentation changes required: N
Added unit tests: (Y/N)
Attached related issue: (Y/N)
Checklist:

[x] Task version was bumped - please check instruction how to do it
[x] Checked that applied changes work as expected

Are we really changing 95 files? Most of them look generated.

Are we really changing 95 files? Most of them look generated.

Yes... we are supposed to checkin changes of _generated folder too after build


global.Mustache triggers JScript runtime error

Because ASP/WSH script environments are language agnostic, one cannot assume it can be managed like a first-class javascript object. By changing the javascript global explicit assignment pattern to implicit assignment pattern (less is more) it works as expected.
Error before the change:
Microsoft JScript runtime error '800a01b6'
Object doesn't support this property or method
/lib/axe/classes/Parsers/mustache.asp, line 163

Thanks for looking after us @nagaozen. I thought I remembered you got something similar merged last year -> https://github.com/janl/mustache.js/commit/60c2da80783a71409a8c3ecbc32a3d4e858df691.
Too ensure this regression doesn't sneek its way back in again, any thoughts about writing a unit tests for this fix?


Dashboard template - ignore "instance"

this may be out of scope for this project, but I containerized the nagios_exporter and I'm running it in k8s - the problem there is that I lose data continuity every time the pod is replaced or restarted because the data in the daashboard is tied to "instance", which is irrelevant for my use case. Any ideas on how I can just skip that distinction?

Hey @V3XATI0N , happy to hear you're using this project!!
In this case I would recommend one of two things:

Relabel instance at scrape time to a static value that represents your Nagios instance

e.g:
relabel_configs:
      - source_labels: [__address__]
        regex: '.*'
        target_label: instance
        replacement: 'nagios_V3XATI0N'

By default I'm fairly sure address gets relabed into instance, which is where your pod hostname comes into play.

Simply drop all instance labels from the dashboard.

I should probably be using jsonnet to template out the Grafana dashboard instead so it's more customizable. However you might be able to just find and remove instance=\"$instance\" and instance=\"$instance\", from the PromQL queries in each panel to make the dashboard instance independent. My homelab is a bit broken right now so can't test it for you but the dashboard I'm linking below may work.
https://gist.github.com/wbollock/86b6b0ad705a0c26ecb7bd6a48e7ac51
Admittedly I haven't had this specific issue before but I hope this helps! Let me know how it goes and if you have any other feature requests

you many want to try re-labeling like this too: https://grafana.com/blog/2022/03/21/how-relabeling-in-prometheus-works/#labelmap
hope this helps! when/if I get the dashboard templated with jsonnet that should help a bunch also to make mass adjustments


Main Menu  cannot click on the buttons

At the start of the game, in the main menu with the buttons new game, load a game I cannot click on the buttons.

Make sure the game window is focused and that triggers/buttons on both of your controllers work.

My controllers are functional and I pass the triggers on the bourons but on the buttons, no button is activated when the trigger passes. I can't select them. I have an Occulus Rift.

It's verified to work on rift so it must be something about your setup. Try asking on the koikatsu discord server if anyone else had problems like this. Post the output log from game directory too.

I changed the game version and it works perfectly. I have the R8 repack and it works well instead of the R10.

That doesn't really help me, can you pinpoint what exactly causes the issue?


[WHITELIST] hb.wpmucdn.com

hb.wpmucdn.com apparently breaks acrcloud.com... if the image isn't obvious enough.
AdAway also blocks it.

Hi @CyanoTex we don't block any of these domains.
Closing.


Updating Tags does not update Elastic Indexes

When updating "Tags" from frontend, it does update backend Neo4j but does not update Elastic indexes. Hence, once tags are updated, they are not searchable.

Currently a job with databuilder ElasticSearchPublisher() – and related Neo4jSearchDataExtractor() and FSElasticsearchJSONLoader() need to run to rebuild the ES index.
If you are exploring the “Quickstart” example you can simply re-run example/scripts/sample_data_loader.py as it runs that job:
https://github.com/lyft/amundsendatabuilder/blob/6e3f64e5f0383b6759670af1393a86329fcdbad2/example/scripts/sample_data_loader.py#L262-L264
I’m wondering if it would be useful to:

Extract the ES job from the example code (no need to re-run Neo4j ingest)
Wrap it as Flask service, that can get pushed by the metadata service on it’s incoming POST/PUT operations.
Even dockerize this service

If you pull master of the Databuilder repo you will now get updated footer about when you last triggered Elasticsearch re-index since this PR just got merged: https://github.com/lyft/amundsendatabuilder/pull/86


[NFC] Move some Type Checker Options into TypeCheckerOptions

Well, and a few other things

Remove TypeChecker::getLangOpts().  You can get it from the ASTContext.
Drop a few SaveAndRestores of type checker debug flags.  These were just being used to control the behavior of the ASTPrinter.  But the ASTPrinter doesn't dump types...
Clean up type checker option parsing a bit.

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

⛵


cosign support in witness: keyless signatures via OIDC issuer

I am reading the code for the signature generation right now. I see that we are using x509 certificates for signing already.
Would it make sense to support keyless signatures via OIDC-issuer, either via Email-based OpenIDs or via Workload Identities.
The first would make sense if a human signs the attestations. The latter makes sense if witness runs in a CI environment and there should be no interaction that needs manual intervention.
If you are not up to date with keyless signatures here is a short walkthrough:
You might know the OpenID Connect standard. The OpenID Connect standard allows identifying yourself via your email to a foreign system. For example, when you login into a new website via your Github ID you are doing this via OpenID Connect.
The sigstore project is currently pushing the idea that we could use this user identity to generate an ephemeral certificate key pair for signing operations. This has the huge advantage that the user does not need to care about private key management at all. No keys in a cold storage, no key backups, no potential key leaks. The only thing the user has to care about is securing their Identity, for example via 2FA.
Workload identities are working the same way. The only difference is that a workload identity is coupled to the CI Pipeline. In terms of Github Actions github will create a token, only valid for this one pipeline run, and sigstore uses that token to generate an ephemeral key pair and so on.
The cosign library manages most of this (including uploading the signature(s) to the rekor transparency log). The transparency log can be compared to a https certificate transparency log... it tracks all signing operations + stores the used public certificate and a digest of the artifact.
On the long term I would like to have this feature in in-toto-golang as well, but unfortunately I am afraid, that we need to make huge changes in in-toto-golang. First step could be a Signer interface (right now most signing operations are pretty much hard coded).
I don't know if this is a feature for witness or for in-toto-golang. As far as I understand the code in witness, you are not using in-toto-golang's signing features at all, right?

Code sample on how to fetch a cert from fulcio here:  https://github.com/sigstore/fulcio/pull/324

We will need to get Fulcio set up to work on this issue.

First MR will be to support witness sign to enable OIDC signing of policy.

We support Fulcio keyless, I'll open other issues specific to Rekor and CA/timestamp policy verification


remove glob chars from spectral target filename

Fixes #1127

problematic filename values in the registry

Yes, I considered that but as I suggested in the issue, I believe that's over-generalizing. I don't imagine that all filenames should be subject to glob validation... that would puts the burden on the API users to have to sanitize perfectly valid filenames.

This is basically what I was thinking: https://en.wikipedia.org/wiki/Robustness_principle :)


Refactoring channels.nim

Refactoring channels.nim

Restoring double locks (partly fixes  #19), per the original algorithm from @mratsim's impl and @aprell's thesis.
Renaming conditional variables.
Marking buffered/unbuffered with static bool, in accordance with the actual usage.
Change private arguments and logic from nonBlocking to blocking.
Swap head and tail lock roles in accordance with common nomenclature.
Fix runnableExamples using unavailable close proc.
Hide unused isClosed proc under when false (same as open and close)

Future work: lifting the buffered/unbuffered distinction to type information. Would like to hear your suggestions on the implementation, as I don't see any obvious ways of doing it without affecting the public interface (how to make RawChannel being generic and hide it?). @mratsim already asked some relevant questions but I'm not sure what applies.

I've adapted my example code from #19 to serve as an additional test, checking passage of messages through two one-> many and many->one channels, checking for both buffered and unbuffered code paths. Should I add the test module as a separate PR or is it ok here? Current test does a bare minimum.


[test-framework] Add control RPC to test service

The RPC allows us to simulate in-mesh traffic from one service to
another.
Creating a copy of the pilot test service under pkg/test. This is a
more appropriate home for the code and also avoids changing the rest
of the code that depends on the old service.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: nmittler
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pkg/OWNERS [nmittler]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Presumably, this is all copied code and does not contain your additions, right?
/lgtm

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: nmittler, ozevren
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pkg/OWNERS [nmittler]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Presumably, this is all copied code and does not contain your additions, right?

@ozevren yes, that's correct.  The only addition is the new handler method for the ForwardedEchoRequest


docs: Re-instantiate Typedocs for hidden exported functions

Since #835  was addressed, there are a number of functions which are relevant to consumers that are currently marked as hidden such as getTransactionsSummaries for instance.  Now that we have a more structured approach to documentation we can discuss how we want to articulate these methods.

Is this issue only about the getTransactionsSummaries or something else, and what exactly?
We must strive to have straightforward and actionable issues that people can work on, ideally without seeking more information than what's on the issue. Otherwise, discussions are welcome.

That's a good point, I've opened this up in the discussion forum


Missing attachments on viewer relogs

Fixed a possible race condition with session global _rezMultupleAttachmentsData.  It is used outside the lock; it might be getting cleared during async processing by another packet, especially if the asset cache isn't warmed up, such as on a different region from where the attachments were added.  Also some small tweaks to some of the inventory/asset reporting to help us investigate issues like this.

Initial lengthy testing by @zauberparacelsus and I today were unable to reproduce the issue after testing on a region with these changes. However, after that we weren't able to reproduce the problem even on non-updated regions. So it's a bit unclear if this resolves the case in question, but it certainly didn't make it worse. Going to merge this for the RC deployment to Sandboxes tonight.


real-time-html not reloading

Hi,
here my goaccessconf file.
I can t make it real time html with automatic refresh. What am I missing ?
fyi, I m using the docker version of allinurl/goaccess

time-format %H:%M:%
date-format %d/%b/%

log-format %h %^[%d:%t %^] "%r" %s %b "%R" "%u" %T %^
config-dialog true
hl-header true
html-report-title dwl web stats
json-pretty-print true
no-color false
no-column-names false
no-csv-summary false
no-progress false
no-tab-scroll false
with-mouse true
addr <my public ip>
origin http://<my public ip>
real-time-html true
ws-url <my public ip>:<my port>
log-file /dwl/var/log/apache2/access.log
debug-file /srv/data/debug.log
config-file /srv/data/goaccess.conf
invalid-requests /srv/data/invalid.log
no-global-config true
agent-list false
with-output-resolver false
http-method yes
http-protocol yes
output-format /srv/data/index.html
no-query-string false
no-term-resolver false
444-as-404 false
4xx-to-unique-count false
all-static-files false
double-decode false
ignore-crawlers false
crawlers-only false
ignore-panel REFERRER
ignore-panel KEYPHRASE
real-os true
static-file .css
static-file .js
static-file .jpg
static-file .png
static-file .gif
static-file .ico
static-file .jpeg
static-file .pdf
static-file .txt
static-file .csv
static-file .zip
static-file .mp3
static-file .mp4
static-file .mpeg
static-file .mpg
static-file .exe
static-file .swf
static-file .woff
static-file .woff2
static-file .xls
static-file .xlsx
static-file .doc
static-file .docx
static-file .ppt
static-file .pptx
static-file .iso
static-file .gz
static-file .rar
static-file .svg
static-file .bmp
static-file .tar
static-file .tgz
static-file .tiff
static-file .tif
static-file .ttf
static-file .fl
geoip-database /srv/data/GeoLiteCity.dat

I'd try commenting out addrand origin just to be sure those are not the conflict options. Also, please take a look at your browser's console and see if you can spot some errors on the client side.

I did it however I got this error in the console

Uncaught DOMException: Failed to construct 'WebSocket': The port 65535 is not allowed.
    at Object.setWebSocket (http://<IP_ADDRESS>:65534/:3396:16)
    at Object.initialize (http://<IP_ADDRESS>:65534/:3371:9)
    at window.onload (http://<IP_ADDRESS>:65534/:4847:11)

fyi, I access the report.html file through another container as I don t know how to access the report within the goaccess container

In which port are you running GoAccess' WebSocket server? It seems like the issue is the URL to which the browser is trying to connect. You certainly don't want http in the WS constructor. So if you know the container's address and port where GoAccess is running, then I'd use that under ws-url. For instance,
ws-url ws://<IP_ADDRESS>:8080

Since GoAccess doesn't run a webserver to host the generated report, then you probably need another container serving the HTML report, but @julianxhokaxhiu probably  can chime in on this, since I'm not entirely if a workaround exists.

here my docker cmd:
docker run --name=goaccess --restart=always -d -p 65535:8080 \
    -v /path/to/log/apache2:/dwl/var/log/apache2 \
    -v /path/to/data:/srv/data \
    allinurl/goaccess

still the same error

Did you replace your ws-url option in the config file? Also if you are using a different port, please make sure to add it in your goaccess' config file.
Second I would use the same HTTP port you use to connect to the web interface, like 80 or 443, in order to avoid any firewall issue that may be living in your distribution.
Third, you should be aware that browsers sometimes locks you from using certain ports, because of their special needs. So really, go for 80 or 443 :)

For my understanding:

what is 7890 port for ?
what is 8080 port for ?
How can I debug a ws url - ws://ip:port ?

FYI, here what I understood about the conf required.
I need 2 + x containers :

x containers from which I'll externalize logs and provide them to goaccess
1 for goaccess which will access the logs I provide to it and generate the report.html
1 to access the report.html over http(s)

Am I right ?

To answer your suggestion @julianxhokaxhiu, To which port should I assign 9000 ? 7890 or 8080 ?
Am I missing something in the documentation ?

You should really go for a reverse proxy logic honestly. See https://github.com/julianxhokaxhiu/vps-powered-by-docker as an example.
Then doesn't matter which port do you use. The traffic to the final browser goes towards port 80 or 443. Internally you can route as you like.
If I'll find time later I'll add goaccess as a module to my project, so you can start from that if you prefer.

You should really go for a reverse proxy logic honestly. - Actually that s what I do once I ve validated some new installation and That s what I did following your advises (but I ll definitly look at your reverse proxy)
Howver could you tell me:

what is 7890 port for ?
what is 8080 port for ?
How can I debug a ws url - ws://ip:port ?

What do you mean my debugging sorry? If you want to debug your application behind WS connection, then you may use your preferred IDE.
If you want to know if it works, then it does or it does not. There are no gray shades.

Just adding to @julianxhokaxhiu's reponse, you can also try using telnet from an external network, e.g.,
telnet <host> <port>

@davask let me play locally with your config file and I'll post back.

@allinurl thanks.
do you need me to send it again ?

@davask Unless you already changed some settings, otherwise, feel free to update the one above.

@allinurl you can go like that :)
thks

@davask Please take a look at this section, I've added some instructions on how to run the docker image. Let me know how it goes.

Is it possible to have a generic binding over ws-url or to auto-guess it from goaccess?
The way to do it, makes goaccess not really scalable :/

@allinurl, I do agree with @julianxhokaxhiu

@julianxhokaxhiu Upon opening the generated report in your browser, the report will try to establish a WebSocket connection to the host name (window.location.hostname) of the generated report. That solves the need of not needing to use --ws-url for most cases.
Now in the case of docker, if you are serving the report from a different container (unless supervisor is being used to run the WebServer on the same container) then you probably need to set --ws-url since the WebSocket server is not running on the same container as  window.location.hostname. Now to make this easier from Docker's side, it would be great though if you could simply override GoAccess' config file by passing command line options such as ws-url when docker is ran. e.g.,
docker run -d -p 7890:7890 allinurl/goaccess goaccess --ws-url=xzy --log-format=VCOMBINED etc...

Yeah, that's part of the custom goaccess.conf file, so I would assume that who wants to use Web Sockets, also is willing to place his own custom config file. With that argument inside :)

@julianxhokaxhiu That I agree.
@davask and @julianxhokaxhiu, I just verified this and even if you leave ws-url empty in v1.2, the generated report will successfully establish a connection to the WebSocket server.
This occurs since Docker will bind to <IP_ADDRESS>:7890, which means that GoAccess WebSocket server is using port 7890 and reachable from <IP_ADDRESS> in addition to your host IP.
Also, if you ever need to run it on a different port, e.g.,
docker run --restart=always -d -p 8080:7890 ...

Then you can simply set the external port within ws-url. e.g., ws-url ws://localhost:8080 and keep GoAccess' internal port in your config file set to port 7890. That should do it.


How to predict relative actions on CALVIN when ∆t = 3?

Hi @bdrhtw @hongtaowu67,
In appendix A.4 you mentioned:

We compare the effectiveness of predicting images at different future steps (i.e. 1, 3, and 5) on CALVIN benchmark. Results are shown in Tab. 5. Pre-training is not used in this ablation. It is observed that increasing the step from 1 to 3 improves success rates. This may be because consecutive frames are very similar and predicting frames that are farther away from the current step helps the robot to understand more about the future. But the improvement saturates soon. We hypothesize that this is because the model is trained to predict local actions and predicting frames that are too far into the future may not be able to provide good guidance for immediate local action prediction.

And in appendix A.1:

Robot data is denser compared to the sampled video frame sequences in pre-training. Therefore, when finetuning on robot data, we set ∆t = 3. And we train the network to predict captured images from the static camera and the gripper camera. The input sequence length is set as 10.

The input and output shapes of the network are:
with torch.no_grad():
    prediction = self.policy(
        rgb=rgb_data,                    # [1, 10, 3, 224, 224], rgb_data[:, 1:]=0 in evaluation
        hand_rgb=hand_rgb_data,          # [1, 10, 3, 224, 224], hand_rgb_data[:, 1:]=0 in evaluation
        state=state_data,                # state_data['arm']: [1, 10, 6], state_data['gripper']: [1, 10, 2]
        language=tokenized_text,         # [1, 77]
        attention_mask=attention_mask,   # [1, 10], it's [[1,0,0,0,0,0,0,0,0,0]] in evaluation (0: this part of input is ignored)
   )
   '''
   In the output:
   prediction['obs_preds']:            [1, 10, 196, 768] if 'fwd_pred' in training_target, otherwise None
   prediction['obs_targets']:          [1, 10, 196, 768] if 'fwd_pred' in training_target, otherwise None
   prediction['obs_hand_preds']:       [1, 10, 196, 768] if fwd_pred_hand in training_target and use_hand_rgb=True, otherwise None
   prediction['obs_hand_targets']:     [1, 10, 196, 768] if fwd_pred_hand in training_target and use_hand_rgb=True, otherwise None
   prediction['arm_action_preds']:     [1, 10, 6]
   prediction['gripper_action_preds']: [1, 10, 1]
   '''

If I understand correctly, when finetuning on the CALVIN dataset, the interval between rgb_data[0, 0] and rgb[0, 1] is ∆t = 3. The interval between prediction['obs_preds'][0, 0] and prediction['obs_preds'][0, 1] is also ∆t = 3.
Since you use relative ee action space, prediction['arm_action_preds'] is actually the sum of 3 consecutive relative actions?
For example,

Step 1, a = 1
Step 2, a = 0.5
Step 3, a = 1.9
Then in your training data, ∆t = 3, the actual action a = 1+0.5+1.9 = 3.4?

Hi @StarCycle ,  the target for the rgb prediction is three timestep in the future. But the interval between two timesteps in an input sequence is one.

Hi @bdrhtw,
If I understand correctly, the loss function will be
rgb_mask = batch['mask'].unsqueeze(-1).unsqueeze(-1)[:, 3:]
loss['rgb_static'] = (F.mse_loss(pred['obs_preds'][:, :-3], pred['obs_targets'][:, 3:], reduction='none')*rgb_mask).mean()
loss['rgb_gripper'] = (F.mse_loss(pred['obs_hand_preds'][:, :-3], pred['obs_hand_targets'][:, 3:], reduction='none')*rgb_mask).mean()

So you use the first 7 items of pred['obs_preds'] to predict the last 7 items in pred['obs_targets']. So

The target for the rgb prediction is three timestep in the future. But the interval between two timesteps in an input sequence is one.

can be achieved.
Is my understanding correct?
Best,
Star Cycle

Yes, it's correct.


Able to finish training with one GPU, but got ERROR with "num_gpus_per_node: 2"

What happened?
After revising ‘num_gpus_per_node’ from 1 to 2 in the debug.yml and run "anemoi-training train --config-name=debug.yaml", I got the error below.
here's my debug.yml
defaults:
- data: zarr
- dataloader: native_grid
- diagnostics: eval_rollout
- hardware: example
- graph: multi_scale
- model: gnn
- training: default
- _self_

### This file is for local experimentation.
##  When you commit your changes, assign the new features and keywords
##  to the correct defaults.
# For example to change from default GPU count:
# hardware:
#   num_gpus_per_node: 1

diagnostics:
  plot:
    enabled: False
hardware:
  num_gpus_per_node: 2
  #num_nodes: 1
  #num_gpus_per_model: 2
  paths:
    data: /home/john/test/anemoi
    output: /home/john/test/anemoi/outputs/
    checkpoints: ${hardware.paths.output}checkpoint/
  files:
    dataset: dataset.zarr
    checkpoint:
      every_n_epochs: anemoi-by_epoch-epoch_{epoch:03d}-step_{step:06d}
      every_n_train_steps: anemoi-by_step-epoch_{epoch:03d}-step_{step:06d}
      every_n_minutes: anemoi-by_time-epoch_{epoch:03d}-step_{step:06d}
    warm_start: null
    # graph: ???
  # num_gpus_per_node: 2
  # num_nodes: 1
  # num_gpus_per_model: 1
model:
  num_channels: 128
dataloader:
  limit_batches:
    training: 100
    validation: 100
training:
  max_epochs: 5

configuration: 4090*2;  ubuntu 22.04;
python env: mainly following anemoi documentation
What are the steps to reproduce the bug?
Following the anemoi documentation do the training

anemoi-training train to generate  debug.yaml and etc.
successfully do the first training with debug.yaml(num_gpus_per_node: 1)
get error when changing num_gpus_per_node from 1 to 2

Version
develop
Platform (OS and architecture)
ubuntu22
Relevant log output
...
usage: anemoi-training [-h] [--version] [--debug] {checkpoint,config,mlflow,train} ...
anemoi-training: error: argument command: invalid choice: 'hydra.run.dir="outputs/2024-10-10/11-02-12"' (choose from 'checkpoint', 'config', 'mlflow', 'train')
[2024-10-10 11:02:24,571][lightning_fabric.utilities.distributed][INFO] - Initializing distributed: GLOBAL_RANK: 0, MEMBER: 1/2
[2024-10-10 11:02:29,494][lightning_fabric.strategies.launchers.subprocess_script][INFO] - [rank: 1] Child process with PID 629502 terminated with code 2. Forcefully terminating all other processes to avoid zombies 🧟
Killed

Accompanying data
No response
Organisation
No response

Hi, we are aware of this issue and are currently testing a fix. Should have an update for you soon.
If you happen to be on a slurm cluster, multi-GPU training does work when you run anemoi-training train via srun.

Thank you so much for your quick response and for working on this! I appreciate the update :)
If it's not too much trouble, would it be possible to share an estimated timeline for when the fix might be available? @gmertes Looking forward to hearing from you.

The PR with a fix should go up by tomorrow and then we will see how long review takes 😄

This should be fixed in 0.2, give it a try and let me know if you're still having issues.
pip install -U anemoi-training


Metafacture core 5.3.1rc

This is the branch for testing metafacture-core release candidate 5.3.1.
It follows up https://github.com/metafacture/metafacture-core/issues/397.
Based on the experience we've made with the release of metafacture-core 5.3.0 (which is deprecated resp. not recommended for usage) it was  discussed with @blackwinter  the need to better test release candidates before actually releasing them (this is also reflected now in the Maintainer Guidelines).
@blackwinter found the dependency upgrade of log4j:1.2.12' to log4j-core:2.14. problematic, so this is reverted in 36ed96920c70a0173abb75f39ed28ca8bbba7bab.
I have tested 5.3.0 against lobid-resources and found an incompatibility which I reverted in 7c1ea04c6012fbea661a86d73e6c0ef8f8f53ff8.
Furthermore, I found out that if you are using a metamorph.xsd of your own and make use of FileMap you have to also update your locally metamorph.xsd (like the one at https://github.com/hbz/lobid-resources/blob/master/src/main/resources/schemata/metamorph.xsd):
-      <attribute name="separator" type="string" use="optional" default="\t">
+      <attribute name="separator" type="string" use="optional" default="	">

As this will occur only quite rarely (is there anybody out there using a local duplicate of metamorph.xsd ?) and this is the result of a bug fix (d528ac973a7b6a1bb0b846a6d90950ace9553742) (in effect the default separator defined (falsely) in metamorph.xsd has had no effect at all) this "breaking" fix will be part of the release (or is anyone against this?) but the release would come with the here mentioned note.

Are there actual release artifacts to test? I can't find any 5.3.1-rc1 on Sonatype or GitHub.

found an incompatibility which I reverted in 7c1ea04.

That's 462c8f8 now, right?

refs/tags/metafacture-core-5.3.1-rc

IMO, release candidates should always be numbered.

That's 462c8f8 now, right?

right

IMO, release candidates should always be numbered.

did so now
The snapshot can be consumed as feature-5.3.1-rc1-SNAPSHOT in your building system. Note to properly configure it, e.g. for maven add to your pom.xml:
<repositories>
    <repository>
        <id>oss.sonatype.org-snapshot</id>
        <url>https://oss.sonatype.org/content/repositories/snapshots</url>
        <releases>
             <enabled>false</enabled>
        </releases>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
    </repository>
</repositories>

See the (hopefully now) comprehensive how-to in the wiki.

don't get why these changes were made in an RC branch.

hmhm.  We should first merge this PR into master and then make an rc, right? Hm, but then how to review properly, because we want to test this PR within our programs.
Do you propose to first merge this PR into master (you have already given your +1 so we can do this right now) and then build an rc and open an issue to test this? (+1 from me)

We should first merge this PR into master and then make an rc, right?

Yes, that would be my suggestion.

Hm, but then how to review properly, because we want to test this PR within our programs.

To me, those are different things. Want to review a pull request/feature branch? Publish a preview release (feature-<branch>). Want to review an upcoming release? Publish a release candidate (from the main/release branch; ideally, without the feature- prefix). I admit that this comes with a bit of overhead as long as the releases have to be performed manually.

you have already given your +1 so we can do this right now

There are multiple reviewers. Shouldn't all of them get a chance to test the release artifacts?

There are multiple reviewers. Shouldn't all of them get a chance to test the release artifacts?

But I understand your proposal to first merge into master and then make release artifacts and open an issue to let them test the release artifacts?

But I understand your proposal to first merge into master and then make release artifacts and open an issue to let them test the release artifacts?

Yes, but the question is what they should test. This PR (they're listed as reviewers after all) or the RC (which, for now, coincides with this PR)?

Thx for the clarification , also thx for the gradle config in the wiki!
It's agreed for normal PRs that one +1 is sufficient, so I've merged this PR.
Opened a new issue for testing the already buildt release artifacts at #429, reassigning the reviewers in question.

OK, thanks! That should make things a little clearer.


Fix the name of amule indexer in docs

:tada: This PR is included in version 1.5.2 :tada:
The release is available on:

GitHub release
v1.5.2

Your semantic-release bot :package::rocket:


pytorch-bin: init at 1.4.0

Not perfect, literally clumped together by copying bits from tensorflow-bin.
Motivation for this change
But in case anyone comes by and wants a CUDA-enabled pytorch without compiling from source for Linux.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

I would prefer that we just take the pytorch built cleanly from source.  I'm working on getting https://github.com/NixOS/nixpkgs/pull/75827 polished up, at which point I don't think we'll need this.


NSUUID cannot be parsed

i got an uuid from my api, i tough it was going to be parsed as a String, then when i tried
let text = json["uuid"].string!
print(text)
``

it prints:
<__NSConcreteUUID 0x15ff16a0> 32D3636A-7B17-447F-95A1-E4C902D4AED0

but when i get the json["uuid"].rawValue it gets a pioner or something like that
if i search json["uuid"].type it says is string.... so i don't know what to do, im getting the last characters of the description of the object but thats not the idea

There hasn't been much activity on this issue for a while so I'm gonna go ahead and consider it fixed. Please let us know it you'd like to reopen it.


Updated links to new repo name

Looks like vercel will also have to be reconfigured to use the new repo name. Sucks that they don't pick it up automatically. @diptanu can you update that?


Projectionist file create commands don't have a folder for context

One thing I'm noticing as far as Projectionist features are concerned is that running commands to create files such as :Econtroller post creates the file in /lib/controllers/post_controller.ex instead of the intended /lib/blog_web/controllers/post_controller.ex.
The controller itself also lacks the project name and results in:
defmodule .PostController do
  use , :controller
end

When switching between files with :A the plugin has the proper context to know what folder sits in the glob for /lib/**/controllers/post_controller.ex, but the creation commands don't have any context.
I'm unsure if we're able to provide that context to Projectionist.

I think you're supposed to do :Econtroller blog_web/post

Ah yeah that does work! I suppose that makes sense with the dirname and basename used in the template and the fact Projectionist really wasn't made for Elixir.
Makes me wish it had a pluggable system that we could use to inject the correct name in when in an Elixir project but that's definitely out of scope here!


Need Silicon Valley

Plz

What exactly are you asking for?

Quotes or other material from the show

Someone had previously submitted a Silicon Valley one that has been merged. I added some more fun SV stuff in PR 1048

@tjgillies Faker now has Silicon Valley support and @russellschmidt kindly jumped in and helped make it more robust. Unless there's anything else you think needs covering are you happy for this issue to be closed? :)


Bloom causes massive performance hit (>50% framerate reduction)

OS = "Win11"
Bevy version = "0.14.0-dev" && "0.13.2"
AdapterInfo { name: "AMD Radeon RX 7700S", vendor: 4098, device: 29824, device_type: DiscreteGpu, driver: "AMD proprietary driver", driver_info: "24.3.1 (LLPC)", backend: Vulkan }
[profile.dev.package."*"]
opt-level = 3
[profile.dev]
opt-level = 3
[profile.release]
lto = true
opt-level = 3
codegen-units = 1
incremental = false
debug = false
What you did
Enabled bloom

What went wrong
Enabling bloom gives a massive hit to performance in both debug and release. In these examples I'll use debug (as nothing changed between them in RenderDoc except overall framerate).
In other game engines I've never personally experienced a hit this large from bloom. This was tested with both a personal example and the bloom_2d example in 0.14, as well as a simple bloom setup in 0.13.2. I tested bloom in Unity with a similar scene for a quick comparison and the performance impact in Unity is around 10% give or take - which iirc from using other engines is about the standard hit.
In Bevy issues I did a quick search for issues related to bloom and didn't see any mention of a performance profile on the bloom pass. With that in mind I figured opening up an issue would be best. If I missed an open issue relating to this please point me in the right direction - as I plan to explore the bloom passes code more in the future.
Bloom off:

Bloom on:

RenderDoc info
Full time:

Bloom pass:

Conclusion:
Based on this information from RenderDoc - it would seem that the bloom pass is heavier on rendering than everything else combined. Bloom takes up more than 50% of the render time when enabled.

RenderDoc is not really a great GPU profiling tool. You'll want to use NSight/RGP/Xcode/GPA/etc (Nvidia, AMD, Apple, Intel) depending on your GPU manufacturer.
CPU metrics are also important for rendering - maybe recording so many render passes is expensive. Tracy will show you that.
I can take a look later this week and figure out why it's expensive.

RenderDoc is not really a great GPU profiling tool. You'll want to use NSight/RGP/Xcode/GPA/etc (Nvidia, AMD, Apple, Intel) depending on your GPU manufacturer.
CPU metrics are also important for rendering - maybe recording so many render passes is expensive. Tracy will show you that.
I can take a look later this week and figure out why it's expensive.

Thanks for the tip.
The performance hit happens without profiling and can be seen in the first two bloom on/off pictures. Those pictures are without profiling. There I am just using bevy frame diagnostics to track the frames per second. Sorry for not clarifying that when using those pictures to explain the issue.
In this case - all the profiling is doing is confirming the >50% reduction is in fact coming from the bloom pass. The bloom pass taking up just over 50% of the render time in the profiling lines up 1:1 with the framerate reduction seen when not profiling.
Currently I'm taking a look into the first two downsampling and last two upsampling passes in the code to see if I can get more information or optimize anything. Please also ignore me open/closing the issue. Misclick!

I reduced bloom.wgsl to its simplest form and only noticed a negligible increase in performance with bevy frame diagnostics and the profiler (somewhere between 5-10 fps improvement?). Here is the reduced code:
struct BloomUniforms {
    threshold_precomputations: vec4<f32>,
    viewport: vec4<f32>,
    aspect: f32,
};

@group(0) @binding(0) var input_texture: texture_2d<f32>;
@group(0) @binding(1) var s: sampler;
@group(0) @binding(2) var<uniform> uniforms: BloomUniforms;

fn rgb_to_srgb_simple(color: vec3<f32>) -> vec3<f32> {
    return pow(color, vec3<f32>(1.0 / 2.2));
}

fn sample_input_4_tap(uv: vec2<f32>) -> vec3<f32> {
  let j = textureSample(input_texture, s, uv, vec2<i32>(-1, 1)).rgb;
  let k = textureSample(input_texture, s, uv, vec2<i32>(1, 1)).rgb;
  let l = textureSample(input_texture, s, uv, vec2<i32>(-1, -1)).rgb;
  let m = textureSample(input_texture, s, uv, vec2<i32>(1, -1)).rgb;
  var sample = (j + k + l + m) * 0.125;
  return sample;
}

fn sample_input_mini_tent(uv: vec2<f32>) -> vec3<f32> {
    let x = 0.004 / uniforms.aspect;
    let y = 0.004;
    let e = textureSample(input_texture, s, vec2<f32>(uv.x, uv.y)).rgb;
    let a = textureSample(input_texture, s, vec2<f32>(uv.x - x, uv.y + y)).rgb;
    let c = textureSample(input_texture, s, vec2<f32>(uv.x + x, uv.y + y)).rgb;
    let g = textureSample(input_texture, s, vec2<f32>(uv.x - x, uv.y - y)).rgb;
    var sample =  (a + c + g + e) * 0.25;
    return sample;
}

@fragment
fn downsample_first(@location(0) output_uv: vec2<f32>) -> @location(0) vec4<f32> {
    //let sample_uv = output_uv;
    var sample = sample_input_4_tap(output_uv);
    return vec4<f32>(sample, 1.0);
}

@fragment
fn downsample(@location(0) uv: vec2<f32>) -> @location(0) vec4<f32> {
    return vec4<f32>(sample_input_4_tap(uv), 1.0);
}

@fragment
fn upsample(@location(0) uv: vec2<f32>) -> @location(0) vec4<f32> {
    return vec4<f32>(sample_input_mini_tent(uv), 1.0);
}

After moving on to the render code.. So far my only clue has been this:

When I divide into the mip dimensions in bloom/mod.rs to reduce it, I get a good portion of the the frames back.

Obviously this isn't a solution or anything. Just sharing what I found before throwing my hands in the air for the day.

On my system, at 1080p, it takes 355 microseconds on the CPU (according to Tracy) to encode rendering commands for bloom, 0.20ms of GPU time (according to NSight) to execute those commands.
2024-04-27T06:53:32.154332Z  INFO bevy_diagnostic::system_information_diagnostics_plugin::internal: SystemInfo { os: "Windows 11 Home", kernel: "22631", cpu: "AMD Ryzen 5 2600 Six-Core Processor", core_count: "6", memory: "15.9 GiB" }
2024-04-27T06:53:32.710031Z  INFO bevy_render::renderer: AdapterInfo { name: "NVIDIA GeForce RTX 3080", vendor: 4318, device: 8710, device_type: DiscreteGpu, driver: "NVIDIA", driver_info: "551.61", backend: Vulkan }

Yes I don't see much of a slowdown between bloom on/off. Just to check are you only toggling bloom? You're leaving tonemapping and Camera::hdr the same between runs?

I'm turning all three on yes, but tonemapping itself can be on/off without a difference. I'll provide the code just to fully clarify.
250fps (hdr on w/ bloom on and tonemapping (optional):
fn setup(mut commands: Commands) {
    commands.spawn((
        Camera2dBundle {
            camera: Camera {
                hdr: true,
                ..Default::default()
            },
            //tonemapping: Tonemapping::AcesFitted
            ..default()
        },
         Name::new("light_camera"),

         BloomSettings::default(),
    ));
 }

700-750+fps (nothing on):
fn setup(mut commands: Commands) {
    commands.spawn((
        Camera2dBundle {
            camera: Camera {
                ..Default::default()
            },
            ..default()
        },
         Name::new("light_camera"),
    ));
 }

500fps (hdr on w/ bloom off):
fn setup(mut commands: Commands) {
    commands.spawn((
        Camera2dBundle {
            camera: Camera {
                hdr: true,
                ..Default::default()
            },
            ..default()
        },
         Name::new("light_camera"),
    ));
 }

And this happens in every case I've tried running it. Different version, local fresh bevy deps, using bevys examples etc.

Could you use frame times instead of frame rates?
700 fps: 1.43ms
500 fps: 2.00ms (so, 0.57ms slower)
250 fps: 4.00ms (an additional 2ms slower)

@superdump sure np i'll make sure to convert to ms whenever I can moving forward. Sorry about that.

It might also help to note that this occurs even without any actual bloom in the scene. Blank screens get the same hit when changing the camera settings.

Just to confirm what @JMS55 was saying, I tested it on an older Nvidia machine with a different display (1050 ti, 1080p display) also using Vulkan and the performance loss was a tad bit less, but for me it was still around 25-30% on average. I'd be interested in seeing benchmarks on different resolutions. For now I'm just going to take the node out, as I don't really need it in my graphics pipeline atm anyways. If there's anything else I can share that will help, just let me know.

It's possible this is due the large number of separate render passes that do very little to no work.
Especially on lower mip levels the GPU does mostly nothing and is stalled by barriers (previous passes) and other overhead:

It also looks like there's a clear pass for each level of the mip chain, which I feel like isn't needed.

I've also noticed this with simple scenes. It should be possible to rewrite bloom to use a compute shader for down- and upscaling like SPD.


Improve performance to ConsumeQueue when reclaiming disk space

When commit log files are deleted, ALL consume queues scan its last file linearly, from beginning to its written position. RocketMQ is designed to serve a large number of topics, thus an even bigger number of consume-queues... This style of scan becomes very inefficient when these numbers grow larger and system memory is tight.
We recently observed this correction procedure takes a significant amount of IO bandwidth, impacting QoS noticeably in production.
A few improvements can be made to mitigate/resolve this issue observed.

For topics/consume queues that are deprecated, this scan should be zero overhead;
Scan should start from the previous min-offset wherever possible, instead of the beginning of the first file;
The scan should employ binary search as entries in consume queue are naturally sorted.

@duhenglucky Regarding the release plan. It's known that RocketMQ has released its LTS version a few days ago. However, LTS normally has follow-up minor updates to further consolidate its support.  For example, Ubuntu has very good release cycle management. See https://wiki.ubuntu.com/Releases
After releasing Ubuntu 20.04 LTS, it further released Ubuntu 20.04.1 LTS, Ubuntu 20.04.2 LTS Ubuntu 20.04.3 LTS Ubuntu 20.04.4 LTS to demonstrate continuous support.
With that said, shall we have a release plan, like <IP_ADDRESS> LTS, to accept fixes similar to this pull request?


[INLONG-10809][SDK] Improvements to TypeConverter field types and CompareValue in OperatorTools

Fixes #10809
Motivation

Modified the SourceData interface to enable it to return multiple data types; Based on this, the CsvSourceData class was modified.
To enable the functionality of field types, modify the FieldInfo class to have a default TypeConverter.
Based on the TypeConverter interface, two converters, DoubleConverter and LongConverter, have been implemented.
In order to enable InLong Transform to parse floating-point data types in SQL statements, the DoubleParser class is added.
Modify the OperatorTool.compareValue method to align the types of both parties being compared.
Add IfFunction class for testing various classes of ExpressionOperator.

Verifying this change
(Please pick either of the following options)

[ ] This change is a trivial rework/code cleanup without any test coverage.

[x] This change is already covered by existing tests, such as:
(please describe tests)

[ ] This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

please add more details about this issue

More information has been added. @vernedeng


GPG key isn't required to be present before the repositories

The outcome from #72 and #73 means that the GPG key now isn't guaranteed to be imported before the Yum repositories are created which means if I install a package found in these repositories, I get the following errors:
GPG key retrieval failed: [Errno 14] curl#37 - "Couldn't open file /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7"

And then a bit further on, I finally see the GPG key get created:
Notice: /Stage[main]/Epel/File[/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7]/ensure: defined content as '{md5}58fa8ae27c89f37b08429f04fd4a88cc'
Notice: /Stage[main]/Epel/Epel::Rpm_gpg_key[EPEL-7]/Exec[import-EPEL-7]/returns: executed successfully

I agree the original dependency chaining was a bit far reaching, but I think there needs to be something here to keep the resources within the context of this module in the correct order.

Fixed in https://github.com/voxpupuli/puppet-epel/pull/86


scripy.sparse.linalg. cgs / cg / bicg   has wrong results (?)  with  complex  linear operator

https://github.com/scipy/scipy/blob/2e5883ef7af4f5ed4a5b80a1759a45e43163bf3f/scipy/sparse/linalg/_isolve/iterative.py#L412
Not only cgs can not support the linear equation of complex numbers ( using linear operator)
Those are claimed to support (such as cg, bicg) also do not support complex

Can you post the code that caused the problem?

(1)  Afun= lambda x: At(A(x))
(2) AtA= spla.LinearOperator(shape=(n,n),matvec= Afun)
(3) x2, info = spla.cg(A=AtA2, b=np.imag(z0), x0=np.imag(x0), tol=1e-05, maxiter=5, M=None, callback=None,atol=None)
Here At， A is also defined using lambda. We can explain it： A is a matrix m×n,  At is a matrix nxm,  x is a vector n×1
At，A, x  are all complex
Line 3 can work out, but the results are wrong.  others such as bicg can not support the Lin3 expression (do not why ?)

@MR4LEAF please follow our template for the issue. We need a clear minimal reproducible example with all variables defined, what you expected numerically, etc.

MATLAB version:
function example()
%%
dims= [128,128];
image= randn(dims);
image= image.exp(1jimage);
%% fft2
ampl_fft= fft2(image)/sqrt(numel(image));
image_true= sqrt(numel(image))*ifft2(ampl_fft);
e1= norm(image- image_true,'fro');
%% iteration
Afun= @(x) getfft(x,dims);
sol=cgs(@(x) Afun(x),ampl_fft(:),1e-3,30,[],[], image(:) + 0.01*randn(numel(image),1));
x0= sol;
%%
e2= norm(x0-image(:),2);
%
figure(1);clf
plot(e1,'b*');hold on;grid on
plot(e2,'rs')
function y= getfft(x,dims)
x_matrix= reshape(x,dims);
y_matrix= fft2(x_matrix)/sqrt(dims(1)*dims(2));
y= y_matrix(:);
end
end

@MR4LEAF thank you but can you also show the output of both? Some people don't have access to Matlab.


ansi-styles and support-color correction

Cannot find module '#ansi-styles' from 'node_modules/chalk/source/index.js'
Cannot find module '#supports-color' from 'node_modules/chalk/source/index.js'
I was doing unit tests with Chalk 5 using Jest and we got these two errors. I made some changes to make it work, I'm using "ansi-styles" in version "^6.1.0" and "support-color" in version "^9.2.2". After securing the latest version, remove the "#" in the import of the two in "source/index.js". At Jest, I work with ECMAScript Modules in their experimental support. This change solved the problem, being the reason for me opening this "pull request", I don't know if it's the best way to solve these errors. But, that's what worked in my application!

Please search next time, this is intended.

Lost in ignorance, but could you tell me why? Honestly, I didn't find about this question on the web. It would be of great help to my learning. Thank you very much in advance

Lost in ignorance, but could you tell me why? Honestly, I didn't find about this question on the web. It would be of great help to my learning. Thank you very much in advance

https://github.com/chalk/chalk/issues/535#issuecomment-1082644840


Enable finding boost::leaf from locally installed boost library.

Describe your problem
boost::leaf has been added into boost's distribution from boost 1.75.0. It can be found from locally installed environment (e.g., installed using brew) to avoid duplicate installation (and possible overwrite the local version).
Additional context
This may requires further work on upstream (on CMake).

Close as CMake's FindBoost.cmake doesn't support that: https://github.com/Kitware/CMake/blob/master/Modules/FindBoost.cmake


Visiting Turbolinks enabled links in Chrome shows the page with no CSS for a split second before showing it normally

How to replicate:

Visit http://photoid.herokuapp.com/users/sign_in on Chrome
Go back and forth between 'Login' page and the 'Register' page from the links on the navbar
You'll see that the page is displayed with no CSS for a second or two before changing to normal

Screenshots:
No CSS:

With CSS:

This doesn't happen in Firefox. I've tested it only on Chrome and Firefox so far so I cannot speak for other browsers. Please look into this. Thanks!

Hey @Arif59! I don't think this is a Turbolinks problem. The issue appears to be with your site's HTML. The stylesheet links, in Chrome, are being rendered inside the <body> element. That's what's causing the flash of unstyled content – your stylesheets are being reloaded on each render.
This is probably because the markup has an errant closing </html>:
<!DOCTYPE html>
<html></html>
<head>


Fix typo. Retrieve SDK key from MoPub dashboard

I fixed an error on the Android adapter for MoPub, the interstitial file was missing a "}" at the end.
Also, I added support so that we can now use the "custom event class data" on MoPub's dashboard to initialize Applovin, the JSON object required is below. In case this field is empty, the behavior of Applovin is the same as before (it needs to edit the AndroidManifest file to ad a  tag).
{"sdk_key":"SDK_KEY_HERE"}

Thanks for the pull request. Looks reasonable to me. If @applovinjosh concurs we'll happily merge.

Hey @mparodi, after a bit of internal discussion we've decided not to merge this into the mainline adapter. There are downstream integration implications to whether the SDK key is used in the manifest or in a getInstance call, particularly if our SDK is additionally used outside the MoPub medation layer.
So we don't really want to have this behavior dependent on what's going on in the MoPub dash :)
That being said we do appreciate your pull request. We'll fix the typo reported. And of course if providing this key from the dash works well for you in your app, obviously there's nothing wrong with that approach.


go generator doesn't handle JS providers

I have a prisma file setup in a Go + TS monorepo as
generator client {
  provider        = "prisma-client-js"
  previewFeatures = ["jsonProtocol", "views"]
}

generator golang {
  provider        = "go run github.com/steebchen/prisma-client-go"
  previewFeatures = ["views"]
}

Running npx prisma migrate dev --schema packages/db/prisma/schema.prisma works
✔ Generated Prisma Client (4.14.0 | library) to ./node_modules/@prisma/clie
nt in 120ms
✔ Generated Prisma Client Go to ./packages/db/prisma/db in 1.28s

However, running equivalent (just generate) with Go CLI fails
Error: Generator<EMAIL_ADDRESS>failed:

node:internal/modules/cjs/loader:936
  throw err;
  ^
Error: Cannot find module '/home/hua/code/monorepo/--max-old-space-size=8096'
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function._resolveFilename (pkg/prelude/bootstrap.js:1951:46)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Function.runMain (pkg/prelude/bootstrap.js:1979:12)
    at node:internal/main/run_main_module:17:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

panic: could not run [generate --schema packages/db/prisma/schema.prisma]: exit status 1

goroutine 1 [running]:
main.main()
        /home/hua/go/pkg/mod/github.com/steebchen/prisma-client-go@v0.18.0/main.go:29 +0x34b
exit status 2

When I comment out the prisma-client-js generator, the golang command works fine. This is on a WSL / amd64 instance.
I am not blocked on this, since the npx command is able to generate correctly.
Edit -- this may be a dupe of #455

Yeah so basically the Go client is a custom generator which means it's supported, however the JS generator is built into Prisma so the Go client can't directly invoke it. So the best thing to do here is using npx to generate both clients. If you do want to generate the Go client with Go only, you should use a single generator and provide the provider variable via an env var depending on which client you want to generate.

Makes sense! I think having a note in the docs to that effect would be helpful, or perhaps handling it in the code (refusing to run the JS generator), but docs are probably sufficient

yes will be done via #880 and #455, closing this in favor of these

related: https://github.com/prisma/prisma/issues/10564


Fix site target failing on Javadocs

Javadocs failed because of a syntax error.
By default, Travis CI skips building Javadocs. This PR also should enable building Javadocs on the Linux without Docker CI job.

Pull Request Test Coverage Report for Build 146

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 99.73%

Totals

Change from base Build 145:
0.0%

Covered Lines:
370

Relevant Lines:
371

💛  - Coveralls


S3 receiver

References -

#34

Looks like abandoned one? Should we drop it and focus on other things?

Its WIP rn,  sort of like an add on. Currently working on imporving the llm one

Converted to draft just to know it's WIP

Cool merging this


Module development (how to)

I'm trying to add my first module on this system
But I'm stuck on registering provider Module.php
It seems like the only way to add module is to manually register the provider to mage2/ecommerce module.
Is there a proper way to add Module?

you can do it inside config/app.php provider section
just use module.php file namespace into config/app.php (Provider section)
Mage2\ModuleName\Module::class,

I hope this would help.
For autoloading file you can add namespace into composer,json and use
composer dump-autoload

thanks

Thanks, I'll go for it :)


APEXCORE-655 Support archetypeVersion=LATEST when generating project.

@davidyan74 please review

@vrozov made the changes


Support Psych v4.0.0

Fixes #20
Ruby master ships with Psych 4.0.0 which makes YAML.load
defaults to safe mode (https://github.com/ruby/psych/pull/487).
Keep compatibility by using unsafe_load.

Since this is basically abandonware, @tmimura39 is there a different public fork you or others are using?


AbemaTV batch 3/3: [AbemaTV] add support for abema.tv

Please follow the guide below

You will be asked some questions, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your pull request (like that [x])
Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

[x] At least skimmed through contributing guidelines including yt-dlp coding conventions
[x] Searched the bugtracker for similar pull requests
[x] Checked the code with flake8

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check one of the following options:

[x] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[ ] Bug fix
[ ] Improvement
[x] New extractor
[ ] New feature

Description of your pull request and other information
This adds support for abema.tv, which is a broadcast service in Japan. Geo-restricted.
Please do not merge before #1686 and #1687. This PR requires new functionality in these PRs. :tada:

This PR is ready to review and merge without waiting for network stack overhaul. I'll rewrite network handler code after these PRs are merged.


Using eventing broker tests here.

yikes... 2020-11-25T23:28:09.4026299Z   message: '0/2 nodes are available: 1 Insufficient cpu, 1 node(s) had taints that 2020-11-25T23:28:09.4027068Z     the pod didn''t tolerate.'

@n3wscott are you still planning on doing something with this?

@n3wscott are you still planning on doing something with this?

I will update, just trying to get back to this repo...

I will update, just trying to get back to this repo...

/close


Fix word detection crashing if string input is starts with a symbol, letter or normal word before the profanity

Fixed when someone swears with a symbol, letter or word at the beginning the bot crashes

Thanks for fixing the error! Should be implemented in the newest version, v0.0.4 👍


[Do not merge] Document ignore exit-code functionality

Summary
Do not merge yet as the feature is not yet publicly available. I'll ping here when it's available (it should be in about a week).

Internal previews

📄 File
🔗 Preview link

docs/core/testing/unit-testing-mstest-runner-exit-codes.md
MSTest runner exit codes

docs/core/testing/unit-testing-mstest-runner-intro.md
MSTest runner overview

Oh damn @IEvangelist, I wasn't expecting that changing title would lead to merge. When is the official doc updated? I can do a follow-up commit to say this will be available only on final.

Oh damn @IEvangelist, I wasn't expecting that changing title would lead to merge. When is the official doc updated? I can do a follow-up commit to say this will be available only on final.

It will go live today. For future reference, you can use Draft PRs to ensure that they don't get merged until you're ready. If a PR is not in draft form, and approved - we like to merge them. :)

Yeah my bad here!


Port DeeplabV3+ from segmentation_models_pytorch

@qubvel I've ported your DeeplabV3+ implementation from segmentation_models_pytorch. I would appreciate your feedback. :)

Hi @kamil-rafalko
DeeplabV3+ implementation requires dilated convolutions in a backbone, however current backbones do not support this kind of option


Verify Auth Challenge Response role name is too long

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
v16.13.1
Amplify CLI Version
7.6.21
What operating system are you using?
Mac
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
No
Amplify Categories
auth
Amplify Commands
push
Describe the bug
I created environment called production and added auth with custom signup flow with and Cognito triggers. I added Verify Auth Challenge Response trigger and tried to push.
I get error:
CREATE_FAILED      LambdaExecutionRole                                                                                AWS::IAM::Role             Mon Mar 07 2022 15:00:47 GMT+0100 (Central European Standard Time) 1 validation error detected: Value '*************VerifyAuthChallengeResponse-production' at 'roleName' failed to satisfy constraint: Member must have length less than or equal to 64 (Service: AmazonIdentityManagement; Status Code: 400; Error Code: ValidationError; Request ID: ; Proxy: null)
CREATE_FAILED      amplify--production-91310-****************VerifyAuth-19IOBUHMU662B AWS::CloudFormation::Stack Mon Mar 07 2022 15:00:48 GMT+0100 (Central European Standard Time) The following resource(s) failed to create: [LambdaExecutionRole].
Total length of Value '****************VerifyAuthChallengeResponse-production is 68 characeters. My app name is 13 characters long
Expected behavior
It should allow me to push changes and fix name automaticaly or allow me to set the shorter name for lambda.
Reproduction steps

Create a new amplify backend with name of 13 characters (that's our name length)
Create a new environment called production.
Add auth and Lambda triggers for Cognito.
Push

? Do you want to configure Lambda Triggers for Cognito? Yes
? Which triggers do you want to enable for Cognito Create Auth Challenge, Define Auth Challenge, Pre Sign-up, Verify Auth Challenge Response
? What functionality do you want to use for Create Auth Challenge Create your own module
? What functionality do you want to use for Define Auth Challenge Create your own module
? What functionality do you want to use for Pre Sign-up Create your own module
? What functionality do you want to use for Verify Auth Challenge Response Create your own module

GraphQL schema(s)
No response
Log output
No response
Additional information
No response

Thanks for reporting it. I have confirmed this is a bug

To avoid these types of limits, we probably shouldn't be providing a role name at all and instead letting CFN generate one for us. I'm not sure what kinds of downstream impacts a change like this will have though particularly on amplify client libraries.


Mrfetcher ignores failed tasks

This changes fixes a couple of things

We have been ignoring Failed Tasks in calculation of resource usage. This handles that.
For failed jobs, the Exception heuristic is supposed to give the stacktrace. That wasn't working currently. this changes fixes that.

I still need to test this change with the FS fetcher and potentially add a couple of unittests.

@akshayrai Fixed your comments. Take a look and merge if it looks good.


Using tor as http proxy

Using current stable version of gnupg and curl it's possible to use tor socks proxy as http-proxy option in keyserver-options. So there is no need for torsocks anymore.
$ gpg --keyserver-options http-proxy=socks5://<IP_ADDRESS>:9050 --search<EMAIL_ADDRESS>gpg: searching for<EMAIL_ADDRESS>from hkps server hkps.pool.sks-keyservers.net
gpg: key<EMAIL_ADDRESS>not found on keyserver

On the other hand current implemation breaks if usage of tor is defined in gnupg configuration (~/.gnupg/gpg.conf). In this scenario torsocks adds a second tor layer and therefore connection fails:
$ torsocks gpg --keyserver-options http-proxy=socks5://<IP_ADDRESS>:9050 --search<EMAIL_ADDRESS>gpg: searching for<EMAIL_ADDRESS>from hkps server hkps.pool.sks-keyservers.net
[Mar 10 01:59:15] WARNING torsocks[10135]: [connect] Connection to a local address are denied since it might be a TCP DNS query to a local DNS server. Rejecting it for safety reasons. (in tsocks_connect() at connect.c:177)
gpgkeys: HTTP search error 7:
gpg: key<EMAIL_ADDRESS>not found on keyserver
gpg: keyserver internal error
gpg: keyserver search failed: keyserver error

Therefore parcimonie.sh could not be used if tor should also be used for all other keyserver operations of gnupg. This is bad.
Tested with gnupg 1.4.16 and curl 7.37.1 running Ubuntu 14.10.

It's bad, but at least it's not bad in a way that compromises the user.
I also rather prefer the torsocks approach because it shields against the case of a bug in gnupg not using the proxy, or poorly implementing it (doing DNS lookups in the clear or whatnot). I have no grudge against gnupg, but the people I trust the most to do proper Tor-ing of applications are the Tor people.
I also can't just make parcimonie.sh silently override --keyserver-options, because that will (probably silently) break people's setups if they have GNUPG_KEYSERVER_OPTIONS unset and a keyserver-options option set in their gnupg config file. I personally have a setup like this, in order to set a custom ca-cert-file value. In order to behave properly in such a setup, parcimonie.sh would need to parse the keyserver-options part of the gnupg config file, merge it together with GNUPG_KEYSERVER_OPTIONS, and finally add an extra http-proxy=socks5://<IP_ADDRESS>:9050 on top in order to act correctly. This isn't impossible, but it's not trivial in Bash either.
How about adding a new configuration variable like USE_TORSOCKS, enabled by default. If explicitly disabled, it is assumed that the user has configured gnupg correctly (either through the config file or through GNUPG_KEYSERVER_OPTIONS)? Would this be OK?

Partly override keyserver-option seems to be possible. So you could just override http-proxy but not ca-cert-file and other keyserver-options configuration:
Overriding http-proxy option in keyserver-options:
$ cat ~/.gnupg/gpg.conf
# FILE CREATED BY SEAHORSE
keyserver hkps://hkps.pool.sks-keyservers.net
keyserver-options ca-cert-file=/home/main/.gnupg/sks-keyservers.netCA.pem http-proxy=socks5://<IP_ADDRESS>:9050

$ torsocks gpg --keyserver-options http-proxy= --search<EMAIL_ADDRESS>gpg: searching for<EMAIL_ADDRESS>from hkps server hkps.pool.sks-keyservers.net
gpg: key<EMAIL_ADDRESS>not found on keyserver

Proving that ca-cert-file option was still used:
$ cat ~/.gnupg/gpg.conf
# FILE CREATED BY SEAHORSE
keyserver hkps://hkps.pool.sks-keyservers.net
# keyserver-options ca-cert-file=/home/main/.gnupg/sks-keyservers.netCA.pem http-proxy=socks5://<IP_ADDRESS>:9050

$ torsocks gpg --keyserver-options http-proxy= --search<EMAIL_ADDRESS>gpg: searching for<EMAIL_ADDRESS>from hkps server hkps.pool.sks-keyservers.net
gpgkeys: HTTP search error 60: server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
gpg: key<EMAIL_ADDRESS>not found on keyserver
gpg: keyserver internal error
gpg: keyserver search failed: keyserver error

It's probably just about having http-proxy= as default in gnupgKeyserverOptions,

Oh, it's cool that it's additive like that. That makes it easier.
I still have two objections:

Using socks5:// still does the DNS resolution locally, leaking the keyserver hostname and the fact that you're connecting to it. There appears to be a non-leaky version called socks5h://, but just the fact that this is not the default worries me. These things are easy to get wrong.
I don't see a way to specify the SOCKS user/password to use. This is necessary in order to force Tor to create unique circuits for each key refresh. I tried socks5g://user@pass:<IP_ADDRESS>:9050/ but that didn't work.

I think I wasn't clear in my last post. I'm convinced that using torsocket in parcimonie.sh is the best way to go. But parcimonie.sh should also run when GnuPG is configured to proxy traffic through TOR by setting http-proxy. Therefore parcimonie.sh should silently override http-proxy by an empty string as default. As shown other keyserver-options are not affected.
I also tested socks5g:// and socks5h://. Both are affected by DNS leaks.
Setting user/password is working, your uri was just wrong formated. It should be: socks5h://user:pass@<IP_ADDRESS>:9050/

Ah I see, makes more sense now. Thanks :)

If I understand this correctly, the
If you already use torify connections to keyservers with gpg's http-proxy keyserver-option in your gpg.conf, you will need to pass http-proxy= here to disable it.
part in the README is now obsolete?

Updated README in ea857e66720b91aa3c6eed2fc214d587149ca034.


'getText' method

Added ->getText() that returns the text content of nodes in a context.
Also short version as ->text() (no parameter).
--> What do you think of PHP_EOL as default? Maybe ',' better? Or '' ?

Coverage remained the same at 99.818% when pulling ec17a3e223e8675dec856508eabd848e468e801a on alepeino:gettext-method into b6fb8de6ff781479c9e4fb6e9f27b5af1e87dc4b on servo-php:master.

Hi dear @alepeino and thank you again for your appreciated PLs.
FluidXML is a library mainly intended for manipulating XML files.
As you can see from the API, it is made by methods used to mutate the document or to export it. Reading the internal state of the document is not a goal for the 1.x release but it will probably change in the 2.x release implementing a reading-specific API style.
Unfortunately implementing the ->text() getter method would break the coherence of the API mainly duplicating features already implemented in the native DOM API.
Even if your effort in this PL is very much appreciated, I can not accept it at least for the 1.x releases. It will be probably part of the 2th main release of FluidXML, but for now I can not merge it.
Thank you very much again for you effort.

Yes, I understand what you mean. In my current project I'm reading/parsing XMLs as much as I am writing/manipulating them. That's how I thought of these features I have sent, since I thought the reading-specific part could also benefit from a fluid, elegant and convenient API. I'll wait for v2, to which I will try to contribute as well.


Couldn't find any documentation about Qrel and run score range

Hi,
In the current api documentation I see some examples of how to structure qrel and run dicts, but there is no detail about what should be the range of qrel relevance score, I assume its 0 for non-relevant cases and 1 or more for graded relevance, but is there a upper limit to integer value?
Similarly, for run score which is a float type, I assume the range is 0.0 to 1.0?
Thank you.

Hi and thanks for your question.
qrels scores can be any integer, positive or negative.
All the implemented metrics but bpref only considers relevant documents, i.e. discard all qrels < 1. You can change this threshold by adding -ln (n is a parameter) to a metric name when using evaluate and compare methods (e.g., recall-l2 means that all qrels < 2 are considered as non-relevant when computing the recall score).
All the implemented metrics but DCG and nDCG consider relevance to be binary, i.e., graded scores are not taken into account.
run scores can be any float, positive or negative. They are used only for sorting as they are model-dependent.
Let me know if you have further questions.
Please consider giving ranx a star if you like it.

Thank you very much.


about multi-channel pic

dear researcher, what should I do if i want to input multi-channel-pic, should I change the model?

please see #2