[BUG] Heartbeat thread pool does not shutdown when MasterRegistry unRegistry

*For better global communication, please give priority to using English description, thx! *
Describe the bug
look at the method of MasterRegistry
    public void unRegistry() {
        String address = getLocalAddress();
        String localNodePath = getMasterPath();
        zookeeperRegistryCenter.getZookeeperCachedOperator().remove(localNodePath);
        logger.info("master node : {} unRegistry to ZK.", address);
    }

The method, which is invoke when close the MasterServer, does not shutdown the Heartbeat thread pool.

Hi,
Good job, please describe the impact of this bug according to the issue template[1], it will help us to make this bug trackable.
Thx a lot.
[1] issue template:
*For better global communication, please give priority to using English description, thx! *

**Describe the bug**
A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior, for example:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Which version of Dolphin Scheduler:**
 -[1.1.0-preview]

**Additional context**
Add any other context about the problem here.

**Requirement or improvement
- Please describe about your requirements or improvement suggestions.

Can you describe it in more detail?

Can you describe it in more detail?

Hi,
You can check the pr #3188 , and there is no executor shutdown when unregistry.


Dev tweaks

Coverage remained the same at 100.0% when pulling bac4621d79d52ac6af3870ca4b49b770b6d7e65c on dev-tweaks into 50f0fa0267c8df16679ab4a524d06b42f56e525c on master.

Coverage remained the same at 100.0% when pulling b3b27bca858122748066fe0dac3d9e8edcaa521e on dev-tweaks into 50f0fa0267c8df16679ab4a524d06b42f56e525c on master.


Python wrapper installation fails

Now it is impossible to install puccini from latest source code.
> pip install git+https://github.com/tliron/puccini#subdirectory=wrappers/python
Defaulting to user installation because normal site-packages is not writeable
Collecting git+https://github.com/tliron/puccini#subdirectory=wrappers/python
  Cloning https://github.com/tliron/puccini to /tmp/pip-req-build-zysluqa9
  Running command git clone --filter=blob:none --quiet https://github.com/tliron/puccini /tmp/pip-req-build-zysluqa9
  Resolved https://github.com/tliron/puccini to commit 8d605c2e3e902df69876543f069e0b4352a4546a
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-req-build-zysluqa9/wrappers/python/setup.py", line 10, in <module>
          with open(os.path.join(os.path.dirname(__file__), 'puccini', '__init__.py')) as f:
      FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-req-build-zysluqa9/wrappers/python/puccini/__init__.py'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

I think this was due to Puccini not having a release in a long time. :) I cut a release. Can you verify that it now works for you?

Sorry, I misread your issue report. You are installing from git, which is indeed broken. I will look into it.

After some consideration, this approach to installing a local build is no longer supported. The scripts/build-wrapper-python should be used. I've updated the documentation accordingly.


Broken zsh shell integration on 0.51+

Checklist

[X] I have read through the manual page (man fzf)
[X] I have searched through the existing issues
[X] For bug reports, I have checked if the bug is reproducible in the latest version of fzf

Output of fzf --version
0.53.0 (c4a9ccd)
OS

[X] Linux
[ ] macOS
[ ] Windows
[ ] Etc.

Shell

[ ] bash
[X] zsh
[ ] fish

Problem / Steps to reproduce
Shell integration appears to be broken on early 5.x.x releases of zsh, starting from from fzf version 0.51.0.
I am using a locked system where zsh version cannot be changed:
% zsh --version
zsh 5.0.2 (x86_64-redhat-linux-gnu)
% ./fzf --version
0.53.0 (c4a9ccd)
% source <(./fzf --zsh)
% vi **<TAB>
vi **__fzf_generic_path_completion:export:23: not an identifier: 40%

The last fzf version which supports this version of zsh is 0.50.0 (f97d275).

The last fzf version which supports this version of zsh is 0.50.0 (f97d275)

I believe the issue has been introduced through commit f864f8b5f7ab62e81fbf9eb2ac2333c7e52bb4e4.
I can confirm the problem occurs with any version up to 5.1, and the change was implemented in a commit^1, which introduced improved assignment handling in zsh for builtin commands such as export.
zsh-5.0.2 ❌
❯ ~/.local/zsh-test/bin/zsh-5.0.2 -fxvc 'export TEST=$(echo "--height 40% --reverse --scheme=path")'
+zsh:1> echo '--height 40% --reverse --scheme=path'
+zsh:1> export 'TEST=--height' 40% --reverse '--scheme=path'
zsh:export:1: not an identifier: 40%

zsh-5.1 ✅
❯ ~/.local/zsh-test/bin/zsh-5.1 -fxvc 'export TEST=$(echo "--height 40% --reverse --scheme=path")'
export TEST=$(echo "--height 40% --reverse --scheme=path")
+zsh:1> echo '--height 40% --reverse --scheme=path'
+zsh:1> export TEST='--height 40% --reverse --scheme=path'

Does splitting declaration and assignment solve the issue for you?
--- a/shell/completion.zsh
+++ b/shell/completion.zsh
@@ -158,5 +158,6 @@ __fzf_generic_path_completion() {
       [ "$dir" != "/" ] && dir="${dir/%\//}"
       matches=$(
-        export FZF_DEFAULT_OPTS=$(__fzf_defaults "--reverse --scheme=path" "${FZF_COMPLETION_OPTS-}")
+        export FZF_DEFAULT_OPTS
+        FZF_DEFAULT_OPTS=$(__fzf_defaults "--reverse --scheme=path" "${FZF_COMPLETION_OPTS-}")
         unset FZF_DEFAULT_COMMAND FZF_DEFAULT_OPTS_FILE
         if declare -f "$compgen" > /dev/null; then


Request to add SSL keys upload to SDK

Requesting functionality to add SSL certs to Alert Logic UI via SDK.  Would greatly simplify upload for users.

@james-hastings https://github.com/alertlogic/alertlogic-sdk-definitions/blob/master/alsdkdefs/apis/credentials/credentials.v2.yaml#L77

please re-open if request is still not satisfied.
cc @carlisom


Java 17 cannot start on AIX 7.2

Please provide a brief summary of the bug
Hello,
I'm trying to run java 17 on an AIX7.2TL05SP05 and I get the following error:
rs95:/tmp/jdk-17.0.7+7-jre/bin# oslevel -s
7200-05-05-2246
rs95:/tmp/jdk-17.0.7+7-jre/bin# ./java -version
Error: dl failure on line 542
Error: failed /tmp/jdk-17.0.7+7-jre/lib/server/libjvm.so, because 0509-022 Cannot load module /tmp/jdk-17.0.7+7-jre/lib/server/libjvm.so.
0509-150 Dependent module libc++.a(shr_64.o) could not be loaded.
0509-022 Unable to load module libc++.a(shr_64.o).
0509-026 System error: A file or directory in the path name does not exist.
0509-022 Unable to load module /tmp/jdk-17.0.7+7-jre/lib/server/libjvm.so.
0509-150 Dependent module /tmp/jdk-17.0.7+7-jre/lib/server/libjvm.so could not be loaded.
I have retrieved the following archive : https://github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.7%2B7/OpenJDK17U-jre_ppc64_aix_hotspot_17.0.7_7.tar.gz
A similar ticket has already been opened, but there was no resolution.
On my system, the xlc++ installed is version XLC 16.1.
Thank you in advance for your help.
Gaël
Please provide steps to reproduce where possible
No response
Expected Results
show java version
Actual Results
rs95:/tmp/jdk-17.0.7+7-jre/bin# ./java -version
Error: dl failure on line 542
Error: failed /tmp/jdk-17.0.7+7-jre/lib/server/libjvm.so, because       0509-022 Cannot load module
0509-150   Dependent module libc++.a(shr_64.o) could not be loaded.
0509-022 Cannot load module libc++.a(shr_64.o).
0509-026 System error: A file or directory in the path name does not exist.
0509-022 Cannot load module /tmp/jdk-17.0.7+7-jre/lib/server/libjvm.so.
0509-150   Dependent module /tmp/jdk-17.0.7+7-jre/lib/server/libjvm.so could not be loaded.
What Java Version are you using?
17.0.7+7
What is your operating system and platform?
AIX 7.2TL05SP05
How did you install Java?
I download from github
Did it work before?
No response
Did you test with the latest update version?
No response
Did you test with other Java versions?
No response
Relevant log output
No response

@gvernier Can you try the 17.0.8 release?   Should be out in a few days.

@karianna I don't think the latest version is available for OS AIX. There is a ppc64 version for Linux. Do you have this version for AIX?

See adoptium.net - the banner there has a link to our status page for getting the builds out this week including AIX.


Hello, there was an error when I was training with a single gpu

Hello, there was an error when I was training with a single gpu，RuntimeError: Default process group has not been initialized, please make sure to call init_process_group.

Hello, there was an error when I was training with a single gpu，RuntimeError: Default process group has not been initialized, please make sure to call init_process_group.

Hi, what command are you using? The example we provide in the README needs 8 GPUs, if you want to run with one GPU, make sure you have adjusted the learning rate, and then try:
bash tools/dist_train.sh configs/selfsup/mask_rcnn.py 1 --work-dir work_dirs/selfsup_mask-rcnn

For more details about how to use mmdet, please refer to their repo and tutorials.


How did you deal with the size different problem in the same batch?

For example, two images in different sizes of one batch, perhaps there is only one dimension equals(h,w). when i check the gt_masks in the same batch, i found they have different sizes, which means the original image size is different.

For example, two images in different sizes of one batch, perhaps there is only one dimension equals(h,w). when i check the gt_masks in the same batch, i found they have different sizes, which means the original image size is different.

I have solved the problem, it processes the images using mmcv.impad!


Roll engine 5ccee9537334..59715b71db8e (13 commits)

https://github.com/flutter/engine/compare/5ccee9537334...59715b71db8e
git log 5ccee95373348854bc4877cfe240024a36847d2d..59715b71db8e3d9d9c3ca47db6555a7a6789c180 --no-merges --oneline
59715b71d Disable build_ios task due to lack of credits. (flutter/engine#8150)
31b289f27 Fix indexing error in dart:ui TextStyle.toString (flutter/engine#8143)
fc2e6b61c Typo "fast an inline" to "fast and inline" (flutter/engine#8142)
0f19b2de0 Reland PerformanceOverlayLayer golden test (flutter/engine#8140)
073aadde1 Fix TextStyle decode misalignment (flutter/engine#8141)
d87d29054 Roll src/third_party/skia 406b068942f0..2eecc3ea3d71 (11 commits) (flutter/engine#8138)
5cef4a022 Use final state passed to dart before initialization as the initial lifecycleState. (flutter/engine#8124)
ffef51be6 Roll src/third_party/skia 665bc64a2dc4..406b068942f0 (8 commits) (flutter/engine#8137)
48efd0fb7 Roll src/third_party/skia 762ddd7e4352..665bc64a2dc4 (2 commits) (flutter/engine#8129)
f666adbbd Roll src/third_party/skia 2932a458957d..762ddd7e4352 (3 commits) (flutter/engine#8128)
8b0df6ded Bugfix #29203: NPE in getAccessibilityProvider in old FlutterView. (flutter/engine#8126)
8f7b18345 Roll src/third_party/skia c6d8781c4036..2932a458957d (2 commits) (flutter/engine#8125)
52b67fdd5 Expose decorationThickness to dart:ui (flutter/engine#8008)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff (jsimmons@google.com), and stop
the roller if necessary.

Trybots failed. These were the failed builds: https://cirrus-ci.com/task/6271816963194880

Commit queue failed; closing this roll.


Plz help

pytorch_lightning.utilities.exceptions.MisconfigurationException: You requested gpu: [0, 1, 2, 3]
But your machine only has: [0]
I am running the code on google colab. Is the issue related to the need of physical GPU?

https://github.com/PediaMedAI/ViTASD/blob/f96561962dbae63fc1e168c081946819e6262e9b/configs/config_vitasd_large.yaml#L26
Set here to 1 may help.

/usr/local/lib/python3.10/dist-packages/pytorch_lightning/callbacks/model_checkpoint.py:616: UserWarning: Checkpoint directory /content/ViTASD/lightning_logs/AffectNet-B exists and is not empty.
rank_zero_warn(f"Checkpoint directory {dirpath} exists and is not empty.")
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
pytorch_lightning.utilities.exceptions.MisconfigurationException: The provided lr scheduler CosineLRScheduler doesn't follow PyTorch's LRScheduler API. You should override the LightningModule.lr_scheduler_step hook with your own logic if you are using a custom LR scheduler
Thanks for your reply. I resolved the previous issue but got a new one as said above. plz tell solution to that.

I am still waiting for your reply brother.

It seems that it is a version error. Try to downgrade pytorch_lightning will solve it.


VS Code snippets do not work with Jupyter (.ipynb) files

I have started using VS Code with my Jupyter notebook files. However I have noticed that my existing Python snippets do not work within VS Code's implementation of Jupyter. Initially I thought it was because my snippets are specifically for .py files, but even after adding my snippet code to the global snippet settings JSON I still can't use them in any .ipynb files. They work with all other file types.
VSCode Version: 1.38.1
OS Version: Windows_NT x64 10.0.17763
Here is the code I am adding to my global.code-snippets json file.
{
	"Python common imports": {
		"prefix": "imports",
		"body": [
			"import pandas as pd",
			"import numpy as np",
			"import matplotlib.pyplot as plt",
			"import statsmodels.api as sm"
		],
		"description": "Imports common python modules"
	}
}

Hey @hannibal90 . Thanks for reporting this. Currently the Python interactive window and notebook editor work via a webview hosting their own editor. As such they don't work with the existing VS Code editor features and extensions (such as search or VIM keybinding extensions or snippits). We'd like to support this, but this might take some work from the VS Code side to support. We have an issue tracking general support for vs code editor features here (which would include snippits). Just follow that issue to see our progress on that.
#7875


systemd example error, keeps in activating status

I followed https://github.com/bachya/ecowitt2mqtt#running-in-the-background to set up ecowitt2mqtt as a service but whenever trying to start it it would time out:
root@server:~# systemctl start ecowitt2mqtt.service
Job for ecowitt2mqtt.service failed because a timeout was exceeded.
See "systemctl status ecowitt2mqtt.service" and "journalctl -xe" for details.

systemctl status would output:
root@server:~# systemctl status ecowitt2mqtt.service
● ecowitt2mqtt.service - ECOWITT2MQTT daemon
   Loaded: loaded (/lib/systemd/system/ecowitt2mqtt.service; enabled; vendor preset: enabled)
   Active: activating (start) since Fri 2022-06-17 11:29:23 CEST; 4s ago
 Main PID: 10868 (ecowitt2mqtt)
    Tasks: 5 (limit: 1026)
   CGroup: /system.slice/ecowitt2mqtt.service
           └─10868 /usr/bin/python3 /home/user1/.local/bin/ecowitt2mqtt -b <IP_ADDRESS> --hass-discovery --output-unit-system metric

Jun 17 11:29:23 homeslave systemd[1]: Starting ECOWITT2MQTT daemon...
Jun 17 11:29:26 homeslave ecowitt2mqtt[10868]: 2022-06-17 11:29:26,528 | ecowitt2mqtt | INFO | Starting ecowitt2mqtt

It would keep hanging in "activating" status even though the process was actually running and working.
After some research I found that changing this setting in the service file would fix this:
[Service]
Type=simple

So now it is working fine. Suggest to change the setting in the readme.

Thanks!


improve grammar

Contributions

[X] I have read the contribution docs, and understand what's expected of me.

Link to the page on docs.getdbt.com requiring updates
https://docs.getdbt.com/guides/best-practices/environment-setup/2-one-deployment-environment
What part(s) of the page would you like to see updated?
multiple developers can easily contributing
Additional information
multiple developers can be easily contributing
OR
multiple developers can easily contribute

Thanks for opening this @MaratAbdullaev! You are welcome to address this in a PR, otherwise, I've triaged it for the docs team.

Hey @MaratAbdullaev thanks for flagging this! I've opened a pr to address and clarify 🙏


Pathfinding

Works as well and as fast as I think it needs to be, in sim. Still needs testing on RIO, but I've added both single-threaded and multithreaded implementations so we can choose depending on its performance there. The API for creating environments may need a little more cleanup but it's usable as it is.

Tested on RoboRIO at State Fair, works! Average 6 ms calc time for full-field path. Probably don't need multithreading, but I'll leave the code there just in case.


Able to add fields to closed structs

Exporting the following:
Foo: {
	listOfCloseds: [...Closed]
}

Closed :: {
	a: int | *0
}

Junk: {
	b: 2
}

Foo & {
	listOfCloseds: [{
		for k, v in Junk {
			"\(k)": v
		}
	 }]
}

outputs:
{
    "listOfCloseds": [
        {
            "a": 0,
            "junk": 2
        }
    ]
}

which seems incorrect, since we shouldn't have been able to add an extra field to a struct unified with Closed.

@mpvl it seems like this is a bug with the struct comprehension as the following works.
Foo & {
	listOfCloseds: [Junk]
}
// listOfCloseds.0: field "b" not allowed in closed struct:

Do you have a idea where their is a good place to start looking ?

@xinau at the evaluator in the cue subdirectory. Most likely binop.go.
I usually copy a snippet in to TestX in resolve_test.go and start debugging from there.

@mpvl. Thank you for the information. will do so next time.

This issue has been migrated to https://github.com/cue-lang/cue/issues/153.
For more details about CUE's migration to a new home, please see https://github.com/cue-lang/cue/issues/1078.


Created a double buffer (with write/read indices) for each serial comm

Added to double buffers and indices to bm_serial context

Merging fix for thread-safe DMA reads


Spanish localisation isn't working

I've built the recent version of the project and I've seen that the Spanish localisation is not working (and I think Russian isn't working too).
I think I was because I forgot to add the translates files to the Xcode project, and I don't know where to add them

I can't find any Spanish localisations in the project, so you would be correct. Can you double-check if  the Russian isn't working?

@markvl91 Russian is working.
The only problem is with Spanish translations. I only added the files but not to the project (duh...)


Fix FileSystemWatcher_RazorFileEvent_Background_AddRemoveDoesNotNotif…

…y flaky test.

Turns out XUnit would occasionally construct a TestProjectSnapshotManagerDispatcher on test threads (there was no guarantee which thread was the constructor) and because of that the test variant of the dispatcher would bind to that thread and use it to attempt to re-dispatch notifications. Well our RazorFileEvent test's final step would be to re-dispatch file notifications via the dispatcher while simultaneously waiting for the notification to be sent on the test side. This resulted in a deadlock killing the test.
Updated the LanguageServerTestBase to rename the existing Dispatcher to LegacyDispatcher and add a new Dispatcher that utilizes the real LSP dispatcher. I took this approach because there's loads of locations that utilize the pre-existing LegacyDispatcher that will need to be iteratively moved to the real dispatcher type to ensure things work as expected. Issue tracking migrating tests to the "real" dispatcher: https://github.com/dotnet/razor-tooling/issues/5683

Fixes #4401

lgtm! I wonder if the issue you filed is also related to #4393.

Ahhhh most definitely is!


build: Update to go version 1.21.8

What this PR does
go version 1.21.8 includes a number of security fixes[1]:

CVE-2024-24783 - https://access.redhat.com/security/cve/CVE-2024-24783
CVE-2023-45290 - https://access.redhat.com/security/cve/CVE-2023-45290
CVE-2023-45289 - https://access.redhat.com/security/cve/CVE-2023-45289

[1] https://groups.google.com/g/golang-announce/c/smSYdsWaO4o

Fixes #
Why we need it and why it was done in this way
The following tradeoffs were made:
The following alternatives were considered:
Links to places where the discussion took place:
Special notes for your reviewer
/cc @dhiller @xpivarc

Checklist
This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR.
Approvers are expected to review this list.

[ ] Design: A design document was considered and is present (link) or not required
[ ] PR: The PR description is expressive enough and will help future contributors
[ ] Code: Write code that humans can understand and Keep it simple
[ ] Refactor: You have left the code cleaner than you found it (Boy Scout Rule)
[ ] Upgrade: Impact of this change on upgrade flows was considered and addressed if required
[ ] Testing: New code requires new unit tests. New features and bug fixes require at least on e2e test
[ ] Documentation: A user-guide update was considered and is present (link) or not required. You want a user-guide update if it's a user facing feature / API change.
[ ] Community: Announcement to kubevirt-dev was considered

Release note

Build KubeVirt with Go version 1.21.8

@brianmcarey any planes for 1.0-2?

@brianmcarey any planes for 1.0-2?

I think it should be straight forward to backport to 1.2 but 1.0 & 1.1 are still on 1.19 so there will be more involved there.

Do we have a policy to bump minor versions of stable releases?

/cherrypick release-1.2


Cornelius wiggles towards food

Description
In the Kyoshi novels, they say the Earthbenders find the Avatar by dividing the earth kingdom in 2, and divining which direction the Avatar is in. Moving in smaller and smaller areas. They don't find Kyoshi though, because she moves, like our food does. Let's evaluate how safe or hazardous an area is, and drive Cornelius that direction!

we could devide the map up into sectors, then we need to quantify the dangers and benefits of a sector IE If a sector has lots of tail it looses points, aswell as having danger tiles it looses points, but if it has food, it also gains points, so corny knows to head in that general direction, in their little wiggly way.
What would need to be thought about: How would we determine sectors programatically, for larger boards putting 5x5 sectors could work well, but for the 7x7 board, that wouldnt make much sense

we could divide the map up into sectors, then we need to quantify the dangers and benefits of a sector IE If a sector has lots of tail it looses points, as well as having danger tiles it looses points, but if it has food, it also gains points, so corny knows to head in that general direction, in their little wiggly way.
What would need to be thought about: How would we determine sectors programmatically, for larger boards putting 5x5 sectors could work well, but for the 7x7 board, that wouldn't make much sense

(3 * Food - Hazards)/squares in space? I just made that function up, but it feels right. Also, I wouldn't divide the board by fixed sectors, I'd divide it based on the location of Cornelius's head.

Also how are you passing this? I wrote it at 10pm because I couldn't sleep!

Also how are you passing this? I wrote it in the middle of the night because I couldn't sleep!

how do you mean passing?

(3 * Food - Hazards)/squares in space?

I think
safty = foodCount - hazards
  where
    hazards = snakeSegmentCount + hazardWallCount

I dont think food needs to be that prioritized, I think avoiding danger could be more important. Maybe 2 * food at most. This is just speculation though, I think we will need to ask Corny to try a few strategies out for us


[Refac] Object handler for INTF (serialize)

Prerequisite for #5658
The coding should first, fill the ABAP type and then pass the type to XML output handling. This facilitates alternative output handling, such as for JSON.

Ready to merge?


On toggle it doesn't cancel previous animation

Clicking on the menu too fast can lead to buttons being visible even when collapsed.
Doesn't seem to properly handle cancellation of previous animations.

Same as #70.


need more then 100 tweet result

i tried tweet search but according to twitter api count limit is 100 per page
client.get('search/tweets', {q: 'node.js'}, function(error, tweets, response){
  console.log(tweet.length)
});

output
100
is there any way to can count total tweet by a username with a hash in specific day
eg:
#ThingsiJustLove from:@xyz_user on 26-apr-2016
i want total number of tweets posted by this user with this hashtag in specific date
if anyone have any idea please share

@isoftmaster unfortunately, this is a limitation of the API.

The number of tweets to return per page, up to a maximum of 100

See: https://dev.twitter.com/rest/reference/get/search/tweets#api-param-search_count
You will have to do your own paging here to get all of the results.  The until, since_id and max_id filters should be helpful.


System cannot find command btrview when running as sudo

I'm having trouble getting started.
My OS is Fedora 39 (KDE spin), and I installed using pip install btrview.  When I try running sudo btrview, I get the error sudo: btrview: command not found. However, when I run just btrview, I get something that suggests that I do in fact have this command on my computer.
$ btrview
WARNING: You're current running this script as user erin.
If you have problems, try rerunning this script with sudo, or as the root user.
Traceback (most recent call last):
  File "/home/erin/.local/bin/btrview", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/erin/.local/lib/python3.12/site-packages/btrview/scripts/btrview.py", line 58, in main
    output = logic(args.labels, args.exclude, args.property,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/erin/.local/lib/python3.12/site-packages/btrview/rich_output.py", line 140, in logic
    subvols = fs.subvolumes(remove)
              ^^^^^^^^^^^^^^^^^^^^^
  File "/home/erin/.local/lib/python3.12/site-packages/btrview/btrfs.py", line 120, in subvolumes
    subvols = [] if "root" in remove else [Subvolume.from_ID("5",mount_point, self.mounts)]
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/erin/.local/lib/python3.12/site-packages/btrview/subvolume.py", line 104, in from_ID
    props = cls._run_cmd(cmd)
            ^^^^^^^^^^^^^^^^^
  File "/home/erin/.local/lib/python3.12/site-packages/btrview/subvolume.py", line 122, in _run_cmd
    out = run(cmd)
          ^^^^^^^^
  File "/home/erin/.local/lib/python3.12/site-packages/btrview/utils.py", line 23, in run
    raise e
  File "/home/erin/.local/lib/python3.12/site-packages/btrview/utils.py", line 20, in run
    out = subprocess.run(tokens, check=True, capture_output=True,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['btrfs', 'subvolume', 'show', '-r', '5', '/home']' returned non-zero exit status 1.

Was there an installation step that I missed?

I'm guessing this is an issue because the $PATH variable (basically a list of which programs you can execute) is different between running as a regular user and running with sudo on fedora. There's some trouble shooting steps listed in this stackoverflow thread.
The top answer of that thread is probably the easiest sudo -E env "PATH=$PATH" <command> [arguments] for a quick fix, but something more permanent would be to edit the secure_path in your sudoers file. Unfortunately I'm not running a machine with fedora on it at the moment so I can't test these solutions myself.

I've updated the README with some new installation instructions, check them out, I think it should fix your issues!


subprocess.CalledProcessError: Command '('krb5-config --cflags krb5',)' returned non-zero exit status 1.

`Collecting krb5
Using cached krb5-0.5.1.tar.gz (221 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [23 lines of output]
'"krb5-config --cflags krb5"' is not recognized as an internal or external command,
operable program or batch file.
Using krb5-config at 'krb5-config'
Traceback (most recent call last):
File "D:\AI\private-GPT\origin-private-GPT\privateGPT-main\CritixScrapy\venv\lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 353, in
main()
File "D:\AI\private-GPT\origin-private-GPT\privateGPT-main\CritixScrapy\venv\lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "D:\AI\private-GPT\origin-private-GPT\privateGPT-main\CritixScrapy\venv\lib\site-packages\pip_vendor\pyproject_hooks_in_process_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "C:\Users\TAOL.klotzsh\AppData\Local\Temp\pip-build-env-3669wbo7\overlay\Lib\site-packages\setuptools\build_meta.py", line 355, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "C:\Users\TAOL.klotzsh\AppData\Local\Temp\pip-build-env-3669wbo7\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in _get_build_requires
self.run_setup()
File "C:\Users\TAOL.klotzsh\AppData\Local\Temp\pip-build-env-3669wbo7\overlay\Lib\site-packages\setuptools\build_meta.py", line 341, in run_setup
exec(code, locals())
File "", line 175, in
File "", line 33, in run_command
File "C:\Users\TAOL.klotzsh\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "C:\Users\TAOL.klotzsh\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '('krb5-config --cflags krb5',)' returned non-zero exit status 1.
[end of output]
`
and I have try run  krb5-config --cflags krb5, it seems like not cofig a envrioment variable
(venv) D:\CritixScrapy>krb5-config --cflags krb5 'krb5-config' is not recognized as an internal or external command, operable program or batch file.
what should I do?

You need to ensure the package the ships krb5-config has been installed. It is used to get the correct flags needed to compile against the Kerberos libs.

sorry I don't have krb5 background knowledge , I try pip install krb5-config, its not work

sorry I don't have krb5 background knowledge , I try pip install krb5-config, its not work

This package relies on the krb5 C library to be present on the system as it compiles against the libkrb5.so. To do this it requires some development tools like the krb5-config command and other headers. How you get these libraries depends on the Linux distribution you use, for example Debian based distros need to do apt-get install libkrb5-dev, RHEL based distros need to do dnf install krb5-devel and so on. There are also some build requirements like gcc and so on. These requirement cannot be fulfilled by pip as they are system package requirements and not Python requirement.


RadialGuageChart - Width and Height

I am using Microcharts.Forms, and I would like to know how to set the size of the chart correctly. For now I am adding a few empty and transparent Entry (ChartEntry) to the chart, so it grows. How to make a bigger RadialGuageChart using only some height and width parameter?
Thanks!

This is an issue for me too and i would like to know the answer please

How do you do that?

@PanagiotisSpyrakis and @biapar
My solution (if I can call it that way) was to put my Graph under and Grid Row with Star (*) sizing.
<microcharts:ChartView                                                     Chart="{Binding Grafico}"                                                    Margin="0"                                                    WidthRequest="170"                                                    HeightRequest="170"                                                    HorizontalOptions="FillAndExpand"                                                    VerticalOptions="FillAndExpand"                                                    BackgroundColor="#2c517e"/>
Setting Height and WidthRequest, FillAndExpand for Horizontal and Vertical.

What platforms is this an issue on?

iOS
Στις Σάβ, 27 Ιουλ 2019, 05:40 ο χρήστης Joseph Nixon <
<EMAIL_ADDRESS>έγραψε:

What platforms is this an issue on?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dotnet-ad/Microcharts/issues/139?email_source=notifications&email_token=AKKU5M3NIS3OK2OVNCO6KSLQBOYRHA5CNFSM4G4PQANKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD26B7QY#issuecomment-515645379,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKKU5M6M6YDISDUJYPX6ATLQBOYRHANCNFSM4G4PQANA
.


question about channel pendingAck implementation.

I see that pluggable component go-sdk transforms streaming into unary mode. In a question-and-answer implementation, there is a paragraph in github.com/dapr-sandbox/components-go-sdk/bindings/v1/handle.go code snippet about pendingAck:
 84                 select {
 85                 case resp := <-pendingAck:
 86                         return resp.data, resp.err
 87                 case <-ctx.Done():
 88                         return nil, ErrAckTimeout
 89                 }

If the channel pendingAck receives data timeout (default: 1s), the ACK API will directly return nil, but after executing the Send API, waiting for pendingAck will hang in/out, and the select <-ctx.Done here is to wait for the upstream to actively complete the request. and returns ErrAckTimeout. In fact, the ErrAckTimeout returned here is not suitable, it should be the upstream end of the request.
I think the correct approach should be: AcknowledgementManager.Ack API should also take ackTimeoutFlagFunc API as a return parameter, and then uniformly handle timeout, is it more appropriate? similar:
                msgID, pendingAck, cleanup, ackTimeoutFlagFunc := ackManager.Get()
                ......
                select {
                case resp := <-pendingAck:
                        return resp.data, resp.err
               case <-ackTimeoutFlagFunc(): // return Ack timeout flag.
                       return nil, ErrAckTimeout
                case <-ctx.Done():
                        return nil, nil
                }

include pubsub/bindings/state...
My main point is that when the Ack API has a processing timeout (default: 1s), the pendingAck returned by the Get API also needs to know this timeout, and then return directly without blocking waiting and relying on upstream active processing. Or should there be absolutely no packet loss?

hey @1046102779 sorry for taking too long for responding back,

If the channel pendingAck receives data timeout (default: 1s), the ACK API will directly return nil, but after executing the Send API, waiting for pendingAck will hang in/out, and the select <-ctx.Done here is to wait for the upstream to actively complete the request. and returns ErrAckTimeout. In fact, the ErrAckTimeout returned here is not suitable, it should be the upstream end of the request.

Please, notice that ctx.Done() is not for the entire upstream request (the input binding stream/pubsub stream), the context in this case is provided by the underlying component, so as a good practice it should be tied to the message consumption process (i.e the timeout to consume the message, redis for instance uses the metadata field processingTimeout)

My main point is that when the Ack API has a processing timeout (default: 1s), the pendingAck returned by the Get API also needs to know this timeout, and then return directly without blocking waiting and relying on upstream active processing. Or should there be absolutely no packet loss?

The default ack timeout of 1s is used when the message was actually ack'ed but there's no consumer waiting for the ack, so it would probably lead in two scenarios:

Message was already ack'ed (because the ack channel has size 1), hence it is duplicated so could be discarded without any problem
Ack was timed out (now the default 1s) because the component couldn't wait enough for the message to be ack'ed

by using this
                msgID, pendingAck, cleanup, ackTimeoutFlagFunc := ackManager.Get()
                ......
                select {
                case resp := <-pendingAck:
                        return resp.data, resp.err
               case <-ackTimeoutFlagFunc(): // return Ack timeout flag.
                       return nil, ErrAckTimeout
                case <-ctx.Done():
                        return nil, nil
                }

we are also saying that the underlying component will consider the message as successfully delivered as the ctx.Done() was reached but the ack was not completed, so it leads in having messages being not at-least-once delivered, or not?
Please, see if it makes sense to you @1046102779

hey @1046102779 sorry for taking too long for responding back,

If the channel pendingAck receives data timeout (default: 1s), the ACK API will directly return nil, but after executing the Send API, waiting for pendingAck will hang in/out, and the select <-ctx.Done here is to wait for the upstream to actively complete the request. and returns ErrAckTimeout. In fact, the ErrAckTimeout returned here is not suitable, it should be the upstream end of the request.

Please, notice that ctx.Done() is not for the entire upstream request (the input binding stream/pubsub stream), the context in this case is provided by the underlying component, so as a good practice it should be tied to the message consumption process (i.e the timeout to consume the message, redis for instance uses the metadata field processingTimeout)

My main point is that when the Ack API has a processing timeout (default: 1s), the pendingAck returned by the Get API also needs to know this timeout, and then return directly without blocking waiting and relying on upstream active processing. Or should there be absolutely no packet loss?

The default ack timeout of 1s is used when the message was actually ack'ed but there's no consumer waiting for the ack, so it would probably lead in two scenarios:

Message was already ack'ed (because the ack channel has size 1), hence it is duplicated so could be discarded without any problem
Ack has timed out (now the default 1s) because the component couldn't wait enough for the message to be ack'ed

by using this
                msgID, pendingAck, cleanup, ackTimeoutFlagFunc := ackManager.Get()
                ......
                select {
                case resp := <-pendingAck:
                        return resp.data, resp.err
               case <-ackTimeoutFlagFunc(): // return Ack timeout flag.
                       return nil, ErrAckTimeout
                case <-ctx.Done():
                        return nil, nil
                }

we are also saying that the underlying component will consider the message as successfully delivered as the ctx.Done() was reached but there's no evidence that the message was processed, so it leads in having messages being not at-least-once delivered, or not?
Please, see if it makes sense to you @1046102779

I'll take a closer look at the entire process. Thank you for your detailed reply.


Feature: create a function in client which can send logs to db without making a network call

We need to create one more function in the client like client.NewClientWithServer - which can take the logs and instead of doing a network call and all, directly start the routine to save logs to the DB (the server side routine) and bypass the network call and rest API and send the logs directly to the channel from which it would be sent to the DB.

#81 closed this one.


Borgs are missing their flashlight after a recent update

Description
I'm not sure when this started, but I noticed a few days ago that Cyborgs no longer spawn with flashlights equipped, and as a result have no way of producing light. This makes them useless in maints, and helpless when power goes out.
Reproduction

Spawn as a Cyborg
Realize you are missing flashlight
Cry

Screenshots

most likely trolled by item toggle rework

I cannot Replecate The Bug maybe This was mostlikley fixed elsewehre

It seems like this was fixed briefly but is once again an issue.

This is resolved now.


Allow web resources to run as a seperate process

So that the main logic parts of the conference bot could be restarted without losing things like streams. Currently there is some shared state (the web resources aren't just static pages) and there needs to be some thought about how to safely split it without having stale data issues.
An attempt has begun in #184

Done in #184


ERROR Unexpected exception: ...failed with exit code 127

We are trying to train mmt on a AWS instance, Sagemaker ml.p3.2xlarge with 61 GiB memory, and recieved an "ERROR Unexpected exception" when the Cleaning Corpora ran the Creating aligner filter. What could be wrong?
We have tried using both the included example data and importing a WikiMatrix dataset, both with same error.
See progress and Error code:
=========== TRAINING STARTED ===========

ENGINE:   default
LANGUAGE: ar > sv

(1/4) Cleaning corpora
    (1/6) Cleaning corpora...                                    DONE in 16s
    (2/6) De-duplicate corpora...                                DONE in 1s
    (3/6) Preprocess corpora...                                  DONE in 4s
    (4/6) Creating aligner filter...
ERROR Unexpected exception: Command '/home/ec2-user/SageMaker/mmt/build/bin/fa_build -s ar -t sv -i /home/ec2-user/SageMaker/mmt/runtime/default/tmp/training/_temp_cleaning/preprocessed_corpora -m /home/ec2-user/SageMaker/mmt/runtime/default/tmp/training/_temp_cleaning/aligner/ar__sv.fam -I 4 --case-insensitive --no-favor-diagonal' failed with exit code 127

Hi @AdlerKosma ,
most likely the problem could be related to the process requesting more RAM than what is installed.
Can you please also paste here the content of the training log file at runtime/<engine-name>/logs/training.log?

Thanks for your reply @davidecaroselli. I added information about the instance in the issue. RAM should be the problem.
Here is the content of training.log:
2021-06-21 09:24:58,237 [INFO] - Training started: engine=default, lang_pair=ar__sv
2021-06-21 09:24:58,263 [INFO] - Training step "clean" started
2021-06-21 09:24:58,264 [INFO] - Training step "clean" started
2021-06-21 09:24:58,264 [DEBUG] - java -DentityExpansionLimit=0 -DtotalEntitySizeLimit=0 -Djdk.xml.totalEntitySizeLimit=0 -Xmx55260m -cp /home/ec2-user/SageMaker/mmt/build/mmt-4.9.5.jar -Dmmt.home=/home/ec2-user/SageMaker/mmt -Djava.library.path=/home/ec2-user/SageMaker/mmt/build/lib eu.modernmt.cli.CleaningPipelineMain -s ar -t sv --input examples/data/train/ --output /home/ec2-user/SageMaker/mmt/runtime/default/tmp/training/_temp_cleaning/clean_corpora --output-format parallel
2021-06-21 09:25:15,032 [INFO] - Training step "clean" completed in 16s
2021-06-21 09:25:15,033 [INFO] - Training step "dedup" started
2021-06-21 09:25:15,033 [DEBUG] - java -cp /home/ec2-user/SageMaker/mmt/build/mmt-4.9.5.jar -Dmmt.home=/home/ec2-user/SageMaker/mmt -Djava.library.path=/home/ec2-user/SageMaker/mmt/build/lib eu.modernmt.cli.DeduplicationMain -s ar -t sv --input /home/ec2-user/SageMaker/mmt/runtime/default/tmp/training/_temp_cleaning/clean_corpora --output /home/ec2-user/SageMaker/mmt/runtime/default/tmp/training/_temp_cleaning/dedup_corpora
2021-06-21 09:25:16,465 [INFO] - Training step "dedup" completed in 1s
2021-06-21 09:25:16,466 [INFO] - Training step "preprocess" started
2021-06-21 09:25:16,466 [DEBUG] - java -cp /home/ec2-user/SageMaker/mmt/build/mmt-4.9.5.jar -Dmmt.home=/home/ec2-user/SageMaker/mmt -Djava.library.path=/home/ec2-user/SageMaker/mmt/build/lib eu.modernmt.cli.TrainingPipelineMain -s ar -t sv --output /home/ec2-user/SageMaker/mmt/runtime/default/tmp/training/_temp_cleaning/preprocessed_corpora --input /home/ec2-user/SageMaker/mmt/runtime/default/tmp/training/_temp_cleaning/dedup_corpora
2021-06-21 09:25:20,590 [INFO] - Training step "preprocess" completed in 4s
2021-06-21 09:25:20,591 [INFO] - Training step "make_filter" started
2021-06-21 09:25:20,591 [DEBUG] - /home/ec2-user/SageMaker/mmt/build/bin/fa_build -s ar -t sv -i /home/ec2-user/SageMaker/mmt/runtime/default/tmp/training/_temp_cleaning/preprocessed_corpora -m /home/ec2-user/SageMaker/mmt/runtime/default/tmp/training/_temp_cleaning/aligner/ar__sv.fam -I 4 --case-insensitive --no-favor-diagonal
/home/ec2-user/SageMaker/mmt/build/bin/fa_build: error while loading shared libraries: libicui18n.so.60: cannot open shared object file: No such file or directory

Actually, the problem is related to a missing library: libicui18n. We have never ever encountered this error, are you running the software on a standard Ubuntu 18 installation?
You can also manually install it with the command: sudo apt-get install libicu-dev


pr template

I personally don't like this kind of template, but if folks on the team find them useful then go for it.
Can you just remove me from the reviewers?


0.8.x Compatibility

The module doesn't seem to be working properly on 0.8.x. Any plans to update?

Yep. Working on it right now.

Ok, I've uploaded a new package - let me know if it installs for you. If it does, then feel free to close the issue.

It does install, and works perfectly! Thank you!


[SPARK-47102][SQL][COLLATION] Adding COLLATION_ENABLED config

What changes were proposed in this pull request?
This PR adds COLLATION_ENABLED config to SQLConf and introduces new error class COLLATION_SUPPORT_NOT_ENABLED to appropriately report error on usage of feature under development.
Why are the changes needed?
We want to make collations configurable on this flag. These changes disable usage of collate and collation functions, along with any COLLATE syntax when the flag is set to false. By default, the flag is set to false.
Does this PR introduce any user-facing change?
Yes. It introduces new error along with an appropriate message.
How was this patch tested?
./build/mvn -Dtest=none -DwildcardSuites=org.apache.spark.sql.errors.QueryCompilationErrorsSuite test
./build/mvn -Dtest=none -DwildcardSuites=org.apache.spark.sql.catalyst.expressions.CollationExpressionSuite test
./build/mvn -Dtest=none -DwildcardSuites=org.apache.spark.sql.CollationSuite test

Was this patch authored or co-authored using generative AI tooling?
No.

One more thing: it would be nice to add the [Collation] tag in the title of the PR.

PR should be ready now, but until I get GitHub Actions fixed for my account, CI runs can't be run before merging. Will ping when that gets fixed.


Plugin for circular navigation

I think that would be interesting to have circular navigation in this plugin. So, this request only contains the code to build it and i'll create a new pull request with documentation for this plugin.
The Circular Navigator plugin, have some limitations that i'll improve later. These limitations consisting in to connect with 'Bullet Navigator' and 'ResponsiveHub Connector', which will be explained in the documentation.

I'm closing this pull request because i've created an updated version with #10, #8 and #5 improves. It will provide a better tree organization.


create_collation bug with multiple connections (iOS)

When using multiple connections I have to call raw.sqlite3_create_collation for every connection to register the collation functions. Doing so I get random NullPointer exceptions from SQLite3Provider.collation_hook_bridge_impl at a later point.
I think I narrowed the problem down to the following (correct me if I'm wrong):
The SQLite3Provider class is implemented as a singleton (static member _imp of class raw) which contains a Dictionary _collation_hooks. Now every time create_collation is called (regardless of the connection or db object) the old collation hook of the _collation_hooks dictionary is released and a new one is created.
So when first connection1 and then connection2 are created and the collations are registered in this order, connection1 contains a collation hook which was released and is therefore unusable.
Another thought: As you do not lock the _collation_hooks dictionary in a critical section, have you tested opening multiple connections simultaneously from different threads? This might produce race conditions...

I only briefly glanced at the code but a possible solution to this would be to use a ConditionalWeakTable to map sqlite3 instances to per connection dictionaries. Regarding the concurrency question, it would probably be easier to define individual sqlite3 instances as not being threadsafe (this is what I do in pretty as well), while the ConditionalWeakTable would provide the necessary thread safety guarantees needed for creating new connections from different threads.

I added the following Expando object to SQLitePCL.pretty to solve some similar problems. It's loosely based on Stephen Cleary's connected properties library. The main difference is that the per object property map is not threadsafe. I can look into patching this issue this week to utilize a similar Expando object in SQLitePCL.raw, but wanted to get your impressions first before jumping in to code a fix.

(1)  I'm pretty sure I understand the problem.
(2)  I don't yet have a proposed solution of my own.  I would need to investigate more deeply.
(3)  I understand bordoley's proposed solution, but only at a shallow level, so I have enough confidence to say "Yes, I am interested in this", but not enough to say "Yes!  Do it!"
(4)  I assume that the same problem exists in the several other places I'm using a Dictionary to keep track of hooks.  :-(

Woah, that was a quick reply ;)
While I think bordoley's solution would be good, the ConditionalWeakTable class is only applicable for reference types, so it does not work for the IntPtrs you're using inside SQLite3Provider (and changing everything to the sqlite3 class would require a lot of changes). Another solution I propose:
Instead of using
Dictionary<string, collation_hook_info> _collation_hooks

you could use a
ConcurrentDictionary<IntPtr, Dictionary<string, collation_hook_info>> _collation_hooks

(which is threadsafe) and then clean up inside the sqlite3_close and sqlite3_close_v2 methods:
int ISQLite3Provider.sqlite3_create_collation(IntPtr db, string name, object v, delegate_collation func)
{
    var collation_hooks = _collation_hooks.GetOrAdd(db, p => new Dictionary<string, collation_hook_info>());
    //the rest of the method stays the same, except for using "collation_hooks" instead of "_collation_hooks"
}

int ISQLite3Provider.sqlite3_close(IntPtr db)
{
    Dictionary<string, collation_hook_info> collation_hooks;
    if (_collation_hooks.TryRemove(db, out collation_hooks))
    {
        //free everything inside collation_hooks
    }
    return NativeMethods.sqlite3_close(db);
}

And of course this would have to be done for collation hooks, function hooks and the commit/update hooks.
However I still think it would be worth the effort as this bug makes the SQLite.raw framework pretty much useless on iOS when you need custom collations/functions/...

The problem with the ConcurrentDictionary solution is that its not available in the PCL profile, and specifically is missing outright from Windows Phone 8, which I believe uses the PInvoke implementation. You can mimic it with locks but that comes with all the associated problems ConcurrentDictionary is used to avoid. You make an excellent point about IntPtr being a struct and therefore not compatible with ConditionalWeakTable . Hmm...

Hmm, I missed that. How about just using a regular Dictionary with lock(){}? As the locking would only be required when creating collations / closing the db connection the potential performance overhead could probably be ignored...

I'm going experiment with a fix based on the ideas above:
(1) One dictionary per connection
(2) lock(){}
I'll let you know how it goes.
--
E
On Sun, Mar 15, 2015 at 10:15 AM, dominik-weber<EMAIL_ADDRESS>wrote:

Hmm, I missed that. How about just using a regular Dictionary with
lock(){}? As the locking would only be required when creating collations /
closing the db connection the potential performance overhead could probably
be ignored...
—
Reply to this email directly or view it on GitHub
https://github.com/ericsink/SQLitePCL.raw/issues/42#issuecomment-81083273
.

You might want to consider defining a custom IConcurrentDictionary interface, and wrapping ConcurrentDictionary on platforms that support it and using locks on those that don't via ifdefs. The connected properties library I linked to above includes such an implementation that you could copy verbatim.

Eric I don't know exactly what you mean with "One dictionary per connection", but it would probably be much cleaner code if you used just one static dictionary and a helper class which contains all the connection-specific properties:
//used to associate connections (IntPtr) with ConnectionHooks objects
//access must be lock-ed or ConcurrentDictionary used on supporting platforms (like bordoley said)
static Dictionary<IntPtr, ConnectionHooks> _connection_hooks;

class ConnectionHooks
{
    object /*or whatever*/ commit_hook;
    //...
    Dictionary<string, object> collation_hooks;
    //...

    public void Dispose() //call this method in sqlite3_close and sqlite3_close_v2
    {
        //free all hooks
    }
}

All I meant was, in deliberately vague terms, what you were proposing.
My focus at first will be to make sure I can reproduce the behavior you are
seeing and then find a solution that fixes the problem.
--
E
On Mon, Mar 16, 2015 at 1:16 PM, dominik-weber<EMAIL_ADDRESS>wrote:

Eric I don't know exactly what you mean with "One dictionary per
connection", but it would probably be much cleaner code if you used just
one static dictionary and a helper class which contains all the
connection-specific properties:
//used to associate connections (IntPtr) with ConnectionHooks objects
//access must be lock-ed or ConcurrentDictionary used on supporting platforms (like bordoley said)
static Dictionary<IntPtr, ConnectionHooks> _connection_hooks;
class ConnectionHooks
{
object /or whatever/ commit_hook;
//...
Dictionary<string, object> collation_hooks;
//...
public void Dispose() //call this method in sqlite3_close and sqlite3_close_v2
{
    //free all hooks
}

}
—
Reply to this email directly or view it on GitHub
https://github.com/ericsink/SQLitePCL.raw/issues/42#issuecomment-81850505
.

Just to clarify something about this remark from @dominik-weber
"as this bug makes the SQLite.raw framework pretty much useless on iOS when you need custom collations"
As far as we know, the following two statements are true, right?
(1)  The bug is not specific to iOS.
(2)  The bug doesn't happen when only one SQLite connection is in play.
Thanks.

OK, maybe I actually don't understand this bug.  I have added test_collation_2() and test_collation_3() to src/cs/test_cases.cs, and neither of them reproduces the problem.  @dominik-weber can you look at the code for these test cases and see if you can spot why they do not bring about the "random NullPointer exceptions from SQLite3Provider.collation_hook_bridge_impl" that you are seeing?  Thanks.

I haven't tested this but one case I can think of is adding two different
collation sequences to two different db connections with the same name.
On Mar 16, 2015 1:28 PM, "Eric Sink"<EMAIL_ADDRESS>wrote:

OK, maybe I actually don't understand this bug. I have added
test_collation_2() and test_collation_3() to src/cs/test_cases.cs, and
neither of them reproduces the problem. @dominik-weber
https://github.com/dominik-weber can you look at the code for these
test cases and see if you can spot why they do not bring about the "random
NullPointer exceptions from SQLite3Provider.collation_hook_bridge_impl"
that you are seeing? Thanks.
—
Reply to this email directly or view it on GitHub
https://github.com/ericsink/SQLitePCL.raw/issues/42#issuecomment-81920724
.

@ericsink I was unfortunately not yet able to create a test case to reproduce the problem. I have a fairly large application which currently uses SQLitePCL and I'm trying to switch to SQLitePCL.raw. I got the exceptions after I changed my code to use SQLitePCL.raw, and I can kind-of reproduce it by working with the app for a little while...
In the mean time I found another way to test if the collations work (it doesn't produce the NullReference exception though):
            string res1, res2, res3;

            using (var c1 = ugly.open(path))
            {
                c1.create_collation("col", null, (o, p1, p2) => string.Compare(p1, p2));
                res1 = c1.query_scalar<string>("select name from applications order by name collate col");

                using (var c2 = ugly.open(path))
                {
                    c2.create_collation("col", null, (o, p1, p2) => string.Compare(p2, p1));
                    res2 = c2.query_scalar<string>("select name from applications order by name collate col");
                }

                res3 = c1.query_scalar<string>("select name from applications order by name collate col");
            }

Notice the changed parameter order for the collation of c2 (c1 sorts ascending while c2 sorts descending).
When running this code you will find that res2==res3 instead of the expected result res1==res3

I installed an update for Xamarin.iOS which replaced the NullReference exception with a ArgumentException "GCHandle value belongs to a different domain". After asking Google I found out that this basically means the same thing - a GCHandle (the commit_hook_info handle) was accessed after it was freed.
(http://forums.xamarin.com/discussion/925/what-is-causing-gchandle-value-belongs-to-a-different-domain, check out the comment from RolfBjarneKvinge)
I think this exception is quite hard to test as it may depend on the VMs memory allocation.
BUT I found a way to reproduce the bug with Xamarin iOS (classic api) on iOS Simulator (iPad 2, iOS 8.2). It turned out we just have to allocate some memory:
            string res1, res2, res3;

            using (var c1 = ugly.open(path))
            {
                c1.create_collation("col", null, (o, p1, p2) => string.Compare(p1, p2));
                res1 = c1.query_scalar<string>("select name from applications order by name collate col");

                for (int i = 0; i < 10000; i ++)
                    GCHandle.Alloc(new byte[10000]);

                using (var c2 = ugly.open(path))
                {
                    c2.create_collation("col", null, (o, p1, p2) => string.Compare(p2, p1));
                    res2 = c2.query_scalar<string>("select name from applications order by name collate col");
                }

                res3 = c1.query_scalar<string>("select name from applications order by name collate col");
            }

When executing this code the line "res3 = ... " throws the GCHandle exception.

I have added test_collation_4(), which is basically the same as what @dominik-weber suggested above (the one where res1 != res3, not the one with the GCHandle exception).  This test currently fails (which is good).  I have also added test_collation_5() which is the same test except using static methods instead of lambdas.  It fails too.
I don't yet have a clean test to repro the memory-related aspect, but I can move forward with a fix for the main issue.

Great, thanks!
I think when you have a lambda which doesn't access any variables or class members it is compiled as a static function, so these should be the same.
Probably a good additional test would be like
var x = new Func<string, string, int>((p1, p2) => string.Compare(p1, p2));
sqlite3_create_collation(db, (o, p1, p2) => x(p1, p2));

or something similar, just to check if x gets garbage collected

Update just to say that I have not forgotten about this and will get it done as soon as possible.  I got a little sidetracked.  :-)

OK, I've finally pushed a fix for this, and I am inviting code review from you guys.
util.cs contains the new class called hooks which manages thread safety and another class called info which keeps a class which contains all the hooks for a given SQLite db connection.
Both sqlite3_pinvoke.cs and sqlite3_cppinterop.cs use this new functionality.  Everywhere they need to deal with a hook function, they request the info object for that connection and use it instead of the previous approach.
This has been implemented for all NINE different kinds of hook functions.
The info object for a connection is removed when that connection is closed, to make sure that another connection which happens to reuse the same IntPtr does not reuse the hooks info object.
Apologies in advance for the poor naming of these new things.  I should at least rename info.  That's lame.
Even greater apologies for the indentation issues which seem to be getting worse every time I make a commit.  I need to run an autoindent tool on all this code, but there never seems to be a good time to do that, since it'll destroy the ability to diff across the barrier.  :-(
With those caveats aside, hopefully I've got the code in place to resolve this issue.
Thanks for your patience.

I'll go through the commit and add detailed comments if I see anything.
Architecturally did you consider changing the layering such that the ISQLite3Provider methods would accept the pointer wrappers (sqlite3, etc.) instead of the raw the IntPtrs? This would allow you to hang the callbacks off the sqlite3 instances as properties similarly to what you are doing with the info class and then maintain a single mapping of IntPtrs to sqlite3 instances. This would also make implementing functions such as sqlite3_collation_needed possible.
Also I'd prefer if a the IntPtr to Info Dictionary was implemented using ConcurrentDictionary on platforms that support it. While I don't expect most users to encounter issues due to locking, SQLite generally (according to my reading of the documentation) provides good isolation between db instances, and it would be good for raw not to introduce additional concurrency concerns for users. In the future, we could even consider depending on the nuget version of System.Collections.Concurrent for the windows phone implementation.
Related to your last point re: formatting, in general I find the code base hard to work with within visual studio/xam studio. Specifically the source files are quite long and I've actually ran into issues where trying to edit the code in VS caused VS to crash or hang. That said if you are primarily using VIM/Emacs I can understand the preference. If I was to reorg the code I would probably probably break up sqlite3_pinvoke.cs into several files, using partial classes to segregate both SQLite3Provider and NativeMethods by function. This preference has to do with how I use tabs in the IDE to jump around between source files.

I was just trying to fix a [somewhat awful] bug without making any major changes.  Changing the layering would be kind of an overhaul.  Risk of breaking something else would be quite high.
More on my perspective on the risk of changes:  http://ericsink.com/articles/Four_Questions.html
FWIW, I have often wished that sqlite provided a way to store one void* pointer inside a sqlite3 conn object.  :-)
I have not yet looked at the source to ConcurrentDictionary, but I assume it would introduce the same "additional concurrency concerns for users" as my simple lock(), while simultaneously adding a package dependency.
If it makes you feel any better, the code is hard to work with within Vim as well.  I find myself switching back and forth between several different editing environments, and things have gotten messy, so I wanted to acknowledge the mess and apologize for it.  But I still don't like to mix functionality changes with formatting changes too much.

Re: Architecture, I totally get that rationale. I wanted tor bring it up though to at least consider it as an option for the future. With the current approach it might be useful to also maintain a weakref to the sqlite3 instance in the info class, so that we can go ahead and implement sqlite3_collation_needed etc.

ConcurrentDictionary internally uses one lock per bucket, so it avoids blocking read/writes across different keys in the dictionary in the common case [*]. Almost all of the supported platforms support it natively (iOS, Android, Net45) within the System.Collections.Concurrent namespace. The lone exception is win phone and in that case i was proposing to follow the bait and switch model in the short term using the locking technique you've implemented, and then consider in the long term switching to the dependent package once its out of beta.

Definitely agree that formatting/organization changes can wait and should be separate.

[*] For more detail see: https://www.simple-talk.com/blogs/2012/02/22/inside-the-concurrent-collections-concurrentdictionary/

@ericsink thanks for the update!
Just looking at your code it seems fine, one thing to think about though: collation/function names in SQLite are case insensitive (at least to some degree, as SQLite doesn't understand foreign languages), where your implementation uses a case sensitive Dictionary. I don't think this is a real problem as it would "only" allow some pointers to still be in memory that aren't used by SQLite anymore, but as you only have a finite many options to upper/lowercase function names the possible memory overhead would also be finite (and removed once the connection is closed). I currently can't imagine a clean way of resolving this issue as C# doesn't have a nocase comparison which works like the one SQLite uses.
Could you please also upload an updated nuget package so I can test the code with my application?

Can the case insensitivity issue be resolved by calling name.ToLower(CultureInfo.InvariantCulture) on the keys during insert and retrieval? Also doesn't SQLite only use case insensitivity when the name is not quoted? Might need to consider this case when determining if the name should be matched case insensitively or not.

I'm not too sure about quotes, but using ToLower or the more elegant version of using the new Dictionary<>(StringComparer.OrdinalIgnoreCase) constructor would be very wrong as C# for example compares Ä and ä to be the same (I'm from germany ;)) where SQLite does only nocase letters A through Z. So you would again end up having released pointers which could still be in use.
Using the case sensitive comparison you at least eliminate the worst case of invalid active pointers while still having the less severe problem of probably having some still alive but no longer in use pointers.
It would probably be good to add a comment to the source code which explains this problem though.

Version 0.8.0-pre2 has been pushed to nuget.

I updated to 0.8.0-pre2 and think I've found a bug (one of my sqlitepcl.pretty tests is failing with a null reference exception).
I think in ISQLite3Provider.sqlite3_close(IntPtr db) the order of operations should be reversed such that the hooks are freed after the datatabase is closed, see below:
int ISQLite3Provider.sqlite3_close(IntPtr db)
{
    int rc;
    rc = NativeMethods.sqlite3_close(db);
    hooks.removeFor(db);
    return rc;
}

The SQLitePCL.pretty test that is failing is this:

public void TestIsAutoCommit()
 {
     using (var db = SQLite3.OpenInMemory())
     {
            Assert.True(db.IsAutoCommit);
            db.Execute("BEGIN TRANSACTION;");
            Assert.False(db.IsAutoCommit);
        }
    }
}

The reason it is failing is that SQLitePCL.pretty exposes the Rollback hook as a C# event on IDatabaseConnection instances. Under the covers, the underlying implementation at construction time registers a rollback hook with SQLite that then publishes the event.
In the above test, the call to dispose on the db instance var, results in a corresponding call to sqlite3_close(). Since there is a hanging transaction a rollback hook call is made, but SQLitePCL.raw has already released the hooks, so I get a null pointer.
The stack trace is below:
Result Message:	System.NullReferenceException : Object reference not set to an instance of an object.
Result StackTrace:
at SQLitePCL.SQLite3Provider.rollback_hook_bridge_impl(IntPtr p)
   at SQLitePCL.SQLite3Provider.NativeMethods.sqlite3_close(IntPtr db)
   at SQLitePCL.SQLite3Provider.SQLitePCL.ISQLite3Provider.sqlite3_close(IntPtr db)
   at SQLitePCL.raw.sqlite3_close(sqlite3 db)
   at SQLitePCL.pretty.SQLiteDatabaseConnection.Dispose(Boolean disposing) in y:\Desktop\Projects\SQLitePCL.pretty\SQLitePCL.pretty\DatabaseConnection.cs:line 1162
   at SQLitePCL.pretty.SQLiteDatabaseConnection.Dispose() in y:\Desktop\Projects\SQLitePCL.pretty\SQLitePCL.pretty\DatabaseConnection.cs:line 1149
   at SQLitePCL.pretty.tests.SQLiteDatabaseConnectionTests.TestIsAutoCommit() in y:\Desktop\Projects\SQLitePCL.pretty\SQLitePCL.pretty.tests\DatabaseConnectionTests.cs:line 251

I daresay you are correct.  Nice catch.  I'll fix.
E
On Thursday, April 2, 2015, Dave<EMAIL_ADDRESS>wrote:

I updated to 0.8.0-pre2 and think I've found a bug (one of my
sqlitepcl.pretty tests is failing with a null reference exception).
I think in ISQLite3Provider.sqlite3_close(IntPtr db) the order of
operations should be reversed such that the hooks are freed after the
datatabase is closed, see below:
int ISQLite3Provider.sqlite3_close(IntPtr db)
{
int rc;
rc = NativeMethods.sqlite3_close(db);
hooks.removeFor(db);
return rc;
}
The SQLitePCL.pretty test that is failing is this:
public void TestIsAutoCommit()
{
using (var db = SQLite3.OpenInMemory())
{
Assert.True(db.IsAutoCommit);
db.Execute("BEGIN TRANSACTION;");
Assert.False(db.IsAutoCommit);
}
}
}
The reason it is failing is that SQLitePCL.pretty exposes the Rollback
hook as a C# event on IDatabaseConnection instances. Under the covers, the
underlying implementation at construction time registers a rollback hook
with SQLite that then publishes the event.
In the above test, the call to dispose on the db instance var, results in
a corresponding call to sqlite3_close(). Since there is a hanging
transaction a rollback hook call is made, but SQLitePCL.raw has already
released the hooks, so I get a null pointer.
The stack trace is below:
Result Message: System.NullReferenceException : Object reference not set to an instance of an object.
Result StackTrace:
at SQLitePCL.SQLite3Provider.rollback_hook_bridge_impl(IntPtr p)
at SQLitePCL.SQLite3Provider.NativeMethods.sqlite3_close(IntPtr db)
at SQLitePCL.SQLite3Provider.SQLitePCL.ISQLite3Provider.sqlite3_close(IntPtr db)
at SQLitePCL.raw.sqlite3_close(sqlite3 db)
at SQLitePCL.pretty.SQLiteDatabaseConnection.Dispose(Boolean disposing) in y:\Desktop\Projects\SQLitePCL.pretty\SQLitePCL.pretty\DatabaseConnection.cs:line 1162
at SQLitePCL.pretty.SQLiteDatabaseConnection.Dispose() in y:\Desktop\Projects\SQLitePCL.pretty\SQLitePCL.pretty\DatabaseConnection.cs:line 1149
at SQLitePCL.pretty.tests.SQLiteDatabaseConnectionTests.TestIsAutoCommit() in y:\Desktop\Projects\SQLitePCL.pretty\SQLitePCL.pretty.tests\DatabaseConnectionTests.cs:line 251
—
Reply to this email directly or view it on GitHub
https://github.com/ericsink/SQLitePCL.raw/issues/42#issuecomment-88976856
.

0.8.0-pre3 (with a fix for the most recent problem above) is now pushed to nuget.

My tests are passing now.

0.8.0-pre3 seems to be working fine with my application, I'll let you know if any bugs are reported by the users. Thanks for your good work! ;)


Patch for 2023-10-24 changes to classes

Discord decided to change all their classes from
class-hash
to
class_hash
So to fix your selectors just replace all the -s with _s.

should be fixed now, sorry for the delay


update label and description.

Relation #219816.

Label -> Language Label
Description -> Language Id
Consistent with Language Mode.
@uRohan

It seems that the https://github.com/microsoft/vscode/labels/polish label would be appropriate for this PR.


nixos/chromium: add the ability to set initial preferences

Description of changes
This adds the ability to configure initial preferences for Chromium. Currently it is only possible to configure the enterprise policies using programs.chromium.extraOpts. When certain policies are enabled, users can't change those settings and they see the "Your browser is managed by your organisation" message in the browser. Initial preferences don't have such restrictions since they are just used as a template for the user's personal settings. More information on policies vs preferences can be found in the Chromium documentation.
The standard location of the initial preferences is next to the Chromium binary. This PR add a patch to read them from /etc/chromium/inital_preferences to make it easier to configure in NixOS.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

LGTM :)
I usually don't review changes to the nixos/chromium module, but this is an exception because it touches the chromium derivation itself as well.
I will start a run of nixpkgs-review on this PR shortly, just to make sure there isn't some obscure electron version that fails to compile because of this newly added change.
Especially because electron does not even support this feature.
Will comment (and maybe merge) after nixpkgs-review is done compiling.
Thank you for your contribution :)

Thank you! Let me know if you find any issues :)

nixpkgs-review is happy :)
Just one more thing:
You need to squash/rebase your last commit into your first commit.
As per our commit conventions we don't have "fix up" commits but rather just rebase and force push all the time.
I'll mark your PR as draft until that's done, so if the rebase goes sideways (happens from time to time), GitHub won't mass-ping all codeowners.
If you need help or have questions let me know.
PS: Do you want this change to be backported to 23.11 stable as well?

Result of nixpkgs-review pr 293251 run on x86_64-linux 1

2 packages marked as broken and skipped:

archivebox
archivebox.dist

2 packages blacklisted:

nixos-install-tools
tests.nixos-functions.nixos-test

1 package failed to build:

redisinsight

73 packages built:

authy
aws-azure-login
bilibili
bitwarden-desktop
bitwarden-directory-connector
bruno
camunda-modeler
chromium
chromium.sandbox
deltachat-desktop
drawio
drawio-headless
electron (electron_28)
electron_26
electron_27
element-desktop
element-desktop-wayland
fast-cli
freetube
headset
heroic
heroic-unwrapped
itch
jitsi-meet-electron
logseq
mattermost-desktop
mermaid-cli
mermaid-filter
micropad
morgen
nix-tour
obsidian
open-stage-control
pandoc-drawio-filter
pandoc-drawio-filter.dist
percollate
playwright-test
pocket-casts
podman-desktop
pritunl-client
puppeteer-cli
python311Packages.mkdocs-drawio-exporter
python311Packages.mkdocs-drawio-exporter.dist
python311Packages.pytest-playwright
python311Packages.pytest-playwright.dist
python312Packages.mkdocs-drawio-exporter
python312Packages.mkdocs-drawio-exporter.dist
python312Packages.pytest-playwright
python312Packages.pytest-playwright.dist
r2modman
revolt-desktop
sharedown
sitespeed-io
standardnotes
stretchly
super-productivity
teams-for-linux
terra-station
thedesk
threema-desktop
uhk-agent
uhk-udev-rules
uivonim
ungoogled-chromium
ungoogled-chromium.sandbox
vesktop
vhs
vieb
wayback
webcord
webcord-vencord
webtorrent_desktop
youtube-music

nixpkgs-review is happy :)
Just one more thing: You need to squash/rebase your last commit into your first commit. As per our commit conventions we don't have "fix up" commits but rather just rebase and force push all the time.
I'll mark your PR as draft until that's done, so if the rebase goes sideways (happens from time to time), GitHub won't mass-ping all codeowners.
If you need help or have questions let me know.
PS: Do you want this change to be backported to 23.11 stable as well?
Result of nixpkgs-review pr 293251 run on x86_64-linux 1
2 packages marked as broken and skipped:
2 packages blacklisted:
1 package failed to build:
73 packages built:

Ah, ok, I amended the commit and rebased on top of master. Thanks for testing it!
I guess it would be nice to backport it to 23.11 if you think it meets the criteria.

Awesome, thanks!
Will try to merge this together with #293845 (most likely tomorrow).


Create NODEJUMPER.toml

Description
All previous genesis validators should name their PRs "Update {validator_alias}.toml for tesntet-15" and provide links to previous PRs merged.
If this is an UPDATE for a previous genesis validator
Checks are made against your net_address. If this has changed since the previous testnet, make sure you provide links of previous prs merged from your previous git username.
Thanks in advance!
Checklist before merging

[ ] Only one toml is added in this PR
[ ] The file being added is indeed a .toml file
[ ] The toml being added is to the correct folder, and only to the correct folder
[ ] The eth_hot_key and eth_cold_key are present
[ ] The email, discord, elements telegram, and twitter fields are present and valid

The transactions.toml file for 62971c7 was deemed invalid with error Invalid signatures: Signatures could not be verified

The transactions.toml file for 62971c7 was deemed invalid with error Invalid signatures: Signatures could not be verified

The transactions.toml file for 989acdc was deemed invalid with error Invalid signatures: Signatures could not be verified

The transactions.toml file for 989acdc was deemed invalid with error Invalid signatures: Signatures could not be verified

The transactions.toml file for 989acdc was deemed invalid with error Invalid signatures: Signatures could not be verified

The transactions.toml file for 989acdc was deemed invalid with error Invalid signatures: Signatures could not be verified

@bengtlofgren hi! I created a new PR with correct validator toml file https://github.com/anoma/namada-testnets/pull/3171. Should I close this PR or update with new correct validator toml file?

The transactions.toml file for 989acdc was deemed invalid with error Invalid signatures: Signatures could not be verified


Wdo3650/breadcrumb additions

Added remove and add methods from master.
Typed variables
Removed class removal utility

Looks good!

Is PullApprove working? I approve this PR.

Approved

Approve

Could you try merging beta/3.0.0 into this branch to see if it clears up the build errors?

Approved (test)


OfType Followed by Some Operations Generates Non-compiling Code

I've only tested for Select and Where as delegates, but it may be more widespread. There are tests covering Skip and Take, and they seem to work fine. It will probably only affect operations that take a selector/predicate.
Here is some example user code that causes the problem:
TestData.ObjectStringArray.Gen().OfType<string>().Select(g => string.IsNullOrEmpty(g) ? '\0' : g[0])

TestData.ObjectStringArray.Gen().OfType<string>().Where(g => string.IsNullOrEmpty(g))

The Select generates (with most excluded for brevity)
internal struct Select_uF<TOut_p2> : IInternalStub<char>
    {
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        internal Select_uF(in OfType_p2<TOut_p2> source, global::System.Func<string, char> selector_uF) : this()
        {
            this.source_n2 = source.source_n2;
            this.selector_uF = selector_uF;
        }
        ...
            [MethodImpl(MethodImplOptions.AggressiveInlining)]
            public bool MoveNext()
            {
               ...
                while (true)
                {
                    ...
                    var current_p2 = (TOut_p2)(object)current_n2;
                    var current_uF = parent.selector_uF.Invoke(current_p2); // Error here
                    ...
                }

                return false;
            }

I think the constructor should take a Func<TOut_p2, char> instead of Func<string, char> as the error is Argument type 'TOut_p2' is not assignable to parameter type 'string'
Locally I inserted these two lines at https://github.com/cathei/LinqGen/blob/main/LinqGen.Tests/OperationTests.tt#L94 to generate failing tests:
("OfTypeSelect", "OfType<string>().Select(g => string.IsNullOrEmpty(g) ? '\0' : g[0])", "OfType<string>().Select(g => string.IsNullOrEmpty(g) ? '\0' : g[0])", castSources),
("OfTypeWhere", "OfType<string>().Where(g => string.IsNullOrEmpty(g))", "OfType<string>().Where(g => string.IsNullOrEmpty(g))", castSources),

Thank you for reporting this issue! It was due to operations directly used the inferred type, not applying generic type arguments. It will be fixed in next release.

Included in v0.2.2


fix: make redis port necessary

Description:
Makes the port 6379 necessary insted of optional, because the local node is unable to start properly if it is already in use.
Related issue(s):
Fixes #
Notes for reviewer:

Checklist

[ ] Documented (Code comments, README, etc.)
[ ] Tested (unit, integration, etc.)

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 51.39%. Comparing base (d34c153) to head (ed03553).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #580   +/-   ##
=======================================
  Coverage   51.39%   51.39%
=======================================
  Files          24       24
  Lines        1181     1181
  Branches      136      136
=======================================
  Hits          607      607
  Misses        550      550
  Partials       24       24

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Release build is failing with minifyDevelopmentReleaseWithR8 with version 15

Task :app:minifyDevelopmentReleaseWithR8 FAILED
ERROR:/project_path/.gradle/caches/transforms-3/2c8d5bc13302d1152f8c7e8d06ed71db/transformed/jetified-sdk-3.6.1-runtime.jar: R8: Type com.exponea.style.AppInboxListViewStyle is defined multiple times: /project_path/.gradle/caches/transforms-3/2c8d5bc13302d1152f8c7e8d06ed71db/transformed/jetified-sdk-3.6.1-runtime.jar:com/exponea/style/AppInboxListViewStyle.class, /project_path/Documents/Alshaya/Alshaya/hm/node_modules/react-native-exponea-sdk/android/build/.transforms/0ea2fd2e68e458c75c7c946d37d3aa92/transformed/classes.jar:com/exponea/style/AppInboxListViewStyle.class

It seems something needs to add in proguard.rules
React Native Exponea SDK (Tried this but not working)
-keep class com.exponea.** { *; }

Working fine with 1.5.1


[Bugs] Facet returns Error JObject not compatible with JArray

When adding a facet to the querybuilder, in my case "Categories.Id", this is of type List in c#.  We are trying to facet on the Id of the categories.id value.
To Reproduce
Steps to reproduce the behavior:

Add a querybuilder to the page
Add a facet of anytime to the querybuilder, in my case .Facet("Categories.Id")
in the response.GetContent() method, the error shows
JsonSerializationException: Deserialized JSON type 'Newtonsoft.Json.Linq.JObject' is not compatible with expected type 'Newtonsoft.Json.Linq.JArray'. Path 'facets.Categories'.

Expected behavior
To return an list of Facets with name and count
Screenshots

Package (please complete the following information):

OS: Windows and Linux(dxp),
version 1.0.0 or client

@JoshuaFolkerts this bug already fixed in Optimizely.Graph.Client version 1.0.1, please try again and confirm that your errors are fixed.
Feel free to reopen this issue if you still get the errors. Thanks!


Finalized version look up

We can only look up the state of the latest version. It would be nice to provide this to choose between the finalized version and the latest version.

Done


config.service.endpoint.disableCertificateValidation

Add 'config.service.endpoint.disableCertificateValidation=true' because of the new Configuration V2 custom domain.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: eab8afa1-e7d8-a645-2858-013dffc1a879
Version Independent ID: 17c69062-41a7-3d25-fd25-0bd48f133484
Content: Reference - Self-hosted gateway container settings - Azure API Management
Content Source: articles/api-management/self-hosted-gateway-settings-reference.md
Service: api-management
GitHub Login: @dlepow
Microsoft Alias: danlep

@tiago-msft
Thanks for your feedback! We will investigate and update as appropriate.

@tomkerkhove Can you please review this?

We don't recommending using it but I've opened a PR to document it in case people know what they are doing.

Doc update completed, so closing this issue. Thanks all.
#please-close


AudioNavigator.seek cannot cross previous resource boundary

Describe the bug
seek seems unusable for a slider component or rewind button in a multi-resource audiobook, because it cannot go further back than the beginning of the current resource.
How to reproduce?

Create an AudioNavigator from an audiobook with multiple resources.
Observe that navigator.seek(to: time) and navigator.seek(by: -time) cannot move the current playback position from resource n to resource n-1. Instead, it moves playback to the beginning of resource n.

Readium version
I tried 2.6.1 and develop
OS version
17.2
Testing device
simulator
Environment
Xcode 15.2
Build version 15C500b

Additional context
No response

It works as intended. An audiobook player usually provides scrubbing control for the current resource only.
Hovewer, I get that it may not fit all needs. How about these alternatives?
enum TimeScope {
    case currentResource
    case publication
}

func seek(to time: Double, in scope: TimeScope = .currentResource)

navigator.seek(to: 30, in: .publication)

and having seek(by:) skip to the previous resource. Although for the next resource, I think it might be more user-friendly to still start from the beginning of the next chapter instead of actually skipping x seconds.
Are you interested in contributing these changes?

I wonder if perhaps we are conflating resources and chapters, or perhaps I am misunderstanding. Does Readium process embedded chapter metadata and convert to "virtual" resources? If not, I would love if there was a unified way to deal with publication chapters for both embedded chapters (like in an M4B) and per-file chapters (like in a zipped audiobook of MP3s). Any thoughts on providing a unified API for navigating chapters?
Regarding the above TimeScope proposal, I'm open to contributing it, but I would quite like to hear your thoughts about chapter navigation abstraction first.

It doesn't, in Readium a "resource" always refers to a single entry in a ZIP package, and/or an entry in a Readium Audiobook Manifest.
There's no plan to parse a chapter list from a M4B directly in the toolkit's AudioParser. But you could implement a custom PublicationParser for M4B which returns an audiobook Publication, which would work with the AudioNavigator.
For chapters (as opposed to resource), we store them in the publication.tableOfContents property, which is a tree structure. We don't have any API to navigate relatively through chapters right now, mainly because in EPUB it gets complicated as sorting chapters require to parse the HTML.
But for an audio table of contents it should be easy, as they would use a time fragment or progression percentage. What would an ideal chapter navigation API look like to you?

I didn't know about publication.tableOfContents. Cool!
Does "no plan" to parse a chapter list in AudioParser mean Readium has decided against it or that it's not specifically planned but would be accepted as a contribution?
I think the ideal case would be the parser populating tableOfContents from the asset in whatever way it is provided (chapterized MP3 files, M4B/MP3 metadata, CUE file, etc.).
I'm not sure about the ideal API for utilizing chapters, but I'll try to talk through my potential use cases. For current playback and book overview, I'd want to be able to display chapter title, duration, and progress. For playback, I'd want to be able to seek forward and backward within chapters and between chapters, essentially what you proposed for resource seeking with time scopes. Perhaps this points to TimeScope needing a case for currentChapter?

Does "no plan" to parse a chapter list in AudioParser mean Readium has decided against it or that it's not specifically planned but would be accepted as a contribution?

It depends on whether this requires a third-party dependency. We aim to minimize them in the toolkit.
In any case, adding extensibility to the AudioParser for the app to integrate a table of contents algorithm could be interesting.

For current playback and book overview, I'd want to be able to display chapter title, duration, and progress.

Maybe we need a tableOfContentsIndex in the MediaPlaybackInfo? I hesitate calling it chapter, as we don't really have this concept in the APIs.

In what ways is a PublicationParser more powerful than a Publication.Builder.Transform?

The PublicationParser creates the components of a Publication.Builder from scratch. Usually you implement it to parse a given publication specification (EPUB, PDF, Zipped audiobook) into a Publication object.
Publication.Builder.Transform is an extension point for modifying Publication.Builder components before creating the Publication object. You can for example adjust the publication metadata or reorder/filter the reading order. As it's independent of a publication format, it can be used across various PublicationParser implementations.

Thanks for the explanation! It sounds like Publication.Builder.Transform will be sufficient for adding table of contents information since it is able to modify the manifest, as long as the audiobook is otherwise parsable by the default parser.

Would you accept a PR that changes AudioParser to include additional information in the manifest without requiring any other dependencies? I would like to include resource durations, bitrates, etc.


Behavior of organization owned entities - editing data

It looks like organization owned entity records are not editable in the UCI. Is that true? and if so, I think it should be called out under the "Entity ownership" topic on this page. Also, it would be good to tell where one would be able to edit the records for such an entity in PowerApps.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d497c807-5338-83e8-bbc9-3a03453cfc9f
Version Independent ID: 0bac7894-b1c3-28e9-aa6e-e6b24eaa74f3
Content: Entity metadata - PowerApps
Content Source: powerapps-docs/developer/common-data-service/entity-metadata.md
Service: powerapps
GitHub Login: @mayadumesh
Microsoft Alias: jdaly

Cannot reproduce.


yarn start command giving  "Module not found: Error: Can't resolve 'react-dom/client' in"

Preflight Checklist

[x] I have read the Contributing Guidelines for this project.
[x] I have searched the issue tracker for an issue that matches the one I want to file, without success.

Issue Details

Operating System:

Mac OS

Browser:

Chrome

Expected Behavior
As per steps given in  https://www.cloudcarbonfootprint.org/docs/create-app , client should be up and running after completing guided install and yarn start
Actual Behavior

created app as per steps given in https://www.cloudcarbonfootprint.org/docs/create-app
Completed guided install and executed yarn start getting below error
yarn start
[1] [INFO] 15:21:49 ts-node-dev ver. 2.0.0 (using ts-node ver. 10.9.1, typescript ver. 4.9.5)
[0] (node:66807) [DEP_WEBPACK_DEV_SERVER_ON_AFTER_SETUP_MIDDLEWARE] DeprecationWarning: 'onAfterSetupMiddleware' option is deprecated. Please use the 'setupMiddlewares' option.
[0] (Use node --trace-deprecation ... to show where the warning was created)
[0] (node:66807) [DEP_WEBPACK_DEV_SERVER_ON_BEFORE_SETUP_MIDDLEWARE] DeprecationWarning: 'onBeforeSetupMiddleware' option is deprecated. Please use the 'setupMiddlewares' option.
[0] Starting the development server...
[0]
[1] 2023-02-01T09:51:55.348Z [server] info: Cloud Carbon Footprint Server listening at http://localhost:4000
[0] Failed to compile.
[0]
[0] Module not found: Error: Can't resolve 'react-dom/client' in '/Users/shruthinarayan/my-carbon-footprint/packages/client/src'
[0] ERROR in ./src/index.tsx 8:0-46
[0] Module not found: Error: Can't resolve 'react-dom/client' in '/Users/shruthinarayan/my-carbon-footprint/packages/client/src'
[0]
[0] webpack compiled with 1 error

To Reproduce

Screenshots

Additional Information

© 2021 Thoughtworks, Inc.

Hi @shruthinarayan11 thanks for bringing this to our attention! The issue seems to be that some of the React dependencies for the create-app template are outdated since we have made the migration to React 18. This must've slipped through the cracks, so apologies for the inconvenience and we'll push out a fix asap!
To fix your current Create-app build, you can run the following command to update the necessary dependencies:
yarn up react react-dom @types/react @types/react-dom
This should update the dependencies referenced in the client and get rid of that error for you :)

Thannks @4upz , I ran yarn up as given by you. I am facing below issue when running yarn start
[1] [INFO] 11:26:40 ts-node-dev ver. 2.0.0 (using ts-node ver. 10.9.1, typescript ver. 4.9.5)
[0] Invalid options object. Dev Server has been initialized using an options object that does not match the API schema.
[0]  - options.allowedHosts[0] should be a non-empty string.
[0] yarn start-client exited with code 1
[1] (node:29252) [DEP0128] DeprecationWarning: Invalid 'main' field in<EMAIL_ADDRESS>of 'dist/index.js'. Please either fix that or report it to the module author
[1] (Use node --trace-deprecation ... to show where the warning was created)
[1] (node:29252) [DEP0128] DeprecationWarning: Invalid 'main' field in<EMAIL_ADDRESS>of 'dist/index.js'. Please either fix that or report it to the module author
[1] (node:29252) [DEP0128] DeprecationWarning: Invalid 'main' field in<EMAIL_ADDRESS>of 'dist/index.js'. Please either fix that or report it to the module author
[1] (node:29252) [DEP0128] DeprecationWarning: Invalid 'main' field in<EMAIL_ADDRESS>of 'dist/index.js'. Please either fix that or report it to the module author
[1] (node:29252) [DEP0128] DeprecationWarning: Invalid 'main' field in<EMAIL_ADDRESS>of 'dist/index.js'. Please either fix that or report it to the module author
[1] (node:29252) [DEP0128] DeprecationWarning: Invalid 'main' field in<EMAIL_ADDRESS>of 'dist/index.js'. Please either fix that or report it to the module author
[1] 2023-02-03T05:56:49.926Z [server] info: Cloud Carbon Footprint Server listening at http://localhost:4000
Can you please guide on this issue

No problem! @shruthinarayan11 As for the new error, I'm unable to reproduce this one after updating the packages, so this may be proxy/host issues that are specific to your environment. Are you running the app locally or in a remote environment?
You can try editing or removing the proxy key/value in the packages/client/package.json file. This Stack Overflow post also has some other useful solutions such as setting up your own middleware for the client to proxy a connection to the server:  https://stackoverflow.com/questions/70374005/invalid-options-object-dev-server-has-been-initialized-using-an-options-object
Let me know if any of that helps

Thanks @4upz I was able to run now without connecting to VPN . I am trying to connect to one of AWS account and get insights from it . I followed guided install and configured AWS account and i can see the environment variables updated in Api env file also
But In the Client ,  I do not see account listed as shown below

can you please guide on this issue

Hi @shruthinarayan11! Apologies, this seems to be due to more underlying dependencies that were outdated in that template. We just recently released new versions of all of our packages -- including the create-app templates. I would recommend creating a new app using the npx command and doing another guided-install so that you're using the most stable version (v. 2.2.1). I just tested it out on my end and as long as the proper credentials are given during the guided-install, you should be able to get up and running more smoothly.

Thank @4upz , I will try and update you

Sounds good! Closing this issue since the original problem was resolved, but feel free to open a new one if you're still facing some of the other problems we discussed