Add MIRNet TFJS Models

Add TFJS version of MIRNet model as proposed by Learning Enriched Features for Real Image Restoration and Enhancement by Zamir et al.  This model is capable of enhancing low-light images upto a great extent.
I also request you to kindly take a look at the headings for the TFJS models, I found that the docs were not very clear so as to what should be entered there for multiple models

Any pull request you open is subject to the TensorFlow Hub Terms of Service at www.tfhub.dev/terms.

@Rishit-dagli thank you for flagging the error. I'm looking into it and will update shortly.

@Rishit-dagli thank you for flagging the error. I'm looking into it and will update shortly.

@Rishit-dagli I identified the reason why the test is failing.
The problem is the saved-model.tar.gz archive. Running tar --list -f saved-model.tar.gz results in:
saved_model.pb
variables/variables.data-00000-of-00001
variables/variables.index

Other tar.gz saved models on tfhub.dev for which the tests pass have the following tar --list output:
./
./saved_model.pb
./variables/
./variables/variables.index
./variables/variables.data-00000-of-00001

Note that the variables/ directory is not explicitly listed in tar --list for the tar.gz is this PR. This is the root cause of the error and I'm investigating a fix in the tfhub library that should resolve the failure.
Our of curiosity, was the tar.gz created by some library/automation or did you manuall create it? I guess you can try recreating the tar.gz using the Linux built-in tar command.

@Rishit-dagli I identified the reason why the test is failing.
The problem is the saved-model.tar.gz archive. Running tar --list -f saved-model.tar.gz results in:
saved_model.pb
variables/variables.data-00000-of-00001
variables/variables.index

Other tar.gz saved models on tfhub.dev for which the tests pass have the following tar --list output:
./
./saved_model.pb
./variables/
./variables/variables.index
./variables/variables.data-00000-of-00001

Note that the variables/ directory is not explicitly listed in tar --list for the tar.gz is this PR. This is the root cause of the error and I'm investigating a fix in the tfhub library that should resolve the failure.
Our of curiosity, was the tar.gz created by some library/automation or did you manuall create it? I guess you can try recreating the tar.gz using the Linux built-in tar command.

@Rishit-dagli I identified the reason why the test is failing.
The problem is the saved-model.tar.gz archive. Running tar --list -f saved-model.tar.gz results in:
saved_model.pb
variables/variables.data-00000-of-00001
variables/variables.index

Other tar.gz saved models on tfhub.dev for which the tests pass have the following tar --list output:
./
./saved_model.pb
./variables/
./variables/variables.index
./variables/variables.data-00000-of-00001

Note that the variables/ directory is not explicitly listed in tar --list for the tar.gz in this PR. This is the root cause of the error and I'm investigating a fix in the tfhub library that should resolve the failure.
Our of curiosity, was the tar.gz created by some library/automation or did you manually create it? I guess you can try recreating the tar.gz using the Linux built-in tar command.

I created this .tar.gz file with the Python  library shutil using the shutil.make_archive method. However, as you suggested I will recreate the .tar.gz using the Linux built-in tar command.

@Rishit-dagli I identified the reason why the test is failing.
The problem is the saved-model.tar.gz archive. Running tar --list -f saved-model.tar.gz results in:
saved_model.pb
variables/variables.data-00000-of-00001
variables/variables.index

Other tar.gz saved models on tfhub.dev for which the tests pass have the following tar --list output:
./
./saved_model.pb
./variables/
./variables/variables.index
./variables/variables.data-00000-of-00001

Note that the variables/ directory is not explicitly listed in tar --list for the tar.gz in this PR. This is the root cause of the error and I'm investigating a fix in the tfhub library that should resolve the failure.
Our of curiosity, was the tar.gz created by some library/automation or did you manually create it? I guess you can try recreating the tar.gz using the Linux built-in tar command.

I created this .tar.gz file with the Python  library shutil using the shutil.make_archive method. However, as you suggested I will recreate the .tar.gz using the Linux built-in tar command.

@Rishit-dagli I identified the reason why the test is failing.
The problem is the saved-model.tar.gz archive. Running tar --list -f saved-model.tar.gz results in:
saved_model.pb
variables/variables.data-00000-of-00001
variables/variables.index

Other tar.gz saved models on tfhub.dev for which the tests pass have the following tar --list output:
./
./saved_model.pb
./variables/
./variables/variables.index
./variables/variables.data-00000-of-00001

Note that the variables/ directory is not explicitly listed in tar --list for the tar.gz in this PR. This is the root cause of the error and I'm investigating a fix in the tfhub library that should resolve the failure.
Our of curiosity, was the tar.gz created by some library/automation or did you manually create it? I guess you can try recreating the tar.gz using the Linux built-in tar command.

Thanks, @maringeo incorporating these suggestions the Markdown Validator tests passed though I am a bit curious about:

Directories not showing up explicitly in .tar.gz created by utilities other than the tar command
Size difference in the model created with shutil and created with tar. The archive created with tar command is 152 MB whereas the one created with shutil is 129 MB 🤔 .

These are not directly related to the model publishing so it would be completely fine if you think it would be better to not address these questions here.

@Rishit-dagli I identified the reason why the test is failing.
The problem is the saved-model.tar.gz archive. Running tar --list -f saved-model.tar.gz results in:
saved_model.pb
variables/variables.data-00000-of-00001
variables/variables.index

Other tar.gz saved models on tfhub.dev for which the tests pass have the following tar --list output:
./
./saved_model.pb
./variables/
./variables/variables.index
./variables/variables.data-00000-of-00001

Note that the variables/ directory is not explicitly listed in tar --list for the tar.gz in this PR. This is the root cause of the error and I'm investigating a fix in the tfhub library that should resolve the failure.
Our of curiosity, was the tar.gz created by some library/automation or did you manually create it? I guess you can try recreating the tar.gz using the Linux built-in tar command.

Thanks, @maringeo incorporating these suggestions the Markdown Validator tests passed though I am a bit curious about:

Directories not showing up explicitly in .tar.gz created by utilities other than the tar command
Size difference in the model created with shutil and created with tar. The archive created with tar command is 152 MB whereas the one created with shutil is 129 MB 🤔 .

These are not directly related to the model publishing so it would be completely fine if you think it would be better to not address these questions here.

Thanks @Rishit-dagli!
I think that calling shutil.make_archive and setting base_dir might result in directories missing from the output of the tar command (see https://docs.python.org/3/library/shutil.html#shutil.make_archive).
The archive size difference between shutil and tar is also very interesting - I have no idea what might be the cause.
Thank you for updating the tar.gz - this PR looks good now and I've started the publishing process. The models should be live on tfhub.dev in the coming days - I'll comment on this thread once the models are live.

Thanks @Rishit-dagli!
I think that calling shutil.make_archive and setting base_dir might result in directories missing from the output of the tar command (see https://docs.python.org/3/library/shutil.html#shutil.make_archive).
The archive size difference between shutil and tar is also very interesting - I have no idea what might be the cause.
Thank you for updating the tar.gz - this PR looks good now and I've started the publishing process. The models should be live on tfhub.dev in the coming days - I'll comment on this thread once the models are live.

Thanks @Rishit-dagli!
I think that calling shutil.make_archive and setting base_dir might result in directories missing from the output of the tar command (see https://docs.python.org/3/library/shutil.html#shutil.make_archive).
The archive size difference between shutil and tar is also very interesting - I have no idea what might be the cause.
Thank you for updating the tar.gz - this PR looks good now and I've started the publishing process. The models should be live on tfhub.dev in the coming days - I'll comment on this thread once the models are live.

Got it, that makes sense, I will further try to explore the cause of the size differences. Thank You @maringeo .

Thanks @Rishit-dagli!
I think that calling shutil.make_archive and setting base_dir might result in directories missing from the output of the tar command (see https://docs.python.org/3/library/shutil.html#shutil.make_archive).
The archive size difference between shutil and tar is also very interesting - I have no idea what might be the cause.
Thank you for updating the tar.gz - this PR looks good now and I've started the publishing process. The models should be live on tfhub.dev in the coming days - I'll comment on this thread once the models are live.

Got it, that makes sense, I will further try to explore the cause of the size differences. Thank You @maringeo .

Hi @Rishit-dagli, models are live at https://tfhub.dev/rishit-dagli/mirnet-tfjs/1

Hi @Rishit-dagli, models are live at https://tfhub.dev/rishit-dagli/mirnet-tfjs/1

Hi @Rishit-dagli, models are live at https://tfhub.dev/rishit-dagli/mirnet-tfjs/1

Thank You @maringeo , there seem to be 2 issues though:

The dataset is not appearing

The results image shows up in the Markdown files but is not rendered on tfhub

Hi @Rishit-dagli, models are live at https://tfhub.dev/rishit-dagli/mirnet-tfjs/1

Thank You @maringeo , there seem to be 2 issues though:

The dataset is not appearing

The results image shows up in the Markdown files but is not rendered on tfhub

Hi @Rishit-dagli , @WGierke identified the problem in 2 and fixed it in https://github.com/tensorflow/tfhub.dev/pull/38 .
The missing dataset should be fixed early next week.

Hi @Rishit-dagli , @WGierke identified the problem in 2 and fixed it in https://github.com/tensorflow/tfhub.dev/pull/38 .
The missing dataset should be fixed early next week.


Add unaligned feature

While the structs defined in this crate have the alignment required by the specifications, some toolchains violate these requirements. Add an unaligned feature that allows these files to be processed, at the cost of potentially slower code.
Fixes #230
I've kept this out of the all feature for now. I'm not sure if this should be enabled by default. @alexcrichton What are your thoughts on what backtrace-rs will want for this? I'm not sure how much the performance of this matters, and whether you want to selectively enable it for some architectures (most likely x86-64).

I think I may be a bit confused, what happens today if anything is unaligned? Does an error get returned in parsing or does it end up being unsound?
The way I thought something like this would look is that by default everything is considered unaligned to be safe, and then there'd be a crate feature for "assume everything is aligned" and alignment checks are added early on in parsing. (which may be higher performance?)

From https://github.com/gimli-rs/addr2line/issues/170:

thread 'main' panicked at 'called Result::unwrap() on an Err value: Error("Invalid ELF section header offset/size/alignment")', examples\addr2line.rs:156:17

Today the alignment checks are already there, and it returns an error if anything is unaligned. The feature has to be "assume everything is unaligned", because features must be additive: it is adding the ability to parse unaligned headers.

Ah ok, makes sense. I don't really have an idea of what the performance impact to this would be, but I doubt that it'd affect the backtrace crate too too much either way.

Based on sentiment here and in the bug report, it sounds like it will be more convenient if this enabled by default, so I've changed it to that.


Update README.md

Added offical discord server to readme

I can confirm that this is valid invite.
It will soon be partnered with Discord, they are currently processing it.


when is the reInit() event fired ?

Would be lovely if this happens when a new breakpoint is reached to help recalculate some stuff inside the slider

+1 same question

More of a support question than issue... see support section  in CONTRIBUTING.markdown


Reporting Dashboard: Report Due Date Displayed for Users

Description:
As an Industry Reporter, I want to see the due date for the annual report so that I am reminded of the deadline for submission.
This work involves showing the users when their Annual Reports are due for the current reporting cycle.
For example, 2024 Annual Reports will be due on May 31 2025.
We need a way to show this "due date" on the Reporting Dashboard.
Wireframe
Acceptance Criteria:
Given that I am an Industry Reporter
When I am in the BCIERS system
Then I want a clear visual to indicate when my annual report is due
Given that I am an Industry Report
When I am on the Reporting Dashboard for the current reporting year
Then I want to see the "report due date" on the page.
Development Checklist:

[ ] The text "Report Due [date]" appears above the grid matching the wireframe.
[ ] The date is in [month] [day], [year] format
[ ] "report_due_date" datetime column in the reporting_year table
[ ] Due date value taken from reporting_year table
[ ] Meets the DOD

Definition of Ready (Note: If any of these points are not applicable, mark N/A)

[x] User story is included
[x] User role and type are identified
[ ] Acceptance criteria are included
[x] Wireframes are included (if required)
[ ] Design / Solution is accepted by Product Owner
[x] Dependencies are identified (technical, business, regulatory/policy)
[x] Story has been estimated (under 13 pts)

Definition of Done (Note: If any of these points are not applicable, mark N/A)

[ ] Acceptance criteria are tested by the CI pipeline
[ ] UI meets accessibility requirements
[ ] Configuration changes are documented, documentation and designs are updated
[ ] Passes code peer-review
[ ] Passes QA of Acceptance Criteria with verification in Dev and Test
[ ] Ticket is ready to be merged to main branch
[ ] Can be demoed in Sprint Review
[ ] Bugs or future work cards are identified and created
[ ] Reviewed and approved by Product Owner

Notes:

@suhafa curious what you're envisioning for this ticket. Is something sent to users (i.e. a notification) or is just information displayed on the app? something else? Let's chat!

@BallardRobinett to write dev checklist

Could a developer share the link to where I can see this? Also looking for confirmation that the dev checklist is complete. Thanks!


Blacklist neg/patmatexhaust-huge.scala.

It exhibits spurious failures, and anyway it's testing the pattern matcher exhaustivity checks, which does not concern Scala.js.

Refer to this link for build results (access rights to CI server needed):
https://scala-webapps.epfl.ch/jenkins/job/scalajs-pr/3903/
https://scala-webapps.epfl.ch/jenkins/job/scalajs-matrix-build/5162/
Test PASSed.


windows_share resource adds everyone group to share

Description
"everyone" is granted read access to shares created with the windows_share resource
Chef Version
14.7.17
Platform Version
Windows Server 2016
Replication Case
windows_share 'MyShare' do
action :create
path archive_folder
full_users [ "domain\domain admins", "domain\user2" ]
read_users [ "domain\user3" ]
end
Client Output
n/a
Stacktrace
n/a

This is the default behavior of New-SmbShare powershell command as a default group needs access, similar to the CREATOR OWNER permission for NTFS.
The everyone group can be removed using Revoke-SmbShareAccess -Name "MyShare" -AccountName "Everyone" -Force.


docs: Add deployment steps for apps in docker directory

Archiving resources, to be deployed on https://github.com/dbrennand/home-ops/issues/2


Inquiries Regarding Ollama Tool Usage

Hi,
Thanks for Ollama tool, it's been a fantastic resource!
I have a couple of inquiries I hope you could assist me with:

I recently executed the following command:
ollama create dolphin.mistral -f Modelfile.dolphin.mistral

The contents of my Modelfile.dolphin.mistral are as follows:
FROM ./dolphin-2.5-mixtral-8x7b.Q4_K_M.gguf
TEMPLATE """{{ if and .First .System }}system
{{ .System }}
{{ end }}user
{{ .Prompt }}
assistant
"""
SYSTEM """You are Dolphin, an uncensored, unbiased AI assistant"""
PARAMETER stop

Could you please guide me on the typical duration for the ollama create process to complete?

I also noticed that the output of ollama create is typically saved in the following directory:
/usr/share/ollama/.ollama/models/

Is it possible to designate an alternative directory for storing this output, apart from the default location?

Sincerely,
E.W.

Hi @ewijaya,

For larger models like mixtral ollama create will take a few minutes, the longest step is that it is validating the model which requires reading the whole thing.
For changing where the models are stored you can set OLLAMA_MODEL as an environment variable when starting the ollama server: https://github.com/jmorganca/ollama/blob/main/docs/faq.md#how-can-i-change-where-ollama-stores-models

Let me know if you have any more questions.


Bug with SetWindowLong

Hey,
I updated to a new version of MahApps.Metro (1.1.2). I want that if the users drag & drop the window, it doesn't show the aero snap (because I have my own). My solution was that I hide the minimize and maximize buttons which prevents the aero snap (SetWindowLong(hwnd, GWL_STYLE, (currentStyle & ~WS_MAXIMIZEBOX & ~WS_MINIMIZEBOX));). But when I do that with the new MahApps.Metro, it looks like that after I enable them again:

When I move the mouse over the XP-styled buttons or if I move the window outside the screen, it goes away.
I made a small sample project. Perhaps someone knows why this is happening :)

@Alkalinee this is not a good solution to avoid aero snap. try this one in your sample
public MainWindow()
{
  InitializeComponent();

  this.SourceInitialized += (sender, args) => {
    var helper = new WindowInteropHelper(this);
    var currentStyle = WindowHelper.GetWindowLong(helper.Handle, WindowHelper.GWL_STYLE);
    currentStyle |= (int)WindowHelper.WindowStyles.WS_OVERLAPPEDWINDOW;
    currentStyle ^= (int)WindowHelper.WindowStyles.WS_THICKFRAME;
    WindowHelper.SetWindowLong(helper.Handle, WindowHelper.GWL_STYLE, currentStyle);
  };
}

private bool _restoreIfMove;

private void UIElement_OnMouseLeftButtonDown(object sender, MouseButtonEventArgs e)
{
  if (WindowState == WindowState.Maximized)
  {
    _restoreIfMove = true;
  }
  DragMove();
}

private void UIElement_OnMouseLeftButtonUp(object sender, MouseButtonEventArgs e)
{
  _restoreIfMove = false;
}

private void UIElement_OnMouseMove(object sender, MouseEventArgs e)
{
  if (_restoreIfMove)
  {
    _restoreIfMove = false;
    var percentHorizontal = e.GetPosition(this).X / ActualWidth;
    var targetHorizontal = RestoreBounds.Width * percentHorizontal;

    var percentVertical = e.GetPosition(this).Y / ActualHeight;
    var targetVertical = RestoreBounds.Height * percentVertical;

    WindowState = WindowState.Normal;

    POINT lMousePosition;
    WindowHelper.GetCursorPos(out lMousePosition);

    Left = lMousePosition.X - targetHorizontal;
    Top = lMousePosition.Y - targetVertical;

    try
    {
      DragMove();
    }
    catch (InvalidOperationException)
    {
    }
  }
}

Thank you, that works :)


Missing OneDrive Files

Hi,
The OneDrive links mentioned in the datasets section of your readme lead to an error page: "The item might not exist or is no longer available".
Can you please re-upload those files so I can run the model training myself?
Thanks.

I updated the links in readme. The new Cloud.THU links should work.


Angular, React, and Vue sample.

A sample that shows how to use .NET Aspire with Angular, React, and Vue.
.NET Aspire dashboard

Angular app

React app

Vue app

When checking out this commit, I followed the readme, ran npm install for all 3 of the JS projects. When running the Aspire AppHost project, the Angular app fails to start: it gets a NaN as a port, with this logged.
><EMAIL_ADDRESS>start
> ng serve --port %PORT%
Node.js version v21.1.0 detected.
Odd numbered Node.js versions will not enter LTS status and should not be used for production. For more information, please see https://nodejs.org/en/about/previous-releases/.
/Users/owen/Developer/aspire-samples/samples/AspireWithJavaScript/AspireJavaScript.Angular/node_modules/rxjs/dist/cjs/internal/util/reportUnhandledError.js:13
            throw err;
            ^
RangeError [ERR_SOCKET_BAD_PORT]: options.port should be >= 0 and < 65536. Received type number (NaN).
    at validatePort (node:internal/validators:424:11)
    at Server.listen (node:net:2000:5)
    at /Users/owen/Developer/aspire-samples/samples/AspireWithJavaScript/AspireJavaScript.Angular/node_modules/@angular-devkit/build-angular/src/utils/check-port.js:55:14
    at new Promise (<anonymous>)
    at checkPort (/Users/owen/Developer/aspire-samples/samples/AspireWithJavaScript/AspireJavaScript.Angular/node_modules/@angular-devkit/build-angular/src/utils/check-port.js:27:12)
    at initialize (/Users/owen/Developer/aspire-samples/samples/AspireWithJavaScript/AspireJavaScript.Angular/node_modules/@angular-devkit/build-angular/src/builders/dev-server/builder.js:112:63) {
  code: 'ERR_SOCKET_BAD_PORT'
}
Node.js v21.1.0

Set up:
Aspire preview 2 workload installed.
Running through dotnet run in AppHost proj
MacOs: 14.2.1
Node: v21.1.0
npm ng --version: 10.2.3
Sorry if I am being too early on this

When checking out this commit, I followed the readme, ran npm install for all 3 of the JS projects. When running the Aspire AppHost project, the Angular app fails to start: it gets a NaN as a port, with this logged.
><EMAIL_ADDRESS>start
> ng serve --port %PORT%
Node.js version v21.1.0 detected.
Odd numbered Node.js versions will not enter LTS status and should not be used for production. For more information, please see https://nodejs.org/en/about/previous-releases/.
/Users/owen/Developer/aspire-samples/samples/AspireWithJavaScript/AspireJavaScript.Angular/node_modules/rxjs/dist/cjs/internal/util/reportUnhandledError.js:13
            throw err;
            ^
RangeError [ERR_SOCKET_BAD_PORT]: options.port should be >= 0 and < 65536. Received type number (NaN).
    at validatePort (node:internal/validators:424:11)
    at Server.listen (node:net:2000:5)
    at /Users/owen/Developer/aspire-samples/samples/AspireWithJavaScript/AspireJavaScript.Angular/node_modules/@angular-devkit/build-angular/src/utils/check-port.js:55:14
    at new Promise (<anonymous>)
    at checkPort (/Users/owen/Developer/aspire-samples/samples/AspireWithJavaScript/AspireJavaScript.Angular/node_modules/@angular-devkit/build-angular/src/utils/check-port.js:27:12)
    at initialize (/Users/owen/Developer/aspire-samples/samples/AspireWithJavaScript/AspireJavaScript.Angular/node_modules/@angular-devkit/build-angular/src/builders/dev-server/builder.js:112:63) {
  code: 'ERR_SOCKET_BAD_PORT'
}
Node.js v21.1.0

Set up: Aspire preview 2 workload installed. Running through dotnet run in AppHost proj MacOs: 14.2.1 Node: v21.1.0 npm ng --version: 10.2.3
Sorry if I am being too early on this

Ah, I think that might be an issue with Windows. I passed the port to the ng serve --port switch using %PORT%, if you're not on Windows that won't work... Try changing that to ng serve --port $PORT instead.

When checking out this commit, I followed the readme, ran npm install for all 3 of the JS projects. When running the Aspire AppHost project, the Angular app fails to start: it gets a NaN as a port, with this logged.

<EMAIL_ADDRESS>start

ng serve --port %PORT%

Node.js version v21.1.0 detected.

Odd numbered Node.js versions will not enter LTS status and should not be used for production. For more information, please see https://nodejs.org/en/about/previous-releases/.

/Users/owen/Developer/aspire-samples/samples/AspireWithJavaScript/AspireJavaScript.Angular/node_modules/rxjs/dist/cjs/internal/util/reportUnhandledError.js:13

        throw err;

        ^

RangeError [ERR_SOCKET_BAD_PORT]: options.port should be >= 0 and < 65536. Received type number (NaN).

at validatePort (node:internal/validators:424:11)

at Server.listen (node:net:2000:5)

at /Users/owen/Developer/aspire-samples/samples/AspireWithJavaScript/AspireJavaScript.Angular/node_modules/@angular-devkit/build-angular/src/utils/check-port.js:55:14

at new Promise (<anonymous>)

at checkPort (/Users/owen/Developer/aspire-samples/samples/AspireWithJavaScript/AspireJavaScript.Angular/node_modules/@angular-devkit/build-angular/src/utils/check-port.js:27:12)

at initialize (/Users/owen/Developer/aspire-samples/samples/AspireWithJavaScript/AspireJavaScript.Angular/node_modules/@angular-devkit/build-angular/src/builders/dev-server/builder.js:112:63) {

code: 'ERR_SOCKET_BAD_PORT'

}

Node.js v21.1.0

Set up: Aspire preview 2 workload installed. Running through dotnet run in AppHost proj MacOs: 14.2.1 Node: v21.1.0 npm ng --version: 10.2.3

Sorry if I am being too early on this

Ah, I think that might be an issue with Windows. I passed the port to the ng serve --port switch using %PORT%, if you're not on Windows that won't work... Try changing that to ng serve --port $PORT instead.

That, fixed it! Thanks!

Note to self, update this to use an .env with DEV_SERVER_PORT=PORT or change PORT to DEV_SERVER_PORT.

LGTM. I do wonder whether we could make these projects show up in VS via the esproj project types.

Yeah, that would be really cool!


RCS message does NOT show properly cannot be replied to

staring from iOS 18 MacOS 15.0, iMessage platform now split RCS message out from text message type.
At the moment, RCS can only be seen from BlueBubbles but does NOT recognised properly nor can be replied to.
Kindly fix please?

staring from iOS 18 MacOS 15.0, iMessage platform now split RCS message out from text message type. At the moment, RCS can only be seen from BlueBubbles but does NOT recognised properly nor can be replied to.
Kindly fix please?

We'll have to explore Sequoia some more. Neither of us currently have it installed


fix: verify than token address is not empty inside ethTokens

Currently, the CI crash because there is a token with an empty address on coingecko
How to test?
Verify that the CI is ✅

hey @NeOMakinG can you please just make this change in your blacklist PR?


Make standard deviations available via LogReport

The standard deviations can be computed by DictSummary.make_statistics() but it is not available in LogReport yet. This patch computes and provides them via log.
This fix is suggested by @ktns.
Output example:
[
    {
        "main/loss": 0.18992580644320697,
        "main/loss.std": 0.19250075864959096,
        "main/accuracy": 0.9421666692197322,
        "main/accuracy.std": 0.05971576333672585,
        "validation/main/loss": 0.08225996830442454,
        "validation/main/loss.std": 0.0,
        "validation/main/accuracy": 0.9739000046253204,
        "validation/main/accuracy.std": 0.0,
        "epoch": 1,
        "iteration": 600,
        "elapsed_time": 26.420904844999313
    },
    {
        "main/loss": 0.0745212121346655,
        "main/loss.std": 0.045783321075002194,
        "main/accuracy": 0.9770833432674408,
        "main/accuracy.std": 0.015275029362245526,
        "validation/main/loss": 0.08226964094443247,
        "validation/main/loss.std": 0.0,
        "validation/main/accuracy": 0.9755000072717667,
        "validation/main/accuracy.std": 0.0,
        "epoch": 2,
        "iteration": 1200,
        "elapsed_time": 54.302615933993366
    },
    ...
]

This change basically LGTM but did you have the time to do some benchmarking?

Benchmark
Summary
make_statistics() is 2-3 times slower than compute_mean(), but I think it has actually no impact on the performance (about 0.015 ms/epoch).

Thanks for the report. I think it's okay in the most general cases too but it assumes that trigger passed to log report is quite rarely fired, e.g. per epoch and not per iterations. I'm not sure it's a good idea to change the default behavior and that it'd be better to add an additional argument to log report to control whether std should be computed or not. What do you think?

Jenkins CI test (for commit ab7750181950f3278047587ca6b60b158138ef6d, target branch master) failed with status FAILURE.

Hmm. I think this is a fully backward-compatible way because this patch simply adds keys suffixed with ".std" for each corresponding metrics. I'm afraid the filtering feature increases complexity of the API. What kind of cases do you think the users will be confused?

I'm thinking of users firing this log report in e.g. each iteration. Wouldn't the performance impact be quite noticeable to them? I do agree of course agree that the API complexity shouldn't be increased unless necessary.

It makes sense. It's just an idea, but how about making _init_summary injectable through the constructor like the following?
An advantage is that the users can replace it with more powerful implementation such like supporting quantiles.
class LogReport(extension.Extension):
    def __init__(self, keys=None, trigger=(1, 'epoch'), postprocess=None,
                 filename=None, init_summary=reporter.DictSummary, **kwargs):
        # init_summary is a callable object (like constructor, function or lambda)
        self._init_summary = init_summary
        ...

    def __call__(self, trainer):
        ...
        if self._trigger(trainer):
            # output the result
            stats = self._summary.make_statistics()
            ...
            # reset the summary for the next output
            self._init_summary()

If we go with it, need to discuss the followings:

Capability of the default DictSummary (just compute mean?)
A name of the default DictSummary (like MeanDictSummary and StdDictSummary)
The spec of make_statistics()

It states "returns a single dictionary that holds mean and standard deviation values..." in the document
Add another method and deprecate the old one?

What do you think?

That sounds like a future proof approach. A concern might be that we'll have to define a new class for each combination of statistics, once they get many, if they'll ever do.

@hvy I'm very sorry for the late response. I tried to add and use new Stats and StatsDict (the name is borrowed from Guava library) in place of Summary. Could you check and give a comment for these new APIs? Thanks.


[Bug] Not able to use clickhouse profile

Astronomer Cosmos Version
main (development)
If "Other Astronomer Cosmos version" selected, which one?
1.7.0
dbt-core version
1.8.7
Versions of dbt adapters
dbt-adapters==1.7.0
dbt-clickhouse==1.8.2
dbt-common==1.10.0
dbt-core==1.8.7
dbt-extractor==0.5.1
dbt-semantic-interfaces==0.5.1
LoadMode
AUTOMATIC
ExecutionMode
DOCKER
InvocationMode
None
airflow version
2.9.2
Operating System
MACOS
If a you think it's an UI issue, what browsers are you seeing the problem on?
No response
Deployment
Docker-Compose
Deployment details
No response
What happened?
Hello,
When I try to connect to clickhouse with astronomer, I've got the error : DB::Exception: default: Authentication failed: password is incorrect, or there is no user with such name.
If I go to :
cosmos/profiles/clickhouse/user_pass.py
I can see :
    required_fields = [
        "host",
        "login",
        "schema",
        "clickhouse",
    ]
    secret_fields = [
        "password",
    ]
    airflow_param_mapping = {
        "host": "host",
        "login": "login",
        "password": "password",
        "port": "port",
        "schema": "schema",
        "clickhouse": "extra.clickhouse",
    }

Shouldn't it be with user instead ?
    required_fields = [
        "host",
        "user",
        "password",
        "dbname",
    ]
    secret_fields = [
        "password",
    ]
    airflow_param_mapping = {
        "host": "host",
        "user": "login",
        "password": "password",
        "port": "port",
        "dbname": "schema",
        "keepalives_idle": "extra.keepalives_idle",
        "sslmode": "extra.sslmode",
    }

My working configuration on DBT is :
project:
  outputs:
    stage:
      type: clickhouse
      user: "{{ env_var('DBT_USER') }}"
      password: "{{ env_var('DBT_PASSWORD') }}"
      #optional fields
      driver: native
      port: 9440
      host: myhost
      secure: true
      cluster: "all-replicated"
  target: stage

My unworking configuration on airflow :

profile_config = ProfileConfig(
    profile_name="project",
    target_name="stage",
    profile_mapping=ClickhouseUserPasswordProfileMapping(
        conn_id=f"clickhouse_{CONN_ENV}",
        profile_args={
            "cluster": "all-replicated",
            "secure": True,
        },
    ),
)

Thanks a lot,
Relevant log output
No response
How to reproduce
Launching airflow on docker and try to connect to clickhouse
Anything else :)?
No response
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Contact Details

Hi,
Update on my issue.
It's working with the user instead of login :
"""Maps Airflow Postgres connections using user + password authentication to dbt profiles."""

from __future__ import annotations

from typing import Any

from ..base import BaseProfileMapping

class ClickhouseUserPasswordProfileMapping(BaseProfileMapping):
    """
    Maps Airflow generic connections using user + password authentication to dbt Clickhouse profiles.
    https://docs.getdbt.com/docs/core/connect-data-platform/clickhouse-setup
    """

    airflow_connection_type: str = "generic"
    dbt_profile_type: str = "clickhouse"
    default_port = 9000
    is_community = True

    required_fields = [
        "host",
        "user",
        "schema",
        "clickhouse",
    ]
    secret_fields = [
        "password",
    ]
    airflow_param_mapping = {
        "host": "host",
        "user": "login",
        "password": "password",
        "port": "port",
        "schema": "schema",
        "clickhouse": "extra.clickhouse",
    }

    def _set_default_param(self, profile_dict: dict[str, Any]) -> dict[str, Any]:
        if not profile_dict.get("driver"):
            profile_dict["driver"] = "native"

        if not profile_dict.get("port"):
            profile_dict["port"] = self.default_port

        if not profile_dict.get("secure"):
            profile_dict["secure"] = False
        return profile_dict

    @property
    def profile(self) -> dict[str, Any | None]:
        """Gets profile. The password is stored in an environment variable."""
        profile_dict = {
            **self.mapped_params,
            **self.profile_args,
            # password should always get set as env var
            "password": self.get_env_var_format("password"),
        }

        return self.filter_null(self._set_default_param(profile_dict))

    @property
    def mock_profile(self) -> dict[str, Any | None]:
        """Gets mock profile."""

        profile_dict = {
            **super().mock_profile,
        }

        return self._set_default_param(profile_dict)

I'm open to create a PR on this.


[bug] copy address success in local dev but. failed when deployed to server

Link to minimal reproducible example
diffucult to reproduce
Summary
In local dev, when i copy the address from modal, it success.
But when I deploy to server, the copy failed with:
Failed to copy

Then, I found this link:  https://stackoverflow.com/questions/69438702/why-does-navigator-clipboard-writetext-not-copy-text-to-clipboard-if-it-is-pro
they say:
Because the navigator.clipboard.writeText method returns a promise and your code does not wait for its result.

so, I think the copy code should be
navigator.clipboard.writeText(val).then(....)

List of related npm package versions
3.1.0

Can you copy it on https://web3modal.com ?

If yes, then do you have a preview of your deployment where this issue happens?
If no, can you provide browser / device details?

Awaiting this is not necessary

Hi, I am having a similar issue, but this is persistent on all mobile devices using Sveltekit.
On desktop, no problem, on mobile "failed to copy" when hitting the copy button to the right of "scan the code"

same, local dev success, product failed

"@wagmi/core": "^1.4.12",
"@web3modal/wagmi": "^3.5.2",
"use-wagmi": "^0.4.9",
"viem": "^1.21.0",
my version


Automated testing

Adds automated tests and bumps the plugin to the latest version to get more consistent caching, and to reduce the size of CI builds (so we don't bump against the GitHub limit of around 12~14 GB).
The changes to the plugin should be merged first.
https://github.com/Islandora-Devops/isle-gradle-docker-plugin/pull/8

This changeset is huge.  I'll try and smoketest this and see if the tests pass, etc...
Also, I see you're getting rid of some more execline :fire: :smiling_face_with_three_hearts:

Things are building fine but I'm getting errors when running ./gradlew test.  Looks like I need to login to docker first?

Ya you’ll need to login to docker. Wise it downloads etcd to run one of the
tests
On Thu, 1 Apr 2021 at 21:02, dannylamb @.***> wrote:

Things are building fine but I'm getting errors when running ./gradlew
test. Looks like I need to login to docker first?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Islandora-Devops/isle-buildkit/pull/128#issuecomment-812139845,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AADW7TMG4SD6UMJGCAOYHCTTGTGNZANCNFSM4Z2KDWMA
.

--
Nigel Banks
@.***

Okay, works fine once you log in.
I've dug through this enough to grasp the general pattern of docker-compose files exec'ing the test commands.
This is a good start for basic sanity checks like making sure the defaults work. And this doesn't appear to affect make demo or make local, so let's pull this in :+1:


[Bug] Error code 100 during install

I am running a HomeAssistant VM through virtualbox.
I see an Error code 100 during add-on install. The dialog message I see is:
The command '/bin/bash -o pipefail -c apt-get update && apt-get install -y --no-install-recommends ca-certificates=20230311ubuntu<IP_ADDRESS> curl=7.81.0-1ubuntu1.16 jq=1.6-2.1ubuntu3 tzdata=2024a-0ubuntu0.22.04 xz-utils=5.2.5-2ubuntu1 && S6_ARCH="${BUILD_ARCH}" && if [ "${BUILD_ARCH}" = "i386" ]; then S6_ARCH="i686"; elif [ "${BUILD_ARCH}" = "amd64" ]; then S6_ARCH="x86_64"; elif [ "${BUILD_ARCH}" = "armv7" ]; then S6_ARCH="arm"; fi && curl -L -s "https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-noarch.tar.xz" | tar -C / -Jxpf - && curl -L -s "https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-${S6_ARCH}.tar.xz" | tar -C / -Jxpf - && curl -L -s "https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-symlinks-noarch.tar.xz" | tar -C / -Jxpf - && curl -L -s "https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-symlinks-arch.tar.xz" | tar -C / -Jxpf - && mkdir -p /etc/fix-attrs.d && mkdir -p /etc/services.d && curl -J -L -o /tmp/bashio.tar.gz "https://github.com/hassio-addons/bashio/archive/${BASHIO_VERSION}.tar.gz" && mkdir /tmp/bashio && tar zxvf /tmp/bashio.tar.gz --strip 1 -C /tmp/bashio && mv /tmp/bashio/lib /usr/lib/bashio && ln -s /usr/lib/bashio/bashio /usr/bin/bashio && curl -L -s -o /usr/bin/tempio "https://github.com/home-assistant/tempio/releases/download/${TEMPIO_VERSION}/tempio_${BUILD_ARCH}" && chmod a+x /usr/bin/tempio && apt-get purge -y --auto-remove xz-utils && apt-get clean && rm -fr /tmp/* /var/{cache,log}/* /var/lib/apt/lists/*' returned a non-zero code: 100

Try the latest version 0.0.12 and let me know if you have any more issues.

Thanks for the quick fix.
Installing 0.0.12, and I see the spinning install logo the past 5 minutes. Does it typically take this long to finish?

It does take some time to download the Ubuntu docker image, update it,
download the s6-overlay, and apply that. Then it installs various Python
dependencies, Playwright, and Playwright Firefox build.  You can monitor
the build process by going to HA Settings, System, Logs, then select
Supervisor from the upper right pull down.  Click the refresh button to
reload the logs.
On Thu, Aug 15, 2024 at 10:21 AM c3p0vsr2d2 @.***>
wrote:

Thanks for the quick fix.
Installing 0.0.12, and I see the spinning install logo the past 5 minutes.
Does it typically take this long to finish?
—
Reply to this email directly, view it on GitHub
https://github.com/thor0215/hassio-xfinity-usage/issues/17#issuecomment-2291511042,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AL7JXGODQS3SPLRJ4YGWV63ZRTBQLAVCNFSM6AAAAABMQSOYUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOJRGUYTCMBUGI
.
You are receiving this because you modified the open/close state.Message
ID: @.***>

thanks @thor0215 - Install finally finished without issues. I am facing some MQTT hiccups, which I will work on sorting out. Please close this issue, and thanks again for the super quick turnaround.


@DynamoDBVersionAttribute with dynamodb-enhanced not working in native mode

First of all, thanks for your work on Quarkus/Quarkiverse. We really enjoy working with Quarkus. :)
We are using Quarkus version 2.14.3.Final and quarkus-amazon-services version 1.3.1
To access DynamoDb we use the quarkus-amazon-dynamodb-enhanced extension.
Now we want to introduce opmtistic locking for an entity. https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBMapper.OptimisticLocking.html
We added a field with the annotation @DynamoDBVersionAttribute which is provided by the aws SDK for this purpose.
Non-native it works as expected (version is set to 1 on insertion, then incremented with each update). Native a ConditionalCheckFailedException is thrown while inserting a new entity.
Based on https://github.com/quarkusio/quarkus-quickstarts/tree/main/amazon-dynamodb-quickstart and https://quarkiverse.github.io/quarkiverse-docs/quarkus-amazon-services/dev/amazon-dynamodb.html#_dynamodb_enhanced_client I prepared a sample project to reproduce the issue.
You can find the sample project at https://github.com/dagrammy/DynamoDbVersionAttribute-test.
./mvnw install -> BUILD SUCCESS
./mvnw install -Dnative -> BUILD FAILURE
Complete log: https://github.com/dagrammy/DynamoDbVersionAttribute-test/blob/main/README.md
 HTTP Request to /fruits failed, error id: 641e581d-abc1-4585-8ddb-bec115b262b3-1: software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException: The conditional request failed (Service: DynamoDb, Status Code: 400, Request ID: ODARM82TCOGF31YLKRI9QIU57OGW2SNPZ1DJTCVW9PFX3NY3D74C)
	at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:125)
	at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:82)
	at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:60)
	at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:41)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36)

Is there a way to use the opmtistic locking feature in native builds?
Thanks in advance!

Could you try adding @RegisterForReflection(targets = VersionRecordAttributeTags.class) to any of your application class and see if it fixes the issue?
If so, we will register it for reflection automatically.

Ah, you were kind enough to provide a reproducer.
I couldn't make it to work with the following patch:
diff --git a/dynamodb-enhanced/deployment/src/main/java/io/quarkus/amazon/dynamodb/enhanced/deployment/DynamodbEnhancedProcessor.java b/dynamodb-enhanced/deployment/src/main/java/io/quarkus/amazon/dynamodb/enhanced/deployment/DynamodbEnhancedProcessor.java
index a808242..6726f3d 100644
--- a/dynamodb-enhanced/deployment/src/main/java/io/quarkus/amazon/dynamodb/enhanced/deployment/DynamodbEnhancedProcessor.java
+++ b/dynamodb-enhanced/deployment/src/main/java/io/quarkus/amazon/dynamodb/enhanced/deployment/DynamodbEnhancedProcessor.java
@@ -24,6 +24,8 @@ import io.quarkus.deployment.builditem.nativeimage.ReflectiveClassBuildItem;
 import io.quarkus.deployment.pkg.steps.NativeBuild;
 import io.quarkus.gizmo.Gizmo;
 import software.amazon.awssdk.enhanced.dynamodb.DefaultAttributeConverterProvider;
+import software.amazon.awssdk.enhanced.dynamodb.internal.extensions.AutoGeneratedTimestampRecordAttributeTags;
+import software.amazon.awssdk.enhanced.dynamodb.internal.extensions.VersionRecordAttributeTags;
 import software.amazon.awssdk.enhanced.dynamodb.internal.mapper.BeanTableSchemaAttributeTags;
 import software.amazon.awssdk.enhanced.dynamodb.mapper.BeanTableSchema;
 import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.DynamoDbBean;
@@ -52,7 +54,7 @@ public class DynamodbEnhancedProcessor {
                 .getAnnotations(DotName.createSimple(DynamoDbBean.class.getName()))) {
             ClassInfo classInfo = i.target().asClass();
             reflectiveClass.produce(
-                    new ReflectiveClassBuildItem(true, false, classInfo.name().toString()));
+                    new ReflectiveClassBuildItem(true, true, classInfo.name().toString()));
         }

         // Register classes which are used by BeanTableSchema but are not found by the classloader
@@ -60,6 +62,10 @@ public class DynamodbEnhancedProcessor {
                 new ReflectiveClassBuildItem(true, false, DefaultAttributeConverterProvider.class.getName()));
         reflectiveClass.produce(
                 new ReflectiveClassBuildItem(true, false, BeanTableSchemaAttributeTags.class.getName()));
+        reflectiveClass.produce(
+                new ReflectiveClassBuildItem(true, false, AutoGeneratedTimestampRecordAttributeTags.class.getName()));
+        reflectiveClass.produce(
+                new ReflectiveClassBuildItem(true, false, VersionRecordAttributeTags.class.getName()));

     }

Not exactly sure what's going on but my guess is that it will require adding some debug logging to the AWS library to understand what's going wrong in native.
@hamburml @Nithanim would you be interested in having a closer look?

Could you try adding @RegisterForReflection(targets = VersionRecordAttributeTags.class) to any of your application class and see if it fixes the issue?
If so, we will register it for reflection automatically.

Yes, we already registered some classes for reflection in the sample project.  But unfortunately without any success.
https://github.com/dagrammy/DynamoDbVersionAttribute-test/blob/main/src/main/java/org/acme/dynamodb/ReflectionConfig.java
@RegisterForReflection(targets = {
    software.amazon.awssdk.enhanced.dynamodb.extensions.AtomicCounterExtension.class,
    software.amazon.awssdk.enhanced.dynamodb.extensions.WriteModification.class,
    software.amazon.awssdk.enhanced.dynamodb.extensions.VersionedRecordExtension.class,
    software.amazon.awssdk.enhanced.dynamodb.extensions.ReadModification.class,
    software.amazon.awssdk.enhanced.dynamodb.extensions.AutoGeneratedTimestampRecordExtension.class,
    software.amazon.awssdk.enhanced.dynamodb.extensions.annotations.DynamoDbVersionAttribute.class,
    software.amazon.awssdk.enhanced.dynamodb.extensions.annotations.DynamoDbAtomicCounter.class,
    software.amazon.awssdk.enhanced.dynamodb.extensions.annotations.DynamoDbAutoGeneratedTimestampAttribute.class,
    software.amazon.awssdk.enhanced.dynamodb.internal.extensions.VersionRecordAttributeTags.class,
    software.amazon.awssdk.enhanced.dynamodb.internal.extensions.DefaultDynamoDbExtensionContext.class,
    software.amazon.awssdk.enhanced.dynamodb.internal.extensions.ChainExtension.class,
    software.amazon.awssdk.enhanced.dynamodb.internal.extensions.AtomicCounterTag.class,
    software.amazon.awssdk.enhanced.dynamodb.internal.extensions.AutoGeneratedTimestampRecordAttributeTags.class
}, ignoreNested = false)

@hamburml @Nithanim would you be interested in having a closer look?

I do not have the time right now. I'll try it at the weekend.

[...] @Nithanim would you be interested in having a closer look?

I'll try next weekend.
But sadly no promises since I completely switched to Azure which uses up all my brain-power. I envy you guys being in AWS :(

So... I lied. I should have gone to bed but went down the rabbit hole instead. Nothing precise yet but I have a long ramble about what I checked and found out; including the path I took.
If you do not want to dig into the problem yourself, the following text is useless to you!

For anyone who wants to dig into stuff like this:
(The text is not proof-read and it is about 01:00 in the morning...)
Normally, an app that has problems with native-image crashes at a very specific point which most likely is the area you have to apply a patch. An example would be the core fix for ddb-enhanced that (if I remember correctly) works around the not implemented feature of runtime-defined lambdas. This crash is at compile time when trying to create this lambda. Another example would be a successful compile but crasing at runtime while accessing stuff via reflection. This is also very obvious because you have the direct point of crashing.
The current issue here is a special one. It compiles and it runs in native-mode, but it just does not work. I personally haven't seen that. The server does not like our request. In a simple world, we would have the raw HTTP communication. There might be ways, but I did not want to screw around to get that so I jumped into the code instead.
The obvious problem is that we cannot just debug the problem since in JVM it works fine. Therefore we have to search indirectly for the problem. It is exremely helpful to have a minimal, working example with tests (thank you!). Also, we know that it is a problem with @DynamoDbVersionAttribute so we use our IDE and jump there. It's just an annotation, so we let it list all usages. We quickly come to @VersionedRecordExtension.
After a quick scrolling though, we find that there is not that much code in there. But both the modifyMetadata and beforeWrite sound very helpful. I don't know (yet) what modifyMetadata does, but beforeWrite sound promising to be responsible for changes to the request to the remote server.
Since out main target with native image is reflective stuff, I made a quick check for that but nothing directly obvious in there. I have no idea what it does and the javadoc is helpful but I am tired, I just set a breakpoint and let the test run in debug mode. Examining the callstack from our service slowly to the current frame, I get some insight where it gets through. I also stepped though at some point but no obvious native-image-braking stuff.
Randomly, I came across software.amazon.awssdk.enhanced.dynamodb.internal.operations.CommonOperation#execute. For most people that would be uninteresting but this is a very lucky find because we have an excellent breakpoint candidate where we have the request and the response directly there. No need for scavanging around for that!
Our main goal is still to find out why the native-image does not work and it must be in the request. So I quickly tested the request#toString() and it makes a nice print:
PutItemRequest(TableName=QuarkusFruits, Item={name=AttributeValue(S=Cherry), description=AttributeValue(S=description-Cherry), version=AttributeValue(N=1)}, ConditionExpression=attribute_not_exists(#AMZN_MAPPED_version), ExpressionAttributeNames={#AMZN_MAPPED_version=version})

Very helpful!
Problem though: how to let it print in native-image since it is compiled in the code. You could program a detour liek other fixes but that would be too much work. Other way: replace the code. How? We get the version of the sdk from the maven deps (2.17.291), clone the aws sdk, checkout this specific version tag, add like System.out.println(request.toString()); and mvn install.
Simple? Wrong: Checkstyle and spotbugs prevent but after a quick google: -Dspotbugs.skip=true -DskipTests=true. Adn now we have a modified veersion in our local maven repo. Everything locally executed (and built) now uses our modifed dependency.
We let it test both JVM and native: mvn verify -Pnative.
In JVM we get:
PutItemRequest(TableName=QuarkusFruits, Item={name=AttributeValue(S=Cherry), description=AttributeValue(S=description-Cherry), version=AttributeValue(N=1)}, ConditionExpression=attribute_not_exists(#AMZN_MAPPED_version), ExpressionAttributeNames={#AMZN_MAPPED_version=version})
PutItemResponse()
PutItemRequest(TableName=QuarkusFruits, Item={name=AttributeValue(S=Pear), description=AttributeValue(S=description-Pear), version=AttributeValue(N=1)}, ConditionExpression=attribute_not_exists(#AMZN_MAPPED_version), ExpressionAttributeNames={#AMZN_MAPPED_version=version})
PutItemResponse()
GetItemRequest(TableName=QuarkusFruits, Key={name=AttributeValue(S=Cherry)})
GetItemResponse(Item={name=AttributeValue(S=Cherry), description=AttributeValue(S=description-Cherry), version=AttributeValue(N=1)})
GetItemRequest(TableName=QuarkusFruits, Key={name=AttributeValue(S=Pear)})
GetItemResponse(Item={name=AttributeValue(S=Pear), description=AttributeValue(S=description-Pear), version=AttributeValue(N=1)})

and for native image:
PutItemRequest(TableName=QuarkusFruits, Item={name=AttributeValue(S=Cherry), description=AttributeValue(S=description-Cherry), version=AttributeValue(N=1)}, ConditionExpression=#AMZN_MAPPED_version = :old_version_value, ExpressionAttributeNames={#AMZN_MAPPED_version=version}, ExpressionAttributeValues={:old_version_value=AttributeValue(N=0)})
2022-12-18 23:48:39,055 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-0) HTTP Request to /fruits failed, error id: 365d4c2b-31af-44f1-a314-4f78f66c277d-1: software.amazon.awssdk.services.dynamodb.model.ConditionalCheckFailedException: The conditional request failed (Service: DynamoDb, Status Code: 400, Request ID: O6S3KIR9QT5O70Y7Z4BBYYHKSCZ1NCBFF6VABTS872VCY2MEQ4E1)

This should be identical, but it is not!
Jumping back to the beforeWrite, we have a pretty big if and also the expressions are built there. See that attribute_not_exists? This should be the condition but is is the one from the else branch.
Intermediate conclusion: The condition of the if evaluates wrong. Why? Who knows. Time to add a print there to confirm for sure and then to find out why it is wrong.
Thanks for listening to my TED-talk!
Maybe I am completely wrong or maybe not but good luck! That's all from me for today.

Thanks @Nithanim  for your work and your TED talk. No matter how interesting but annoying I consider the problem, your TED Talk made me laugh. 😄
I also went down this rabbit hole on Friday, but not as deep as you.
After the terrific groundwork, I dove even deeper into the hole and tried to find the problem....
Like you wrote, the problem arises with the if in VersionedRecordExtension.
if (!existingVersionValue.isPresent() || isNullAttributeValue(existingVersionValue.get())) {
https://github.com/aws/aws-sdk-java-v2/blob/master/services-custom/dynamodb-enhanced/src/main/java/software/amazon/awssdk/enhanced/dynamodb/extensions/VersionedRecordExtension.java#L112
So whats the value of existingVersionValue?
JVM:
empty Optional
Native:
AttributeValue(N=0)
Now if you go further up the chain, you will eventually get to the line Map<String, AttributeValue> itemMap = tableSchema.itemToMap(item, alwaysIgnoreNulls); in the PutItemOperation class.
If you output the values before and after the row:
JVM:
Fruit{name='Cherry', description='description-Cherry', version=null}
{name=AttributeValue(S=Cherry), description=AttributeValue(S=description-Cherry)}

Native:
Fruit{name='Cherry', description='description-Cherry', version=null}
{name=AttributeValue(S=Cherry), description=AttributeValue(S=description-Cherry), version=AttributeValue(N=0)}

Time to dig even deeper!
The itemToMap method is implemented in StaticImmutableTableSchema
https://github.com/aws/aws-sdk-java-v2/blob/master/services-custom/dynamodb-enhanced/src/main/java/software/amazon/awssdk/enhanced/dynamodb/mapper/StaticImmutableTableSchema.java#L502
Again, the output of the following statement:
AttributeValue attributeValue = attributeMapper.attributeGetterMethod().apply(item);
JVM:
AttributeValue(NUL=true)
Native:
AttributeValue(N=0)
Time for some "serious" work. 😉  To be continued....

It's late, I'm tired but still in the rabbit hole....and somehow arrived back at the start. 😉
https://github.com/quarkiverse/quarkus-amazon-services/blob/main/dynamodb-enhanced/deployment/src/main/java/io/quarkus/amazon/dynamodb/enhanced/deployment/DynamodbEnhancedProcessor.java#L102
                    } else if (name.equals("getterForProperty")) {
                        return new ReplaceMethodBody(
                                originalMethodVisitor,
                                getMaxLocals(descriptor),
                                visitor -> {
                                    visitor.visitCode();
                                    visitor.visitVarInsn(Opcodes.ALOAD, 0);
                                    visitor.visitVarInsn(Opcodes.ALOAD, 1);
                                    Type type = Type.getType(descriptor);
                                    visitor.visitMethodInsn(
                                            Opcodes.INVOKESTATIC, TARGET_METHOD_OWNER, name, type.getDescriptor(), false);
                                    visitor.visitInsn(Opcodes.ARETURN);
                                });
.....

If I understand it correctly (and honestly I only partially do right now) at this point methods are replaced for native builds which are generated in the AWS SDK during runtime.
Is it possible that there may be issues with autoboxing/conversion between Long and long?
Does it possibly explain that Long => null becomes long => 0.

I am sorry, I still haven't had any time :(

If I understand it correctly (and honestly I only partially do right now) at this point methods are replaced for native builds which are generated in the AWS SDK during runtime.

It is always replaced, not only for native builds.

Thanks for the clarification. I missed that one and made a wrong assumption.

Does it possibly nonetheless explain that Long => null becomes long => 0?

Hm... I do not think that graalvm converts object-types to their primitive counterparts. If you like you could try to check the parameters given to the method here (https://github.com/quarkiverse/quarkus-amazon-services/blob/main/dynamodb-enhanced/deployment/src/main/java/io/quarkus/amazon/dynamodb/enhanced/deployment/DynamodbEnhancedProcessor.java#L97). You should get the arguments via Type.getMethodType(...). See here on stackoverflow. https://stackoverflow.com/questions/67952646/how-to-get-count-and-types-of-method-parameters-using-org-objectweb-asm-methodvi
I hope this helps.

I also can not explain why Graal should make such a transformation at this point.
Thank you also for the hints where I can continue next. I hope that tomorrow I will be able to take a closer look at the mentioned places.

Unfortunately I didn't have much time today. But I have included debug statements in BeanTableSchemaSubstitutionImplementation / GetterWrapper#apply and the entity.
try {
  R retVal = (R) mh.invoke(t);
  System.out.println("GetterWrapper returns: " + retVal);
  if (retVal != null) {
      System.out.println("GetterWrapper returns instance of: " + retVal.getClass().getName());
  }
  return (R) mh.invoke(t);
  ....

and
  @DynamoDbVersionAttribute
  public Long getVersion() {
      System.out.println("getVersion() called and returns: " + version);
      return version;
  }

The result:
JVM:
getVersion() called and returns: null
GetterWrapper returns: null

Native:
getVersion() called and returns: null
GetterWrapper returns: 0
GetterWrapper returns instance of: java.lang.Long

Then I tried to figure out what happens with the parameters @ https://github.com/quarkiverse/quarkus-amazon-services/blob/main/dynamodb-enhanced/deployment/src/main/java/io/quarkus/amazon/dynamodb/enhanced/deployment/DynamodbEnhancedProcessor.java#L102
So I added some debug statements (thx to your SO link):
    Type methodType = Type.getMethodType(descriptor);
    int sizes = methodType.getArgumentsAndReturnSizes();
    System.out.println("return type: "
            + methodType.getReturnType().getClassName() + " (" + (sizes & 3) + ')');

    Type[] argTypes = methodType.getArgumentTypes();
    System.out.println(argTypes.length + " arguments (" + (sizes >> 2) + ')');
    for (int ix = 0; ix < argTypes.length; ix++) {
        System.out.println("arg" + ix + ": " + argTypes[ix].getClassName());
    }

Output:
return type: java.util.function.Function (1)
2 arguments (3)
arg0: java.beans.PropertyDescriptor
arg1: java.lang.Class

But these are just the return type & parameters (PropertyDescriptor propertyDescriptor, Class<T> beanClass) of BeanTableSchema#getterForProperty.
I'm at my wit's end (or...attention, very poor 1:1 translation from german: "I am at the end with my latin" 😉)
So I am stuck right now and don't know how to get even deeper.
I will nevertheless continue to try my best to track down the reason for this strange behavior.

It is very kind of you that you tracked it all the way back to the MethodHandles, thank you! It gets a lot easier when someone else does the work for you :stuck_out_tongue:  and you can trim everything away and get a smaller example project: https://github.com/Nithanim/quarkus-native-bug-test
So for some weird reason, the MethodHandle for Long is broken. The problem is that there is not a whole lot of things I can think of trying right now. It is pretty much the border to the arcane magic of native image.
Questions:

Only Long specifically, or other primitive wrappers too?
Does Quarkus do some magic? => Even more minimal project without Quarkus; call native-image manually. It would be pretty weird to have such an obvious native-image bug.
wtf is going on?
Is there a difference between Mandrel and GraalVM? (Most likely not, but...)
Maybe there is something in the issue tracker, like https://github.com/oracle/graal/issues /5209 *. Seems related at least.
...

* I don't want github to cross-link the issues (yet).
Sorry, that I don't have a lot of value to add yet.

I'm at my wit's end (or...attention, very poor 1:1 translation from german: "I am at the end with my latin" wink)

At work I am doing something with "underwriting"s and "bookholding". Hah! :smile:

It is very kind of you that you tracked it all the way back to the MethodHandles, thank you! It gets a lot easier when someone else does the work for you 😛 and you can trim everything away and get a smaller example project: https://github.com/Nithanim/quarkus-native-bug-test

You are welcome! 😊 Well, it's a team effort! 😉 So thank you for example project!

So for some weird reason, the MethodHandle for Long is broken. The problem is that there is not a whole lot of things I can think of trying right now. It is pretty much the border to the arcane magic of native image.

I took your example project and stripped it down even further (no Quarkus, no maven....just simple javac, java and native-image). https://github.com/dagrammy/graal-native-bug-test

Questions:

Only Long specifically, or other primitive wrappers too?

It's reproducible using Integer and Boolean too.

Does Quarkus do some magic? => Even more minimal project without Quarkus; call native-image manually. It would be pretty weird to have such an obvious native-image bug.

I don't think so...

wtf is going on?

This is the right question of all questions. 😆

Is there a difference between Mandrel and GraalVM? (Most likely not, but...)

Maybe there is something in the issue tracker, like https://github.com/oracle/graal/issues /5209 *. Seems related at least.

...

I don't want github to cross-link the issues (yet).

I also did some research in the issue tracker, but was not able to find sth. So I decided to open an issue using the sample project as reproducer.
https://github.com/oracle/graal/issues/ 5672
(also messed up the link intentionally)

Sorry, that I don't have a lot of value to add yet.

I'm at my wit's end (or...attention, very poor 1:1 translation from german: "I am at the end with my latin" wink)

At work I am doing something with "underwriting"s and "bookholding". Hah! 😄

To quote a colleague. "I should have learned something serious. Carpenter or something like that..."
With this in mind, I wish everyone happy holidays.

I took your example project and stripped it down even further (no Quarkus, no maven....just simple javac, java and native-image). https://github.com/dagrammy/graal-native-bug-test

You really took "minimal" to the next level, even stripping out poor maven :cry: Really nice, thank you!

It's reproducible using Integer and Boolean too.
This is the right question of all questions. laughing

This is so weird. I am really eager to find out what is going on.

I also did some research in the issue tracker, but was not able to find sth. So I decided to open an issue using the sample project as reproducer.
https://github.com/oracle/graal/issues/ 5672 (also messed up the link intentionally)

Perfect! You are doing all the complicated stuff :smile:
You could have let Github link the issues because it is directly related but ¯\_(ツ)_/¯

To quote a colleague. "I should have learned something serious. Carpenter or something like that..."

With this in mind, I wish everyone happy holidays.

Happy holidays to you too!

You really took "minimal" to the next level, even stripping out poor maven 😢 Really nice, thank you!

I also couldn't remember the last time I typed javac into a terminal. 😄

This is so weird. I am really eager to find out what is going on.

Me too...me too!!

Perfect! You are doing all the complicated stuff 😄 You could have let Github link the issues because it is directly related but ¯_(ツ)_/¯

Fixed the link so the issue should be linked (if that works for an edit).

And et voilà...it works! https://github.com/dagrammy/graal-native-bug-test/blob/extended/MethodHandleTest.java#L31
Method getValue = beanClass.getDeclaredMethod("getValue");
Long v = (Long) getValue.invoke(bean);
System.out.println(">>>>> " + v + " <<<<<");

returns null for native and JVM!
I also added some simple test for the setter MethodHandles and they seem to work with native. https://github.com/dagrammy/graal-native-bug-test/blob/extended/SetterTest.java
So I came up with the following PR:
https://github.com/quarkiverse/quarkus-amazon-services/pull/443
I tested the dynamodb-enhanced Quarkus integration with my initial reproducer at https://github.com/dagrammy/DynamoDbVersionAttribute-test and it works (version attribute is set to 1 on insert and incremented afterwards)
What do you think of the PR respectively of the idea?

Welcome to 2023 :)
I hope the graalvm guys will soon reply to your issue. It is great that you found the problem. Ideally we would not need your workaround but instead get a new graalvm version.

What do you think of the PR respectively of the idea?

I would suggest adding some tests for the use of DynamoDBVersionAttribute. When we use a new graalvm version and set it back to MethodHandle, it should still work and with tests we can ensure that.

Welcome to 2023 :)

Happy new year too! 🥳

I hope the graalvm guys will soon reply to your issue. It is great that you found the problem. Ideally we would not need your workaround but instead get a new graalvm version.

I also hope that the graalvm team will address the issue soon. That would be the best solution, of course.

I would suggest adding some tests for the use of DynamoDBVersionAttribute. When we use a new graalvm version and set it back to MethodHandle, it should still work and with tests we can ensure that.

That's an excellent idea. To be honest, I hadn't even seen the tests for the enhanced client. 😉  But now I have discovered the integrationtests.
I updated the existing test classes and added new fields with annotations @DynamoDbVersionAttribute and @DynamoDbAutoGeneratedTimestampAttribute.  I also extended the test as @QuarkusIntegrationTest so the test is run in native mode. A new commit was pushed to the PR.
With the latest version the test fails, with the PR workaround the test succeeds.

1.5.0 has hit Central with the fix contributed by @dagrammy


Update Libraries

Also fixed settings for Unit Testing.
I replaced XCTest with swift-tap.

#59 is fixed by this.


resize not being set for editor in modal

I have instantiated Jodit editor but the set size is not shown when the editor shows in a modal window.
Jodit Version:  3.1.xxxxx
*var editor = new Jodit("#editor", {
"allowResizeY": false,
"showCharsCounter": false,
"showWordsCounter": false,
"toolbarButtonSize": "small",
"showXPathInStatusbar": false,
"askBeforePasteHTML": false,
"askBeforePasteFromWord": false,
"disablePlugins": "fullsize,mobile",
"height": 320,
"buttons": "|,bold,underline,italic,|,|,ul,ol,|,outdent,indent,|,,image,file,video,table,link,|,align,undo,redo"
});
Instead, when clicking in the editor, the editor then resizes.  Is there a method to update the editor to set the size?

Hi try send resize event
var editor = new Jodit('#editor')
after open modal
editor.events.fire('resize');

If I resize the modal to be smaller (after making it larger) the Jodit editor stays the same size and mangles the modal, how can I fix this and have it shrink also?


KullbackLeibler gradient

At the moment, there are two implementations of the gradient of the KullbackLeibler
Numba:
https://github.com/TomographicImaging/CIL/blob/701c4999c53e77c6ae3b587b1776f74accff2335/Wrappers/Python/cil/optimisation/functions/KullbackLeibler.py#L123-L130
and Numpy
https://github.com/TomographicImaging/CIL/blob/701c4999c53e77c6ae3b587b1776f74accff2335/Wrappers/Python/cil/optimisation/functions/KullbackLeibler.py#L344-L353
The numpy implementation above is not correct, see #1269 and will be corrected by PR #1272.
However, in the case where x and eta have zero values, numba implementation fails as
File ~/Edo/build/INSTALL/python/cil/optimisation/functions/KullbackLeibler.py:487, in KullbackLeibler_numba.gradient(self, x, out)
    485 else:
    486     out_np = out.as_array()
--> 487     kl_gradient(x.as_array(), self.b.as_array(), out_np, self.eta.as_array())
    488     out.fill(out_np)

ZeroDivisionError: division by zero

Reproduce: NEMA phantom + KullbackLeibler + FISTA.

So this is issue is now Handle ZeroDivisionError in KullbackLeibler gradient method?

There have been changes to the code so the above isn't up-to-date anymore. Current status:
Documentation
https://github.com/TomographicImaging/CIL/blob/2f92286b00381e703f4871e14b97cd87be7ea4ad/Wrappers/Python/cil/optimisation/functions/KullbackLeibler.py#L37-L44
(Note that the last line of this comment is superfluous, it's handled in the "ifs" above).
numpy implementation:
https://github.com/TomographicImaging/CIL/blob/2f92286b00381e703f4871e14b97cd87be7ea4ad/Wrappers/Python/cil/optimisation/functions/KullbackLeibler.py#L138-L143
Therefore, this will actually return 0 when u>0, v=0, which is not what was documented at the start, and strictly speaking incorrect for KL. (Note that the gradient is consistent with the above choice)
numba implementation
https://github.com/TomographicImaging/CIL/blob/2f92286b00381e703f4871e14b97cd87be7ea4ad/Wrappers/Python/cil/optimisation/functions/KullbackLeibler.py#L350-L360
Does return infinity.
conclusion
numby and numpy implementations are still inconsistent. The numpy implementation is a practical choice that avoids problems, and is also what STIR/SIRF does. I don't really like it, but it does happen for instance when the reconstructed image size is smaller than the FOV. The forward projections in the "outer" part of the sinogram will therefore be zero, while the measured data might not be. This would cause problems tracking KL improvement. (Not any actual problems in the reconstruction, as the backprojection of those LORs will never contribute to the image). Of course, in real life, the scatter/randoms estimate should not be zero, and therefore this situation shouldn't ever occur in the "proper" case, but people do crazy things...
I'd be fine with either choice (return inf or 0), but I would expect both implementations to give the same result, and to match the documentation.
Maybe @MargaretDuff has an opinion...


Change to define context menu order in DataView

The goal is to let the context menu in data views to have a consistent behaviour with the context menu in top-down & bottom-up views:
All actions that could be invoked from this view are visible, and unavailable (i.e., disabled) actions are greyed out.
We will change to let DataView define the menu order in GetMenuWithGrouping,
and let each individual views decide the visible actions and action visibilities in GetActionVisibilities.
The GetActionVisibilities method will return a map with the visible action name as key, and action availability as value.
Data view context menu - after the change: screen/7m8YadWCTsg8bMS
(Update: The "Add Iterator(s)" action is moved to be together with the "Jump To" actions.
Data view context menu - before the change: screen/kMgaChvZVnqDNdW
Top-down view context menu: screen/BPm2TRpsMqoF6u9
TODO:

Split this into three smaller PRs and this will be the last one.

Bug: http://b/205676296

Closed this one as we change the implementation to add DataView::GetActionStatus.
See https://github.com/google/orbit/pull/3387 and https://github.com/google/orbit/pull/3389 for details


Import .ttl file to large language model

Hi @gtfierro, I'm trying to let large language model (LLM) understand the ontology using llamaindex and RDF loader so user can query/ explore the Brick model using natural language:
Google Colab example:
https://colab.research.google.com/drive/16Ygf2IyGNkb725ZqtRmFQjwWBuzFX_kl?usp=sharing
RDF loader:
https://llamahub.ai/l/file-rdf
However I encounter some issue when I import the Brick.ttl and building ontology, the RDF loader only accept n-triples (.nt) format, so I need to convert it to nt format, if the file contain array like relationship such as:
bldg:floor_1 a brick:Floor ;
    brick:area [ brick:hasUnit unit:M2 ;
            brick:value 1e+03 ] ;
    brick:hasPart bldg:room_1 .

After the conversion using g.serialize, the object will become a unreadable string like:
<http://www.example.com/tlvl#TLVL-RF-CHWP-01.AUTO> <https://brickschema.org/schema/Brick#timeseries> _:nf16794e190c744ccb6be0cd71650e015b58 .
Is it possible to retain the array information after the conversion? Thanks.

Hi @s0301132 -- cool project! What you are looking at there is a blank node, not an array (see this page for some explanation, or the note here for an example). You might want to look at the Concise Bounded Description of a node in order to define subgraphs of related nodes/edges.
I've done some experiments with LLMs and Brick before and if you are looking to do RAG then you should really consider how to carefully partition the model up

Hi @gtfierro , thanks for the reply, I'm trying to import the building ontology to NebulaGraph with llm query and hoplly have a great result, closing this issue now.

Keep me updated! I'm curious how it works out


fix: use correct dirname for manifest

Description 📖
Uses path.dirname instead of path.basename so we correctly determine the manifestDir in the assets-manifest plugin.
Background 📜
This was happening because the package was using path.basename, which gives the file name and not the directory name.
The Fix 🔨
Change path.basename to path.dirname
Screenshots 📷
N/A

Thanks Joseph!


0.16.1 patch release: Add back CtxRef::begin_frame/end_frame

begin_frame, end_frame is more convenient when using egui in a game engine. In particular, 0.16.0 was incompatible with https://github.com/mvlabat/bevy_egui.

Published!


New HD geo with two separate active volumes

New HD geo with two separate active volumes (separated by central APA) to allow for background simulation emulating the HD lateral APA.
To be merged together with https://github.com/DUNE/dunesim/pull/66.

Waiting to hear back from Laura on dunesim pr66

The default is to create one "TPCInner" volume that is half the full volume and place it twice, rotating one of them to get things in the right position. This way, in the end, you have two volumes with the same name. In this script, two volumes are created independently (but using the same algorithm and also rotating one of them) so that they have different names but the procedure is the same in both. So it doesn't just add the labels, it creates a different second logical volume that is identical to the first and places it the same way as before. I don't think this is the best option but on the other hand I don't see how this could create any inconsistency with v5 geo.
As a side note, in largeant, both the old (TPCInner) and the new volumes (TPCInner0 and TPCInner1) are listed so that old geometries can also be run without problems


Install Issues

Hi. I have Rclone and a Config already
I am getting the error with the plugin
plugin.tar.gz extracted
install.sh returned a non-zero value:
Error with install script

Error Log
2022:01:21-16:10:59: (1)Execle error on /usr/local/directadmin/plugins/rclone_backup/scripts/install.sh : No such file or directory

same here

Thank you for pointing this out.  The issue which was due to CRLF instead of LF should now be fixed in 1.0.6.

Cool!

The Install worked, But there is now an error on the CronJob Page

Page Keeps Refreshing

Do you have any domains setup under the admin account or are these setup under new accounts.
What's the output of this (replace IP address) - https://SERVER IP:2222/CMD_API_DOMAIN?json=yes&action=document_root

I have domains under the admin account
{
"users":
{
"admin":
{
"domains":
{
"example1.com":
{
"private_html": "/home/admin/domains/example1.com/private_html",
"public_html": "/home/admin/domains/example1.com/public_html",
"subdomains":
{
}
},
"example2.com":
{
"private_html": "/home/admin/domains/example2.net/private_html",
"public_html": "/home/admin/domains/example2.net/public_html",
"subdomains":
{
}
}
}
}
}
}

Does adding var_dump($domainJSON); die; on line 29 give the same output?
string(1234) "{ "users": { "admin": { "domains": { "DOMAIN.com": { "private_html": "/home/admin/domains/DOMAIN

Now get an error in the ui "string(70) "use https:string(70) "use http"
Same output at  https://SERVER IP:2222/CMD_API_DOMAIN?json=yes&action=document_root

Sorry I don't understand that message.  Can you post the full output please (with domains redacted).
Is there any log entries referencing this?
Do you use HTTP or HTTPS for admin?

Installing the version below should product a similar output to the image below.  Please confirm this is the case.
rclone_backup.tar.gz

i have the same issue as the post above, after installing the custom version made by you above , any hopes that this might be fixed soon ?

Progress has been made.  Now need to add changes and test.

Version 1.0.7 has now been published.


Enable browser support for SVG2 geometry CSS properties

Currently I receive "unknown property" warnings for SVG2 geometry CSS properties, such as:

cx
cy
r
rx
ry
x
y

While these are registered in the css-schema.json file, their entries have browser set to none, so they are likely being filtered out.
If these entries are being automatically fetched and generated from MDN or another source, and that source has not updated their documents in a way that is understood by the automation script, is there a way to update the schema so these are registered appropriately? MDN does have docs for these with compatibility tables, such as this page for cx, but I'm not certain where the data in the schema is coming from.

The problem used to be because MDN doesn't have the data for it so it'll need the one in the css-schema.json. But seems like MDN added it recently so updating mdn-data and regenerate the data should be able to solve this. (Repeating here for the VSCode maintainers)

@jasonlyu123 what needs to get done to solve this issue? happy to give it a shot at creating a PR!


Change version and dependency jars to v22.02.0

Change version and dependency jars to v22.02.0
Signed-off-by: Tim Liu<EMAIL_ADDRESS>

Note: merge with "Create a merge commit"


add mypy to the build and some typing

Since we're dropping support for old versions of python (pre-3.6) I thought it might be appropriate to include some typing in the 3.x series, along with mypy in the build to check typing where appropriate.

Thanks for working on this. However, after my experience adding typing to the Pallets libraries, I am only confident in myself adding typing in one go, rather than others adding it bit by bit. Nothing against your work here, just don't feel comfortable merging it.


other-transcode no arguments fails - --nvenc fails but --hevc works

Thanks for this really interesting tool!
I am using an AMD system with a Nvidia RTX 3090.
When using default other-transcode I get a fail
other-transcode h:\Video\encode\test\clip_001.mkv
or if I use --nvenc it fails
other-transcode --nvenc h:\Video\encode\test\clip_001.mkv
Here is the fail message I get
[h264_nvenc @<PHONE_NUMBER>27f580] No capable devices found [vost#0:0/h264_nvenc @<PHONE_NUMBER>4e1b00] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
But if I use
other-transcode --hevc h:\Video\encode\test\clip_001.mkv
it is successful and pegs the GPU video encoder at 100% which is better then transcode-video's (handbrake) 80%
Any ideas on whats going on?
Also, if you have any examples of passing on quality arguments to ffmpeg that would be great, right now the quality of transcode-video --nvenc quality is a little better then ffmpeg --hevc in my testing, but ffmpeg is much faster and the file size is much smaller.
thanks again!

@Gattung My apologies for taking so long to respond.
This is very odd: For this command:
other-transcode --hevc h:\Video\encode\test\clip_001.mkv

Can you change that to:
other-transcode --dry-run --hevc h:\Video\encode\test\clip_001.mkv

... and copy the output of that command here.
Thanks.

No worries! here's the output.
Verifying "ffprobe" availability... Verifying "ffmpeg" availability... Verifying "mkvpropedit" availability... Finding encoders... Trying "hevc_nvenc" video encoder... Scanning media... duration = 00:09:15.982 Stream mapping:  0 = hevc_nvenc / 8000 Kbps  1 = copy Command line: ffmpeg -loglevel error -stats -i h:\Video\encode\test\clip_001.mkv -map 0:0 -c:v hevc_nvenc -pix_fmt:v p010le -b:v 8000k -maxrate:v 25000k -bufsize:v 25000k -color_primaries:v bt2020 -color_trc:v smpte2084 -colorspace:v bt2020nc -metadata:s:v title= -disposition:v default -map 0:1 -c:a:0 copy -metadata:s:a:0 title= -disposition:a:0 default -sn -metadata:g title= -default_mode passthrough clip_001.mkv

@Gattung Thanks! OK, it's definitely using the Nvidia encoder then. Hmmmmm.
OK, just in case... What happens if you change this:
other-transcode --nvenc h:\Video\encode\test\clip_001.mkv

...to this:
other-transcode -dry-run --nvenc h:\Video\encode\test\clip_001.mkv

Verifying "ffprobe" availability...
Verifying "ffmpeg" availability...
Verifying "mkvpropedit" availability...
Finding encoders...
Scanning media...
duration = 00:09:15.982
Stream mapping:
 0 = h264_nvenc / 6000 Kbps
 1 = copy
Command line:
ffmpeg -loglevel error -stats -i h:\Video\encode\test\clip_001.mkv -map 0:0 -filter:v scale=1920:1080 -c:v ``h264_nvenc -b:v 6000k -maxrate:v 20000k -bufsize:v 20000k -profile:v high -color_primaries:v bt2020 -``color_trc:v smpte2084 -colorspace:v bt2020nc -metadata:s:v title= -disposition:v default -map 0:1 -c:a:0 copy -``metadata:s:a:0 title= -disposition:a:0 default -sn -metadata:g title= -default_mode passthrough clip_001.mkv

@Gattung OK, that looks perfectly normal. It's selected the correct encoder. Aaaaand... I have no idea why it would fail.
Honestly, I've never seen anything like that.
My only thought is that there's either something wrong with ffmpeg or with your video drivers.

OK thanks! Will investigate further.
On Mon, Jul 17, 2023, 10:35 PM Don Melton @.***> wrote:

@Gattung https://github.com/Gattung OK, that looks perfectly normal.
It's selected the correct encoder. Aaaaand... I have no idea why it would
fail.
Honestly, I've never seen anything like that.
My only thought is that there's either something wrong with ffmpeg or
with your video drivers.
—
Reply to this email directly, view it on GitHub
https://github.com/donmelton/other_video_transcoding/issues/190#issuecomment-1639516027,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAB3UAS4Z6E74PUVVNICF6DXQYOCJANCNFSM6AAAAAA2LNBYAY
.
You are receiving this because you were mentioned.Message ID:
@.***>


TooltipTemplate

When you use tooltipTemplate:, it works for single dataset and not two datasets.
Two dataset:
https://jsfiddle.net/wckfhrsa/10/
One dataset:
https://jsfiddle.net/4pp64drv/1/

@jt4000 multiTooltipTemplate is used when there is more than one dataset

The result from the jsfiddle is totally different from your statement. I strongly recommend you to take a look at the link

@jt4000 I did look at the JSfiddle. When there is more than one dataset (and thus more than one item to show) a different template option is used. I modified the JSFiddle and it worked as expected.
The option that you want to use is multiTooltipTemplate
Please see: http://www.chartjs.org/docs/#getting-started-global-chart-configuration

Thank your for your help!

Closing as resolved.


[runix/pgadmin4] invalid .secretName check at tls

Describe the bug
A clear and concise description of what the bug is.
The fix
https://github.com/rowanruseler/helm-charts/blob/b9e7c0b3a2cffcbb5fbee9c2df9e583a950659b9/charts/pgadmin4/templates/ingress.yaml#L32
from PR https://github.com/rowanruseler/helm-charts/pull/190 for isssue https://github.com/rowanruseler/helm-charts/issues/189 introduces a new error due invalid "path":
Error: INSTALLATION FAILED: template: pgadmin4/templates/ingress.yaml:32:18: executing "pgadmin4/templates/ingress.yaml" at <.Values.ingress.tls.secretName>: nil pointer evaluating interface {}.ingress

since it doesn't honor that it's executed in a "range" context
https://github.com/rowanruseler/helm-charts/blob/b9e7c0b3a2cffcbb5fbee9c2df9e583a950659b9/charts/pgadmin4/templates/ingress.yaml#L29
just using
{{- if .secretName }}

should fix it
Version of Helm and Kubernetes:
helm:
Version:"v3.8.0", GitCommit:"d14138609b01886f544b2025f5000351c9eb092e", GitTreeState:"clean", GoVersion:"go1.17.5"

k8s:
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.2", GitCommit:"f66044f4361b9f1f96f0053dd46cb7dce5e990a8", GitTreeState:"clean", BuildDate:"2022-06-15T14:14:10Z", GoVersion:"go1.18.3", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.4", GitCommit:"b969368e201e1f09440892d03007c62e791091f8", GitTreeState:"clean", BuildDate:"2022-12-16T19:44:08Z", GoVersion:"go1.19.3", Compiler:"gc", Platform:"linux/amd64"}

Which chart:
runix/pgadmin4 1.14.2 (6.19)
What happened:
installations throws the following error:
Error: INSTALLATION FAILED: template: pgadmin4/templates/ingress.yaml:32:18: executing "pgadmin4/templates/ingress.yaml" at <.Values.ingress.tls.secretName>: nil pointer evaluating interface {}.ingress

What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
values.yaml:
ingress:
  enabled: true
  tls:
    - secretName: my-secret
      hosts:
        - my-host.com

helm install --dry-run -f ./values.yaml test runix/pgadmin4

Anything else we need to know:

provided PR with fix https://github.com/rowanruseler/helm-charts/pull/192


Update PHSMethods course

I have updated the course with the new functions and some renamed functions.

Changes accepted, merged PR, and redeployed app.


0.2.0 search result not sort as distince

Describe the bug
0.2.0 search result not sort as distince
Steps/Code to reproduce behavior
not sort after find from mongo
Expected behavior
search result sort with distince
Environment details

Docker info
Phantoscope version v0.2.0

already fix it in pre release version


New French specs-updated 1224

15 updates in total:

Add new French specs for 11 existing files
Create 4 new files

@XiaoxiaoMa0815, please also add the number of new or edited specs per file in the PR description above. Currently it only mentions the files, which is not enough to track parity with English.

@GaoyanCheerup, FYI.

@XiaoxiaoMa0815, feel free to close the PR and open new ones if you prefer. But that's not necessary. PRs can also be updated.
Choose the option that best fits your process. But if you close a PR that has comments, please mention the solution to those comments in the relevant new PRs.


[feature request] dark mode switcher

Hi,
Some elements are difficult to read and I think it is because of the dark mode.
Maybe a button to switch it could be a first step to improve it?
Example:

Hi @mykiwi,
thanks for the report !
Indeed, I hadn't noticed.
It's relatively easy to fix, are you interested? It's HTML in the file src/Report/Html/templates/index.html. Would you be interested in opening a PR ?


a space is interpreted as a number

Plot.plot({ marks: [Plot.barX([{ x: 200, y: " " }], { x: "x", y: "y" })] }) wrongly recognizes the space as a number, then warns about y being numbers.

It’s because it’s coercible to a number: +" " equals 0.


Enable parsing of python version 3.10+ in startup script

Previous code with [:3] was parsing 3.10 as 3.1 and not getting the
correct site-packages directory for launch
Fix #518

I propose a better alternative that doesn't require much parsing, by using the platform module:
import platform
print(".".join(platform.python_version_tuple()[:2]))

Thanks a lot. I have just verified that this will also work with Python 2.7.
I will merge it in the next days.

this diff works currently on python 3.10:
diff --git a/wxglade b/wxglade
index 2506b53d..25720533 100755
--- a/wxglade
+++ b/wxglade
@@ -38,7 +38,7 @@ for INTERPRETER in $INTERPRETER_LIST; do
   PYTHON_BIN=$INTERPRETER

   # determined current python version
-  PY_VERSION=$(${PYTHON_BIN} -c 'import sys; print(sys.version[:3])')
+  PY_VERSION=$(${PYTHON_BIN} -c 'import platform; print(".".join(platform.python_version_tuple()[:2]))')

   EGG_DIR="wxGlade-${WXG_VERSION}-py${PY_VERSION}.egg"


kotlin module uses different kotlin version than what boot provides, prevents boot 3.1 compatibility builds

This causes build failures when testing compatibility with boot 3.1.x. It also sets the target jvm version to 1.8, it should be 17
https://github.com/spring-cloud/spring-cloud-function/blob/9e39f2c991a8f1349e38369e1dc61518046edb44/spring-cloud-function-kotlin/pom.xml#L64-L99
One potential issue is the use of kotlin-maven-allopen which I don't think is managed anywhere. Maybe we can ask the boot team to manage it?
Here is a similar change that gateway made https://github.com/spring-cloud/spring-cloud-gateway/commit/4c4c6f33b088c39b18466005b74575695b5166c0

Not sure upgrading to 1.8.10 is the right answer. Boot 3.0 uses 1.7.x. Will a boot 3.0 app work if this is compiled with 1.8.x?