let with elvis don't work in Compose desktop

Simple example:
val test: String? = null
test?.let {
    Text("not null")
?: println("null")

prints nothing.

It looks like a pure kotlin code. How is it related to Compose?

On my env everything works fine. Try this example:
@Composable
fun repro1236() {
    val test: String? = "1"
    test?.let {
        println("not null")
        Text("not null")
    }
}

https://android-review.googlesource.com/c/platform/frameworks/support/+/2077826 - was fixed in upstream. We'll have it after the next rebase.

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.


[yacas mode] fix multiline comments

Obvious fix for a silly error in yacas mode causing multiline comments to be at most one line long

Thanks. Merged as 060445d


Set recording dir, etc, with NetworkControl object?

Hi there -- thanks for having written this package! I'm wondering if the NetworkControl class can send info about recording dir, prepend text, etc, similar to what's done here or discussed here. I'm happy to fork and implement it, just curious if it already exists. Thanks!

Hi Jonah! We are about to release a new version of the GUI that includes a built-in configuration server. The Network Events module will still work, but it will be easier to make changes using the built-in server, which is launched automatically at startup and doesn't require a separate plugin. We will soon be updating the NetworkControl class to be compatible with the new API, including commands for setting the recording directory.
Stay tuned for an announcement about the next release!

Great, thank you! Looking forward to it.

On Dec 21, 2021, at 4:07 PM, Josh Siegle @.***> wrote:
Hi Jonah! We are about to release a new version of the GUI that includes a built-in configuration server. The Network Events module will still work, but it will be easier to make changes using the built-in server, which is launched automatically at startup and doesn't require a separate plugin. We will soon be updating the NetworkControl class to be compatible with the new API, including commands for setting the recording directory.
Stay tuned for an announcement about the next release!
—
Reply to this email directly, view it on GitHub https://github.com/open-ephys/open-ephys-python-tools/issues/8#issuecomment-999094592, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQKOLZFNOCHNWNPVZ75O2YTUSDT75ANCNFSM5KOZ5DUQ.
You are receiving this because you authored the thread.


Cross skill the team (create cross knowledge documentation/primers)

Describe the issue
The tech team finds the non-tech "boring" and likely vice versa.  It will be valuable to our users for all of our team members to understand the full nature of the work we do, including compliance/procurement etc because as our communities expand everyone will need to know enough to connect people to help them get answers, rather than just saying "I don't know".
Additional context
Add any other context, attachments or screenshots
Definition of done

Create a few primers or cue cards for support issues that might not be tech related per se.

not relevant


Add flux consumption model from I. Savona

Replaces #4
Added inductance functionality and flux functionality to POPCON for time-dependent and time-independent calculations.
Compared to #4

Refactored to eliminate unnecessary use of wraps_ufunc
Adapted to use the new Algorithm.register_algorithm method

Keeps the same test suite as the original merge request.
N.b. All physics in this merge request was implemented by I. Savona

@IsaacSavona Could you please check if this merge request matches your original implementation?

@IsaacSavona Could you please check if this merge request matches your original implementation?
I've marked a few points where I had questions with TODO: please clear these.
Also, the Jupyter notebook needs to be ported to the new style. Could you do this?

I am looking into this now. The only thing I immediately see missing from the commit is the algorithm files, but I guess you have modified this to be Algorithm.register_algorithm structure.

@IsaacSavona Could you please check if this merge request matches your original implementation?
I've marked a few points where I had questions with TODO: please clear these.
Also, the Jupyter notebook needs to be ported to the new style. Could you do this?

Hi guys,
Having some trouble porting the new changes to my notebook. I have tried running getting_started.ipynb to no avail. I get an error at algorithm.update_dataset(dataset).
Issue
Running algorithm.update_dataset(dataset)
Error
It is too long to warrant pasting the entire thing, but I have attached the very end below. It seems like it has something to do with the key-value pairs of the impurities...
...
File [~/Documents/CFS/cfspopcon24/cfspopcon/cfspopcon/algorithm_class.py:119](https://file+.vscode-resource.vscode-cdn.net/Users/isaacsavona/Documents/CFS/cfspopcon24/cfspopcon/docs/doc_sources/~/Documents/CFS/cfspopcon24/cfspopcon/cfspopcon/algorithm_class.py:119), in Algorithm.update_dataset(self, dataset, allow_overwrite)
    [114](https://file+.vscode-resource.vscode-cdn.net/Users/isaacsavona/Documents/CFS/cfspopcon24/cfspopcon/docs/doc_sources/~/Documents/CFS/cfspopcon24/cfspopcon/cfspopcon/algorithm_class.py:114)         sorted_default_keys = ", ".join(sorted(self.default_keys))
    [115](https://file+.vscode-resource.vscode-cdn.net/Users/isaacsavona/Documents/CFS/cfspopcon24/cfspopcon/docs/doc_sources/~/Documents/CFS/cfspopcon24/cfspopcon/cfspopcon/algorithm_class.py:115)         raise KeyError(
    [116](https://file+.vscode-resource.vscode-cdn.net/Users/isaacsavona/Documents/CFS/cfspopcon24/cfspopcon/docs/doc_sources/~/Documents/CFS/cfspopcon24/cfspopcon/cfspopcon/algorithm_class.py:116)             f"KeyError for {self._name}: Key '{key}' not in dataset keys [{sorted_dataset_keys}] or default values [{sorted_default_keys}]"
    [117](https://file+.vscode-resource.vscode-cdn.net/Users/isaacsavona/Documents/CFS/cfspopcon24/cfspopcon/docs/doc_sources/~/Documents/CFS/cfspopcon24/cfspopcon/cfspopcon/algorithm_class.py:117)         )
--> [119](https://file+.vscode-resource.vscode-cdn.net/Users/isaacsavona/Documents/CFS/cfspopcon24/cfspopcon/docs/doc_sources/~/Documents/CFS/cfspopcon24/cfspopcon/cfspopcon/algorithm_class.py:119) result = self._function(**input_values)
    [120](https://file+.vscode-resource.vscode-cdn.net/Users/isaacsavona/Documents/CFS/cfspopcon24/cfspopcon/docs/doc_sources/~/Documents/CFS/cfspopcon24/cfspopcon/cfspopcon/algorithm_class.py:120) return xr.Dataset(result).merge(dataset, join="left", compat=("override" if allow_overwrite else "no_conflicts"))
...
--> [304](https://file+.vscode-resource.vscode-cdn.net/Users/isaacsavona/Documents/CFS/cfspopcon24/cfspopcon/docs/doc_sources/~/Documents/CFS/cfspopcon24/cfspopcon/cfspopcon/formulas/read_atomic_data.py:304)     max_temp, min_temp, max_density, min_density = self.grid_limits[species]
    [305](https://file+.vscode-resource.vscode-cdn.net/Users/isaacsavona/Documents/CFS/cfspopcon24/cfspopcon/docs/doc_sources/~/Documents/CFS/cfspopcon24/cfspopcon/cfspopcon/formulas/read_atomic_data.py:305)     electron_temp = np.minimum(electron_temp, max_temp)
    [306](https://file+.vscode-resource.vscode-cdn.net/Users/isaacsavona/Documents/CFS/cfspopcon24/cfspopcon/docs/doc_sources/~/Documents/CFS/cfspopcon24/cfspopcon/cfspopcon/formulas/read_atomic_data.py:306)     electron_temp = np.maximum(electron_temp, min_temp)

KeyError: <AtomicSpecies.Helium: 2>

Hi Isaac, Can you run poetry install poetry run radas ? It looks like you’re missing the atomic data files. Cheers, Tom https://cfs.energy/ Thomas Body Scientist, Boundary and Divertor Physics Commonwealth Fusion Systems https://cfs.energy/ he/him/his
…
On Mon, 27 May 2024 at 10:14 PM, IsaacSavona @.**> wrote: @IsaacSavona https://github.com/IsaacSavona Could you please check if this merge request matches your original implementation? I've marked a few points where I had questions with TODO: please clear these. Also, the Jupyter notebook needs to be ported to the new style. Could you do this? Hi guys, Having some trouble porting the new changes to my notebook. I have tried running getting_started.ipynb to no avail. I get an error at algorithm.update_dataset(dataset). Issue Running algorithm.update_dataset(dataset) Error It is too long to warrant pasting the entire thing, but I have attached the very end below. It seems like it has something to do with the key-value pairs of the impurities... ... File ~/Documents/CFS/cfspopcon24/cfspopcon/cfspopcon/algorithm_class.py:119, in Algorithm.update_dataset(self, dataset, allow_overwrite) 114 sorted_default_keys = ", ".join(sorted(self.default_keys)) 115 raise KeyError( 116 f"KeyError for {self._name}: Key '{key}' not in dataset keys [{sorted_dataset_keys}] or default values [{sorted_default_keys}]" 117 ) --> 119 result = self._function(input_values) 120 return xr.Dataset(result).merge(dataset, join="left", compat=("override" if allow_overwrite else "no_conflicts")) ... --> 304 max_temp, min_temp, max_density, min_density = self.grid_limits[species] 305 electron_temp = np.minimum(electron_temp, max_temp) 306 electron_temp = np.maximum(electron_temp, min_temp) KeyError: <AtomicSpecies.Helium: 2> — Reply to this email directly, view it on GitHub <#57 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A32XUMNRBEWHBUCQUCCCLETZEOH2RAVCNFSM6AAAAABH6O7Z3KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZUGAYDMOJVG4 . You are receiving this because you authored the thread.Message ID: @.>

Hi Isaac, Can you run poetry install poetry run radas ? It looks like you’re missing the atomic data files. Cheers, Tom https://cfs.energy/ Thomas Body Scientist, Boundary and Divertor Physics Commonwealth Fusion Systems https://cfs.energy/ he/him/his
…
On Mon, 27 May 2024 at 10:14 PM, IsaacSavona @.**> wrote: @IsaacSavona https://github.com/IsaacSavona Could you please check if this merge request matches your original implementation? I've marked a few points where I had questions with TODO: please clear these. Also, the Jupyter notebook needs to be ported to the new style. Could you do this? Hi guys, Having some trouble porting the new changes to my notebook. I have tried running getting_started.ipynb to no avail. I get an error at algorithm.update_dataset(dataset). Issue Running algorithm.update_dataset(dataset) Error It is too long to warrant pasting the entire thing, but I have attached the very end below. It seems like it has something to do with the key-value pairs of the impurities... ... File ~/Documents/CFS/cfspopcon24/cfspopcon/cfspopcon/algorithm_class.py:119, in Algorithm.update_dataset(self, dataset, allow_overwrite) 114 sorted_default_keys = ", ".join(sorted(self.default_keys)) 115 raise KeyError( 116 f"KeyError for {self._name}: Key '{key}' not in dataset keys [{sorted_dataset_keys}] or default values [{sorted_default_keys}]" 117 ) --> 119 result = self._function(input_values) 120 return xr.Dataset(result).merge(dataset, join="left", compat=("override" if allow_overwrite else "no_conflicts")) ... --> 304 max_temp, min_temp, max_density, min_density = self.grid_limits[species] 305 electron_temp = np.minimum(electron_temp, max_temp) 306 electron_temp = np.maximum(electron_temp, min_temp) KeyError: <AtomicSpecies.Helium: 2> — Reply to this email directly, view it on GitHub <#57 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A32XUMNRBEWHBUCQUCCCLETZEOH2RAVCNFSM6AAAAABH6O7Z3KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZUGAYDMOJVG4 . You are receiving this because you authored the thread.Message ID: @.>

Hi Tom, I have already done these things...

Hey @tbody-cfs! I am trying to keep up with your changes. I have updated my notebook and have two questions (there is a preamble to each but the main questions are bolded):

I need to update my SPARC_PRD_results.nc file in order to pull the changes you have made here. When I run python generate_regression_results.py within the regression results directory it gives me the error FileNotFoundError: atomic_data_directory (/Users/isaacsavona/Documents/CFS/cfspopcon24/cfspopcon/tests/regression_results/radas_dir) does not exist which is true because the /radas_dir exists just under /cfspopcon. How do I properly keep my SPARC_PRD_results.nc file up to date?
When I make changes to the dataset object which comes from the input.yaml in my notebook I do not want to have to worry about figuring out all the dependent parameters which I need to update with algorithms (because there are many). Instead, I just want to run all the algorithms to ensure the whole dataset is updated. When I run dataset = algorithm.update_dataset(dataset) after modifying the parameters of the dataset it throws an error about the dataset not being hashable. How can I update the whole dataset in one go once I have made changes to it?

Hi @IsaacSavona,
To update the regression results, you've got the correct command poetry run python tests/regression_results/generate_regression_results.py. It might be that you're running this from the wrong directory, however — you need to run it from the top-level directory (the one containing the tests folder).
However, for this merge request you shouldn't need to update the regression results yourself.
For updating the dataset yourself, I'm not sure if I understand the error that you're encountering. The following shows how you'd calculate all of the terms to run your flux consumption model.
import xarray as xr
import numpy as np
import matplotlib.pyplot as plt

from cfspopcon import Algorithm, named_options
from cfspopcon.unit_handling import ureg

algorithms = [
    "calc_minor_radius_from_inverse_aspect_ratio",
    "calc_elongation_at_psi95_from_areal_elongation",
    "calc_average_ion_temp_from_temperature_ratio",
    "calc_f_shaping_for_qstar",
    "calc_q_star_from_plasma_current",
    "calc_beta_toroidal",
    "calc_beta_poloidal",
    "calc_effective_collisionality",
    "calc_ion_density_peaking",
    "calc_electron_density_peaking",
    "calc_bootstrap_fraction",
    "calc_inductive_plasma_current",
    "calc_Spitzer_loop_resistivity",
    "calc_resistivity_trapped_enhancement",
    "calc_neoclassical_loop_resistivity",
    "calc_loop_voltage",
    "calc_cylindrical_edge_safety_factor",
    "calc_internal_inductivity",
    "calc_internal_inductance_for_cylindrical",
    "calc_external_inductance",
    "calc_vertical_field_mutual_inductance",
    "calc_invmu_0_dLedR",
    "calc_vertical_magnetic_field",
    "calc_internal_flux",
    "calc_external_flux",
    "calc_resistive_flux",
    "calc_poloidal_field_flux",
    "calc_flux_needed_from_solenoid_over_rampup",
    "calc_max_flattop_duration",
    "calc_breakdown_flux_consumption",
]

dataset = xr.Dataset(
    data_vars=dict(
        major_radius = 1.85 * ureg.m,
        areal_elongation = 1.75,
        triangularity_psi95 = 0.3,
        magnetic_field_on_axis = 12.2 * ureg.T,
        plasma_current = 8.7 * ureg.MA,
        inverse_aspect_ratio = 0.57 / 1.85,
        elongation_ratio_areal_to_psi95 = 1.025,
        average_electron_density = 2.5 * ureg.n20,
        average_electron_temp = 9.0 * ureg.keV,
        ion_to_electron_temp_ratio = 1.0,
        surface_inductance_coefficients = named_options.SurfaceInductanceCoeffs.Barr,
        total_flux_available_from_CS = 35.0 * ureg.Wb,
        ejima_coefficient = 0.6,
        z_effective = 1.5,
        electron_density_peaking_offset = -0.1,
        ion_density_peaking_offset = -0.2,
        temperature_peaking = 2.5,
        dilution = 0.85,
    )
)

for algorithm in algorithms:
    alg = Algorithm.get_algorithm(algorithm)
    dataset = alg.update_dataset(dataset)

print(dataset["max_flattop_duration"])

Could you pull the latest changes, update the demo notebook and then push your changes. Then, we'll merge this in.

Hi @IsaacSavona,
To update the regression results, you've got the correct command poetry run python tests/regression_results/generate_regression_results.py. It might be that you're running this from the wrong directory, however — you need to run it from the top-level directory (the one containing the tests folder). However, for this merge request you shouldn't need to update the regression results yourself.
For updating the dataset yourself, I'm not sure if I understand the error that you're encountering. The following shows how you'd calculate all of the terms to run your flux consumption model.
import xarray as xr
import numpy as np
import matplotlib.pyplot as plt

from cfspopcon import Algorithm, named_options
from cfspopcon.unit_handling import ureg

algorithms = [
    "calc_minor_radius_from_inverse_aspect_ratio",
    "calc_elongation_at_psi95_from_areal_elongation",
    "calc_average_ion_temp_from_temperature_ratio",
    "calc_f_shaping_for_qstar",
    "calc_q_star_from_plasma_current",
    "calc_beta_toroidal",
    "calc_beta_poloidal",
    "calc_effective_collisionality",
    "calc_ion_density_peaking",
    "calc_electron_density_peaking",
    "calc_bootstrap_fraction",
    "calc_inductive_plasma_current",
    "calc_Spitzer_loop_resistivity",
    "calc_resistivity_trapped_enhancement",
    "calc_neoclassical_loop_resistivity",
    "calc_loop_voltage",
    "calc_cylindrical_edge_safety_factor",
    "calc_internal_inductivity",
    "calc_internal_inductance_for_cylindrical",
    "calc_external_inductance",
    "calc_vertical_field_mutual_inductance",
    "calc_invmu_0_dLedR",
    "calc_vertical_magnetic_field",
    "calc_internal_flux",
    "calc_external_flux",
    "calc_resistive_flux",
    "calc_poloidal_field_flux",
    "calc_flux_needed_from_solenoid_over_rampup",
    "calc_max_flattop_duration",
    "calc_breakdown_flux_consumption",
]

dataset = xr.Dataset(
    data_vars=dict(
        major_radius = 1.85 * ureg.m,
        areal_elongation = 1.75,
        triangularity_psi95 = 0.3,
        magnetic_field_on_axis = 12.2 * ureg.T,
        plasma_current = 8.7 * ureg.MA,
        inverse_aspect_ratio = 0.57 / 1.85,
        elongation_ratio_areal_to_psi95 = 1.025,
        average_electron_density = 2.5 * ureg.n20,
        average_electron_temp = 9.0 * ureg.keV,
        ion_to_electron_temp_ratio = 1.0,
        surface_inductance_coefficients = named_options.SurfaceInductanceCoeffs.Barr,
        total_flux_available_from_CS = 35.0 * ureg.Wb,
        ejima_coefficient = 0.6,
        z_effective = 1.5,
        electron_density_peaking_offset = -0.1,
        ion_density_peaking_offset = -0.2,
        temperature_peaking = 2.5,
        dilution = 0.85,
    )
)

for algorithm in algorithms:
    alg = Algorithm.get_algorithm(algorithm)
    dataset = alg.update_dataset(dataset)

print(dataset["max_flattop_duration"])

Could you pull the latest changes, update the demo notebook and then push your changes. Then, we'll merge this in.

I am getting on this now


Check if gbase64 (GNU) is available on Mac and use it in preference to BSD base64

Previously it was switching to GNU versions of base64 even if base64 was the BSD version
Fixes #568
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

It works fine now with and without coreutils on Mojave. Thanks!


Implement imgui-sdl2-support crate.

This PR implements a SDL 2 support crate for imgui-rs, similar to the existing imgui-winit-support crate. This is mainly due to the fact that the existing SDL 2 support crate is severely outdated and a "official" crate is beneficial. This also checks off the last point from the list at the end of #444.
Currently most things except locked/rounded DPI are implemented, and I don't think there is a straightforward way to implement that. Please let me know if there is something that has to be changed, or can be improved.
(sorry for the new PR!)

The only thing now left is adding top-level doc-comments and some other small doc changes. I am not sure what to do about locked/rounded DPI (also adding that I don't have much knowledge about the same), the existing third party SDL support crate also doesn't handle it.

I am a bit busy the next day, so I will address the other points a bit later. Sorry!

Good call -- when this is merged, I will make the same change to the winit crate!
And don't worry about timing -- I didn't review until Sunday night, so I understand that we've all got work/classes in the week


GUI colors not showing

I really like this pure lua theme. It's fine in terminal but when I try to use neovim gui neovide,colors not showing.
I read this FAQ:
https://neovide.dev/faq.html#the-terminal-displays-fallback-colorsterminal-does-not-show-my-colors

@vintingb I do not use GUI clients like Neovide, so I actually did not know about this issue. So, unfortunately, you have to figure out it by yourself. If you do so, you can submit a pull request.
Thanks for using the project and Happy Coding ❤


add moment, and get stuff to display on the 4th screen,

hey @StephanieEA @bekahlundy ,
we added moment.js to our file, and got it so that we can display info on the screen!
Dylan

🎲 📧 ❗


Use PAM module system [feature request]

Idea:
I think it would be fine to use PAM as third party authentication application.
Benefits
RADIUS, LDAP and many other thirdparty auth systems can be used without implementing them in BookStack, OS deals with them by itself with appropriate pam module

Thank you for the suggestion @kanlas-net. Unfortunately I would look to not include this feature for the following reasons:

I'd like to not extend out our authentication options any further unless there's high value to our users since the current systems already consume a lot of time.
Implementing this would not replace the existing auth systems, due to current supported options and supported operating systems, hence it will add to the current maintenance effort of our auth systems.
I'd prefer not to implement OS-specific features where possible.
Support for PAM with the PHP space looks to have dropped off meaning a lot of effort would be required to get this working, or at least concern for possible security issues in existing implementation options.

Therefore I'm going to close this off as out-of-scope.

Currently I use gdocs. I think about migrating to bookstack because of user management. Bookstack is createt not as a system for one-person but for group work - so the login process/user management should be consider as priority.
Check again that situation. Now - it's very easy to implement.
Example of authentication module - ready to use code - few lines only:
https://github.com/rohanbbb/php-radius-user-login/blob/master/userAuthenticate.php
"Implementing this would not replace the existing auth systems" - radius is supplement not necessity.
Radius is not OS-specific. (There are radius servers for Windows/FreeBSD)
As I remember correctly You don't need to use PAM module - for sure not at bookstack app side.

@interduo My thoughts remain the same as three years ago, in that I would not look to expand our scope of support to additional authentication systems without significant value, and I don't see Radius as meeting that threshold since there's been little desire.
If handy with PHP, you may be able to hack on something basic using our logical theme system.

My thoughts remain the same as three years ago, in that I would not look to expand our scope of support to additional authentication systems without significant value, and I don't see Radius as meeting that threshold since there's been little desire.

The client is outside bookstack, loaded by composer - client is supported by other maintainers in other repository. On bookstack there would be only one line in composer.json and few lines in login code whitch would not change probbably during next many years. What kind of support You got on mind?
How do You measure that little desire? Centralized user management is everyday case that need to be discused, when software is implemented in institutions/company. What scope of auth types are supported by bookstack now? Radius is just most popular in linux.

If handy with PHP, you may be able to hack on something basic using our logical theme system.

Thanks for info - I will check that.

On bookstack there would be only one line in composer.json and few lines in login code whitch would not change probbably during next many years.

Initial MVP implementation is rarely ever my concern for adding features. It's everything else.

What kind of support You got on mind?

I'd need to get familiar with these technologies, I'd need to have testing environments for this, we'd need to mock the core functions for testing, we'd have to update and re-test on changes, we'd have to document the usage of this; then, most significantly, we'd have to support user queries and requests. We'd have to debug when a user reports an issue, we'd have to re-test things when necessary. From experience, even when implementing fairly strong standards, variance and edge-cases always pop up and bring their complexities.

How do You measure that little desire?

By your request being just the first for this, that I can remember, over the last 8 years. (The original issue opened here requests implementing PAM as a potential means for RADIUS, not for RADIUS itself).

Centralized user management is everyday case that need to be discused, when software is implemented in institutions/company. What scope of auth types are supported by bookstack now?

We already support email+password/LDAP/SAML2/OIDC, in addition to secondary auth via some specific OAuth services.

The original issue opened here requests implementing PAM as a potential means for RADIUS, not for RADIUS itself

I didn't want to new open similar issue for that. Grouping informations in less issues in long term benefits.
If You start considering radius as big need and decide for implement it - feel free to contact me for a comprehensive testing. For now I will just use email+pass auth.
Thank You for Your time.


Figure out how to work with bower dependencies

I am having trouble using the sbt snippet from the README:
// Include only the `src` directories from the bower packages
(sourceDirectories in purescript in Assets) ++=
    (baseDirectory.value / "bower_components" * AllPassFilter / "src").get

Does this issue cover that? Or is this issue just old and it actually works?
Haven't figured out what is wrong yet. Definitely not picking up the sources from bower_components. I will come back to it and get it to work.


Simple storage

This checklist is used to make sure that common guidelines for a pull request are followed.

For new extensions:

[ ] My extension description/summary conforms to the Extension Summary Guidelines.

If this PR is for a new extension or change to an existing extension, use the following to try out the changes in this PR:
docker run -it microsoft/azure-cli:latest
export EXT=<NAME>
pip install --upgrade --target ~/.azure/cliextensions/$EXT "git+https://github.com/williexu/azure-cli-extensions.git@simple_storage#subdirectory=src/$EXT&egg=$EXT"


Id Class enhancements

Some of our classes i feel need revamps for better adherence to SLAP and prevent Strong coupling (dependencies) between classes

after much discussion, ID class seems okay and no changes are required


tracer: skywalking reported wrong instance name

Title: skywalking reported the service name as instance name in tracing segment data
Description:
from docs, skywalking tracer extions could config servicename and instance name as following:
...
          tracing:
            provider:
              name: envoy.tracers.skywalking
              typed_config:
                "@type": type.googleapis.com/envoy.config.trace.v3.SkyWalkingConfig
                grpc_service:
                  envoy_grpc:
                    cluster_name: skywalking
                  timeout: 0.250s
                client_config:
                  service_name: service1-envoy
                  instance_name: service1-envoy-1
...

but, I found that skywalking oap(skywalking backend, a commponent to receive tracing data from envoy) received the instance name as the same as the service name. That's may be wrong.

cc skywalking extension owners @wbpcode @dio @lizan @Shikugawa

Thanks, we will check it.

Here is minor bug. We will fix it.


Upgrade packages to minimize the size of dependency graph

Hi@AnderssonPeter, I found an issue in the SampleWithSerilog.csproj:
Packages Hangfire.AspNetCore v1.7.10, Serilog.AspNetCore v3.2.0,Serilog.Sinks.Async v1.4.0, Serilog.Sinks.Console v3.1.1 and Serilog.Sinks.File v4.1.0 transitively introduce 101 dependencies into Hangfire.Console.Extensions’s dependency graph (see dependency graph before upgrades), while Hangfire.AspNetCore v1.7.24, Serilog.AspNetCore v3.4.0,Serilog.Sinks.Async v1.5.0, Serilog.Sinks.Console v4.0.0 and Serilog.Sinks.File v5.0.0 can only introduce 71 dependencies (see dependency graph after upgrades).
These upgrades can help project minimize the size of dependency graph.
Hope the PR can help you.
Best regards,
sucrose

Sorry for the huge delay, but a new package is on its way with this feature!


cleanup: replace &(*x) with x.as_ref() for smart pointer derefs

The only instances of the &(*x) dereferencing pattern in the codebase, is used with Completion smart pointers. Changed it to use as_ref(). No functional changes.

thanks!


Edit SearchDashboards sql limit from 1000 to 10000

Sql query has been refactored/fixed in develop.

:+1:


savefig() not working for tables

Hi, I very much enjoy using Winston to make plots in Julia. I came across an issue when attempting to save a table comprised of two plots to a file. Here, savefig() does not save the table, but only the most recent plot. An example is below.
using Winston
xx = collect(0.0:0.1:1.0)
p1 = plot(xx,xx.^2)
p2 = plot(xx,xx.^3)
tbl = Table(1,2)
tbl[1,1] = p1
tbl[1,2] = p2
figure()
display(tbl)
savefig("fig1.png")

Hi Nick. Just tell savefig what object do you actually want to print into the file:
savefig(tbl, "fig1.png")

ahh, thank you!

On May 25, 2016, at 7:00 PM, Helios De Rosario<EMAIL_ADDRESS>wrote:
Hi Nick. Just tell savefig what object do you actually want to print into the file:
savefig(tbl, "fig1.png")
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub https://github.com/nolta/Winston.jl/issues/265#issuecomment-221733047


[ENH] Auto-generation of ufuncs

Is your feature request related to a problem? Please describe.
The Numpy C API defines a way to generate "ufuncs" from a C function pointer. The C API takes a C callable and converts it into a Python callable that works on arrays of varying dimensions (broadcasting), and also handles reductions and broadcasting I think.
Describe the solution you'd like
The Numpy C API for this is accessible in Cython (via the pxd files) but is not hugely intuitive. It is also based around void*s to access the data, rather than the nicer Cython types like typed memoryviews). Therefore it might be nice to have a decorator to convert Cython cdef functions into Numpy ufuncs to allow them to work well with array broadcasting (people sometimes try to solve this problem with fused types of double[:], double[:,:], double[:,:,:]` etc., which is OK, but a bit ugly).
Example code:
cimport cython

@cython.numpy_ufunc
cdef double add(double x, double y):
    return x+y  # obviously a trivial example which is in Numpy already

Additional context
Numba provides a similar feature https://numba.pydata.org/numba-doc/dev/user/vectorize.html, so there's precedent (and people would probably understand it)
The Numpy C API also provides "generalised ufuncs". These would also be nice, but probably not part of an initial implementation.
This'd introduce a runtime+compile-time dependency on Numpy. Cython typically tries to be fairly dependency-free by default. I've proposed the decorator name numpy_ufunc to try to make that dependency as clear as possible. There's also a question: is there a way to do this without tying it to Numpy.
In the more distant future this might be a good place to introduce other backends (e.g. gpu, parallelization?). But definitely not for the initial version.
I have no immediate plans to implement this for the moment - posted mainly to get a bit of feedback about whether it's likely to be useful

This would be great to have! I played around with a ufunc-cython integration a bit here because I needed to evaluate the likelihood of a distribution repeatedly. But the interface in the linked repo is still pretty clunky. See here for the application.

Just throwing my 2 cents in as a happy Cython user, this would be a great feature for developers of other libraries who want to use Cython either to implement their own functions or to wrap external libraries, and who want to expose their work as ufuncs.
See for example https://github.com/uber/h3-py/issues/141 and https://github.com/uber/h3-py/issues/205, where the authors of H3-Py implemented only a bare minimal subset of ufunc behavior because it wasn't obvious or easy how to use Numpy to do it.

@gwerbin A few comments:

you can always test the preliminary implementation now (https://github.com/cython/cython/pull/4803),
it doesn't actually do anything that you can't do yourself (I think...) - it does a bit of auto-generation of Cython code but it's all code that a user could write. The main reason to do it in Cython that the Numpy interface is pretty confusing and that we can hide that from you

@da-woods Do you have an example of the generated Cython code? It would be very helpful to at least have an equivalent of this page https://numpy.org/doc/stable/user/c-info.ufunc-tutorial.html#example-numpy-ufunc-with-multiple-arguments-return-values in Cython.
I tried to browse your PR diff but the codebase is too unfamiliar to me and it would take me a very long time to work through it. I'm not much of a "Cython hacker" nor am I deeply familiar with C. My use cases tend to be pretty low on the complexity scale and my knowledge is correspondingly beginner-level.

@gwerbin Check out my PR, go to this line (https://github.com/cython/cython/blob/c7ba0c83d7fa7c0db60742b117b9e0207493013f/Cython/Compiler/UFuncs.py#L156) and add print(code.impl) and that should show the generated utility code I hope.
Note that the one thing that doesn't work in multiple return types where one of the types is a PyObject. That's really a consequence of the syntax I've used than a fundamental block.
There's also a small amount of C utility code. Think that all could be written in Cython but isn't (mainly for convenience). I think you'd add print(TempitaUtilityCode.load("UFuncConsts", "UFuncs_C.c", context=context).impl) (near https://github.com/cython/cython/blob/c7ba0c83d7fa7c0db60742b117b9e0207493013f/Cython/Compiler/UFuncs.py#L230) to see that.


Feature: Browser Extension

(Copy of open-sauced/hot#423)
Type of feature
🍕 Feature
Current behavior
We aren't getting submissions on the platform
Suggested solution
make an extension that suggests submissions for opensauced when user stars a repo.
Would love ideas on this approach.
Additional context
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Contributing Docs

[ ] I agree to follow this project's Contribution Docs__

Copy of comment from @nickytonline
Some background about browser extensions
I've worked on a bunch of extensions including one for a previous job. If you're looking to hook into GitHub.com, I'd recommend taking a peek at the Refined GitHub browser extension. They integrate seamlessly with GitHub.
All the code you write for the extension will work in Chromium based browsers that support extensions, and Firefox basically took on the same API as the Chrome Extension API, so for the most part the extension will be cross-browser. The main difference is Chromium based browser use the version 3 of the manifest while last I checked, Firefox still only supports version 2 of the manifest, although they are working on v3 support.
I'd also recommend using Parcel as the bundler for the project if this goes ahead as they support reloading the extension during development with the manifest version 3.
Suggested Approach
The approach you probably want to take is to integrate directly into the GitHub user interface by adding a Hot Sauce button. You could probably also add an option to the extension that auto-suggests to opensauced when the user stars a repo.
To implement this, you could add a click event listener to the button or link that stars the repo. The click event listener would see if the user has auto-suggest to opensauced, and if it does, it'd make an API call to opensauced to auto-suggest the repo. You could get the repo URL to send most likely via window.location.href
If the option to auto-suggest isn't enabled, surface a button to auto-suggest a submission to open sauced.
Happy to discuss this further and even pair with folks.

I’m going to close this one and transfer the original (didn’t know that was a thing 🤷♂
Cc @bdougie @nickytonline


Traefik doesn't work with Docker Swarm Mode on armhf Odroid C1 board

Related issue with HypriotOS team https://github.com/hypriot/image-builder-odroid-c1/issues/38
What version of Traefik are you using (traefik version)?
1.2.1
What is your environment & configuration (arguments, toml...)?
HypriotOS Debian Jessie on Odroid C1 ARMv7 32 Bits

docker service create
--name traefik
--constraint=node.role==manager
--publish 80:80 --publish 8080:8080
--network traefik-net
hypriot/rpi-traefik
--docker
--docker.swarmmode=true
--docker.domain=docker.localhost
--docker.watch=true
--docker.exposedbydefault=true
--web

What did you do?
Followed this tutorial step by step : https://docs.traefik.io/user-guide/swarm-mode/
What did you expect to see?
Traefik SERVICE running properly on the manager node.
What did you see instead?
The deployment never goes through and I keep seeing this error:

If applicable, please paste the log output in debug mode (--debug switch)

It seems that when I remove the --publish 80:80 --publish 8080:8080 with the --network traefik-net flags the service runs without a problem except that it's impossible to access Traefik within the browser.

docker service create
--name traefik
--constraint=node.role==manager
--mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock
bambuserver1:5000/ergu/traefik
--docker
--docker.swarmmode=true
--docker.watch=true
--docker.domain=docker.localhost
--docker.exposedbydefault=true
--web

The problem seems to be related with EXPOSING PORTS + LINKING NETWORK

This ticket is resolved by the HypriotOS Team you can set it to fixed.


Rename repository to be more generice (e.g., object-tagging object-tagging-app, visual-tagging...)

How about VOTT : Visual Object Tagging Tool

Like
Get Outlook for iOShttps://aka.ms/o0ukef
On Thu, Mar 23, 2017 at 2:56 PM +0200, "PythicCoder" <notifications<EMAIL_ADDRESS>wrote:
How about VOTT : Visual Object Tagging Tool
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCatalystCode%2FCNTK-Video-Tagging-Tool%2Fissues%2F98%23issuecomment-288710080&data=02|01|limorl%40064d.mgd.microsoft.com|f5c3303f5e38482a2a9a08d471ec14cb|72f988bf86f141af91ab2d7cd011db47|1|0|636258706157265691&sdata=d8dafr7sjZX4zhkX9%2FW%2By3SapnEnMXCSQNIqbfDao1A%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABCUbEGohDwvkoMhg7J8viF_LaYMmGmXks5romwUgaJpZM4MihU-&data=02|01|limorl%40064d.mgd.microsoft.com|f5c3303f5e38482a2a9a08d471ec14cb|72f988bf86f141af91ab2d7cd011db47|1|0|636258706157265691&sdata=QtxmO8bmEmcUMQlCwxF1R%2B74LmkSXWV1Hy4iyoH0KHA%3D&reserved=0.


Addition of TanhExp activation function - issue #45929

Addition of TanhExp activation function based on issue #45929 that leads changes to tensorflow/python/keras/activations.py and tensorflow/python/ops/nn_impl.py

@googlebot I signed it!

The API owners are moving to a rotation based system instead of the meetings and assigning several owners to follow along and review.  Adding @fchollet, @manivaradarajan

This should be opened against master, not a release branch

This should be opened against master, not a release branch

A new pull request has been created for the master branch - https://github.com/tensorflow/tensorflow/pull/47566


Feral Combat Training

Feral combat feat isn't working properly with claws given by Bloodrager/Dragon Disciple and Demon

That feat has a lot of interactions. Can you specify what's wrong?

The flurry of blows seems to still not add the extra attacks when using claws vs when using unarmed

Duplicate of this https[https://github.com/Truinto/DarkCodex/issues/35#issuecomment-1121095903]://[https://github.com/Truinto/DarkCodex/issues/35#issuecomment-1121095903]github.com[https://github.com/Truinto/DarkCodex/issues/35#issuecomment-1121095903]/[https://github.com/Truinto/DarkCodex/issues/35#issuecomment-1121095903]Truinto[https://github.com/Truinto/DarkCodex/issues/35#issuecomment-1121095903]/[https://github.com/Truinto/DarkCodex/issues/35#issuecomment-1121095903]DarkCodex[https://github.com/Truinto/DarkCodex/issues/35#issuecomment-1121095903]/issues/35#issuecomment-1121095903[https://github.com/Truinto/DarkCodex/issues/35#issuecomment-1121095903]
Flurry does indeed not work completely. But differently than you think. Feral Combat Training never grants more attacks from Flurry. Read the original link for an in depth explanation.

That explains everything 👍

Mh. Somehow my phone botched that hyperlink real good. Anyway, the only changes I would make would actually reduce the number of attacks. Not something I usually do. Will need to think about this.

Duplicate of this https://github.com/Truinto/DarkCodex/issues/35#issuecomment-1121095903
Flurry does indeed not work completely. But differently than you think. Feral Combat Training never grants more attacks from Flurry. Read the original link for in deep explanation.


Bug

I changed the line "sides: 5 + Math.round(Math.random() * 3)" to make the dots round and only 90% of the dots are round now there are a few dots that still only have like 4 to 6 sides on them am i missing something?

What did you change it to?

I believe that's what he changed it to.

I like turtles
On Wednesday, October 21, 2015, Logan7432<EMAIL_ADDRESS>wrote:

I changed the line "sides: 5 + Math.round(Math.random() * 3)" to make the
dots round and only 90% of the dots are round now there are a few dots that
still only have like 4 to 6 sides on them am i missing something?
—
Reply to this email directly or view it on GitHub
https://github.com/huytd/agar.io-clone/issues/337.

Should be fixed.


Theming?

I see a reference to "theme url" in the code, is there an example of using custom colors with a theme?

Yes, see https://themes.getbootstrap.com.


MM-30048 Implement new websocket events for collapsed threads

Documentation for: https://mattermost.atlassian.net/browse/MM-30048
Updated:

Self-Managed Admin Guide > Configure Mattermost > Configuration Settings > Experimental Settings only in config.json > Service Settings

Added new Collapsed Threads configuration setting

Hello @cwarnermm,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

Hello @cwarnermm,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

May also need to be documented in the Telemetry doc.

May also need to be documented in the Telemetry doc.

@reflog - In the Telemetry documentation, CollapsedThreads is currently documented as a bool, rather than a string. Please let me know if that's technically correct or if changes are needed. Thanks!

@reflog - In the Telemetry documentation, CollapsedThreads is currently documented as a bool, rather than a string. Please let me know if that's technically correct or if changes are needed. Thanks!

@cwarnermm - it was a boolean initially, but not it's a tri-state: "off/default_on/default_off".
off - turned off completely
default_off - each user has to opt-in
default_on - each user has to opt-out

@cwarnermm - it was a boolean initially, but not it's a tri-state: "off/default_on/default_off".
off - turned off completely
default_off - each user has to opt-in
default_on - each user has to opt-out

Thanks, @reflog -- are tri-states also documented as bools in the Telemetry documentation? (they appear to be - just want to confirm).

Thanks, @reflog -- are tri-states also documented as bools in the Telemetry documentation? (they appear to be - just want to confirm).

Yup, it's the same in telemetry

Yup, it's the same in telemetry

@cwarnermm let's hold off on any docs for PRs for collapsed reply threads for now. In fact we should probably figure out our approach to documenting the entire feature once we are closer to implementation rather than piecemeal

@cwarnermm let's hold off on any docs for PRs for collapsed reply threads for now. In fact we should probably figure out our approach to documenting the entire feature once we are closer to implementation rather than piecemeal

I'll close this for now so that it doesn't get introduced to the v5.31-documentation branch.

I'll close this for now so that it doesn't get introduced to the v5.31-documentation branch.


frint-store: Support epics

As can be seen in action here: https://redux-observable.js.org/

How to Epic in frint-store
Constants
// constants/index.js
export const PING = 'PING';
export const PONG = 'PONG';

Action Creators
// actions/pings.js
// nothing here so far

Reducers
// reducers/pings.js
import { PING, PONG } from '../constants';

const INITIAL_STATE = {
  isPinging: false,
};

export default function pings(state = INITIAL_STATE, action) {
  switch (action.type) {
    case PING:
      return {
        isPinging: true,
      };

    case PONG:
      return {
        isPinging: false,
      };

    default:
      return state;
  }
}

and root reducer:
// reducers/index.js
import { combineReducers } from 'frint-store';
import pings from './pings';

export combineReducers({
  pings: pings,
});

Epics
// epics/pings.js
import { PING, PONG } from '../constants';

// - take an incoming PING
// - wait for 1 second
// - send a PONG
export default function pingEpic$(action$, store) {
  return action$ // this can be customized to fit our needs even more, right now very raw with plain RxJS
    .filter(action => action.type === PING)
    .delay(1000)
    .map(action => ({ type: PONG }))
}

and root epic:
// epics/index.js
import { combineEpics } from 'frint-store';

import pings$ from './pings';

export default combineEpics(pings$, someOtherEpic$, ...);

Store in App
// app/index.js
import { createApp } from 'frint';
import { createStore } from 'frint-store';

import rootReducer from '../reducers';
import rootEpic from '../epics';

export default createApp({
  name: 'MyApp',

  providers: [
    {
      name: 'store',
      useFactory: function () {
        const Store = createStore({
          reducer: rootReducer,
          epic: rootEpic,
        });

        return new Store();
      }
    }
  ]
});

Dispatching
Same like before:
app.get('store').dispatch({ type: 'PING' });

Summary
Fully backwards compatible. And one key thing to understand is, Epics are not a direct replacement of Action Creators in any way. In fact, they can even work together better.
And not to mention, doing things with Epics can lead to better handling of cancellation of async operations.

Closed via #271


Incorrect colors when loading JPEG images.

When loading a JPG with get-pixels, colors are, more often than not a little bit off. Take this simple image and code, for example:

var fs = require('fs');
var getPixels = require('get-pixels');

getPixels("png_test.png", function(err, pixels) {
    for (var pixel = 0; pixel < 3; pixel++) {
        i = pixel * 4;
        console.log("R: " + pixels.data[i] + ", G: " + pixels.data[i + 1] + ", B: " + pixels.data[i + 2]);
    }
});

All it does is print out the what the RGB values of the first three pixels are, according to get-pixels. The result is:
R: 231, G: 183, B: 105
R: 212, G: 166, B: 103
R: 207, G: 162, B: 117

These colors are, in fact, all wrong. The actual colors of the first three pixels are (230, 183, 103), (211, 165, 103), (206, 162, 117). Most of the values are off by one or two! This is a bit of a problem, especially when doing color analysis. I assume it's due to a rounding error or a mistake in the JPEG interpretation algorithm somewhere.
I also tested with a PNG - the same small errors did not appear there, so I assume the problem is JPEG-specific.

It could be a bug in the JPEG decoder.  Also JPEG is a lossy format, so you shouldn't expect to get the same colors out as you put in.

Right, if so it's the decoder get-pixels uses, not the encoder whatever saved the image used. The "actual colors" I presented are the actual colors of the JPEG, when a 100% working deccoder is used.

The issue is in jpeg-js then.  If you have a better decoder let me know and I'll update the dependency.

if you need exact pixel values every time, where 1-2 off errors matter, then jpeg is the wrong format. By definition it is a lossy format, and the expectation from different codecs is a perceptually similar result (to the point the naked eye can't tell the difference), *not the excact same output every time. If a certain codec vendor decided that they could get what they consider better results by adding a random number to each pixel for some reason, that would be acceptable.
You should be using a lossless format where the exact result, pixel-to-pixel, is encoded into the data. PNG is the most likely candidate format, here.
@scijs I recommend closing this issue.

@gunderson Could you source that? I asked on StackOverflow a while back if JPEGs have canonical representation, or if the spec allows for some leeway here and there, but I never got an answer.
It's not about whether compression is lossy or not - if the same file differs between jpeg-js and more mainstream decoders, that might be a problem. Of course the JPEG spec has some limit as to how inexact representations can be - otherwise just displaying a single color and calling it "slightly inexact" would be up to spec.
If JPEGs do have an exact canonical representation, this should be fixed. If not, well, sure. Small discrepancies between decoders are fine for most cases.

Here is the actual spec https://www.w3.org/Graphics/JPEG/itu-t81.pdf
The relevant sections are 22 & 23 (pp 6,7). The requirement for any codec is only encode/decode to "appropriate accuracy". There is no canonical representation of the compressed data or decompression from that data.

@gunderson Thank you very much for the reference! Wonderful. "Appropriate accuracy", huh. I guess this is fine, then.
Oh, and it may seem like I'm being nit-picky, but even lossy formats may have rigid requirements for decoding. Even if it doesn't match the raw image, for some lossy formats there may still be an objectively correct representation.


Automatic launch of the demo application

Let's Rename _Tests folder to Tests?
Let's move BlazorApp to the Test folder?


Support for tpl function in extra env values

Hello,
Having support to use tpl on the extra env values will be of great help to support environment variables to be able to refer to external secrets resolving names from the release name, chart name or any other variable.
Thanks!!

This sounds useful, but I can't validate it by looking at it as I'm not familiar with the tpl syntax. Could you add a values yaml to the ci folder of the chart that set a regular envvar and one that uses tpl? That way we can have CI check it while also documenting.
Might also be worth mentioning in the chart Readme.

This sounds useful, but I can't validate it by looking at it as I'm not familiar with the tpl syntax. Could you add a values yaml to the ci folder of the chart that set a regular envvar and one that uses tpl? That way we can have CI check it while also documenting.
Might also be worth mentioning in the chart Readme.

@erkannt Thank you for looking into this, I have added the values file with tpl function tests and also have updated readme. Cheers!

LGTM

Thank you @erkannt , Please let me know how this makes it into main. I believe I won't be able to do it myself!

@abhilashjoseph could you please rebase?

@abhilashjoseph could you please rebase?

@pierluigilenoci, I have resolved the conflicts, should be able to merge now, thanks!

@erkannt please take a look.

Resolved conflict again as there were more main commits after my last conflict resolution.

Thank you @erkannt, I'm unable to merge this, can someone help to get this merged?

@desaintmartin I can't approve despite being a maintainer now.
Is it because this branch was forked earlier?
@abhilashjoseph this PR need a Chart version bump 😉

@desaintmartin I can't approve despite being a maintainer now.
Is it because this branch was forked earlier?
@abhilashjoseph this PR needs a Chart version bump 😉

The main chart version changed again, I'll bump it up now

@pierluigilenoci I think you need to be added as member of this repo (see the related issue).

@abhilashjoseph IMHO maybe makes sense to rebase from the master.

@abhilashjoseph IMHO maybe makes sense to rebase from the master.

@pierluigilenoci  Yes, I just rebased from master and pushed the changes, thank you!

Let me do this in the meantime.
@abhilashjoseph thanks!


Code Cleanup

removed the errant site directory
ran style-fix
ran rector-fix
ran clean

Seems to be some conflicts between what PhpCS and Rector want with = alignments. Will look into that later.


#14375: Optimize usage of unordered_map in tt::Cluster

Ticket
https://github.com/tenstorrent/tt-metal/issues/14375
Problem description
Cluster::get_driver consumes a lot of resources according to the profiler. Primarily there are two reasons: it's being called too often, and it uses an std::unordered_map where an std::vector would work
What's changed
Describe the approach used to solve the problem.
Summarize the changes made and its impact.
Checklist

[ ] Post commit CI passes
[ ] Blackhole Post commit (if applicable)
[ ] Model regression CI testing passes (if applicable)
[ ] Device performance regression CI testing passes (if applicable)
[ ] New/Existing tests provide coverage for changes

@sminakov-tt is the IndexMap here similar to the SlotMap introduced by @eyonland  and @patrickroberts ?
Just want to make sure we don't duplicate if possible
fyi @pgkeller

Sorry for the clang-tidy noise on this PR.
This PR has a stale .clang-tidy file which will be fixed once this branch is rebased.
I enabled clang-tidy today to run one simple check on using namespace;

@davorchap From what I can see, SlotMap is very different in both purpose and implementation compared to IndexMap. IndexMap is just an api wrapper over std::vector<std::optional> to mimic std::map api.

@ayerofieiev-tt I currently believe that each index is roughly in range [0; numberOfChipsInTheCluster)

@pgkeller
IndexMap is just a simple vector with optionals. The reason for optionals is that currently there are a lot of checks that the requested element is actually present either using .at() method or even more explicit ones like:
    if (this->sdesc_per_chip_.find(chip) == this->sdesc_per_chip_.end()) {
        TT_THROW(
            "Cannot access soc descriptor for {} before device driver is initialized! Call "
            "initialize_device_driver({}) first",
            chip,
            chip);
    }

And there are also some iterations, which are supposed to only iterate over existing elements, for example:
for (const auto &[chip, mmio_device_id] : this->device_to_mmio_device_)

This is why I think we need to have the optionals there, and IndexMap is just a simple wrapper to preserve map-like api.

@davorchap From what I can see, SlotMap is very different in both purpose and implementation compared to IndexMap. IndexMap is just an api wrapper over std::vectorstd::optional to mimic std::map api.

@sminakov-tt awesome! I'm glad you liked them. I'm happy to enable more checks in the main line incrementally.
The mistake I made last time, was enable everything at once.

Based on the review from @abhullar-tt, I will wait on https://github.com/tenstorrent/tt-metal/pull/13949, and then replace IndexMap with a simple std::vector in the mentioned places.
I would really appreciate if we can get similar feedback on devices_grouped_by_assoc_mmio_device_, tunnels_from_mmio_device, device_eth_routing_info_, and ethernet_sockets_
It would be really nice to just have std::vector everywhere, so we can kill IndexMap. The less code the better 🎉

@pgkeller IndexMap is just a simple vector with optionals. The reason for optionals is that currently there are a lot of checks that the requested element is actually present either using .at() method or even more explicit ones like:
    if (this->sdesc_per_chip_.find(chip) == this->sdesc_per_chip_.end()) {
        TT_THROW(
            "Cannot access soc descriptor for {} before device driver is initialized! Call "
            "initialize_device_driver({}) first",
            chip,
            chip);
    }

And there are also some iterations, which are supposed to only iterate over existing elements, for example:
for (const auto &[chip, mmio_device_id] : this->device_to_mmio_device_)

This is why I think we need to have the optionals there, and IndexMap is just a simple wrapper to preserve map-like api.

I guess I'm ok w/ this.  From 10K feet it feels like we're using IndexMap to preserve the map-like API when using a map was bad design to begin with.  So, is this a bandaid to easy transition or is there value to how these are being used?


[Tree] Is there something broken with the tree?

Hi
I would like to start using the tree behaviour but I think there is something not right ATM.
In the repository, root entries have an empty materialized path:

    public function getRootNodesQB($rootAlias = 't')
    {
        return $this->createQueryBuilder($rootAlias)
            ->andWhere($rootAlias.'.materializedPath = :empty')
            ->setParameter('empty', '')
        ;
    }

In the Node, the root node has the separator:

    public function isRootNode()
    {
        return self::getMaterializedPathSeparator() === $this->getParentMaterializedPath();
    }

What is right and how does this all work?
Also, when I fetch a Node and asks it to getParentNode(), it always returns null.
Maybe I got something wrong but at least the first example ( root == '' and root == '/') seems to be completely broken. Can you please look at it?
Greetings
Thomas

hi!
It's definitely possible something is wrong with path comparison. I have however not planned to look at this problem, until I have the need to.
If you can provide a fix for that, it would be awesome!
Concerning:

Also, when I fetch a Node and asks it to getParentNode(), it always returns null.

you have to build the tree first. Fetching a node by itself won't populate the parent and children until you pass by the buildTree method.

I'm going through the old issues/PRs and closing most of them so we can focus on now.
This one is missing any feedback or reasoning so I cannot deduce anything from it.
Thanks for you though, but closing for reasons above.
If the bug still remains, please send failing test case to verify it. Only with test we'll be albe to help.


Add resolutions to package.json type definition

Technically workspace packages can have resolutions too, not just root level package.json's.
Ran into this missing when trying to use @manypkg/get-packages to clean up workspace resolutions

Ran into this missing when trying to use @manypkg/get-packages to clean up workspace resolutions

Does manypkg depend on this type from the @changesets/type package?

@Andarist Correct, so if I do
const { tool, root, packages } = await getPackages(process.cwd());
packages.map(pkg => pkg.packageJson.resolutions);

I get a typecheck error since resolutions is not on the interface, this PR addresses that


[develop]  added eosio-blocklog options to take last good blocks

Change Description

eosio-blocklog --trim-blocklog allows you to take a block.log + block.index and repair it to a usable situation.  You need to know what is a good block to repair to.  If you choose a block in the future, it fails.
New options:
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --help
--check-files               Check  blocks.index and blocks.log.
Must give 'blocks-dir'.
--update-index           Update blocks.index with the best
effort to avoid creating a new index
file. Must give 'blocks-dir'. Give
'output-file' relative to current
directory or absolute path (default is
/blocks.index).
--fix-corrupted-file     fix the corrupted end of blocks.log and
blocks.index files by trimming the
incomplete block. Must give
'blocks-dir'.
Example)
~/tmp/blocks> ls -al
-rw-r--r--  1 jongho.kim staff  128 Jun  9 13:52 blocks.index
-rw-r--r--  1 jongho.kim staff 3198 Jun  9 13:47 blocks.log
drwxr-xr-x  3 jongho.kim staff   96 Jun  9 13:47 reversible
// truncate blocks.log
~/tmp/blocks> truncate -s 3000 blocks.log
~/tmp/blocks> ls -al
-rw-r--r--  1 jongho.kim staff  128 Jun  9 13:52 blocks.index
-rw-r--r--  1 jongho.kim staff 3000 Jun  9 14:27 blocks.log
drwxr-xr-x  3 jongho.kim staff   96 Jun  9 13:47 reversible
// smoke-test crashes because of the corrupted file
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --smoke-test
Smoke test of blocks.log and blocks.index in directory "/Users/jongho.kim/tmp/blocks"
zsh: segmentation fault  ./eosio-blocklog --blocks-dir ~/tmp/blocks --smoke-test
~/contracts/eos/build/programs/eosio-blocklog>
// check-file got the assertion
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --check-files
Check blocks.index and blocks.log in "/Users/jongho.kim/tmp/blocks"
info  2020-06-09T18:29:08.512 thread-0  block_log.cpp:673             check_corrupted_bloc ]  blocks.index file looks good with size of 128 bytes.
info  2020-06-09T18:29:08.513 thread-0  block_log.cpp:754             check_corrupted_bloc ] Found incomplete 186 bytes in blocks.log file
error 2020-06-09T18:29:08.513 thread-0  main.cpp:381                  main                 ] 3190000 block_log_exception: Block log exception
corrupted blocks.log/blocks.index.
{}
thread-0  block_log.cpp:598 check_files
// fix-corrupted-file fixed the blocks.log
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --fix-corrupted-file
Fix corrupted blocks.log and blocks.index files in "/Users/jongho.kim/tmp/blocks"
info  2020-06-09T18:30:02.781 thread-0  block_log.cpp:673             check_corrupted_bloc ]  blocks.index file looks good with size of 128 bytes.
info  2020-06-09T18:30:02.781 thread-0  block_log.cpp:754             check_corrupted_bloc ] Found incomplete 186 bytes in blocks.log file
info  2020-06-09T18:30:02.781 thread-0  block_log.cpp:758             check_corrupted_bloc ] Fixed blocks.log file by trimming 186 bytes.
no problems found
// check-file still asserts because of blocks.index not updated
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --check-files
Check blocks.index and blocks.log in "/Users/jongho.kim/tmp/blocks"
info  2020-06-09T18:30:38.378 thread-0  block_log.cpp:673             check_corrupted_bloc ]  blocks.index file looks good with size of 128 bytes.
info  2020-06-09T18:30:38.378 thread-0  block_log.cpp:747             check_corrupted_bloc ] blocks.log file looks good with size of 2814 bytes.
info  2020-06-09T18:30:38.378 thread-0  block_log.cpp:658             check_corrupted_file ] No corrupted block or index found!
info  2020-06-09T18:30:38.378 thread-0  block_log.cpp:608             check_files          ] blocks.log file /Users/jongho.kim/tmp/blocks/blocks.log has 14 blocks
info  2020-06-09T18:30:38.378 thread-0  block_log.cpp:609             check_files          ] blocks.index file /Users/jongho.kim/tmp/blocks/blocks.index has 16 blocks
error 2020-06-09T18:30:38.378 thread-0  main.cpp:381                  main                 ] 3190000 block_log_exception: Block log exception
Need to update index...
{}
thread-0  block_log.cpp:633 check_files
// smoke-test failed, too.
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --smoke-test
Smoke test of blocks.log and blocks.index in directory "/Users/jongho.kim/tmp/blocks"
error 2020-06-09T18:31:35.742 thread-0  main.cpp:381                  main                 ] 3190000 block_log_exception: Block log exception
/Users/jongho.kim/tmp/blocks/blocks.log says it has 14 blocks which disagrees with 16 indicated by /Users/jongho.kim/tmp/blocks/blocks.index
{"block_file_name":"/Users/jongho.kim/tmp/blocks/blocks.log","log_num_blocks":14,"index_num_blocks":16,"index_file_name":"/Users/jongho.kim/tmp/blocks/blocks.index"}
thread-0  block_log.cpp:456 block_log_archive
// update-index
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --update-index
Update blocks.index to blocks.log in "/Users/jongho.kim/tmp/blocks"
info  2020-06-09T18:32:21.662 thread-0  block_log.cpp:539             fix_index_file       ] Log is nonempty
info  2020-06-09T18:32:21.663 thread-0  block_log.cpp:546             fix_index_file       ] Index is nonempty
info  2020-06-09T18:32:21.663 thread-0  block_log.cpp:554             fix_index_file       ] The last block positions from blocks.log and blocks.index are different, Reconstructing index...
info  2020-06-09T18:32:21.663 thread-0  block_log.cpp:557             fix_index_file       ] Need to update index file
no problems found
// check-files ok
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --check-files
Check blocks.index and blocks.log in "/Users/jongho.kim/tmp/blocks"
info  2020-06-09T18:32:26.275 thread-0  block_log.cpp:673             check_corrupted_bloc ]  blocks.index file looks good with size of 112 bytes.
info  2020-06-09T18:32:26.275 thread-0  block_log.cpp:747             check_corrupted_bloc ] blocks.log file looks good with size of 2814 bytes.
info  2020-06-09T18:32:26.275 thread-0  block_log.cpp:658             check_corrupted_file ] No corrupted block or index found!
info  2020-06-09T18:32:26.275 thread-0  block_log.cpp:608             check_files          ] blocks.log file /Users/jongho.kim/tmp/blocks/blocks.log has 14 blocks
info  2020-06-09T18:32:26.275 thread-0  block_log.cpp:609             check_files          ] blocks.index file /Users/jongho.kim/tmp/blocks/blocks.index has 14 blocks
info  2020-06-09T18:32:26.275 thread-0  block_log.cpp:635             check_files          ] Index is up-to-date
no problems found
// smoke-test ok
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --smoke-test
Smoke test of blocks.log and blocks.index in directory "/Users/jongho.kim/tmp/blocks"
info  2020-06-09T18:32:28.436 thread-0  block_log.cpp:1225            smoke_test           ] blocks.log and blocks.index agree on number of blocks
no problems found
// 2 blocks truncated
~/tmp/blocks> ls -al
-rw-r--r--  1 jongho.kim staff  112 Jun  9 14:32 blocks.index
-rw-r--r--  1 jongho.kim staff 2814 Jun  9 14:30 blocks.log
Change Type
Select ONE

[ ] Documentation

[ ] Stability bug fix

[ ] Other

[ ] Other - special case

Consensus Changes

[ ] Consensus Changes

API Changes

[ ] API Changes

Documentation Additions

[ ] Documentation Additions

Modified to use blocks.index to repair damaged blocks.log.

Duplicate fix implemented in PR
https://github.com/EOSIO/eos/pull/9184


Update gemspec, add rspec, add in meta_instance

Still not 100% coverage, but close.

Hi @NullVoxPopuli !
I have good news and I have bad news.  The good news is that, with my cursory overview of your pull request, there is some great changes in there. I would love to see them worked in. The bad news is that some of the changes I can't accept, primary among them is dependency on ActiveSupport. This library is intended to be lowish level, so a huge dependency like that just doesn't blend.
But I am willing to work with you, if you would like to discuss the various changes and resubmit them in smaller more imaginable pull requests. I am even willing to turn over general maintenance if we can come to terms on a few of basic tenants. That means you will probably have to do a couple of things in ways you are not accustomed. But likewise I have no problems with making accommodations for things I am not accustomed to as well, e.g. I have never used codeclimate.
It's never been my intention not to maintain this library, it's just been a lower priority and I have many other things going on -- you know how it is.
Of course you are welcome to fork the project as "meta_instance" and do as you like with it. You are even welcome to change the license to MIT if you wish, just please keep my in the copyrights and a mention of the original project in the README would be great.
Let me know how you'd like to proceed.

Hey! thanks for the reply!
I'm only using ActiveSupport's Core Extensions for the delegate functionality. It cleans up a lot of code, however, it, and possibly a few other things break compatibility with pre-ruby 2 (and the non MRI rubies)
I totally get the smaller pull request thing. Small pull requests are awesome - easier to manage and review, etc. Most of the diff is just renaming / re-organizing files (which may not have been required to get rspec / the new gem format working -- but multiple modules / classes in the same file drive me bonkers).
I've actually already released the gem, overriding a gem that I previously had also by the name meta instance.
The released gem is here: https://rubygems.org/gems/meta_instance
you are listed as an author, as you still wrote most of the code. :-)
I don't know what the best course of action is - cause the Instance gem had compatibility with like.. 6 interpreters or something.
I'd like to eventually clean it up a bit more - finish implementing rspec tests, etc.

Cleaning up my PRs. If this work is still desired, feel free to cherry-pick


how do we revert Wakala Escrow payments?

Let#s say an agent accepts a request, money is sent to escrow.
If something breaks, or the user never confirms, how and when do we send the money back?

We have the breakdown of the transacted cash/USD within the transaction object.
On the smart contract side what can happen is basically send all the money back to the client if it's a withdrawal transaction.
Send all the  USD cash to the agent if it's a deposit.
This should only be done if the transaction has not been confirmed by any of the two.
Once it`s confirmed by any of the two we have a game theory problem. We do not know who is telling the truth to cancel.
This brings us to the questions.

Who is allowed to cancel??
Should we ask them why they are canceling? And should we store this information on chain??
When is it too late to cancel the transaction?
How do we handle fraud??  i.e An agent can cancel a withdrawal transaction once the client has sent him/her fiat outside Wakala.
@steffenkrogmann, @ClaireLagat , @Chegemartinx


Roadblocks in converting from Hugo Academic

Hi @apreshill!
Thanks for outlining some steps on Twitter on how to approach converting to Apéro from Hugo Academic. I followed the steps (listed below) but am unable to render the site using blogdown::serve_site() 😢
Steps followed, with corresponding commit in my own repo:

Install theme alongside Academic (https://github.com/spcanelon/silvia/commit/cc5d24d93676990675abc52145fd7a369c7bffa6), change in config.toml(https://github.com/spcanelon/silvia/commit/499cc959a947761b4dbc518f5a1bf6312527b517)
Copy all Academic shortcodes to layouts/ root (https://github.com/spcanelon/silvia/commit/f3c7d5334b4effbd850b204eb267425f6740b4af)
Remove all assets (https://github.com/spcanelon/silvia/commit/3843c76a5da6184b2d9b547b18f96ec6810a695a)
Remove all custom layouts (https://github.com/spcanelon/silvia/commit/1ad7e3d491d309e71e1b4fa0bbad1e3af6b9d322)
Copy over Apéro example site config.toml file (https://github.com/spcanelon/silvia/commit/db37289e768640522130f98353996de4a6e0abfc)
Remove Academic config directory (https://github.com/spcanelon/silvia/commit/5541f38871911d5067c6c8856936d54d183b3ec9)

The error I'm stuck on at the moment is the following:

Error: Error building site: failed to render pages: render of "page" failed: execute of template failed: template: _default/single.html:3:8: executing "_default/single.html" at <partial "head.html" .>: error calling partial: "/Users/scanelon/Documents/Repos/Website/silvia/themes/hugo-apero/layouts/partials/head.html:14:53": execute of template failed: template: partials/head.html:14:53: executing "partials/head.html" at : can't evaluate field Build in type string

I'm understanding this to mean there's an issue with using the templates in themes/hugo-apero, but I'm not sure why. I'm sure I missed something along the way 😅
Any pointers you have would be much appreciated!

I tried to use the RStudio new blowdown site feature or calling new_site() with hugo-apero/apero as the theme and got exactly the same error. I have an existing blog that is using beautifulhugo, but that is in a different project. I get exactly the same error as described here, except it refers to my new apes project folder. So this does not relate specifically to converting from Academic.

Progress: rather than go the blowdown::new_site() route, I cloned https://github.com/hugo-apero/apero onto my Mac. After doing blogdown::install_hugo("0.79.0") (from version 0.59) I was successful at getting the site to start up via serve_site(). At this point I'm not sure whether the key was simply to upgrade to 0.79 or whether I also needed to clone the repo rather than use blowdown::new_site(), but in any case I can move on from here to see where I fail next. Also I supposed I better go back to my production web site to see whether I have broken it after the Hugo upgrade.

Thanks for sharing @johngoldin! I am currently on Hugo v0.69.2 so your success with updating to v0.79 gives me hope. I'll try that next and report back.

Thanks for the guidance @johngoldin and @apreshill, updating the Hugo version worked! Here are the steps I took:

blogdown::install_hugo() -- installed latest version of Hugo (v0.82.1)
Updated .Rprofile per instructions, and restarted R
Updated netlify.toml
Updated Hugo version on Netlify.com
blogdown::check_site() -- suggested I use blogdown::install_hugo("0.82.1") so I did, but I'm not sure why I needed to do it again. I made sure to restart R here also.
blogdown::check_site() -- all checks passed
blogdown::serve_site() -- site preview worked! 🎉


ZFS Pool IO stats are always 0%

After adding ZFS pools to my disks to monitor, the storage is represented but not IO Stats like "normal" disks.  Presume this is due to the ZFS pool being represented as a group and not a single disk.

No expert but believe this is due to iostats being captured separate on ZFS. I only know of zpool iostat to produce metrics that show operations and bandwidth.
$ zpool iostat pool02
              capacity     operations     bandwidth
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
pool02      9.03T  7.33T     26      1  8.86M   331K

Understanding is that the metrics is an average since boot of machine but can have flags -y TIME (count) to pick an average across a time frame and the number of times to capture (otherwise indefinite). Additionally I can get no headers to format and then use awk to get just the bandwidth metrics
$ zpool iostat pool02 -H -y 1 1 | awk '{print $6, $7}'
0 0

One unfortunate behavior I have seen is that the time indicated is the start of the averaging so if you were to want 60 second averages it would need to run for 60 seconds (makes sense) but we cannot get a snapshot otherwise, meaning if 1 second was gathered which happened to be no activity may not represent the best metrics.
Would be nice to have ZFS stats if possible given my OCD and seeing the persistent 0 stat graphs for multiple pools. Alternatively if there was a way to filter those as well, that would be awesome. Not sure if there is the ability to tell the agent which stats to capture or not.
Love the project, thank you.

I don't use ZFS so I probably won't be able to add any direct integration with ZFS utilities.
Do the devices that make up the pool show up if you run cat /proc/diskstats?
Maybe as a workaround I can add a way for you to map the sum total I/O of multiple devices to the pool. Something like pool02_IO=sdc,sdd,sde.

Thanks. Yep looks like I can get the individual disk stats with `cat /proc/diskstats'
cat /proc/diskstats
 259       0 nvme1n1 1506586 0 228868910<PHONE_NUMBER>01417 0<PHONE_NUMBER> 44571380 0 42303316 60623102 103422 0<PHONE_NUMBER><PHONE_NUMBER>452 15526537
 259       2 nvme1n1p1 1506442 0 228864038<PHONE_NUMBER>01417 0<PHONE_NUMBER> 44571380 0 42303284 45096543 103422 0<PHONE_NUMBER> 107140 0 0
 259       3 nvme1n1p9 38 0 304 2 0 0 0 0 0 16 2 0 0 0 0 0 0
 259       1 nvme2n1 1511239 0 229581638<PHONE_NUMBER>09248 0<PHONE_NUMBER> 38301785 0 39336024 51438886 103419 0<PHONE_NUMBER><PHONE_NUMBER>491 12601546
 259       4 nvme2n1p1 1511095 0 229576766<PHONE_NUMBER>09248 0<PHONE_NUMBER> 38301785 0 39335996 38837320 103419 0<PHONE_NUMBER> 115364 0 0
 259       5 nvme2n1p9 38 0 304 0 0 0 0 0 0 8 0 0 0 0 0 0 0
 259       6 nvme0n1<PHONE_NUMBER>46 28280576<PHONE_NUMBER>5350 15793242 526448866 63327031 0 33930024 66396994 101187 0<PHONE_NUMBER> 66803 1090594 2631275
 259       7 nvme0n1p1 195 6327 10638 477 2 0 2 8 0 164 509 8 0 8276416 23 0 0
 259       8 nvme0n1p2<PHONE_NUMBER>9 28244114<PHONE_NUMBER>4640 15792368 526436240 63321656 0 33924352 63759764 101179 0<PHONE_NUMBER> 66780 0 0
 259       9 nvme0n1p3 285 20 6472 40<PHONE_NUMBER>4 5363 0 6652 5403 0 0 0 0 0 0
   8       0 sda 50991523 2801<PHONE_NUMBER>8 302941532 2848924 54384<PHONE_NUMBER> 27799594 0 123105296 331867050 0 0 0 0 43246 1125923
   8       1 sda1 50991369 2801<PHONE_NUMBER>8 302941193 2848924 54384<PHONE_NUMBER> 27799594 0 123104976 330740788 0 0 0 0 0 0
   8       9 sda9 38 0 304 61 0 0 0 0 0 72 61 0 0 0 0 0 0
   8      48 sdd 71421488 3047<PHONE_NUMBER>2 250393782 2826984 66182<PHONE_NUMBER> 26877846 0 122065348 278372007 0 0 0 0 43246 1100378
   8      49 sdd1 71421334 3047<PHONE_NUMBER>2 250393448 2826984 66182<PHONE_NUMBER> 26877846 0 122065048 277271295 0 0 0 0 0 0
   8      57 sdd9 38 0 304 60 0 0 0 0 0 72 60 0 0 0 0 0 0
   8      32 sdc 2859360 84436 145041618 21616561 27455244 4334787<PHONE_NUMBER> 720119264 0 115697556 831925478 0 0 0 0 2574871 90189652
   8      33 sdc1 2859246 84436 145037058 21616342 27455244 4334787<PHONE_NUMBER> 720119264 0 115697356 741735606 0 0 0 0 0 0
   8      16 sdb<PHONE_NUMBER>4 202385242<PHONE_NUMBER>9326 4888077<PHONE_NUMBER> 22663024 0 10797204 24158455 22012 0<PHONE_NUMBER><PHONE_NUMBER>74 446252
   8      17 sdb1<PHONE_NUMBER>4 202380906<PHONE_NUMBER>9326 4888077<PHONE_NUMBER> 22663024 0 10797164 23712180 22012 0<PHONE_NUMBER> 157152 0 0

Disks in ZFS pools (in output above)
lsblk -o NAME,FSTYPE -dsn | grep -A 1 'zfs'
sda1      zfs_member
└─sda
--
sdd1      zfs_member
└─sdd
--
nvme1n1p1 zfs_member
└─nvme1n1
--
nvme2n1p1 zfs_member
└─nvme2n1

The summation is not a bad idea if not too much effort.

Great, thanks for confirming. I'll add this to the list.
I do need to work on a different project for a bit to pay the bills so it will take some time.

Awesome and totally understand, even as it stands today this has been one of my favorite additions to my homelab environment so thanks again.

If I my ask @kernelkaribou, how did you reference your zfs pool in your docker compose? By directory or device-name (and if so, what device)?
I get 0 data when using the directory method.
Just wanted to create a new discussion for this before found your issue.

@kernelkaribou I would also appreciate it if you could share your zfs monitoring configuration for Beszel? Thanks!

Not the OP, but the directory method is working for me.  For example from my docker compose:

/home is a zfs mounted volume, and /home/.beszel is a directory created there.

Not the OP, but the directory method is working for me. For example from my docker compose:

/home is a zfs mounted volume, and /home/.beszel is a directory created there.

Thanks for this. Do you know if it possible to monitor free space rather than space used?

@jerias When you say that its working for you, are you seeing the stats as well or just that its presenting the storage space.
I have never done the .beszel method and both before and after adding that it still only shows the storage usage.

Sorry, I should have been more clear.  My response was directed at @theAlevan and @2600box .   The directory method is working for me for providing the usage data only.  The I/O stats would require the proposed change from @henrygd mentioned previously.

Understood, I was assuming as much. I had never used the method of a .beszel directory, just the main directory and it worked fine (for storage usage) up until this point and thus the confusion. Hopefully the fix idea is something that can be made available in the future.

I am not using ZFS but LUKS + LVM and I have a similar issue:
df relevant output
/dev/mapper/vg_crypt-lv_data       7.3T  4.1T  2.9T  59% /mnt/encrypted_lvm

docker compose
 beszel-agent:
    image: 'henrygd/beszel-agent'
    container_name: 'beszel-agent'
    restart: unless-stopped
    network_mode: host
    volumes:
      - /mnt/usb-samsung-hd:/extra-filesystems/sdb:ro
      - /mnt/encrypted_lvm:/extra-filesystems/encrypted_lvm:ro
    environment:
      EXTRA_FILESYSTEMS: encrypted_lvm

blkid relevant output
/dev/sdd: UUID="7e1e48e0-fa7b-42ba-bfc3-f1f05f1d5e2d" TYPE="crypto_LUKS"
/dev/mapper/encrypted_sdc: UUID="6AYeP1-8A0n-FVBY-Bsgg-4A4j-uasY-84Pfzr" TYPE="LVM2_member"
/dev/mapper/encrypted_sdd: UUID="bImlfD-Ghfm-CO2c-xfiI-S5Sq-f7pC-lIyLIH" TYPE="LVM2_member"
/dev/mapper/vg_crypt-lv_data: UUID="233069af-5d0d-4392-9a96-02f14b3b32c1" BLOCK_SIZE="4096" TYPE="ext4"

Can I do anything to fix this on my side?

@rcbop You need to figure out associated device / partition that has an entry in /proc/diskstats.
Then rename mountpoint within the container to use that name. For example, if it's sdc1, do this:
volumes:
   - /mnt/encrypted_lvm:/extra-filesystems/sdc1:ro

Try running lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINT. That should show you the associated luks-UUID and device, hopefully.
You can also try running sudo cryptsetup status luks-UUID to find the device.
If you're still unsure, paste the output of those commands here. Also restart the agent with the env var LOG_LEVEL=debug. That will log a lot more stuff. Find the lines near the top starting with Disk partitions= and DEBUG Disk I/O diskstats= and include those here as well.

lsblk output
NAME                        SIZE TYPE  FSTYPE      MOUNTPOINT
sda                       238.5G disk
├─sda1                        1M part
├─sda2                        2G part  ext4        /boot
└─sda3                    236.5G part  LVM2_member
  └─ubuntu--vg-ubuntu--lv   100G lvm   ext4        /
sdb                       465.8G disk
└─sdb1                    465.8G part  exfat       /mnt/usb-samsung-hd
sdc                         3.6T disk  crypto_LUKS
├─sdc1                    882.7G part
└─encrypted_sdc             3.6T crypt LVM2_member
  └─vg_crypt-lv_data        7.3T lvm   ext4        /mnt/encrypted_lvm
sdd                         3.6T disk  crypto_LUKS
└─encrypted_sdd             3.6T crypt LVM2_member
  └─vg_crypt-lv_data        7.3T lvm   ext4        /mnt/encrypted_lvm

cat /proc/diskstats
   8       0 sda 72977920 26162919<PHONE_NUMBER> 19396798 43680399 85917731<PHONE_NUMBER> 51872822 0 20275899 75073332 80242 0 238096560 50075 6086187 3753636
   8       1 sda1 487 0 14512 83 0 0 0 0 0 90 83 0 0 0 0 0 0
   8       2 sda2<PHONE_NUMBER>8 278<PHONE_NUMBER>20 3149 0 1393 3547 42 0 3935552 119 0 0
   8       3 sda3 72975675 26162590<PHONE_NUMBER> 19396261 43679764 85917479<PHONE_NUMBER> 51869672 0 22414111 71315889 80200 0 234161008 49955 0 0
 252       0 dm-0 99136660 0<PHONE_NUMBER> 27363082 129567573 0<PHONE_NUMBER> 250945012 0 26448411 278357436 80200 0 234161008 49342 0 0
   8      16 sdb 13546933 1092382<PHONE_NUMBER> 49147361 3966328<PHONE_NUMBER>58173 15857979 0 38347019 65005341 0 0 0 0 0 0
   8      17 sdb1 13546772 1092382<PHONE_NUMBER> 49145572 3966328<PHONE_NUMBER>58173 15857979 0 39288646 65003551 0 0 0 0 0 0
   7       8 loop8 326 0 11192 120 0 0 0 0 0 84 120 0 0 0 0 0 0
   8      32 sdc 34603913<PHONE_NUMBER>667593 333291523 16901115 1389077<PHONE_NUMBER>0 279176947 0 124368334 626486923 0 0 0 0<PHONE_NUMBER>8453
   8      33 sdc1 393 0 30984 970 0 0 0 0 0 907 970 0 0 0 0 0 0
   8      48 sdd 7996243<PHONE_NUMBER>049476 135487935 10870719 3187770<PHONE_NUMBER> 172090710 0 61493600 320210541 0 0 0 0<PHONE_NUMBER>1895
 252       1 dm-1 35384226 0<PHONE_NUMBER> 446557138 12439606 0<PHONE_NUMBER>2 619691027 0 125441401<PHONE_NUMBER> 0 0 0 0 0 0
 252       2 dm-2 8351368 0<PHONE_NUMBER> 194770340 10627039 0<PHONE_NUMBER><PHONE_NUMBER> 0 62919797<PHONE_NUMBER> 0 0 0 0 0 0
 252       3 dm-3 43733999 0<PHONE_NUMBER> 641487717 16351549 0<PHONE_NUMBER>0<PHONE_NUMBER> 0 178153655<PHONE_NUMBER> 0 0 0 0 0 0

I tried monitoring the disks individually by mounting in the docker agent /dev/sdc and /dev/sdd but it didn't worked. The metrics vanished from the dashboard.
beszel-agent  | 2024/12/14 16:02:35 DEBUG Disk I/O diskstats="map[dm-0:{\"readCount\":99137191,\"mergedReadCount\":0,\"writeCount\":129572428,\"mergedWriteCount\":0,\"readBytes\":664000070656,\"writeBytes\":601138872320,\"readTime\":27363653,\"writeTime\":250948542,\"iopsInProgress\":0,\"ioTime\":26453793,\"weightedIO\":278361537,\"name\":\"dm-0\",\"serialNumber\":\"\",\"label\":\"ubuntu--vg-ubuntu--lv\"} dm-1:{\"readCount\":35384309,\"mergedReadCount\":0,\"writeCount\":12439606,\"mergedWriteCount\":0,\"readBytes\":4041527762944,\"writeBytes\":6416360955904,\"readTime\":446557651,\"writeTime\":619691027,\"iopsInProgress\":0,\"ioTime\":125441889,\"weightedIO\":1066248678,\"name\":\"dm-1\",\"serialNumber\":\"\",\"label\":\"encrypted_sdc\"} dm-2:{\"readCount\":8351518,\"mergedReadCount\":0,\"writeCount\":10627039,\"mergedWriteCount\":0,\"readBytes\":859668082688,\"writeBytes\":3789827194880,\"readTime\":194771958,\"writeTime\":2868384134,\"iopsInProgress\":0,\"ioTime\":62921401,\"weightedIO\":3063156092,\"name\":\"dm-2\",\"serialNumber\":\"\",\"label\":\"encrypted_sdd\"} dm-3:{\"readCount\":43734232,\"mergedReadCount\":0,\"writeCount\":16351549,\"mergedWriteCount\":0,\"readBytes\":4901144899584,\"writeBytes\":10203931566080,\"readTime\":641489849,\"writeTime\":2168006850,\"iopsInProgress\":0,\"ioTime\":178155738,\"weightedIO\":2809496699,\"name\":\"dm-3\",\"serialNumber\":\"\",\"label\":\"vg_crypt-lv_data\"} loop0:{\"readCount\":352,\"mergedReadCount\":0,\"writeCount\":0,\"mergedWriteCount\":0,\"readBytes\":3835904,\"writeBytes\":0,\"readTime\":96,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":102,\"weightedIO\":96,\"name\":\"loop0\",\"serialNumber\":\"\",\"label\":\"\"} loop1:{\"readCount\":774,\"mergedReadCount\":0,\"writeCount\":0,\"mergedWriteCount\":0,\"readBytes\":7830528,\"writeBytes\":0,\"readTime\":185,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":174,\"weightedIO\":185,\"name\":\"loop1\",\"serialNumber\":\"\",\"label\":\"\"} loop2:{\"readCount\":256,\"mergedReadCount\":0,\"writeCount\":0,\"mergedWriteCount\":0,\"readBytes\":6078464,\"writeBytes\":0,\"readTime\":78,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":69,\"weightedIO\":78,\"name\":\"loop2\",\"serialNumber\":\"\",\"label\":\"\"} loop3:{\"readCount\":486,\"mergedReadCount\":0,\"writeCount\":0,\"mergedWriteCount\":0,\"readBytes\":10618880,\"writeBytes\":0,\"readTime\":209,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":130,\"weightedIO\":209,\"name\":\"loop3\",\"serialNumber\":\"\",\"label\":\"\"} loop4:{\"readCount\":479,\"mergedReadCount\":0,\"writeCount\":0,\"mergedWriteCount\":0,\"readBytes\":10522624,\"writeBytes\":0,\"readTime\":211,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":116,\"weightedIO\":211,\"name\":\"loop4\",\"serialNumber\":\"\",\"label\":\"\"} loop5:{\"readCount\":376,\"mergedReadCount\":0,\"writeCount\":0,\"mergedWriteCount\":0,\"readBytes\":4014080,\"writeBytes\":0,\"readTime\":98,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":93,\"weightedIO\":98,\"name\":\"loop5\",\"serialNumber\":\"\",\"label\":\"\"} loop6:{\"readCount\":496,\"mergedReadCount\":0,\"writeCount\":0,\"mergedWriteCount\":0,\"readBytes\":9672704,\"writeBytes\":0,\"readTime\":190,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":128,\"weightedIO\":190,\"name\":\"loop6\",\"serialNumber\":\"\",\"label\":\"\"} loop7:{\"readCount\":1893,\"mergedReadCount\":0,\"writeCount\":0,\"mergedWriteCount\":0,\"readBytes\":68467712,\"writeBytes\":0,\"readTime\":428,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":424,\"weightedIO\":428,\"name\":\"loop7\",\"serialNumber\":\"\",\"label\":\"\"} loop8:{\"readCount\":326,\"mergedReadCount\":0,\"writeCount\":0,\"mergedWriteCount\":0,\"readBytes\":5730304,\"writeBytes\":0,\"readTime\":120,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":84,\"weightedIO\":120,\"name\":\"loop8\",\"serialNumber\":\"\",\"label\":\"\"} sda:{\"readCount\":72978298,\"mergedReadCount\":26163072,\"writeCount\":43683178,\"mergedWriteCount\":85919807,\"readBytes\":664176902144,\"writeBytes\":601434050560,\"readTime\":19397134,\"writeTime\":51874709,\"iopsInProgress\":0,\"ioTime\":20276689,\"weightedIO\":75075899,\"name\":\"sda\",\"serialNumber\":\"\",\"label\":\"\"} sda1:{\"readCount\":487,\"mergedReadCount\":0,\"writeCount\":0,\"mergedWriteCount\":0,\"readBytes\":7430144,\"writeBytes\":0,\"readTime\":83,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":90,\"weightedIO\":83,\"name\":\"sda1\",\"serialNumber\":\"\",\"label\":\"\"} sda2:{\"readCount\":856,\"mergedReadCount\":329,\"writeCount\":635,\"mergedWriteCount\":252,\"readBytes\":41235456,\"writeBytes\":295178240,\"readTime\":278,\"writeTime\":3149,\"iopsInProgress\":0,\"ioTime\":1393,\"weightedIO\":3547,\"name\":\"sda2\",\"serialNumber\":\"\",\"label\":\"\"} sda3:{\"readCount\":72976053,\"mergedReadCount\":26162743,\"writeCount\":43682543,\"mergedWriteCount\":85919555,\"readBytes\":664100058112,\"writeBytes\":601138872320,\"readTime\":19396596,\"writeTime\":51871560,\"iopsInProgress\":0,\"ioTime\":22414943,\"weightedIO\":71318113,\"name\":\"sda3\",\"serialNumber\":\"\",\"label\":\"\"} sdb:{\"readCount\":13546933,\"mergedReadCount\":1092382,\"writeCount\":3966328,\"mergedWriteCount\":212087,\"readBytes\":659745610240,\"writeBytes\":456221784576,\"readTime\":49147361,\"writeTime\":15857979,\"iopsInProgress\":0,\"ioTime\":38347019,\"weightedIO\":65005341,\"name\":\"sdb\",\"serialNumber\":\"\",\"label\":\"\"} sdb1:{\"readCount\":13546772,\"mergedReadCount\":1092382,\"writeCount\":3966328,\"mergedWriteCount\":212087,\"readBytes\":659743070720,\"writeBytes\":456221784576,\"readTime\":49145572,\"writeTime\":15857979,\"iopsInProgress\":0,\"ioTime\":39288646,\"weightedIO\":65003551,\"name\":\"sdb1\",\"serialNumber\":\"\",\"label\":\"\"} sdc:{\"readCount\":34603996,\"mergedReadCount\":781774,\"writeCount\":16901115,\"mergedWriteCount\":1389077,\"readBytes\":4041569853952,\"writeBytes\":6416464015360,\"readTime\":333292021,\"writeTime\":279176947,\"iopsInProgress\":0,\"ioTime\":124368812,\"weightedIO\":626487421,\"name\":\"sdc\",\"serialNumber\":\"\",\"label\":\"\"} sdc1:{\"readCount\":393,\"mergedReadCount\":0,\"writeCount\":0,\"mergedWriteCount\":0,\"readBytes\":15863808,\"writeBytes\":0,\"readTime\":970,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":907,\"weightedIO\":970,\"name\":\"sdc1\",\"serialNumber\":\"\",\"label\":\"\"} sdd:{\"readCount\":7996393,\"mergedReadCount\":356096,\"writeCount\":10870719,\"mergedWriteCount\":3187770,\"readBytes\":859691591680,\"writeBytes\":3789930254336,\"readTime\":135489526,\"writeTime\":172090710,\"iopsInProgress\":0,\"ioTime\":61495160,\"weightedIO\":320212132,\"name\":\"sdd\",\"serialNumber\":\"\",\"label\":\"\"}]"
beszel-agent  | 2024/12/14 16:02:35 WARN Partition details not found filesystem=/dev/sda1
beszel-agent  | 2024/12/14 16:02:35 ERROR Invalid filesystem name=sdc err="no such file or directory"
beszel-agent  | 2024/12/14 16:02:35 ERROR Invalid filesystem name=sdd err="no such file or directory"

@rcbop Thanks, I think dm-3 is what you want. Try this:
volumes:
   - /mnt/encrypted_lvm:/extra-filesystems/dm-3:ro

Make sure I/O populates, then do a test read or write of a largish file to make sure it shows properly.
It's helpful to see your output. I should be able to handle this automatically in a future release.

It works like a charm, thanks for this awesome project!


shared from subworkflow no longer works

This example
[process: shared='result']
result = 100

[default]
sos_run('process')
print(f"Result from subworkflow process is {result}")

does not work.

[process: shared='result']
result = 100

[default]
sos_run('process', shared='result')
print(f"Result from subworkflow process is {result}")

works.


return errors where appropriate

Currently the package returns a boolean indicating success.
I've also done some minor tweaks for readability and adhering to conventions.

I've changed base from master to v2. I try to look into this further later (maybe tomorrow).

(sorry, I didn't see you changed the PR, so I was opening a new one when I saw what you did)

Don't worry, the Github UI can be confusing at times...
(Looks like I need to update the CI tooling. The test matrix still targets Go 1.13 :eyes:)

oh. there are errstr all over the place now. I'll fix.

Thanks for looking at this. Sorry for the errstr stuff.
The question is what we're suppose to do with the deprecated stuff. I can have a stab a clearing it out once this PR is merged, or perhaps you wanna do it? Let me know and I can plan accordingly.

Sorry for the errstr stuff.

Don't worry, things happen.

The question is what we're suppose to do with the deprecated stuff. I can have a stab a clearing it out once this PR is merged, or perhaps you wanna do it? Let me know and I can plan accordingly.

Feel free to clear them out, while you're at it :)
I've pushed a few commits to update dependencies and the CI workflows, so you may want to rebase your PR on top of them.

Removed the deprecated calls as requested. Gone over the existing todos and resolved as many as I could. There is one left, which is wrt to a potential error which might show up during handling anonymous sections, something I know know nothing about.
I've been unable to figure out how to get switch my fork to use the v2 branch directly, so I haven't been able to rebase. Hope it doesn't cause much problems.
I'm quite happy with the state of it as it is now. Lemme know.

I've been unable to figure out how to get switch my fork to use the v2 branch directly

Assuming git remote -v displays something like
origin	git@github.com:perbu/go-uci (fetch)
origin	git@github.com:perbu/go-uci (push)
upstream	git@github.com:digineo/go-uci.git (fetch)
upstream	git@github.com:digineo/go-uci.git (push)

i.e. upstream points to this repo, and origin to yours, to rebase you'll need to:

Fetch my v2 branch
$ git fetch upstream

Move your commits onto that branch:
$ git rebase upstream/master

Note: This may or may not) create a merge conflict you'd need to resolve before continuing (I've fixed some linter issues, which you might have as well - if we've applied the same solution, git won't complain). Should things go south, you can abort the rebase attempt with git rebase --abort, even after solving one or more conflicts.

Push your changes (forcefully):
$ git push --force-with-lease origin master

Would you try that? If not, or if problems arise, let me know, I should be able to do this on my end.

I think I managed to pull it off.

Codecov Report
Attention: 14 lines in your changes are missing coverage. Please review.

Comparison is base (37c7b10) 89.85% compared to head (570bdc9) 87.97%.
Report is 4 commits behind head on v2.

Files
Patch %
Lines

uci.go
77.27%
9 Missing and 1 partial :warning:

errors.go
50.00%
2 Missing :warning:

parser.go
50.00%
1 Missing :warning:

types.go
0.00%
1 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##               v2      #21      +/-   ##
==========================================
- Coverage   89.85%   87.97%   -1.88%
==========================================
  Files           7        7
  Lines         680      682       +2
==========================================
- Hits          611      600      -11
- Misses         60       73      +13
  Partials        9        9

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Looks good. I'll merge this when I'm back home.

I've squashed your commits into 60c1481, tagged as v2.0-rc.1 and cleaned some minor things up. The PR somehow contained a merge commit with master, which led to the commits being duplicated... Not sure how this could happen :)
There still was a (*tree).Set method lying around, but Set was already removed from the Tree interface. I'm now considering renaming SetType to Set, since one argument already contains the Type keyword:
uci.SetType("pkg", "sect", uci.TypeOption, "val")
// vs.
uci.Set("pkg", "sect", uci.TypeOption, "val")