Issues 319 and 257

OK, here's another fairly significant pull request.
This does two major things:

Simplifies the PID calculations.  The PID library we were using was unnecessarily obscuring some fairly simple calculations and some of the math was suspect.  The new version is much easier to read and will make future enhancements easier to implement.  This will continue to evolve as we introduce the pinch valves, but the updated code is a good starting place.
Splits the main code into a low priority background loop and a high priority task that's run from a timer interrupt.  This ensures that the PID loop runs at a fixed rate which is important.  It also opens up more options for improving the A/D sampling.  The A/D on this chip can oversample the inputs to get better resolution, but that was problematic to implement due to timing concerns with the old code.
For the moment I haven't changed the PID update rate, but that will definitely increase to at least 1kHz soon.

I'm sorry about all the big pull requests, this should calm down soon.  Edwin wants to try to do some testing with the pinch valve this weekend and I've still got a lot of work to do before that can reasonably happen.
I know we're all just volunteering on this project, but it would be really helpful to get some of these PRs accepted, or if there are issues please highlight them.

it would be really helpful to get some of these PRs accepted

If you're not able to make progress, that's a problem that I think we should address.  Can you elaborate on how you're blocked if this does not get merged into master before the weekend?  Perhaps we can assist somehow.

if there are issues please highlight them.

@asmodai27 has been carefully testing and fixing issues in the PID code.  Are there tests of the new code?

Steve, sorry, I strongly disagree with inlining the PID library into the controller code. While the calculations might be simple, we've had enough bugs in simple calculations (example: https://github.com/RespiraWorks/VentilatorSoftware/pull/314) that testability is a top priority, and a well-encapsulated library is much easier to unit test, especially for corner cases, when the PID logic is not intertwined with reading sensors etc. Happy to revisit the other aspects of the PR when the library is back.

Hi Justin, sorry if I'm pushing too hard here.
My problem is that I'm working on a couple different changes, this and adding the debug interface, and I'm coming to a juncture where the two are starting to merge.  I can't add the next very important part of the debug interface without first fixing the synchronous loop problem, so I've come to something of a blocking point.
Perhaps this is because I'm working directly with the RW repository.  I noticed that both you and Eugene have your own forks of the repo that you work on.  I had been wondering why you did that, but it's now becoming clear as it allows you to continue making progress while waiting for lower level branches to be committed to the main repo.
I guess that's how I'll start working as well although I'm somewhat concerned that this could lead to our different lines of development diverging.

Perhaps this is because I'm working directly with the RW repository. I noticed that both you and Eugene have your own forks of the repo that you work on.

Not sure if that is as easily applicable to bigger changes, but you can use any branch as the base branch for new development.
I kind of had a similar problem with the controller refactoring last weekend: started working, and realized that I had to change the PID library, which had a big pending PR that would mean I couldn't actually work on it since I had started working from master.
If you started around the same point in time (same master branch), it's easy, I could just rebase from the PR's branch:
git rebase PR_branch
That imports the commits from PR_branch into the current branch, and you should be fine.
If you started from a more recent master, it gets a bit trickier, you may have to rebase the PR_branch from master before being able to do the rebase on the current branch:
git checkout PR_branch
git rebase master
git push -f #not actually necessary but since you did the rebase, you might as well push it to github...
git checkout current_branch
git rebase PR_branch
And you should be all set to resume work

Hi Eugene, I think you're looking at the problem wrong.
I agree that testability is important, but right now we're testing the wrong thing.  We need to test the controller function, not the PID library that it calls.  Controller::ComputeFanPower() is what we should be running the unit tests on.  If we test the higher level function then we would also catch errors in the subordinate library, but not the other way around.
In the previous version of the code ComputeFanPower() didn't really do anything but call the PID library, so it was little more then a thin veneer over that code.  The control functions that we need to implement are probably not best served by just a standard PID loop however, so it's very likely that we will either need to change the PID library so it's not really a PID library anymore, or add a bunch of other calculations to the compute function which will in turn need testing.
Moving the actual calculations into ComputeFanPower doesn't reduce our ability to unit test this, it just makes it more likely that we'll be testing the right thing.
A couple reasons why I think the calculations should be in the compute function rather then in the PID library:

The code is without question easier to follow when written this way.  It's important that we write understandable code where possible, not just because it makes it easier for us to find problems but because other people in the future will be looking at this trying to understand how it works.  Generally speaking, simpler is better here.
There's nothing magic about the PID calculations.  PID tends to be used as a generic term for any type of feedback control algorithm, but those three calculations aren't the only or necessarily the best way to calculate a control loop, they're just a fairly common way.  Its very likely that our control algorithm will not fit neatly into a PID type structure, especially not now that we've got multiple outputs to control (both fan power & pinch valve).  Trying to force this into a fixed PID structure only works against us.
The PID library we're using is not particularly well written.  In particular, the D term isn't actually a D term the way we're using it, it's a feed forward term.  I have nothing against feed forward gains in a control loop, they're very helpful, but calling it the D term is just wrong.  Also the way the integral is being limited is too simplistic and is likely to lead to loop saturation.

So yes, we should have unit tests on our loop calculations but I don't think that's a good argument against the change I just made.
As to those unit tests, I think it's still a bit early to spend a lot of time writing them.  The problem is that the overall system design is currently in flux.  We know that simply controlling the blower isn't meeting our requirements (at least not with this blower) and we're at the stage where we're experimenting with other options.  We may very well find that the pinch valve approach has its own set of problems and move on to a completely different control structure.  All this could easily have a big impact on how the control loop works, so until that's nailed down I don't think there's a lot of value in writing unit tests on control loops which will likely continue to evolve.
This is also why I'm pushing so hard to get these changes implemented quickly.  We need to nail down the basic system design fairly soon but we can't do that until we can run some tests to see if the current design is workable.  Running those tests requires a bunch of software infrastructure to be written, so there's some pressure to get this integrated quickly.  A couple things that really need to get done to perform proper tests on the system design are tightening up the timing of the control loop (part of this change), and adding some real time debugging capability to allow us to more easily evaluate the system performance.

As to those unit tests, I think it's still a bit early to spend a lot of time writing them.

That is a big difference from the software development philosophy that I am familiar with.
I can't speak to whether your code ever has bugs, but mine does all the time.  Writing good unit tests as we go is the only I know to ensure that we as a group can make progress.
If you need proof of that, just take the typo you found in the sensor code.  That wasted a whole day of my time, plus I involved others.  Or take the bug @asmodai27 found in #314 -- this has wasted huge amounts of time.  Both could have been avoided with better testing (and I'm as much at fault here as anyone).  The unit tests we do have have saved me from writing numerous bugs.
If this is not motivating to you, I'll put it a different way.  Eugene and I could spend today arguing with you about libraries and nanopb and testing, or we could spend today reviewing your beautiful, tested code.  If you want to get your stuff in asap, I suspect that writing tests and using libraries is the way to do it.

you can use any branch as the base branch for new development.

Exactly.  My tool https://github.com/jlebar/git-pr-chain basically automates the workflow you describe.  It's really handy for not getting blocked.

That is a big difference from the software development philosophy that I am familiar with.

Ha, ya I figured that would be controversial statement!
All I mean is that I think the actual loop calculations are still very much in flux and it probably makes some sense trying to use them to tune the valve based system before I go too far into writing tests for them.
I need to hash out the whole question of whether or not to use the PID library with Eugene first anyway.  I'm not going to bother writing unit tests for code that's not going to be permitted into the repo in the first place.
If we have to use the PID library then there are already some unit tests for it.  I'll deal with trying to fix it's many shortcomings later.  If not, I'll add some tests for the new code as soon as I've had a chance to try closing the loop with it.

My tool https://github.com/jlebar/git-pr-chain basically automates the workflow you describe.

OK, cool.  I'll check that out.  Thanks

Okay, there's several issues at play here:
You're blocked from making progress before certain PRs are merged. This is unquestionably frustrating for everyone - for you because you're blocked, for e.g. me because I feel like I have to apply less than my usual level of scrutiny in order to avoid slowing you down, or worse, potentially frustrating you enough to lose you as a contributor. I think once you're able to make progress and experiment without waiting on us, things will go much smoother. Fixing this is the top priority, please reach out to Justin or me to hash this out.
Decomposition into libraries I agree with some of what you're saying. E.g. the PID library isn't that great (you should have seen it before #259!); and I defer to experts (e.g. you) on its mathematical properties (I'm not a PID expert). Happy to see improvements there.
I also can kinda see where you're going with saying that the control loop won't be a clean application of PID soon. I agree that, when it becomes impossible to structure the control loop as just applying PID, then we'll need to structure it differently and at that point I don't mind dumping the PID library. I just don't want to do this prematurely and before understanding what the control loop will look like; and when we do this, I'd like it to still be a well-encapsulated library - e.g. it cannot be intertwined with reading and writing pins; one major reason for that is that we have to be able to wire it up with Modelica. The current Controller class API does a decent job at this though.
I also agree that we need tests of the whole controller loop, not just the PID library - but IMO higher-level testing doesn't give us a free pass on low-level testing because higher-level tests might not trigger all the lurking corner cases of low-level code, and then one day we might refactor the higher-level code in a way that it still passes higher-level tests but now can trigger those corner cases in some other situations, and boom, we're hitting an untested codepath.
I don't agree in general that inlining the library makes the code "without question easier to follow" - it is definitely less code, but when the code is encapsulated and tested, then when reading the call site I don't need to follow it at all. As far as I can tell, @jlebar and @asmodai27 are in the same boat, so inlining would make it harder for us to work with it and easier for us (or other contributors) to accidentally break it. That said, it's not a dogma, and we'll just have to take a "know it when you see it" approach.
Testing
To save us both time, I will say upfront that I will not approve a PR that adds any math without adding sufficient unit tests for it in the same PR, because even in the short timespan of this project everybody who wrote any math has already written it wrong, or broken someone else's code because it didn't have enough tests, costing themselves and others lots of time in debugging. But I believe this should be a lot less of a concern when we figure out how to decouple your experimentation with different approaches vs. our reviews.

Hi Eugene, thanks for the response.  Just to be clear, I'm not getting annoyed or frustrated by this exchange.  I value hearing your perspective on these subjects and while we don't see eye on some of these topics I'll willing to work within the constraints of the current environment.
I think I'll try using Justin's tool to manage my repository so I can continue to work without being blocked by delayed PRs.  That should allow me to continue making progress where I need to and merge things into the main repo when possible.
As to the PID changes, I'll back that out.  All I really care about right now is fixing the timing problems and I can hold off on fixing the broken PID calculations until another time.


Make sure documentation makes sense.

If anyone wants to get into this project, this could be a good place to start taking a look at.

Hi, I´m looking for a place to start. Where do you think I should start in this specific issue. Thanks

I've left a lot of docstrings throughout the code.

If you find and correct any spelling/grammar mistakes you find that would be awesome.
I also was not sure what to write in init.py files and at the top of some modules. Suggestions are welcome.
I was thinking about taking a look at how each of them behave when using print(help(package_name.module_name)) and how the descriptions would make the most sense. You could look into this too.

Once you've read all docstrings and the code, it should probably be easier to work on the other issues.

Thanks I really want to help and be a part of this community, it s just a little bit overwhelming at the beginning.


org.apache.ibatis.io.Resources as a singleton instead of static methods

It should be good to have the org.apache.ibatis.io.Resources as a singleton with instance methods instead of static methods. In this way one can change the resource loading behaviour.
In this way, it should be easy to load the necessary mapper xml according to the current environment (oracle or db2...)
(There is a "default classloader", which can be set, but the first classloader is the classloader of the mapper interface which is called to get the mapper. So if there is a default mapper in the classpath, there is no way to load a different mapper xml. Now the only solution to remove the original mapper from the jar containing the mapper interface (or to provide the necessary mapper xml somewhere earlier in the classpath...))

For your use, you need You need to use registerAlias method that takes Class as a parameter.


[FEATURE REQUEST] Custom update scripts

Please confirm these before moving forward

[X] I have searched for my feature proposal and not found a work-in-progress/duplicate/resolved/discarded issue.
[X] This proposal is a completely new feature. If you want to suggest an improvement or an enhancement, please use this template

Describe the new feature
Allow users to set a script, to update from, when an update is available.
This would be useful for modified apps like "block the spot" and "better discord".
*User installs app
*User adds custom update variables
*Winget finds update for app "app"
*User checks and selects "update selected packages"
*Custom update script is ran
Describe how this new feature could help users.
It will stop functionality break, and make updating moded apps more convenient.

This kind-of already exists via "custom command line argument", but it's not as simple as copy pasting the powershell script from block the spot's readme....
If there's an easy way to tell it to skip udpating from repo and instead running the script via the existing interface lmk.
So far it seems more convenient to run the powershell script on start up. Than it is to run it via winget.
Perhaps this is similar to https://github.com/marticliment/WingetUI/issues/1117
But I really just want to ignore, existing winget etc, and on 'update found' trigger powershell script

So, you mean something like custom post-install scripts?

Ya, I guess.
For example block the spot just uses:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; Invoke-Expression "& { $(Invoke-WebRequest -UseBasicParsing 'https://raw.githubusercontent.com/mrpond/BlockTheSpot/master/install.ps1') } -UninstallSpotifyStoreEdition -UpdateSpotify"
I'd need to run this every time I update spotify. Or download the .dll and drop it in the spotify folder every time there's an update to spotify. Just a minor inconvenience.
Same with better discord, a discord update often needs a reinstall of better discord.
Also would be convenient if you could right click apps in the installed list to go to the app folder. I assume the current logging could already see the output folder[s]? It also seems wingetui checks against the registry, which registry has the installed folder.

Also would be convenient if you could right click apps in the installed list to go to the app folder. I assume the current logging could already see the output folder[s]? It also seems wingetui checks against the registry, which registry has the installed folder

This is not true. There is no viable way of getting the install location from WingetUI.
Regarding the feature, it could be done, i don't see why not

This is not true. There is no viable way of getting the install location from WingetUI.

Well for example the %LocalAppData%\Programs\WingetUI\choco-cli\logs writes SOME install locations "2023-07-29 00:09:27,786 8424 [INFO ] -   Software installed to 'C:\Program Files\Proton\VPN'" Also obviously the windows registry has install locations.
Querying the choco log for software locations and caching the locations would be fairly simple
`$LogPath = "$env:LocalAppData\Programs\WingetUI\choco-cli\logs\chocolatey.log"
$SearchString = "Software installed to "
$OutputFile = "Software install location.log"
Get-ChildItem $LogPath -Filter .log | ForEach-Object {
$Content = Get-Content $_.FullName
$Matches = [regex]::Matches($Content, "$SearchString'([^'])'")
foreach ($Match in $Matches) {
$InstallLocation = $Match.Groups[1].Value
Add-Content $OutputFile $InstallLocation
}
}
`

This is not true. There is no viable way of getting the install location from WingetUI.

Well for example the %LocalAppData%\Programs\WingetUI\choco-cli\logs writes SOME install locations "2023-07-29 00:09:27,786 8424 [INFO ] - Software installed to 'C:\Program Files\Proton\VPN'" Also obviously the windows registry has install locations. Querying the choco log for software locations and caching the locations would be fairly simple
`$LogPath = "$env:LocalAppData\Programs\WingetUI\choco-cli\logs\chocolatey.log" $SearchString = "Software installed to " $OutputFile = "Software install location.log"
Get-ChildItem $LogPath -Filter .log | ForEach-Object { $Content = Get-Content $.FullName $Matches = [regex]::Matches($Content, "$SearchString'([^']_)'") foreach ($Match in $Matches) { $InstallLocation = $Match.Groups[1].Value Add-Content $OutputFile $InstallLocation } } `

Ok, let's assume we get the paths of all the chocolatey packages. How do we get the winget ones? and the scoop ones? they don't use logs from where locations can be extracted...

Ok, let's assume we get the paths of all the chocolatey packages. How do we get the winget ones? and the scoop ones? they don't use logs from where locations can be extracted...

Does scoop not put all packages in %userprofile%\scoop\apps ? So just parse the name of the scoop app from that dir?

this seems to be an open issue for Winget tho https://github.com/microsoft/winget-cli/issues/2298

Get-WmiObject -Class Win32_Product | Select-Object Name, InstallLocation
will also display installlocations of a few apps

Would really like post-install update scripts, or even just the ability to run a specific command after install to be able to call a batch file. For example "Git For Windows" adds a couple of stupid context menu entries on every update, and I have a batch script I have to run that removes the registry entries for it. Would be nice to have this automatic.
I see in #2160 someone requested the same thing but that was marked as a duplicate but it's really not, it's subtly different.

For example "Git For Windows" adds a couple of stupid context menu entries on every update

I avoid this context menus by installing git with custom parameters like this winget install --no-upgrade --scope machine -h --accept-package-agreements --accept-source-agreements Git.Git --custom '"/COMPONENTS="icons,assoc,assoc_sh,,,,gitlfs,icons\quicklaunch" /o:SSHOption=ExternalOpenSSH"'


Inheritance diagram.

@caelanradford and @jbloom might find this helpful.
I just discovered pylint's UML (unified model language) tool, pyreverse. You can call pyreverse from your command line on any python file and get a diagram of the inheritance tree. It's pretty handy.
For examples, here's what I got when I ran it on the spline model in the epistasis package:
pyreverse -my -A -o png epistasis/models/nonlinear/spline.py

Note, you have to install graphviz (installation depends on your OS) and pylint (pip installable).

@harmsm might also find this useful when trying to understand Zach's unreviewed, "grad-student-doesn't-know-what-he's-doing" code.

This is great, thanks.


Fix wrong use of std::move in message.h

This might give some performance gains.

Codecov Report

Merging #38 (cf8be3f) into master (7a8367c) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master      #38   +/-   ##
=======================================
  Coverage   99.68%   99.68%
=======================================
  Files          26       26
  Lines         631      631
=======================================
  Hits          629      629
  Misses          2        2

Impacted Files
Coverage Δ

include/protopuf/message.h
98.68% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


File Manager: Code highlight for certain files

Hello Ilia...
all text code now is white, there is no style anymore for
PHP, CSS and so on...
then, is possible to have the code highlighting even for smarty .tpl?
thanks

It used to be white for all the time, right?
It would be better to have them colored, is that what you're saying?

indeed no....before there was the code highlighter...how is it possible? Are you sure that it was white before?

Hmm.  I can't even see the highlight on the .sh scripts.
I will take a look. We need the highlight!

you're great...please, try to make it for .php .css .tpl... if possible :-D 👍 🥇

There are so many little things to watch after certain changes. Not everything happened to be properly abstracted.
Anyway, code highlight is back for ALL files, including .php and .tpl (Smarty) and all other.
Please give it a test.

ok, now it works... ;-) thank you

bug is back after latest update....and also it shows 18.47 version instead of 18.48
thanks

The funny thing. I had it too on the first install.
The reason was for that files on Webmin Themes/Install weren't updated :open_mouth:
I don't know, just update from theme settings or better try to do the same thing again but manually, download the package and feed the link to the installer.

..and the reason why the bug is back, because the theme announces itself as 18.47 to the browser and loading old stuff. If you go to authentic-theme directory you will see that authentic-init.pl and authentic-lib.pl weren't updated. I have absolutely no idea why.
I think in the future I will do all updates using Git as it's more stable and reliable.
Re-running update fixes it.

in theme settings there's no force update anymore....

I just explained on the comment above.


what is this error?

i ran the web ui user and it gives me this error
Error loading script: two_shot.py
Traceback (most recent call last):
File "/content/stable-diffusion-webui/modules/scripts.py", line 229, in load_scripts
script_module = script_loading.load_module(scriptfile.path)
File "/content/stable-diffusion-webui/modules/script_loading.py", line 11, in load_module
module_spec.loader.exec_module(module)
File "", line 848, in exec_module
File "", line 219, in _call_with_frames_removed
File "/content/stable-diffusion-webui/extensions/stable-diffusion-webui-two-shot/scripts/two_shot.py", line 10, in
from modules.script_callbacks import CFGDenoisedParams, on_cfg_denoised
ImportError: cannot import name 'CFGDenoisedParams' from 'modules.script_callbacks' (/content/stable-diffusion-webui/modules/script_callbacks.py)

Did you run this command?
git apply --ignore-whitespace extensions/stable-diffusion-webui-two-shot/cfg_denoised_callback-ea9bd9fc.patch

I have same error.
I think the patch is actually applied but it still happens.
I tried re-download webui for new directory and install cleanly, but same error.
webui version:
3715ece0adce7bf7c5e9c5ab3710b2fdc3848f39
(master branch 2023/02/13 14:12:51)
git diff:
diff --git a/modules/script_callbacks.py b/modules/script_callbacks.py
index 4bb45ec..a47e117 100644
--- a/modules/script_callbacks.py
+++ b/modules/script_callbacks.py
@@ -46,6 +46,18 @@ class CFGDenoiserParams:
         """Total number of sampling steps planned"""

+class CFGDenoisedParams:
+    def __init__(self, x, sampling_step, total_sampling_steps):
+        self.x = x
+        """Latent image representation in the process of being denoised"""
+
+        self.sampling_step = sampling_step
+        """Current Sampling step number"""
+
+        self.total_sampling_steps = total_sampling_steps
+        """Total number of sampling steps planned"""
+
+
 class UiTrainTabParams:
     def __init__(self, txt2img_preview_params):
         self.txt2img_preview_params = txt2img_preview_params
@@ -68,6 +80,7 @@ callback_map = dict(
     callbacks_before_image_saved=[],
     callbacks_image_saved=[],
     callbacks_cfg_denoiser=[],
+    callbacks_cfg_denoised=[],
     callbacks_before_component=[],
     callbacks_after_component=[],
     callbacks_image_grid=[],
@@ -150,6 +163,14 @@ def cfg_denoiser_callback(params: CFGDenoiserParams):
             report_exception(c, 'cfg_denoiser_callback')

+def cfg_denoised_callback(params: CFGDenoisedParams):
+    for c in callback_map['callbacks_cfg_denoised']:
+        try:
+            c.callback(params)
+        except Exception:
+            report_exception(c, 'cfg_denoised_callback')
+
+
 def before_component_callback(component, **kwargs):
     for c in callback_map['callbacks_before_component']:
         try:
@@ -283,6 +304,14 @@ def on_cfg_denoiser(callback):
     add_callback(callback_map['callbacks_cfg_denoiser'], callback)

+def on_cfg_denoised(callback):
+    """register a function to be called in the kdiffussion cfg_denoiser method after building the inner model inputs.
+    The callback is called with one argument:
+        - params: CFGDenoisedParams - parameters to be passed to the inner model and sampling state details.
+    """
+    add_callback(callback_map['callbacks_cfg_denoised'], callback)
+
+
 def on_before_component(callback):
     """register a function to be called before a component is created.
     The callback is called with arguments:
diff --git a/modules/sd_samplers_kdiffusion.py b/modules/sd_samplers_kdiffusion.py
index f076fc5..ab82c08 100644
--- a/modules/sd_samplers_kdiffusion.py
+++ b/modules/sd_samplers_kdiffusion.py
@@ -8,6 +8,7 @@ from modules import prompt_parser, devices, sd_samplers_common
 from modules.shared import opts, state
 import modules.shared as shared
 from modules.script_callbacks import CFGDenoiserParams, cfg_denoiser_callback
+from modules.script_callbacks import CFGDenoisedParams, cfg_denoised_callback

 samplers_k_diffusion = [
     ('Euler a', 'sample_euler_ancestral', ['k_euler_a', 'k_euler_ancestral'], {}),
@@ -136,6 +137,9 @@ class CFGDenoiser(torch.nn.Module):

             x_out[-uncond.shape[0]:] = self.inner_model(x_in[-uncond.shape[0]:], sigma_in[-uncond.shape[0]:], cond={"c_crossattn": [uncond], "c_concat": [image_cond_in[-uncond.shape[0]:]]})

+        denoised_params = CFGDenoisedParams(x_out, state.sampling_step, state.sampling_steps)
+        cfg_denoised_callback(denoised_params)
+
         devices.test_for_nans(x_out, "unet")

         if opts.live_preview_content == "Prompt":
(END)

when i ran this extension by itself, i didnt get the error.
when i ran it with my other extensions, i got the error.
i think it's just an extension conflict.

Same problem here, the patch return this btw:
C:\Users\User\Downloads\stablediffusion\stable-diffusion-webui>git apply --ignore-whitespace extensions/stable-diffusion-webui-two-shot/cfg_denoised_callback-ea9bd9fc.patch
error: patch failed: modules/script_callbacks.py:68
error: modules/script_callbacks.py: patch does not apply
error: patch failed: modules/sd_samplers_kdiffusion.py:8
error: modules/sd_samplers_kdiffusion.py: patch does not apply

With the webui update the other day, you no longer need to use the patch.  You can use it just by installing it.

I was having this error with a google colab I was using and found that it started to work after I removed the embeddings from: https://huggingface.co/nolanaatama/embeddings

Just for future readers, as of the time writing this, if you are still getting an import error after running the patch commands, it might be because they aren't actually getting applied. The CFG patch worked for me, but the 0001-Adding-after_ui_callback-for-scripts.patch was still needed and wasn't getting applied. I copied it over manually, but this stackoverflow page might help?
https://stackoverflow.com/questions/4770177/git-patch-does-not-apply

I have the same issue like yours, anyone has suggestion?

In my case it was fixed by updating stable diffusion web ui, by executing 'git pull' on the root directory (usually ...\stable-diffusion-webui) it also fixed the slow starting, im assuming because of updating old dependencies shared with the plugins I have installed.
I hope this works for you too!


New versions

After my PR https://github.com/Ysj291823/egg-swagger-doc/pull/27 , unfortunately  you don't release a version and I see that you have huge changes and some releases
But I see that it is not currently stable and usable
When I can have a stable version?

it is the endpoint, is there err log

@Ysj291823 no error log , just 404 not found page when I open http://<IP_ADDRESS>:7001/swagger-ui.html

you can see https://github.com/Ysj291823/egg-example-api ,  Are there any differences？

i have meet the same problem, when upgrade the old version plugin to new. i will check it.

you can try for it , i push a new version


chore: sync recent rwkv.cpp version

Sync recent rwkv.cpp version
Important
Since rwkv.cpp has merge recent version of ggml, which leads format break of old quantized models.
will work on with cuBlas feature soon

fixing the lint problem

checking compile error on macos ....

CUDA is definitely not supported by Apple...

checking compile error on macos ....

I think you can use cargo feature flag which serves CUDA as an optional

The feature has not been implemented in this pr yet, gonna to sync the recent version of rwkv.cpp before bring CUDA here

I opened a PR to fix this issue on rwkv.cpp
https://github.com/saharNooby/rwkv.cpp/pull/79
Will keep updating

@hlhr202 The compile error of rwkv.cpp has been fixed. ready to be merged


Error when using HTTPs RESTful APIs to access keylime registrar with curl

Is your issue a feature request? If so, please raise it as an enhancement
No
Environment

Keylime version: master

Description
When I use https RESTful APIs to access the keylime registrar with curl,  there is an error that tlsv13 alert certificate required, errno 0.
The steps to reproduce problem and relevant logs are as followings. What should I do to fix this problem?
Steps to reproduce problem
curl --cacert /var/lib/keylime/server-cert.crt -k --key /var/lib/keylime/server-private.pem -v  "https://<IP_ADDRESS>:8891/v2.1/agents"  | jq .

Relevant logs
Attach any relevant log files that can help to debug your issue.

curl logs:

keylime registrar logs:

@Jingshui1037 Hello, could you please try removing the -k option from the curl command? The -k option (equivalent to --insecure) makes the client to not provide its certificate.
And also you should be using the client certificates for your client, and not the server certificates. Try with something like:
curl --cacert /var/lib/keylime/cv_ca/cacert.crt --key /var/lib/keylime/cv_ca/client-private.pem --cert /var/lib/keylime/cv_ca/client-cert.crt -v  "https://<IP_ADDRESS>:8891/v2.1/agents"  | jq .

You still need the -k because hostnames are validated in curl otherwise and I think there is a option to only disable that. As @ansasaki pointed out the client certificate is still needed. Something like this works:
curl  --key /var/lib/keylime/cv_ca/client-private.pem --cert /var/lib/keylime/cv_ca/client-cert.crt -k  "https://<IP_ADDRESS>:8891/v2.1/agents"  | jq .

Thanks @THS-on , I succeed when I used your command. This is my test result.

Now I close this issue. Thanks @THS-on @ansasaki

Hi @THS-on @ansasaki,
Now I can succeed to use HTTPs RESTful APIs to access keylime registrar and keylime verifier with curl according to your command. However, when I use the curl command to access keylime agent with HTTPs RESTful APIs, there is an new error like the following.  What should I do to fix this problem?
curl --key /var/lib/keylime/cv_ca/server-private.pem --cert /var/lib/keylime/cv_ca/server-cert.crt -k  https://<IP_ADDRESS>:9002/v2.1/version -v | jq .

@Jingshui1037  Please provide the logs from the agent. You can get them by running the agent with the RUST_LOG environment variable set like RUST_LOG=keylime_agent=debug keylime_agent
I'd say there is a fair probability that the agent don't have access to the CA certificate generated by the verifier in /var/lib/keylime/cv_ca/cacert.crt. If that is the case, you need to chown to provide access to the user running the agent.

Thanks @ansasaki ,
These are the logs of my keylime agent in the first picture below. And when I chown  those three files (/var/lib/keylime/cv_ca/cacert.crt, /var/lib/keylime/server-cert.crt , and /var/lib/keylime/server-private.pem) to keylime:tss  according to your suggestion in the second picture below, the error changes to error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca in the last picture below.

@Jingshui1037 OK, then the last issue is that you have to provide the agent certificate in the --cacert option. Differently from the other components, the agent generates a self-signed certificate, so it is not signed by the CA certificate from /var/lib/keylime/cv_ca/cacert.crt.
Also, you should not be using the certificates generated by the agent as your client's certificate. Use the ones generated by the verifier from /var/lib/keylime/cv_ca instead. Try:
curl  --key /var/lib/keylime/cv_ca/client-private.pem --cert /var/lib/keylime/cv_ca/client-cert.crt --cacert /var/lib/keylime/server-cert.crt -k  "https://<IP_ADDRESS>:9002/v2.1/version" -v  | jq .

@ansasaki , according to your suggestion, I find a problem the agent machine doesn't have those two files(/var/lib/keylime/cv_ca/client-private.pem、/var/lib/keylime/cv_ca/client-cert.crt). So I have to copy these files from the keylime registrar machine and chown them to keylime:tss.

Then I use the command you provided to access the agent, the error logs change to
error:1409441B:SSL routines:ssl3_read_bytes:tlsv1 alert decrypt error which is similar to the first issue about my agent above

So I'm still confused about which private key and cert should I use.

@ansasaki , according to your suggestion, I find a problem the agent machine doesn't have those two files(/var/lib/keylime/cv_ca/client-private.pem、/var/lib/keylime/cv_ca/client-cert.crt). So I have to copy these files from the keylime registrar machine and chown them to keylime:tss.

Right, the agent machine will not have those files because the verifier is responsible for generating them by default. If you are running the components in different machines, then some provision steps are needed as the default configuration assumes you are running all the components in the same machine (for testing).
By default, he verifier will create a mini CA and generate key pairs and respective certificates for the server role and client role. They are placed in /var/lib/keylime/cv_ca/ in the verifier machine.

The verifier needs the server keys for its REST API server, as well as the client keys to access the other components (registrar and agent)
The registrar only needs the server keys and certificate as it does not access other components as clients
The tenant needs only the client keys and certificate as it does not act as a server

All these certificates are signed by the same verifier mini CA.
For the agent, it is different. The agent generates its own self-signed certificate to use as its server TLS certificate (the so-called mtls certificate)

The agent needs only server keys and certificate because when acting as a client (to access the registrar) it does not use any certificate (TLS is not used for this connection)
BUT it needs the CA certificate for the clients it trusts (configured via the trusted_client_ca in the config file) to verify the clients in the incoming connections. It means you need to provide the CA certificates that signed the certificates of all clients that are accessing the agent (tenant and verifier, or in your case curl).

In the default case, the verifier will generate all the certificates and put in /var/lib/keylime/cv_ca. The agent, by default, uses the CA certificate from that location (/var/lib/keylime/cv_ca/cacert.crt).
You need to copy the CA certificate from the verifier and put in /var/lib/keylime/cv_ca/cacert.crt in the agent machine. Alternatively, you can copy it to some location the user running the agent process can access and configure the path in the trusted_client_ca configuration option. If you are using different CAs for the components, you need to provide all of them as a list. For example, in the agent machine, add a configuration snippet file in /etc/keylime/agent.conf.d/ containing:
[agent]
trusted_client_ca = "['/path/to/a/CA/certificate', '/path/to/another/CA/certificate']"

Alternatively you can override the agent configuration option by setting the KEYLIME_AGENT_TRUSTED_CLIENT_CA environment variable. Something like:
$ RUST_LOG=keylime_agent=debug KEYLIME_AGENT_TRUSTED_CLIENT_CA="['/path/to/a/CA/certificate', '/path/to/another/CA/certificate']" keylime_agent &

then, in the curl command you need to provide the client certificate (signed by one of the CA certificates you provided in the agent configuration) and key:
curl  --key /var/lib/keylime/cv_ca/client-private.pem --cert /var/lib/keylime/cv_ca/client-cert.crt -k  "https://<IP_ADDRESS>:9002/v2.1/version"  | jq .

@ansasaki , Thank you very much for your detailed reply, now I understand it and can succeed to access agent with curl and HTTPS RESTful APIs.

Now I close this issue.


Update config for WebPack 4

WebPack 4 requires that you specify { type: "javascript/auto" } in the loader config. If you don't, the output of messageformat-loader will be passed to the default JSON loader, which will fail because the content is no longer JSON.
I'm actually not sure if it's possible to do this inside of the require statement.

The change should be applied to example/webpack.config.js as well -- which should probably be updated to use webpack v4 and webpack-cli. It's unfortunate that it looks like it may not be possible to define the default mode from within the loader itself, which I think could be a cleaner option.

I'd really prefer it if we could find a way for the loader itself to take care of this. It seems onerous to require users of the plugin to do this. Besides, what if they want to chain the output of messageformat-loader to another loader that expects JS? I asked this question in the webpack/webpack gitter today (Mar 20) at 14:55 PT. No answer yet. I'll probably need to make a Stack Overflow for it. This is also going to be a problem for lots of other loaders (val-loader, to-string-loader, apply-loader, etc) so it'll be interesting to see how it gets resolved.
I tried hackily setting this._module.type in the loader itself but that didn't work. I'll keep trying.

Adding this to the loader right before the return seemed to work for me (I commented here also).
this._module.type = 'javascript/auto';
this._module.parser = new JavascriptParser();
this._module.generator = new JavascriptGenerator();

where the parser is webpack/lib/Parser.js and the generator is webpack/lib/JavascriptGenerator.js.
Not sure if we should commit this or wait for a better resolution from the webpack team.

Since the linked issue is closed, I've filed webpack/webpack#7057. I'll merge the above hack in the meantime to support webpack 4.


Enhancement that disables BLJing similar as on DS

A little hacky but might be the shortest way

I thought the goal of this project was to make it as accurate and well-documented as possible?

There is already a ifdef for the shindou edition that fixes BLJs, though to make it more accessible it should probably be a bugfix option in config.h like many other such changes are.

OK
Le sam. 27 févr. 2021 22:43, Nadia Holmquist Pedersen <
<EMAIL_ADDRESS>a écrit :

There is already a ifdef for the shindou edition that fixes BLJs, though
to make it more accessible it should probably be a bugfix option in
config.h like many other such changes are.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/n64decomp/sm64/pull/63#issuecomment-787157836, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AL2ZQXBCP4GWF3CWJUFBND3TBFRPRANCNFSM4YI5LLDA
.

this is the sm64 n64 decomp repo, being an exact decomp is the goal, this does not belong here

will leave it as a pull request for people who are interested in it


Adding links to projects: Ruby SDK and Vantage

@boyney123 Just checking in here - anything I need to do to help get this merged in?

Hey,
Sorry @Bensign, just seen this! Not sure how it fell through.
Yeah merged it in, thanks!

Hey,
Sorry @Bensign, just seen this! Not sure how it fell through.
Yeah merged it in, thanks!

@boyney123 Thanks! No worries - figured you were busy :)

@boyney123 Thanks! No worries - figured you were busy :)


Fees selection and usage

[closes #81]

From a code standpoint it looks good to me, but I think tests for the core functions should be added like: calcSpotPrice, calcTotalAssetPrice and generateSwapExactAmountOutTx.

I think tests for the simple calculations should be straight forward to add as part of this pr. Though I think the transaction generation functions are a bigger piece of work as we need to figure out how to mock the polkadot js api

generateSwapExactAmountOutTx

Yeah looking at it I think you only need to create a unit test for calcInGivenOut and calcOutGivenIn since thats the part that does the client side calculation logic before creating the tx.


column profiling metadata fix

@bastienboutonnet the other changes were added by accident - I think.  I'll merge the branch.


Responsiveness of controls are slow

Describe the bug
Looking at a couple examples on the product page, there is a number of examples of slow front-end controls.
The canonical example in the Intro has a counter. It has a noticeable lag, looks like ~300ms[1], but feels like near a second.
Looking at the React wrapping demo, there is a noticeable latency when dragging the color circle around, but when I use the demo site for the original component, it has near-zero lag
To Reproduce
Uses the above and you should feel lag
Expected behavior
That all the Pyncone examples are fast with no extra effort. The simple example of a counter should show no lag, and the wrapped React counter should perform at the same as the component it is wrapping.
Screenshots
[1]
** Specifics (please complete the following information):**

Browser: Chrome Version 110.0.5481.177 (Official Build) (x86_64)

Got it thanks for digging into this. I'll look into this more but I think this has to do with larger apps. We are working on ways to prevent other components that don't update from re-rendering and you should see performance improvements on the main website as a result.

Quick follow up. On the counter, I did not realize it was hitting the server to update the value. Assuming your UI does not need long-term state and instead wants to store up changes locally (e.g. building a dynamic form to submit), can you bind props to a local state store instead of server hosted state?

I am facing the same issue with the responsiveness. Please check the video attached for the tutorial in docs page:
https://user-images.githubusercontent.com/119093969/229191379-1731a48f-52f5-4ff5-a9b0-90579ec9cff7.mov

When running the same tutorial locally, it doesn't even respond.

When running the same tutorial locally, it doesn't even respond.

Are you running in dev mode or prod mode?

closing this issue for now


[master] Auto-update dependencies

Produced via:
./hack/update-deps.sh --upgrade && ./hack/update-codegen.sh
/assign ImJasonH vdemeester
/cc ImJasonH vdemeester

/ok-to-test

@mattmoor hum, why the only changes are on the third_party folder :thinking: (and none on go.sum, go.mod, …)

@vdemeester sorry I filter most of these mails, so I didn't see your comment.  I see changes to go.*, but I'd guess there were differences when you looked because someone didn't run things properly and it made it into HEAD?

@mattmoor that's what's worrying me a bit… it should have shown in the CI (if those file were missing in a PR)… This might mean we have a bad check somewhere :stuck_out_tongue_closed_eyes:
/lgtm

/cc @bobcatfish @afrittoli @sbwsg @dibyom
@mattmoor might conflict with #2846 but is independent right ?

@vdemeester I really like the check we have here: https://github.com/vmware-tanzu/sources-for-knative/blob/master/.github/workflows/go-verify.yml

Also, yes it will conflict :)


Parse  inside  and  into category attribute

(Prompted by this post on our mailing list.)
RSS 2.0 can have <category> tags in <channel> and <item> (see the spec for details). Those should be parsed and put into feed- and article-level category attributes, respectively.
TODO:

[ ] Check if other versions of RSS, and Atom, have categories too

For Atom I can tell, that it has categories in both feed and article levels: https://tools.ietf.org/html/rfc4287#section-4.2.2


Swirl of shades - color must be int or tuple

Seen from example documentation below - fill results in "TypeError: color must be int or tuple", likely from handling of None returns where no relevant color.
import shades

canvas = shades.Canvas()
shade = shades.SwirlOfShades(
  noise_field=shades.NoiseField(scale=0.005),
  shades=([
    (0.4, 0.6, shades.BlockColor((63, 151, 197)))
  ]),
)

shade.fill(canvas)
canvas.show()

(raised this to go along with another issue on documentation)
The issue was caused by the swirl passing back 'None' where a color isn't appropriate. There had been error handling her previously I think from a messy 'try' loop, which I'd taken out without realising the impact.
Now, the abstract shade object won't make a point on the canvas if there isn't a color return for the determine_shade method, which should make hacking around a lot easier as well.
Added in some unit testing as well to capture if this happens again, plus ammended the documentation.
All released up to pip as v1.1 so updating library should fix this.


feat(bigquery): implement custom minute time grains

SUMMARY
Currently the BigQuery engine spec only supports full seconds/minutes/hours due to the dialect not natively supporting custom intervals. This PR adds support for the missing default 5, 10, 15 and 30 min grains by utilizing some custom timestamp handling functions:
Screenshot

TEST PLAN
Verified to work on TIMESTAMP, DATETIME and DATE types + added new unit tests.
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

Codecov Report

Merging #12581 (ae64370) into master (0f243c6) will decrease coverage by 3.58%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master   #12581      +/-   ##
==========================================
- Coverage   66.75%   63.16%   -3.59%
==========================================
  Files        1015      486     -529
  Lines       49634    29968   -19666
  Branches     4839        0    -4839
==========================================
- Hits        33131    18930   -14201
+ Misses      16380    11038    -5342
+ Partials      123        0     -123

Flag
Coverage Δ

cypress
?

javascript
?

python
63.16% <100.00%> (-0.80%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

superset/db_engine_specs/bigquery.py
95.74% <ø> (ø)

superset/db_engine_specs/base.py
84.43% <100.00%> (-1.39%)
:arrow_down:

superset/db_engines/hive.py
0.00% <0.00%> (-85.72%)
:arrow_down:

superset/sql_validators/postgres.py
50.00% <0.00%> (-50.00%)
:arrow_down:

superset/db_engine_specs/hive.py
54.61% <0.00%> (-29.24%)
:arrow_down:

superset/db_engine_specs/sqlite.py
65.62% <0.00%> (-9.38%)
:arrow_down:

superset/databases/commands/create.py
83.67% <0.00%> (-8.17%)
:arrow_down:

superset/databases/commands/update.py
85.71% <0.00%> (-8.17%)
:arrow_down:

superset/connectors/sqla/models.py
84.31% <0.00%> (-6.28%)
:arrow_down:

superset/databases/commands/test_connection.py
84.78% <0.00%> (-4.35%)
:arrow_down:

... and 541 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0f243c6...ae64370. Read the comment docs.

Codecov Report

Merging #12581 (ae64370) into master (0f243c6) will decrease coverage by 3.58%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master   #12581      +/-   ##
==========================================
- Coverage   66.75%   63.16%   -3.59%
==========================================
  Files        1015      486     -529
  Lines       49634    29968   -19666
  Branches     4839        0    -4839
==========================================
- Hits        33131    18930   -14201
+ Misses      16380    11038    -5342
+ Partials      123        0     -123

Flag
Coverage Δ

cypress
?

javascript
?

python
63.16% <100.00%> (-0.80%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

superset/db_engine_specs/bigquery.py
95.74% <ø> (ø)

superset/db_engine_specs/base.py
84.43% <100.00%> (-1.39%)
:arrow_down:

superset/db_engines/hive.py
0.00% <0.00%> (-85.72%)
:arrow_down:

superset/sql_validators/postgres.py
50.00% <0.00%> (-50.00%)
:arrow_down:

superset/db_engine_specs/hive.py
54.61% <0.00%> (-29.24%)
:arrow_down:

superset/db_engine_specs/sqlite.py
65.62% <0.00%> (-9.38%)
:arrow_down:

superset/databases/commands/create.py
83.67% <0.00%> (-8.17%)
:arrow_down:

superset/databases/commands/update.py
85.71% <0.00%> (-8.17%)
:arrow_down:

superset/connectors/sqla/models.py
84.31% <0.00%> (-6.28%)
:arrow_down:

superset/databases/commands/test_connection.py
84.78% <0.00%> (-4.35%)
:arrow_down:

... and 541 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0f243c6...ae64370. Read the comment docs.

Tested manually - ok :) I had only problem with connection to physical table (not from sql editor). But time columns mentioned in description worked. 🟢

Tested manually - ok :) I had only problem with connection to physical table (not from sql editor). But time columns mentioned in description worked. 🟢

@adam-stasiak good to know - can you open a ticket about that? I'm pretty sure it's unrelated, but needs to be fixed at any rate but in a different PR.

@adam-stasiak good to know - can you open a ticket about that? I'm pretty sure it's unrelated, but needs to be fixed at any rate but in a different PR.


pytest command should use pyargs instead of path based option

For compiled projects, path-based pytests will fail. See the failed tests with mdaencore.
https://github.com/MDAnalysis/mdaencore/actions/runs/5648200654/job/15300027899#step:8:34
This is because the *.so files are not found. Using pyargs ensures that the proper package is imported.
pytest -n 2 -v --cov=mdaencore --cov-report=xml --color=yes --pyargs <PACKAGENAME>

Side note: should we replace -n 2 with -n auto?

should we replace -n 2 with -n auto?

I would second this.

I'm not sure why the *.so files wouldn't be found. using --pyargs has a lot of disadvantages (including not being able to use local extensions properly). I wouldn't advocate for it in nearly all cases.

To clarify here - plenty of projects don't use pyargs and have .so files. I think there's a deeper issue here that's being overlooked.


VSTS modal dialog close event

Hi,
I am using microsoft VSTS modal dialog and I want a confirmation message when modal dialog is closed from 'x' (closed button of modal dialog), I get close event which fired after event dialog is closed.
I want an event before closing the dialog. How can I do that?
here is the code:
VSS.getService(VSS.ServiceIds.Dialog).then(function(dialogService)
{
       var contributionId = //id
       var dialogOptions = { title: "ABC", close: onDialogCloseByCrossButtonClick };
       dialogService.openDialog(contributionId, dialogOptions).then(
               function (dialog) {
                    // do something
        });
}

like its appearing in TFS WorkItem window.

@javeriaanwer  you will want to post your question on the following repo: https://github.com/Microsoft/vss-web-extension-sdk

@tedchamb already posted, thanks anyway.