Disable cache expiry

We use js-localization (thanks :-)) in a Laravel 5.1 project, and it's working really well.
We do have one issue, though, I believe it's to do with the cache.
Some of the longest running processes on our site is /js-localization/messages . My guess is that it's because the message cache expires and the messages json is rebuilt. Is it possible to disable the cache expiry and only have the messages rebuilt using the artisan command? Or did I misunderstand what is actually going on in that step? :-)

Hi @Jarlskov!
Interesting point, but I fear there must be another cause. Cache expiry can't be it I think, since the it's cached using Cache::forever: https://github.com/andywer/laravel-js-localization/blob/laravel-5/src/Caching/AbstractCachingService.php#L64
Maybe you can further investigate that for your use case, like:

is it permanently slow or just occasionally
do you use a fast caching provider
how does disabling the message cache affect performance (just monkeypatch the code)

Hope that helps.

Hej Andy
Thanks for the reply.
It makes sense, so the issue probably is that we're currently just using the disk as a cache, so disk IO is the load time. It just seems strange that it would be that much slower than basically any other request, but that might just be a local issue in our setup.
I should probably look into having some proper caching mechanism set up.

Any news on that @Jarlskov?

Not yet, @andywer , I'm still on the issue. We just had some larger server issues and had to do a pretty panicky server change so I want to make sure that everything runs smoothly before I change too much in our setup :-)
My plan is to change between Christmas and New Year.

I've moved to using Redis as a cache now, and most of it seems pretty good. We do still see a few spikes where the request takes a long time. We have a request about 15 min ago where New Relic reported the slowest component to be 	/\JsLocalization\Http\Controllers\JsLocalizationController@createJsMessages taking 60 sec

Me neither, I just wanted to keep you updated on the current state, even though you of course can't help much with whatever limited info I can currently provide :-)
I'll keep an eye on it, and see if anything of interest pops up.

I've looked a bit more into it, and it seems like one issue is that we always hit a PHP endpoint. This means that Apache can't cache it properly and we need to boot Laravel all of the time.
Do you think it would be a good idea to allow saving the generated JS to a file?
Making it a static file should make it easier to use caching headers, and it would allow us to include it in our JS compilation flow, and maybe even minify it, and version it to provide cache invalidation.
It could probably be made as an optional setting, to allow backwards compatibility.
Do you think this would be a good approach? Then it should probably be possible for me to create a PR to add the feature, if you'd include it in the project.

Hey @Jarlskov. Thanks for the update :)
Yeah, actually I think any decent, up-to-date library for exporting the translations should be able to create static files, also because many people will want to bundle it with their other JS sources (#21).
Unfortunately I don't have the time right now and my interest in writing PHP code is near zero 🙈
But if you open a PR I will review it :)


⚠ Bofrost.lu has degraded performance

In cc1bf8b, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1831 ms

Resolved: Bofrost.lu performance has improved in adf66b5 after 13 minutes.


Fix inputs initialization

Try to set value to parameter.value. Skip if parameter was not initialized yet.
Should fix the managed window example in issue #90.

When I am testing this PR (and nr 95), I now get a managed window, but

the queued procedures don't do anything (just stay "queued").
Hitting Open results in

  File "<snip>\pymeasure\pymeasure\display\widgets.py", line 381, in _setup_ui
    self.layout().addWidget(preview_tab, 0, 5, 4, 1)

AttributeError: 'NoneType' object has no attribute 'addWidget'

Also, to run at all the managed window example misses an import sys and an import tempfile.

did you run it after patching #95, #96 and #97? they are all needed for the
managed window example.
on what platform did you run the example? I'll try to run the latest
version on my pc (Linux, x64, python 3.6.1 via anaconda).
On Tue, Jul 25, 2017, 17:05 Christoph Buchner<EMAIL_ADDRESS>wrote:

When I am testing this PR (and nr 95), I now get a managed window, but

the queued procedures don't do anything (just stay "queued").
Hitting Open results in

File "\pymeasure\pymeasure\display\widgets.py", line 381, in _setup_ui
self.layout().addWidget(preview_tab, 0, 5, 4, 1)
AttributeError: 'NoneType' object has no attribute 'addWidget'

Also, to run at all the managed window example misses an import sys
and an import tempfile.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ralph-group/pymeasure/pull/96#issuecomment-317748308,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKuWBm_Iaa0_LBvf7ZiazGGHniHAuTKaks5sRfY3gaJpZM4OdYiX
.

Yes, Windows 10, Python 3.6. via anaconda.

Did the following:
git checkout master
git pull upstream master
git checkout -b test_959697
git merge fix_log
git merge fix_inputs
git merge fix_serialization

Then run this:
import logging
import random
import sys
import tempfile
from time import sleep

from pymeasure.display.Qt import QtGui
from pymeasure.display.windows import ManagedWindow
from pymeasure.experiment import IntegerParameter, FloatParameter, Parameter
from pymeasure.experiment import Procedure, Results

log = logging.getLogger(__name__)
log.addHandler(logging.NullHandler())

class RandomProcedure(Procedure):
    iterations = IntegerParameter('Loop Iterations')
    delay = FloatParameter('Delay Time', units='s', default=0.2)
    seed = Parameter('Random Seed', default='12345')

    DATA_COLUMNS = ['Iteration', 'Random Number']

    def startup(self):
        log.info("Setting the seed of the random number generator")
        random.seed(self.seed)

    def execute(self):
        log.info("Starting the loop of %d iterations" % self.iterations)
        for i in range(self.iterations):
            data = {
                'Iteration': i,
                'Random Number': random.random()
            }
            self.emit('results', data)
            log.debug("Emitting results: %s" % data)
            sleep(self.delay)
            if self.should_stop():
                log.warning("Caught the stop flag in the procedure")
                break

class MainWindow(ManagedWindow):
    def __init__(self):
        super(MainWindow, self).__init__(
            procedure_class=RandomProcedure,
            inputs=['iterations', 'delay', 'seed'],
            displays=['iterations', 'delay', 'seed'],
            x_axis='Iteration',
            y_axis='Random Number'
        )
        self.setWindowTitle('GUI Example')

    def queue(self):
        filename = tempfile.mktemp()

        procedure = self.make_procedure()
        results = Results(procedure, filename)
        experiment = self.new_experiment(results)

        self.manager.queue(experiment)

if __name__ == "__main__":
    app = QtGui.QApplication(sys.argv)
    window = MainWindow()
    window.show()
    sys.exit(app.exec_())

It seems to work:

The exact packages are
freeze_959697.yml.txt

Can you try to install the environment from the attached file, and check whether it works?

so, what happens if you hit "Open"? Also, in Spyder, your kernel does not die?

Yes, I can try this, but probably only tomorrow.

OK after adapting your instructions to work correctly, I ran the code in spyder, now on my Ubuntu 16.04 home machine.
On first run the window opens, I can queue procedures, but they never start. Open button works correctly here, though (at least opens a file dialog). Running a second time after closing the GUI gives the same "kernel died" message.

very strange.
may you please include the result of conda env export?
On Tue, Jul 25, 2017, 23:09 Christoph Buchner<EMAIL_ADDRESS>wrote:

OK after adapting your instructions to work correctly, I ran the code in
spyder, now on my Ubuntu 16.04 home machine.
On first run the window opens, I can queue procedures, but they never
start. Open button works correctly here, though (at least opens a file
dialog). Running a second time after closing the GUI gives the same "kernel
died" message.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ralph-group/pymeasure/pull/96#issuecomment-317857207,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKuWBkMacIDUFrwnU15jtpHe3vKx2Ljqks5sRkuEgaJpZM4OdYiX
.

OK, your env file is not even installable on the Windows machine because of lots of missing packages, NoPackagesFoundError: Package missing in current win-64 channels for
- cairo 1.14.8 0
- curl=7.52.1=0
- dbus=1.10.10=0
- fontconfig 2.12.1 3
etc...

I'll send the result for conda env export on the linux machine later.

OK, so now, on the Linux machine, after starting your example a second time in spyder, I get
/home/bilderbuchi/anaconda3/envs/pymeasure/lib/python3.6/multiprocessing/semaphore_tracker.py:129: UserWarning: semaphore_tracker: There appear to be 28 leaked semaphores to clean up at shutdown
len(cache))

I'm officially thoroughly confused on the repro for these issues, I suspect some pyqt or whatever dark fairies spitting into the soup.
If you can't repro any of these issues I'm seeing at all, I just have to conclude that it's on my end (although it would be good if someone else could give this a spin, too... @cjermain?). Thanks in any case for the follow-up!
Just for completeness, here's my conda env export output:
my_conda_export.txt

Clicking on Open opens a file dialog. Everything continue to work as expected. (didn't try to open a file).

After installing an environment based on your yaml file, it still works as expected.

+1 for the dark fairies hypothesis.

did you try running the example twice? i.e. run, close it, run again (F5 in spyder)?

I use pycharm to run it.
I ran it in many ways: one-after-another, two instances at the same time,
etc. Always worked as expected.
On Wed, Jul 26, 2017 at 11:38 PM Christoph Buchner<EMAIL_ADDRESS>wrote:

did you try running the example twice? i.e. run, close it, run again (F5
in spyder)?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ralph-group/pymeasure/pull/96#issuecomment-318175221,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKuWBjbhB5WAhD7nUuwaEVkk7CoTWQ30ks5sR6OugaJpZM4OdYiX
.

Hm, yeah, I can try with pycharm, too. Thanks.

Agreed, see #105

Nice work @feinsteinben and @bilderbuchi!
I am planning to work on this and merge things during the weekend, and ship a new minor release.


Fix py.typed file creation

Currently, when building a distribution, if a py.typed file needs to be created, this happens during the generate_setup_file() step. The file is added to the typeshed directory and will get copied to the tmp dir during the copy_stubs() step. This is problematic for a few reasons:

Conceptually, generate_setup_file() makes no modifications. It just returns the setup.py file as a string. The exception being creating the py.typed file if necessary.
The build process now modifies the typeshed directory, when it should be considered "read-only".
This also means that the build process can't be started twice before cleaning the typeshed dir.
This is especially problematic in local development environments, where your local typeshed directory gets modified and you can't run the integration tests twice.

The solution is to create a new step between copy_stubs() and copy_changelog() that creates the py.typed marker directly in the tmpdir. I'll send a PR after the current batch of PRs was processed.

Btw I found why py.typed files are missing from namespace packages. The path added to package_data is just wrong. Something like this outta do it:


How to start buffering before play?

Clappr Version: 0.2.35
I'm using:
player.play();
player.pause();

This is will start buffering, but i think it's wrong way.
I can't find right api method for start buffering.

I think you can add a "preload" attribute to your video tag. It gives a hint to the browser whether it should load the video or not, although it's up to the browser to decide. You can find more about the preload attribute here (https://developer.mozilla.org/en-US/docs/Web/HTML/Element/video).
I didn't find anything on the Clappr API about the preload though.

You can use the preload and set it to auto.
Like @thalleshmm said though it is still then up to the browser if it wants to honour it or not.
This currently only applies to the html5 video playback though, not hls. Do you need this to apply to hls as well?

@param {String} [options.preload]
video will be preloaded according to preload attribute options default: 'metadata'

Thank you all!


clear_cache should have an --all option

Hi,
It would make life a bit easier if the clear_cache command had an --all option, to clear all caches instead of having to name them individually.
Cheers

FYI: django.core.cache.caches.all() only returns those caches that have been touched at least once (tested on 1.11 and 2.0). So if we want to implement a "clear all caches", we probably (AFAIU) need to loop over settings.CACHES instead of relying on django.core.cache.caches.all().

Yeah sounds really useful.
More then happy to accept a PR for this !

@trbs happy to oblige, here you go: #1203 :)


There are nvidia-specific features in the mainline_merge branch that allow Nvidia to work without glamor, because before their ~500 Linux driver release (I can't remember the exact version) they did not have egl.
The next wrinkle, however, is the DRI3 extension, which I verified with Jay Nvidia does not support yet. This suggests you may get some benefits of the Nvidia acceleration on glamor, or it'll simply crash. We should experiment with this. Note that there is one caveat here. Nvidia broke something with the "present" extension in their latest drivers on Linux, so you'll need this patch in your xserver: https://gitlab.freedesktop.org/xorg/xserver/-/commit/22d5818851967408bb7c903cb345b7ca8766094c
Or you'll need this patch:
diff --git a/module/rdpMain.c b/module/rdpMain.c
index 192cd70..7148560 100644
--- a/module/rdpMain.c
+++ b/module/rdpMain.c
@@ -46,6 +46,15 @@ rdp module main
 #include <xf86xv.h>
 #include <xf86Crtc.h>

+#if XORG_VERSION_CURRENT < XORG_VERSION_NUMERIC(21, 1, 4, 0, 0)
+#define XACE_DISABLE_DRI3_PRESENT
+#endif
+
+#ifdef XACE_DISABLE_DRI3_PRESENT
+#include <xacestr.h>
+#include <xace.h>
+#endif
+
 #include "rdp.h"
 #include "rdpInput.h"
 #include "rdpDraw.h"
@@ -208,6 +217,27 @@ xorgxrdpDamageDestroy(DamagePtr pDamage, void *closure)
     LLOGLN(0, ("xorgxrdpDamageDestroy:"));
 }

+#ifdef XACE_DISABLE_DRI3_PRESENT
+/*****************************************************************************/
+static void
+xorgxrdpExtension(CallbackListPtr *pcbl, void *unused, void *calldata)
+{
+    XaceExtAccessRec *rec = calldata;
+    LLOGLN(10, ("xorgxrdpExtension:"));
+    LLOGLN(10, ("  name %s", rec->ext->name));
+    if (strcmp(rec->ext->name, "DRI3") == 0)
+    {
+        LLOGLN(10, ("  disabling name %s", rec->ext->name));
+        rec->status = BadValue;
+    }
+    if (strcmp(rec->ext->name, "Present") == 0)
+    {
+        LLOGLN(10, ("  disabling name %s", rec->ext->name));
+        rec->status = BadValue;
+    }
+}
+#endif
+
 /******************************************************************************/
 /* returns error */
 static CARD32
@@ -231,6 +261,12 @@ xorgxrdpDeferredStartup(OsTimerPtr timer, CARD32 now, pointer arg)
             LLOGLN(0, ("xorgxrdpSetupDamage: DamageRegister ok"));
             TimerFree(g_timer);
             g_timer = NULL;
+#ifdef XACE_DISABLE_DRI3_PRESENT
+            if (getenv("XORGXRDP_NO_XACE_DISABLE_DRI3_PRESENT") == NULL)
+            {
+                XaceRegisterCallback(XACE_EXT_ACCESS, xorgxrdpExtension, NULL);
+            }
+#endif
             return 0;
         }
     }

As to the BUS ID, can you provide more detail so we can fix this? Sounds like a bug in the script still.
ah, I worked it out, sed is missing -i so the changes never get written.

Also can add -y to the apt commands for speedy testing.
Cheers!

@noaho Does
sudo sed -i -E 's/(BusID "PCI:)[[:digit:]]+(:0:0")/\1'$BUS_ID'\2/' /etc/X11/xrdp/xorg_nvidia.conf
Work for you now?
I updated the apt commands

It did work this morning when I added the -i earlier. I didn't test it if you changed anything else.

Is it possible to enable H.264 in Hyper-V with virtual GPU enpowered by GPU-PV technology? I've managed to configure GPU-PV working properly with OpenGL.

@seflerZ I'm trying to figure this out myself. It would be so nice if I could figure out a way to do Hyper-V GPU accelerated XRDP or using it with WSL2. It would greatly ease development, at least. However, I'm not hopeful at this point... I think GPU passthrough with Hyper-V doesn't work unless you have Windows Server, and I've not been able to get it working with WSL2 in spite of the Nvidia drivers appearing to work correctly...

@Nexarian Your efforts are highly appreciated. I'll try to figure it out too.

@seflerZ Here's my issue on trying to get things working in WSL2: https://github.com/neutrinolabs/xrdp/issues/2328

@Nexarian Got it. In the mean time, I wonder if we could use the FreeRDP core lib. As I suggested before, the GNOME-REMOTE-DESKTOP is based on it and has the H.264 fully supported both on CPU and GPU. This maybe much easier.

@seflerZ My goal is excellent remote desktop on Linux in whatever form that may take. Currently, I've come to believe that XRDP is the best path forward for that. If you have a compelling case to make that this isn't true, I'm all ears.
There are a couple caveats to what you say:

If you use the mainline_merge branches that I maintain, XRDP is fully supported on CPU and GPU as well... In fact, I'd argue that it works better than Gnome-Remote-Desktop. GRD in Ubuntu 22.04 has serious limitations, though I do acknowledge that in the future some of this will be solved.
What do you believe isn't supported beyond (1)?
FreeRDP and XRDP used to be sister projects. They no longer collaborate as heavily due to historical reasons (I wasn't there). However, I believe the two biggest problems are that FreeRDP is C++ (XRDP is strictly C), and FreeRDP's interface changes rapidly and it's hard to keep up with all the changes, which is why GRD is largely written by the same people who maintain FreeRDP.

In fact, I'd argue that it works better than Gnome-Remote-Desktop.

You can do that as a politician. Testing is one proper way as a base, instead of spreading speculations. Unlike xrdp, g-r-d does not impose an FPS limit of 25 frames per second.
Currently, g-r-d works best with the NVIDIA drivers (and NVIDIA gpu) on the server side present.
Grab mstsc and measure the time periods for frame encodings for a selection of setups, frame rates (you do have to ensure, that the screen changes (g-r-d culls unnecessary updates out)).
But also account for the client side: g-r-d accounts the client decoding and display rate too. If the client is too slow, frames might need to be dropped (which is actually why measuring the time frame for encodings is more accurate than measuring any FPS values).
Also, you do not account for one important aspect: The frame latency, which is also impacted by the RTT.
Without RTT handling, you cannot handle slow(er) clients at the same time, while also being able to handle clients that are fast, but where the network is impacted by medium or high round trip times (RTT).

FreeRDP and XRDP used to be sister projects.

They are two different projects. FreeRDP is not xrdp-ng (or FreeRDS or ogon). Jay, as one of the FreeRDP members from its beginning (at least on github), can confirm that.

However, I believe the two biggest problems are that FreeRDP is C++

Not true in the slightest way. FreeRDP is written in C.

FreeRDP's interface changes rapidly

Wrong. This changed long time ago. With the release of FreeRDP 2.0 two and a half years ago, this is no longer the case: FreeRDP has the master branch (development) and a stable branch for the stable releases.
Also without these stable releases, g-r-d wouldn't be able to use the FreeRDP library.

which is why GRD is largely written by the same people who maintain FreeRDP.

Also not true. I don't know why you are make up things here, but this is just a lie.
The correct story is:
g-r-d as a project was created 2015 by Jonas as a replacement for vino (which is now long deprecated and the repo is archived).
(vino is x11 only (x11 is btw also in the process of dying (upstream xorg development stalled (effort is on XWayland in that repo)))).
Headless sessions were never really supported via vino, so g-r-d was written for the remote assistance scenario (to be able to replace vino) (which is much easier and also a requirement for the headless scenario).

In April 2020, I experimented a bit with the RDP backend in weston and the VNC backend in g-r-d and created an initial RDP backend for g-r-d.
Coincidentally, few weeks earlier FreeRDP 2.0 was released.
And no, I was not part of any FOSS project at that time.
Over the time, the RDP backend in g-r-d grew in functionality, clipboard support (text, images, files (or folders with files (both sides))), extended with clipdata locking support (g-r-d is btw the only RDP server on linux supporting clipdata locking (quality of experience feature)), support for the graphics pipeline, support for virtual monitors and dynamic resizing of virtual monitors, etc..
Meanwhile, Jonas added support for virtual monitors in mutter (where I added the hook-up in g-r-d), as initial part of headless session support (which is ongoing). The project itself is maintained by Jonas (mutter dev) and can himself only dedicate a small portion of his time to g-r-d.
Me, I am just a free time contributor (on g-r-d) and can also only dedicate a small portion of time to g-r-d (I am the one writing the RDP specific code).
And these are all g-r-d devs, no FreeRDP maintainer or what else you make up here.
Small project, two people dedicating some time on that project (The Ubuntu desktop team (or anyone else from Canonical) does not participate and never contributed anything related to the screencast and remote-desktop abilities in GNOME).
Just because I contribute occasionally to the FreeRDP project does not make me a FreeRDP maintainer. Just because I was added to the FreeRDP member list recently, does not make me a maintainer. Same applies to Jay, who is btw a FreeRDP member too: https://github.com/orgs/FreeRDP/people
Your premise and your conclusion are therefore wrong.
 I don't know your motivation here, you don't have it necessary to make things here up.
Do some research, instead of making speculations.
Speculations and other nonsense is already spread by users (they don't read documentation or similar), no need to support their effort in this area.

GRD in Ubuntu 22.04 has serious limitations

While you haven't mentioned any limitation here, three things:

Like written on the wiki, g-r-d is currently remote assistance only. (And as mentioned above headless session work is ongoing) (I suspect you thought about this here.)
Ubuntu takes due to its SRU process a very long time to push updates to users. I partly have to "stand behind the Ubuntu desktop team with a whip", so that they package any required updates and for the SRU part (where the SRU team pushes the packaged package to the proposed repo to let it wait >= 1 week in there before pushing it to updates), nobody can accelerate that. As of today, Ubuntu is still several mutter updates (stable releases) behind, which are necessary for g-r-d.
As mentioned above, the Ubuntu desktop team does not participate in upstream development regarding anything remote desktop related.

@pnowack Lets keep it polite hear, you will not make many friends that way.
You can correct people without swearing.
I do feel you've said some things that are not true.
Create a discussion issue if you want to talk more about a common GFX library.

@pnowack Most of us are part-timers doing the best we can, and if I have mischaracterized or misunderstood something, it was in good faith. Am I biased? Yes, a little, because I've been contributing to XRDP, I want to see it succeed, but I also concede that I haven't tried GRD yet. I am happy to discuss a way forward. As I said, my ultimate goal, and I think the ultimate goal of all of us, is to have an excellent, frictionless Linux remote desktop experience. RDP seems to be a good way to do that, and I would support a unification of efforts so we can all row our oars in the same direction as we're all ultimately in the same ocean.
We still need to answer @seflerZ's question. I think for now, XRDP and GRD are separate projects.

Let's work all together to make an excellent, frictionless Linux remote desktop experience. @Nexarian I've compile and configured your H264 mainline_merge branch. It works great! Thanks for you hard work! Though I don't know why the FPS was hard coded to 24 but it can be changed easily. After switching to 60fps, the experience is smooth enough.
So the only question is how to activate the hardware acceleration? I'm using GPU-PV in Hyper-V Linux configured with Intel HD 770 Graphics. I've seen the decoder works when playing H264 movies using VLC player. I'll take a look into it. Theoretically. it won't be hard to take advantage of the hardware unless Microsoft has limitation on the GPU-PV technology.

@Nexarian I've done some research and found that currently it most likely not being able to enable H.264 hardware decoding in WSL because the GPU-PV WSL used is a kernel API passthrough technology. They only has Mesa implementation at present which means openGL and some 3D accerleration are supported. As for the hardware decoding using VA-API, there is no driver now. There is no way (I suppose) to have full GPU features except using DDA(Discrete Device Assignment) in VMs.

@seflerZ That is really annoying, but is in line with what I have come to expect from Windows virtualization. Note that the opposite works extremely well. Windows VMs hosted on Linux can have full access to the GPU passed through from the Linux host, just not the other way around (which is what we'd want!)
Maybe when WSL 3 comes out...
Thanks for the valiant effort! If you still have time or motivation to help, I can always use assistance on getting the mainline_merge branches reviewed and merged into devel!

@Nexarian Okay, I've been using this branch for a while and found it stable enough to merge. There're two issues at present:

When using RFX Pro, there are scraped images some times.
The frame is locked to 25, I think we can change it to 60 and it behaves fine at present in my computer.

Should we fix them or merge the banch first? If do merging first, I'll try it.

@seflerZ So we can't just merge this wholesale. The changes are gigantic and have been in the works for years. We need to take it slow. The current PR I have for the next step is here: https://github.com/neutrinolabs/xrdp/pull/2338
To see how big it would be to merge it in take a look:
https://github.com/neutrinolabs/xrdp/compare/devel...Nexarian:xrdp:mainline_merge?expand=1
https://github.com/neutrinolabs/xorgxrdp/compare/devel...Nexarian:xorgxrdp:mainline_merge?expand=1
These need to go through code review and testing and release gradually, otherwise we could cause issues.

glad to see there is some progress. After too long time I had time to look again on the progress of the egfx progress.
The version I tested was https://github.com/Nexarian/xrdp mainline_merge and https://github.com/Nexarian/xorgxrdp mainline_merge
My findings are the following

running desktop session, not the transmission)
nvidia accelerated works, but not through glamor (this is not a issue, you only need to know since nvidia configuration differ from others)
other gpu's work using glamor.
transmission with h264
works with helper, but only with nvidia. you get hardware accelerated encoding. When launching plasma I get glitches after the splash screen, but after reconnecting it works like it should (I think this is a issue of nvenc or the ffmpeg decoder in freerdp)
without hardware encoder (that means without using the helper) the required code is present, but two "issues" prevent it to kick in. Some small code changes make it work, but I would like to discuss this with someone who understands this more deeply. Namly why only XR_RDPGFX_CAPVERSION_104 allows enabling h264 (and not ex. XR_RDPGFX_CAPVERSION_105; xfreerdp sends this one) and what lp.size should be set to (that is missing and prevents encoding since out_uint32_le thinks there is no space:

    out_data_bytes = 16 * 1024 * 1024;
    index = 256 + 16 + 2 + enc->num_drects * 8;
    out_data = g_new(char, out_data_bytes + index);
    if (out_data == NULL)
    {
        return 0;
    }

    s = &ls;
    g_memset(s, 0, sizeof(struct stream));
    ls.data = out_data + 256;
    ls.p = ls.data;

To be able to make a broader test without the need of custom compiles everywhere I made myself a ubuntu ppa where I've uploaded both xrdp and xorgxrdp matching Nexarians mainline_merge branch with some Ubuntu specific tweaks and the two changes mentioned in my previous post.
Given the size of the mainline_merge diff imho it would not be a bad idea if more people are able to test this branch. This ppa should it make easier for many to test out and if issues arise at least we know that it is not a configure or other type of compile issue. If there is no veto from @Nexarian  or @jsorg71 I will post a link to a page with further information.

@akarl10 The default framerate of Nexarian's branch is limited to 25. I think it's  okay to change it to 60 to have a better experience. Besides Nexrarian uses Nvidia's xorg configuration by default which is not suitable for every hardware, I think.

@akarl10 The default framerate of Nexarian's branch is limited to 25. I think it's okay to change it to 60 to have a better experience. Besides Nexrarian uses Nvidia's xorg configuration by default(can be easily changed) which is not suitable for every hardware, I think.

@seflerZ I could change that, but currently I left that how it is.Rregarding the xorg.conf, there is a postinstall script that takes care of it (checks if its nvidia, changes sesman.ini, patches bus id). In theory it should work oob for "everyone".

Hello, I'm having a weird problem that nvenc works fine on one machine but fails to open a encode session on the other. Both of them run the latest Debian 11, and their software setups are nearly identical, GPU details are as follows:

GPU
Driver
Work?
Comment

Quadro K420
470.141.03, from Debian non-free
✅
470 is the last version supporting K420

NVIDIA T600
520.61.05, from cuda's deb repo
❌

My xrdp and xorgxrdp are built from @Nexarian's latest mainline_merge branches, with some downstream Debian patches applied, the forks can be found at https://github.com/MoetaYuko/xrdp/tree/mainline_merge and https://github.com/MoetaYuko/xorgxrdp/tree/mainline_merge. Hardware rendering works on both machines according to glxgears -info.
The command to connect from client side is xfreerdp /bpp:32 /v:SRV /u:USER /p:PWD /w:1024 /h:768 /gfx:avc420, and I've done the following experiments with the T600 one:

With the standard setup, xfreerdp shows a black screen and exits soon after starting, nvEncOpenEncodeSessionEx returns NV_ENC_ERR_UNSUPPORTED_DEVICE according to xorgxrdp_helper.14.log
Replacing /gfx:avc420 with /gfx (so it uses RFX codec), RDP works fine.
Commenting XRDP_USE_HELPER=1 in /etc/xrdp/sesman.ini (so it uses X264 software encoding?), RDP works fine.

Hence, the XRDP stack works okay in general, except for nvenc accelerated h264 encoding. The returned NV_ENC_ERR_UNSUPPORTED_DEVICE from nvenc API isn't much informative.
In addition, I compiled and tested the official nvenc example for OpenGL, and I'm pretty sure nvEncOpenEncodeSessionEx called from there does NOT return an error, which means my driver setup should be fine.
Are there any ideas to fix the issue?

NV_ENC_ERR_UNSUPPORTED_DEVICE means that your GPU does not support NVENC. In general nvEncOpenEncodeSessionEx is never guaranteed to succeed, i.e. it should never be assumed, that it succeeds.
For example, if you already have 3 NVENC sessions running, nvEncOpenEncodeSessionEx will also not succeed, unless you patch your NVIDIA driver with the keylase patch (by default NVIDIA imposes a max session limit).

Thanks for the reply!

NV_ENC_ERR_UNSUPPORTED_DEVICE means that your GPU does not support NVENC. In general nvEncOpenEncodeSessionEx is never guaranteed to succeed, i.e. it should never be assumed, that it succeeds.

T600 supports most NVENC capabilities according to https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new

For example, if you already have 3 NVENC sessions running, nvEncOpenEncodeSessionEx will also not succeed, unless you patch your NVIDIA driver with the keylase patch (by default NVIDIA imposes a max session limit).

I think there aren't other programs using NVENC, and nvEncOpenEncodeSessionEx called from the official nvenc example for OpenGL succeed. But anyway, how do I know the number of running sessions?

But anyway, how do I know the number of running sessions?

It is literally in the link with the matrix, that you posted (Max # of concurrent sessions).

T600 supports most NVENC capabilities according to

I don't see a T600 here in that table. General note: After NVIDIA released the 470 driver, all Kepler cards were removed from that table. They still support NVENC though.

nvEncOpenEncodeSessionEx called from the official nvenc example for OpenGL succeed

Don't know what that example really calls under the hood. In any case, if the driver tells you, that the device does not support NVENC, it is either the driver or the device, i.e. nothing API user related.

I don't see a T600 here in that table. General note: After NVIDIA released the 470 driver, all Kepler cards were removed from that table. They still support NVENC though.

T600 is listed in the second Professional (NVIDIA RTX / Quadro) tab, Consumer (GeForce) is selected by default.

nvEncOpenEncodeSessionEx called from the official nvenc example for OpenGL succeed

Don't know what that example really calls under the hood. In any case, if the driver tells you, that the device does not support NVENC, it is either the driver or the device, i.e. nothing API user related.

The example app sets the options for opengl variant (the one used by xrdp) from here and calls nvEncOpenEncodeSessionEx at here.
Besides, I tested the scenario with over 3 nvenc sessions on my K420 machine, and nvEncOpenEncodeSessionEx returns NV_ENC_ERR_OUT_OF_MEMORY instead.

T600 is listed in the second Professional (NVIDIA RTX / Quadro) tab, Consumer (GeForce) is selected by default.

Ah, right.

The example app sets the options for opengl variant (the one used by xrdp) from here and calls nvEncOpenEncodeSessionEx at here.

For the variant with the GL device, the GL context needs to be current. The other way is using NVENC with a CUDA device.

For the variant with the GL device, the GL context needs to be current. The other way is using NVENC with a CUDA device.

Well, they're implementation-wise considerations. Now that the same program worked on one machine but not on the other...

I have good news! After a lot of testing and working with @jsorg71, I've been able to get a prototype script for Intel YAMI H264 acceleration working on XRDP on Ubuntu 20.04!
xrdp-intel-setup.sh
I've also codified the latest updates for Nvidia acceleration as well, though I haven't tested it recently:
xrdp-nvidia-setup.sh

Not sure if this is the right place to report this, but issues and discussions don't seem to be able on the fork. I've noticed that it becomes impossible to connect to a remote host running the nvidia xrdp/xorgxrdp variant if the host has been suspended and re-awakened. Prior to suspend, it's perfectly reachable. After suspend, Remmina goes into a reconnection loop and this never succeeds.
I've tried restarting xrdp, xrdp-sesman. I've also tried stopping those services, killing any linger xrdp related processes and Xorg sessions, then restarting xrdp, xrdp-sesman. Nothing seems to get the system back to a state where a connection can be made with xrdp. Rebooting the remote host will get it back again.
I do not notice this behavior so far with regular xrdp/xorgxrdp - suspending and awakening seems to be OK.
I'm not sure how to debug this or provide sufficient information in case this is not already known (a search here didn't turn up mentions of suspend other than pipewire)

@philstopford Sorry for the late reply! I've had my hands full debugging AVC444, which is still not working unfortunately.
Anyway, to answer your question, it's very likely that the culprit is something @jsorg71 created to enable Gnome 3 to work in the first place called lrandr -- In it, we replace the RandR extension with a facsimile of the same name in order to make Nvidia work. It is highly probable that one of the functions in the facsimile is either not doing the right thing on suspend/resume or we need to implement one of the few that's left. Looking in the ~/.xorgxrdp.*.log files should tell you when an unimplemented version is being called.
The other thing you can try is switching your session to use Mate or Cinnamon instead of Gnome 3, which doesn't require LRandR for Nvidia use-cases (https://ubuntu-mate.org/, https://github.com/linuxmint/Cinnamon)
To do this, rebuild and install xorgxrdp without the enable-lrandr flag (you can just modify the install script to not have it if you're not comfortable tweaking this manually and re-run the script)
Then, in ~/.xsession change the command to mate-session or cinnamon-session and then move ~/.xsessionrc to ~/.xsessionrc.bak
Then reboot.

@Nexarian : will investigate. I was remiss in not noting that I've been using Plasma for the RDP sessions. I used to use Cinnamon for RDP, but couldn't get that going in recent attempts. I will re-check that as well.

I don't know if Plasma NEEDS the LRandR extension or not. You can also try rebuilding xorgxrdp without it and see if it helps.

@Nexarian : nonNV.xorgxrdp.10.log is from the default xrdp/xorgxrdp pairing. NV.xorgxrdp.10.log is after I removed the default xrdp/xorgxrdp, and install using the script (without lrandr), then reboot. In the former, Cinnamon is started nicely and works, although slowly. In the latter, I get an error that the X server could not be started.
The xorg_nvidia.conf file looks OK.
notNV.xorgxrdp.10.log
NV.xorgxrdp.10.log

To be able to make a broader test without the need of custom compiles everywhere I made myself a ubuntu ppa

@akarl10 - Thanks for this, a few years ago I created a batchfile to build XFCE and KDE Neon desktops in WSL that use xrdp instead of requiring an xserver.  I just switched to a rebuild of your DEBs and rendering only with Mesa in WSL1.
Even with a sofware-only GPU pipeline, there is a very noticable performance improvement over main.  I can force full desktop composition in both desktop environments and they're flying on modest hardware.  Basic games like TuxRacer are actualy playable --Excellent work @Nexarian!

@DesktopECHO Most of the credit goes to @jsorg71. I largely took his code and maintained a branch that works off of rebased devel. The resizing code was mostly my work though. I appreciate the +1 :)

I see now almost a decade in the making... https://xrdp-devel.narkive.com/kAr4I3mP/codec-mode-xrdp
Thank you @jsorg71 for your work on this incredible tool.

The resizing code was mostly my work though.

I use a Mac and accicdentally ran into this grabbing an RDP window.  When the window resized and the RDP resolution snapped along with it I was in shock!

:) It took me over a year to get that code stable! In any case, the next things we're working on:

Multi-monitor (mostly Jay)
AVC444 (Full color so no more weird/bad text colors)
Relative mouse X/Y axis support so gaming will work.

Then very long term we're thinking:

Wayland support
Network Level Authentication support
Prototype AV1 encoding for even-more-faster performance! Since the RDP protocol doesn't ACTUALLY support that yet, we'd have to use FreeRDP as our client, but if we can get it the performance will be worth it!

@Nexarian - I've had some thoughts about NLA recently which I'll be happy to share with you when the time comes.

@Nexarian @jsorg71 I can't express how much I wanted good RDP performance on Linux hosts. You guys did an amazing job by bringing this to Linux!
As I'm not very technical, could you please clarify which of those features have been integrated into the main XRDP project, if any? If they haven't been integrated yet, what is the timeline for this? so I can just apt install it.

I'm also curious to know whether this is getting upstreamed.

@rcarmo - follow #2891 which is pretty active right now.

* Prototype AV1 encoding for even-more-faster performance! Since the RDP protocol doesn't ACTUALLY support that yet, we'd have to use FreeRDP as our client, but if we can get it the performance will be worth it!

Awesome work on all the progress so far! Thanks for your time
I'm just curious, why would AV1 be faster than H264? I thought H264 would be using hardware codec and the latency would be hard to beat? Or do you mean compression/efficiency improvements?

#2891 has been merged. The next version v0.10.0 supports GFX. H264 is not yet supported at the moment.


Error: 'jsx' is not exported by ../../node_modules/react/jsx-runtime.js

Describe the bug
I got this error when building:
Error: 'jsx' is not exported by ../../node_modules/react/jsx-runtime.js, imported by ../../node_modules/react-dnd/dist/esm/core/DndProvider.js
i've added this to rollup.config.js
plugins: [
   commonjs({
       include: '../../node_modules/**',
       namedExports: {
           '../../node_modules/react/jsx-runtime.js': ['jsx']
       }
   }),
]

Reproduction
react.js version: 17.0.2
react-dnd version: 14.0.2
typescript version: 4.2.4

nvm it's a typo


update supabase js

What kind of change does this PR introduce?
Update to supabase-js 2.0.4
What is the current behavior?
We are currently missusing setSession(session) to refresh the session in sveltekit.
What is the new behavior?
Use the new refreshSession() method

This seems a duplicate of #331

No, this PR makes use of the new method refreshSession as described above

@david-plugge do we need this to be a new release?

Does it make sense to split the PR in

Update supabase-js to most recent version
Switch to refreshSession

So it's more clear, what is going on

Does it make sense to split the PR in
1. Update supabase-js to most recent version

2. Switch to refreshSession

So it's more clear, what is going on

Its only a small change but we can write this in the changeset.


Add descriptor keys with support for secrets and signers

I'm opening this right PR now mostly to gather feedback, since many things are still missing and need a lot of polishing.
This PR builds on top of #93 (here's the diff with it), so it incorporates the DescriptorKey concept from @sgeisler and extends it to also support parsing of secrets (WIF or xprv) in Descriptors.
It implements pretty much what @apoelstra described in #93, which is having a way to parse a descriptor with secrets, but immediately separate them out and store them in a map, and only have public types in the descriptor. This is achieved with the SplitSecret trait that can be implemented on types that are MiniscriptKey that may contain secrets, and it adds a method split_secret() that when called returns a tuple of, roughly speaking, (PublicType, Option<SecretType>).
This PR also adds the Signer trait which at the moment is very simple but again, I wanted to hear some feedback on how to move forward with it, and especially how to make it interact in a nice way with the Satisfier trait.
Also, I don't particularly like the name I chose for some structs, like DescriptorKeyWithSecrets or DescriptorAndSigners so if you have better ideas I'd love to hear them!

I pushed some experiments I've been doing with the Signer trait. I still don't feel 100% confident about the interface, specifically if it's better to take a &mut psbt::PartiallySignedTransaction and apply the signatures inside the signer (what's currently implemented) as opposed to taking a normal reference, returning the signature and letting the caller handle everything else.
I initially liked the first approach better because it hides the logic of pushing the signature in the partial_sigs map from the caller, but that actually forces me to clone the psbt somewhere, which is not optimal.
I guess the &mut approach is better if we expect people to directly use the signers outside of the code provided by this crate, while the other one is less "user-friendly" but it allows for some minor optimizations inside this code.
The other big question is whether I should add an explicit flag to ask for a BIP143 signature or try to automatically determine it by looking at the witness_utxo/non_witness_utxo fields like I'm doing now. Considering the recent issue with Trezors and the fact that they decided to require non_witness_utxos for segwit psbts too, I think it might be necessary to have an explicit flag to avoid issues down the road.

The other big question is whether I should add an explicit flag to ask for a BIP143 signature or try to automatically determine it by looking at the witness_utxo/non_witness_utxo fields like I'm doing now. Considering the recent issue with Trezors and the fact that they decided to require non_witness_utxos for segwit psbts too, I think it might be necessary to have an explicit flag to avoid issues down the road.

I don't completely follow this, what is the issue to determine by looking at witness_utxo field? Can you elaborate how this might lead to more issues ahead?
As for the role of miniscript in PSBT stack, we originally thought that miniscript should only be required for finalizing psbt. Other roles must fill in the partial sigs as required. I read BIP32, BIP 174 a couple days ago, so maybe I am wrong with these. We only considered the following workflow

The creator creates a psbt with unsigned tx and passes it on for signing.
The signers can analyze the Miniscript, infer which keys are in the miniscript; sign the tx with appropriate sighash flag and add the partial sigs to the pbst.
The finalizer finally looks at the partial sigs and each inputs and creates the transaction. The role of satisfier triat is to construct the final pbst as implemented here. https://github.com/rust-bitcoin/rust-miniscript/blob/87a47e242dc19c517ca06c558cf79b43eec22abf/src/psbt/mod.rs#L125

Please note that the pbst implementation is incomplete and not tested.
My understanding that the overall flow should be as follows: (cc @apoelstra )
fn main() {
    // get descriptor and signers
    let descriptor, signers = .....

    /// Create unsigned initial psbt
    let psbt: bitcoin::util::psbt::PartiallySignedTransaction = .......

    // Note that this could be a distributed process
    // Sign each input and
    for input in psbt.inputs{
    // Add sig for each input. Again could be distributed process.
        for signer in signers{
        //Add partial sig here.
        signer.sign(&mut psbt, input_index)
        }
    }
    // Pass on to the finalizer. The Satisfier trait is already implemented for psbt input.
    pbst::finalize(& mut psbt);
}

I did not look at all of the code yet, but most of it looks good.
A couple of questions that I am still unclear about:
Is there any special need for DescriptorWithSigners struct to implement FromTree. I think that this only requires FromStr and then the struct MiniscriptWithSigners should no longer be required.

I don't completely follow this, what is the issue to determine by looking at witness_utxo field? Can you elaborate how this might lead to more issues ahead?

I don't know how realistic this is, I was just thinking that maybe some wallets could decide in the future to always include non_witness_utxos and potentially remove witness_utxos entirely to save some space, because after all you can find everyting you need from the non_witness_utxo. If that was the case, PSBTs built by these wallets would be mistaken for non-segwit with the logic I have implemented currently.

As for the role of miniscript in PSBT stack, we originally thought that miniscript should only be required for finalizing psbt. Other roles must fill in the partial sigs as required.

If you feel like the signers are out of scope I can totally remove them. There was a comment on #93 (https://github.com/rust-bitcoin/rust-miniscript/pull/93#discussion_r450488856) that said "and have our [signers] map use the Satisfier trait to allow users to produce signatures etc with it." so I interpreted that as "implement Satisfier on SignersContaner so that it can produce signatures on-demand", but I might have just misunderstood something.

I think that this only requires FromStr and then the struct MiniscriptWithSigners should no longer be required.

Yeah, I think you are right. I was trying to keep the structs "parallel" (Miniscript -> MiniscriptWithSigners, Descriptor -> DescriptorWithSigners) but now that I think about it, the signers pretty much only make sense with descriptors, so I could remove it MiniscriptWithSigners. I'll try working on that today.

I agree that PSBT signing is out of scope ... if only because it is so simple. As Sanket says, you just compute the sighash (which rust-bitcoin has functionality for), produce a signature (i.e. call sign in rust-secp) and attach it to the PSBT (and the PSBT struct exposes a BTreeMap for you to insert the signature in). Given this, I'm not sure I see the purpose of the Signer trait.
As for implementing Satisfier on a map from public keys to private keys (and from Pk::Hash to private keys), I think this seems totally reasonable. And I agree that the way to do this is to create something like the PSBTSigningContext structure which contains a reference to a PSBT, a reference to a map, and an input index. But the implementation in this PR seems overly complicated. I will do a detailed review with more comments.

Some more thoughts:

I think DescriptorWithSigners and and MiniscriptWithSigners and DescriptorKeyWithSecrets is too clever by half. I think it'd be better to have two concrete types, DescriptorPublicKey and DescriptorSecretKey, which are basically the same except that one has secret-key variants of the other.
Then I think rather than having a separate type DescriptorWithSigners where we impl FromStr on it, we should just add a method to Descriptor called parse_secret or something, which returns a Descriptor<DescriptorPublicKey> as well as a HashMap<DescriptorPublicKey, DescriptorPrivateKey>. I think it's fine to wrap the HashMap in a new type, maybe call it KeyMap
Then you can implement a method on KeyMap which takes an input index and a mutable reference to a PSBT, which computes a signature and inserts it in the right place. (Notice that this does not actually involve Miniscript or descriptors at all, it's just a convenience function...I am ambivalent about whether we ought to have it in this library at all.)
Then you can implement Satisfier directly on Psbt, which takes signatures from the input map, scripts from the input map, and outputs a valid witness.

Regarding my comment in 93 about "using Satisfier to produce signatures", I was just wrong. The Satisfier trait's job is to produce a witness, given signatures, not to produce signatures (which it cannot do because the API provides nowhere to input a sighash nor anywhere to output a signature). If we add such a function, and I don't necessarily think we should, it should be a standalone function.

The reason why I kinda liked the idea of connecting the signing flow with the Satisfier trait is that it seemed nice to be able to only generate the signatures you require and nothing more, on-demand, but that really makes a difference only when somebody has the ability to provide multiple signatures in the same script, which is kind of a weird situation and probably doesn't justify all the complexity that signing carries: it's much easier to just sign with every key you have (probably just one in most cases) and then the satisfier will only pick the ones it needs.
So here's what I'll try to do for this PR:

Remove the Signer trait, PSBTSigningContext, etc
Rename DescriptorKey to DescriptorPublicKey
Rename DescriptorKeyWithSecrets to DescriptorSecretKey, but don't make it MiniscriptKey since we won't care about parsing directly in Descriptor::from_str (which also means removing all the custom implementation for Eq, Ord, etc since we won't need them anymore)
Add Descriptor::parse_secret that parses the string as Descriptor<String> internally first, then parses and maps every individual key using translate_pk() and at the same time populates the KeyMap with secrets, if they keys contain any
Modify ScriptContext::sighash to use transactions instead of PSBTs and move the PSBT stuff in a separate, standalone function

Hope I understood everything right, thanks for the feedback!

Just so we are on the same page: Descriptor::parse_secret would return Result<(Descriptor<DescriptorPublicKey>, HashMap<DescriptorPublicKey, DescriptorSecretKey>), _>?
Another thing I noticed is that we might want to add a Hash variant to the DescriptorPublicKey enum. It's not pretty as some methods will need to panic or we'll need to sacrifice the ToPublicKey impl. But not being able to build a pkh(<hash>) descriptor is very annoying because one doesn't even know the pubkey.

That's a neat idea to generate only the actually-used signatures. Unfortunately it doesn't really fit into the workflows we imagined or the API of the Satisfier trait, and I don't think it's worth the added complexity to try to support it.
If you wanted to do this, it is possible to do outside the library, by writing your own type that knows how to sign (I guess it would include a sighash or means to compute this) and implementing Satisfier on it. But my feeling is that this is too niche to be worth including in the library.
Everything else you say @afilini sounds good to me! @sgeisler that does sound like the right method signature.

@sgeisler regarding a Hash variant, can you bring this up on IRC and ping me and sipa? Even with PSBT, I don't think there should be any situations where you don't know a public key but still want to create a descriptor.

Even I don't know why but there is an addr descriptor in bitcoin-core that is roughly equivalent to Hash. But, even I am curious about use-cases where one might need Hash descriptor.
Also, the current implementation of PSBT finalizer is incomplete. I hope to have PR for it tomorrow.

@afilini what do you think about extending xpub/xpriv derivation paths with arbitrary tweaks, which can be useful for situations like in RGB?

It's probably out of scope in my opinion, considering that you can very easily define your own MiniscriptKey types outside of this library. It's trivial to wrap DescriptorPublicKey and store whatever you need in your custom struct.

Hi @afilini, i'd be pretty interested to see this moving forward (at least the xpub part).
I'd happily help you with it, thought about making a PR for the DescriptorPublicKey of your above list mainly based of #93. What do you think ?

I think I will have some time this weekend to come back to this and finish it up. I think most of the code is already there, it needs cleanup and docs. I was also planning to add commit https://github.com/rust-bitcoin/rust-miniscript/pull/93/commits/d9fa9455b1e8e815a7babb65bc27e2abb86d9dcb from @sgeisler in this branch since I think it could be useful.
The only thing that's missing with the "refactoring" I've done is the sighash computation part, which I was trying to rewrite as suggested to take a transaction instead of a PSBT, but for whatever reason I got kinda stuck (IIRC the interface was awful and I couldn't figure out a way to improve it) and never ended up finishing it. But since this PR doesn't do signing anymore maybe we could also go ahead without sighash and open a separate PR if we really want it.
The only thing (other than me rebasing and finishing the code) that could block merging this PR is the fact that it needs a git version of rust-bitcoin, since I don't think there has been a release yet since the required changes have been merged. I don't know the release schedule of rust-bitcoin, but I think it's been a while since the last release, so hopefully that's coming soon.
As a side note, I've been using this branch in some of my projects for the past few weeks and I don't think I've noticed any particular issue/regression with it, which is nice I guess.

The only thing that's missing with the "refactoring" I've done is the sighash computation part, which I was trying to rewrite as suggested to take a transaction instead of a PSBT, but for whatever reason I got kinda stuck (IIRC the interface was awful and I couldn't figure out a way to improve it) and never ended up finishing it. But since this PR doesn't do signing anymore maybe we could also go ahead without sighash and open a separate PR if we really want it.

Yeah, @sanket1729 has been doing some PSBT stuff related to sighash too lately if i'm not mistaking.

The only thing (other than me rebasing and finishing the code) that could block merging this PR is the fact that it needs a git version of rust-bitcoin, since I don't think there has been a release yet since the required changes have been merged. I don't know the release schedule of rust-bitcoin, but I think it's been a while since the last release, so hopefully that's coming soon.

It has just been bumped, and my cleaned up / improved version of #93 is rebased on top of #130 can be found here https://github.com/darosior/rust-miniscript/commits/descriptor_publickey (currently WIP, need some more tests squashed and raw pubkey with origin implementation: working on it locally).

This is now rebased on #131

Just rebased this on master, it's ready to be reviewed :)

Thanks, sorry it took me a while to fix this!
I think most of the comments should have been addressed: everything public is now documented and I removed pretty much all the ephemeral secp contexts except for one in Descriptor::parse_secret, mostly because I think it's not a critical method in terms of performance, and in general I think it keeps the public API a bit more "clean". But if you think this should also be removed and taken as argument I'm happy to change it, I don't have any strong opinion on that and it was just my personal preference.
Other than that, I don't think I see what you mean with your comment on Descriptor::to_string_with_secret: I mean, I can see that it does many allocations and it clones the public keys needlessly, but I don't think &String is a MiniscriptKey, so I don't think you can avoid cloning them, unless you completely change approach. If it's good for you to do that in a separate PR I could try to work on it, I guess I just need a bit of help!

What I was thinking with to_string_with_secret was that you'd make a type like
struct MapKey<'a, 'b> {
    map: &'a KeyMap,
    key: &'b DescriptorPublicKey,
}

then impl fmt::Display on this in a way that did the lookup and then either wrote the secret key or public key to the fmt::Formatter without allocating.
Unfortunately you still have to translate the input descriptor to a descriptor which uses this weird type as keys, so you can't avoid the allocations inherent in that.

I think we can fix the dissat thing after the fact.
@afilini can you rebase so that acb1164d4779e36062ce6d25c333954a9f9474d8 compiles?

Lol, I meant "audit" not "allocate"


enable tests for sample apps

adds functional_tests for codegen, rag, summarizer
adds image build for ^^ and llamacpp-cuda model-server
adds placeholder for cuda functional_tests

partially resolves https://github.com/containers/ai-lab-recipes/issues/204

We have to make sure people pay attention to this. BTW I broke make bootc

@lmilbaum PTAL

LGTM


Add darkness threshold feedback

Allow Howdy to meaningfully distinguish between a registration or compare that failed due to insufficient lighting as opposed to bad camera reads or a facial recognition miss. Report this distinction to the user.
I found that properly tuning the dark_threshold config variable, or even knowing that it needs to be tuned, was exceedingly difficult, and this change provides the user with the feedback necessary to adjust dark_threshold appropriately.
In particular, on my Dell XPS 13 9300 integrated Windows Hello camera, I found that 100% of frames were being rejected as "too dark" before even being considered for face detection under the default settings, unless I situated myself in front of a wall. These changes assisted me in finding a working value for dark_threshold.

Thanks a bunch!


Publish SonaType Central: [GENERIC_ERROR] java/net/http/HttpTimeoutException

Hi guys,
I am trying to publish to my new sonaType central and getting this generic error:
2024-11-03 07:29:17.136Z  info [SonatypeCentralClient] Uploading bundle /home/runner/work/***/***/target/sonatype-staging/1.0.1-bundle/bundle.zip to Sonatype Central  - (SonatypeCentralClient.scala:72)
2024-11-03 07:29:17.205Z error [Sonatype] [GENERIC_ERROR] java/net/http/HttpTimeoutException  - (Sonatype.scala:494)

Currently my build.sbt looks like this: build.sbt
I have verified that public gpg signature is registered to https://keyserver.ubuntu.com/
All environment variable has been added an
Probably going to try a manual publish to Sonatype central to see what have gone wrong

Looks like it's not an sbt-ci-release issue but sbt-sonatype:  https://github.com/xerial/sbt-sonatype/issues/518


URL deep-linking breaks app (blank page) on iOS

Expected behavior:
The app should start normally when being started through a custom-URL scheme.
Actual behavior:
When using a url scheme to start an app that is not running (cold-start), the app then shows a blank screen. To make it worse, the blank screen persists, even if the app gets force closed.
Steps to reproduce:

Install app1
Start app1, everything works fine
Force close app1
Click on a url scheme to open app1 (cold start)
App shows a white screen.
Force-close app1 and reopen. White screen is still there. Even restarting the device doesn't help.

The only way to resolve the blank screen issue is to remove and reinstall the app.
We currently have not set up universal links, so I can't test that.
When opening the Safari Debugger, I can see that the index.html gets loaded, but it only contains:
<!DOCTYPE html>
<meta charset="utf-8">

The XCode logs (when starting the app again after it broke):
2018-10-26 15:49:49.356528+0200 App1[2259:139119] Apache Cordova native platform version 4.5.3 is starting.
2018-10-26 15:49:49.356989+0200 App1[2259:139119] Multi-tasking -> Device: YES, App: YES
2018-10-26 15:49:49.365818+0200 App1[2259:139119] CDVWKWebViewEngine: trying to inject XHR polyfill
2018-10-26 15:49:49.671690+0200 App1[2259:139119] CDVWKWebViewEngine will reload WKWebView if required on resume
2018-10-26 15:49:49.671790+0200 App1[2259:139119] Using Ionic WKWebView
2018-10-26 15:49:49.672481+0200 App1[2259:139119] [CDVTimer][console] 0.123978ms
2018-10-26 15:49:49.672639+0200 App1[2259:139119] [CDVTimer][handleopenurl] 0.077009ms
2018-10-26 15:49:49.674504+0200 App1[2259:139119] [CDVTimer][intentandnavigationfilter] 1.837015ms
2018-10-26 15:49:49.674594+0200 App1[2259:139119] [CDVTimer][gesturehandler] 0.056982ms
2018-10-26 15:49:49.674682+0200 App1[2259:139119] CDVIonicKeyboard: resize mode 1
2018-10-26 15:49:49.675409+0200 App1[2259:139119] [CDVTimer][keyboard] 0.735998ms
2018-10-26 15:49:49.680010+0200 App1[2259:139119] [CDVTimer][splashscreen] 4.550934ms
2018-10-26 15:49:49.683888+0200 App1[2259:139119] [CDVTimer][statusbar] 3.825903ms
2018-10-26 15:49:49.684009+0200 App1[2259:139119] [CDVTimer][socialsharing] 0.084043ms
2018-10-26 15:49:49.684475+0200 App1[2259:139119] [CDVTimer][ionicdeeplinkplugin] 0.431061ms
2018-10-26 15:49:49.684505+0200 App1[2259:139119] [CDVTimer][TotalPluginStartup] 12.222052ms

Sadly I don't know how to get the native logs of the app start when cold-starting the app with a deep-link. If someone can explain how to do that and if it's helpful, I can post those logs as well.
It looks obvious that for some reason, a wrong index.html gets loaded and therefor the app never starts (correctly). But I'm not sure where this index.html is coming from. I first thought it might be this: https://github.com/ionic-team/cordova-plugin-ionic-webview/blob/570b1f7ef88ccf70d1c47fa445b921c34221018e/src/ios/GCDWebServer/Responses/GCDWebServerErrorResponse.m#L80, but that looks a little different.
Additional Info:
We currently have our app in the store with cordova-plugin-ionic-webview version ^1.2.1. We noticed that when cold-starting the app via deep-links, only assets could get loaded that have been loaded previously when the app was running. But other assets (images, js files, etc) that were not used before, simply were not found and returned a 404.
We noticed this because some images, as well as a js file used in a webworker, were simply not loading sometimes. It took quite a while to figure out that it was due to the deep-linking cold start.
We would be perfectly happy with v1 if this issue was resolved, but I suppose you only plan on fixing bugs in v2?
Info:
Ionic:

   ionic (Ionic CLI)  : 4.1.2 (/Users/andreas/.nvm/versions/node/v10.3.0/lib/node_modules/ionic)
   Ionic Framework    : ionic-angular 3.9.2
   @ionic/app-scripts : 3.1.9

Cordova:

   cordova (Cordova CLI) : 7.1.0
   Cordova Platforms     : android 7.1.0, ios 4.5.4
   Cordova Plugins       : cordova-plugin-ionic-keyboard 2.1.3, cordova-plugin-ionic-webview 2.2.0, (and 11 other plugins)

System:

   Android SDK Tools : 26.1.1 (/Users/andreas/Library/Android/sdk)
   ios-deploy        : 1.9.2
   NodeJS            : v10.3.0 (/Users/andreas/.nvm/versions/node/v10.3.0/bin/node)
   npm               : 6.1.0
   OS                : macOS
   Xcode             : Xcode 10.0 Build version 10A255

com.lampa.startapp 6.1.6 "startApp"
cordova-plugin-add-swift-support 1.7.2 "AddSwiftSupport"
cordova-plugin-app-version 0.1.9 "AppVersion"
cordova-plugin-device 1.1.7 "Device"
cordova-plugin-inappbrowser 1.7.2 "InAppBrowser"
cordova-plugin-ionic-keyboard 2.1.3 "cordova-plugin-ionic-keyboard"
cordova-plugin-ionic-webview 2.2.0 "cordova-plugin-ionic-webview"
cordova-plugin-qrscanner 2.6.0 "QRScanner"
cordova-plugin-splashscreen 5.0.2 "Splashscreen"
cordova-plugin-statusbar 2.4.2 "StatusBar"
cordova-plugin-whitelist 1.3.3 "Whitelist"
cordova-plugin-x-socialsharing 5.4.1 "SocialSharing"
cordova-sqlite-storage 2.4.0 "Cordova sqlite storage plugin"
es6-promise-plugin 4.2.2 "Promise"
ionic-plugin-deeplinks 1.0.17 "Ionic Deeplink Plugin"

One thing that I just noticed today: If I open the Safari Debugger while the app shows the blank page and refresh the app there, it loads correctly. So it seems to me that the deep-linking somehow "bypasses" certain startup mechanisms, which causes the app to load before the web server is ready with the actual content? I'm still not sure where that blank HTML page is coming from though.

I've noticed similar problem (after SEVERAL hours of deep debugging)
Deep links stopped working after upgrading from "cordova-plugin-ionic-webview": "^1.2.1".
I use Google Firebase Authentication with Cordova (https://firebase.google.com/docs/auth/web/cordova) and they relies on cordova-universal-links-plugin-fix (I also tried cordova-universal-links-plugin) to implement deep links.
After upgrading to cordova-plugin-ionic-webview" deep links stopped working. I had to downgrade to 1.2.1 where everything works ok!

In my case, the problem occured because both apps (the one source and target of the deep link) were running on the same port. Related issue: https://github.com/ionic-team/cordova-plugin-ionic-webview/issues/165


[QUESTION]  composer update -o 异常

Before you submit this issue, you has been search all existed issues and search the documentation

环境如下:
Linux e5d770f177de 3.10.0-123.el7.x86_64 #1 SMP Mon Jun 30 12:09:22 UTC 2014 x86_64 Linux
PHP 7.2.25 (cli) (built: Nov 22 2019 13:45:46) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
hyperf/amqp                          v1.1.22  A amqplib for hyperf.
hyperf/cache                         v1.1.22  A cache component for hyperf.
hyperf/command                       v1.1.21  Command for hyperf
hyperf/config                        v1.1.22  An independent component that provides configuration container.
hyperf/constants                     v1.1.24  A constants component for hyperf.
hyperf/contract                      v1.1.22  The contracts of Hyperf.
hyperf/crontab                       v1.1.22  A crontab component for Hyperf.
hyperf/database                      v1.1.22  A flexible database library.
hyperf/db-connection                 v1.1.22  A hyperf db connection handler for hyperf/database.
hyperf/devtool                       v1.1.22  A Devtool for Hyperf.
hyperf/di                            v1.1.24  A DI for Hyperf.
hyperf/dispatcher                    v1.1.22  A HTTP Server for Hyperf.
hyperf/event                         v1.1.22  an event manager that implements PSR-14.
hyperf/exception-handler             v1.1.6   Exception handler for hyperf
hyperf/filesystem                    v1.1.23  flysystem integration for hyperf
hyperf/framework                     v1.1.25  A coroutine framework that focuses on hyperspeed and flexible, specifically use...
hyperf/guzzle                        v1.1.23  Swoole coroutine handler for guzzle
hyperf/http-message                  v1.1.24  microservice framework base on swoole
hyperf/http-server                   v1.1.24  A HTTP Server for Hyperf.
hyperf/logger                        v1.1.22  A logger component for hyperf.
hyperf/memory                        v1.1.22  An independent component that use to operate and manage memory.
hyperf/model-listener                v1.1.22  A model listener for Hyperf.
hyperf/paginator                     v1.1.22  A paginator component for hyperf.
hyperf/pool                          v1.1.23  An independent universal connection pool component.
hyperf/process                       v1.1.22  A process component for hyperf.
hyperf/redis                         v1.1.24  A redis component for hyperf.
hyperf/server                        v1.1.24  A base server library for Hyperf.
hyperf/session                       v1.1.22  A session library
hyperf/task                          v1.1.22  A task library for Hyperf.
hyperf/testing                       v1.1.23  Testing for hyperf
hyperf/translation                   v1.1.17  An independent translation component, forked by illuminate/translation.
hyperf/utils                         v1.1.24  A tools package that could help developer solved the problem quickly.
hyperf/validation                    v1.1.22  hyperf validation
hyperf/view                          v1.1.22  A view library for Hyperf.
sy-records/think-template            v2.0.9   use think template engine for hyperf.
swoole
Swoole => enabled
Author => Swoole Team<EMAIL_ADDRESS>Version => 4.4.12
Built => Dec  5 2019 21:54:13
coroutine => enabled
epoll => enabled
eventfd => enabled
signalfd => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 1.1.1d  10 Sep 2019
http2 => enabled
zlib => 1.2.11
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
mysqlnd => enabled
async_redis => enabled
Directive => Local Value => Master Value
swoole.display_errors => On => On
swoole.enable_coroutine => On => On
swoole.enable_library => On => On
swoole.enable_preemptive_scheduler => Off => Off
swoole.unixsock_buffer_size => 8388608 => 8388608
swoole.use_shortname => Off => Off

Describe your question
composer update 报错.
/var/www/company-home # php -d memory_limit=-1 /usr/bin/composer update -o
Loading composer repositories with package information
Updating dependencies (including require-dev)
Nothing to install or update
Writing lock file
Generating optimized autoload files
Warning: Ambiguous class resolution, "Symfony\Contracts\EventDispatcher\Event" was found in both "/var/www/company-home/vendor/symfony/event-dispatcher-contracts/Event.php" and "/var/www/company-home/vendor/symfony/contracts/EventDispatcher/Event.php", the first will be used.
Warning: Ambiguous class resolution, "Symfony\Contracts\EventDispatcher\EventDispatcherInterface" was found in both "/var/www/company-home/vendor/symfony/event-dispatcher-contracts/EventDispatcherInterface.php" and "/var/www/company-home/vendor/symfony/contracts/EventDispatcher/EventDispatcherInterface.php", the first will be used.
Warning: Ambiguous class resolution, "Symfony\Contracts\Service\ResetInterface" was found in both "/var/www/company-home/vendor/symfony/service-contracts/ResetInterface.php" and "/var/www/company-home/vendor/symfony/contracts/Service/ResetInterface.php", the first will be used.
Warning: Ambiguous class resolution, "Symfony\Contracts\Service\ServiceLocatorTrait" was found in both "/var/www/company-home/vendor/symfony/service-contracts/ServiceLocatorTrait.php" and "/var/www/company-home/vendor/symfony/contracts/Service/ServiceLocatorTrait.php", the first will be used.
Warning: Ambiguous class resolution, "Symfony\Contracts\Service\ServiceProviderInterface" was found in both "/var/www/company-home/vendor/symfony/service-contracts/ServiceProviderInterface.php" and "/var/www/company-home/vendor/symfony/contracts/Service/ServiceProviderInterface.php", the first will be used.
Warning: Ambiguous class resolution, "Symfony\Contracts\Service\ServiceSubscriberInterface" was found in both "/var/www/company-home/vendor/symfony/service-contracts/ServiceSubscriberInterface.php" and "/var/www/company-home/vendor/symfony/contracts/Service/ServiceSubscriberInterface.php", the first will be used.
Warning: Ambiguous class resolution, "Symfony\Contracts\Service\ServiceSubscriberTrait" was found in both "/var/www/company-home/vendor/symfony/service-contracts/ServiceSubscriberTrait.php" and "/var/www/company-home/vendor/symfony/contracts/Service/ServiceSubscriberTrait.php", the first will be used.
Warning: Ambiguous class resolution, "Symfony\Contracts\Translation\LocaleAwareInterface" was found in both "/var/www/company-home/vendor/symfony/translation-contracts/LocaleAwareInterface.php" and "/var/www/company-home/vendor/symfony/contracts/Translation/LocaleAwareInterface.php", the first will be used.
Warning: Ambiguous class resolution, "Symfony\Contracts\Translation\TranslatorInterface" was found in both "/var/www/company-home/vendor/symfony/translation-contracts/TranslatorInterface.php" and "/var/www/company-home/vendor/symfony/contracts/Translation/TranslatorInterface.php", the first will be used.
Warning: Ambiguous class resolution, "Symfony\Contracts\Translation\TranslatorTrait" was found in both "/var/www/company-home/vendor/symfony/translation-contracts/TranslatorTrait.php" and "/var/www/company-home/vendor/symfony/contracts/Translation/TranslatorTrait.php", the first will be used.
ocramius/package-versions:  Generating version class...
ocramius/package-versions: ...done generating version class

同时启动也报错了
/var/www/company-home # php bin/hyperf.php start
Scanning app ...
Scan app completed, took<PHONE_NUMBER>4768 milliseconds.
Scanning vendor ...
Scan vendor completed, took<PHONE_NUMBER>9836 milliseconds.
PHP Fatal error:  Declaration of Hyperf\Framework\SymfonyEventDispatcher::dispatch($event) must be compatible with Symfony\Contracts\EventDispatcher\EventDispatcherInterface::dispatch(object $event, ?string $eventName = NULL): object in /var/www/company-home/vendor/hyperf/framework/src/SymfonyEventDispatcher.php on line 24
Fatal error: Declaration of Hyperf\Framework\SymfonyEventDispatcher::dispatch($event) must be compatible with Symfony\Contracts\EventDispatcher\EventDispatcherInterface::dispatch(object $event, ?string $eventName = NULL): object in /var/www/company-home/vendor/hyperf/framework/src/SymfonyEventDispatcher.php on line 24

删除 composer.lock 文件后重新 composer update 即可

删除 composer.lock 文件后重新 composer update 即可

这个操作我试过几次了. 并没有用
/var/www/company-home # rm -rf ./composer.lock
/var/www/company-home # pc^C
/var/www/company-home # php -d memory_limit=-1 /usr/bin/composer update
Loading composer repositories with package information
Updating dependencies (including require-dev)
Nothing to install or update
Writing lock file
Generating autoload files
ocramius/package-versions:  Generating version class...
ocramius/package-versions: ...done generating version class
/var/www/company-home # php bin/hyperf.php start
Detected an available cache, skip the app scan process.
Detected an available cache, skip the vendor scan process.
PHP Fatal error:  Declaration of Hyperf\Framework\SymfonyEventDispatcher::dispatch($event) must be compatible with Symfony\Contracts\EventDispatcher\EventDispatcherInterface::dispatch(object $event, ?string $eventName = NULL): object in /var/www/company-home/vendor/hyperf/framework/src/SymfonyEventDispatcher.php on line 24
Fatal error: Declaration of Hyperf\Framework\SymfonyEventDispatcher::dispatch($event) must be compatible with Symfony\Contracts\EventDispatcher\EventDispatcherInterface::dispatch(object $event, ?string $eventName = NULL): object in /var/www/company-home/vendor/hyperf/framework/src/SymfonyEventDispatcher.php on line 24

rm -rf vendor
rm composer.lock
composer require "symfony/event-dispatcher:^4.3"
composer update -o

应该是这个,不然还是会出错
composer require "symfony/event-dispatcher:~5.0"

我也有这个错误，试了几次，还是没有解决，我是重1.1版本升级到2.1版本后出现的。
环境
php =>7.4
swoole => 4.5.5

我也有这个错误，试了几次，还是没有解决，我是重1.1版本升级到2.1版本后出现的。
环境
php =>7.4
swoole => 4.5.5


core/image: fix usage of "unknown" platform

"unknown" platform should not be returned from Platforms() as a valid platform type for an image.
Fixes: #10196

/cherry-pick release/1.7

/cherrypick release/1.7

Is this one also needed for 1.6 ?

Yes. It is needed. I was not able to reproduce the issue yesterday when working on 1.6 branch. Hence didnt raise the cherry-pick. Just now reproduced on HEAD of release/1.6

/cherrypick release/1.6


Uncaught SyntaxError: Cannot use import statement outside a module

Some rails apps don't get the js playground/application.js

Fixed in f604fb87837992e88154e76adf6b463361cbefcd


tests do not pass on current head for rustc 1.70.0

tests do not pass when following instructions in README.md
version of Rust compiler
~/src/lightarti-rest$ rustc --version
rustc 1.70.0 (90c541806 2023-05-31)

output:
$ make dircache
[...]
$ cargo test
[...]
    Finished test [unoptimized + debuginfo] target(s) in 40.07s
     Running unittests src/lib.rs (target/debug/deps/lightarti_rest-67ccfbb00f1b27cf)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/client.rs (target/debug/deps/client-04261347f66a5a5e)

running 4 tests
2023-06-19T15:30:46.676640Z DEBUG arti_client::client: It appears we have the lock on our state files.
2023-06-19T15:30:46.676640Z DEBUG arti_client::client: It appears we have the lock on our state files.
2023-06-19T15:30:46.676641Z DEBUG arti_client::client: It appears we have the lock on our state files.
2023-06-19T15:30:46.676640Z DEBUG arti_client::client: It appears we have the lock on our state files.
2023-06-19T15:30:46.676702Z DEBUG lightarti_rest::flatfiledirmgr: consensus.txt loaded
2023-06-19T15:30:46.676706Z DEBUG lightarti_rest::flatfiledirmgr: consensus.txt loaded
2023-06-19T15:30:46.676706Z DEBUG lightarti_rest::flatfiledirmgr: consensus.txt loaded
2023-06-19T15:30:46.676712Z DEBUG lightarti_rest::flatfiledirmgr: churn.txt loaded
2023-06-19T15:30:46.676715Z DEBUG lightarti_rest::flatfiledirmgr: consensus.txt loaded
2023-06-19T15:30:46.676717Z DEBUG lightarti_rest::flatfiledirmgr: churn.txt loaded
2023-06-19T15:30:46.676719Z DEBUG lightarti_rest::flatfiledirmgr: churn.txt loaded
2023-06-19T15:30:46.676723Z DEBUG lightarti_rest::flatfiledirmgr: churn.txt loaded
2023-06-19T15:30:46.683194Z DEBUG tor_circmgr: Lock not held; no state to flush.
2023-06-19T15:30:46.683194Z DEBUG tor_circmgr: Lock not held; no state to flush.
2023-06-19T15:30:46.683200Z DEBUG tor_circmgr: Lock not held; no state to flush.
2023-06-19T15:30:46.683249Z DEBUG tor_circmgr: Lock not held; no state to flush.
test test_get_short ... FAILED
test test_get_many_headers ... FAILED
test test_post ... FAILED
test test_get_long_response ... FAILED

failures:

---- test_get_short stdout ----
thread 'test_get_short' panicked at 'create client: create tor client

Caused by:
    0: tor: Tor network protocol violation (bug, incompatibility, or attack): Unable to bootstrap a working directory
    1: Unable to bootstrap a working directory
    2: Directory object expired or not yet valid
    3: Object is not currently valid', tests/client.rs:66:14

---- test_get_many_headers stdout ----
thread 'test_get_many_headers' panicked at 'create client: create tor client

Caused by:
    0: tor: Tor network protocol violation (bug, incompatibility, or attack): Unable to bootstrap a working directory
    1: Unable to bootstrap a working directory
    2: Directory object expired or not yet valid
    3: Object is not currently valid', tests/client.rs:66:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test_post stdout ----
thread 'test_post' panicked at 'create client: create tor client

Caused by:
    0: tor: Tor network protocol violation (bug, incompatibility, or attack): Unable to bootstrap a working directory
    1: Unable to bootstrap a working directory
    2: Directory object expired or not yet valid
    3: Object is not currently valid', tests/client.rs:66:14

---- test_get_long_response stdout ----
thread 'test_get_long_response' panicked at 'create client: create tor client

Caused by:
    0: tor: Tor network protocol violation (bug, incompatibility, or attack): Unable to bootstrap a working directory
    1: Unable to bootstrap a working directory
    2: Directory object expired or not yet valid
    3: Object is not currently valid', tests/client.rs:66:14

failures:
    test_get_long_response
    test_get_many_headers
    test_get_short
    test_post

test result: FAILED. 0 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

error: test failed, to rerun pass `--test client`

there were no issues in the compiling step

I created a minimal Dockerfile and reproduced on a newly cloned repo:
FROM rust:1.70-bookworm

COPY . .
RUN apt-get update && apt-get install make
RUN make dircache
ENTRYPOINT ["cargo", "test"]

The error is a bit strange:
    0: tor: Tor network protocol violation (bug, incompatibility, or attack): Unable to bootstrap a working directory

So either the tor network updated the nodes, and the arti-version we're using cannot connect to it anymore, or there is something not-tor-compatible going on in your network setup.
Can you try to re-run the github workflows? IIRC, they also run the tests.

@ineiti I'm not really sure that I understand how all of this is working, but given that the error refers to the directory, is it possible that the outdated directory cache from https://github.com/c4dt/lightarti-directory/releases/tag/2022-12-11 is the problem?
the README says it'd get updated every day, but the latest update is from 12/22

it seems that GitHub disables scheduled workflows after 2 months of inactivity:

This scheduled workflow is disabled because there hasn't been activity in this repository for at least 60 days. Enable this workflow to resume scheduled runs.

https://github.com/c4dt/lightarti-directory/actions/workflows/update.yaml
shall I re-enable the workflow?


documentation error fix

Hello.
I have just started to use nextra, it`s really inspiring, thanks!
Reading the documentation meet a simple error that break my brains for a while.
It's written to use wrong import:
import 'nextra-theme-blog/style.css'
instead of:
import 'nextra-theme-docs/style.css'
That`s follow with error import when starting project from the scratch instead of copy the repository.
Hope this save some time to somebody in the future.
Thanks.

It seems that I made a mistake on my side, i used old fork of code ...
Just followed the instruction again and got succeed.


Improve property for descendantFiles

Based on initial insight from @wemrysi
Supersedes #1037

Current coverage is 82.11%

Merging #1041 into master will not affect coverage as of 50ad01f

@@            master   #1041   diff @@
======================================
  Files          133     133
  Stmts         8923    8923
  Branches       286     286
  Methods          0       0
======================================
  Hit           7327    7327
  Partial          0       0
  Missed        1596    1596

Review entire Coverage Diff as of 50ad01f

Powered by Codecov. Updated on successful CI builds.

:+1:


Allow sinceRef to work with non-root cwds

When sinceRef is used with a cwd that is not at the repository root, newHashes isn't calculated properly, resulting in valid changesets being filtered out in the read changesets response.
Details
Let's say the changeset root was at project/subfolder/.changeset, and there was a changeset some-changeset.md present in this directory.
In filterChangesetsSinceRef, newChangesets would be — ['project/subfolder/.changeset/some-changeset.md'].
https://github.com/atlassian-forks/changesets/blob/27a5a82188914570d192162f9d045dfd082a3c15/packages/read/src/index.ts#L13-L19
But since newHashes incorrectly always takes the second half of a string split by /, it would result in newHashes being subfolder instead of some-changeset.md.
Instead, its safer to peek at the last part of the path fragment (which would always refer to the file).

LGTM, just missing a changeset


When deploying, it does an error

What happened?
I'm adding a CertManager controller to my Kubernetes configuration, but when trying to deploy it I receive this error:
  kubernetes:helm.sh/v3:Release (cert-manager-helm):
    warning: Helm release "cert-manager-helm-eb578c1e" was created but has a failed status. Use the `helm` command to investigate the error, correct it, then retry. Reason: failed post-install: timed out waiting for the condition
    error: 1 error occurred:
    	* Helm release "cert-manager/cert-manager-helm-eb578c1e" was created, but failed to initialize completely. Use Helm CLI to investigate.: failed to become available within allocated timeout. Error: Helm Release cert-manager/cert-manager-helm-eb578c1e: failed post-install: timed out waiting for the condition

Expected Behavior
I expect the controller to be properly installed
Steps to reproduce
This is the configuration I'm using
cert_manager_ns_name = "cert-manager"
cert_manager_ns = Namespace("cert-manager", metadata={"name": cert_manager_ns_name})
cert_manager_controller = pulumi_kubernetes_cert_manager.CertManager(
    "cert-manager",
    install_crds=True,
    helm_options=pulumi_kubernetes_cert_manager.ReleaseArgs(
        namespace=cert_manager_ns_name,
    ),
    opts=pulumi.ResourceOptions(provider=k8s_provider, depends_on=[cert_manager_ns]),
)

Output of pulumi about
CLI
Version      3.54.0
Go Version   go1.20
Go Compiler  gc

Plugins
NAME                      VERSION
docker                    3.6.1
gcp                       6.46.0
kubernetes                3.23.1
kubernetes-cert-manager   0.0.5
kubernetes-ingress-nginx  0.0.10
python                    unknown

Host
OS       darwin
Version  13.0.1
Arch     x86_64

This project is written in python: executable='/Users/primet/work/gouach/code/gouach-backend/infra-arch-pulumi/venv/bin/python3' version='3.10.9
'

Current Stack: maelp/infra-arch-pulumi/staging

TYPE                                              URN
pulumi:pulumi:Stack                               urn:pulumi:staging::infra-arch-pulumi::pulumi:pulumi:Stack::infra-arch-pulumi-staging
docker:image:Image                                urn:pulumi:staging::infra-arch-pulumi::docker:image:Image::web-app-image
pulumi:providers:gcp                              urn:pulumi:staging::infra-arch-pulumi::pulumi:providers:gcp::gcp
pulumi:providers:kubernetes                       urn:pulumi:staging::infra-arch-pulumi::pulumi:providers:kubernetes::cluster
pulumi:providers:kubernetes-ingress-nginx         urn:pulumi:staging::infra-arch-pulumi::pulumi:providers:kubernetes-ingress-nginx::default_0_0_10
pulumi:providers:gcp                              urn:pulumi:staging::infra-arch-pulumi::pulumi:providers:gcp::default_6_46_0
kubernetes-ingress-nginx:index:IngressController  urn:pulumi:staging::infra-arch-pulumi::kubernetes-ingress-nginx:index:IngressController::ingress-ctrl
gcp:artifactregistry/repository:Repository        urn:pulumi:staging::infra-arch-pulumi::gcp:artifactregistry/repository:Repository::gcp-docker-repository
pulumi:providers:kubernetes                       urn:pulumi:staging::infra-arch-pulumi::pulumi:providers:kubernetes::default
kubernetes:helm.sh/v3:Release                     urn:pulumi:staging::infra-arch-pulumi::kubernetes-ingress-nginx:index:IngressController$kubernetes:helm.sh/v3:Release::ingress-ctrl-helm
kubernetes:apps/v1:Deployment                     urn:pulumi:staging::infra-arch-pulumi::kubernetes:apps/v1:Deployment::web-app-1
kubernetes:apps/v1:Deployment                     urn:pulumi:staging::infra-arch-pulumi::kubernetes:apps/v1:Deployment::web-app-2
kubernetes:core/v1:Service                        urn:pulumi:staging::infra-arch-pulumi::kubernetes:core/v1:Service::web-app-1-svc
kubernetes:core/v1:Service                        urn:pulumi:staging::infra-arch-pulumi::kubernetes:core/v1:Service::web-app-2-svc
kubernetes:networking.k8s.io/v1:Ingress           urn:pulumi:staging::infra-arch-pulumi::kubernetes:networking.k8s.io/v1:Ingress::web-app-ingress

Found no pending operations associated with staging

Backend
Name           pulumi.com
URL            https://app.pulumi.com/maelp
User           maelp
Organizations  maelp

Dependencies:
NAME                             VERSION
pip                              23.0.0
pulumi-docker                    3.6.1
pulumi-gcp                       6.46.0
pulumi-kubernetes-cert-manager   0.0.5
pulumi-kubernetes-ingress-nginx  0.0.10
setuptools                       66.0.0
wheel                            0.38.4

Pulumi locates its logs in /var/folders/6d/tgjjs7m13yngmk287mk9cv9h0000gp/T/ by default

Additional context
No response
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

On the other hand, installing directly from Helm seems to work
cert_manager_chart = Chart(
    "cert-manager",
    config=ChartOpts(
        fetch_opts={"repo": "https://charts.jetstack.io"},
        chart="cert-manager",
        version="v1.11.0",
        namespace=cert_manager_ns_name,
        values={"installCRDs": True},
    ),
    opts=pulumi.ResourceOptions(provider=k8s_provider, depends_on=[cert_manager_ns]),
)

But then, when I add a TLS to my Ingress and deploy with the helm chart, I get this:
  kubernetes:batch/v1:Job (cert-manager/cert-manager-startupapicheck):
    warning: This resource contains Helm hooks that are not currently supported by Pulumi. The resource will be created, but any hooks will not be executed. Hooks support is tracked at https://github.com/pulumi/pulumi-kubernetes/issues/555 -- This warning can be disabled by setting the PULUMI_K8S_SUPPRESS_HELM_HOOK_WARNINGS environment variable
    error: 2 errors occurred:
    	* the Kubernetes API server reported that "cert-manager/cert-manager-startupapicheck" failed to fully initialize or become live: Resource 'cert-manager-startupapicheck' was created but failed to initialize
    	* [BackoffLimitExceeded] Job has reached the specified backoff limit

This might be because I'm deploying to GKE AutoPilot?

It seems reasonable that the issue is related to AutoPilot, and I updated the title to that effect. Here's a ticket in cert-manager side with some general tips on deploying to AutoPilot:
https://github.com/cert-manager/cert-manager/issues/3717
I would suggest that we re-test on a GKE AutoPilot cluster using the above instructions, and then incorporate the learnings into our documentation.


--quiet doesn't suppress output

When I pass --quiet, the tool still spits out lots of info - presumably coming from the browser process?
I expect a tool to have zero output when a "quiet" option is on.
Also, I couldn't find a quiet option in the Python API.
Is the command line / Python API not equivalent?

At the moment, there is no way to disable the browser's output from both the CLI and Python package, it is feasible though and will be implemented in a later update. See #55.
Even if the goal is to keep the "interface" very similar between the CLI and the Python API, they are not completely equivalent in some ways. Indeed, CLIs often have quiet/verbose modes to control the amount of information that is automatically displayed after running a command, where you would have to print() them yourself while using the Python package.
For instance, with the CLI, the paths of the generated files are printed by default if the quiet flag is not set, while these paths are returned by the Html2Image.screenshot() method of the Python package.

When will this be added? Looking forward to this :)
Would also donate

I need to run this often and the ouput bothers me


Do not close Statement when checking for more Result Sets (#792)

Possible fix for #792 (PreparedStatement::getMoreResults() closes Prepared Statement itself)

@hadrabap my attempts to implement multiple result set handling have proven more complicated than expected, mostly because of the intertwining of classes.
Thus i would like to merge your change.
I did a few updates and added more tests, if you could have a look before i merge i would appreciate.


feat: osmosis delegation support

Description
Osmosis bounty -- delegation support. Delegating / undelegating / claiming works.
It shows 0% apr because we need to update unchained but this should be everything web needs to support osmosis staking.
This is behind a feature flag and will not show the "staking" section of the osmosis page unless osmosis is enabled.
Notice

[x] Have you followed the guidelines in our Contributing guide?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Pull Request Type

[ ] :bug: Bug fix (Non-breaking Change: Fixes an issue)
[x] :hammer_and_wrench: Chore (Non-breaking Change: Doc updates, pkg upgrades, typos, etc..)
[ ] :nail_care: New Feature (Breaking/Non-breaking Change)

Issue (if applicable)
Osmosis bounty -- delegation
Risk
Low risk behind feature flag.
Testing
Make sure cosmos staking still works as expected when osmosis is disabled.
There is no need for ops to test osmosis delegating in this PR as it is feature flagged and will be tested when we are ready to release it
Screenshots (if applicable)

Moving to draft until previous requested changes are tackled

Also noticed one regression vs. Cosmos staking: APR is currently zero, but it's actually coming as 0 from unchained so for the scope of this web PR, this isn't a bug. Related PR: shapeshift/unchained#434

yes this was noted in the PR description

this is from the defi/earn page

total nitpick - not a regression, but a bit of margin/padding would be nice

some nit pickish changes + defi/earn page is busted

Added a 8px one in 48fead11ea703a37d706a4f748ff29124efee881, to be consistent with the "Asset allocation" row:

this is from the defi/earn page - not seeing console errors, but that capitalized Osmosis in the url raises an eyebrow

Nice catch! Fixed in 4388b2431924cdac75974a3427edd6998fd8e42c

some nit pickish changes + defi/earn page is busted

All fixed

merging as previously approved by gomes, stylistic changes only


Point Timber package to a matching version

Hi,
i've noticed the composer.json was slightly outdated and due to that it was pointing to version of the timber package which doesn't exist anymore, i've pushed the following commit to address that.
Thank you,
Cezar

Good catch