Refactor fate

Refactor Fate to allow removing of generics and be able to improve serialization
Move Fate classes from core to Manager
Create new FateZooStore in core and make ZooStore extend it, consolidating
methods between the 2 classes. This allows sharing between Manager and Shell
Extract inner classes to allow reuse and consolidation across packages
Drop Generics from many different Fate interfaces and classes
Combine Fate admin utilities into one FateAdmin
Drop master FateAdmin
Drop AdminUtil in favor of FateCommandHelper and use in FateCommand
Rename method from checkGlobalLock() to isManagerLockValid()
Move TStatus to core and rename to FateTransactionStatus
Closes #2473

@keith-turner I ran into a dependency snag. The FateCommand uses a bunch of Fate types and is in the Shell package. Moving the Fate stuff to manager, breaks this. I don't know of a good solution other than rewrite FateCommand and make new types that use API. Any ideas?

@keith-turner I ran into a dependency snag. The FateCommand uses a bunch of Fate types and is in the Shell package. Moving the Fate stuff to manager, breaks this. I don't know of a good solution other than rewrite FateCommand and make new types that use API. Any ideas?

@milleruntime I missed this comment.  I have not looked at this in a while, I can take a fresh look at it.  Is this PR in a state where you want a review or are you still changing some things?

@keith-turner I ran into a dependency snag. The FateCommand uses a bunch of Fate types and is in the Shell package. Moving the Fate stuff to manager, breaks this. I don't know of a good solution other than rewrite FateCommand and make new types that use API. Any ideas?

@milleruntime I missed this comment. I have not looked at this in a while, I can take a fresh look at it. Is this PR in a state where you want a review or are you still changing some things?

Yeah it is in a good state, if you want to take a look at it. This PR isn't a priority though as it will probably have to go into 3.0. I pulled some of these changes out into https://github.com/apache/accumulo/pull/2541. Also, @Manno15 is working on changes here: https://github.com/apache/accumulo/pull/2215, which should be able to get into 2.1 but have some new API additions you might want to look at as well.

@keith-turner I ran into a dependency snag. The FateCommand uses a bunch of Fate types and is in the Shell package. Moving the Fate stuff to manager, breaks this. I don't know of a good solution other than rewrite FateCommand and make new types that use API. Any ideas?

@milleruntime I missed this comment. I have not looked at this in a while, I can take a fresh look at it. Is this PR in a state where you want a review or are you still changing some things?

Yeah it is in a good state, if you want to take a look at it. This PR isn't a priority though as it will probably have to go into 3.0. I pulled some of these changes out into #2541. Also, @Manno15 is working on changes here: #2215, which should be able to get into 2.1 but have some new API additions you might want to look at as well.

If you want to wait for me to merge the latest changes in main, I am working on that now.

I plan to circle back to this next week and review it again.

I am going to rework these changes for 3.0, incorporating the deprecating of FateCommand, but I may have to just create a new branch.

Closing this PR. It is too broad and contains too many different changes.


[ feature ] Show in-progress preview in the Image Preview node

If an Image Preview node is added to the Controller, it would show the KSampler in-progress preview
This could be useful for people that want to build a whole GUI, like SwarmUI or InvokeAI.
The "big picture area" can also show the in-progress preview.

There is a workaround for this, but it may be too complex for new users:
In Comfy UI, you can create a group node with: KSampler + Vae Decode + Image Preview.
This way, the final image will show up in the same node as the in progress preview.
You can use the SamplerCustomAdvanced node, so that there are no other widgets.
The you can add the group node the the Controller.

I think that the preview is only available on sampler nodes, you can associate a preview node to a ksampler latent preview (I guess).
Maybe a future workaroud would be through the "hide widgets" feature created by Pythongosssss.

It's easy enough to catch the preview image - we already do it, in fact, in order to show the preview even if the KSampler node is minimised (minimised nodes don't get images for obvious reasons). So showing the preview wouldn't be technically hard.
Wondering what the UI would be like, though - a workflow might have multiple Image Preview nodes, (and/or multiple KSamplers) and it wouldn't make sense to show all KSampler in progress previews on all Image Previews.
It could be done if they are in the same group, perhaps, or if they are directly connected to each other in the workflow? Or would we need a way for the user to specify?
I'm leaning to the same group (and maybe only if there is exactly one preview and exactly one sampler)

Wondering what the UI would be like, though - a workflow might have multiple Image Preview nodes, (and/or multiple KSamplers) and it wouldn't make sense to show all KSampler in progress previews on all Image Previews.

You would only show in in-progress preview, in the Image Preview nodes that are in the Controller.
You would place the in-progress preview in the same exact area were the final image will show up.
And once the generation is done, the in-progress preview would be replaced by the final image.
The main use case is a layout like this, with a single image preview.
I want to see the preview, while the image is being generated, like in a traditional GUI.

or if they are directly connected to each other in the workflow?

I would choose this option, to associate the preview node to the KSampler that is generating the image.

or if they are directly connected to each other in the workflow?

I would choose this option, to associate the preview node to the KSampler that is generating the image.

Yes, exactly.

I'm using the rgthree progress bar it right now!
I don't think we need to replicate this functionality.

And ( unlike I suggested before ) we actually could use a progress bar  for the controller.
It could be shown below the in progress preview !

or if they are directly connected to each other in the workflow?

I would choose this option, to associate the preview node to the KSampler that is generating the image.

Actually, it's not directly connected, there is a VAE decode in between.

Actually, it's not directly connected, there is a VAE decode in between.

@chrisgoringe Damn it! 😆

It could be done if they are in the same group

But this way you will have both KSampler and Preview Image displaying the latent preview.

I think there are a few heuristics that could be used to link a preview node (or save image, for that matter) to a sampler.
One would be to follow the dataflow upstream - what node did the image come from? What node sent that node either an image or a latent? etc until you hit a sampler.

Um... I think I might have made this happen...

It works, but it's glitchy...
https://github.com/user-attachments/assets/823fbbd6-cc3d-44ad-8540-67d2716b1ff1

It works, but it's glitchy...

It seems that between steps, it's shown the final image, that was previously generated.

It seems the glitch happens in certain zoom/canvas position (for example, it doesn't glitch if you are far from KSampler)

There's also a different bug:
An image preview is showing up in the Load VAE node

It seems the glitch happens in certain zoom/canvas position (for example, it doesn't glitch if you are far from KSampler)
Yeah, with more "zoom in" and the Sampler out of the canvas, this does not happen.

@chrisgoringe Now the preview it's perfecly working in simple workflows with one sampler+preview, but with 2 pass + face detailer there are some issues:

the 1st and 2nd pass previews are displayed in both 2pass preview and face detailer preview
when in face detailer, the glitch is back
after the generation, both previews in Controller show the latent previews and not the actual images

https://github.com/user-attachments/assets/d138bf74-a68a-46ac-9500-35d171d475b0

Yeah, this is where the “follow upstream” logic becomes less reliable!
One option would be to only show the preview if generated up to N links
upstream. N=2 would cover the basic case (sampler-vae decode-preview).
But if you do sampler-refiner-decode-preview, I think you would want the
preview to show the first and second samplers? That would be N=3…
I don’t do very complicated workflows so I don’t really have an instinct
for how it ought to work!
On Sun, 3 Nov 2024 at 6:56 PM, LukeG89 @.***> wrote:

@chrisgoringe https://github.com/chrisgoringe Now the preview it's
perfecly working in simple workflows with one sampler+preview, but with 2
pass + face detailer there are some issues:

the 1st and 2nd pass previews are displayed in both 2pass preview
and face detailer preview
when in face detailer, the glitch is back
after the generation, both previews in Controller show the latent
previews and not the actual images

https://github.com/user-attachments/assets/d138bf74-a68a-46ac-9500-35d171d475b0
—
Reply to this email directly, view it on GitHub
https://github.com/chrisgoringe/cg-controller/issues/263#issuecomment-2453333163,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABBLMA23TF5TE2UM7QURCCTZ6XJJDAVCNFSM6AAAAABRBQYHZ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJTGMZTGMJWGM
.
You are receiving this because you were mentioned.Message ID:
@.***>

Could post a screenshot of the workflow corresponding to your video?

I don’t do very complicated workflows so I don’t really have an instinct for how it ought to work!

Yeah, I just made one from scratch because I know ComfyUI users like complicated things! 😆

Could post a screenshot of the workflow corresponding to your video?

Here's the workflow in case you need
1.3 - Group Tabs testing.json

@chrisgoringe, Don't change this behavior, I love it. It's perfect !
The current "in progress" preview is also shown in the Image Preview node of the next steps !!!
Makes perfect sense. Because the next stages will be affected by changes in the previous ones.

And I can see all "in-progress previews", with a single image window open ( the one in the last stage ).
We can have only one big window for multiple stages, like a traditional GUI does.

If I make a change in the first Sampler. It shows the "in progress" preview in all windows.

If I make a change in the 2 stage ( refiner ), it only shows the  "in progress" preview in the refiner and the next step ( face detailer )

And if I make a change in the last stage ( face detailer ) it shows the "in progress" preview only in the face detailer.

But there are a couple of things that can be improved:

When the generation is done, show the final image ( PNG ).
Now, if it do try to save the image via right click, I get a jfif file ( preview ), not the PNG

If a stage is bypassed, whe don't need to the the "in-progress" preview

@JorgeR81 Are you experiencing these bugs?

when in face detailer, the glitch is back
after the generation, both previews in Controller show the latent previews and not the actual images

after controlnet preprocessor
(no problem with that, except that it was previewing the Load Image connected to the preprocessor, but only before the generation)

when in face detailer, the glitch is back
after the generation, both previews in Controller show the latent previews and not the actual images

Yes, I just updated my previous post with these issues.
I will check controlnet preprocessor.

Except for the issues mentioned, it works as expected with all enabled.
And it looks fantastic !
But there are CPU spikes and ComfyUI slows down, when we try to move the canvas while generating.

Just pushed an update which fixes some of these:

I think the final image glitch should be gone
bypassed nodes don't get updated
image moved above widgets

I'll look at Load Image next.
A question about the final image shown in the sampler node. Comfy leaves the sampler node showing the approximate image, not the finished one; that's currently what we're doing too. Personally, I'm ok with that - I think you can just watch the Image Preview node to see progress and final outcome.
The problem with "show the final image" is "which final image?". If there are multiple Image Preview nodes, which show different stages of the process, which image should the sampler show?
I think it makes more sense for the sampler just to show the interim images, with the final node showing both.

OK, I think I've also fixed the issues with the Load Image node. LMK if it's still got problems.

Not the glitch is gone, but I still have this problem:

after the generation, both previews in Controller show the latent previews and not the actual images

At first I see the final image, but when I switch group, I see the latent preview.

Anyways, something happened to all the previews, now they are over the widgets (even in Load Image node)

after the generation, both previews in Controller show the latent previews and not the actual images

At first I see the final image, but when I switch group, I see the latent preview.

Same happens to me.
First I can get the PNG.
But if I switch groups in the Controller I get the jfif file ( preview ).

@chrisgoringe, can you also hide the in-progress preview in the Sampler, if the Controller also as an Image Preview
Having the preview in the Sampler it's redundant now. It just takes space.

I can create a new group ( in blue ) just with an "Image loader", after the final stage.
And this "big picture area" shows all in-progress images and final previews, for the whole workflow, even with bypassed stages.

And I can collapse  the "big picture area" to see the full workflow.

@chrisgoringe, about the progress bar, the rgthree bar works well, on top of the workflow.
What we could use in the Controller is that mini progress bar that shows up in the Sampler header.

@chrisgoringe, can you also hide the in-progress preview in the Sampler, if a Controller also has an Image Preview, displaying that same in-progress preview ?

Maybe we don't even need to hide the Sampler previews.
With this configuration, we don't need an Image Preview in the other Controllers.
We can get the PNG file from the "Big Picture area".

With this more complex configuration, sometimes ComfyUI gets very slow.
Then sometimes, the whole ComfyUI page gets white, and I get an Out of Memory error code, right in the browser's page.
If I Reload / Refresh the browser, it starts working again, and it's fast again.
I use Google Chrome, in Windows 10, and I have 32 GB RAM.

@chrisgoringe, can you also hide the in-progress preview in the Sampler, if a Controller also has an Image Preview, displaying that same in-progress preview ?

A better solution would be to show the final PGN image, in the KSampler, once the generation is done.
This way we don't need to have an Image Preview node in the Controller !
Is that possible ?

This is probably the more complex use case.
I added 2 more Controllers:

with a Ultimate SD Upscale node ( tiled upscaler )
and finally and a regular upscaler ( with the x1_ITF_SkinDiffDetail_Lite_v1 model to give some "texture" to the image ).

@chrisgoringe, can you also hide the in-progress preview in the Sampler, if a Controller also has an Image Preview, displaying that same in-progress preview ?

A better solution would be to show the final PGN image, in the Sampler, once the generation is done.
This way we don't need to have an Image Preview node in the Controller !

I think I mentioned this above. The problem is, which final image to show?
If you have two stages and an image preview after each, what should the first sampler show when it finishes?
You get the situation where the behaviour of a node depends on what is connected to it downstream (where are image previews?) which is contrary to the way node based programming works and would, I think, inevitably lead to counterintuitive outcomes.
Comfy leaves the final interim image in place.

I think there are two issues mentioned above that I'll look at:
The image preview node returning to the last interim image
Images "forgetting" their size
(and maybe performance?)

I think I mentioned this above. The problem is, which final image to show?

What about the first alternative?
Don't show a Sampler preview, if we already have it, in an Image Preview is on that same Controller.
And if that's not possible, maybe an user option to disable all Sampler previews.

(and maybe performance?)

Yeah, this is weird, because sometimes it's quite slow, but if I refresh the browser it's fast again.

Don't show previews in sampler if the image preview is in the same controller might be possible.

Anyways, something happened to all the previews, now they are over the widgets (even in Load Image)

That's a change I made, so that the image is in the same place on all nodeblocks - I think it looks better?

This is getting way long, so I've added a few new issues to track bits and pieces...
#273 - don't show in sampler if showing in preview
#274 - preview node returns to interim image if you swap groups
#275 - images 'forget' their size
Closing this one as done since the basic functionality is there.

Anyways, something happened to all the previews, now they are over the widgets (even in Load Image)

That's a change I made, so that the image is in the same place on all nodeblocks - I think it looks better?

@LukeG89, I ask for that:  https://github.com/chrisgoringe/cg-controller/issues/263#issuecomment-2453387891
Now finally the Load Image is aligned with Image Preview !
I usually prefer to copy the "native" ComfyUI style, but not in this case.
I wanted this ( with nodes ) since I started using ComfyUI !

I'm having some issues with the live previews:

In this 3 stage workflow, the stage 1 preview is not shown in the big picture area.
It was working 2 day ago, when I took the screenshots above.

Also, the live preview in stage 3 sampler ( Face Detailer ), sometimes gets wider during the generation.
This was also not hapening before.

UPDATE:

Also, the live preview in stage 3 sampler ( Face Detailer ), sometimes gets wider during the generation.

I don't seem to have this issue anymore

In this 3 stage workflow, the stage 1 preview is not shown in the big picture area.

In my workflow the big image preview is plugged to stage 3.
I tried unplugging and plugging the wire and this seems to fix it !
Sometimes, it does not work. But if I plug it to stage 1 and then to stage 3, it also fixes the issue.
It's not about changing the group configuration. It's just plugging and unplugging the wires that fixes the issue.
But even if I save the "fixed" workflow, when I open it again it's also broken again.

Could you raise this as a new issue, with the workflow included? I'd prefer to keep feature issues and bugs separate where possible, when the bug isn't so serious as to render the feature unusable.

Could you raise this as a new issue, with the workflow included?

I've made a simpler 3 stage workflow, without custom nodes, but I was not able to replicate the bug.
I will keep testing this and will open an issue, when I have a better understanding of the cause of the issue, in the order workflow.

great. I'll leave this closed; we'll treat it as a bug if you reproduce.
On Wed, 6 Nov 2024 at 13:45, JorgeR81 @.***> wrote:

Could you raise this as a new issue, with the workflow included?
I've made a simpler 3 stage workflow, without custom nodes, but I was not
able to replicate the bug.
I will keep testing this and will open an issue, when I have a better
understanding of the cause of the issue, in the order workflow.
—
Reply to this email directly, view it on GitHub
https://github.com/chrisgoringe/cg-controller/issues/263#issuecomment-2458619407,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABBLMAZT5HF7LMK5LQVL7T3Z7F7ERAVCNFSM6AAAAABRBQYHZ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJYGYYTSNBQG4
.
You are receiving this because you modified the open/close state.Message
ID: @.***>

OK, I understand the issue.
This is caused by a switch I have in stage 2, to upscale the latent. ( I've only added it today ).
I don't think it's necessary to open an issue, since this is caused by this custom node ( a switch from the WAS node suite ).
A better solution would be to support the Trung0246 switch in the Controller ( https://github.com/chrisgoringe/cg-controller/issues/283 )
Because it has more features and it seems to work better with the Controller.
It replaced the WAS switch, with Trung0246 switch, in the workflow, the issue was also fixed.
The problem with the Trung0246 switch, is that it can't be added to the Controller.

OK. Adding a reference to this in #296 as well.


Performance drops dramatically in controller mode when transientStorePoolEnable=true

Problem
Broker在自动故障转移模式下，设置transientStorePoolEnable=true导致benchmark性能测试急剧下降

transientStorePoolEnable=true测试性能

transientStorePoolEnable=false测试性能

Environment

RocketMQ 5.0 1Master-1Slave Dledger Controller Mode
broker配置

  brokerClusterName = DefaultCluster
  brokerName = broker-a
  brokerId = -1
  deleteWhen = 04
  fileReservedTime = 12
  enableControllerMode = true
  namesrvAddr = <IP_ADDRESS>:9876;<IP_ADDRESS>:9876;<IP_ADDRESS>:9876
  controllerAddr = <IP_ADDRESS>:59878;<IP_ADDRESS>:59878;<IP_ADDRESS>:59878
  allAckInSyncStateSet = true

  flushDiskType = ASYNC_FLUSH

  transientStorePoolEnable=true
  transientStorePoolSize=3

  listenPort = 59011

  storePathRootDir=/home/data/rocketmq/broker-a/store
  storePathIndex=/home/data/rocketmq/broker-a/store/index
  storeCheckpoint=/home/data/rocketmq/broker-a/store/checkpoint
  abortFile=/home/data/rocketmq/broker-a/store/abort
  storePathCommitLog=/home/data/rocketmq/broker-a/store/commitlog
  storePathConsumerQueue=/home/data/rocketmq/broker-a/store/consumequeue
  storePathEpochFile=/home/data/rocketmq/broker-a/store/storePathEpochFile

Hi @riki-wang

Little performance impact

Hi @riki-wang   Little performance impact
首次启动会有这个问题

Hi @riki-wang   Little performance impact
首次启动会有这个问题，而且是要在controller模式下，1master-1slave

我试下首次启动，我上面的模式就是Controller的A-HA的自主切换模式。

@riki-wang

When first start on controller model, broker has performance impact.
I will try to find it and fix.
thanks for you catch.

Hi @riki-wang, the issue of enabling transientStorePool in controller mode is still fixed in #5722 . After the repair is completed, we can continue to test.

本来是准备给生产环境引入自动故障转移的，就突然测出这个性能bug了

Hi @riki-wang, the issue of enabling transientStorePool in controller mode is still fixed in #5722 . After the repair is completed, we can continue to test.

本来是准备给生产环境引入自动故障转移的，就突然测出这个性能bug了

@riki-wang PR #5722 has fixed it. You can try this PR code.

close as fixed

Rocketmq v5.1.0 still has this issue


SSL

Web-Part already uses a Let's Encrypt-Certificate, but is there a way to use them for imaps and smtps to?
I only found the following:
https://github.com/zokradonh/kopano-docker/issues/136
https://github.com/zokradonh/kopano-docker/issues/16
but didn't get how it works now..

Hi @theoneandonly-vector,
the two tickets you have linked were closed because of inactivity, not because what they described was implemented.
Being able to connect Pop3/IMAP clients is not important to me personally so I have not invested time into it. If someone else wants to adapt the compose file for it I'll gladly review and merge it.

there seem to be an issue with the mail-docker:
https://github.com/tomav/docker-mailserver/issues/1369

running this inside the mail-docker-container and restarting it works for me (re-using the cert which is used by kopano-web):
mkdir -p /etc/postfix/ssl
cp "$SSL_CERT_PATH" /etc/postfix/ssl/cert
cp "$SSL_KEY_PATH" /etc/postfix/ssl/key
chmod 600 /etc/postfix/ssl/cert
chmod 600 /etc/postfix/ssl/key
now we only need to create a crontask to automatically refresh it.

so I use this now:
crontab -e
0 0 1 * * /opt/kopano-docker/thisrunsonceamonth.sh
thisrunsonceamonth.sh:
docker exec -it kopano_mail bash -l -c 'mkdir -p /etc/postfix/ssl && cp "$SSL_CERT_PATH" /etc/postfix/ssl/cert && cp "$SSL_KEY_PATH" /etc/postfix/ssl/key && chmod 600 /etc/postfix/ssl/cert && chmod 600 /etc/postfix/ssl/key'


introspecting Docopt usage strings

It seems like there is an opportunity to expose some useful information about Docopt usage strings: http://stackoverflow.com/questions/28210260/parsing-docopt-argument-types-any-language/28214541?noredirect=1#comment44799324_28214541
The parser in this implementation currently drops it on the floor.

Doesn't this type of introspection raise problems? E.g. if you have somthing like:
Usage: myprog --coordinates ( (<cartesian_x> <cartesian_y>) | (<polar_angle> <polar_radius>) )

There is no way for docopt to determine wheter in the call myprog --coordinates 30 72 the coordinates are of type "cartesian" or "polar" faux-type.

Those are positional arguments. The SO post is asking for the name of a flag argument. e.g., for --flag=FILE extract FILE for flag flag.

But --flag=FILE is equivalent to --flag <file>.

Yes... That should work too. Your example contains a flag with no arguments and two pairs of alternate positional arguments. So that would not fall under the question of "given a flag with an argument named NAME in the usage, return the NAME."

I see. Getting the "type" of positional arguments, however, would be the logical extention of the SO request. It might also be nice for my beloved autocompletion. If it is known something has a FILE or HOSTNAME or USER "type" argument, one can uses the appropriate pre-defined completion functions that come with bash.

In this case, the "type" of the positional argument is just its name, which is available in standard Docopt. The key is that the name of a flag's argument is dropped on the floor (because it has no semantic value other than  its existence).

You're right, I didn't think. Sorry. For sake of staying on topic consider deleting this and your previous post.

Question is whether it, in con-f-use's example, should be possible to determine that the type of the positional arguments as [ ("<cartesian_x>","<cartesian_y>"), ("<polar_angle>", "<polar_radius>") ], i.e. a list of everything it could possibly be resolved to. I'm not sure whether that in itself adds much value, but being able to resolve positional arguments definitely does.

Nah, this is good discussion! I had to go back and re-read the SO post myself, so having clarification in more detail is good. Thank you! :-)


F40: failed to write bootloader configuation when installing from Anaconda ISO

The OpenQA tests have discovered that the Anaconda ISO install path is failing with the error failed to write bootloader configuation
https://openqa.fedoraproject.org/tests/2857297#step/_do_install_and_reboot/115
It's believed that the most recent greenboot update (https://bodhi.fedoraproject.org/updates/FEDORA-2024-1d32971a85) is the likely culprit.
It is pulling in bootupd as part of the Requires (see https://github.com/fedora-iot/greenboot/pull/129) and the Anaconda installer is trying to use bootupd to install the bootloader (https://github.com/rhinstaller/anaconda/blob/443d5e6d7e3f97477dbdc870d73957fcc34d0fc5/pyanaconda/modules/payloads/payload/rpm_ostree/installation.py#L504-L509).
Fedora IoT 40 isn't equipped to support bootupd at this time, so we likely need to remove the Requires: bootupd from the spec file

One thing I'll point out here is that I think IoT should stop being its own base image and build on top instead of fedora-bootc - the bootupd integration is already tested and working there and it's not a really good idea to try to re-do that I would say.
I think we can refactor things such that the bootupd integration is only enabled in greenboot when installed on a derivation of fedora-bootc.
For example, the container image could do something like this:
FROM quay.io/fedora/fedora-bootc:41
RUN dnf -y install greenboot && mv /etc/grub2.d/greenboot.conf /usr/lib/bootupd/grub2-static/configs.d

or so. Or, the more painful way would be to try to split the package. Or, change it so it doesn't Requires: bootupd but still unconditionally installs the static config (leaving the /etc/grub2.d as a symlink) - I think that'd be nearly equally easy.

As for why things are failing, it's probably because the IoT builds aren't doing https://gitlab.com/fedora/bootc/base-images/-/blob/main/tier-0/bootupd.yaml?ref_type=heads#L31 but again I think the better fix here is to get to a world of container derivation, not where IoT is its own base image.

One thing I'll point out here is that I think IoT should stop being its own base image and build on top instead of fedora-bootc - the bootupd integration is already tested and working there and it's not a really good idea to try to re-do that I would say.

Let's explore that idea in a separate issue?  Maybe #53 or #57? Or create a new one if you'd like

As for why things are failing, it's probably because the IoT builds aren't doing https://gitlab.com/fedora/bootc/base-images/-/blob/main/tier-0/bootupd.yaml?ref_type=heads#L31 but again I think the better fix here is to get to a world of container derivation, not where IoT is its own base image.

We've done that for F41 and F40, so I'm not sure why we are hitting the failure. Since we are breaking F40 builds, dropping bootupd from the greenboot spec feels like the quickest and easiest option right now (https://github.com/fedora-iot/greenboot/pull/152).

We've done that for F41 and F40, so I'm not sure why we are hitting the failure. Since we are breaking F40 builds, dropping bootupd from the greenboot spec feels like the quickest and easiest option right now (fedora-iot/greenboot#152).

Oh right, we also reverted it for F40 - https://pagure.io/fedora-iot/ostree/c/dc36d6caf078e60eebbf22b1786e53535aee7a15?branch=f40
So that explains why F40 was failing

One thing I'll point out here is that I think IoT should stop being its own base image and build on top instead of fedora-bootc - the bootupd integration is already tested and working there and it's not a really good idea to try to re-do that I would say.

One issue is that fedora-bootc I believe is too fat for IoT.

I opened https://gitlab.com/fedora/bootc/base-images/-/merge_requests/48 related to this. @nullr0ute @miabbott interested in your thoughts from the IoT side.

@pcdubs please have a look at Jonathan's proposal above :arrow_up:

Short term, I would remove the bootupd dependency in greenboot Fedora 40.
Hopefully we keep bootupd enabled in F41 and it can stay there.

As noted above, this was fixed and no longer an issue.


Authentication Database support

Hi @ericmj.  Thanks for this.  I was wondering if you would be interested in a PR to support authentication via an Authentication Database?  If so, would master be the correct branch to submit the PR to?  I've noticed some tests are failing on master and that there is another branch that seems to be active.  Thanks :)

Aha! It looks like this is supported in the emj-dbconn branch.  Sorry for the noise.


CDK init options for RDS and SSL cert validation

Coverage: 93.96% (+0.006%) from 93.953% when pulling 7a068bdbff16e0937f058f64231340f9854e1686 on cody-more-cdk-init-options into 6a0046e6fcf4917c3329a2e22cea585d1240b061 on main.


Inverse normalizations

It could be great to be able to inverse each of the normalization transformations.
When running normalization, we can store some attributes in adata.uns and then we can re-use them for the inverse transformation.
I have already implemented this for a few normalization functions in scyan, so I can do a PR if you find it interesting.

I think that's a good idea. I think that keeping the parameters for normalization would also help to increase the interoperability of pytometry with FlowJo. Both tools have fairly different approaches when it comes to data transformation or normalization, but I think that keeping the parameters and implementing inverse normalizations is a step forward.

Great, I'll work on this!
I'll try to do a PR next week with inverse_asinh, inverse_logicle, and maybe inverse_biexp

Certainly agree on storing the parameters in adata.uns for posteriority.
About the inverse transfromations, could you clarify when you need them instead of just keeping the untransformed data in an adata.layer?

That's a great point.
By default, pytometry does not keep an untransformed version of the data right now, which is why it would make sense to me to implement inverse transformations. @grst has a point, though, and an alternative solution is to create a copy of the untransformed data in adata.layers for all normalizations and keep the normalization parameters for interoperability.
Given those two solutions, I would prefer @grst's suggestion.

I completely agree that keeping the untransformed data is better. Still, the inverse transformation can be used on batch corrected data, for which we don't have the untransformed values (for instance, after batch effect correction with Scyan). This way, we can visualize the corrected data in FlowJO.
But I agree that it's a very specific use case, and maybe a bad practice to do that anyway. So I'll only work on storing the parameters in adata.uns!

Ok, then let's do it like that for now. If it turns out there are more broadly applicable use-cases we can still add inverse transformations later.
Related: in that case working with layers should become more convenient, e.g. all preprocessing functions should support storing the result in a separate layer. There are some ideas how the API for this could look like in https://github.com/scverse/anndata/issues/706:
sc.pp.compensate(adata, in_layer="raw", out_layer="raw_compensated")


manifest: Update to latest upstream main branch

Upstream CI checks, as well as our own daily build show green, so update to the latest upstream revision.

@jerome-pouiller can you check the compliance check failure? Is it a valid one? I wonder why we haven't seen it before.

It's as if the validation isn't seeing our soc/silabs/ hierarchy in the zephyr-silabs tree.

@jerome-pouiller I suspect it might be because we're missing source soc/Kconfig from the root Kconfig file.

@jerome-pouiller I suspect it might be because we're missing source soc/Kconfig from the root Kconfig file.

I added a separate commit to fix this part, but now the issue seems to be that nothing is apparently including soc/silabs/silabs_siwx917/siwg917/Kconfig.soc.

I added a separate commit to fix this part, but now the issue seems to be that nothing is apparently including soc/silabs/silabs_siwx917/siwg917/Kconfig.soc.

Seems like that should be covered by this:
https://github.com/zephyrproject-rtos/zephyr/blob/858a68705294a27c41392bdae88a1f95a7e8fdea/soc/Kconfig.v2#L38-L39

I added a separate commit to fix this part, but now the issue seems to be that nothing is apparently including soc/silabs/silabs_siwx917/siwg917/Kconfig.soc.

Seems like that should be covered by this: https://github.com/zephyrproject-rtos/zephyr/blob/858a68705294a27c41392bdae88a1f95a7e8fdea/soc/Kconfig.v2#L38-L39

If it wasn't, I don't understand how we could compile for siwg917...

If it wasn't, I don't understand how we could compile for siwg917...

Agreed, quite strange. I think there's some additional magic in the build system where it may include things which were not explicitly included. I will investigate this independently of this PR. I now reverted back to just updating west.yml.


[SCHEDULE] Enhance cache_write to enable layout change.

cc @ZihengJiang @Laurawly

Examples in https://github.com/dmlc/tvm/pull/432/files#diff-92e5db064c4949635c76e1527e3452b1R176

It seems that test_schedule_cache_relayout4() is not added.


Practical Assessment Infrastructure

We should allow missions to be protected with a URL parameter. We will use the same Source Academy as always, without any need of starting a separate server. We will hide the URL parameter in a shortcut on the desktop of the lab PCs.
So for example we will have a shortcut "PE" on the desktop, which has the following link:
https://sourceacademy.nus.edu.sg?key=9d8f7g6h5j4k3l2
In this example, the key 9d8f7g6h5j4k3l2 unlocks the PA mission in the assessment section of the Source Academy. The students will need to log in, as usual, using LumiNUS, and go to the PA.
This way, we can:

avoid installing any other server
test the PA setup in advance by supplying the correct key, without any time pressure

In addition: log the access of such 'protected' assignment/mission:

student who accessed and time of access.

I think this is a really good idea.  It would be mean we can totally bypass setting up a sperate instance for the PE.
How will the implementation and flow look like?
A trivial implementation will look something like this.

We add an additional optional field to the mission XML specifying the password.
This password can be generated and added days or a week before the PE.
When a student requests a password protected mission he/she will be prompted for a password.
Validation of this password is done on the backend by comparing the password in the database against the one provided by the student.
During the PE we just need to provide students with the key to unlock the missions.

Some issues that I can think of with regards to this implementation

Correct me if I'm wrong but avengers have access to the private cs1101s repo. The valid key will be stored in the mission XML which will be accessible to all Avengers.
If we do give baggy bounds with regards to the time for the PE. How are we gonna prevent students from logging back into source academy PE mission after the PE since they already have the key?

Maybe we can refine the implementation together. If anyone has any suggestions or ideas do let me know!

**Some issues that I can think of with regards to this implementation
Correct me if I'm wrong but avengers have access to the private cs1101s repo. The valid key will be stored in the mission XML which will be accessible to all Avengers.**
Yes, to be discussed.

Some issues that I can think of with regards to this implementation: If we do give baggy bounds with regards to the time for the PE. How are we gonna prevent students from logging back into source academy PE mission after the PE since they already have the key?
That's where the logging comes in: We tell the students that they must not try to get access after the exam, and we log and go after anyone who does not follow this advice.

Some issues that I can think of with regards to this implementation: If we do give baggy bounds with regards to the time for the PE. How are we gonna prevent students from logging back into source academy PE mission after the PE since they already have the key?
That's where the logging comes in: We tell the students that they must not try to get access after the exam, and we log and go after anyone who does not follow this advice.

Another approach to this is we force students to finialize their submission at the end of the PE. This will mean they can no longer adjust their answers.

Okay lets split this down to smaller problems

Add password feature to assessments
Add support for an additional repo

we should consider a private submodule implementation over an additional repo as well!! love to hear feedback on this!

I feel a bit squeamish about storing the password in plain text in the assessment XML.
A more fancy solution:
(1) The XML specifies that the assessment is password-protected.
(2) There is a procedure in the backend how to set the password, maybe via ssh with a command-line tool.
(3) The command-line tool is protected so that only sysadmin can call it.
It would work something like this:
% assessmentpasswd
Enter assessment number

M3B
Enter password

Re-enter password

Password successfully changed
%

I agree. This is all good.

Note that we now have a separate private repo for Story XML files. See the wiki of the assessment repo for the specification:
https://github.com/source-academy/assessments/wiki


feat: e2e general tests

Description
Add menu tests to e2e testing, which correspond to 'general tests' of our testing plan
Fixes FL-1076

\e2e --tests frontend
I think this should run your new tests

/e2e --tests frontend
I think this should run your new tests

End to end tests: :hourglass_flowing_sand: STARTING

/e2e --tests sdk,frontend

End to end tests: :hourglass_flowing_sand: STARTING

The image building fails due to this error: Step #2: error building image: error building stage: failed to optimize instructions: failed to get files used from context: failed to get fileinfo for /workspace/substra-frontend/e2e-tests/cypress.json: lstat /workspace/substra-frontend/e2e-tests/cypress.json: no such file or directory
Because we forgot to update the dockerfile to match #219
I think we only need edit this line: https://github.com/Substra/substra-frontend/blob/main/docker/substra-frontend-tests/Dockerfile#L5

/e2e --tests sdk,frontend

End to end tests: :hourglass_flowing_sand: STARTING


Process finished with exit code 139 (interrupted by signal 11: SIGSEGV) when loading a hub module

In the following environment:
Python Version : 3.6
Tensorflow Version : 1.14.0
TF Hub Version : 0.5
OS: Ubuntu 19.04

if I run the following code: https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/retrieval_with_tf_hub_universal_encoder_qa.ipynb
when loading the hub module:
module_url = 'https://tfhub.dev/google/universal-sentence-encoder-multilingual-qa/1'
# Set up graph.
g = tf.Graph()
with g.as_default():
    module = hub.Module(module_url)  # load tfhub module

I get the following error:
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

This happens both using PyCharm and Jupyter Notebook.

Solved. The problem was tensorflow's version. You need tensorflow==1.13.1


etherwallet-3.5.0: read contract returns blank text-box.

I am using myetherwallet on my private Ethereum network. Under Contracts=>"Read/Write Contract", I was able to access methods of my contract(entering its address and abi) successfully and I was able to write into my contract successfully. I do check through geth, that string "hello" is written into my contract.

But I was not able to read my contract's method(ex: mycontract.getSize()) which returns nothing. Please note that when I can read my contract inside geth which returns a valid information.
Example code:
  function getSize() constant returns ( uint ) {
    return 100;
  }

As you can see my contract's constant method returned value show as a blank text box.

[Q] Is there anything I could do to fix this issue that my method's return value would show up?
Thank you for your valuable time and help. By the way, this upgrade is great.

Is there any way you can provide me with Address / ABI so I can look into this further. You mentioned its on a private net but is there anything on the main net or (currently broken Ropsten) thats creating this same issue?

Sure:
Address= "0x027d798dadb260836c18f955fe3cc13723285920"
abi=[{"constant":true,"inputs":[],"name":"get","outputs":[{"name":"num","type":"uint32"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"number","outputs":[{"name":"num","type":"int8"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"val","type":"int8"}],"name":"set","outputs":[],"payable":false,"type":"function"},{"inputs":[],"payable":false,"type":"constructor"}]

My Contract: (Here is a simple contract that I have tried.) get() does not return 100.
pragma solidity ^0.4.6;
contract C {
    int8 i;
    function C() {
      i = -1;
    }
    function number() constant returns (int8 num) {
      return i;
    }
    function set(int8 val) {
      i = val;
    }
    function get() constant returns (uint32 num) {
      return 100;
    }
}

Sorry I have not tried on the main net or Ropsten.

I have a feeling you might not have your mew connected to local node, or the testnet  you are using. use the top right node switcher and make sure you are connected to the correct node. I just deployed your contract at 0x7047c65D21b8e0942B69f5fb7719034e8addBc79 and used mew to read the get value and it returned 100. you can also use the above address to test

Thank you @kvhnuke. I have tried this on my private ethereum network (http://ethereum.stackexchange.com/a/11240/4575). I have followed your guide to use the wallet for private network as well. I think my mew is connected to my local node where I can get my contract balance and send transaction to other addresses.
I can see the correct balance:

I can send transaction of my contract's function, it does updated on the blockchain as well:

Interesting part is, I can't read the function, which returns blank text.

Please note that I am using v.3.5.0.

can you please open up the browser console and send us a screenshot, it might be failing due to a different error,
if you can send the network tab screenshot it'll be great too. Ill try to test it on a custom network as soon as I get a chance

etherwallet-master.js
        if (hasApply) {
          return function () {
            var args = [];
            forEach(arguments, function (arg) {
              args.push(formatError(arg));
            });
            return logFn.apply(console, args); //ERROR is coming from this line.
          };
        }

Call order of the error:
completeRequest(callback, status, response, xhr.getAllResponseHeaders(), statusText);=>
callback(status, response, headersString, statusText);=>
if (!$rootScope.$$phase) $rootScope.$apply();=>
$rootScope.$digest();=>
asyncTask.scope.$eval(asyncTask.expression, asyncTask.locals);=>
return $parse(expr)(this, locals);=>
exceptionHandler(errorMessage);=>
$log.error.apply($log, arguments);=>
return logFn.apply(console, args);
Please not that, I have added few lines to the etherwallet-master.js so the error lines probably not correct on the original code.

There is this error always show up.

Let me re-try with the latest version v3.5.6 and updated you based on that without changing the original code. @kvhnuke

Same error, lines are on the original code: v3.5.0. @kvhnuke

@kvhnuke Status update on this?

@avatar-lavventura is this still happening?

Is there any update related to issue, with the v3.5.0 yes still happening. @tayvano

@avatar-lavventura is this still an issue?

Yeah, I think so.


🌐 Add Japanese translation for docs/ja/docs/tutorial/body-fields.md

Relates to https://github.com/tiangolo/fastapi/issues/1572
This PR is a translation of docs/ja/docs/advanced/Fieid-Body.md.
I am the first PR to this repository. I would appreciate a code review. Please let me know if it has any problems.

📝 Docs preview for commit 7f9daca96a21f0ca0f984d1bc4f0308a379afc22 at: https://45b5b0d4.fastapitiangolo.pages.dev

📝 Docs preview for commit b84ff4bcd5d6e2a57e5c796869db7630fda78c38 at: https://6a0f0d99.fastapitiangolo.pages.dev

📝 Docs preview for commit 967a68456fa8aecb580e4291ef75b8f9cdb267f1 at: https://b8773f1d.fastapitiangolo.pages.dev

📝 Docs preview for commit 64067c232001338bff9f218b1345cd42b18c6efc at: https://164ef3a2.fastapitiangolo.pages.dev

@torumanabe
Thx for writing!  If possible, could you divide two files into each PR as Tiangolo said?
After that, I will review and approve each file :)

@tamtam-fitness
I sincerely appreciate your review for my first PR.
following your advice,  I divided this  PR into each of https://github.com/tiangolo/fastapi/pull/10067 and https://github.com/tiangolo/fastapi/pull/10068.
I'll be waiting for your review when you have time.

Awesome! Let's continue on those two PRs, I'm gonna close this one now. 🤓


Failure to deploy correctly.

Using a trial instance of Automation Pilot stood up in early June 2024, the resulting YAML from a "Build Run Configuration" does not resulting in a running POD.
First issue is the environment variables passed in the spec are not named correctly. In release v0.0.4 the update to internal/kubernetes/metadata/metadata.go changed OPERATOR_ID, ENVIRONMENT, and INSTANCE_ID to RWP_OPERATOR_ID, RWP_ENVIRONMENT, and RWP_INSTANCE_ID but the generated YAML uses the older names without the RWP_
Second issue is the order of stanzas in the YAML seem to be off. In the spec section, adding the volume pointing to the secretName failed when the definition of the secret was in its default position at the bottom of the file. Even though the secret was created, the POD complained that it didn't exist. Moving the definition up to the second stanza (after Namespace) seemed to correct this. Other stanzas like the Service account didn't have an issue. Maybe a timing issue since the Secret is kind of large and my Kyma env is also a trial and therefore limited.
Last comment, there is mention of examples in the README.md, and I don't see any. As someone new to Automation Pilot and Kubernetes, would be nice to see examples of what can be done.

Hello,
This remote Automation Pilot agent is still under development and not Generally Available.
The support for it in the Automation Pilot backend services is also under development.
The k8s manifest provided by the AutoPi UI is not updated yet.
Backwards incompatible changes are expected in 0.0.x releases, as it is a Beta feature.
Can you please provide some insight into your use case?
Is this remote worker a key component of your automation flow?

Currently I am just exploring this for future possibilities.
I understand the Beta nature, but it will be hard to make it past Beta if
every person that goes to test it out hits the roadblock of the basic
implementation not even working.
The breaking changes were made in March. It is almost July and AutoPI is
not updated. I would expect SAP to be better coordinated.
On Thu, Jun 27, 2024 at 2:12 AM Rangel Ivanov @.***>
wrote:

Hello,
This remote Automation Pilot agent is still under development and not
Generally Available.
The support for it in the Automation Pilot backend services is also under
development.
The k8s manifest provided by the AutoPi UI is not updated yet.
Backwards incompatible changes are expected in 0.0.x releases, as it is a
Beta feature.
Can you please provide some insight into your use case?
Is this remote worker a key component of your automation flow?
—
Reply to this email directly, view it on GitHub
https://github.com/SAP/remote-work-processor/issues/18#issuecomment-2193880323,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AF3TSEG362V34QEZ7TNBNELZJOUM5AVCNFSM6AAAAABJ6WNFIOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJTHA4DAMZSGM
.
You are receiving this because you authored the thread.Message ID:
@.***>

To run this remote agent in a Kubernetes environment with the current version of Automation Pilot, please use the v0.0.3 release, instead of latest.
We appreciate the concerns you have about this Beta feature and are working on improving the user experience.


Default to exact matches when specifying job queue name

But also allow the use of * to denote wildcard/like searching.
Partial #908.

Probable alternative to #1652


Implicit begin blocks

https://github.com/bbatsov/ruby-style-guide#begin-implicit

Thanks for the contribution!


Replace Mock User IDs with Real User IDs and Implement New User Profile Management

Description
Frontend:

Replaced all mock user data with real user data.
Added a default 5-minute stale time for the global query client to reduce unnecessary fetching.
Removed one of the duplicated sign-out buttons.
Added validation for name input.
Implemented snackbarNotifications in auth-related components.
Created two new React Query hooks for fetching and creating user profiles.
Enhanced auth components to use the new RH hooks

Backend:

Implemented a new user route with functionalities to fetch user profile and create user profiles.
Updated FirebaseAuthMiddleware by adding a method to find users by UID and improved how user data is passed down to controllers.
Updated the UserModel to better integrate with Firebase auth and the future user profile management feature.
Updated logging in various controllers and services.

Fixes #289
Type of change

[X] ✨  New feature (non-breaking change which adds functionality)

How Can this be tested? Testing Plan to review this PR

[X] Where in the app does this change take place?
Auth
[X] What are the steps to test this change?

At the root directory, run npm run dev.

Sign Up
2. Click Sign In, then Sign Up, and test Google sign-up and email-password sign-up (GitHub sign-up is not fully implemented yet).
Sign Out
3. Click Sign Out.
Sign In
4. Click Sign In, and use the information used during sign-up to test Google sign-in and email-password sign-in.

[X] What is the expected output?

You should be able to see your username displayed on the side navigation upon sign-in/sign-up.

Checklist:

[X] I have performed a self-review of my own code.
[X] I have commented on my code, particularly in hard-to-understand areas.
[ ] I have made corresponding changes to the documentation.
[X] My changes generate no new warnings.
[ ] Any dependent changes have been merged and published in downstream modules.
[X] I have reviewed the Security Best Practices Document.
[X] I have followed the Naming Conventions Guide for my changes.
[X] I have ensured that my pull request title is descriptive.

Additional Information
Related documentation updates will be handled in #276

@LucieSeidler and I identified that when a user signs up, the path correctly switches to /generate, but sometimes the UI continues to display the auth page. Since this issue doesn't impact the functionality of this feature I created a bug ticket for it #313

@LucieSeidler and I identified that when a user signs up, the path correctly switches to /generate, but sometimes the UI continues to display the auth page. Since this issue doesn't impact the functionality of this feature I created a bug ticket for it #313

@kinskiwu Are you able to confirm that this bug is not introduced by this PR? I have not experienced the described behavior or heard of anyone else experiencing it on another branch.

@relandboyle This error appears when we use an account previously signed up with linkta before this feature was implemented. In this new feature, a user profile is created after signing up (which we didn't have previously thus it is showing a user profile not found error). To test this feature, please first sign up with your account.

@kinskiwu I deleted my accounts from Firebase and I'm seeing the same or similar behavior.
Can we schedule a Zoom discussion / review for this? Continued via Slack...

Converting this PR to draft to fix some issues

Just fyi in case any of you are following this PR:

@relandboyle @ejwoods @chrisr0892 and I had a live pair programming session today and we were able to identify and fix a couple of issues causing the errors.

Changes I made to fix the residual SignUp Page issue: I added a userEffect to redirect a user to `/generate’  when a user is authenticated

Changes I made to ensure a user signing in through oauth gets a new userprofile generated:  updated handleGoogleAuthCick & handleGithubAuthClick to create a new userProfile if it is nonexistent upon sign-in


Tests do not show which method failed

When a test fails, the only way to tell which method failed the test is to dig into the stack traces.
Need to rewrite the test suite a bit so that it it is clear which test failed.

Closed by #9

Closed by #9


feat(DEVOPS-1225): parametrize Docker image name with git branch

Please check if the PR fulfills these requirements

[X] The commit message follows Talend standard
[ ] Tests for the changes have been added (for bug fixes / features)

What is the current behavior? (You can also link to an open issue here)
Current Docker image names provided by Maven (with maven-docker-plugin) are using an hardcoded name for the Talend private registry & they do not include the Git branch name.
What is the new behavior?
Now Docker image names are using the 'talend_docker_registry' as the registry name in prefix & the git branch name as a suffix.
This allows integration of these docker images in Talend CI.
Does this PR introduce a breaking change?

[ ] Yes
[X] No

If this PR contains a breaking change, please describe the impact and migration path for existing applications: ...
Other information:

Pull request test result: success :star:

Pull request test result: success :star:


[DOCS] Broken/missing picture

https://www.yiiframework.com/extension/yiisoft/yii2-app-advanced/doc/guide/2.0/en/structure-environments
At the end of the page there is a broken picture:

Looking at the repository, the image seems to be there at the expected path, so I don't know what is wrong.

Moved.


feat: ✨ Add delete account functionality and error checking

please change the PR title

related to #46


feat(api): Support enabled TLS

What this PR does / why we need it:
Optional TLS enabled.

Which issue(s) this PR fixes:

Fixes #
Which docs this PR relation:
Ref https://github.com/nervosnetwork/muta-docs/pull/52
Which toolchain this PR adaption:
No Breaking Change
Special notes for your reviewer:
/cc @LycrusHamster @zeroqn

/cc @LycrusHamster @zeroqn

/retest

/lgtm

/hold

/hold cancel

/lgtm

/lgtm


piecewise scale should be redesigned (and isn't well-enough documented)

i'm coming all the way from vega-lite, so i might be missing d3-specific information, happy to be enlightened.  i don't know enough to know whether only vega-lite and vega should change, or if d3-scale should as well.
i'm using a piecewise symlog scale in vega-lite to represent a field as a y-value in a line graph and a color on a choropleth.  i want to use the same scale for both.  the purpose of the piecewise is to reduce the dynamic range allocated to some unimportant negative values.
there's no doc on how to use piecewise in vega/vega-lite, other than specifying that domain and range should have matching numbers of elements (if one makes it all the way to https://github.com/d3/d3-scale#continuous_domain, we also learn that domain should be monotonic).  based on how scales work, i expected the units to be arbitrary, serving only to express relative proportions.  i expected the encoding to automatically adapt to the range of outputs automatically, as usual.
but it turns out that the output range is interpreted directly as concrete units in the encoding, so the user has to know specifics, such as the pixel locations of the top+bottom of a graph, or the particulars of how colors are indexed.  moreover, y-values are negative-going compared to the data domain, whereas colors are not!
instead, piecewise transforms should precede the nonlinear transform and similarly use arbitrary units.
data domain
-> piecewise range (expresses relative proportions in abstract units)
-> nonlinear (symlog, etc -- input to this layer automatically matched to range of outputs of previous layer, user only cares about shape, not units)
-> concrete presentation (pixels, colors, user doesn't know or care about units cuz again the input to this layer is automatically matched to output of previous)
original report here: https://github.com/vega/vega-lite/issues/7013
i also filed this on vega: https://github.com/vega/vega/issues/2954

instead, piecewise transforms should precede the nonlinear transform and similarly use arbitrary units.

Sounds like the sort of thing that needs to fixed at the Vega layer, so closing as duplicate of vega/vega#2954. I don’t understand concretely how you’d like d3.scaleSymlog or the like to behave differently, but I’m happy to reconsider if you want to clarify your desire.


.NET Core or .NET Standard Package

Any plans to update the package for either .NET core or .NET standard?

depend on https://github.com/OData/WebApi/tree/feature/netcore
after Microsoft.AspNetCore.ODat release a stable version, I will update the package for .Net core

It seems that the dependency has been released so this can procede. I will try to see if this is easy to do when I add the filter #9

There may be have some problem to  support .Net core 3.1, you can refer this: https://github.com/OData/WebApi/issues/1748
I'm busy with another project, next version maybe base on .Net core 5 which Release scheduled for November 2020(https://github.com/dotnet/core/blob/master/roadmap.md)


Pre-step of restructuring / modifying the props before composing

Hi, I'm wondering whether it is possible to restructure / modify the props as a pre-step before composing a component when using graphql from react-apollo?
I have the scenario below where I query for a list of contacts. The jobTitlevariable of a contact contains a key I use to look up i18n translations using a meteor package TAPi18n. The setData method illustrates what I want to do, in a manner.
import ContactList from '../components/contact_list';
import { TAPi18n } from 'meteor/tap:i18n';
import { composeAll } from 'mantra-core';
import { graphql } from 'react-apollo';
import gql from 'graphql-tag';

const GET_CONTACTS = {
  query: gql`
    query contacts {
      contacts {
        fullName,
        jobTitle,
        email,
        phone
      }
    }`,
  options: () => ({
    pollInterval: 60000,
    setData: (data) => {
      return {
        contacts: data.contacts && data.contacts.map((contact) => {
          contact.jobTitle = TAPi18n.__(contact.jobTitle);
          return contact;
        })
      };
    }
  })
};

export default composeAll(
  graphql(GET_CONTACTS.query, { options: GET_CONTACTS.options })
)(ContactList);

@bompi88 Are you wanting to change the shape of the props that get sent to the child component? If so take a look at http://docs.apollostack.com/apollo-client/react.html#graphql-props
Otherwise, I'm not sure I follow

@jbaxleyiii thanks, i must have overlooked it while reading the docs. It seems that I have to put the logic I wanted inside the component anyways, as I did not manage to reactively update the props while the language changes. If you have any suggestions I'd be very grateful. Thanks again :)

Ah, I see. If the user changes their language, you want to translate the query result, not requery the server? Yeah I think that is best inside the component


Simplify Makefile by using pattern matching for tests

Apply the same technique used in #27 to tests.

Addressed in #29


Structural diagram queries

May I ask why the author has made a residual connection between the feature maps of different scales in the downsampling process and the upsampling process? (red arrows in the figure) Or is my understanding wrong? Hope to get your answer.

Did you figure it out? I am also confused about this.


Suggest creating "Description:" heading in Event details modal.

Please provide a summary of the feature

The event details modal inserts the word "Description: " as additional text before the actual description from the event creator. Semantically it would be better to include this in a heading tag, preferably above the text box area.
This would also improve the user experience when recreating events. Currently, when copying and pasting a previous event description in order to recreate a new event description the "Description:" can be copied and inadvertently duplicated as well.
Example:

"Description: Description: An event where people gather to do things."

Are there any potential issues you foresee with this feature
Aesthetics in regards to #17 👀
Is this feature related to any current or past issues? Please provide the
Additional context
Add any other context or screenshots about the feature request here.
Which Branch should this fix be Pulled into?
PR into [branchName]
If you're interested in working on this feature, please comment so you can be assigned

link to some relevant code:
https://github.com/Caleb-Cohen/Together/blob/343d7378306ad752cbada653b4de1f6d6dab829b/client/src/features/modal/EventModal.jsx#L64-L66

Just a heads up this has been solved in a future redesign in #285. Redesign will probably take some time so if someone wanted to take this on it will be available until #285 is pulled into development.

Hi. This seems like an easy fix. I would like to be assigned this if possible. I already have a solved branch ready that will close this issue.

This issue should be closed. It was resolved by PR #444


Sqlite persistence experiment - Do not merge

Why

We wanted to test out the quick benefits of using the simple sqlite db instead of file based storage.

Notes:

We get atomic writes which is something we had to simulate with file based storage.
We don't keep the db connection open but open and close each time we need to execute a query since this is considered best practise.

[x] CHANGELOG updated or not needed
[x] Documentation updated or not needed
[x] Haddocks updated or not needed
[x] No new TODOs introduced or explained herafter

I think we need to find a way configure sqlite since the haskell library we used does not provide this. Perhaps we should also explore using some different library since I believe this PR should bring us benefits if we are able to configure sqlite for performance.


full derivation of invariant error dynamics

Thanks for your sharing. The example code works well with my own biped robot walk data.
But I am puzzled with the derivation of some equations in the paper titled

Contact-Aided Invariant Extended Kalman Filtering for Legged Robot State Estimation.

Could you please provide the full derivation of equation (20), the individual terms of the invariant error dynamics?
Thank you!

RIEKF_ErrorDynamics.pdf

Let me know if that answers your question. It is simply applying the chain rule and making a first-order approximation.

@RossHartley Thanks very much for your file. It exactly answers my question :)


Fix always restarting mesos cloud with multiple frameworks

Resolves Issue #270

This fix has been running on our Jenkins test boxes for ~7 days with no issues.  The updateScheduler is now firing correctly for updates to multiple Mesos clouds on Jenkins.  This was tested on Jenkins 1.642.2 with Mesos version 1.0.1 and Jenkins 2.32.3 with Mesos version 1.2.0

Thanks @lightcapda for the PR!


Add doc section on map printing

This adds info on maps being printed on script
exit and re-adds info for how to trigger map printing using SIGUSR1.
Checklist

[ ] Language changes are updated in man/adoc/bpftrace.adoc
[ ] User-visible and non-trivial changes updated in CHANGELOG.md
[ ] The new behaviour is covered by tests

@ajor Noticed that the info on SIGUSR1 was removed from the docs accidentally. I probably did this during the migration so I'm adding it back


add non-breaking space to stop symbols list

non-breaking space symbol is not allowed in url

Hey, thanks for the PR!
Do you have a reference for why the U+A0 NO-BREAK SPACE is not allowed in URLs? I looked at RFC 3987 and it explicitly allows the character (search for %xA0). Having said that, I think it makes sense to exclude whitespace characters in this library. Maybe we should list some others as well.

link
There can not be any of the types of spaces in the url. This is not allowed. In RFC 3987
It is indicated that the use of this symbol is allowed in ucschar. I do not know what is ucschar = /
Also, this link can be useful:
wikipedia

Thanks, merged. I decided I'll add some other Unicode whitespace characters as well, see the list here: https://en.wikipedia.org/wiki/Whitespace_character#Unicode

Released in 0.7.0: https://github.com/robinst/autolink-java/blob/master/CHANGELOG.md#070---2017-08-31

@robinst Thank you!


build: move production build to Docker

Resolves #519

/preview CI

/preview CI

:tada: This PR is included in version 1.6.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add a less broken alternative to ID3D12Device_GetAdapterLuid()

Could we have a HRESULT ID3D12Device_GetAdapterLuid(ID3D12Device*, LUID*) instead? Figuring out the proper way to call this from C is a pain. Some examples of the problems caused by this function:
https://github.com/amerkoleci/Vortice.Windows/issues/201
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26338
https://stackoverflow.com/questions/65197140/msvc-crash-when-invoking-a-method-directly-via-the-vtable-that-returns-by-value

The latest version of the header was updated to support non-MSVC compilers. Does this help your issue?
#if defined(_MSC_VER) || !defined(_WIN32)
        virtual LUID STDMETHODCALLTYPE GetAdapterLuid( void) = 0;
#else
        virtual LUID *STDMETHODCALLTYPE GetAdapterLuid(
            LUID * RetVal) = 0;
#endif

I don't think so, since I was compiling with MSVC when I faced this issue. It's a problem with the C API.
I ended up adding a C++ module with wrappers like this
HRESULT ID3D12Resource_GetDescNonBroken(ID3D12Resource *t,
                                              D3D12_RESOURCE_DESC *d) {
	*d = t->GetDesc();
	return S_OK;
}


Email notifier doesn't work with Gmail app password (both SSL on 465 and TLS on 587)

Describe the bug
I want to set up notifications, but the email notifier doesn't work with my Gmail account. I have 2-Step Verification set up in Gmail, which is why I set up an app password. However, both combinations of secure=true with port 465 and secure=false with port 587 always end with the same error.
Screenshots or Logs
><EMAIL_ADDRESS>entrypoint-config
> node dist/src/entrypoint-config.jsSetting
timezone: Europe/Berlin
Run on startup: true
[2023-02-02 11:52:05.000 +0100] INFO: Started epicgames-freegames-node
	COMMIT_SHA: "5fc6de6ce0306d51e1867b10e76e9dbd1343faa0"
	BRANCH: "master"
	DISTRO: "alpine"
[2023-02-02 11:52:05.523 +0100] INFO: Testing all configured notifiers
[2023-02-02 11:52:07.214 +0100] ERROR: Error sending email. Please check your configuration
	user<EMAIL_ADDRESS>
	reason: "TEST"
	emailConfig: {
		"type": "email",
		"secure": false,
		"smtpHost": "smtp.gmail.com",
		"smtpPort": 587,
		"emailSenderAddress"<EMAIL_ADDRESS>		"emailSenderName": "Epic Games Captchas",
		"emailRecipientAddress"<EMAIL_ADDRESS>	}
[2023-02-02 11:52:07.214 +0100] ERROR: Mail command failed: 530-5.7.0 Authentication Required. Learn more at
530 5.7.0  https://support.google.com/mail/?p=WantAuthError f8-20020a05600c44c800b003dc433bb5e1sm4333056wmo.9 - gsmtp
	err: {
		"type": "Error",
		"message": "Mail command failed: 530-5.7.0 Authentication Required. Learn more at\n530 5.7.0  https://support.google.com/mail/?p=WantAuthError f8-20020a05600c44c800b003dc433bb5e1sm4333056wmo.9 - gsmtp",
		"stack":
			Error: Mail command failed: 530-5.7.0 Authentication Required. Learn more at
			530 5.7.0  https://support.google.com/mail/?p=WantAuthError f8-20020a05600c44c800b003dc433bb5e1sm4333056wmo.9 - gsmtp
				at SMTPConnection._formatError (/usr/app/node_modules/nodemailer/lib/smtp-connection/index.js:784:19)
				at SMTPConnection._actionMAIL (/usr/app/node_modules/nodemailer/lib/smtp-connection/index.js:1566:34)
				at SMTPConnection. (/usr/app/node_modules/nodemailer/lib/smtp-connection/index.js:1041:18)
				at SMTPConnection._processResponse (/usr/app/node_modules/nodemailer/lib/smtp-connection/index.js:947:20)
				at SMTPConnection._onData (/usr/app/node_modules/nodemailer/lib/smtp-connection/index.js:749:14)
				at TLSSocket.SMTPConnection._onSocketData (/usr/app/node_modules/nodemailer/lib/smtp-connection/index.js:189:44)
				at TLSSocket.emit (node:events:513:28)
				at addChunk (node:internal/streams/readable:324:12)
				at readableAddChunk (node:internal/streams/readable:297:9)
				at TLSSocket.Readable.push (node:internal/streams/readable:234:10)
		"code": "EENVELOPE",
		"response": "530-5.7.0 Authentication Required. Learn more at\n530 5.7.0  https://support.google.com/mail/?p=WantAuthError f8-20020a05600c44c800b003dc433bb5e1sm4333056wmo.9 - gsmtp",
		"responseCode": 530,
		"command": "MAIL FROM"
	}
[2023-02-02 11:52:07.218 +0100] WARN: Current version
	COMMIT_SHA: "5fc6de6ce0306d51e1867b10e76e9dbd1343faa0"
	BRANCH: "master"
	DISTRO: "alpine"

Additional context
I looked up the error code at Google Support, see here.

530, "5.7.0", Must issue a STARTTLS command first.

Don't think I can do anything about this on my site.
Would be happy to help debug the issue and hope you find the time to look into it.

587 is for STARTTLS, try with port 456. and be sure your smtp service is enable in google mail setting

BTW, you should set secure to true if you re using the google mail to notify

587 is for STARTTLS, try with port 456. and be sure your smtp service is enable in google mail setting

Did you mean port 465? As stated in my original post I already tested port 465 with secure set to true, but without success.
Since I stopped using google mail, I simply moved on to Gotify, which works like a charm =)


Consider AG Grid as recommended default over Dash data table

Originally posted by @antonymilne in https://github.com/mckinsey/vizro/issues/114#issuecomment-1770347893
Just from quickly playing around with the the example dashboard here, I prefer the AG Grid. I wonder whether we should be recommending that instead of Dash data table as the default table for people to use in the (I guess) most common case that someone just wants to draw a nice table?

for small screen sizes the table has its own little scroll bar rather than stretching out the whole screen
I can rearrange column order by drag and drop
the active row is highlighted
the style just feels a bit more modern and slicker somehow (though dark theme is obviously not usable right now)

I'd guess some of these you can probably also achieve with Dash data table by using appropriate styling parameters or arguments though?
These are just my impressions from playing around for a few seconds though, so don't take them too seriously - it's the first time I've used both sorts of table, so I don't know what each is capable of or how easily we can get all the features we want out of each. But curious what other people think - should Dash data table or AG Grid be the "default" table that we suggest people to use?

@maxschulz-COL @huong-li-nguyen @petar-qb interested in your thoughts on the above since you're all more familiar with what people like to do with our tables.
Either way I think we will eventually have both dash_data_table and dash_ag_grid available in our codebase and hence we will also need some brief documentation on which to use when (even if this just links to comparison pages already written on other sites).
So following #114 my proposed next steps would be:

implement dash_ag_grid (will help to ensure all the table filtering code etc. is sufficiently generic)
play around with AG Grid and DataTable some more to get a better feel for them
write docs/example apps to help users decide which to use when
Though none of this is high priority as far as I'm concerned.

Thanks for raising this issue @antonymilne!
I agree that we need to enable both implementations and probably many others, but right now I'm just being introduced to the capabilities and limitations of dash.dataTable. Here is a concrete example of what is possible with it and what we implemented in a previous project (link to video). Looks good? 😄 (there are some features that are not presented here, but are part of the same example, so contact me to show you all if you are interested).
I'm not very familiar with dash AgGrid capabilities (but I know it does not support server paging) so I can't say much about it. All I can say is that I'm happy with the customisation benefits that dash datatable' provides, but there's still a chance that dash AgGridis even more customisable and may have better performance benefits, but I can't say. However, I believe (but we need to prove this) thatdash datatable` is more widespread among the audience and that Vizro users are accordingly more familiar with this approach.

This looks like a fantastic project and nice to see it available open source.
It would be great to  see AG Grid as the default table component in Vizro.  DataTable is a good component, but I think Dash AG Grid  has a better UI and more features.  It's also easier to style and customize.

it would be great to compare the most commonly requested features in a table and provide some side-by-side comparisons.

I'm part of the community team developing Dash AG Grid and I'm also familiar with DataTable.  I could help with creating examples and doing side-by-side comparisons.

I'm not very familiar with dash AgGrid capabilities (but I know it does not support server paging)

Server side pagination is available:   https://dash.plotly.com/dash-ag-grid/infinite-row-model#pagination
More info on pagination here: https://dash.plotly.com/dash-ag-grid/pagination

a dark theme is not usable right now

Is this still an issue?  If so, I could probably help with that.  For example, this site demos Dash AG Grid in light and dark color modes for all the themes available in the Dash Bootstrap Components library.

dash datatable is more widespread among the audience and that Vizro users are accordingly more familiar with this approach.

I expect this is true since  DataTable has been around for a long time.  However,  it's not under active development -- there have been (almost)  no new features for years.   I think it won't be long before Dash AG Grid is the preferred component  for working with tabular data in Dash.

Hello @AnnMarieW, amazing to see such a Dash celebrity here! 🤩 I am constantly referring to your Dash pages posts and examples in particular - thank you for all your work on the amazing feature! What we're doing just wouldn't have been possible without everything you've done. I'm curious to know how you found out about Vizro if you don't mind sharing?
Thanks for all the comments on Dash AG Grid, it's great to get some more context on it. I didn't realise that DataTable wasn't actively developed any more, but from what you say it sounds like AG Grid should be the preferred option going forward 👍
The comment about the dark theme not being usable right now is not at all a problem with AG Grid, it was just because we hadn't put any effort into styling yet. I'm sure we  will be able to style it properly when we get a chance to do so (or can ask you for help if we have difficulty!).
Since I first made this issue, things have changed a bit so just to update you: we actually released the table functionality last week with vizro 0.1.6. By design there is a single Table model which can take a (somewhat) arbitrary table function in its figure argument. Hence it's actually already sort of possible to use Dash AG Grid in Vizro just by using a custom table function. We deliberately did not tie the Table model to Dash DataTable so that we could enable AG Grid easily in future.
The reason I say sort of possible is that, while the grid will render and respond correctly to filters, we do not yet have:

built in vizro.tables.dash_ag_grid function
nice styling applied (also work in progress for the Dash DataTable actually to update the styling so it matches designs)
ability to use all the different callbacks with Dash AG Grid natively with our actions system (e.g. to filter a graph in response to clicking on the table)

All of these are very much on the backlog though and I hope we will be able to finish implementing them by the end of year!
Due to the Table model structure there is no good way for us to specify a default Table.figure function though (just like we don't have a default Graph.figure function). In this sense all our built in table functions (dash_data_table and soon dash_ag_grid also) are "equally preferred", since one or the other must be explicitly chosen by the user. It sounds like AG Grid should be first choice for most users though, so we should definitely make that recommendation clear in our documentation once we've enabled it properly.

Hi @antonymilne

"a Dash celebrity"

Haha - you are too kind  :blush:   . I'm glad you found my contributions helpful. I think I discovered Vizro in the trending repos section in my GitHub homepage feed. I also read your Medium article.
It can be challenging to create a low code solution that is also scalable and   customizable.   It looks like you are on track to achieve all these objectives with Vizro.  I'm a huge fan of open source projects, and I'm happy to help with any Dash related issues.


Visual Selector and Live Visual Tree like WPF / UWP in Visual Studio

Is your feature request related to a problem? Please describe.
It is often very laborious going from the application to where the AXAML code is, especially in the case of ResourceDictionaries and templated controls
Describe the solution you'd like
A clear and concise description of what you want to happen. When developing WPF / UWP applications with Visual Studio 2022 there is a little toolbox on the top of the application containing the visual selector.
By clicking on it and then selecting an element, it will select and focus it in the Live Visual Tree, from which it is possible to click on "View Source" which will bring you immediately to the XAML source code:

This seems like a trivial functionality, but it saves an significant amount of time that is otherwise spent on looking for stuff.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered. If the above is not possible, even some meta-data in the existing Dev Tools visual tree would be pretty useful. Something like "<AXAML_File>::".
Additional context
Labels : [area-dev-tools] [enhancement]

We have developer tools already.
While technically possible to link dev-tools to XAML in runtime, it is unlikely to happen soon.

Is it possible to manually split the axaml into subfiles and organize them as a file tree in the solution panel?
In a main axaml file, one uses  to include subfiles.

@jabobian please use Discussions for questions. Thx.


Remove redundant module explicit qualification from cors_catch/2 definition

As cors_catch/2 is declared as a meta-predicate, the meta-argument is implicitly qualified by the runtime. Moreover, the predicate clause doesn't make use of the module qualifier.

Thanks!


onTouchStart, onTouchEnd, onTouchMove on Mesh?

Hi, 👋
In what way can I trigger touchStart, touchEnd, touchMove events on Mesh if I have separate logic from mouse click logic ?
<mesh
onTouchStart={}
...
/>

You can do onPointerDown/Up/Move, pointer events are for mouse, touch and even pens.

But if I have another logic on touchstart event. For example add the check:
 if (e.changedTouches.length === 1) { ... }
 if  (e.touches.length === 2)  { ... }
get the value of e.changedTouches.length

you have e.buttons which is 0 i believe if no mouse button is pressed. Overall you can go to mdn and read about PE, they have all the informations you would need. it's def possible to handle touch differently than mouse.


feat: get information about global resource availability

Sometimes users end up with notebooks in a pending state without knowing that there aren't enough resources available in the cluster. This is especially true for GPUs that are very limited in some deployments.
We could get this information from kubernetes and use it for one (or both) of the following:

Provide a /stats API
Fail early in the environment launch phase when resources are unavailable

Check if 1) we are already getting this information and 2) if it's clear to the user
We should already get this
Step: PodScheduled
Reason: Unschedulable
Message: 0/13 nodes are available: 1 Insufficient memory, 12 node(s) didn't match node selector.

We may need to improve the message or provide adequate documentation

@lorenzo-cavazzi can we clarify what the /stats api should do? cc @olevski

The initial idea was to get information about the availability of the resources to prevent stalling when starting a new environment requiring currently unavailable resources (e.g. when asking for 1 gpu and no gpus are available).
A few things changed in the meanwhile, and I guess it's not easy to have something like this in our environment -- with different nodes, it may not be obvious where the pod for a new interactive environment ends up.
A better solution could be to fail the lunch with a reasonable error message when no resources are available. That seems easier to implement and the UX won't be much worse than the original proposal.

I believe it should be possible to query the kubernetes API about resources and provide constraints on the types of nodes you want to consider. imho preventing a launch is better than recovering from a failed launch

Ok, then for the UI the idea would be to get extra information about the current availability of the resources.
The GET /options endpoint returns the available resources for the pods. Let's say we know an interactive environment can request from 1 to 4 cpu and from 0 to 2 gpu. The /stats endpoint should tell us how many cpu and gpu are currently available so that we can further limit the resources -- or even better notify the user that it's temporarily not possible to start a session with more than <currently_available> resources.
If that is feasible, I can quickly formalize the endpoint proposal by sketching it in SwaggerHub.

Apparently this is not easily solvable (see this discussion) but  some tools exist: https://github.com/davidB/kubectl-view-allocations

I am sorry for being late to this. My github notifications are not setup right so I get way too many emails about random things and I miss things where I am needed.
@rokroskar that is a very good link you posted. In there I found a command like this, which will give you the utilization of every pod in a namespace (or all namespaces).
kubectl get po -o=jsonpath="{range .items[*]}{.metadata.namespace}:{.metadata.name}{'\n'}{range .spec.containers[*]}  {.name}:{.resources.requests.cpu}{'\n'}{end}{'\n'}{end}"

So if we use this command on the backed and combine it with something like top nodes or similar to see what is the total number and total capacity of the nodes then I believe we can get the data that @lorenzo-cavazzi is looking for.
Some other questions (that I think can be resolved) if we pursue this are:

If I am not mistaken we do not use taints, affinities, etc to limit that the user servers run on a limited set of pods? So then the question is how does one determine the total capacity that is available for user servers. To resolve this properly we may need to designate which nodes will run user servers and enforce this with affinities, taints, tolerations.
Not sure whether the credentials that the API uses to access the k8s cluster have enough permissions to allow it to run a command like kubectl get nodes or something similar to get the total capacity of the nodes.


Proposal for #39

This fixes bug #39 on Windows.
Not being familiar with Darwin, I have made some changes that I believe make sense, but they need a double-check.
Moreover, I didn't dig deep with Linux since I'm currently unable to test.
TODO:

[x] Test Darwin build
[ ] Fix for Linux
[x] Fix for Darwin

Looks like a good start, thanks

I was just working on macOS for this and it seems like the parent parameter is not needed. The following code worked in my tests:
- (void) add_separator:(NSNumber*) menuID
{
  NSMenuItem* menuItem = find_menu_item(menu, menuID);
  if (menuItem != NULL) {
    [menuItem.submenu addItem: [NSMenuItem separatorItem]];
  } else {
    [menu addItem: [NSMenuItem separatorItem]];
  }
}

Closing in favour of #41


Spire Agent kubelet verification: curl: option --caPath: is unknown

It looks like in version 0.23.0 the spire-agent.workloadAttestors.k8s.verification.type feature does not work properly. Curl does not have a --caPath option. It looks like it's case sensitive and should be --capath. Location in the source code
I could verify that in the container image of the gather-host-cert container as follows:
docker run cgr.dev/chainguard/bash:latest@sha256:f5c85affd2aa0f55fc1ead7dc07952577ad82741bbbba742ead0fd9dde2de14a "curl --caPath"
curl: option --caPath: is unknown

Passing the option all lowercase looks better:
docker run cgr.dev/chainguard/bash:latest@sha256:f5c85affd2aa0f55fc1ead7dc07952577ad82741bbbba742ead0fd9dde2de14a "curl --capath"
curl: option --capath: requires parameter

Thanks @avanschie. --caPath works fine on the mac where we did this testing. Will submit PR to make it lower case.


No matching distribution found for blockfrost-python

I'm having an issue installing via pip. I have tried different machines but it's the same issues.
Tried Python 3.6.8 and 3.6.9.
Any assistance appreciated.
(development_env) root@tars:~$ pip install blockfrost-python
Collecting blockfrost-python
  Could not find a version that satisfies the requirement blockfrost-python (from versions: )
No matching distribution found for blockfrost-python

@tars01 Thanks for reaching out.
Python 3.6 has exceeded end life support.
So this package has been configured with Requires: Python >=3.7, <4.
That is why a version for 3.6 is not showing.
I will highly suggest you upgrade your python version, to a version that receives security fixes.
Hope this clears up the issue you encountered.

You are welcome.
I will close this issue, feel free to reach out if you encounter any other issues.


Show gas cost estimation for publish

For each method we call during publish we have a corresponding method to estimate its gas usage in ocean.js. So make use of those in the submission screen and display estimated cost (in ETH + user's currency, or just user currency) for each step beside the transaction count labels

are these the methods i need to call, right?
file<EMAIL_ADDRESS>methods:

Step 1 (Create Tokens & Pricing)

estGasCreateNFT

Step 2 (Construct & Encrypt DDO)

none

Step 3 (Publish DDO)

estGasCreateNftWithErc20

I've been trying to debug these params but for what i can see all params are being populated, I'm not sure if the information is actually right thought. Here's the full payload for each call.
Free Pricing - method: estGasCreateNftErc20WithDispenser
accountId: 0xC50ce98B13Aa9782cAc4aF6809007B2b035E599e, nftCreateData : {     "name": "Ocean Asset NFT",     "symbol": "OCEAN-NFT",     "templateIndex": 1,     "tokenURI": "data:application/json;base64,eyJuYW1lIjoiT2NlYW4gQXNzZXQgTkZUIiwic3ltYm9sIjoiT0NFQU4tTkZUIiwiZGVzY3JpcHRpb24iOiJUaGlzIE5GVCByZXByZXNlbnRzIGFuIGFzc2V0IGluIHRoZSBPY2VhbiBQcm90b2NvbCB2NCBlY29zeXN0ZW0uIiwiZXh0ZXJuYWxfdXJsIjoiaHR0cHM6Ly9tYXJrZXQub2NlYW5wcm90b2NvbC5jb20iLCJiYWNrZ3JvdW5kX2NvbG9yIjoiMTQxNDE0IiwiaW1hZ2VfZGF0YSI6ImRhdGE6aW1hZ2Uvc3ZnK3htbCwlM0Nzdmcgd2lkdGg9Jzk5JyBoZWlnaHQ9Jzk5JyBmaWxsPSd1bmRlZmluZWQnIHhtbG5zPSdodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyclM0UlM0NwYXRoIGZpbGw9JyUyM2ZmNDA5Mjc3JyBkPSdNMCw5OUwwLDIxQzgsMjMgMTcsMjUgMjUsMjVDMzIsMjQgMzgsMjIgNDcsMjBDNTUsMTcgNjUsMTYgNzUsMTdDODQsMTcgOTEsMjAgOTksMjNMOTksOTlaJy8lM0UlM0NwYXRoIGZpbGw9JyUyM2ZmNDA5MmJiJyBkPSdNMCw5OUwwLDQ2QzUsNDUgMTEsNDUgMjEsNDZDMzAsNDYgNDIsNDcgNTIsNTBDNjEsNTIgNjgsNTUgNzYsNTZDODMsNTYgOTEsNTQgOTksNTNMOTksOTlaJyUzRSUzQy9wYXRoJTNFJTNDcGF0aCBmaWxsPSclMjNmZjQwOTJmZicgZD0nTTAsOTlMMCw2OUM4LDcwIDE3LDcyIDI1LDcyQzMyLDcxIDM5LDY4IDQ3LDcwQzU0LDcxIDYzLDc3IDcyLDc5QzgwLDgwIDg5LDc2IDk5LDcyTDk5LDk5WiclM0UlM0MvcGF0aCUzRSUzQy9zdmclM0UifQ==" }, ercParams : {     "templateIndex": 2,     "minter": "0xC50ce98B13Aa9782cAc4aF6809007B2b035E599e",     "feeManager": "0xC50ce98B13Aa9782cAc4aF6809007B2b035E599e",     "mpFeeAddress": "0x9984b2453eC7D99a73A5B3a46Da81f197B753C8d",     "feeToken": "0x8967bcf84170c91b0d24d4302c2376283b0b3a07",     "feeAmount": "0",     "cap": "1000",     "name": "Comely Cod Token",     "symbol": "COMCOD-10" }, dispenserParams: {     "dispenserAddress": "0x5FfDe8179E095803de22CBe1D405Eeef18406852",     "maxTokens": "1000000000000000000",     "maxBalance": "1000000000000000000",     "withMint": true,     "allowedSwapper": "0x0000000000000000000000000000000000000000" }
Fixed Pricing - method: estGasCreateNftErc20WithFixedRate
accountId: 0xC50ce98B13Aa9782cAc4aF6809007B2b035E599e, nftCreateData : {     "name": "Ocean Asset NFT",     "symbol": "OCEAN-NFT",     "templateIndex": 1,     "tokenURI": "data:application/json;base64,eyJuYW1lIjoiT2NlYW4gQXNzZXQgTkZUIiwic3ltYm9sIjoiT0NFQU4tTkZUIiwiZGVzY3JpcHRpb24iOiJUaGlzIE5GVCByZXByZXNlbnRzIGFuIGFzc2V0IGluIHRoZSBPY2VhbiBQcm90b2NvbCB2NCBlY29zeXN0ZW0uIiwiZXh0ZXJuYWxfdXJsIjoiaHR0cHM6Ly9tYXJrZXQub2NlYW5wcm90b2NvbC5jb20iLCJiYWNrZ3JvdW5kX2NvbG9yIjoiMTQxNDE0IiwiaW1hZ2VfZGF0YSI6ImRhdGE6aW1hZ2Uvc3ZnK3htbCwlM0Nzdmcgd2lkdGg9Jzk5JyBoZWlnaHQ9Jzk5JyBmaWxsPSd1bmRlZmluZWQnIHhtbG5zPSdodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyclM0UlM0NwYXRoIGZpbGw9JyUyM2ZmNDA5Mjc3JyBkPSdNMCw5OUwwLDIxQzgsMjMgMTcsMjUgMjUsMjVDMzIsMjQgMzgsMjIgNDcsMjBDNTUsMTcgNjUsMTYgNzUsMTdDODQsMTcgOTEsMjAgOTksMjNMOTksOTlaJy8lM0UlM0NwYXRoIGZpbGw9JyUyM2ZmNDA5MmJiJyBkPSdNMCw5OUwwLDQ2QzUsNDUgMTEsNDUgMjEsNDZDMzAsNDYgNDIsNDcgNTIsNTBDNjEsNTIgNjgsNTUgNzYsNTZDODMsNTYgOTEsNTQgOTksNTNMOTksOTlaJyUzRSUzQy9wYXRoJTNFJTNDcGF0aCBmaWxsPSclMjNmZjQwOTJmZicgZD0nTTAsOTlMMCw2OUM4LDcwIDE3LDcyIDI1LDcyQzMyLDcxIDM5LDY4IDQ3LDcwQzU0LDcxIDYzLDc3IDcyLDc5QzgwLDgwIDg5LDc2IDk5LDcyTDk5LDk5WiclM0UlM0MvcGF0aCUzRSUzQy9zdmclM0UifQ==" }, ercParams : {     "templateIndex": 2,     "minter": "0xC50ce98B13Aa9782cAc4aF6809007B2b035E599e",     "feeManager": "0xC50ce98B13Aa9782cAc4aF6809007B2b035E599e",     "mpFeeAddress": "0x9984b2453eC7D99a73A5B3a46Da81f197B753C8d",     "feeToken": "0x8967bcf84170c91b0d24d4302c2376283b0b3a07",     "feeAmount": "0",     "cap": "1000",     "name": "Comely Cod Token",     "symbol": "COMCOD-10" }, freParams: {     "fixedRateAddress": "0xdf29DA8FF979170eBfEd7133068dA247dc04CdC9",     "baseTokenAddress": "0x8967bcf84170c91b0d24d4302c2376283b0b3a07",     "owner": "0xC50ce98B13Aa9782cAc4aF6809007B2b035E599e",     "marketFeeCollector": "0x9984b2453eC7D99a73A5B3a46Da81f197B753C8d",     "baseTokenDecimals": 18,     "datatokenDecimals": 18,     "fixedRate": "5",     "marketFee": "0",     "withMint": true,     "allowedConsumer": "0x0000000000000000000000000000000000000000" }
Dynamic Pricing - method: estGasCreateNftErc20WithPool
accountId: 0xC50ce98B13Aa9782cAc4aF6809007B2b035E599e, nftCreateData : {     "name": "Ocean Asset NFT",     "symbol": "OCEAN-NFT",     "templateIndex": 1,     "tokenURI": "data:application/json;base64,eyJuYW1lIjoiT2NlYW4gQXNzZXQgTkZUIiwic3ltYm9sIjoiT0NFQU4tTkZUIiwiZGVzY3JpcHRpb24iOiJUaGlzIE5GVCByZXByZXNlbnRzIGFuIGFzc2V0IGluIHRoZSBPY2VhbiBQcm90b2NvbCB2NCBlY29zeXN0ZW0uIiwiZXh0ZXJuYWxfdXJsIjoiaHR0cHM6Ly9tYXJrZXQub2NlYW5wcm90b2NvbC5jb20iLCJiYWNrZ3JvdW5kX2NvbG9yIjoiMTQxNDE0IiwiaW1hZ2VfZGF0YSI6ImRhdGE6aW1hZ2Uvc3ZnK3htbCwlM0Nzdmcgd2lkdGg9Jzk5JyBoZWlnaHQ9Jzk5JyBmaWxsPSd1bmRlZmluZWQnIHhtbG5zPSdodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyclM0UlM0NwYXRoIGZpbGw9JyUyM2ZmNDA5Mjc3JyBkPSdNMCw5OUwwLDIxQzgsMjMgMTcsMjUgMjUsMjVDMzIsMjQgMzgsMjIgNDcsMjBDNTUsMTcgNjUsMTYgNzUsMTdDODQsMTcgOTEsMjAgOTksMjNMOTksOTlaJy8lM0UlM0NwYXRoIGZpbGw9JyUyM2ZmNDA5MmJiJyBkPSdNMCw5OUwwLDQ2QzUsNDUgMTEsNDUgMjEsNDZDMzAsNDYgNDIsNDcgNTIsNTBDNjEsNTIgNjgsNTUgNzYsNTZDODMsNTYgOTEsNTQgOTksNTNMOTksOTlaJyUzRSUzQy9wYXRoJTNFJTNDcGF0aCBmaWxsPSclMjNmZjQwOTJmZicgZD0nTTAsOTlMMCw2OUM4LDcwIDE3LDcyIDI1LDcyQzMyLDcxIDM5LDY4IDQ3LDcwQzU0LDcxIDYzLDc3IDcyLDc5QzgwLDgwIDg5LDc2IDk5LDcyTDk5LDk5WiclM0UlM0MvcGF0aCUzRSUzQy9zdmclM0UifQ==" }, ercParams : {     "templateIndex": 2,     "minter": "0xC50ce98B13Aa9782cAc4aF6809007B2b035E599e",     "feeManager": "0xC50ce98B13Aa9782cAc4aF6809007B2b035E599e",     "mpFeeAddress": "0x9984b2453eC7D99a73A5B3a46Da81f197B753C8d",     "feeToken": "0x8967bcf84170c91b0d24d4302c2376283b0b3a07",     "feeAmount": "0",     "cap": "1000",     "name": "Comely Cod Token",     "symbol": "COMCOD-10" }, poolParams: {     "dispenserAddress": "0x5FfDe8179E095803de22CBe1D405Eeef18406852",     "maxTokens": "1000000000000000000",     "maxBalance": "1000000000000000000",     "withMint": true,     "allowedSwapper": "0x0000000000000000000000000000000000000000" }

Maybe try to work with local version of ocean.js and debug there as well

Now i'm at a good point, but there's an issue is driving kind of nuts 😅
Basically, the "Create token & pricing" estimates seems right, but the "PublishDDO" estimates are way off.

Now I'm trying to debug this issue.

You can estimate them both before hitting submit. If you estimate the second after submit it kinda misses the point. To estimate the second you can just pass some dummy addresses ( i think it should work) to make the estimation. This would be ideal if possible

I did it that way because it seems that the estGasSetMetadata need erc721Address which I get after the first transaction and encryptedDdo and metadataHash (which I the ddo that I  get on the second transaction)
Basically the query I'm trying to do is this one:
await nft.estGasSetMetadata(     feedback['1'].erc721Address, // this is provided by the first transaction     accountId,      0,     services['0'].providerUrl.url,     '',     flags,     feedback['2'].encryptedDdo, // this is provided by the second transaction     '0x' + metadataHash // this is provided by the second transaction   )
Of course I'm assuming this by the setMetadata and whats in the estGasSetMetadata in the lib.
Can you share some dummy data or is there a way to find an example call to copy from like a documentation of some sort? Because unfortunately this methods doesn't return any issue as discussed, just 10000 as value (probably because the params are not correct)

Problem solved. It has been solve on ocean.js. Now i get the connect estimate for the publishDDO step.

You can estimate them both before hitting submit. If you estimate the second after submit it kinda misses the point. To estimate the second you can just pass some dummy addresses ( i think it should work) to make the estimation. This would be ideal if possible

I'm still trying to figure out what data to send as Dummy data, can i just use whatever data? can i just hardcode the data of a  real successful call?
because as previously discussed, the metadataHash is made of getHash(JSON.stringify(feedback['2'].ddo)), where I need to process the first transaction to get the feedback['2'].ddo. The same for the _encrptedDdo parameter. I get these params only after processing the first transaction (after the createNftErc20 calls).

at the moment we have this behaviour:

the Create Tokens & Pricing step displays the correct gas fees, but the Publish DDO shows 0 as values. Until this the first transaction si succeful

after the transaction is succesful, the Publish DDO step will show the correct gas fees. As discussed, this is not very effective as you get the gas fees at the moment of payment which is being trigger immediately. So, this becomes kind of useless.

The possible solutions I proposed are:

implement dummy data as @mihaisc suggested or
allow the user to trigger himself the Publish DDO payment by clicking again to the submit button before running the transaction. So the user has the time to see the gas fees before transacting

reopen if anybody starts using ETH mainnet


Fixes for gradle, AndroidThings 1.0, and joystick demo

Hello! I tried to build and run the demo app in this project on AndroidThings 1.0 and found a few build errors. Here are the fixes! I also enabled the joystick demo and can confirm that it is working after a small change.
Cheers!

sorry for the late reaction, i'm currently on leave and won't have time to check this before the 12th of october. Maybe @OlafKa  can have a look.

Review was succesful! Thx, for your support. We appreciate it very much.
Your fix will be included in Version 1.4