Feat: Multisig creation with remark

closes https://github.com/novasamatech/nova-spektr/issues/1756

There are still quite some things to iron out, such as error management for the form submission, UI glitches, and I'm investigating an issue I found in the multisig indexer, but I'd like to have your opinion on things such as:

do we need to support proxy signers
should I organize models differently (right now there's 1 for the flow, 1 for most of the data, 1 for the confirmation). I basically took example on the "add proxy" flow

I removed most comments. The multisig creation doesn't take into account proxies for now, and only left the commented logic for the flexible multisig wizard.

There are still quite some things to iron out, such as error management for the form submission, UI glitches, and I'm investigating an issue I found in the multisig indexer, but I'd like to have your opinion on things such as:

do we need to support proxy signers
should I organize models differently (right now there's 1 for the flow, 1 for most of the data, 1 for the confirmation). I basically took example on the "add proxy" flow

Proxy signatories is not a part of this feature.
The UI should match mockup because:

For multishard wallet, WalletConnect, Nova Wallet the user has to select network before selecting acounts
There should be an option to add signatory by address only (without contact book)

May I receive reviews on the content of the PR @tuul-wq please?

I made the following changes:

base this PR on top of the matrix removal one https://github.com/novasamatech/nova-spektr/pull/1898 so that we can keep subjects isolated
I made the change to have first the network selection + name then the signatories selection

Since I did not received any feeback on the effector logic yet, I'd like to start with that. While I'll get the UI to look closer to what the Figma wireframe looks like

@Tbaut I'm getting error when create a multisig wallet.

Seems that Signer Wallet is undefined

But I'm not able to select wallet on the screen where filling signatoies.

@pgolovkin the issue you encountered is probably because the first account is supposed to be one of your own account. This is not something that is enforced yet, and the selection of your wallet doesn't work yet. You can type in some address though, and if you do so, the first account should be yours, then the error disapears, you can type in other signatories etc. Then the signing should work (it did for me, I just tested). The submission to the blockchain failed bc of "wasm trap" but this is not linked to this PR, it's linked to the v12 api unless I missed something.

@pgolovkin the issue you encountered is probably because the first account is supposed to be one of your own account. This is not something that is enforced yet, and the selection of your wallet doesn't work yet. You can type in some address though, and if you do so, the first account should be yours, then the error disapears, you can type in other signatories etc. Then the signing should work (it did for me, I just tested). The submission to the blockchain failed bc of "wasm trap" but this is not linked to this PR, it's linked to the v12 api unless I missed something.

thanks for explanation!

@Tbaut now I'm getting TooFewSignatories error. But anyway signing works!
https://westend.subscan.io/extrinsic/21911935-2

I'll take care of this now that I know there doesn't seem to be big changes incoming in the architecture. From your Tx, I can see there are indeed 0 signatories, as other_signatories is null.

The list of issues to be fixed:

The "multisig" in chains.json must be added only for networks where the multisig indexer is running. So that the list of available networks for the multisig will be limited

Validation error should be added only when the user proceeds. Now it's opened when the user gets to the 2nd screen of the Create multisig flow

There is no additional cost (network fee + multisig deposit) that the user has to pay for the system_remark call on the 2nd screen of the Create multisig flow. Figma mockup link

Confirmation screen doesn't match the [mockup]
(https://www.figma.com/design/HgFuj8QZd0xqEbkG10DIfs/Flexible-multisig-|-Milestone-4?node-id=528-42184&t=h20RjewkkjpFoFmn-4).

Agree, that operation should not be added into the basket.

When the user successfully signs system_remark operation then the new wallet has to be selected and perations page has to be opened. As I remember it's a common flow for multisig operations, please check it.

4. https://www.figma.com/design/HgFuj8QZd0xqEbkG10DIfs/Flexible-multisig-|-Milestone-4?node-id=528-42184&t=h20RjewkkjpFoFmn-4

let's clarify with designer.

@pgolovkin I addressed your comments in the last commit.

I now take the proxy url when the network has the multisig option
The alerts are only displayed after the user made a mistake and clicked on next
I now display all additional costs on all screens
My screen is now very close to the mock up. The only difference is that it's larger. The reason I think it's better this way, is so that I can re-utilize the nice "Sign with XYZ" button. And this together with the "additional costs" really doesn't look good on small modal, e.g

When the user successfully signs system_remark operation then the new wallet has to be selected and perations page has to be opened. As I remember it's a common flow for multisig operations, please check it.

I need help, maybe @Asmadek @sokolova-an or @johnthecat can tell me how to do that.

ok
ok
ok
let's clarify with designer

I've adapted the confirmation step to the new design.
The only difference I kept on purpose is the conversion in $ value present on my implementation but not on the design, because I think it brings value to users

@Tbaut looks great!
Could you please check how the following scenario works:

The user selects more then 1 accounts that the user owns (1 is polkadot vault, 1 is nova wallet)
The user proceed
Which account will be selected for signing?

Also please check with Sergy which popup has to be used for showing signatories on the confirmation screen. For example for staking it looks like on the screenshot

I implemented the modal for signatories:

The user selects more then 1 accounts that the user owns (1 is polkadot vault, 1 is nova wallet)
The user proceed
Which account will be selected for signing?

To answer here, what I answered in tg:
For now, the first account selected has to be an account that is owned, and this is the default signer. Let's discuss the design if you want me to add/change something.

I added the signer selection, with a "Back" button and without the token info for now. I need some more info to manage the multi-shard accounts. I pinged Sergey to have a chat.

Now showing the Wallet rather than the accounts @pgolovkin in the signer selection.

@Tbaut looks great! Thanks!


Lumen 5.6 exporting as file instead of binary response

Hi contributor I have read and used your library that is really a very good piece of code to utilize in project. I have done import part but I am stuck on exporting part can someone help me that which thing I am doing wrong;
what I am using

Lumen 5.6
Laravel-Excel 3.1.7

In my Export Class
<?php
namespace App\Modules\Marketing\Requests\Submission;

use Maatwebsite\Excel\Concerns\FromQuery;
use Maatwebsite\Excel\Concerns\Exportable;
use Maatwebsite\Excel\Concerns\WithMapping;
use App\Modules\Marketing\Models\SubmissionModel;

class SubmissionExport implements FromQuery, WithMapping
{

    use Exportable;

    public function __construct($requestData, $dataMap = [])
    {
        $this->requestData = $requestData;
        $this->dataMap = $dataMap;
    }

    public function map($data): array
    {
        $mappedData = [];
        foreach ($this->dataMap as $key => $value) {
            $tempArray[] = $data->$value;
        }
        return $tempArray;
    }

    public function query()
    {
        $response = SubmissionModel::query();
        //here check if directory param is given
        foreach ($this->requestData as $key => $field) {
                $response = $response->where($key, $field);
        }
        return $response;
    }
}

In my Request Class
namespace App\Modules\Marketing\Requests\Submission;

use App\Core\Base\Request;
use App\Modules\Marketing\Models\SubmissionModel;
use Excel;

class ExportRequest extends Request
{

    private $submissionModel;

    function __construct()
    {
        $this->submissionModel = new SubmissionModel();
    }

    public function process($requestData)
    {
        //here transform the response in data
        return Excel::download(new \App\Modules\Marketing\Requests\Submission\SubmissionExport($requestData, ['title','description']), 'submisssion.xlsx');
    }

Here in request class I am using Excel because I have defined class_alias(Maatwebsite\Excel\Facades\Excel::class, 'Excel'); in bootstrap/app.php
Specified request return me data in binary form instead of dowlaoding file as submission.xlsx as

o�WNG�D�Xo�WNG�D�Xo�WNG�D�Xo�WNG�D�Xo�WNG�D�Xo�WNG�D�Xo�WNG�D�Xo�WNG�D�Xo�WNG�D�Xo�WNG�D�Xo�WNG�D�Xo�WNG�D�Xo�WNG�D�Xo�WNG�D�X

and so on more data like this. Please help me how I can download this file.

Hey @mudasserzahid we currently don't really support Lumen, however we have in the past added some things to improve support for Lumen. I haven't tested downloads in Lumen, but I'm assuming it should work the same as in Laravel. download() returns a BinaryFileResponse in Laravel. (Whichs triggers a download) You could try and dd(Excel::download() and see if that's the same for Lumen.
Make sure the actual Excel::download response is returned in the controller and not converted to a string or something. I'm not familiar with the Request::process method you are using, if that's being called in a route or controller, make sure you return that response.

Thanks for your response @patrickbrouwers. I think we should test this library also with API point of view.
Here are some solution which I have tried with this library while returning the binary file. I set the response header as Content-Type', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet . That thing was helpful in case receiving file in attachments like
http://prntscr.com/mpbq6q
but issue is still same there is no file download in request.
After solution given by you and Here is the response after dd(Excel::(download)
http://prntscr.com/mpbqgd
Let me know more information, that we can make this library more compatible. Happy coding :+1:

How are you requesting the download, are you doing an ajax call from the frontend?

I am doing specified request using Postman.

FYI. Here is some testing which has done by my side, I changed something in Maatwebsite\Excel\Writer, there is a method named tempFile. I just concatenated .xls with string.
Now it is storing the accurate output in temp directory like if I do,
$response = Excel::download();
print_r($response->getFile());

Specified code will return me a string (path to excel file in temp directory). Further I am still reading your library.

If you use postman/xhr/ajax call than it's logical you see the binary response. If you want to download the excel file in the frontend, you should let people directly call that URL,not via an ajax call.

Yes, you are right in your point, but I need this as an API point of view. Because Lumen is for writing web services. In other cases, I just explored some libraries like mPDF what they are doing in Lumen while the download is as,
$output = $this->output();
return new Response($output, 200, array(
        'Content-Type' => 'application/pdf',
        'Content-Disposition' =>  'attachment; filename="'.$filename.'"'
));

in our case this should work like this
$response = Excel::download(new ExportTransfer, 'submissions.xls');
$response->sendHeaders(array(
      'Content-Type: application/vnd.ms-excel',
      'Content-Disposition: attachment; filename="submissions.xls"'
));
return $response;

But still there is not any proper response, in header we can see a file but physically there is no existance of file.

You could try to use Excel::raw and stream that ?

I have not tried Excel::raw there is not any method named as in the library.

Here is the solution of specified exception, the thing about which I am saying was a bad response actually is an accurate response. All the thing you need is work with Blob on front and export file to your desired extension. For the time being, I am using Angular JS as a front-end application. I will share also source code for Angular JS if that would be relevant to this repository.


SAML error: SAML message intended destination endpoint did not match the recipient endpoint

I am fairly naive about SAML and the spring framework in general, but I am trying to configure our ShinyProxy server to use my organization's SAML server, and I'm receiving the following error after what appears to be a successful authentication:
2021-01-21 19:22:52.748 ERROR 1 --- [  XNIO-1 task-1] o.o.c.b.decoding.BaseSAMLMessageDecoder  : SAML message intended destination endpoint 'https://myapphost/saml/SSO' did not match the recipient endpoint 'http://myapphost/scheduler/saml/SSO'
The issue appears to be with something related to the https/http protocol being mismatched. This app is behind an Apache reverse proxy, which from what I've read might be causing the issue. The RP redirects from https://myhostname/myapp to the exposed Docker host port 8888 (as defined in my application.yml file).
The apache config for this remote proxy is:
# Websocket stuff
# Needs the rewrite and proxy_wstunnel modules.
        RewriteEngine On
        RewriteCond %{HTTP:Upgrade} =websocket [NC]
        RewriteRule /scheduler/(.*)    ws://<IP_ADDRESS>:8888/myapp/$1 [P,L]

#
# Docker stuff for the scheduler.  Requires the proxy_http
# module to be enabled.
        ProxyPass /myapp http://<IP_ADDRESS>:8888/myapp
        ProxyPassReverse /myapp http://<IP_ADDRESS>:8888/myapp

Some related pages I've found in my search:

https://github.com/openanalytics/containerproxy/pull/32
https://docs.spring.io/spring-security-saml/docs/2.0.x/reference/html/configuration-advanced.html#configuration-load-balancing
https://github.com/johannestang/shinyproxy-lb
https://github.com/openanalytics/shinyproxy/issues/247

Here is my application.yml (with some private information changed/removed):
proxy:
  authentication: saml
  title: My app
  favicon: file:///opt/shinyproxy/templates/favicon.ico
  admin-groups: admins
  logo-url: file:///opt/shinyproxy/templates/my_logo.png
  template-path: /opt/shinyproxy/templates/mytemplate
  bind-address: <IP_ADDRESS>
  port: 8888
  # change the landing-page to restrict shinyproxy to one application:
  landing-page: /myapp/app/myapp
  saml:
    idp-metadata-url: <URL TO metadata XML>
    app-entity-id: <MY APP's DEPLOYMENT PATH>
    app-base-url: <MY APP's DEPLOYMENT PATH>
    name-attribute: http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn
    roles-attribute: http://schemas.microsoft.com/ws/2008/06/identity/claims/role
  support:
    mail-to-address<EMAIL_ADDRESS>  docker:
    internal-networking: true
    container-network: shinyproxy-net
  container-log-path: ./container-logs
  specs:
    - id: myapp
      display-name: My tool
      description: Description for my app
      container-cmd: ["R", "-e", "shiny::runApp('/root')"]
      container-image: my-image
      container-network: "${proxy.docker.container-network}"

# for email
spring:
  mail:
    host: my.smtp.server
    # TLS: 587 SSL: 465 Plain: 25
    port: 25

server:
  useForwardHeaders: true
  forward-headers-strategy: native
  servlet.session.timeout: 3600
  servlet:
    context-path: /myapp

logging:
  file:
    shinyproxy.log

I have tried this with ShinyProxy 2.4.3 deployed in Docker using the following Dockerfile:
FROM openjdk:8-jre

RUN mkdir -p /opt/shinyproxy/
RUN wget https://www.shinyproxy.io/downloads/shinyproxy-2.4.3.jar -O /opt/shinyproxy/shinyproxy.jar
COPY application.yml /opt/shinyproxy/application.yml
COPY templates/ /opt/shinyproxy/templates/

WORKDIR /opt/shinyproxy/
CMD ["java", "-jar", "/opt/shinyproxy/shinyproxy.jar"]

Any ideas if I can fix this issue by changing a configuration setting somewhere?

Hi @jat255
This error typically happens when you access ShinyProxy through a load balancer or reverse proxy on a different path (/URL) than that is configured in ShinyProxy. I think this is also the case here.
For example, this is the case when you access ShinyProxy using https://mydomain/scheduler and the proxy forwards these requests to https://mydomain/myapp.
There are two solutions to this:

streamline all configuration.

make sure the contextpath which is configured in ShinyProxy is also the contextpath you will effectively use when accessing ShiynProxy. Let's say you access ShinyProxy at /abcd then ensure that the proxy is setup to forward requests at /abcd to /abcd on ShinyProxy and make sure you configure the contextpath in ShinyProxy to be /abcd
ShinyProxy should know that it is accessed over HTTPS, even if the reverse proxy accesses ShinyProxy over HTTP. This is done using the X-Forwarded-Proto and X-Forwarded-For headers. I see you already have ShinyProxy configured to check for these headers, however, I think you also have to configure apache to set the correct headers (see https://webmasters.stackexchange.com/a/107445 and https://serverfault.com/a/257643/261145 or any other resource).

use the Spring SAMLContextProviderLB option (https://github.com/openanalytics/containerproxy/pull/32). I think you already chooses for this route.

Hi @jat255
This error typically happens when you access ShinyProxy through a load balancer or reverse proxy on a different path (/URL) than that is configured in ShinyProxy. I think this is also the case here.
For example, this is the case when you access ShinyProxy using https://mydomain/scheduler and the proxy forwards these requests to https://mydomain/myapp.
There are two solutions to this:

streamline all configuration.

make sure the contextpath which is configured in ShinyProxy is also the contextpath you will effectively use when accessing ShiynProxy. Let's say you access ShinyProxy at /abcd then ensure that the proxy is setup to forward requests at /abcd to /abcd on ShinyProxy and make sure you configure the contextpath in ShinyProxy to be /abcd
ShinyProxy should know that it is accessed over HTTPS, even if the reverse proxy accesses ShinyProxy over HTTP. This is done using the X-Forwarded-Proto and X-Forwarded-For headers. I see you already have ShinyProxy configured to check for these headers, however, I think you also have to configure apache to set the correct headers (see https://webmasters.stackexchange.com/a/107445 and https://serverfault.com/a/257643/261145 or any other resource).

use the Spring SAMLContextProviderLB option (https://github.com/openanalytics/containerproxy/pull/32). I think you already chooses for this route.

@LEDfan thanks for the response!
RE: your first point, I believe I have all my configuration set up correctly, not changing scheduler to myapp in my posted apache config was just an artifact of not fully redacting my config files, whoops! What I don't think I have set up is the X-Forwarded-Proto and X-Forwarded-For headers on apache. I believe I have it working using the SAMLContextProviderLB option, but I would rather be able to use the native ShinyProxy build so we have fewer things to maintain. I'll give the headers a shot and report back here if it works.

@LEDfan thanks for the response!
RE: your first point, I believe I have all my configuration set up correctly, not changing scheduler to myapp in my posted apache config was just an artifact of not fully redacting my config files, whoops! What I don't think I have set up is the X-Forwarded-Proto and X-Forwarded-For headers on apache. I believe I have it working using the SAMLContextProviderLB option, but I would rather be able to use the native ShinyProxy build so we have fewer things to maintain. I'll give the headers a shot and report back here if it works.

Ok, I've added those headers and confirmed things seem to be working properly. Thank you for your help!

Ok, I've added those headers and confirmed things seem to be working properly. Thank you for your help!


header on a title and toc pages

Is there a way in theme customization to have header on a title and toc page?
I have the following in a theme:
 header:
   font_size: $base_font_size_small
   height: $base_line_height_length * 6.0
   vertical_align: middle
   recto:
       columns: "<30% 40% >30%"
       left:
         content: 'image:logo/logo-1.png[a, scaledwidth="20%"]'
       right:
         content: 'image:logo/logo-2.png[a, scaledwidth="20%"]'
   verso:
       columns: "<30% 40% >30%"
       left:
         content: 'image:logo/logo-1.png[a, scaledwidth="20%"]'
       right:
         content: 'image:logo/logo-2.png[a, scaledwidth="20%"]'

I don't believe so currently.  I've resorted to changing the following in converter.rb (line 161 in my version):
num_front_matter_pages = page_number - 1

to
num_front_matter_pages = (doc.attr 'pdf-start-running-content-page', page_number).to_i - 1

This allows me to put the following into my document to control the page on which running headers and footer start while existing documents get the current behaviour:
:pdf-start-running-content-page: 1

Hope this helps.

I have a very similar requirement to have headers and footers displayed on the Front Page and TOC (it's for the aerospace/military industry and have zero flexibility on this!).
It would be nice to at least have the option to switch on headers and footers on these pages, even if by default it's switched off for the rest of the community.

Would be good for me too. Could it also have running headers/footers, roman numeral page numbers for the TOC and preface, and the first chapter starting at page 1?

As @deevodavis71 I have Quality Documentation to generate without flexibility, that's (one of ) the last hurdle to switch people from MS Office to Asciidoc.

I would like to +1 this request

+1 for the record. We need it to comply with the legal requirements.

Chiming in, someone on my team pointed out that the TOC doesn't have the same header/footer as the rest of the document. I had never noticed it but now that I have, I recognize that it's wrong and needs to be fixed.
I can also see the utility of having the header/footer on the title page. We don't need that now, but might in the future. Allowing a user to specify a specific (possibly different) header/footer for the title page might actually be a more useful feature.

Similar to the above, when writing technical documentation (especially for Regulated environments) it's mandatory to have Header & Footer on every page.
I guess a flag/option to enable/disable it or a way to control it via the custom theme yaml would be a great way to support both use cases

I have tried to find where it could be tinkered in the src...
So it is converter.rb line 203 (here I have changed it to 0):
num_front_matter_pages = 0<EMAIL_ADDRESS>= page_number) - 1

This is the number of skipped pages to generate header and footer later on lines 215-222:
if page_count > num_front_matter_pages
  unless doc.noheader || @theme.header_height.to_f.zero?
    layout_running_content :header, doc, skip: num_front_matter_pages
  end
  unless doc.nofooter || @theme.footer_height.to_f.zero?
    layout_running_content :footer, doc, skip: num_front_matter_pages
  end
end

So basically, with that small change I can see header and footer on title and toc pages.

The only issue I can immediately see is with page numbering -- it starts from the title page now. Maybe this is not an issue - not sure yet. :)
For sure it is not "production" ready -- it clearly should controlled from theme file -- and I am not sure how it has to be implemented.

And example of header theming:
header:
  pages: all

change in converter.rb:
if page_count > num_front_matter_pages
  unless doc.noheader || @theme.header_height.to_f.zero?
    if (@theme.header_pages == 'all')
      layout_running_content :header, doc, skip: 0
    else
      layout_running_content :header, doc, skip: num_front_matter_pages
    end
  end
  unless doc.nofooter || @theme.footer_height.to_f.zero?
    layout_running_content :footer, doc, skip: num_front_matter_pages
  end
end

@mojavelinux I can try to come up with a PR implementing it, but not sure what kind of stuff should be implemented (and the probable impact on another parts of asciidoctor).

Name of the option(s) and values?
Should the page numbering always start after the toc?
Should we have option to put header/footer on title+toc, toc without title VS to put it everywhere, everywhere except title and toc.

When we started out, we had headers and footers on the toc page. People complained they shouldn't be on that page. So we removed them. Now we have a request to add them back. :scream: So what does that mean? An option, of course!
@habamax Thank you for offering to create PR. Based on your proposal, my advice is to break this down into smaller parts. Trying to do multiple changes at once almost always leads to a stalled change. If we take it one thing at a time, the changes will be much more likely to go through.
Let's first address the issue of which pages the header / footer should be on. This is definitely something that belongs in the theme file. I'd be happy to start with your proposal.
header:
  pages: all

I think we can assume that "all" does not mean the cover pages.
Another possible option is:
header:
  pages: +title,+toc

That might be more clear and scalable.
For now, stick with numbering pages as they are. That should be a separate issue.

@themowski wrote:
someone on my team pointed out that the TOC doesn't have the same header/footer as the rest of the document. I had never noticed it but now that I have, I recognize that it's wrong and needs to be fixed

you mean the toc page doesn't have a header/footer at all, right?

Please don't implement only half the solution.

I'm not suggesting that we do. But combining two issues into one means you get 0 solutions (which I can say speaking from lots of experience). These changes need to be handled in separate PRs.

@themowski wrote:
someone on my team pointed out that the TOC doesn't have the same header/footer as the rest of the document. I had never noticed it but now that I have, I recognize that it's wrong and needs to be fixed

you mean the toc page doesn't have a header/footer at all, right?

Ah, yes, sorry for the misleading wording! You're correct, I mean to say that there is no header/footer on the TOC.
Also -- +1 for adding customization to specify which, if any or both, of the title/toc get headers. I like the proposed pages: +title,+toc option for themes. Having the ability to put information (such as PROPRIETARY labels) on the title page in the footer would be a huge benefit, if we can't do that already.

Absolutely, but I'm afraid the issues may not be so easy to separate. For example, it's easy to make a change that appears to work, but where the page number in the header is out of sync with the TOC and/or the outline. The header theming example by @habamax above has that problem.
It may be easier to split the change something like this:

Add options to say whether the TOC, title+TOC, or none of them shall be considered part of the front matter. Numbering in this case will, as today, start from the page after the front matter and thus on the first page with a header (since headers are only rendered on non-front matter pages).
Add options to specify where numbering shall start, and make sure that headers/footers before that page gets numbered correctly (possibly with roman numerals, as in the outline?). This is probably a more complicated and less well-defined issue.

In any case, when testing, make sure to try with both header and footer containing "content: '{page-number}({page-count})'" and that they get numbers that correspond to both the TOC and the outline.

@habamax Are you willing to submit a proposal to implement support for the following?
header:
  pages: +title,+toc

Just split on comma, then look for +title or +toc, similar to what your patch was doing when looking for all.

@mojavelinux sure, I have just to find some time to figure out how to do it with PRs. Last time I did it was years ago :)

There's no need to use any PRs. Just upgrade to the latest release of Asciidoctor and you should see that the issue is resolved.

@mojavelinux I guess you've mixed our conversations: the one with diagrams and this one :)

Ooops! My mistake.
In that case, let me know if you need any assistance getting set up again.

Hi @mojavelinux,

I wasn't able to implement theming with +title,+toc.
It was not clear how to treat only +title. Should we have header on title and all except toc pages? If so -- I don't know how to do it :)
Instead I have startfrom: title|toc option meaning the header(footer) will start either from title or toc pages.

I couldn't find how to fix page numbering issue...
PS, PR stuff is not easy when you have 2 years old fork.

It looks like it is better to have this option not per header and footer, but probably more general one, like for a page.
For now page, with my PR, numbering works if both header and footer are started from the same page. Otherwise there is a discrepancy: header and footer shows different page numbers.

@habamax Thank you for implementing this. I had the same issue for the same reason. I've pulled your pull request into my local repo and it has worked great so far.

@landshark666, I am thinking of changing the theme params to have only 1 common option both for header and footer, just to avoid page numbering issue. Just not sure how to name it :)
Also @mojavelinux approvement would be nice to have

@habamax I don't have a use for it, but someone might find the ability to start the header/footer on different pages useful. Could a new command such as :pageNumber: 1 be introduced which explicitly sets the page from the current page forward?

@landshark666 I will have a look into it, but I doubt it would be easy to do.

In my document, I set
start_at: toc
footer:
...
recto:
right:
content:  '{chapter-title}'
I got

How can I change  'Preface' to 'TOC' on footer of TOC or just remove this word from footer?

@Alyaksej Please open a separate issue. This one is closed and what you're asking for is an enhancement (albeit a logical one).


gitAndTools.gh: 0.12.0 -> 1.0.0

https://github.com/cli/cli/releases/tag/v1.0.0

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@ofborg build gitAndTools.gh


VpnManagementAgent::GetProfilesAsync does not work in a Win32 app, but works in a UWP app

Describe the bug
VpnManagementAgent::GetProfilesAsync is supposed to return a list of your VPN profiles. But it gives me an empty vector in C++/WinRT. Running as an administrator does not help.
Steps to reproduce the bug
#include <winrt/Windows.Foundation.Collections.h>
#include <winrt/Windows.Networking.Vpn.h>

using namespace winrt::Windows::Networking::Vpn;

int main() {
    winrt::init_apartment();

    VpnManagementAgent vpn{};
    auto profiles = vpn.GetProfilesAsync().get();
    printf("Found %d profiles\n", profiles.Size());
}

Expected behavior
The very same function however returns all my vpn profiles in this UWP app.
Edit. I thought I'd add that strangely VpnManagementAgent::AddProfileFromObjectAsync works, if I run as administrator.  VpnManagementAgent::GetProfilesAsync seems to be have faulty logic.  I suspect that its logic presumes that a caller is always a UWP app and therefore not only has a package manifest but also has the required capabilities declared, which will not be true, if it is a Win32 app.
Screenshots
No response
NuGet package version
None
Packaging type
No response
Windows version
Windows 11 version 22H2 (22621, 2022 Update)
IDE
No response
Additional context
Visual Studio 2022 17.5.0 Preview 1.0.

We reached out to the owner of this API who confirmed it doesn't work in Win32, we don't have a timeframe of when or if this will be supported in the future.


Docker image stuck in "restarting" upon system reboot

Hi there, I've installed your app via its docker image, but whenever I have to reboot the server hosting it, Benotes fails to start up - it just keeps on restarting itself unsuccessfully. The only relevant line in the logs is this:
error: failed switching to "root": operation not permitted
Restarting the app with docker-compose down and docker-compose up works, but restarting the docker service or rebooting the machine results in the same failure.
I worked around this by adding the following line to the docker-compose.yml file within the service block for "app":
user: root
However, I am way too unfamiliar with docker to know if this is a cause for concern or not.

Hey, thank you for reporting this issue !
I will look into it once I'm back at home in a couple of days.
A quick search resulted in a lot of different causes for this error, it's seems quit common. But I have to try to recreate your bug and see how things go from there.

As far as I understand this is not a cause for concern.
So based on this issue on the base image I hopefully fixed your bug.
Let me know if it's still a problem or you find another one.
If not please close this issue.

Thanks for looking into this, and sorry for taking so long to get back to you/close. All the best.

With pleasure and no worries, I was just a little concerned something was wrong, but I'm glad that wasn't the case ;) All the best for you too.


[BW Clock] No font included

@peerdavid BW Clock loads quicker if using bitmap fonts, unsuprisingly :p I'm thinking I could maybe add app customization so the user can choose between "Fancier Looks" or "Faster Loading". What do you think?
The changes as they are now is what I have tested performance with. It is not intended for merging as is.

Oh wow thats great. Is there a huge difference how it looks when "fast loading"? If not I would guess fast loading should be the default right?

Here's a graphics dump with bitmap font:

Here's with vector font:

Yea it looks quite different, I honestly liked the original font -- in this case I prefer the "slow" solution which we should still provide

Yes, I think that would work well using app customization via the app loader as mentioned in the opening comment :)
I don't know how that's done yet, but I can try to learn if you don't particularly feel you want to implement it? ;)

BW Clock loads quicker if using bitmap fonts, unsuprisingly :p

Do you have any numbers?
Just thinking aloud here - if all the clock_info code apart from rendering ends up as part of the clock_info library, it feels like there's not actually that much code in the clock itself - and you'd likely have to change all the positioning for different fonts anyway.
It might be worth moving all the complicated stuff out into utility libraries so it's just 90% rendering code and then just forking the app?
Another option is to look at loading fonts from Storage. It could be done quite easily, and while rendering might be a smidge slower, startup time would be a lot better.

Do you have any numbers?

Not more than counting in my head, shy of maybe 0.7 seconds vs shy of 0.5 seconds. I could record a video for comparison. I don't know how to do timings programmatically.

It might be worth moving all the complicated stuff out into utility libraries so it's just 90% rendering code and then just forking the app?

I don't understand precisely what this means, so I don't have an opinion on it, maybe @peerdavid has one?

Another option is to look at loading fonts from Storage. It could be done quite easily, and while rendering might be a smidge slower, startup time would be a lot better.

That sounds like it could be sensible to me.

Not more than counting in my head,

I'd be very vary of making quite big sweeping changes without any numbers to back them up. Just add t=getTime() at the start of the app, and print(getTime()-t) at the end - and you'll get the time in seconds it took to load the app.

I don't understand precisely what this means,

I just mean that if there's very little code in the clock app itself (so it's just ~100 lines of code that is rendering the clock face) then potentially it just makes sense to create a copy that doesn't use fonts, since almost everything in the clock would change anyway in order to tweak the font positions.

Timings based on PR #2471:
Regular load(): 1.58590698242 s
Fast loading:    0.91567993164 s
Timings when using bitmap font in this PR:
Regular load():  1.22711181640 s
Fast loading:     0.55581665039 s

Do you think this is ready to merge?
It's still marked as draft but as it's a separate app I'd think it's fine to pull in?

I've been meaning to tweak font sizes and positions, but will probably not get around to it soon.
I've used it for a while myself.
So sure, go ahead 👍

Ok, thanks!


Implement JWT authentication

Reference:
https://tutorialedge.net/golang/authenticating-golang-rest-api-with-jwts/

@RahmaYasser please link the pull request to this issue. So it can get closed immediately the pull request is merged.

I think I don't have permissions to link pr to an issue


[srp-client] exclude sub-type PTR when service is being removed

This commit updates the SRP client to not include sub-types PTR
records when the base service is being removed.

Please see https://github.com/project-chip/connectedhomeip/issues/18507#issuecomment-1178050134 for background
Addresses https://github.com/project-chip/connectedhomeip/issues/18507

Thank you ! You just did something that the whole human being would be grateful!


Event when clicking date inputs

This question is relevant to react-dates but any third party components we use in our projects.
I want to bind an event to the inputs (#startDate, #endDate) but react-dates doesn't reveal an onClick prop or a ref for these inputs.
How does one bind an event to a DOM element inside a 3rd party component? Typically you'd just select the element by ID and bind an event, but that's not possible in React as far as I know.
Any feedback would be awesome, been stuck on this for hours. I must be missing something.

I've ran into this issue before as well, a temporary workaround would be to check when the focused element changes (which you have to store in state and pass to the date picker anyways). You won't be able to have access to "incomplete" dates that are typed with this approach, though.

Thanks for your quick response.
So now I really think I'm misunderstanding something because I keep getting the same answer about using refs and callbacks.
With react-dates, how do I add a ref to either of the start/end inputs? I understand how to do this when it's my own component, but not with a 3rd party component.

Hi @NathanCH, in general you can't access the ref of an installed 3rd-party package unless that package has explicitly exposed the ref. In particular, in react-dates, we try not to expose refs on our individual dom elements/components to maintain isolation of our components and not have any unexpected side-effects. Instead, we expose explicit callbacks (like onFocusChange and onDatesChange) that allow you to do something when a particular action takes place.
If you really want to select the id and do a thing, our inputs do have ids (in fact you can set them yourself) which you can then select on and do whatever. This is... however... a bad pattern in react and not recommended.
Can you explain a little bit about what you are trying to do in this event handler and maybe we can guide you to using a prop that would suit your usecase better?

Hi @majapw, thanks for taking the time to explain this to me. What you've said about component isolation makes sense.
I'll quickly show you what I'm trying to do, and the solution I came up with.
I have a select which allows users to choose predefined date ranges, or a custom date range. While the predefined date ranges are selected, I've disabled the react-dates inputs.

What I wanted to do was enable the react-dates inputs when they are clicked like this:

I think you could make the argument that the inputs should not be disabled in the first place, and this issue is moot. But I wanted to make sure I understood the limitations of 3rd party components so I created this thread.
As you can see in my gif I did get it to work by using findDOMNode, but from what I've been reading this will be deprecated soon. Further, I'm not actually selecting the inputs with findDOMNode, it just happens to work in my case.
 ReactDOM.findDOMNode(this).addEventListener('click', this.props.clickInput)

You could wrap the inputs with a <div/> that has a click handler which sets state.isFocused which will open the calendar, maybe?


Add metric to track Hooks feature usage

Describe the feature
To better understand usage/adoption of the lakeFS Hooks functionality, add a metric event that triggers when hooks are used.

Add new events for each of pre-merge, post-merge, pre-commit, and post-commit hook events please!


docs fix spelling issues

Hey ! In order to ensure a better understanding I fixed several spelling issues.

@jonathanpberger @zmanian @fadeev Hello! Can someone take a look at this pull request and merge it if everything looks good? Let me know if any adjustments are needed. Appreciate it!


Rebuild for libprotobuf 4.23.3

Checklist

[ ] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

@conda-forge-admin, please rerender

@conda-forge-admin, please rerender

If the re-render didn't do anything, this change isn't working. Would suggesting taking a closer look

@conda-forge-admin, please rerender

Might want to look at extended_keys

Might want to look at extended_keys

@jakirkham I looked over extended_keys but I'm not sure if I understand how it would be used here. libprotobuf is already part of a zip_keys with libgrpc, which seems like it might act differently than the example in the docs which is only one key. I pushed some commits that seem to give the right rerendering, but I'm very open to alternatives if this isn't the best approach.

@conda-forge-admin, please rerender

Might want to look at extended_keys

@jakirkham I looked over extended_keys but I'm not sure if I understand how it would be used here. libprotobuf is already part of a zip_keys with libgrpc, which seems like it might act differently than the example in the docs which is only one key. I pushed some commits that seem to give the right rerendering, but I'm very open to alternatives if this isn't the best approach.

Am worried that what we are doing here is a bit fragile since we are overriding global pinnings and manually adding values from migrators
Think it would be better if we used some method to append our values to the keys (instead of overriding them). Either using something from conda-build or by manually adding a migrator of our own to add the missed values

In terms of a migrator (which may wind up being the best/simplest solution), would try...

Copying one of the existing migrators to a new file
Adding use_local: true
Tweaking it to match whatever versions we are wanting to add
Re-rendering to apply the changes

This should add new builds with the expected versions (if not repeat 3-4)

In terms of a migrator (which may wind up being the best/simplest solution), would try...
1. Copying one of the existing migrators to a new file

2. Adding [`use_local: true`](https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/4400/commits/8c5323d2a65a49be297c216a9b4b3cd0135a3776)

3. Tweaking it to match whatever versions we are wanting to add

4. Re-rendering to apply the changes

This should add new builds with the expected versions (if not repeat 3-4)

This is a good idea. I like it!

May also need to make this change
-   - libprotobuf                            # [build_platform != target_platform]
+   - libprotobuf

@conda-forge-admin , please re-render


"yarn.resourcemanager.zk-address" has been deprecated

… "hadoop.zk.address".

Purpose of the pull request

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:

(or)
If your pull request contains incompatible changes, you should also pull request the documentation to https://github.com/datasophon/datasophon-website

https://issues.apache.org/jira/browse/HADOOP-15093

"yarn.resourcemanager.zk-address" has been deprecated and replaced by "hadoop.zk.address".


Paypal oauth is not working

Hi there,
I tried to add paypal authentication (with both sandbox and live options) but no luck. The application redirects to me the login page. Have anybody encountered this issue?

could you provide more details about the issue? Did you create your PayPal app correctly?

Hi @0xNacho , I created the PayPal app from here: https://developer.paypal.com/developer/applications/
I also specified the return Url: http://localhost:3000/api/auth/paypal/callback
Is it the right way to do it?
The app was created by my sandbox account (which is generated by paypal). The country of that sandbox account is US but my server is in EU. Does that matter?

I don't think so.
After logging in, are you getting a URL like this ?
http://localhost:3000/?error_uri=http://localhost:3000&error_description=Client Validation Error: Invalid scope.&error=invalid_scope

I'm not getting any errors. It doesn't even call the callback function: function (req, accessToken, refreshToken, profile, done) {} in paypal strategy.

Hi @0xNacho ,
Actually I got that error from the server logs. In the oauthCallback funtion, after passport authenticates the user == false, so the application redirects to the signin page. That why I didn't see the the url. Anyway, what could be the issue?

Could you gist your file?

https://gist.github.com/anphu7492/b34b28c881006db1585c#file-logs
Here it is

On line 4, I consoled value of error and user inside passport.authenticate function

I'm sure your problem is in the callback URL.
Did you change your paypal.callbackURL param into your  /config/env/development.js file?
It should be: /api/auth/paypal/callback
Also, you need to specify your callbackURL into your PayPal App.Change your Return URL param as follows:
``http://localhost:3000/api/auth/paypal/callback```
Did this solve your problem?

Unfortunately not, my configuration was exactly like you said, in the config file /config/env/development.js file:
paypal: {
clientID: process.env.PAYPAL_ID || 'myid',
clientSecret: process.env.PAYPAL_SECRET || 'mysecret',
callbackURL: '/api/auth/paypal/callback',
sandbox: true
}
callbackURL was specified like that as default. And I also specified the return URL on paypal developer dashboard. I also tried to clone the fresh meanjs project and configured paypal like my current project. I got the same issue

@0xNacho I got it working by configuring the advanced options of "Log In with PayPal, using Basic authentication doesn't work :) Thank you anyway :smile:

I wrongly assumed you had this option enabled :)


ChromeEC: No valid hash (status=0 size=2058478072). Compute one...

This string is always present in cbmem log, so hash is not cached which creates additional delays and potentially problematic: #186
Google Chrome EC: Initializing
EC returned error result code 1
Google Chrome EC: version:
        ro: fizz_v1.1.7899-258e4a3d2
        rw: fizz_v1.1.7909-6fc5ebe65
  running image: 1
CBFS: Found 'ecrw.hash' @0x9d200 size 0x20 in mcache @0x7abfd214
ChromeEC SW Sync: Expected hash: c5a1127f71d172f246e943dfb5487f1cac288a23d1aee75946c1e34fb2df16ff
ChromeEC: Getting hash:
ChromeEC: Getting hash:
ChromeEC: Getting hash:
ChromeEC: No valid hash (status=0 size=2058478072). Compute one...
ChromeEC: Starting EC hash:
ChromeEC SW Sync: current EC_RW hash: c5a1127f71d172f246e943dfb5487f1cac288a23d1aee75946c1e34fb2df16ff
ChromeEC SW Sync: EC_RW is up to date
Google Chrome EC: version:
        ro: fizz_v1.1.7899-258e4a3d2
        rw: fizz_v1.1.7909-6fc5ebe65
  running image: 2
PNP: 0c09.0 init finished in 989 msecs

I'm not sure why you think this is a problem, but it's not. The delay to calculate a hash on a normal boot is sub 5ms. And the issues that you're seeing on your device w/r/t USB-PD are not only not reproducible by me, but also not reported by any other Fizz users.

I don't think this is necessary the problem, but it might be, directly or indirectly.
Anyway, as I understood you this is not expected behavior, as it should cache and compute the hash only once.
The fact you can't reproduce it doesn't tell us much, I guess.
Do we know many users with my use-case? With my display model?
But we know a user on reddit not being able to boot with PD, also. IIRC, there were a few such reports.

Anyway, as I understood you this is not expected behavior, as it should cache and compute the hash only once.

it seems Fizz is different with using multiple RW ec fw options.

The fact you can't reproduce it doesn't tell us much, I guess.

the log is the same as mine; I can't reproduce your USB-PD issues

closing as this isn't a script issue


If BTC is currency then we get redundant pricing

If BTC we shouldn't show another BTC in parenthesis.

Recommended solution: in the item short and item templates, add logic so that if the currency code is BTC the btc value is not shown.

Or list it in Dogecoin

This is fixed/implemented?

Yes, this was resolved.


Plans to support MSSQL Always Encrypted feature?

Reviewing StackOverflow there is no major information about support.
https://stackoverflow.com/questions/65512096/always-encrypted-with-r2dbc
Any plan to support the Always Encrypted feature as JDBC does?
https://docs.microsoft.com/en-us/sql/connect/jdbc/using-always-encrypted-with-the-jdbc-driver?view=sql-server-ver15
Regards

We would be happy to accept a pull request that introduces support for Always Encrypted.


Add maze_generator, terminal_images and a dmm PR

I have read the CLA Document and I hereby sign the CLA.

I have read the CLA Document and I hereby sign the CLA

Thanks for catching this bug on our end

Sorry, that should be fixed now.

sorry for the merge conflicts. it should be fixed now

if you could fill the above form i can merge this pr

[x]  I have included my module name
[x]  I have published my module to Nest.land
[x]  I have published my module to Deno.land
[x]  I have included my name (first and/or last)
[x]  I have included my GitHub username
[x]  I have created an Arweave Wallet and included my wallet address
[x]  I confirm that this is my work and my module is not malicious>
[x]  I am a member of the Nest.land Discord
[x]  I am following @nestdotland on Twitter
[x]  I am following @deno_land on Twitter
[ ]  I am following @ArweaveTeam on Twitter
[ ]  I am following @peer2peerE on Twitter


Package 'cryptography' is missing from non-amd64 images in ansible-operator images > v1.26.0

Bug Report

What did you do?
Use cryptography in ppc64le and s290x images based on ansible-operator. Minimal example:
RUN pip3 install python-cryptography-fernet-wrapper

What did you expect to see?
Package cryptography can be used (python-cryptography-fernet-wrapper installs successfully like it did in <= 1.26.0)
What did you see instead? Under which circumstances?
Package 'cryptography' is missing in ppc64le and s390x images (didn't check arm64), causing an installation failure because cryptography is no longer preinstalled and the rust toolchain required to build cryptography from source is missing:
v1.26.1: Pulling from operator-framework/ansible-operator
Status: Downloaded newer image for quay.io/operator-framework/ansible-operator:v1.26.1
 ---> 3b5e14e0cb90
Step 3/14 : RUN pip3 install python-cryptography-fernet-wrapper
 ---> Running in 99ac64582c30
Defaulting to user installation because normal site-packages is not writeable
Collecting python-cryptography-fernet-wrapper
  Downloading python_cryptography_fernet_wrapper-1.0.4-py3-none-any.whl (14 kB)
Collecting cryptography
  Downloading cryptography-39.0.1.tar.gz (603 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Requirement already satisfied: cffi>=1.12 in /usr/local/lib64/python3.8/site-packages (from cryptography->python-cryptography-fernet-wrapper) (1.15.1)
Requirement already satisfied: pycparser in /usr/local/lib/python3.8/site-packages (from cffi>=1.12->cryptography->python-cryptography-fernet-wrapper) (2.21)
Building wheels for collected packages: cryptography
  Building wheel for cryptography (PEP 517): started
  Building wheel for cryptography (PEP 517): finished with status 'error'
Failed to build cryptography
  ERROR: Command errored out with exit status 1:
:
     =============================DEBUG ASSISTANCE=============================
      If you are seeing a compilation error please try the following steps to
      successfully install cryptography:
      1) Upgrade to the latest pip and try again. This will fix errors for most
         users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
      2) Read https://cryptography.io/en/latest/installation/ for specific
         instructions for your platform.
      3) Check our frequently asked questions for more information:
         https://cryptography.io/en/latest/faq/
      4) Ensure you have a recent Rust toolchain installed:
         https://cryptography.io/en/latest/installation/#rust
      Python: 3.8.13
      platform: Linux-5.4.0-135-generic-ppc64le-with-glibc2.17
      pip: n/a
      setuptools: 67.2.0
      setuptools_rust: 1.5.2
      rustc: n/a
      =============================DEBUG ASSISTANCE=============================
  error: can't find Rust compiler

Environment
Operator type:

/language ansible

Kubernetes cluster type:
OpenShift
$ operator-sdk version

v1.26.1
v1.27.0

$ go version (if language is Go)

$ kubectl version

Possible Solution

Additional context
Seemingly a regression caused by https://github.com/operator-framework/operator-sdk/pull/6269

I wrote a little test script to check the contents of ansible-operator images:
#site-packages!/usr/bin/env bash

version=${1:-v1.27.0}
outdir=${2:-site-packages}-${version}
image_name=quay.io/operator-framework/ansible-operator

function check() {
  local arch=$1
  docker pull --quiet --platform ${arch} ${image_name}:${version}
  docker tag ${image_name}:${version} ${image_name}:${version}-${arch}
  docker rmi ${image_name}:${version}
  mkdir -p ${outdir}/${arch}
  local container=${image_name##*/}-${arch}
  docker create --name ${container} ${image_name}:${version}-${arch}
  docker cp --quiet ${container}:/usr/local/lib64/python3.8/site-packages/. ${outdir}/${arch}
  docker rm ${container}
}

for arch in amd64 ppc64le s390x; do
  check ${arch}
done
ls -ld ${outdir}/*/crypto*

With v1.26.0 I get:
drwxrwxrwx 1 vagrant vagrant 4096 Dec  8 22:53 site-packages-v1.26.0/amd64/cryptography
drwxrwxrwx 1 vagrant vagrant 4096 Dec  8 22:53 site-packages-v1.26.0/amd64/cryptography-3.3.2.dist-info
drwxrwxrwx 1 vagrant vagrant 4096 Dec  8 23:05 site-packages-v1.26.0/ppc64le/cryptography
drwxrwxrwx 1 vagrant vagrant 4096 Dec  8 23:05 site-packages-v1.26.0/ppc64le/cryptography-3.3.2.dist-info
drwxrwxrwx 1 vagrant vagrant 4096 Dec  8 23:05 site-packages-v1.26.0/s390x/cryptography
drwxrwxrwx 1 vagrant vagrant 4096 Dec  8 23:05 site-packages-v1.26.0/s390x/cryptography-3.3.2.dist-info

but with v1.26.1 and v1.27.0 I only get this:
drwxrwxrwx 1 vagrant vagrant 4096 Feb  2 00:12 site-packages-v1.27.0/amd64/cryptography
drwxrwxrwx 1 vagrant vagrant 4096 Feb  2 00:12 site-packages-v1.27.0/amd64/cryptography-39.0.0.dist-info

@efussi Thanks for raising this issue! We updated our cryptography dependency in https://github.com/operator-framework/operator-sdk/pull/6269
I would have assumed our build process would have properly built and fetched the dependencies correctly but seems this wasn't the case... I will do some digging on this to try and figure out what went wrong
/assign

Interestingly enough there seems to have been an error that occurred when building the images for ppc64le (here) and s390x (here), although they say immediately after "Will try again!" and appears to succeed...
It looks like it was successful for arm64 (here).
After doing some digging, it looks like ppc64le and s390x prebuilt wheels don't exist as the maintainers of the project state they can't test these architectures (although it looks like ppc64le may be coming later this year?) :

https://github.com/pyca/cryptography/issues/6189
https://github.com/pyca/cryptography/issues/7723
https://github.com/pyca/cryptography/issues/7010
https://cryptography.io/en/latest/faq/#will-you-upload-wheels-for-my-non-x86-non-arm64-cpu-architecture

I think we have a few options here:

Revert https://github.com/operator-framework/operator-sdk/pull/6269 and release patch releases of each affected version (I didn't think this change would break stuff and was just a dependency update)
Update documentation around ansible operators for ppc64le and s390x architectures to mention that the cryptography package can no longer be provided prebuilt in the image for these architectures.
Update the base images to always build the cryptography package for all architectures (this means we are knowingly issuing the cryptography package in architectures that are unsupported and untested)

I will make sure to raise this issue for discussion in the next community issue triage meeting on Monday (02/13/2023) and gather some thoughts on what we should do.

Thanks @everettraven for your quick assessment!
Taking a step back, I am wondering why the missing cryptography package on ppc64le and s390x isn't causing unit test or runtime issues -- isn't cryptography a dependency that is needed at ansible-operator runtime? If yes, then I think (2) isn't a viable option. If not, may I ask why cryptography was included in the first place? If it was a courtesy to consumers, then imho option (2) is a breaking change and might require a major version bump according to semver (one could argue about this, if this courtesy wasn't officially documented :-) ).
Is it possible to do a variant of option (1), reverting just on those platforms where no prebuilt wheels are available?
To throw in one last data point: Anaconda provides cryptography for a variety of platforms, see https://anaconda.org/main/cryptography. But this is not readily installable in a plain pip environment (if at all). There might also be license considerations.

Taking a step back, I am wondering why the missing cryptography package on ppc64le and s390x isn't causing unit test or runtime issues -- isn't cryptography a dependency that is needed at ansible-operator runtime?

To be honest, I'm not sure. I'm not very familiar with all the python side of things for the ansible operator but am slowly learning a bit more at a time to try and help cover a bit of a knowledge gap we have in the currently active maintainers (all of our maintainers that were SMEs on the ansible side of things have since moved on to other things). My assumption is that if the unit tests aren't failing it isn't required but more so a courtesy as you mentioned.

If it was a courtesy to consumers, then imho option (2) is a breaking change and might require a major version bump according to semver (one could argue about this, if this courtesy wasn't officially documented :-) )

I 100% agree here and option 2 is my least favorite option but figured I would include it since it is one path that could be taken.

Is it possible to do a variant of option (1), reverting just on those platforms where no prebuilt wheels are available?

IIUC how we currently build the images it uses the same Dockerfile and uses docker buildx for building for multiple architectures. Unfortunately I think this limits us to either reverting for all platforms (option 1) or updating the build process for all platforms (essentially option 3). If we tried to do a bit of a hybrid revert I feel it would add way more complexity to our build/release process.

To throw in one last data point: Anaconda provides cryptography for a variety of platforms, see https://anaconda.org/main/cryptography. But this is not readily installable in a plain pip environment (if at all). There might also be license considerations.

That's good to know, thanks for sharing!

I ran another quick test using this small Dockerfile based on the ones in https://github.com/operator-framework/operator-sdk/blob/master/images/ansible-operator:
FROM registry.access.redhat.com/ubi8/ubi
RUN yum update -y \
 && yum install -y python38-devel python38-pip python38-setuptools \
 && pip3 install ansible~=2.9.15

This installs plain ansible and the build output is as follows:
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting ansible~=2.9.15
  Downloading https://files.pythonhosted.org/packages/5b/99/ce005d0314840e1a6eef34e0faf0ba4f7bccd8172b33cc84fee21afab7ad/ansible-2.9.27.tar.gz (14.8MB)
Collecting jinja2
  Downloading https://files.pythonhosted.org/packages/bc/c3/f068337a370801f372f2f8f6bad74a5c140f6fda3d9de154052708dd3c65/Jinja2-3.1.2-py3-none-any.whl (133kB)
Collecting PyYAML
  Downloading https://files.pythonhosted.org/packages/d7/42/7ad4b6d67a16229496d4f6e74201bdbebcf4bc1e87d5a70c9297d4961bd2/PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701kB)
Collecting cryptography
  Downloading https://files.pythonhosted.org/packages/bb/03/20b85e10571c919fd4862465c53ae40b6494fa7f82fd74131f401ce504f6/cryptography-39.0.1-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.2MB)
Collecting MarkupSafe>=2.0
  Downloading https://files.pythonhosted.org/packages/9d/80/8320f182d06a9b289b1a9f266f593feb91d3781c7e104bbe09e0c4c11439/MarkupSafe-2.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Collecting cffi>=1.12
  Downloading https://files.pythonhosted.org/packages/b7/8b/06f30caa03b5b3ac006de4f93478dbd0239e2a16566d81a106c322dc4f79/cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442kB)
Collecting pycparser
  Downloading https://files.pythonhosted.org/packages/62/d5/5f610ebe421e85889f2e55e33b7f9a6795bd982198517d912eb1c76e1a53/pycparser-2.21-py2.py3-none-any.whl (118kB)
Installing collected packages: MarkupSafe, jinja2, PyYAML, pycparser, cffi, cryptography, ansible
    Running setup.py install for ansible: started
    Running setup.py install for ansible: finished with status 'done'
Successfully installed MarkupSafe-2.1.2 PyYAML-6.0 ansible-2.9.27 cffi-1.15.1 cryptography-39.0.1 jinja2-3.1.2 pycparser-2.21

We see that this installs cryptography even though cryptography is not explicitly requested. That is, ansible itself must declare it as a dependency package. Could be indirect, though. Trying to get more details by explicitly installing cryptography first:
FROM registry.access.redhat.com/ubi8/ubi
RUN yum update -y \
 && yum install -y python38-devel python38-pip python38-setuptools \
 && pip3 install cryptography==3.3.2 \
 && pip3 install ansible~=2.9.15

We get this output:
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting cryptography==3.3.2
  Downloading https://files.pythonhosted.org/packages/c6/d1/800ec785c9e66cc6d0ac587bd666eb22f7b2ff6c150e053d35881acd2f57/cryptography-3.3.2-cp36-abi3-manylinux2010_x86_64.whl (2.6MB)
Collecting six>=1.4.1
  Downloading https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl
Collecting cffi>=1.12
  Downloading https://files.pythonhosted.org/packages/b7/8b/06f30caa03b5b3ac006de4f93478dbd0239e2a16566d81a106c322dc4f79/cffi-1.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (442kB)
Collecting pycparser
  Downloading https://files.pythonhosted.org/packages/62/d5/5f610ebe421e85889f2e55e33b7f9a6795bd982198517d912eb1c76e1a53/pycparser-2.21-py2.py3-none-any.whl (118kB)
Installing collected packages: six, pycparser, cffi, cryptography
Successfully installed cffi-1.15.1 cryptography-3.3.2 pycparser-2.21 six-1.16.0
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting ansible~=2.9.15
  Downloading https://files.pythonhosted.org/packages/5b/99/ce005d0314840e1a6eef34e0faf0ba4f7bccd8172b33cc84fee21afab7ad/ansible-2.9.27.tar.gz (14.8MB)
Collecting jinja2
  Downloading https://files.pythonhosted.org/packages/bc/c3/f068337a370801f372f2f8f6bad74a5c140f6fda3d9de154052708dd3c65/Jinja2-3.1.2-py3-none-any.whl (133kB)
Collecting PyYAML
  Downloading https://files.pythonhosted.org/packages/d7/42/7ad4b6d67a16229496d4f6e74201bdbebcf4bc1e87d5a70c9297d4961bd2/PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701kB)
Requirement already satisfied: cryptography in /usr/local/lib64/python3.8/site-packages (from ansible~=2.9.15) (3.3.2)
Collecting MarkupSafe>=2.0
  Downloading https://files.pythonhosted.org/packages/9d/80/8320f182d06a9b289b1a9f266f593feb91d3781c7e104bbe09e0c4c11439/MarkupSafe-2.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Requirement already satisfied: cffi>=1.12 in /usr/local/lib64/python3.8/site-packages (from cryptography->ansible~=2.9.15) (1.15.1)
Requirement already satisfied: six>=1.4.1 in /usr/local/lib/python3.8/site-packages (from cryptography->ansible~=2.9.15) (1.16.0)
Requirement already satisfied: pycparser in /usr/local/lib/python3.8/site-packages (from cffi>=1.12->cryptography->ansible~=2.9.15) (2.21)
Installing collected packages: MarkupSafe, jinja2, PyYAML, ansible
    Running setup.py install for ansible: started
    Running setup.py install for ansible: finished with status 'done'
Successfully installed MarkupSafe-2.1.2 PyYAML-6.0 ansible-2.9.27 jinja2-3.1.2

The relevant line is
Requirement already satisfied: cryptography in /usr/local/lib64/python3.8/site-packages (from ansible~=2.9.15) (3.3.2)

So cryptography is a direct dependency of ansible which means option (2) might (partially) break the ppc64le and s390x ansible-operator images because a dependency of ansible itself is missing.

I think the Pipfile is (also) a flattened dependency tree for the purpose of pinning versions.

Going to bring this up at the ansible community call next week, will report back.

We discussed this in the ansible community meeting and it was recommended that we first try updating our pip version that we install when building the images from 21.1.0 to the latest (currently 23.0.1).

@everettraven  Do we have any updates on this ?

@ppadashe This was closed by #6333 which updated our image build process to ensure that all images produced contain the cryptography package. This fix should be included in the next release.

fwiw, https://github.com/operator-framework/operator-sdk/pull/6333 was reverted through https://github.com/operator-framework/operator-sdk/pull/6348, see also https://github.com/operator-framework/operator-sdk/issues/6342#issuecomment-1464038685.


replay is not the same as the captured image

Description
I capture a game, save rdc file, but replay is not the same as the captured image
Steps to reproduce
open rdc file,  I will email you the rdc file, because rdc file is too big.  nearly 22M.
Environment

RenderDoc version: RenderDoc v1.7 @ dd69f8642, 2020-04-01
Operating System: Android 9
Graphics API: OpenGL ES 3.2

For future information, it's rude to send 40MB as a direct email attachment. You should upload such files to a file sharing website and share a link instead. It's especially rude to send two identical 40MB emails with duplicated attachments.
The RenderDoc version you have listed is not correct, that build listed is not from today 2020-04-01 it's from 2020-02-12. Lying about what version you are using will make it hard to determine the real bug. Your build is an unstable nightly build from several months ago - please test at least with the latest stable version v1.7, optionally with a recent nightly build if you want to check to see if the bug has been fixed since v1.7.
The captures you sent seem to be using invalid programs which have been created but not linked and have no shaders attached. If this is a renderdoc bug then it's something going wrong at capture time. Please ensure that your program is not invoking any undefined GL behaviour, if there are no errors reported by the driver and the latest stable version of RenderDoc still shows the problem I'd need to investigate the capture time to see more about what's going wrong. Would you be able to share the application with me privately?

The RenderDoc reports that glProgramBinaryOES and glGetProgramBinaryOES are not supported. We wrote two functions based on glProgrameBinary. If we don't do this, the game will crash. Is this correct?
发件人: Baldur Karlsson<EMAIL_ADDRESS>发送时间: 2020年4月1日 22:57
收件人: baldurk/renderdoc
抄送: zhouhuamin; Author
主题: Re: [baldurk/renderdoc] replay is not the same as the captured image (#1807)
For future information, it's rude to send 40MB as a direct email attachment. You should upload such files to a file sharing website and share a link instead. It's especially rude to send two identical 40MB emails with duplicated attachments.
The RenderDoc version you have listed is not correct, that build listed is not from today 2020-04-01 it's from 2020-02-12. Lying about what version you are using will make it hard to determine the real bug. Your build is an unstable nightly build from several months ago - please test at least with the latest stable version v1.7, optionally with a recent nightly build if you want to check to see if the bug has been fixed since v1.7.
The captures you sent seem to be using invalid programs which have been created but not linked and have no shaders attached. If this is a renderdoc bug then it's something going wrong at capture time. Please ensure that your program is not invoking any undefined GL behaviour, if there are no errors reported by the driver and the latest stable version of RenderDoc still shows the problem I'd need to investigate the capture time to see more about what's going wrong. Would you be able to share the application with me privately?
—
You are receiving this because you authored the thread.
Reply to this email directly, view https://github.com/baldurk/renderdoc/issues/1807#issuecomment-607298719  it on GitHub, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH5IROHA2UWPZSTJMBAHSMDRKNI2DANCNFSM4LY2Z7FQ .  https://github.com/notifications/beacon/AH5IRODVOIIEZTETL7JKFDDRKNI2DA5CNFSM4LY2Z7F2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEQZKJHY.gif

RenderDoc deliberately forces applications to upload shader source instead of program binaries, since program binaries are driver-specific and opaque. GL_OES_get_program_binary is not reported as supported so it's illegal to use glProgramBinaryOES, and the glProgramBinary variant from GLES 3.0 is emulated to act as if the uploaded binary is incompatible.
If your application uses glProgramBinary it must then check that the program successfully linked. If a binary is not supported by a driver then the link status will be set to false and you should fall back to uploading shader source. When running under RenderDoc binaries will always be reported as unsupported, but this can happen as well with any driver if the driver's binary format changes or the binary you are uploading was obtained on an incompatible driver.
If your application is using glProgramBinary and then not checking for success and uploading shader source as a fallback, then it is invoking undefined behaviour and this would explain the problem.

Hi Baldur , Can you easily connect my computer remotely for debugging?

No I can't do that at all. Did you make sure that your application has a fallback path that doesn't use glProgramBinary or else correctly handles its failure? Are you able to share a reproducible test case with me?

This game is quite special. You need to fill in the authentication information. You may not use it?
Can you use the free TeamViewer software to connect to my computer?  Is it inconvenient for you to install the free TeamViewer software? I can send you the TeamViewer software version

If it is convenient, would you come over and take a look? See if you can connect to my computer?

If you can't share the program with me then that's fine, but I will not do any remote debugging for many reasons.
You didn't answer my question - is your program correctly handling glProgramBinary failures?

I believe my program has been processed correctly, you can not debug, you can open the latest version of RenderDoc tools and try to grab a look, you may find many problems? Then see if there is a better way? What do you think?

I have opened the RenderDoc V1.7 program on my computer and connected to the phone. You can check the log information. You do n’t need to debug.

I'm afraid I don't understand what you're asking. If the problem is the same as before then the log won't show me enough information, but given what you've said before the problem was most likely that you were using glProgramBinary and not handling its failure correctly.
If you've fixed your program to not use glProgramBinary can you send another capture to me? Again as I said before do not attach it to the email, use a file sharing service to email me a link.

I use the latest version of RenderDoc V1.7  ：RenderDoc_1.7_64.msi, try to grab again, the game crashes after opening the game, so the most direct way is to let you connect to my computer and see if the log information Useful for you.

I will not connect to your computer. Please stop bringing that up as it is not helpful. If you continue to do so I will close this issue.
If the program is crashing that's a different problem than the one that was originally described. It wasn't crashing previously as you were able to get a capture, what has changed since then? Can you give me any details about the crash such as where the callstack indicates it's crashing?

I checked the log information with adb logcat and found that the program with process number 25507 hung up somehow and there was no stack information.

Line 2278: 04-03 20:50:09.072 25507 25718 D native(MiLinkSDK)(misdk): Thread-3=>jni/base/../../kernel/TcpConnection.cpp:tid-25718 begin to select... sockfd:83 pipefd:81 maxfd:83

Line 2279: 04-03 20:50:09.073 25507 25831 I AudioTrack: AUDIO_OUTPUT_FLAG_FAST successful; frameCount 0 -> 1536

Line 2284: 04-03 20:50:09.077 21178 21370 D Avrcp_ext: AudioManager Player: ID:23647 -- type:OpenSL ES AudioPlayer (Buffer Queue) -- u/pid:10409/25507 -- state:started -- attr:AudioAttributes: usage=USAGE_MEDIA content=CONTENT_TYPE_UNKNOWN flags=0x0 tags= bundle=null

Line 2318: 04-03 20:50:09.083   732  3432 E AudioFlinger: open /proc/25507/cmdline error

Line 2323: 04-03 20:50:09.086 25507 25831 I AudioManager: In isBluetoothScoOn(), calling application: com.netease.blqx.mi

Line 2381: 04-03 20:50:09.111 25507 25611 I netease.blqx.m: ProcessProfilingInfo new_methods=302 is saved saved_to_disk=1 resolve_classes_delay=8000

Line 2390: 04-03 20:50:09.507 25507 25780 W GaidUtils: Google Play services is missing.

Line 2392: 04-03 20:50:09.581 25507 25507 W SdkDouYinShareAward: unknown function:setLanguage

Line 2392: 04-03 20:50:09.581 25507 25507 W SdkDouYinShareAward: unknown function:setLanguage

Line 2393: 04-03 20:50:09.581 25507 25806 E UniSDK AmapLocation: invalid method tag

Line 2427: 04-03 20:50:10.028 25507 25782 I trace   : StorageToFileProxy [write] info=udid(可提供给平台方协助查询):4e88c054963a3f1b

Line 2434: 04-03 20:50:10.362 25507 25831 I AudioManager: In isBluetoothScoOn(), calling application: com.netease.blqx.mi

Line 2436: 04-03 20:50:10.407 25507 25780 I NeoX    : getRealSize p.x=2160 ,p.y=1080

Line 2437: 04-03 20:50:10.410 25507 25780 I NeoX    : getRealSize p.x=2160 ,p.y=1080

Line 2440: 04-03 20:50:10.712 25507 25780 W GaidUtils: Google Play services is missing.

 I asked before - what changed in your code between you being able to capture and not? That narrows down the area the problem could be.
The logcat looks like your program has a custom crash handler of some kind. Either use that to get more information about what is failing, or else disable it so that the default crash handler will print a callstack and tell you where the crash is happening.

Hello, finally got the rdc file, can you see what the problem is? The file blqx-003.rdc is placed in the following link: https://github.com/zhouhuamin/testcase
Version v1.8 (built from fcb82ae7)

The capture linked is corrupted and cannot be opened, it contains some mismatched chunk data that leads to a failure to load. Please confirm that you see the same thing and can't open the capture at all, since there's no way this could be successfully opened on the build you mentioned.
Was this captured on a completely unmodified nightly build of RenderDoc, or was this captured from a customised version of RenderDoc with source changes? If this was captured from an unmodified nightly build then I can't explain what went so wrong unless your application is trampling on RenderDoc's memory. This is another different bug, not related to what you originally described, and if it's a real bug I would definitely need to be able to debug the application at capture time. Unlike the original problem which could theoretically be debugged by running builds with extra logging.

Hello,  got the rdc file, can you see what the problem is? The file blqx-004.rdc is placed in the following link: https://github.com/zhouhuamin/testcase
Version v1.8 (built from 1d64fffd)

I'm not going to investigate this any more unless you give clear and specific answers to the questions I've already asked. By not answering when I try to help you are wasting my time either deliberately or through carelessness, and I'm not going to continue with a guessing game where I can't get enough information to find the problem.

The last file blqx-003.rdc was generated by the customized RenderDoc. The customized code may have problems. It has been deleted, and the latest RenderDoc code has been pulled back for testing. The problem at the beginning can be entered. it is good

OK thank you for clarifying that. In future absolutely do not make any local changes to RenderDoc while I'm trying to investigate this to help you, or else I cannot tell what problems are real bugs, what problems are from your program, and what problems are from your changes to RenderDoc.
Have you confirmed that your program is not using glProgramBinary in any way?

In the program, glProgramBinary is used, and the source shader used after the failure is relinked. The program seems to be multi-threaded. Do you have any plans for multi-threaded support?

Multi-threading is supported on most APIs, OpenGL's threading support is terrible so it's not supported if you're rendering from multiple threads simultaneously. I recently added a hack that may work in some scenarios but it's not guaranteed.
That wouldn't explain the capture that still has programs that don't have any shaders attached and haven't been linked.
Since you can't share your application, can you download and run this build with extra logging. Take a capture with it and share both the capture as well as the complete diagnostic log (from Help -> View diagnostic log -> Save).

This version crash when I launch my application. my version have a change about RenderDoc, I followed my changes in #1758. I don't know which extension caused this problem.
Attach the log information of this version https://github.com/zhouhuamin/testcase/20200409.log, I hope you will give me a test version after fixing the bug, grab the rdc file and log file and send it to you.

You're still talking about running versions with some unknown local changes even after I specifically asked you not to. You are wasting my time through this process and I'm not going to continue this investigation further trying endlessly to obtain your co-operation.
I'm willing to help guide an investigation and work together to gather the necessary information as I understand that not everyone can freely share their programs or their captures. I've done that successfully with many other people. You have consistently omitted important information, avoided answering questions, run custom builds, and changed the discussion of what the bug actually is more than once. You have exhausted my willingness to work with you to determine what the problem is. In future if you do not provide a precise and clear reproducible test case for any issues I will close them, as I won't be drawn into this again.
To anyone else please don't take this as an indication that I'm unwilling to help with bugs. As long as you are willing to co-operate and help me to help you, there is no problem.


Categories: Alphabetical order

Shouldn't this list be in Alpha order?

/cc @holly-cummins

I must admit that some are not that comprehensible (eg.

HA i totally agree that is why i was looking at what Category to put something in I had no idea what some of these meant.

OK its for Change Data Capture technology but that really should just be "Data" category.

They're sorted by popularity, so that the obscure ones are at the bottom. Within that, I think they're alphabetical.
However, @gastaldi's point is also a good one. We're seeing a proliferation of categories, and it's supposed to be - unlike keywords - a relatively small set. Anything that's not a category in the list on http://code.quarkus.io won't be searchable by that category on code.quarkus. Fixing that will need a combination of hand-curation when we spot outliers, and also more help when people create extensions to show them the current list and invite them to select something from it unless they have a really good reason not to.

This issue should be over on https://github.com/quarkusio/extensions, I think. I should put a link on the extensions site to make raising issues and changing source easier.

Ok I will close this one.


initial api changes implementing #54

This looks good. I'll give it a go on my main project and report back in an hour or so.

There's a small problem with DefaultDropTarget#getDropEffects. Since it's the same for both legacy and new-style drop target, the first parameter will either be allowedEffects or component.
It's just a single method so we can put a temporary hack there. For example, we can make it LegacyDefaultDropTarget, add a new one with modern signature and switch between them based on the legacy flag.


Minor error in example

Very minor Issue, quickly fixable.
docs/getting_started.rst in line 66:
ret = pypret.Retriever(pnps, "copra", verbose=True, max_iterations=300)
The argument max_iterations has no effect.
According to the documentation in pypret/retrieval/retriever.py in line 291 it should be maxiter instead.

Thanks! Fixed in https://github.com/ncgeib/pypret/commit/cf4068ed8bcc2bc7e35e17db9ea6da0c9c99538b


Cleanup

Codecov Report
Patch coverage has no change and project coverage change: +6.01 :tada:

Comparison is base (d04a510) 87.68% compared to head (1c91c0a) 93.70%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #76      +/-   ##
==========================================
+ Coverage   87.68%   93.70%   +6.01%
==========================================
  Files           8        6       -2
  Lines         138      127      -11
  Branches       11       10       -1
==========================================
- Hits          121      119       -2
+ Misses         11        2       -9
  Partials        6        6

Flag
Coverage Δ

unittests
93.70% <ø> (+6.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Cannot open browser, asked to update Playwright from 1.30 to 1.33

Describe the bug
When running on my CI pipeline using the latest version docker (FROM marketsquare/robotframework-browser/rfbrowser-stable:16.1), I was told that the Playwright version needs to be updated from 1.30 to 1.33.
This issue does not happen when I install the RF Browser in my local machine using pip. So it's exclusively on the docker version.
I have tried adding Playwright installation on my CI script using npx playwright install before running my test, which successfully install Playwright 1.33, but it did not solve the issue.
Anyone can help me solve this?
Screenshots
The error shown on the test log:

The console log of Playwright 1.33 installation being initiated before the test:

I am astonished that there is no response nor other person who commented on this even after two weeks since its post date.
Am I the only one using the latest Browser version? Or does everyone else not encounter the same issue?

Hi @frirasyidi
This sometimes happen, on Open-Source projects that are totally free to use.
Sometimes we just have no time to answer or no clue why this is not working on your maching.
Please try it again with the newest docker image.

@Snooz82 thanks for your reply, most appreciated.
I fully understand the limitation of an open source project, just amazed that with the popularity of the project, even after 2 months, nobody else seemed to face the same issue as I do.
So, closing the issue now.


Blocks are aligning on opposite ends

I'm interested in this technique, but I can't seem to get the blocks to be left aligned when there is a strange number of them.
I just added 2 image elements to your demo and this is the result:

Do you happen to know how to fix this?

Changing the pseudo selector to from width: 100% to auto kinda works but breaks the gutters.

I'd just give last-of-type { margin-left: 30px; } but what happens if there are only 4 blocks in the grid? Then the 4th one would be pushed off the left side.
I'd do nth-of-type(3n) { margin-right: 30px; } but what if I want my 2nd block to be 2/3 instead of 1/3? Then it breaks the nth stuff.

I’m out all day today so can’t really give an elaborated response. But this has to do with having 5 elements while there is room for 6. Check figure 5 of this article, that explains the problem and a possible solution rather well!
To be honest, I would just use flexbox. Browser support is pretty good and you can use inline-blocks as fallback for when flexbox is not supported.

Adding empty elements isn't ideal and flexbox suffers from the same problem. https://www.google.com/search?q=flexbox+space+between+last+row

Found a solution you might like, http://codepen.io/anon/pen/zrVOYj?editors=0100
Basically adds a right margin to the last element when needed. In the Pen it's nth-child(3n) because of the <h1>, but otherwise it would be nth-child(3n - 1).
.image:nth-child(3n):last-child {
  @include grid-append(3, 9);
}

The logic is similar to the empty elements, without the empty elements. You need n- 2 selectors where n is the number of columns (in this case 3 so that was easy).
Example with 4 columns, http://codepen.io/anon/pen/mVZbeb?editors=0100
&:nth-child(4n - 1):last-child {
  @include grid-append(3);
}
&:nth-child(4n - 2):last-child {
  @include grid-append(6);
}

Yeah, the unfortunate thing is we would have to know how many elements go in a row for nth to work. What if I want an element that is 1/3 and an element that is 2/3 in the first row, and just three 1/3 elements in the second row? nth stuff gets pretty hairy there.
I'm calling this "THE Problem" nowadays.
Justify (whether inline-block or flexbox) seems to be the only way to get elements to space out correctly without having to use margins (that would have to be wrapped in a superfluous element).
The unfortunate part is that last row... It gets justified and elements get spaced out.

...or if you make the :after { width: auto; } it pushes all the last row elements to the left:

...and when you try to add a margin to the last row of elements, you fall back into the "having to know how many elements" trap.
Here's a bare bone pen if you're interested in fighting with it, but I've been fighting with it for several days now.
http://codepen.io/corysimmons/pen/OMYjaP?editors=1100
My only hope is there is some obscure CSS property, or some crrrrrazy combination of nth selectors that will make this kind of stuff possible.
It'd be so nice to have the flexibility of Bootstrap-like grids without the extra markup.

That complicates things... Not sure if this is solvable in just CSS. If you know on the server how many columns you need to fill you can set a class that applies margin-right though. That would be the option I'd go for.
By the way with :after { width: auto; } you might as well remove the :after pseudo.

Yeah but the rule is we can't know how many or the size of the elements they're putting in the container. :)

By the way with :after { width: auto; } you might as well remove the :after pseudo.

Yup. :+1:

Sorry about not responding anymore. I actually have no clue how to solve your issue.

Not a problem. I'm pretty confident it's unsolvable unless there is some weird CSS rule I haven't thought of using yet. I've fought with it for years. =)


revise internal helper _is_perturbed to _maybe_perturbed, fix logic

@patrick-kidger FYI (and feel free to leave a review!)
fixes #9567

As a fix for correctness this LGTM!
Since JAX sees a whole program then in principle it should be able to do this precisely without heuristics though. Correctness is good but correctness and efficiency is better.
I think I can see how this should be doable for all practical use cases. The fix is to adjust custom_jvp and custom_vjp and deprecate closure_convert. AFAIK it's only ever needed around these anyway.
We clearly can't have a non-heuristic closure_convert that works in arbitrary contexts, as we need to defer binary classification until autodiff time and I don't think there's any way to encode things appropriately in the jaxprs before then. But since we only ever (a) use it around custom autodiff, and (b) only ever use the hoisted constants in one way (as args back into the provided function), then we can handle everything at once just by pushing the whole lot into the custom autodiff operations.
Have a look at Equinox's filter_custom_vjp. This currently uses similar floating-point heuristics as it's constrained by its use of jac.custom_vjp, but it demonstrates an API in which nondifferentiability/differentiablility is a detail that can be hidden from the end user.
(I recognise this is a lot of work, and invites similar Equinox-filtering-style changes elsewhere, and that something like this is undoubtedly low priority.)

Thanks for taking a look @patrick-kidger !

Since JAX sees a whole program then in principle it should be able to do this precisely without heuristics though.

We clearly can't have a non-heuristic closure_convert that works in arbitrary contexts, as we need to defer binary classification until autodiff time and I don't think there's any way to encode things appropriately in the jaxprs before then.

I may not be following but aren't these two statements contradictions? ("should be able to do this precisely without heuristics" and "can't have a non-heuristic closure_convert") What's "this" mean in the first sentence?
Actually it's not quite true that JAX sees the whole program. Someone could always use jax.make_jaxpr to stage out a function, at which point the _maybe_perturbed function would be called, and later when interpreting that jaxpr differentiation may or may not be involved. For that reason we simply can't know for sure if differentiation is going to eventually be applied. (If we rule out the use of jax.make_jaxpr and jaxpr interpreters, and only think about e.g. higher-order primitives like jax.lax.scan, then indeed I believe we could have a complete and sound check.)

The fix is to adjust custom_jvp and custom_vjp and deprecate closure_convert.

Have a look at Equinox's filter_custom_vjp.

Could you say more about the fix you have in mind, and/or provide some code links? I can't tell what you mean :)
I looked at the docstring for filter_custom_vjp but since it also seems to work on heuristics (as you say) I don't see what the diff is.

I may not be following but aren't these two statements contradictions? ("should be able to do this precisely without heuristics" and "can't have a non-heuristic closure_convert") What's "this" mean in the first sentence?

Right, sorry for being unclear. "should be able to do this precisely without heuristics" -> 10,000 foot view that JAX should be able to figure out which arrays participate in autodiff, as it sees the whole program. [Yes it does! See my next point.]
"can't have a non-heuristic closure_convert" is then digging into the implementation details of how that might happen; at this point I'm making the comment that closure_convert is a bad abstraction because what is or isn't participating in autodiff isn't known at the time that it is parsed. As you say:

Someone could always use jax.make_jaxpr to stage out a function, at which point the _maybe_perturbed function would be called, and later when interpreting that jaxpr differentiation may or may not be involved.

which is precisely my point as well. _maybe_perturbed is being called at the wrong time! So I agree with you ;)

Could you say more about the fix you have in mind, and/or provide some code links? I can't tell what you mean :)
I looked at the docstring for filter_custom_vjp but since it also seems to work on heuristics (as you say) I don't see what the diff is.

So the problem is that _maybe_perturbed is being called at the wrong time. It's being called at first-trace-time; it should be called at autodiff time. Given the ubiquity of closure_convert around custom autodiff, I'm proposing that filtering differentiable from nondifferentiable be something that's handled inside jax.custom_jvp and jax.custom_vjp, rather than having closure_convert as a standalone function.
[Side note! closure_convert only filters differentiable from nondifferentiable for implicit args -- but around custom autodiff we need to the same thing for explicit args as well. Once again closure_convert isn't quite the right abstraction; using just the facilities of JAX we'd have to define a new function closing over the explicit args and then closure_convert that in order to filter all arguments to the function.]
The implementation of filter_custom_vjp indeed has an implementation that depends on heuristics. I'm offering it just as an example of an API that doesn't have to depend on heuristics. Imagine filter_custom_vjp was actually a primitive that got preserved in the jaxpr across transformations (like custom_vjp already is). We could then use it to perform filtering at autodiff time rather than first-trace-time.
By the way, notice how the word "filter" keeps coming up in the above discussion. Filtering is just a really useful thing to be doing around transformations, and so much so that Equinox is built around this idea. (Along with the parameterised-functions-as-pytrees idea.)