Reorder arguments in decorator methods

I have a few decorator methods that have various behaviours including rate limiting, timing, and of course, circuit breaking! I apply these in a chain, as per the decorator pattern.
CircuitBreaker.decorateSupplier has a type signature Supplier<T> decorateSupplier(Supplier<T> supplier, CircuitBreaker circuitBreaker), so chaining multiple decorator methods looks like:
RateLimiter.decorateSupplier(rateLimitConfig,
    CircuitBreaker.decorateSupplier(
        Stopwatch.decorateSupplier(stopwatchConfig,
            mySupplier()), circuitBreaker));

Notice that the CircuitBreaker instance is far from where it is applied in the chain, and looks out of place.
If CircuitBreaker.decorateSupplier had a type signature of Supplier<T> decorateSupplier(CircuitBreaker circuitBreaker, Supplier<T> supplier), then the same code would look like:
RateLimiter.decorateSupplier(rateLimitConfig,
    CircuitBreaker.decorateSupplier(circuitBreaker
        Stopwatch.decorateSupplier(stopwatchConfig,
            mySupplier()));

Now it's easier to associate the CircuitBreaker instance with the call to CircuitBreaker.decorateSupplier

Hi,
indeed. Looks much better that way.
If you use the library, I would love to get your GitHub Star.

@tomfitzhenry Did you publish your RateLimiter somewhere?

@RobWin @tomfitzhenry I've request limits implementation inspired by your library here . But it is on early stages.
There is:

Contracts and interfaces similar to your javaslang-circuitbreaker library
Stupid simple implementation of request limiter
100% tests coverage

To make it fully usable I plan to add:

Add full javadoc
Write README
Create example directory
Write JMH benchmark for implementation of request limiter

I know that current implementation of request limiter is too straightforward and possibly can have some performance problems but I decided to go with it at first. And after JMH benchmark available I will be able rewrite something based on ring buffer of timestamps.

I added the new signatures and deprecated the old signatures in the upcoming v0.7.0

Would you like to contribute such a feature to javaslang-circuitbreaker?

@tomfitzhenry Yep. I've already look through this and several other implementations. And have overall understanding of pros and cons of different algorithms. But I don't sure that such feature should be added to javaslang-circuitbreaker because it is already concise and beautiful single responsibility library.
On my current project I've created abstraction called Gateway. Every API call is proxied through this Gateway and it is responsible for invocation of circuitBreaker, requestLimit and/or metrics.
Like that:
public class GatewayInterceptor implements MethodInterceptor {
    private final Logger log = LoggerFactory.getLogger(GatewayInterceptor.class);
    private final ApiType apiType;
    private final Try.CheckedFunction<MethodInvocation, Object> gatewayInvoke;

    public GatewayInterceptor(ApiType apiType, RequestLimit requestLimit, CircuitBreaker circuitBreaker) {
        this.apiType = apiType;
        Try.CheckedFunction<MethodInvocation, Object> circuitBreakerInvoke = CircuitBreaker.decorateCheckedFunction(
            MethodInvocation::proceed,
            circuitBreaker
        );
        Try.CheckedFunction<MethodInvocation, Object> requestLimitInvoke = RequestLimit.decorateCheckedFunction(
            circuitBreakerInvoke,
            requestLimit
        );
        gatewayInvoke = requestLimitInvoke;
    }

    @Override public Object invoke(final MethodInvocation invocation) throws Throwable {
        Object result = Try.success(invocation)
            .mapTry(gatewayInvoke)
            .onFailure(throwable -> log.error("Error in invocation of {} API: {}", apiType, throwable.getMessage()))
            .get();
        return result;
    }
}

@tomfitzhenry What do you think about it? Should we javaslang-circuitbreaker or create additional library that will compose  few other libraries?

Let's move the discussion to #12. Could you copy your comment into this issue?
Besides that, @tomfitzhenry is not the maintainer, but it's me :)
If a RateLimiter implementation is lightweight in the sense that we don't need any other 3rd party libraries, we can add it to javaslang-circuitbreaker. A RateLimiter is also some sort of CircuitBreaker. A RateLimiter can forbid further calls, if a rate limit has exceeded.
The retry functionality is also not related to CircuitBreakers.


MSUnmerged: Imlplement the actual deletion logic fro unmerged files

Impact of the new feature
MSUnmerged
Is your feature request related to a problem? Please describe.
Once we have the proper set of unmerged files per RSE established - issue: #10657, we should implement the execution of the actual deletion command per RSE.
Describe the solution you'd like
This deletion should be implemented in 3 steps:

Fetch the default deletion interface for the RSE from Rucio and fill it in the MSUnmergedRSE['delInterface'] field

In [1]: from rucio.client import Client

In [2]: rcl = Client()

In [3]: rcl.lfns2pfns(rse='T2_US_Wisconsin',
   ...:               lfns=['cms://store/unmerged/Run2016B/JetHT/MINIAOD/ver2_HIPM_UL2016_MiniAODv2-v2/140000/388E3DEF-9F15-D04C-B582-7DD036D9DD33.root'],
   ...:               scheme='gsiftp',
   ...:               operation='delete')
Out[3]: {'cms://store/unmerged/Run2016B/JetHT/MINIAOD/ver2_HIPM_UL2016_MiniAODv2-v2/140000/388E3DEF-9F15-D04C-B582-7DD036D9DD33.root': 'gsiftp://cms-lvs-gridftp.hep.wisc.edu:2811/hdfs/store/unmerged/Run2016B/JetHT/MINIAOD/ver2_HIPM_UL2016_MiniAODv2-v2/140000/388E3DEF-9F15-D04C-B582-7DD036D9DD33.root'}

Try to aggregate as much as possible files in a single deletion execution
Execute the deletions and collect counters

Describe alternatives you've considered
Additional context

While working on the previous Issue #10657 #10666, I figured out a possibly risky situation regarding the information coming from Rucio Consistency Monitor about some sites in error state. I am putting the information here, so that I may follow in this issue and still merge the previous one.
I was trying to fetch list of unmerged files for T2_DE_DESY but the result was 0. If checked at https://cmsweb.cern.ch/rucioconmon/unmerged/stats  the RSE is in error. There is already some protection in the code regarding RSE status [1], but currently it is relying on the RSEs status filed to be set explicitly to error, while in the case with T2_DE_DESY the RSE status is set to done:
In [24]: rseStats['T2_DE_DESY']
Out[24]:
{'scanner': {'type': 'xrootd', 'version': '1.3'},
 'server_root': '/store/',
 'server': 't2-cms-xrootd01.desy.de:1094',
 'start_time':<PHONE_NUMBER>.7156065,
 'end_time':<PHONE_NUMBER>.356299,
 'status': 'done',
 'error': 'status: 54 [ERROR] Server responded with an error: [3011] Unable to locate /store/unmerged; no such file or directory',
 'root_failed': True,
 'failed_subdirectories': 0,
 'files': 0,
 'directories': 0,
 'empty_directories': 0}

So the logical question is: Which is the full set of markers/flags we need to check in the unmerged/stats interface, so that we may consider the information coming from unmerged/files interface as reliable?
[1]
https://github.com/dmwm/WMCore/blob/d66237302581b8d6b6a41295d82bb64c95a5296c/src/python/WMCore/MicroService/MSUnmerged/MSUnmerged.py#L229-L231

But is this really a concern?  If MSUmmerged receives an empty list from the consistency checking, it won't clean anything up, right?  That won't cause any issue in the short term. It's much better than cleaning up a file that's actively in use. I think we should rely on the consistency checking service and monitoring of that service to detect and correct it's own error condition. MSUnmerged can just focus on cleaning whatever files it gets sent. What do you think?

Hi  @klannon Thanks.
Well, looking things that way, yes they do sound logical, but there are few more use cases I need to cover.

First - the checks about data reliability I do in a very early step so I can avoid even proceeding such RSEs, because we want to save CPU cycles and redundant calls to external systems for sure, but also
Second - because if we ignore the case of an RSE with 0 file counter and broken status and do not evacuate it from the functional pipeline early, then at the end of the line it's counter would be checked and this RSE would be considered clean. This is how it will enter the service counters in the report. And this is already a well hidden error, about which we will forget and in a month from now some of us will have to go answer the question "Why a broken RSE was not reported in the logs" and start counting RSE by RSE until we (eventually) find the problematic one.


fix(picker): spinner position in RTL direction

Description
Aims to fix https://github.com/adobe/spectrum-css/issues/2566
How and where has this been tested?
Please tag yourself on the tests you've marked complete to confirm the tests have been run by someone other than the author.
Validation steps

Open the docs for the picker component (where is storybook deployed? 🙈 ):

[x] is-loading LTR - margin is applied between the spinner and the label
[x] is-loading RTL - margin is applied between the spinner and the label
[x]  is-invalid LTR - margin is applied between the spinner and the icon
[x] is-invalid RTL - margin is applied between the spinner and the icon
[x] not invalid - label has no margin end
[x] not loading - label has no margin end

Regression testing
Validate:

The documentation pages for at least two other components are still loading, including:

[x] The pages render correctly, are accessible, and are responsive.

If components have been modified, VRTs have been run on this branch:

[ ] VRTs have been run and looked at.
[ ] Any VRT changes have been accepted (by reviewer and/or PR author), or there are no changes.

Screenshots

To-do list

[x] I have read the contribution guidelines.
[x] I have updated relevant storybook stories and templates.
[ ] I have tested these changes in Windows High Contrast mode.
[x] If my change impacts other components, I have tested to make sure they don't break.
[x] If my change impacts documentation, I have updated the documentation accordingly.
[ ] ✨ This pull request is ready to merge. ✨

Thanks @mdt2, great feedback. I have no additional feedback, just +1 to her suggestions.

One thing I observed, unrelated to this change because it was happening before too, is the size of the warning icon in XL Picker, these seem like a suspicious size:

One thing I observed, unrelated to this change because it was happening before too, is the size of the warning icon in XL Picker, this seems like a suspicious size:

This does look strange. I took a quick peek and it looks like the token definition matches the S1 design spec, but we have a margin-inline-start value that seems to be limiting it at the XL size:

Guys, I have no more things to add to this PR, happy to implement any other feedback, or else we can merge it.
cc: @mdt2 @castastrophe


Change MarginAdmin enum to be constant uint8

Pull Request Test Coverage Report for Build 5847

4 of 4 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 5831:
0.0%

Covered Lines:
935

Relevant Lines:
935

💛  - Coveralls


captive portal added in main.cpp PsychicHttp example

Thanks for the great job done on PsychicHttp 👍
1) SUMMARY
For my project, there is a need for captive portal, ie web page which opens automatically when user connects Wifi network (eg "PsychitHttp").
Captiveportal is implemented in ESPAsyncWebServer https://github.com/me-no-dev/ESPAsyncWebServer/blob/master/examples/CaptivePortal/CaptivePortal.ino and in arduino-esp32 examples https://github.com/espressif/arduino-esp32/blob/master/libraries/DNSServer/examples/CaptivePortal/CaptivePortal.ino
This feature can be implemented with Psychichttp with a dedicated handler, as shown in updated "main.cpp" example.
If needed, captive portal could be added in "main.cpp" PsychicHttp example.
Thanks
Code is added below for reference.
2) CODE
Definitions
// captiveportal
// credits https://github.com/me-no-dev/ESPAsyncWebServer/blob/master/examples/CaptivePortal/CaptivePortal.ino
//https://github.com/espressif/arduino-esp32/blob/master/libraries/DNSServer/examples/CaptivePortal/CaptivePortal.ino
#include <DNSServer.h>
DNSServer dnsServer;
//PsychicWebHandler captiverequest;
class CaptiveRequestHandler : public PsychicWebHandler { // handler
public:
  CaptiveRequestHandler() {};
  virtual ~CaptiveRequestHandler() {};
  bool canHandle(PsychicRequest*request){
    // ... if needed some tests ... return(false);
    return true;  // activate captive portal
  }
  esp_err_t handleRequest(PsychicRequest *request) {
   //PsychicFileResponse response(request, LittleFS, "/captiveportal.html"); // captive portal page, if any
   //return response.send();                                                 // return captive portal page
   return request->reply(200,"text/html","Welcome to captive portal !");  // simple text, comment if captive portal page
  }
};
CaptiveRequestHandler *captivehandler=NULL;             // handler for captive portal

setup()
    // captive portal
    dnsServer.start(53, "*", WiFi.softAPIP());                    // DNS requests are executed over port 53 (standard)
    captivehandler= new CaptiveRequestHandler();                 // create captive portal handler, important : after server.on since handlers are triggered on a first created/first trigerred basis
    server.addHandler(captivehandler);                           // captive portal handler (last handler)

loop()
 dnsServer.processNextRequest();       // captive portal
result (web page opened automatically)

Captive portal already works

server.on("/redirect", HTTP_GET, [](PsychicRequest* request)
{ return request->redirect(redirectURL); });
server.on("/connecttest.txt", HTTP_GET, [](PsychicRequest* request)
{ return request->redirect(redirectURL); });
server.on("/hotspot-detect.html", HTTP_GET, [](PsychicRequest* request)
{ return request->redirect(redirectURL); });
server.on("/generate_204", HTTP_GET, [](PsychicRequest* request)
{
return request->redirect(redirectURL); });
server.on("/gen_204", HTTP_GET, [](PsychicRequest* request)
{ return request->redirect(redirectURL); });
i do it like this. any better versions?

server.addHandler(&firstBootHandler).setFilter(ON_AP_FILTER);

class FirstBootHandler : public PsychicWebHandler{
    public:
        esp_err_t handleRequest(PsychicRequest *request) override;
};
inline FirstBootHandler firstBootHandler;

esp_err_t FirstBootHandler::handleRequest(PsychicRequest *request){
   request->loadBody();
   request->loadParams();
   if (request->path() == "/something") {
       // handle "/something"
   }else if (request->path() == "/something1") {
       // handle "/something1"
   }else{
       // hanle "/"
   }
}

I'm happy to merge these changes, but would you mind removing the captive portal code from the main demo code and adding it as a separate sketch/program with a simple "Hello World" page?  The main example is getting pretty complex as it is.  Similar to how your OTA example is its own separate sketch.

Ok, I agree with you, let's keep the main example as it is.
So I will update/resubmit PR and include 2 examples "captive portal" and "OTA update" as separate examples. I will do this in the coming days.
Will add them in the Arduino section since I am using Arduino IDE.
Maybe it could be worth to describe each example as a separate directory, as in https://github.com/espressif/arduino-esp32/tree/master/libraries/WebServer/examples ? Today there is one example for PlatformIO, one for Arduino and one for ESP-IDF, so it is not "easy" to add new examples.
Thanks !

Just name them arduino_ota and arduino_captive_portal for now, I'll see
about shuffling things around later.
On Tue, May 28, 2024, 02:12 06GitHub @.***> wrote:

Ok, I agree with you, let's keep the main example as it is.
So I will update/resubmit PR and include 2 examples "captive portal" and
"OTA update" as separate examples. I will do this in the coming days.
Will add them in the Arduino section since I am using Arduino IDE.
Maybe it could be worth to describe each example as a separate directory,
as in
https://github.com/espressif/arduino-esp32/tree/master/libraries/WebServer/examples
? Today there is one example for PlatformIO, one for Arduino and one for
ESP-IDF, so it is not "easy" to add new examples.
Thanks !
—
Reply to this email directly, view it on GitHub
https://github.com/hoeken/PsychicHttp/pull/88#issuecomment-2134418954,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABEHSXQMPLNU6PJVBNBLP3ZEQN5ZAVCNFSM6AAAAABHWT77W2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZUGQYTQOJVGQ
.
You are receiving this because you commented.Message ID:
@.***>


Better client

merging in for now as the only goal of this was to allow you to use this plugin without having the "optional" dependencies installed such as redis.


HQ items not seen as HQ from Etro Import

Currently, if comparing to a gear/BiS list, an HQ crafted item from an Etro list is seen as an NQ item, which prevents correct matching.
Example:
Currently equipped: Diadochos Headband of Fending (HQ), DH+36/DH+36/DH+36/DH+12/DH+12
Etro list: Diadochos Headband of Fending (HQ), DH+36/DH+36/DH+36/DH+12/DH+12
Lootmaster shows from Etro list: Diadochos Headband of Fending, DH+36/DH+36/DH+36/DH+12/DH+12
These should match, but because it's importing the HQ item from Etro as NQ, it fails to match.

Fixed with 2dea28292c04b43dfde18213fdcf4af12fc1a277 and will be part of Version <IP_ADDRESS>
You will probably need to click "Update from etro.gg" for all affected BiS once the update is available


Allow add/remove users to workspace chat

Details

Allow anyone to be added/removed to the workspace chat via the Invite/Remove button of the members page using the RoomMembersPage instead of ReportParticipantsPage
Fixed Issues

$ https://github.com/Expensify/App/issues/35391
PROPOSAL: https://github.com/Expensify/App/issues/35391#issuecomment-2040742869
Tests

Login to New Expensify app
Go to a Workspace chat
Go to the Workspace chat Details page
Go to Members page
Click on the Invite Button
Verify you can Add Users to the Workspace chat (Add several users with different types of personal details: Known, Unknown, expesifail, ...)
Still on the  Members page click on Remove button
Verify you can Remove Users to the Workspace chat

[x] Verify that no errors appear in the JS console

Offline tests

Same as Tests

[x] Verify that no errors appear in the JS console

QA Steps

Same as Tests
PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

https://github.com/Expensify/App/assets/95647348/6dab48eb-fe27-481f-82a8-a394bcbb8184

iOS: mWeb Safari

https://github.com/Expensify/App/assets/95647348/5ef28785-f9c6-4b20-97c5-806cf48000aa

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/95647348/2a145e66-a668-43ff-839c-8f9a423144bd

MacOS: Desktop

https://github.com/Expensify/App/assets/95647348/4f28f674-1849-49f6-8d31-a4c56c351089

Will review this PR today/tomorrow.

@brunovjk Is this PR ready for review. I am not sure if the change is working. I can see errors in the code. I also don't see any screencasts added for any platforms. Can you please take care of this? I

Sorry @mananjadhav, I had made videos for other platforms except Android, since we have the "merges" frozen, I'll finish fixing my Android build and record the videos by Monday, is it ok? Thank you.

Yeah that's fine. I'll try to test at my end as well.

 I'll test this over the weekend.

Okay, are you better now? Can you help me with videos for Android? I'm waiting for a flat cable to arrive for my main computer, on this notebook I can't build Android :/

fixing errors

fixing errors

I had to redo our PR, I couldn't solve the problem with signing the commits.


Return response for edugain metadata

see https://www.pivotaltracker.com/story/show/117599315

@thijskh

:+1:


Unable to link .ton Level 3 subdomain

Hey there,
I've downloaded latest distribution of the binary and started it with --domain three.mydomain.ton (trying to link level 3 subdomain to my TON site).
It prompted me to submit a transaction with QR-code, however, it stuck at this point and won't go any further no matter how long I am waiting since transaction execution.
Ubuntu 24.04 LTS, firewall disabled.

As investigation shown, TON does not support Level 3 domains out of the box and it requires bit more efforts (e.g. implementation of custom contract for DNS resolving of subdomains).


jdk: 15.0.1-ga -> 16+36

Built jdk16 on x86_64-linux and i686-linux.
The version string doesn't look quite right yet:
OpenJDK Runtime Environment (build 16+0-adhoc..source)

Motivation for this change

JDK 16 reached General Availability on 16 March 2021

https://openjdk.java.net/projects/jdk/16/
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 116717 run on x86_64-linux 1

18 packages marked as broken and skipped:

aldor
foo-yc20
jboss
junit
junixsocket
octave-jit
octavePackages.fem-fenics
octavePackages.vibes
octavePackages.vrml
openfire
pig
plm
purePackages.faust
python38Packages.skein
python39Packages.skein
spark
spring
springLobby

5 packages failed to build and already failed to build on hydra master:

graalvm8: log was empty
jvmci8: log was empty
magnetophonDSP.MBdistortion: log was empty
sweethome3d.furniture-editor: log was empty
sweethome3d.textures-editor: log was empty

18 packages failed to build and are new build failures:

kodi: log was empty
(kodiPackages.kodi): log was empty
kodi-gbm: log was empty
kodi-wayland: log was empty
kodiPackages.inputstream-adaptive: log was empty
kodiPackages.joystick: log was empty
kodiPackages.kodi-platform: log was empty
kodiPackages.pvr-hdhomerun: log was empty
kodiPackages.pvr-hts: log was empty
kodiPackages.pvr-iptvsimple: log was empty
kodiPackages.steam-controller: log was empty
kodiPackages.vfs-libarchive: log was empty
kodiPackages.vfs-sftp: log was empty
mindustry: log was empty
mindustry-server: log was empty
mindustry-wayland: log was empty
plantuml-server: log was empty
yed: log was empty

313 packages built:

alchemy
alda
ammonite (ammonite_2_13)
ammonite_2_12
apache-directory-studio
apache-flex-sdk
apktool
archi
arduino
arduino-core
asciidoc-full-with-plugins
asciidoctorj
aspectj
astrolabe-generator
atlassian-cli
avro-tools
aws_mturk_clt
awstats
axis2
basex
bfg-repo-cleaner
bftools
bitwig-studio1
bitwig-studio2
bloop
bonnmotion
boot
briss
ccemux
cfr
checkstyle
clooj
closurecompiler
confluent-platform
coursier
cryptomator
davmail
dex2jar
ditaa
domination
ec2_api_tools
emem
emscripten
epubcheck
ergo
exhibitor
faust (faust2)
faust2alqt
faust2alsa
faust2csound
faust2firefox
faust2jack
faust2jackrust
faust2jaqt
faust2ladspa
faust2lv2
faustlive
flexibee
flink
flyway
frostwire
frostwire-bin
gama
ganttproject-bin
geoipjava
gephi
global-platform-pro
gogui
gpsprune
graylog
gremlin-console
groove
groovy
h2
haxePackages.hxjava
hdfview
hsqldb
html5validator
hydraAntLogger
imagej
ino
ipscan
jadx
jameica
jbake
jbang
jd-gui
jdiskreport
jdk (jdk16 ,jre ,openjdk ,openjdk16)
jre_headless (jdk16_headless ,openjdk16_headless ,openjdk_headless)
jedit
jflex
jicofo
jing (jing-trang)
jitsi-videobridge
jmeter
jnetmap
josm
jre_minimal
jruby
jugglinglab
jython
kapitonov-plugins-pack
key
keycloak
kotlin
ktlint
languagetool
leiningen
leo3-bin
libreoffice-fresh
libreoffice-fresh-unwrapped
libreoffice-qt
libreoffice-still
libreoffice-unwrapped (libreoffice-still-unwrapped)
librsb
liquibase
logisim
logstash (logstash6)
logstash6-oss
logstash7
logstash7-oss
lombok
ma1sd
macse
magnetophonDSP.CharacterCompressor
magnetophonDSP.CompBus
magnetophonDSP.ConstantDetuneChorus
magnetophonDSP.LazyLimiter
magnetophonDSP.RhythmDelay
magnetophonDSP.VoiceOfFaust
magnetophonDSP.faustCompressors
magnetophonDSP.pluginUtils
magnetophonDSP.shelfMultiBand
makemkv
martyr
marvin
maven (maven3)
mediathekview
metals
micronaut
mill
minc_widgets
minecraft
minecraft-server
mkgmap
mkgmap-splitter
mooSpace
msgviewer
munin
mxisd
nailgun
netlogo
nexus
nzbhydra2
octave
octaveFull
octavePackages.arduino
octavePackages.audio
octavePackages.bim
octavePackages.bsltl
octavePackages.cgi
octavePackages.communications
octavePackages.control
octavePackages.data-smoothing
octavePackages.database
octavePackages.dataframe
octavePackages.dicom
octavePackages.divand
octavePackages.doctest
octavePackages.econometrics
octavePackages.financial
octavePackages.fits
octavePackages.fpl
octavePackages.fuzzy-logic-toolkit
octavePackages.ga
octavePackages.general
octavePackages.generate_html
octavePackages.geometry
octavePackages.gsl
octavePackages.image
octavePackages.image-acquisition
octavePackages.instrument-control
octavePackages.interval
octavePackages.io
octavePackages.level-set
octavePackages.linear-algebra
octavePackages.lssa
octavePackages.ltfat
octavePackages.mapping
octavePackages.matgeom
octavePackages.miscellaneous
octavePackages.msh
octavePackages.mvn
octavePackages.nan
octavePackages.ncarray
octavePackages.netcdf
octavePackages.nurbs
octavePackages.ocl
octavePackages.octclip
octavePackages.octproj
octavePackages.optics
octavePackages.optim
octavePackages.optiminterp
octavePackages.parallel
octavePackages.quaternion
octavePackages.queueing
octavePackages.signal
octavePackages.sockets
octavePackages.sparsersb
octavePackages.splines
octavePackages.statistics
octavePackages.stk
octavePackages.strings
octavePackages.struct
octavePackages.symbolic
octavePackages.tisean
octavePackages.tsa
octavePackages.video
octavePackages.windows
octavePackages.zeromq
odpdown
omegat
opa
open-ecard
openapi-generator-cli
openapi-generator-cli-unstable
openems
opengrok
openjdk16-bootstrap
openjump
papermc
perl530Packages.NetZooKeeper
perl532Packages.NetZooKeeper
picard-tools
picolisp
plantuml
portmod
prometheus-jmx-httpserver
python38Packages.databricks-connect
python38Packages.python-csxcad
python38Packages.python-openems
python38Packages.sphinxcontrib_plantuml
python39Packages.databricks-connect
python39Packages.python-csxcad
python39Packages.python-openems
python39Packages.sphinxcontrib_plantuml
quantomatic
rascal
riemann
runelite
saxon
saxon-he
saxonb (saxonb_8_8)
saxonb_9_1
sbt (simpleBuildTool)
sbt-extras
sbt-with-scala-native
scalafmt
schemaspy
selendroid
selenium-server-standalone
seyren
shattered-pixel-dungeon
smc
snpeff
soapui
solr
sonar-scanner-cli
sourcetrail
spring-boot-cli
squirrel-sql
sshoogr
stm32cubemx
storm
subgit
subsonic
swagger-codegen
sweethome3d.application
swingsane
swt
tabula
tabula-java
tambura
teensyduino
tessera
tomcat-native
tomcat_connectors
tr-patcher
turbovnc
tuxguitar
tvbrowser-bin
umlet
unciv
unoconv
varscan
vassal
vimPlugins.lens-vim
vimPlugins.vim-grammarous
visualvm
vue
vuze
weka
workcraft
xmind
xmloscopy
youtrack
yuicompressor
zap
zipkin
zkfuse
zookeeper_mt
zxing

Please do not remove JDK major versions. Projects tend to be picky about those.

Seems like a good idea to call it 16.nix instead of default.nix. That will save renaming when 17 comes out.

I've overestimated upstream's support window. 15 should be marked insecure.

IMO we should not carry insecure, unsupported JDK versions in a rolling-release distro like NixOS / nixpkgs; we should carry the current LTS release and the current release, nothing more.  If users need an intermediate release like 15, they should pin nixpkgs at an older revisions where it was present.

IMO we should not carry insecure, unsupported JDK versions in a rolling-release distro like NixOS / nixpkgs; we should carry the current LTS release and the current release, nothing more. If users need an intermediate release like 15, they should pin nixpkgs at an older revisions where it was present.

The problem with this approach is that they are then using outdated dependencies. This works for Debian Sid or Arch but not nix.

Well, the JDK itself is an outdated dependency in this scenario, so I'm not sure it's really a new problem if the JDK's dependencies are also outdated.
The JDK in the previous stable NixOS release should see those dependency updates at least for a while, so that would also be an option for users who need it, at least for a short time.
All this being said, I certainly wouldn't reject a PR that gets JDK 16 into nixpkgs only because it also retains JDK 15!  :-)

I was testing the openjdk 16 version from this PR, and it works fine, I didn't find any problems :) thanks @petabyteboy

I renamed the file to 16.nix to make further renames betwen <version>.nix and default.nix unnecessary.
I also added some options to make the version string more sensible:
[pbb@onyx:~/proj/nixpkgs]$ result/bin/java -version
openjdk version "16" 2021-03-16
OpenJDK Runtime Environment (build 16+36-nixos)
OpenJDK 64-Bit Server VM (build 16+36-nixos, mixed mode, sharing)

[pbb@onyx:~/proj/nixpkgs]$

I added another commit that updates the binary zulu jdk distribution for darwin, which is available under the same openjdk16 attribute.

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 116717 run on x86_64-darwin 1

26 packages marked as broken and skipped:

jboss
junixsocket
kodiPackages.inputstream-adaptive
kodiPackages.inputstream-ffmpegdirect
kodiPackages.inputstream-rtmp
kodiPackages.joystick
kodiPackages.pvr-hdhomerun
kodiPackages.pvr-hts
kodiPackages.pvr-iptvsimple
kodiPackages.steam-controller
kodiPackages.vfs-libarchive
kodiPackages.vfs-sftp
mindustry
mindustry-server
mindustry-wayland
octave-jit
odpdown
openfire
picolisp
plm
portmod
python38Packages.skein
python39Packages.skein
shattered-pixel-dungeon
spark
vuze

6 packages failed to build and already failed to build on hydra master:

graalvm8: log was empty
haxePackages.hxjava: log was empty
librsb: log was empty
octaveFull: log was empty
scenebuilder: log was empty
scenic-view: log was empty

3 packages failed to build and are new build failures:

plantuml-server: log was empty
python39Packages.sphinxcontrib_plantuml: log was empty
sourcetrail: log was empty

121 packages built:

ammonite (ammonite_2_13)
ammonite_2_12
apktool
asciidoc-full-with-plugins
asciidoctorj
aspectj
astrolabe-generator
awstats
axis2
bfg-repo-cleaner
bftools
bloop
boot
briss
cfr
clooj
closurecompiler
confluent-platform
davmail
ditaa
domination
emem
emscripten
epubcheck
ergo
exhibitor
flink
flyway
frostwire
gama
geoipjava
global-platform-pro
gogui
gpsprune
gremlin-console
groove
groovy
hsqldb
html5validator
hydraAntLogger
imagej
jadx
jameica
jbang
jd-gui
jdiskreport
jedit
jflex
jing (jing-trang)
jmeter
jmusicbot
jruby
jugglinglab
key
kotlin
leiningen
liquibase
logisim
lombok
ma1sd
macse
maven (maven3)
mediathekview
micronaut
mill
minc_widgets
minecraft-server
mkgmap
mkgmap-splitter
msgviewer
mxisd
nexus
octave
opa
papermc
perl530Packages.NetZooKeeper
perl532Packages.NetZooKeeper
picard-tools
plantuml
prometheus-jmx-httpserver
python38Packages.databricks-connect
python38Packages.sphinxcontrib_plantuml
python39Packages.databricks-connect
quantomatic
rascal
riemann
runelite
saxon
saxon-he
saxonb (saxonb_8_8)
saxonb_9_1
sbt (simpleBuildTool)
sbt-extras
sbt-with-scala-native
selendroid
selenium-server-standalone
seyren
snpeff
soapui
solr
sonar-scanner-cli
spring-boot-cli
sshoogr
stm32cubemx
storm
subgit
subsonic
swingsane
tabula
tabula-java
tomcat-native
tomcat_connectors
umlet
varscan
vassal
visualvm
weka
xmloscopy
yuicompressor
zipkin
zookeeper_mt

Thanks for testing! I don't see the jdk in there, but it must be because those things depend on it.
I would say let's apply this and fix the fallout. In the worst case, some packages will have to be downgraded to jdk11 until there's a new LTS release (I think 17 will be LTS?).


Don't fail the build if the issue key is not found

Not every build needs an issue key, just skip posting the comment to JIRA if the issue isn't found.

@dagio do you think this should be an option, or can I change the step to only actually fail if it tried to post to JIRA and couldn't?

There a is_skippable property that you can set for any steps in your workflow. I think that's what you are looking for ?
More details here : http://devcenter.bitrise.io/tips-and-tricks/dont-mark-build-failed-if-step-fails/

Indeed, thank you!


Pretrained model

Hi eleurent,
Thank you very much for your excellent work. I also noticed that you have another repo "rl-agents" for training agents in this environment. I have set up the DQN agents using your baseline config. However, I cannot generate a policy like your Gif. Could you please help me?

And I cannot run most of your "rl-agents" config. which use different algorithms. Some of them have trouble saving the trained models.

Hi @songanz,
The DQN gif on the Readme was generated about a year ago, and both the environment and agent source codes have evolved since, it is possible that some further tuning is required.
I'll try to run a training again and get back to you.

I cannot run most of your "rl-agents" config. which use different algorithms. Some of them have trouble saving the trained models.
Which agents are you talking about exactly? For some agents there is simply no training phase / model to save, so they may produce a simple warning when loading them.

The current config dates from last may, see: https://github.com/eleurent/rl-agents/issues/21

Thank you very much for the feedback, there was indeed a regression caused by the recent refactoring of observations: the vehicles used to be sorted by increasing distance, but were not anymore.
I reverted to this behaviour as default, since it is helps simple models to converge (front vehicles are always in the first rows of the input tensor).
Here are the results I get, before (red) and after the fix (grey).

You can try to updgrade and run experiments.py evaluate configs/HighwayEnv/env.json configs/HighwayEnv/agents/DQNAgent/baseline.json --train --episodes=1500.
Let me know if this helped :)

Thank you very much for your excellent work.

By the way, what should I do if I want to cite your rl-agents repo?

Thanks for asking!
I added a bibtex in the readme: https://github.com/eleurent/rl-agents#citing

I have tried your newer version and I get the same result as in your reply. Thank you very much.

Thank you very much for the feedback, there was indeed a regression caused by the recent refactoring of observations: the vehicles used to be sorted by increasing distance, but were not anymore.
I reverted to this behaviour as default, since it is helps simple models to converge (front vehicles are always in the first rows of the input tensor).
Here are the results I get, before (red) and after the fix (grey).

You can try to updgrade and run experiments.py evaluate configs/HighwayEnv/env.json configs/HighwayEnv/agents/DQNAgent/baseline.json --train --episodes=1500.
Let me know if this helped :)


[BUG] Queue misaligned in playback bar

The Queue Button misaligned is in playback bar.

Windows 11
Spotify für Windows (64 Bit)
<IP_ADDRESS>1.ge66eb7bc
Spicetify v2.36.11
xpui_2024-05-03_1714757530304_e66eb7b
cef_122.1.13+gde5b724+chromium-122.0.6261.130
Runtime: Chrome
Build System: CMake

it is fixed now check new update

yup fixed :)


Resolve licensing issues discovered by Fossology scan

Iroha source was scanned using Fossology to determine license for files contained in the following repositories

https://github.com/hyperledger/iroha.git
https://github.com/hyperledger/iroha-javascript.git
https://github.com/hyperledger/iroha-ametsuchi.git
https://github.com/hyperledger/iroha-scala.git
https://github.com/hyperledger/iroha-ios.git
https://github.com/hyperledger/iroha-android.git
https://github.com/hyperledger/iroha-network-tools.git
https://github.com/hyperledger/iroha-docker-develop.git
https://github.com/hyperledger/iroha-python.git

From the Hyperledger Charter (https://www.hyperledger.org/about/charter) Section 13, all files that are included in Hyperledger projects should either be licensed with the Apache License, Version 2.0 for source code or Creative Commons Attribution 4.0 International License for documentation.
There are files in the Iroha source base that do not meet these requirements.  See the attached file for results. iroha-2017-06-05.xlsx
The files listed under Attribution, Other, and No License Found need to be resolved. For the Attribution and Other tab, these files should be replaced with code meeting the Apache 2.0 license.
For the files listed as No License Found, first check with the original contributors to make sure that they can contain an Apache 2.0 license (for source code) or a Creative Commons with Attribution 4.0 license (for documentation). Once this has been verified, the editable files must be annotated with the correct license text and/or the SPDX short identifiers (see  https://spdx.org/sites/cpstandard/files/pages/files/using_spdx_license_list_short_identifiers.pdf).
For binary files, a license.txt file should be included in the directory calling out the file types and the associated license.
Any files that are not covered by Apache 2.0 or CC by 4.0 that need to remain with the project will need to go through an exception process with the governing board.

Ok thanks
In now, WIP in  this pull request.
https://github.com/hyperledger/iroha/pull/349

This scan should probably be run again on the develop branch, as a lot has changed.

@takemiyamakoto : is this true for all Iroha repositories or just this one? Meaning, if I pulled the develop branch for all Iroha repositories listed above would that be the right scan to do?

Just this repository.

Tracy, can you please help us and run this scan once again? Thanks

Tracy, if this problem remains, please reopen this issue. I will close it since there was no answer for a month 😃


When add computed property in Group will result InternalServerError

This block works very well
with S := (select SellingOrderItem{id,sellerSku,qty} filter exists .sellerSku),
groups := (  group S{sellerSku} by .sellerSku )
select groups {sku := .key.sellerSku, key, sum := sum(.elements.qty)} order by .sum

but when I add a computed property is select it will result error.
with S := (select SellingOrderItem{id,sellerSku,qty,   size :=.sellingProductListing.sellingProduct.size    } filter exists .sellerSku),
groups := (  group S{sellerSku} by .sellerSku )
select groups {sku := .key.sellerSku, key, sum := sum(.elements.qty)} order by .sum

When I move computed property to schema and refer to it with property name size, it will works fine again.

EdgeDB Version: 2.9

EdgeDB CLI Version:

OS Version: WSL

Could you share your schema?

Sorry for my late reply.
Here is schema
module default {
    abstract type CommonType {
        property createdAt -> datetime {default := datetime_current();}
        property updatedAt -> datetime;
    }

    abstract type OrderItem {
        property sellerSku -> str;
        property unitPrice -> float32;
        property qty -> int16;
        property qtyShipped -> int16;
    }

    type SellingOrderItem extending OrderItem, CommonType {
        property orderItemId -> str {constraint exclusive;};
        link sellingProductListing -> SellingProductListing;
        # property size :=.sellingProductListing.sellingProduct.size;
    }

    type SellingProductListing extending Listing, CommonType {
        link sellingProduct -> SellingProduct;
        property listingSku -> str;
        property customsDeclarationPrice -> float32;
    }

    abstract type Listing {
        property listingId -> str;
        property maxOrderQty -> int16;
        property unitPrice -> float32;
        property listingUrl -> str;
    }

    type SellingProduct extending Product, CommonType {
        property hsCode -> str;
        property productBrief -> str;
        property size -> str;
    }

    abstract type Product {
        property unitPrice -> float32;
        property sku -> str;
        property discontinued -> bool;
        property remark -> str;
    }

}

This looks like it is fixed in 3.x


OpenCV in 1.4

@dwhitena I think we are going to have to change the beginner tutorial for v1.4. Logs being processed in the way we do the fruit stand actually isn't a good use case in v1.4. We dont support large files that you constantly append to very well. We also dont support append-based incrementality at all and don't have /prev anymore to do the Sum pipeline efficiently.
OpenCV just seems much cleaner and is actually a more interesting overall demo IMO. The only part I'm not sure about is i'd like the tutorial to not include pachctl mount but viewing images so you can see your data is tough without it. Perhaps we need to think of something else for beginner tutorial entirely.

Yep, that's great @JoeyZwicker.  I'll think about the mount.

AFAIK opencv is currently the beginner tutorial in 1.4 but there are some issues with it. @JoeyZwicker is that correct and could you expand on what the issues are.

OpenCV is used for both the beginner tutorial but also our live demo. The tutorial is pretty good as is, but there are a few places the demo could be improved in v1.4.

Mounting with FUSE was a great way to demonstrate the commit structure of PFS when users could see master/0, master/1, etc. In v1.4 you only see UUID's in random order and it's was less visually meaningful. Having commit names aliased as described in #1591 might be a great option.

Having more info displayed in pachctl inspect-* commands or potentially in the mount as well if that's possible, would also improve the demo experience.

@dwhitena and I have talked a lot about adding another pipeline step after the edge detection, perhaps just a crop or resize to show pipeline stages, updates, provenance, etc. Dan, I actually dont remember the specifics of what we discussed, but can you add them here if you do?

Yes, @JoeyZwicker related to (3) above, I got feedback from a number of Google advocates that the demo could be made much stronger by having a very simple second stage.  This could be a crop or resize for example.  We could literally add and run this in a couple of second, but it would really demonstrate the power of pipelines in Pachyderm.  The one stage pipeline doesn't differentiate as much from other things like Domino, for example.

One idea I had for a second stage was using ImageMagick to turn the outputted images into a montage. You can do some pretty cool stuff with a simple command line invocation: http://www.imagemagick.org/Usage/montage/ and I think it would work well for the demo because you could just open up a single image at the end which shows the original images and their edges side by side. And of course the montage would automatically update as new images come in.

@jdoliner that's brilliant.  Great idea and I like how it is sort of a reduce after the map, and a different glob pattern.  We could also do it in simple bash if we wanted.

The montage addition was submitted in https://github.com/pachyderm/pachyderm/pull/1850
So I think this can be closed? @JoeyZwicker ?

Only #3 on my list above have been added and we haven't actually adding the montage step to the beginner tutorial yet -- the UI might solve this once we have the file browser ready to go, but we're not there yet.

@JoeyZwicker I have done the OpenCV demo quite a few times using display along with list-file and get-file.  It seems to work pretty smooth.  I'm not quite sure what the FUSE mount adds at this point, especially given that it is somewhat confusing to explain for many people (that is, explaining the mounting of the distributed file system seems to be a hurdle for many people that could be avoided by just listing files and getting them).  I guess you are showing multiple commits, but I think that is pretty clear when you list-commit and list-file.  The only piece that would be missing would be the timestamp.
These are just my opinions of course.  @JoeyZwicker, you have a ton of experience with the demo.
And, yes, @sjezewski we need to add the docs for the montage, which is part of the 1.5 checklist.


do not fail if no git

closes #89

This pr fixes a bug that where the prompt doesn't ask for all the values if it's not a git repo
Also we clean up the prompt a little to make sure the user has context


ADCE: Remove OpDecorateStringGOOGLE

Also fix a few failures to set "modified" status when removing
global values.
Fixes #1492

Can OpDecorateStringGoogle get applied to a group?

From
https://github.com/KhronosGroup/SPIRV-Registry/blob/master/extensions/GOOGLE/SPV_GOOGLE_decorate_string.asciidoc
"Target is the  to decorate. It can potentially be any  that is a forward reference, except it must not be the  of an OpDecorationGroup."

In any case, you should add this to the decoration sorting rules in DecorationLess to ensure a good processing order with groups.

Will do.

Rebased, squashed, and pushed into master as 152b9a681e


Point Singularity at the "external" interface where mesos-master is listening.

Let's add this to docker-compose.yml as well as should be gtg 👍

🚢


&preserve-view=true

Docs Build status updates of commit 8e595d7:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

project-xml-data-interchange/resource-elements-and-xml-structure.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


KMAC FIFO_Empty interruption is not firing when one byte is written at a time

Problem
When the KMAC DIF writes one byte at a time to the FIFO the FIFO_Empty interruption never fires.
Cause
When the Firmware fill the FIFO by writting one byte at a time the KMAC consume the data faster than the FW can put. It causes the FIFO State Register never leaves the empty state. Thus, there's no rising edge on the State register to trigger the interruption.
KMAC registers Waveforme

What should be done?
Is this behavior a bug? If not what would be the correct use case for the interruption?

@mundaym for visibility

@doreis-lowrisc: I realise that you know this already because we were chatting about it yesterday, but it might interest others...
Assuming you can push data fast enough across to KMAC to fill up its buffer (see https://github.com/lowRISC/opentitan/pull/6470) then you might want to go to sleep or do something else until it drains again. The only awkward thing is that it's quite difficult to be certain that you'll get the interrupt. I think the following logic would work:

Write some stuff to the KMAC block
Enable the buffer empty interrupt, if necessary
Read KMAC's status register to see whether the buffer is empty.
If so, go back to (1). Apparently, we didn't write fast enough.
If not, go and do something else or WFI.

The point is that if the buffer isn't empty when read at (3) then we know it has been nonempty at some point since the interrupt was enabled and so we are guaranteed to get a "posedge" of the buffer empty signal, which causes the interrupt.
It's still a bit finicky: whoever writes the KMAC driver will need to know about this (and maybe it should be flagged up explicitly in the HW documentation).

@rswarbrick I also tested writhing 4 bytes at time to the FIFO and I got the interruption. So indeed your flow will work.
Making that behavior clear in the HW documentation could be the solution.
@eunchan Any comment?

It's still a bit finicky: whoever writes the KMAC driver will need to know about this (and maybe it should be flagged up explicitly in the HW documentation).

We should definitely flag this in the docs if it isn't already. I think I understand why it works this way but it is 'weird' that you can write something to the FIFO yet never see a transition out of the empty state.

Thanks for the issue. Empty interrupt won't be asserted as KMAC (SHA3) consumes data right at the same cycle. The FIFO is set to pass the data to reduce one clock latency. We can delay one cycle by disabling Pass parameter and make sure it always fires the empty interrupt.

Correct approach is to change prim_fifo_sync to have empty output that is cleared when depth is 0 and write port pushes data. The module is widely used in OT designs.
@tjaychen @mwbranstad @msfschaffner @rswarbrick @martin-lueker What do you think? Can we add empty port in prim_fifo_sync and clears the empty signal when the write port asserts wvalid while depth == 0?

Hi @eunchan, I apologize, but I'm not following, could you perhaps draw a figure to illustrate?

Hi @eunchan, I apologize, but I'm not following, could you perhaps draw a figure to illustrate?

Are you saying that the empty port would be deasserted in the cycle when the data is inserted, but before the data is available on the other side?

rvalid_o indicates "not_empty" today, so the inverted version of that signal is "empty". It seem like you want some additional function here. If you are saying that on cycle N the fifo state is a depth of zero, but if the wvalid_i signal is also active on cycle N, that the rvalid_o reflects not_empty on cycle N, then that seems like incorrect behavior anyway. I would prefer not to add a new port unless a parameter makes it optional. I am also guessing that if the fifo works as described above, there are current designs that may depend on this and the behavior needs to be preserved, hence a new signal. Do i have this all correct?

Ah.. Thanks @martin-lueker @mwbranstad . I totally forgot rvalid can be an inversion of empty :)
I think adding empty is not needed.
KMAC MSGFIFO used depth to calculate empty status of the FIFO. I think the logic should use rvalid rather than depth. Thanks :)

The PR #9276 has been merged. @doreis-lowrisc Could you please check again?

@eunchan  It seens that your PR broke the CI and got reverted.
However I tested it and the interruptions fired when one byte is inserted into the FIFO, but the interruption handler keeps being called even after the acknowledge (kmac.INTR_STATE=0 )
You can use this code to test it using the simulator and listening the uart for logs.

@doreis-lowrisc Can you paste a copy of the UART output in this issue? If the log is big then you might want to use <detail> tags to make it collapsible.

I just came up with the Idea that the new behavior could be an undesired feature rather than a bug. As the FW is slower than the HW, the FIFO could be getting empty for every byte written.
I'll test it again, collect logs and wave forms to post here.

I just came up with the Idea that the new behavior could be an undesired feature rather than a bug. As the FW is slower than the HW, the FIFO could be getting empty for every byte written. I'll test it again, collect logs and wave forms to post here.
That's what is happening as you can see on the log and wave form below:

Serial log
```
I00000 boot_rom.c:54] Version:    opentitan-earlgrey_silver_release_v5-2034-g8a42d5c61
Build Date: 2021-11-19, 11:56:53
I00001 boot_rom.c:70] Boot ROM initialisation has completed, jump into flash!
I00000 kmac_interrupt_test.c:263] Running KMAC Interrupt test...
I00001 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00002 kmac_interrupt_test.c:293] test_main() :Test 0
I00003 kmac_interrupt_test.c:301] test_main() :Writing Block with 128 bytes
I00004 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00005 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00006 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00007 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00008 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00009 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00010 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00011 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00012 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00013 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00014 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00015 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00016 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00017 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00018 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00019 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00020 kmac_interrupt_test.c:304] test_main() :128 bytes written
I00021 kmac_interrupt_test.c:301] test_main() :Writing Block with 17 bytes
I00022 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00023 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00024 kmac_interrupt_test.c:304] test_main() :17 bytes written
I00025 kmac_interrupt_test.c:180] handle_kmac_isr() :FifoEmpty
I00026 kmac_interrupt_test.c:184] handle_kmac_isr() :KmacDone
I00027 test_status.c:28] PASS!
</details>

![image](https://user-images.githubusercontent.com/92785679/142660550-9264b118-c7cf-4d70-8a1f-63aabc76287d.png)

I just came up with the Idea that the new behavior could be an undesired feature rather than a bug. As the FW is slower than the HW, the FIFO could be getting empty for every byte written. I'll test it again, collect logs and wave forms to post here.

SHA3 throughput is faster than the SW feed rate. So, except SHA3 computation runs in HW (72 rounds with enough entropies), the SW may get the interrupt almost every time.
There's no way we can avoid this. This is same in HMAC too.
I recommend that DIF to ignore empty case in OpenTitan. Just keep checking payload depth and push the amount of available data.
Maybe if KMAC is used in faster CPU, that time empty interrupt may be used.

I recommend that DIF to ignore empty case in OpenTitan. Just keep checking payload depth and push the amount of available data.

@doreis-lowrisc is working on the chip level DV test for KMAC which needs to check all the interrupts work so the DIF does need to support the empty interrupt (albeit only in a very basic way). I think getting an empty interrupt every time is fine for the purposes of the test.
It looks like everything is now working as expected (thanks @eunchan!). Shall we close this issue?

@mundaym Not yet. I was about to finalize the chip level test, and I noted that for small messages the empty interruption never fires. Does not matter if the FIFO is fed 4 bytes at a time or less.
In the log below I was trying to compute the SHA-3 of a message with 4 bytes (just like the smoketest does):
I00000 kmac_interrupt_test.c:263] Running KMAC Interrupt test...
I00001 kmac_interrupt_test.c:294] Test 0
I00002 kmac_interrupt_test.c:306] Absorbing 4 bytes, address 0x8000282c, alignment 0
E00003 kmac_interrupt_test.c:313] CHECK-fail: Timed out after 10000 usec (5000 CPU cycles) waiting for fifo_empty_irq
I00004 test_status.c:34] FAIL!

@eunchan If you need I can colect register traces.

@cindychip Could you please check if we have 4B test in KMAC DV? Does DV also check the interrupt?

@eunchan @doreis-lowrisc
I am super sorry for the late reply, somehow I totally missed this comment and thought it was fixed - We do have DV checks the fifo empty interrupt, and I did see some corner case as below:
Looks like the signal in prim_fifo_sync, the signal wvalid_i will gate the fifo_empty signal, even though the ptr_rd and ptr_wr are equal.

I tried a test sequence that only stream in 4 bytes of data. Looks like FIFO empty is not asserted until KMAC is done.
Is that work SW is waiting for?

Thanks for testing @cindychip . As we discussed offline, this shouldn't be a problem right? Can we close this issue?

Thanks @eunchan and @cindychip, I'll test it again and let you know the results.


Has many not being populated with potential records as options on select.

Locally this works, but on the production server it doesn't.
I have an album, and a photo.
I am using Mongoid.
Album
has_many :photos
Photo
belongs_to :album
When I am editing an album locally, any photo that is in the db shows up in the side by side column to add to the records collection.
When I am editing an album on a production instance with thousands of photos, no photos show up in the side by side view to associate a photo to the album collection.
The only way to associate a photo with an album on the production side is to edit the photo and then associate the album from the photo page. Unable to do "bulk" additions to the album collection from the album edit page.

+1


The time required to receive a response is too long

It will take 30 to 60 seconds to receive a response. Is that normal speed? Is there a way to get a response faster?

What cookies did you use? I did not find the _U cookie. Thank you

It will take 30 to 60 seconds to receive a response. Is that normal speed? Is there a way to get a response faster?

What cookies did you use? I did not find the _U cookie. Thank you


Outcomes in WEB and R don't match

Hi @josesho,
I am opening a new issue as I don't see an answer to this on the previous issue on the same topic.
I have run the same paired data (see below) on the website and in R but the results do not match. Do you know why?
In R: Paired mean difference of CS+ (n = 30) minus CS-(n = 30)  0.207 [95CI  -2.04; 2.59]
In WEB: Paired mean difference of CS+ (n = 30) minus CS-(n = 30)  0.207 [95.0%CI 0.0687, 0.388]
This is the raw data:
CS-  | CS+
8.33 | 8.57
1.30 | 1.26
6.57 | 6.57
7.11 | 7.64
7.16 | 7.46
6.69 | 6.44
4.77 | 4.72
3.21 | 3.46
13.17 | 13.17
5.33 | 5.71
3.04 | 2.66
11.74 | 11.64
3.34 | 3.58
4.39 | 3.88
2.90 | 2.84
6.98 | 6.83
7.84 | 8.59
18.94 | 19.86
19.91 | 21.43
8.42 | 8.88
7.41 | 6.82
11.85 | 12.82
4.99 | 5.31
3.51 | 3.79
10.17 | 10.29
6.48 | 6.62
5.95 | 5.85
3.67 | 3.95
2.90 | 3.13
5.50 | 6.02
Thanks,
Francesca
Originally posted by @francescastarita in https://github.com/ACCLAB/dabestr/issues/94#issuecomment-766978809

Hi,
Sorry for the delay, I am looking into this. I will be consolidating the problem on the earlier issue thread, so closing this for now. Thanks!


For Cli How can I pass and use parameters?

Hi, do you mean passing additional parameters to use in your own template?

Hi, do you mean passing additional parameters to use in your own template?

Yes. for cli, using custom template

It wasn't possible using the current implementation to pass custom parameters.
In the interests of keeping this feature across the CLI and Test Logger implementations, I am adding this feature in #31

Feature published in 1.4.3-beta, there's some details in the readme, but here is a sample from the scripts.
liquid `
--inputs "File=**/*sample.trx;Folder=../Test/LiquidTestReports.Cli.Tests/TrxTestInput;Format=Trx;RunId=123" `
--template "../Test/LiquidTestReports.Cli.Tests/TemplateTestInput/example.md" `
--parameters "Environment=UAT;TicketId=abc123" `
--output-file custom.md

Closing issue with this update


docs: generate JSON openapi schema

Related to https://github.com/hirosystems/docs/issues/426

:tada: This PR is included in version 1.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[benchmark] Just kill the tmux session at the end

This kills the session we started instead of the whole server.
The session we start is named:

here for local runs
here for remote runs

Fixes #152.
Tmux man page:
https://www.man7.org/linux/man-pages/man1/tmux.1.html

I'll revisit in a few.


Look into adding support for industrial_robot_status_controller

Instead of publishing the RobotStatus messages directly from the hardware_interface, expose a IndustrialRobotStatusHandle as a resource and update that periodically.
The IndustrialRobotStatusController can then be made responsible for publishing the message itself, removing the need for a RealtimePublisher in the HardwareInterface.
Ref: industrial_robot_status_controller.

I'm closing this as we've officially deprecated this package.
Refer to the announcement on ROS Discourse.

https://github.com/UniversalRobots/Universal_Robots_ROS_Driver/issues/32 is a follow-up to this for ur_robot_driver.


I can not send mail to outlook/live mail servers

I can not send mail to outlook/live mail servers.
It is said:
host    outlook-com.olc.protection.outlook.com[<IP_ADDRESS>] said: 550 5.7.1
Unfortunately, messages from [<IP_ADDRESS>] weren't sent. Please contact
your Internet service provider since part of their network is on our block
list (S3140). You can also refer your provider to
http://mail.live.com/mail/troubleshooting.aspx#errors.
[DB8EUR06FT039.eop-eur06.prod.protection.outlook.com] (in reply to MAIL
FROM command)

Looks like it's a very common issue:
https://answers.microsoft.com/en-us/outlook_com/forum/all/hotmailoutlook-block-list-s3140-blocks-all-new/699f3a56-406e-4804-97e2-cbe23b9bb01c?auth=1&page=1

Response from Microsoft:
Dear [censored]
We have completed reviewing the IP(s) you submitted. The following table contains the results of our investigation.
Not qualified for mitigation
<IP_ADDRESS>
Our investigation has determined that the above IP(s) do not qualify for mitigation.
Please ensure your emails comply with the Outlook.com policies, practices and guidelines found here: http://mail.live.com/mail/policies.aspx.
To have Deliverability Support investigate further, please reply to this email with a detailed description of the problem you are having, including specific error messages, and an agent will contact you.
Regardless of the deliverability status, Outlook.com recommends that all senders join two free programs that provide visibility into the Outlook.com traffic on your sending IP(s), the sending IP reputation with Outlook.com and the Outlook.com user complaint rates.
Junk Email Reporting program (JMRP) When an Outlook.com user marks an email as "junk", senders enrolled in this program get a copy of the mail forwarded to the email address of their choice. It allows senders to see which mails are being marked as junk and to identify mail traffic you did not intend to send. To join, please visit
https://sendersupport.olc.protection.outlook.com/snds/JMRP.aspx
Smart Network Data Services program (SNDS). This program allows you to monitor the ‘health’ and reputation of your registered IPs by providing data about traffic such as mail volume and complaint rates seen originating from your IPs. To register, please visit http://postmaster.live.com/snds/.
There is no silver bullet to maintaining or improving good IP reputation, but these programs help you proactively manage your email eco-system to help better ensure deliverability to Outlook.com users.
Thank you,
Outlook.com Deliverability Support

Outlook is not blocking us anymore. Still marks emails as spam, but there is not much we can do about it now.


History view - Infinite wallet rest calls when transaction sent to expired namespace alias

Preconditions:
Have an account with the transaction in history which was sent to namespace alias that expired (example account TCETF4ZS2C6AKXQIUJHNXIRCBXVF2NRNMVEFL4I).
Steps:

Go to Home

Result: Wallet is trying to load expired namespace  in a loop
Example transaction for which are problems:
http://ngl-dual-001.testnet.symboldev.network:3000/transactions/confirmed/DEC1A07796044E412E144365725C93272D7148198ED2FEC292A4D01918E128F6
Video: https://share.getcloudapp.com/lluoEADw

Fixed in dev.


Properties window is now shown near mouse cursor position

Resolved / Related Issues

Closes #9285

Details of Changes
Validation

[x] Built and ran the app

Curious: does this work when more than one monitor is connected?

Yeah, subtracting display region's work area offset from pointer position handles that.

Nice, the only issue I'm seeing is that this shows the properties window partially outside the screen bounds when the pointer is close to an edge.
Moving the call to RequestMoveRelativeToDisplayRegion after TryShowAsync appears to work correctly.
bool windowShown = await appWindow.TryShowAsync();
if (windowShown)
{
    // Set window size again here as sometimes it's not resized in the page Loaded event
    appWindow.RequestSize(new Size(460, 550));
    appWindow.RequestMoveRelativeToDisplayRegion(displayRegion,
        new Point(pointerPosition.X - displayRegion.WorkAreaOffset.X, pointerPosition.Y - displayRegion.WorkAreaOffset.Y));
}

Updated

Curious: does this work when more than one monitor is connected?

@gave92 good catch! I only tested this on a single monitor.


[$1000] Settings - Unable to remove legal name after adding it

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Open the app and login to any account
Navigate to Settings > Profile > Personal Details  > Legal Name
Add a legal name and click save
Go back to legal name page and clear the name and click on save to remove the previously added legal name

Expected Result:
Legal name is removed
Actual Result:
Unable to remove legal name
Workaround:
Unknown
Platforms:
Check off any platforms that are affected by this issue
--->
Which of our officially supported platforms is this issue occurring on?

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

Version Number:  1.3.46-1
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/93399543/1f56321c-6438-4432-8540-67a0a5af707c
https://github.com/Expensify/App/assets/93399543/8cd09ec1-1699-4fb6-8187-9480ce58803f
Expensify/Expensify Issue URL:
Issue reported by: @rayane-djouah
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1690565635722659
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01b7d169a10c733391
Upwork Job ID:<PHONE_NUMBER>438637056
Last Price Increase: 2023-08-09

Proposal
Please re-state the problem that we are trying to solve in this issue.
Unable to reset legal name
What is the root cause of that problem?
We are validating the empty string to prevent submitting the form.
What changes do you think we should make in order to solve the problem?
Remove the empty condition(else if) and error setting for both legal first and last name.
https://github.com/Expensify/App/blob/eb020063789091eb2e3086b5b54af3ead8436645/src/pages/settings/Profile/PersonalDetails/LegalNamePage.js#L56-L58
https://github.com/Expensify/App/blob/eb020063789091eb2e3086b5b54af3ead8436645/src/pages/settings/Profile/PersonalDetails/LegalNamePage.js#L62-L64

Normally, we should allow a user to clear fields, but maybe there's a reason why we prevent users from removing it. I don't know the answer, so I asked in Slack!

No response in Slack so I asked one more time.

Still discussing in Slack.

Nobody has raised any concerns around users removing their legal name. Let's move forward with allowing users to remove their legal name after adding it.

Proposal
Please re-state the problem that we are trying to solve in this issue.
Clearing the legal name fields after the initial correct submission is not possible due to the empty form value checks.
What is the root cause of that problem?
The empty form value checks execute on every given input, no matter the context.
What changes do you think we should make in order to solve the problem?

Since we still need the empty form value checks for half-empty submissions (such as a user entering the first name without the last name) we will need to only enforce the empty form value checks when either one of the name fields has a valid value.
What alternative solutions did you explore? (Optional)
Adding a button/clickable text placed under the form that will update and clear the user's legal name via an API write. This approach would be more intuitive for the user, and I am available to produce either result depending on the reviewer's opinion.

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~01c8ed3a25aa63010d

Contributor details
Your Expensify account email<EMAIL_ADDRESS>Upwork Profile Link: https://www.upwork.com/freelancers/~0126f3665fe23ff297
Proposal
Please re-state the problem that we are trying to solve in this issue.
The legal name is removed
What is the root cause of that problem?
Need to remove required validation
What changes do you think we should make in order to solve the problem?
There is a check mark that the legal first name and legal last name, so remove the bellow code
if (_.isEmpty(values.legalFirstName) && _.isEmpty(values.legalLastName)) {
return errors;
}
So it will allow blank value in both fields
What alternative solutions did you explore? (Optional)
N/A

Discussing this in internally.


Update Azure.Provisioning to latest version

Description
Update Azure.Provisioning to the 1.0.0-beta.1 version officially released on nuget.org.
FYI @tg-msft @davidfowl
Checklist

Is this feature complete?

[x] Yes. Ready to ship.

Are you including unit tests for the changes and scenario tests if relevant?

[x] Yes

Did you add public API?

[x] No

Does the change make any security assumptions or guarantees?

[x] No

Does the change require an update in our Aspire docs?

[x] No

Microsoft Reviewers: Open in CodeFlow

This PR is now ready for review.

LGTM even though we have to litter the code with this static helper everywhere 😭

@tg-msft I have a chunk of code I want to delete that does custom interpolation flattening. When will that change be made? Do I need to file a bug?


Unable to add new fields to any Drag and Drop components

I have been using the Drag and Drop components to create a task scheduler and it is working very well. Recently (within the last few weeks), I lost the ability to add new fields to any drag and drop components. I am able to view and select fields, but when I click OK to add them, they do not get added.
I have the same issue for every drag and drop control in my app. I created a new app from scratch and added a drag and drop component and had the same issue.
Has anyone else experience this? If so, is there any known fix? I suspect it may be related to a Power Platform update.

Hallo ich habe das gleiche Problem, kann auch keine Felder hinzufügen. Ich benutze eine Collection

+1
I agree. I believe this issue is related to the Canvas version. It seems that I can no longer change to a version earlier than 3.24054.19.

Thank you for confirming, Rolf and Yaser. Glad it's not just me! Hopefully, it will be addressed soon.

+1

Having the same issue here

+1 Também estou tendo problemas relacionados a isso.

+1

+1!! I am having the same issue too. I thought i did something wrong, but i am not.

+1, having the same issue; cannot add Fields to any components

Same issue here, i cannot add new fields to the component

Same issue, we are also unable to add new fields..is this going to be addressed?

+1 - thought i was going insane. has to be related to the new canvas update

+1 - thought i was going insane. has to be related to the new canvas update

Also - created a brand new app with all preview functionality turned off (i.e. no 'New Analysis Engine') and still get the same behavior. Unable to add fields to a Power Drag and Drop control

This has to do with the new version of powerapps the way colletions work has changed. We are having the same issues. The retired feature that i expect causes this issue "Collection access in component scope". You can enable it in the settings but it does not solve the issue.

Hello - this is due to a regression with the most recent version of Power Apps which - a fix is being worked on - in the meantime you can use the fix described here - https://github.com/microsoft/powercat-code-components/issues/349

Hello - this is due to a regression with the most recent version of Power Apps which - a fix is being worked on - in the meantime you can use the fix described here - microsoft/powercat-code-components#349

Many thanks, fix works as described, and will allow you to add fields


Has anyone successfully run chromedp in Lambda?

What versions are you running?

github.com/chromedp/chromedp v0.1.3
go version go1.12.1

What did you do?
I've written a small headless chromedp utility which runs fine on my laptop (Ubuntu) but it fails in Lambda with this error "fork/exec /usr/bin/google-chrome: operation not permitted".
This appears to be an issue with the restricted Lambda environment
What did you expect to see?
I expected my CLI tool to run successfully.
What did you see instead?
The above error.

chromedp itself would run just fine under AWS Lamba or any other FaaS that supports Go, since chromedp does not have any CGO dependencies or some such. Not being able to launch a process is a limitation of the Lamba FaaS, and not chromedp, as such you should launch/connect Chrome through some other mechanism. Since you are using AWS, I'd suggest using a dedicated EC2 or EKS system for launching the actual Chrome processes.

@kenshaw I'm not explicitly running Chrome but perhaps that's what is happening behind the scenes - I've basically adopted this code and running a slightly modified version of the devToolHandler function: https://github.com/chromedp/chromedp/issues/252#issuecomment-449334679

@matthewcummings chromedp launches an instance (process) of Chrome. Most FaaS platforms are not going to allow the launching of arbitrary processes (ie, Chrome). As such, the way to do this would be to launch Chrome manually on a platform that allows you to launch Chrome, and then configure the Lambda code running chromedp to manually connect to the manually launched Chrome process.

Oh OK, I misunderstood your original response.  Yes, that makes sense to me.  I was planning to go with ECS or EC2 if I couldn't find a workaround for running chromedp in Lambda.  I've been googling this a bit and apparently there are node and Python analogs to chromedp which can run in Lambda.  Not sure what they're doing differently.
Thanks for the replies and for starting this project, Chromedp is an excellent tool.

They'd have the same process launching issue, and they are likely spinning it up separately. If you find some other chromedp analog that can launch a Chrome process on Lambda, feel free to point it out, and I can take a look. Anything that project is doing can be done with chromedp.

I haven't tried any of them yet but these are the tools/guides I've found:
Python:
https://github.com/21Buttons/pychromeless
https://robertorocha.info/setting-up-a-selenium-web-scraper-on-aws-lambda-with-python/
Javascript:
https://github.com/sambaiz/puppeteer-lambda-starter-kit
https://zeit.co/blog/serverless-chrome

I've only looked at the first robertorocha.info blog article, but what they're using is a special build of Chrome against amazonlinux, specifically this project: github.com/adieuadieu/serverless-chrome
You should be able to use that build of Chrome with chromedp in the same fashion, however please note that the version of Chrome currently available on that project is extremely old (at least when measured how by Chrome's rapid development pace).

Excellent, thanks @kenshaw.  It seems I should be able to follow their guide here and build serverless-chrome from a more recent build of Chromium: https://github.com/adieuadieu/serverless-chrome/blob/master/docs/chrome.md
I'm on the fence with this, I'd like to get something running soon and I love Lambda but it might just be easier to Dockerize my CLI tool or just run it on an EC2 instance.

The problem with using the various Chrome builds, is that they're not updated routinely. This is a perfect example where it says it's "updated automatically" but obviously the project stopped getting updates last September. This is also precisely why I have consciously decided not to have any "official" builds of Chrome put out by the chromedp project.

@matthewcummings did you run into any conclusion on this? I'm having the same problem now - not being able to fork the process in chromedp. Although I am using the serverless application repository's version of Chromium layer: https://github.com/pahud/lambda-layer-headless-chromium

@falnyr I gave up on that approach and just went with an EC2 based deployment, however I think it might be as simple as chmodding the chrome binary and making it executable in the Lambda environment. I saw a blog post a while back which implied that this was the underlying issue. I haven't tried it but you should see if that works for you.

@falnyr Did you manage to solve it ?

@mathantunes unfortunately not, I went with the approach of actually using go-rod/rod with headless chrome running in ECS and provisioned based on the size of an SQS queue.

@matthewcummings @falnyr bumped into same issue as you guys. Recently AWS announced support of container images in Lambda, so I thought to give it a go.
Packed my code with zenika/alpine-chrome image, works well locally with docker run, but fails in Lambda with fork/exec /usr/bin/chromium-browser: operation not permitted :(

@AndriiOmelianenko the problem is with the Lambda's runtime interface. You need a container that is compliant with it, then you could essentially swing running the Chrome not even using a container but without the limitations for size etc.
See: https://docs.aws.amazon.com/lambda/latest/dg/images-test.html and https://github.com/aws/aws-lambda-runtime-interface-emulator

@falnyr not quite sure I understand. I have impression that RIE is just webserver-proxy that allows you to test your container-images lambdas locally. Code that you write for lambda (regular or container) is missing webserver part, as you basically just write a Handler that accepts event as parameter.
RIE listens on a port, accepts the payload (json) and passes event to your handler = webserver.
I already created a few container image lambdas (alpine images + some python dependencies + code that calls it) and it works fine with or without RIE. It's kind of optional.
One of the issues I see right now - if you run container image in aws lambda - whole filesystem is ReadOnly, except /tmp

Oh, I'm pretty sure your understanding is better than mine. Although there is a javascript project that manages to run the Chromium executable which I'm not sure if that would be possible without writeable directory outside /tmp 🤷
Anyways, I managed to get it working using https://github.com/go-rod/rod instead with Chromium running on ECS which proved to be very efficient.

For anyone pulling their hair out getting this to work under Lambda, I've managed to get it running using the following parameters:
        ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
	defer cancel()

	opts := []chromedp.ExecAllocatorOption{
		chromedp.DisableGPU,
		chromedp.NoSandbox,
		chromedp.Headless,
		chromedp.Flag("no-zygote", true),
		chromedp.Flag("single-process", true),
		chromedp.Flag("homedir", "/tmp"),
		chromedp.Flag("data-path", "/tmp/data-path"),
		chromedp.Flag("disk-cache-dir", "/tmp/cache-dir"),
		chromedp.Flag("remote-debugging-port", "9222"),
		chromedp.Flag("remote-debugging-address", "<IP_ADDRESS>"),
		chromedp.Flag("disable-dev-shm-usage", true),
	}

	allocCtx, cancel := chromedp.NewExecAllocator(ctx, opts...)
	defer cancel()

	ctx, cancel = chromedp.NewContext(allocCtx, chromedp.WithLogf(log.Printf))
	defer cancel()

	var res string

	err = chromedp.Run(ctx,
		// Your use case
	)

Very nice, thank you for posting @sspruijt !

@sspruijt
I tried it with your settings and got an error. Do you use layers etc.? How do you make it possible to call chromium?
exec: "google-chrome": executable file not found in $PATH


Cherry-pick criticial fixes into 0.28.x branch for 0.28.3 release

Still going through release process, so there may be more necessary to get a full release, but at a minimum this backports fixes onto 0.28.x in preparation for a 0.28.3

Codecov Report

Merging #2073 (62f1752) into v0.28.x (ca93125) will decrease coverage by 0.02%.
The diff coverage is 53.84%.

@@              Coverage Diff              @@
##             v0.28.x    #2073      +/-   ##
=============================================
- Coverage      83.08%   83.05%   -0.03%
- Complexity      2264     2265       +1
=============================================
  Files            322      322
  Lines          10557    10568      +11
  Branches        1048     1049       +1
=============================================
+ Hits            8771     8777       +6
- Misses          1448     1452       +4
- Partials         338      339       +1

Impacted Files
Coverage Δ
Complexity Δ

...io/opencensus/trace/unsafe/ContextManagerImpl.java
100.00% <ø> (ø)
6.00 <0.00> (ø)

.../java/io/opencensus/trace/unsafe/ContextUtils.java
100.00% <ø> (ø)
4.00 <0.00> (ø)

...io/opencensus/trace/unsafe/ContextHandleUtils.java
78.57% <33.33%> (-12.34%)
6.00 <1.00> (+1.00)
:arrow_down:

.../io/opencensus/implcore/stats/MutableViewData.java
92.07% <60.00%> (-1.52%)
4.00 <0.00> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca93125...62f1752. Read the comment docs.

Codecov Report

Merging #2073 (62f1752) into v0.28.x (ca93125) will decrease coverage by 0.02%.
The diff coverage is 53.84%.

@@              Coverage Diff              @@
##             v0.28.x    #2073      +/-   ##
=============================================
- Coverage      83.08%   83.05%   -0.03%
- Complexity      2264     2265       +1
=============================================
  Files            322      322
  Lines          10557    10568      +11
  Branches        1048     1049       +1
=============================================
+ Hits            8771     8777       +6
- Misses          1448     1452       +4
- Partials         338      339       +1

Impacted Files
Coverage Δ
Complexity Δ

...io/opencensus/trace/unsafe/ContextManagerImpl.java
100.00% <ø> (ø)
6.00 <0.00> (ø)

.../java/io/opencensus/trace/unsafe/ContextUtils.java
100.00% <ø> (ø)
4.00 <0.00> (ø)

...io/opencensus/trace/unsafe/ContextHandleUtils.java
78.57% <33.33%> (-12.34%)
6.00 <1.00> (+1.00)
:arrow_down:

.../io/opencensus/implcore/stats/MutableViewData.java
92.07% <60.00%> (-1.52%)
4.00 <0.00> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca93125...62f1752. Read the comment docs.


Use only stable features, allowing compilation in 1.0.0

Can't build it with beta because of #![feature(test)] and#![feature(core)], they don't plan to make it stable in 1.0.

THIS IS IMPORTANT!

@Eilie please note that writing in ALL CAPS on the Internet is considered yelling, and yelling at people isn't very polite. Yes, open-source maintainers are people too. :smile_cat:
That said, I would like to support 1.0, but will have to evaluate exactly what we can do. There have been recent commits that improve the 1.0 compatibility story. One thing that I can say for sure is that the stable version is going to be slower than the unstable version, and we have the ability to make some really good performance gains with gated language features!
I'm also genuinely interested why this compatibility is so important to you. I'd be excited to hear if you are using this library for something real!

@shepmaster I just need XPath library for Rust to get some tag/attribute values out of my XML file and it looks like the only option is https://github.com/shepmaster/sxd-xpath which depends on this library so it would be cool to have it stable because unstable is a bad thing(because it's "unstable" obviously).

@shepmaster Thank you!

No problem! I also updated the XPath library to be compilable with the betas. I hope these libraries serve you well, and I look forward to any pull requests if you experience any problems!


Create item type Armor

Fulfilled, however the display of items on both character sheets are not correct. To close after correction.

character sheet visuals have been improved in 0.1.4


WIP: Working kubeadm setup in Vagrant

This is currently using a branch of https://github.com/GoogleCloudPlatform/terraform-google-k8s-gce over at https://github.com/ii/terraform-google-k8s-gce/pull/1


Disable plugin for conversion of foreign Sprite2D to different types

Disables conversion of foreign Sprite2D nodes to different types since the result will not be properly saved.

Thanks!


Changed TRACK_ID from base16 to base62 so that it is equal with the official Spotify app

This fixes #232

This makes sense but will break setups expecting base16 strings
I therefore suggest merging this in a few days to avoid surprises


Add builds for different versions of psutil

Travis is failing due to OSX issues, but looks like everything is working fine otherwise, and OSX passes locally


Fix build warnings about missing template variables

The .cs.in file had undefined variables and was causing a build warning.

Merging as the changes are minor and tests are passing on CI.

cref https://github.com/aspnet/Coherence-Signed/issues/601


Complete project gallery

Also will take care of #200 as part of this effort.

Doing in https://github.com/okfn/foundation/issues/257


Using thread on a function can't ever really hurt.


[Feature] Add a tool to find broken files.

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
Motivation
After preparing the dataset, the data may be broken. add a tool to find out all the broken files.
Modification
Add a verify_dataset.py tool
BC-breaking (Optional)
No.
Use cases (Optional)
python tools/misc/verify_dataset.py ${CONFIG_PATH}  --num-process ${CPU_TO_USE} --phase ${PHASE} --out-path ${OUT}

Checklist
Before PR:

[x] Pre-commit or other linting tools are used to fix the potential lint issues.
[x] Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
[x] The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
[x] The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

[x] If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
[x] CLA has been signed and all committers have signed the CLA in this PR.

I have used verify_dataset.py to check my datasets, and there were 1885 broken files. My datasets were organized according to ImageNet and they were all jpg images. I wander why they were broken, and I want to know how to handle the 1885 broken files to make them pass the verify_dataset.py.


vazir-fonts: 30.1.0 -> 32.0.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/vazir-fonts/versions.
meta.description for vazir-fonts is: "A Persian (Farsi) Font - قلم (فونت) فارسی وزیر"
meta.homepage for vazir-fonts is: "https://github.com/rastikerdar/vazir-font"
meta.changelog for vazir-fonts is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 32.0.0 in filename of file in /nix/store/9vyvl17fqqrwsad8pv24m21nhm50gqhn-vazir-fonts-32.0.0

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
vazir-fonts

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/9vyvl17fqqrwsad8pv24m21nhm50gqhn-vazir-fonts-32.0.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A vazir-fonts https://github.com/r-ryantm/nixpkgs/archive/e92f338feb1c487a2252a30ab9ed53b43800bfe3.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/9vyvl17fqqrwsad8pv24m21nhm50gqhn-vazir-fonts-32.0.0
ls -la /nix/store/9vyvl17fqqrwsad8pv24m21nhm50gqhn-vazir-fonts-32.0.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

vazir-fonts

Maintainer pings

Build result LGTM, tested with gnome-font-viewer.


MAINT: Improve unit test execution speed

Port of https://github.com/dsgibbons/shap/issues/42
Unit tests currently take ~20 mins to complete. If we could reduce that it could help reduce the time it takes to validate PRs, improving our effectiveness as reviewers.
The pytest logs show the names of the lowest 20 tests, which is a good way to prioritise tests to improve.
TODO

[ ] (Ongoing) refactor the slowest tests to run faster
[x] Cache the installation of the dependencies, which currently take several minutes from each test job.
[x] Investigate parallelising the tests to run on multiple CPUs.

WIP attempt abandoned as it seems to actually make the tests run even slower!

[x] Get CodeCov working, to help us evaluate relevant test coverage when reviewing PRs.

After #3286 , we have unfortunately lost the ~5min speedup of our test suite. I'm keen to investigate a suitable replacement translation model that we can use in the unit tests.
A few ideas come to mind:

We could pick an existing model from HuggingFace, looking for one with a small file size
We could train a small model from scratch as a fixture, including all required code in this repo
We could publish a dedicated model to HuggingFace that is specifically designed for lightweight testing.

I currently favour option 3, as training a translation model requires potentially a fair bit of code that is largely irrelevant to shap.


AddressRegistrationTests failures on OSX

[RegisterAddresses_IPv6ScopeId_Success(addressInput: "http://[fe80::1%1]:0/", testUrls: Func`2 { Method = System.String[] GetTestUrls(Microsoft.AspNetCore.Hosting.Server.Features.IServerAddressesFeature), Target = null })] Microsoft.AspNetCore.Server.Kestrel.Transport.Libuv.Internal.Networking.UvException : Error -49 EADDRNOTAVAIL address not available
[RegisterAddresses_IPv6ScopeId_Success(addressInput: "http://[fe80::14b8:20f9:2ac2:d22b%4]:0/", testUrls: Func`2 { Method = System.String[] GetTestUrls(Microsoft.AspNetCore.Hosting.Server.Features.IServerAddressesFeature), Target = null })] Microsoft.AspNetCore.Server.Kestrel.Transport.Libuv.Internal.Networking.UvException : Error -49 EADDRNOTAVAIL address not available
[RegisterAddresses_IPv4_Success(addressInput: "http://aspnetci-mac.guest.corp.microsoft.com:0/", testUrls: Func`2 { Method = System.String[] GetTestUrls(Microsoft.AspNetCore.Hosting.Server.Features.IServerAddressesFeature), Target = null })] System.Net.Sockets.SocketException : Network is unreachable

Dupe of #1729 ?

The last error is.

Dup of https://github.com/aspnet/KestrelHttpServer/issues/1733

@davidfowl Have you seen this kind of failure since #1754 was merged?
I created that PR to fix the EINVALs and EADDRNOTAVAILs, but it is really odd to me that binding to "http://:0/" would fail with EADDRNOTAVAIL considering there must be some port available.

Seems to work now, you can close.

Seems to pass now, you can close.