JOSS review: Automated tests

@miniufo - well done on your submission to JOSS.
As it stands, your package doesn't meet the following requirement on the reviewer checklist:

Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?

(See https://github.com/openjournals/joss-reviews/issues/5510 for details.)
You've got some test code available at tests/ and associated data files at Data/, but there is no guidance provided on how to run those tests manually or better still there is no automated testing.
In addition, it looks like your test code culminates in the plotting of figures. While that can be useful for visual verification, common practice would also include a suite of tests that have the following components:

a fixture, which is the thing being tested (e.g., an input data array);
an actual result, which is what the xinvert package produces when given the fixture; and
an expected result that the actual result is compared to.

The comparison between the actual result and expected result would take the form of an assertion, so that the test can be said to pass or fail. You can then use a test runner like pytest in conjunction with GitHub actions to setup automated testing.
You may already be familiar with these testing concepts, but if not let me know and I can point you to some resources that explain how to setup automated testing.

You are right, I know this concept, probably using assert keyword in the program, but not every detail.  Could you please send me some concrete examples that I can follow?
Since it is a numerical solver, I cannot expect the repeated runs of the codes reproduce the same results.  I only visualize the plots to see if they are close to those plots in the related papers.

This chapter in Research Software Engineering with Python gives an overview of general testing concepts:
https://merely-useful.tech/py-rse/testing.html
This page gives more information on using pytest:
https://realpython.com/pytest-python-testing/
In terms of a concrete example, here's a test file I wrote recently:
https://github.com/climate-innovation-hub/qqscale/blob/master/test_qdm.py
While you can't expect repeated runs to reproduce exactly the same results, you should hopefully be able to create a fixture that produces an expected result within some tolerance (i.e. assert np.allclose(expected_result, actual_result)).
Once you've got your tests setup, you can use GitHub Actions to run the tests every time you make a commit to your repository. That involves creating a tests.yml file in a new .github/workflows/ directory. It will look something like:
https://github.com/climate-innovation-hub/qqscale/blob/master/.github/workflows/tests.yml
Hope that helps :smile:

Thanks very much for your kind help.  I will follow these to make auto-tests for xinvert soon.

Hi @DamienIrving, I just followed your guide and made a action for tests.  Everything works fine locally on my laptop.  But the action build raised an error: did not find a match in any of xarray's currently installed IO backends ['netcdf4', 'scipy']
I made both netcdf4 and scipy in the requirement.txt file, so I don't know how to resolve this.  Could you help me find why this is happening?
PS: I use git lfs this time because some of the sample netcdf files are larger than github limits.

If you add an extra step in tests.yml after the "install dependencies" step to list the installed packages you'll see that netCDF4 is successfully installed, so that isn't the problem.
      - name: List installed packages
        shell: bash -l {0}
        run: pip list

It might be that the files are too large... you could try a test with a very small data file?

Hi @DamienIrving, thanks for your tries and suggestion.  I guess I found the reason.  All netcdf files in Data has been tracked by LFS, which are then not recognized by xr.open_dataset.  I modified a small nc file but leave it outside of LFS, then it is passed (you may check the log).
So this is really embarrassing, because I need some global dataset to make a test, which exceeds the limit by github.  LFS seems to upload big file transparently but not openable online.  Do you have any suggestion?

It sounds like you need to refactor the tests you want GitHub Actions to run so they don't require large input datasets.
For automated testing, it's most common to either have the test code create it's own fake data (e.g. https://github.com/climate-innovation-hub/qqscale/blob/master/test_qdm.py#L15) so that you don't need to include any data files in your repo, or if using "real" data is unavoidable then it's common to use a netCDF file containing only a single grid point or small spatial area.
You can do your own testing offline where you process and plot large global datasets, but those offline tests wouldn't be included in the tests run by GitHub Actions.
Does that make sense?

Well, that's right.  I should skip those tests requiring large datasets (>100MB), and do them locally by hands.

Hi @DamienIrving, all the pytests has been passed.  So please see if this issue is fixed now.  Thank you very much for you kind help with this.

Thanks for implementing automated testing.
Just so I understand, what are the tests in test_Poisson.py checking?
I would expect that each test would have three basic steps:

Read in vorticity and divergence data from a file (that's the test "fixture")
Calculate the streamfunction and velocity potential (that's the "actual result")
Compare the actual result to an "expected result" to see if the test passes (i.e. using an assertion)

Do you have an expected result? If not, is there some vorticity and divergence data that you can create (i.e. simple example data) where you know what the streamfunction and velocity potential values should be?

Hi, @DamienIrving thanks for your careful check on this.  You are right that one could find some analytical results to do the tests.  But here I adopted an indirect way here.  Basically, we are not checking the streamfunction itself, but its gradient (or velocity components).
According to Helmholtz decomposition theory, a vector field, say, $\mathbf u=(u, v)$, can be decomposed into rotational and divergent components $\mathbf u = \mathbf u_r + \mathbf u_d$.  The rotational component fully recovers the vorticity of the original $\mathbf u$ and has exactly zero-divergence, whereas the divergent component fully recovers the divergence of $\mathbf u$ and has exactly zero-vorticity.  So we can theck this to see if the decomposition (i.e., Poisson inverion) is right.
That is why I put asserts to see if the divergence of $\mathbf u_r$ is zero, and if the vorticity of $\mathbf u_d$ is zero, which are expected results.  The slicing from 1 to -1 is because the finite difference at the boundaries (north-south poles) could result larger errors above the tolerance so it is better to check the interior.  But zonal is periodic so not a problem here.
One of the potential problem with analytical expression is that, we use finite difference scheme here.  If we use larger grid size, there would be grid-scale truncation error relative to analytical results, which may be above the tolerance.  But the current asserts do not depend on grid size, because the inversion for streamfunction, as well we the calculation of its gradient or velocity components, all use the consistent finite difference scheme, which turns out to be quite good that the expected $\nabla\cdot\mathbf u_r=0$ is achieved with default numpy tolerance.
Also, there will be an arbitary constant to Poisson solution.  So any streamfunction, after adding a constant, is still a solution to Poisson equation.  I guess this could be a problem with windspharm to determine the arbitary constant.  That is why I do really prefer the indirect way of assertion here, as it avoid the arbitary constant.

Thanks for the explanation - that's a nice way to write the tests!


CIS Benchmark: Scan Create form shows empty content

Steps to reproduce

Install CIS Benchmark cluster tool helm chart
Cluster --> Tools --> install CIS Benchmark --> refresh
Navigate to resource menu CIS Benchmark --> Scans --> Create

Actual behavior
No content renders on page. No errors or warnings in log.
Expected behavior

From Epic:

I've added QA backlog task to explore the chart install automation effort as I didn't find a CIS covered in the current code
https://github.com/rancher/qa-tasks/issues/1547

Moving again to closed as per information from previous PR and QA.


Evaluating {{ .Namespace }} in a gotmpl file no longer works since v0.126.0

I have a line in one of my values files for a release like this:
  service:
    local:
      annotations:
        metallb.universe.tf/allow-shared-ip: {{ .Namespace }}

It gives this error:
ailed to render values files "myapp.yaml.gotmpl": failed to render [myapp.yaml.gotmpl], because of template: stringTemplate:18:48: executing "stringTemplate" at <.Namespace>: can't evaluate field Namespace in type state.releaseTemplateData

I checked out v0.125.0 and v0.125.9 and they don't have this issue. It must be something that changed in v0.126.0. It is also present in v0.128.0.
Maybe https://github.com/roboll/helmfile/pull/1415 is the source of the regression?

@mumoshu Can you take a look at this?

@mattymo Thanks for reporting! I think #1466 should fix this one.


Bump version and change download button to jitpack

Reason for change
In order to simplify the release process for this project, it may be a good idea to switch from bintray to jitpack. Then, in order to release a new version, the following steps are required:

the version number must be bumped
a corresponding tag for the version must be created

Then jitpack automatically builds the release package and provides it as dependency.
Additionally i bumped the version number to 0.6.0 as there are significant changes since the release of version 0.5.0. E.g. now it is supported to return collections without wrapper object.
Disadvantages
Jitpack does not use the correct groupId to reference the artifact. Thus, in build.gradle the project must be referenced as implementation 'com.github.nielsfalk:ktor-swagger:0.5.0' instead of compile 'de.nielsfalk.ktor:ktor-swagger:0.5.0'

LGTM!

I just pushed tag v0.6.0 to the repo.
Thanks!!


[UI] Finalise Provider-UI Lint

Signed-off-by: kushthedude<EMAIL_ADDRESS>Fixes #696

Finalises ESlint for Provider-UI.

Review after #797

LGTM!

So far so good here, I think, but more left to tie-off, right?

Yeah some parts are left off, completing by today!

@leecalcote It is good to go now !


etherpad not support rtl languages (like Hebrew and Arabic)

The text in etherpad align to left but Hebrew and Arabic written from right to left and them need to align to right.
I think that need to add button align text to right and set direction rtl.

I'm not a geek but I am assuming this issue should not be rocket science and would make Etherpad so much more relevant for RTL users. Any chance of getting a coder on it? Maybe using an existing language detection lib or something? A quick search found this: http://plugins.jquery.com/project/direction-detector

I think that GMail in rich text mode uses contentEditable (though i might be wrong) and RTL support there is better than in EtherPad. Not perfect, but better.

solved in my tree: https://github.com/slim/etherpad-lite/commit/7f38efbaff5604b27948f0e43edb4d91858533f6
it's a little bit hackish but it works. At least in chrome.
@mushon Y U NO ASK ME ;)

Awesome!
I should've asked you… :)
BTW, I just came across ICE from the NYTimes, might be another solution for a similar need:
http://nytimes.github.com/ice/demo/
How are you doing?
--
Mushon Zer-Aviv
Mushon.com (http://Mushon.com) | Shual.com (http://Shual.com) | @Mushon (http://twitter.com/mushon)
On Thursday, February 9, 2012 at 3:59 AM, Slim Amamou wrote:

solved in my tree: https://github.com/slim/etherpad-lite/commit/7f38efbaff5604b27948f0e43edb4d91858533f6
it's a little bit hackish but it works. At least in chrome.
@mushon Y U NO ASK ME ;)

Reply to this email directly or view it on GitHub:
https://github.com/ether/pad/issues/243#issuecomment-3880534


Where can I file an issue for the dotnet SDK?

I've been looking everywhere and I cannot find the Github repo for Microsoft.Azure.Kusto.Data. I'm guessing is not open source?
If not, where can I report a bug?

https://github.com/Azure/azure-kusto-dotnet/issues


Desktop layout interim solution

The pages look really janky on desktop. http://disclosure.caciviclab.org/#!/committee/1 Let's just come up with an interim solution. We're not using any of the bootstrap grid on the main app. Even making the sections 1/2 or 1/3 width would be a little better.
If we could get a final desktop design, that would be even better :)
cc @elinaru @sseiter

I must be showing my East Coast origins... I had to look up janky :smile_cat:

I grew up in NJ....

Eh... throw me a bone @adborden ! :wink:

@adborden I think you meant "hella janky"

On Wed, 2016-01-27 at 22:50 -0800, Tom Dooner wrote:

@adborden I think you meant "hella janky"

:D
--
Aaron D Borden
Human and Hacker

Working on mock-ups for this now --

We are looking good on desktop https://projects.invisionapp.com/share/TK8O67TEZ#/screens/190014301
I think we can close this, right @elinaru ?


How to transform response body using CLI?

Hi,
I'm checking out this library and is really easy to use! I've been able to run the server using the module approach (jsonServer router, middlewares, etc.).
However I'm trying to achieve the same thing using only the CLI, but I'm stuck trying to transform the response to add custom data to it. Using the module approach you can simply override the router.render method, but with the CLI, I can just provide middlewares but I can't transform the response in there.
Any ideas how I can achieve this?

Hi @christophercr,
Thanks for the feedback :)
I'd say it's not possible and not sure if it's something wished (i.e trying to make the CLI as flexible as Express code to save a few lines of code).
That would mean having either a --renderer custom.js option or having some kind of json-server.config.js where you would export an option object.
But the downside of a config file is that you need to learn to use it, instead of just using your existing Express knowledge.

Alright, indeed it would add more complexity to the CLI and the learning curve would be even higher.


Run of the auto formatter (#48)

This pr closes #48.

Most of the contest code just looks even worse, particularly the ones with the old dbg template. Need to fix that at some point


fix: The issue of ambiguous language reply solved

Purpose
To resolve the bug for language ambiguity when said Hi to the bot the reply is in spanish
Does this introduce a breaking change?

[x] Yes
[ ] No

How to Test

Get the code

git clone [repo-address]
cd [repo-name]
git checkout [branch-name]
npm install

What to Check
Verify that the following are valid
When bot is communicated in particular language it replies in the language itself

@Harmanpreet-Microsoft please see https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator/pull/1149#issuecomment-2235692030. It needs addressing before this PR can be approved and merged. Thanks

@microsoft-github-policy-service agree company="Microsoft"

@microsoft-github-policy-service agree company="Microsoft"

Hi @adamdougal
I have completed the needful task


Train station

About the PR
added some miscellaneous trivia. Some of it is just a convenience for mapping, some of it is specifically for the new station.

Added new "Train lattice" that cannot be created, they have the full functionality of regular lattice, and cause players to experience new ambient sound effects in a space train theme.
Added space train sound effects.
Added 2 new types of portals using the Gateway portals sprite. Needed for mapping purposes.
The statues have now gotten noRot: true, and rotate beautifully with the camera.
Added a blue comfy bench and construction recipe for it.
All colored poweredLight lamps (crystal lamps, sodium lamps, LEDs) now glow the right color on mapping (previously all were white on an uninitialized map).
Added stained glass windows and construction recipe for them. [Deleted]
Added cool parallax with meteorites flying down. For the new station.
FloorHullReinforced now Indestructible: true (used only on Pirate shuttle before). Made to make it harder to separate the train with explosions

Added Train Station
not added to rotation. Only playtest.
Media

Render:
Train-0
Changelog
:cl: TheShuEd, Vsem, Jeb

add: Added Train station! Only for Playtest

want art director opinion on the windows imo

Deleted stained window

Space trains?
Sunless Skies pr based

what is that background parallax

Shit, no

walking simulator map

God we need transit tubes from ss13 badly for bigger maps

we have that its called disposals

god forbid you accidentally bump one of the windoors when going between sections :trollface:

we have that its called disposals

disposals are loud and steel transit pipes are quiet and glass
also transit pipes have diagonals (impossible)

Is this actually gonna be a station? One syndie bomb and the whole thing is getting cut in two. I love it

idk if its just maprender tripping but there are no grilles on the meteor shields so they wont do anything

Its ok, WIP

minplayers: 10

>no maxpop
>potentially highpop map with 8 botany trays
>2 botanists
>8 trays
botany is already squabbling over tray space with 10 trays on packed and 9 on saltern

Alright we're gonna need space lube bottles and make a whole line from top to bottom
Let's see how that works for fast travel :trollface:

lube express on the side , big line of lube puddles with a break between each section for you to get off

Updated render

New engi
New maints, new disposal travel pipes
Many fixes

You left a safe in the armory unlocked

Also the west maints in the bridge are extremely undefended, just one depth charge or c4 and boom, free entry to caps locker, suit, spare ID and Renault if they are buckled to their bed assuming that it is a normal syndie, imagine nukies or even solo-op with a china lake in space :godo: . Maybe make it 2 layers of walls and glass on both sides and change the maints doors into command doors?

So, I just remembered something. Bagel's AI core has reinforced plating over a disposals pipe, which makes the disposals pipe completely untamperable.


Fixing cargo test warnging message

Many warning message without this patch. this is on master branch.

[me@rocky9t01a demo]$ cargo test
warning: unused import: `super::prepare_data`
 --> tests/requests/notes.rs:1:5
  |
1 | use super::prepare_data;
  |     ^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

warning: unused import: `axum::http::HeaderValue`
 --> tests/requests/notes.rs:2:5
  |
2 | use axum::http::HeaderValue;
  |     ^^^^^^^^^^^^^^^^^^^^^^^

warning: function `compression_header` is never used
  --> tests/requests/prepare_data.rs:56:8
   |
56 | pub fn compression_header() -> (HeaderName, HeaderValue) {
   |        ^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(dead_code)]` on by default

warning: `blo` (test "mod") generated 3 warnings (run `cargo fix --test "mod"` to apply 2 suggestions)
    Finished test [unoptimized + debuginfo] target(s) in 0.34s
     Running unittests src/lib.rs (target/debug/deps/blo-38b25b6eccb26803)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running unittests src/bin/main.rs (target/debug/deps/blo_cli-ec5a0cbf488290d8)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/cli_tests.rs (target/debug/deps/cli_tests-54fc97a4f9c08f46)

running 1 test
test cli_tests ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/mod.rs (target/debug/deps/mod-2b269e28e404685f)

running 26 tests
test models::users::can_reset_password ... ok
test models::users::can_find_by_pid ... ok
test models::users::can_set_forgot_password_sent ... ok
test models::users::can_find_by_email ... ok
test models::users::can_create_with_password ... ok
test requests::auth::can_login_without_verify ... ok
test requests::auth::can_login_with_verify::case_1 ... ok
test models::users::test_can_validate_model ... ok
test requests::auth::can_reset_password ... ok
test requests::notes::can_delete_note ... ok
test requests::auth::can_login_with_verify::case_2 ... ok
test requests::notes::can_get_note ... ok
test requests::notes::can_get_note_gzip ... ok
test requests::auth::can_register ... ok
test requests::notes::can_get_notes::case_3 ... ok
test models::users::can_verified ... ok
test models::users::handle_create_with_password_with_duplicate ... ok
test requests::notes::can_get_notes::case_2 ... ok
test requests::notes::can_get_notes::case_4 ... ok
test requests::notes::can_get_notes::case_5 ... ok
test requests::user::can_get_current_user ... ok
test models::users::can_verification_token ... ok
test requests::notes::can_add_note ... ok
test requests::notes::can_get_notes::case_1 ... ok
test tasks::seed::test_can_seed_data ... ok
test requests::user::can_get_current_user_with_api_key ... ok

test result: ok. 26 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 19.00s

   Doc-tests blo

running 1 test
test src/tasks/seed.rs - tasks::seed (line 14) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.21s

[me@rocky9t01a demo]$

closing this PR since it has another commit beside the real one.


mget

i can‘t find "RedissonClient.loadBucketValues()"  in version 3.16.0

I read the Chinese document, the Chinese document and the English document are inconsistent


Azima: Workflows: week 2

[ ] I wrote my markdown locally on a new branch
[ ] My bio file is named exactly the same as my GitHub username
[ ] My bio link in the class README opens to my student bio file
[ ] I pulled and merged main into my branch before pushing
[ ] I pushed my branch to the class repository and opened a PR
[ ] I went through the PR checklist
[ ] All the CI check pass
[ ] I assigned a coach to review my PR
[ ] I
linked my PR to this issue

For the label: You are in week 1 of the module 'workflow', next week you will use the label 'week-2'.


Remove some empty lines from generated config files

Use Jinja2's stripping abilities to reduce the number of empty lines in generated files

Awesome! :beers: :panda_face:


Cherry pick changes from #25456

Fixes the license check

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

LGTM

Self hosted runners are down which is blocking the checks here, I'm not going to block on that since this clearly fixes the problem


Client Allocation Request for: Legend Corp

Client Allocation Questions
Core Information

Name: Legend Corp
Website / Social Media: https://aiqicha.baidu.com/company_detail_18439124798299
DataCap Requested: 5TiB
Addresses to be Notarized: f1dgiaypmdsubmonsxf6hd3vs2cszr7ib7rwrpqmi
Notary Requested: Rin-huang

Client Allocation Questions
Core Information

Name: Legend Corp
Website / Social Media: https://aiqicha.baidu.com/company_detail_18439124798299
DataCap Requested: 5TiB
Addresses to be Notarized: f1dgiaypmdsubmonsxf6hd3vs2cszr7ib7rwrpqmi
Notary Requested: Rin-huang

We have just published our Client onboarding guidelines (Filecoin plus notary) on GitHub. Please check our link https://github.com/filecoin-project/filecoin-plus-client-onboarding/tree/main/Fenbushi Capital.


TensorFlow 2.0.0-rc1

The release candidate 1 of TensorFlow 2.0.0 has been tagged:
https://github.com/tensorflow/tensorflow/tree/v2.0.0-rc1
PyPI has packages for all the betas, but not this release candidate:
https://pypi.org/project/tensorflow-rocm/#history
Is there a package coming up?

Hi @Bengt , yes, we're planning to publish the PyPi packages when ROCm2.8 is out.
The package will be built using the following branch:
https://github.com/ROCmSoftwarePlatform/tensorflow-upstream/tree/r2.0-rocm-rc1
You can build it from source if you want to try it earlier.

Hi @sunway513, thanks for the pointer, I will give that branch a spin on ROCm 2.7.

@Bengt , we have released the tensorflow-rocm 2.0.0 whl package, please try it out and let us know if you have any feedback.
Thanks!

Awesome! Thanks for the update. I will give that a spin next week.
Peng<EMAIL_ADDRESS>schrieb am Fr., 11. Okt. 2019, 03:40:

@Bengt https://github.com/Bengt , we have released the tensorflow-rocm
2.0.0 whl package, please try it out and let us know if you have any
feedback.
Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ROCmSoftwarePlatform/tensorflow-upstream/issues/641?email_source=notifications&email_token=AAAC2NY6Y3DUTQSWZUQAIATQN7KQTA5CNFSM4IYAOAE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEA6O3JY#issuecomment-540863911,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAC2N4LXANPVNJYDJDBGA3QN7KQTANCNFSM4IYAOAEQ
.

@sunway513 Congrats! Could you give a hint with which ROCM version it was compiled with, always some guess work to do to get the pip and ROCM version right.
Yes, I know there are docker images :)

@sunway513 what is the branch/tag of the newly released whl? :)

@sunway513 Congrats! Could you give a hint with which ROCM version it was compiled with, always some guess work to do to get the pip and ROCM version right.
Yes, I know there are docker images :)
Ah, got it to work with ROCM 2.8!

Thanks!
TF2.0.0 whl package was built with ROCm2.9 user bit, it might also work with ROCm2.8.
I'm working on a document that specifies the ROCm compatibility against our released whl packages on PyPI, will keep you posted.

@witeko , you can find the release tag here:
https://github.com/ROCmSoftwarePlatform/tensorflow-upstream/releases/tag/v2.0.0-rocm
Please note the source code is pretty much identical to what we have been able to upstreamed to official tensorflow r2.0 release branch, other than minor cleanups to build with ROCm2.9 user bit packages:
https://github.com/tensorflow/tensorflow/commits/r2.0

Hi @kinred , please refer to the following doc that specifies the tensorflow-rocm ROCm build version and the compatibility:
https://github.com/ROCmSoftwarePlatform/tensorflow-upstream/blob/develop-upstream/rocm_docs/tensorflow-rocm-release.md


moduleExtensions should enforce -loader, not duplicate.

I'm submitting a...

[x] bug report

Webpack version:
2.1.0-beta.13
Please tell us about your environment:
OSX 10.11.5
Current behavior:
When loaders are defined with -loader (e.g. babel-loader), because the default value of moduleExtensions is -loader, module resolution attempts to find babel-loader-loader, which causes an erroneous npm install with this plugin:

https://github.com/ericclemmons/npm-install-webpack-plugin

Expected/desired behavior:
Expected that that this enforces the -loader extension, so that babel becomes babel-loader.  It should noop if /-loader$/ matches.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem along with a gist/jsbin of your webpack configuration.

I believe this is due to these two files:

https://github.com/webpack/enhanced-resolve/blob/d154f01b6f74b90f3ac3c5ea6cca89abead5236c/lib/ResolverFactory.js#L164-L169
https://github.com/webpack/enhanced-resolve/blob/master/lib/ModuleAppendPlugin.js

Can you check this against webpack 2 rc?

Please re-open if you see this still.


Event Request Page 10120 "Bank Rec. Worksheet" OnBeforeSetupRecord

Hello, could you please add the event OnBeforeSetupRecord to the procedure SetupRecord on page 10120 "Bank Rec. Worksheet" like this?
procedure SetupRecord()
var
    IsHandled: Boolean; // new change
begin
    IsHandled := false; // new change
    OnBeforeSetupRecord(Rec, IsHandled); // new change
    if IsHandled then // new change
        exit; // new change
    SetRange("Date Filter", "Statement Date");
    CalcFields("Positive Adjustments",
               "Negative Adjustments",
               "Positive Bal. Adjustments",
               "Negative Bal. Adjustments");
end;

// new change
[IntegrationEvent(false, false)]
local procedure OnBeforeSetupRecord(var BankRecHeader: Record "Bank Rec. Header"; var IsHandled: Boolean)
begin
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 53382.


btrbk: 0.28.0 -> 0.28.3

Motivation for this change
fixing updates @r-ryantm  cannot.
Things done

[X] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
[X] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

Notify maintainers
cc @the-kenny
$ nix path-info -Sh ./result
/nix/store/jsv5rvjmj8b3pma6kdnwzq0gvq1155y2-btrbk-0.28.3         157.2M
$ nix path-info -Sh ./result
/nix/store/g6l7dcycr9ssdgca2y9qsnnbiqh4586p-btrbk-0.28.0         157.2M

[1 built, 0.0 MiB DL]
1 package were build:
btrbk

@GrahamcOfBorg build btrbk


{advisor} Bug fix recommendation list

Related command
az advisor recommendation list
Description
Fixes error while executing az advisor recommendation list cmdlet.
Fixes 25657
Testing Guide
az advisor recommendation list --category Cost --refresh
History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change
[Component Name 2] az command b: Add some customer-facing feature

This checklist is used to make sure that common guidelines for a pull request are followed.

[x] The PR title and description has followed the guideline in Submitting Pull Requests.

[x] I adhere to the Command Guidelines.

[x] I adhere to the Error Handling Guidelines.

advisor


compress/lz4: Use default Compressor for Fast LZ4 level

I was wondering why the LZ4 compression was so much slower than Snappy, even in Fast mode (almost x5 slower). I found this issue: https://github.com/pierrec/lz4/issues/60 on LZ4's repo, which points out to the fact that using lz4.CompressBlockHC is slow and lz4.CompressBlock should be used.
So this PR does just that. It uses the default Compressor (lz4.Compressor) in Fast mode, and lz4.CompressorHC otherwise.
I'm not quite sure whether a compression level is actually necessary or wanted, I haven't found any reference of it in Parquet docs: https://github.com/apache/parquet-format/blob/master/Compression.md
Maybe there should rather be another Enum level for the Default Compressor?

CompressorHC gives  better compression ratio, switching to Compressor sacrifices this. We will need benchmarking numbers to justify this change.
Please share your user case and scenario where you think Compressor suits best. By default best compression ratios makes sense.

Agreed, CompressorHC is the best compression ratio, but it's far from the "best", as its compression speed can be very low compared to Compressor. I added a commit to my PR (which shouldn't get merged as is), which adds a couple of test data from https://github.com/klauspost/compress/tree/master/testdata for the compression tests and benchmarks. I added different levels of LZ4 HC compression, and added a new level called Fastest to use a Compressor.
Here are the results, from running
go test -benchmem -bench "BenchmarkEncode/(lz4|snappy)" ./compress/ -v

Compression Ratio
| e | gettysburg | html | Tom Sawyer | Pi | PNG
-- | -- | -- | -- | -- | -- | --
Snappy | 77.84% | 75.32% | 36.45% | 53.43% | 78.05% | 8.78%
LZ4 – Fastest | 100.39% | 85.47% | 41.04% | 58.46% | 98.14% | 5.34%
LZ4 – Fast | 69.52% | 76.23% | 35.11% | 48.69% | 69.58% | 4.64%
LZ4 – L1 | 69.52% | 76.23% | 35.13% | 48.69% | 69.58% | 4.88%
LZ4 – L5 | 69.52% | 76.23% | 35.11% | 48.69% | 69.58% | 4.70%
LZ4 – L9 | 69.52% | 76.23% | 35.11% | 48.69% | 69.58% | 4.64%
Compression Speed
| e | gettysburg | html | Tom Sawyer | Pi | PNG
-- | -- | -- | -- | -- | -- | --
Snappy | 248.71 MB/s | 612.96 MB/s | 730.10 MB/s | 255.77 MB/s | 240.25 MB/s | 1976.59 MB/s
LZ4 – Fastest | 12243.05 MB/s | 379.48 MB/s | 677.99 MB/s | 249.67 MB/s | 395.64 MB/s | 3147.58 MB/s
LZ4 – Fast | 59.75 MB/s | 15.67 MB/s | 52.50 MB/s | 27.20 MB/s | 58.64 MB/s | 6.11 MB/s
LZ4 – L1 | 58.54 MB/s | 15.74 MB/s | 57.35 MB/s | 27.10 MB/s | 58.96 MB/s | 155.30 MB/s
LZ4 – L5 | 58.59 MB/s | 15.58 MB/s | 52.76 MB/s | 26.48 MB/s | 59.23 MB/s | 14.41 MB/s
LZ4 – L9 | 60.27 MB/s | 15.65 MB/s | 53.74 MB/s | 26.26 MB/s | 58.16 MB/s | 6.17 MB/s
  |   |   |   |   |   |
Speed up (Fastest vs. Fast) | 204.90 | 24.22 | 12.91 | 9.18 | 6.75 | 515.15
We can see that the compression ratio is only worse in very random data (Pi and e), but the compression speed as at least 10x faster, up to 500x
If these results are appealing, let me know what would be the best way to integrate this new "compression level". A new Fastest level might be the best solution, as it doesn't change the default; although it could be argued that the default should actually be this level.

Changing the default could be considered a breaking change, so adding a new level might be a better option.

Any update on this feature ? :slightly_smiling_face:

I believe this is backwards compatible and will continue to read old files correctly. So we're good. Seems like a solid perf improvement.


Supporting LargeUtf8 and LargeBinary for avro schema

convert LargeUtf8 and LargeBinary schema to avro::String and avro::Bytes when converting from datatype to avro schema.
adding a test for LargeUtf8 and LargeBinary io

Woops, forgot to convert the schema :/ Many thanks, @illumination-k !


Add support for awaiting IAsyncOperation from UnityEngine.Resource…

The new AddressableAssets plugin currently returns IAsyncOperation for many operations which can dynamically load assets.  Currently they don't have formal async/await support though it is on the roadmap.  This PR adds GetAwaiter support for IAsyncOperation.

Thanks


⚠ Samsung has degraded performance

In cb72bf0, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2205 ms

Resolved: Samsung performance has improved in e293ee1.


Retrieve pull requests by date range

It seems hub pr only allows to retrieve the last X (given by -L X) pull requests filtered by state.
It would be great (if the github API allows) if we could retrieve all PRs that were opened/closed/... within a given date range.

This would be nice, especially because it would match the capabilities of the web view too. Unfortunately, it's not possible using the REST API yet. See https://github.com/github/hub/issues/2079#issuecomment-472507867 for explanation why we would need to switch to using Search for this

Agree, we need date filters by either of the following
createdAt
closedAt
updatedAt
or
mergedAt
This is my graphql query, cannot find a way out to filter the data
query query($owner: String!, $name: String!) {   repository(owner: $owner, name: $name) {     pullRequests(first: 50, states: MERGED, orderBy: {field: UPDATED_AT, direction: ASC}) {       edges {         node {           author {             login           }           createdAt           closedAt           updatedAt           mergedAt           id           title           url           comments {             totalCount           }           additions           changedFiles           deletions         }         cursor       }     }   } }

@namanbada0606
{
  search(first: 100, query: "repo:github/hub is:pr is:open created:2019-10-31..2020-01-01", type: ISSUE) {
    nodes {
      ... on PullRequest {
        title
        url
      }
    }
  }
}

Thank you @mislav  for a workaround solution, that stuff worked for me, Ideally i wish to have it at pullRequests level.
really appreciate your timely help. You are awesome!!


ElasticSearch connector does not work against AWS-ES service

It appears that the connector does not work with ES as managed by aws. Although the initial connect appears to be succesful (using endpoint at port 443), the first call after makes it disconnect again:
2016-05-12 12:40:49,485 @main INFO ElasticsearchEmitter ElasticsearchEmitter using elasticsearch endpoint search-<...>.eu-west-1.es.amazonaws.com:443 ElasticsearchEmitter.java(118)
2016-05-12 12:40:49,640 @main INFO plugins [Leap-Frog] loaded [], sites [] PluginsService.java(151)
2016-05-12 12:41:44,595 @LeaseCoordinator-1 INFO LeaseTaker Worker cfe43d56bde02857:-651163f0:154a48db5ff:-8000 saw 36 total leases, 35 available leases, 1 workers. Target is 36 leases, I have 1 leases, I will take 35 leases LeaseTaker.java(403)
2016-05-12 12:42:51,789 @main INFO transport [Leap-Frog] failed to get node info for [#transport#-1][pbox-VirtualBox][inet[search-<...>.eu-west-1.es.amazonaws.com/<IP_ADDRESS>:443]], disconnecting... TransportClientNodesService.java(398)
org.elasticsearch.transport.SendRequestTransportException: [][inet[search-<...>.eu-west-1.es.amazonaws.com/<IP_ADDRESS>:443]][cluster:monitor/nodes/info]
at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:286)
at org.elasticsearch.transport.TransportService.submitRequest(TransportService.java:243)
at org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler.doSample(TransportClientNodesService.java:376)
A similar issue is reported @ https://github.com/awslabs/cloudwatch-logs-subscription-consumer/issues/9
Perhaps the elasticsearch-hadoop (spark) receiver gives some hints how to approach this: https://www.elastic.co/guide/en/elasticsearch/hadoop/current/reference.html; specifically the option:
conf.set("es.nodes.wan.only", "true"); (https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html)

I found that the ElasticsearchEmitter uses the TransportClient from ES; since the aws es service does not support this client the emitter should be rewritten to work with a client using the rest/api instead.


Error from feature code lookup function

This bug is very weird. This sentence runs fine:
geo.geoparse("Congress and in the legislatures of Alabama, California, Florida, and Missouri.")

...but fails when you replace "Missouri" with "Michigan":
geo.geoparse("Congress and in the legislatures of Alabama, California, Florida, and Michigan.")

It fails here with a KeyError.

The bug was caused by a geonames entry that didn't have a feature_code.
('geonameid': '5740345', 'place_name': 'Michigan (historical)', 'feature_class': 'S', 'feature_code': '')


Next Rep Date card is generated on manually changing Next rep date.

Is this still happening for you?

Yes


Formatter

Is there any way to get formatting for teal files in vscode?

Hello! Unfortunately, I'm not currently aware of any code formatting tool for Teal code. This would be pretty useful.

It's a pity. Teal seem promising but still far from convenient to use. I was searching for a statically type language for my C application and as Lua is super easy to integrate using Teal was a great option. However, not having proper integration with VScode is a deal breaker for me. I endup still to keep lua and now use lua-language-server for type checking and it does pretty good job actually, at least good enough for my needs.


Introduce eventually matcher to system tests.

Summary:
Using a hamcrest like assert_that we can wrap retrying once in an
eventually matcher so that we don't have to rewrite them each time.
The assertions become much cleaner this way. The Python library
precisely provides a nice pattern to define custom matchers.

I'm mixed about this... for 1 there is an attempt to make generic without doing it. There is a lot of additional code without any gained benefit. The core change L269-270 of marathon_common isn't changed much. I get the desire to change it and have it more clear for more occurrences of such a test

I hesitated to change too many test to not blow up the diff. The idea is to use eventually everywhere instead of the retrying. If you like to, I'll add it in more place.

The biggest issue I had is if we are going to make it more generic then it should go in shakedown... but in the end the matcher function isn't generic. has_failure_message has AppFailureMessage which is marathon specific and app specific. The method has_failure_message should be something like has_app_failure_message

AppFailureMessage is a bad example from my side. It is indeed very specific. I was aiming more for assert_that and eventually. Also, the tests are specific right now. assert_that introduces a pattern to make them more generic. I'll try to make the failure message generic. I had something like this in mind
assert_that(lambda: client.get_app(app_def["id"]),
                   eventually(property(['lastTaskFailure', 'message'], contains_string("No such user 'bad'")),max_attempts=3))

Instead of nesting we could go for chaining
assert_that(lambda: client.get_app(app_def["id"])).eventually(max_attempts=30).
                    property(['lastTaskFailure', 'message']).contains_string("No such user 'bad'")

What do you think?

It is also a large change from 30 attempts to 3... if it is eventually why do you want such a small number of attempts? And why to configure the number of attempts for this test while also still using the default?

I used 3 for debugging and forgot to change it to 30 again.
I do plan to introduce the matchers to Shakedown once we have a feel for it. I hope that Shakedown will evolve to a collection of fixtures, matchers and helpers.

also.. I'm happy with either of the code examples provided... I think I lean towards
assert_that(lambda: client.get_app(app_def["id"]),
            eventually(property(['lastTaskFailure', 'message'], contains_string("No such user 'bad'")),max_attempts=30))


Add migration script for users with None as email_verifications

Purpose

Create a migration script so that users with email_verifications that are None are changed to an empty dict, as is the new default.
Closes #3087

Changes

Add script to scripts/migration/migrate_none_as_email_verification.py

A few minor changes and this is good to merge. :point_up:

Changes made!

@barbour-em This addresses an bug that is on production. Can you resend this as a hotfix?

@barbour-em Disregard my previous comment. Talked with @lyndsysimon -- there are no more users affected by the issue on production, so this can stay as a feature.


Add file download feature

Add a feature so every time a file is saved to the browser, a local copy is also attempted (e.g. an auto-download begins).
This might be a bit aggressive but let's try this initially because it's better to err on the user having a copy of the file on their disk than losing it because of a browser cache clear.

Just reviewed this feature. While reviewing it, I closed issues #7, #9, and #10.
If you can QC this and then close this issue, I'll push to production.
FYI first beta units got shipped today, so this is fairly urgent.


logcat -c fails randomly with multiple iterations

Hi,
not sure this is related to WA or if it is how logcat is called but if I run an agenda with multiple iterations, sometimes the test fails at the end because of:
ERROR            Got exit code 1
ERROR            from target command: logcat -c
ERROR            OUTPUT: failed to clear the 'main' log

Hi,
I have not come across that issue before, do you get that same error if you run that command multiple times yourself? / Do you get this with multiple devices?

Yes, I get the issue if I run the command multiple times.
Yes, this issue has been reported with another device but same platform (hikey960)

Perhaps, this is related to the error I can see in the console:
[ 1192.961243] binder_alloc: 13020: binder_alloc_buf, no vma
[ 1192.961243] binder_alloc: 13020: binder_alloc_buf, no vma
[ 1192.972464] binder: 2724:10218 transaction failed 29189/-3, size 108-0 line 3135
[ 1192.972464] binder: 2724:10218 transaction failed 29189/-3, size 108-0 line 3135
[ 1193.014989] binder: undelivered transaction 338114, process died.
[ 1193.014989] binder: undelivered transaction 338114, process died.
[ 1193.031581] binder: undelivered transaction 338093, process died.
[ 1193.031581] binder: undelivered transaction 338093, process died.

Is that with the the same image on the different devices? The other crashes in the console likely indicate that there is something else wrong with the Android system rather than a WA issue itself.

Yes, it is with the same image. Agree it must be an Android related issue. Closing the issue.

It seems like this error can happen when there are multiple instances of logcat running at the same time. I'm wondering if WA could have miss a 'wait' after exec'ing the adb logcat -c (or may be adb is closing the connection before waiting the child process?). I'm reopening this issue in order to double check if that can be the case from the WA side.

The actual clearing of logcat on the device is performed via devlib (https://github.com/ARM-software/devlib/blob/master/devlib/target.py#L1386) and the implementation there uses a lock to prevent multiple instances and will wait until the command is completed to check the error code so I think these cases should already be guarded against.
When you say the error occurs with multiple instances of logcat running, do you still experience it if you are not also monitoring logcat yourself?

Sorry I missed your answer.
I reproduced the issue and monitored a 'logcat' instance. There is a:
shell        28556 24458 0 08:36:59 pts/1 00:00:00 logcat
stuck on the 'recvmsg' syscall on a socket.
I verified before running the scenario no instance was running. So somehow, the logcat is blocked on the socket and WA3 does not wait it (or does not return an error if a timeout occurs leading the error for the next run).

Agenda triggering the issue:
sections:

id: default
runtime_params:
brightness: 0
screen_on: true
airplane_mode: true
classifiers:
tag: dflt

workloads:

name: jankbench
id: jankview_list_view
classifiers:
tag: iter_30
workload_parameters:
test_ids: list_view
iterations: 10

Thank you very much for you agenda, I managed to reproduce the issue here and the jankbench monitor was clearing logcat itself rather than using the method available in devlib.
Would you be able to try my fix (https://github.com/ARM-software/workload-automation/pull/900) and see if this fixes your issue?

Yes, I confirm I do no longer see the error now. Thanks !

Cool thank you, I've merged the fix into master.

Hi. I do see this problem a lot too. I did take the latest fixes but logcat fails with this error message more than 1/3rd of the time
ERROR              File "/mnt/data/src/lisa/tools/workload-automation/wa/framework/job.py", line 134, in setup
ERROR                self.workload.setup(context)
ERROR              File "/mnt/data/src/lisa/tools/workload-automation/wa/workloads/jankbench/__init__.py", line 104, in setup
ERROR                self.monitor = JankbenchRunMonitor(self.target)
ERROR              File "/mnt/data/src/lisa/tools/workload-automation/wa/workloads/jankbench/__init__.py", line 211, in __init__
ERROR                self.target.clear_logcat()
ERROR              File "/home/qyousef/src/lisa/libs/devlib/devlib/target.py", line 1387, in clear_logcat
ERROR                adb_command(self.adb_name, 'logcat -c', timeout=30)
ERROR              File "/home/qyousef/src/lisa/libs/devlib/devlib/utils/android.py", line 474, in adb_command
ERROR                output, _ = check_output(full_command, timeout, shell=True)
ERROR              File "/home/qyousef/src/lisa/libs/devlib/devlib/utils/misc.py", line 198, in check_output
ERROR                raise subprocess.CalledProcessError(retcode, command, output='\n'.join([output or '', error or '']))
ERROR
ERROR            Command 'adb -s HIKEY960_1 logcat -c' returned non-zero exit status 1
ERROR            OUTPUT:
ERROR
ERROR            failed to clear the 'system' log
ERROR
ERROR

Applying this change fixes it for me
diff --git a/wa/workloads/jankbench/__init__.py b/wa/workloads/jankbench/__init__.py
index d4c60cef..1238533b 100644
--- a/wa/workloads/jankbench/__init__.py
+++ b/wa/workloads/jankbench/__init__.py
@@ -208,13 +208,13 @@ class JankbenchRunMonitor(threading.Thread):
         self.daemon = True
         self.run_ended = threading.Event()
         self.stop_event = threading.Event()
-        self.target.clear_logcat()
         if self.target.adb_name:
             self.command = ['adb', '-s', self.target.adb_name, 'logcat']
         else:
             self.command = ['adb', 'logcat']

     def run(self):
+        self.target.clear_logcat()
         proc = subprocess.Popen(self.command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
         while not self.stop_event.is_set():
             if self.run_ended.is_set():

Is there some python rule about starting processes and waiting for it in init() context? I can push a PR with the fix if it's the right thing to do.

Hi. There are no special rules as far as I'm aware of, however I'm not sure how this should be affecting the board this way.
Would you be able to send me the run logs from before and after your fix to see what is happening?

Sure. My 'fix' could be a workaround then but it reliably fixes the issue for me and my colleague too.
AFAICT the 'failed to clear 'system' log' message is produced by adb itself
https://github.com/aosp-mirror/platform_system_core/blob/master/logcat/logcat.cpp#L1496
Why logcat fails to clear 'system' (and at other times 'main') logs sometimes when called from __init__() is werid. I am on Android Q by the way - and did try to ensure that my local adb version matches the server on the device (v40) but that didn't help.

When the failure happens I see this interesting snippet:
908 2018-10-04 11:44:38,175 DEBUG       android:         adb -s HIKEY960_1 shell am kill-all
 909 2018-10-04 11:44:38,251 DEBUG       android:         adb -s HIKEY960_1 logcat -c
 910 2018-10-04 11:44:48,299 DEBUG       android:         adb -s HIKEY960_1 shell dumpsys power
 911 2018-10-04 11:44:48,352 DEBUG       android:         adb -s HIKEY960_1 logcat -c
 912 2018-10-04 11:44:51,431 DEBUG        signal:         Sending error-logged from <wa.utils.log.ErrorSignalHandler object at 0x7f89ba974850>
 913 2018-10-04 11:44:51,432 DEBUG        signal:         Disconnecting <bound method Executor._error_signalled_callback of executor> from error-logged(<Sender: Any>)
 914 2018-10-04 11:44:51,431 ERROR        signal:           File "/mnt/data/src/lisa/tools/workload-automation/wa/framework/job.py", line 134, in setup
 915 2018-10-04 11:44:51,431 ERROR        signal:             self.workload.setup(context)
 916 2018-10-04 11:44:51,431 ERROR        signal:           File "/mnt/data/src/lisa/tools/workload-automation/wa/workloads/jankbench/__init__.py", line 104, in setup
 917 2018-10-04 11:44:51,431 ERROR        signal:             self.monitor = JankbenchRunMonitor(self.target)
 918 2018-10-04 11:44:51,431 ERROR        signal:           File "/mnt/data/src/lisa/tools/workload-automation/wa/workloads/jankbench/__init__.py", line 211, in __init__
 919 2018-10-04 11:44:51,431 ERROR        signal:             self.target.clear_logcat()
 920 2018-10-04 11:44:51,431 ERROR        signal:           File "/home/qyousef/src/lisa/libs/devlib/devlib/target.py", line 1387, in clear_logcat
 921 2018-10-04 11:44:51,431 ERROR        signal:             adb_command(self.adb_name, 'logcat -c', timeout=30)
 922 2018-10-04 11:44:51,431 ERROR        signal:           File "/home/qyousef/src/lisa/libs/devlib/devlib/utils/android.py", line 474, in adb_command
 923 2018-10-04 11:44:51,431 ERROR        signal:             output, _ = check_output(full_command, timeout, shell=True)
 924 2018-10-04 11:44:51,431 ERROR        signal:           File "/home/qyousef/src/lisa/libs/devlib/devlib/utils/misc.py", line 198, in check_output
 925 2018-10-04 11:44:51,431 ERROR        signal:             raise subprocess.CalledProcessError(retcode, command, output='\n'.join([output or '', error or '']))
 926 2018-10-04 11:44:51,431 ERROR        signal:
 927 2018-10-04 11:44:51,432 DEBUG        signal:         Sending error-logged from <wa.utils.log.ErrorSignalHandler object at 0x7f89ba974850>
 928 2018-10-04 11:44:51,433 DEBUG        signal:         Disconnecting <bound method Executor._error_signalled_callback of executor> from error-logged(<Sender: Any>)
 929 2018-10-04 11:44:51,432 ERROR        signal:         Command 'adb -s HIKEY960_1 logcat -c' returned non-zero exit status 1
 930 2018-10-04 11:44:51,432 ERROR        signal:         OUTPUT:
 931 2018-10-04 11:44:51,432 ERROR        signal:
 932 2018-10-04 11:44:51,432 ERROR        signal:         failed to clear the 'system' log
 933 2018-10-04 11:44:51,432 ERROR        signal:
 934 2018-10-04 11:44:51,432 ERROR        signal:

Using my workaround
466 2018-10-04 10:33:08,681 DEBUG       android:         adb -s HIKEY960_1 shell am kill-all
 467 2018-10-04 10:33:08,785 DEBUG       android:         adb -s HIKEY960_1 logcat -c
 468 2018-10-04 10:33:18,837 DEBUG       android:         adb -s HIKEY960_1 shell dumpsys power
 469 2018-10-04 10:33:18,900 DEBUG        signal:         Sending successful-workload-setup from wk1 (jankbench) [1]
 470 2018-10-04 10:33:18,900 DEBUG       android:         adb -s HIKEY960_1 shell "ls /data/local/tmp > /dev/null"
 471 2018-10-04 10:33:18,900 DEBUG        signal:         Sending after-workload-setup from wk1 (jankbench) [1]
 472 2018-10-04 10:33:18,901 INFO            job:     Running job wk1 (jankbench) [1]
 473 2018-10-04 10:33:18,904 DEBUG        signal:         Sending before-workload-execution from wk1 (jankbench) [1]
 474 2018-10-04 10:33:18,905 DEBUG       android:         adb -s HIKEY960_1 shell am start -W -S -n com.android.benchmark/.app.RunLocalBenchmarksActivity --ei com.android.benchmark.EXTRA_RUN_COUNT 1
 475 2018-10-04 10:33:18,967 DEBUG       android:         adb -s HIKEY960_1 shell '(ls -1); echo "
 476 2018-10-04 10:33:18,967 DEBUG       android:         $?"'
 477 2018-10-04 10:33:19,041 DEBUG       android:         ls command is set to ls -1
 478 2018-10-04 10:33:19,042 DEBUG       android:         adb -s HIKEY960_1 logcat -c
 479 2018-10-04 10:33:20,297 DEBUG     jankbench:         Stopping: com.android.benchmark
 480 2018-10-04 10:33:20,297 DEBUG     jankbench:         Starting: Intent { cmp=com.android.benchmark/.app.RunLocalBenchmarksActivity (has extras) }
 481 2018-10-04 10:33:20,297 DEBUG     jankbench:         Status: ok
 482 2018-10-04 10:33:20,297 DEBUG     jankbench:         Activity: com.android.benchmark/.app.RunLocalBenchmarksActivity
 483 2018-10-04 10:33:20,297 DEBUG     jankbench:         ThisTime: 853
 484 2018-10-04 10:33:20,297 DEBUG     jankbench:         TotalTime: 853
 485 2018-10-04 10:33:20,297 DEBUG     jankbench:         WaitTime: 1053
 486 2018-10-04 10:33:20,297 DEBUG     jankbench:         Complete
 487 2018-10-04 10:33:20,297 DEBUG     jankbench:

The interesting bit is the 2 logcat -c sandwiching dumpsys power. My workaround moves the 2nd logcat -c to happen later. I'll try removing the logcat -c altogether to see what happens.
What does dumpsys powe do? I tried running this yesterday and spamming adb with logcat -c didn't generate any failure in over 1500 runs
i=0; while true; do i=$((i+1)); echo "$i"; adb logcat -c; sleep 1; if [ $? -ne 0 ]; then echo "***** FAILED *****"; break; fi; done

I tried the above with and without the sleep 1 by the way and both were happy. So maybe dumpsys power has a side effect.
I can email you the full run.log files if you want.

Removing self.target.clear_logcat() from __init__() seems to make the problem disappear too. I'll continue running with that to see if the problem completely disappears

I am on Android Q by the way - and did try to ensure that my local adb version matches the server on the device (v40) but that didn't help.

Thank you for providing the logs, I'm now using the same android and adb versions as specified and I've managed to encounter a similar error. I can however also reproduce it by calling the clear_logcat method by itself so not sure if this is the exact same issue.

What does dumpsys power do? ... I tried the above with and without the sleep 1 by the way and both were happy. So maybe dumpsys power has a side effect.

This is what we use to help determine the state of the device e.g. the screen state etc. However I don't think it should interact with logcat and I've tried running the two commands in a loop and I haven't managed to find a link so far. If you try adding the extra command into your testing loop do you see the same behaviour?

Removing self.target.clear_logcat() from init() seems to make the problem disappear too. I'll continue running with that to see if the problem completely disappears

I think it might be worthwhile removing the call from the logcat poller as the 2 calls happen so close to each other, however I think this would still be more of a workaround rather than a fix for the underlying issue.

There's something fishy and it's worthwhile trying to understand it I think. With the call to self.targer.clear_logc() commented out from __init__() I hit a similar error but this time when running exoplayer workload (first time I hit it btw)
INFO     Running job default-exoplayer_ogg_128kbps_30s
INFO         Configuring augmentations
INFO         Configuring target for job default-exoplayer_ogg_128kbps_30s (exoplayer) [2]
INFO         Setting up job default-exoplayer_ogg_128kbps_30s (exoplayer) [2]
ERROR              File "/mnt/data/src/lisa/tools/workload-automation/wa/framework/job.py", line 134, in setup
ERROR                self.workload.setup(context)
ERROR              File "/mnt/data/src/lisa/tools/workload-automation/wa/workloads/exoplayer/__init__.py", line 185, in setup
ERROR                self.monitor.start()
ERROR              File "/home/usr/src/lisa/libs/devlib/devlib/utils/android.py", line 601, in start
ERROR                self.target.clear_logcat()
ERROR              File "/home/usr/src/lisa/libs/devlib/devlib/target.py", line 1387, in clear_logcat
ERROR                adb_command(self.adb_name, 'logcat -c', timeout=30)
ERROR              File "/home/usr/src/lisa/libs/devlib/devlib/utils/android.py", line 474, in adb_command
ERROR                output, _ = check_output(full_command, timeout, shell=True)
ERROR              File "/home/usr/src/lisa/libs/devlib/devlib/utils/misc.py", line 198, in check_output
ERROR                raise subprocess.CalledProcessError(retcode, command, output='\n'.join([output or '', error or '']))
ERROR
ERROR            Command 'adb -s HIKEY960_1 logcat -c' returned non-zero exit status 1
ERROR            OUTPUT:
ERROR
ERROR            failed to clear the 'main' log
ERROR
ERROR
INFO         Recording screen state...
INFO         Completing job default-exoplayer_ogg_128kbps_30s
ERROR    Job default-exoplayer_ogg_128kbps_30s iteration 2 completed with status FAILED. Max retries exceeded.

From run.log
2364 2018-10-04 13:44:03,941 DEBUG       android:         adb -s HIKEY960_1 shell am kill-all
2365 2018-10-04 13:44:04,024 DEBUG       android:         adb -s HIKEY960_1 logcat -c
2366 2018-10-04 13:44:14,083 DEBUG       android:         adb -s HIKEY960_1 shell dumpsys package com.google.android.exoplayer2.demo
2367 2018-10-04 13:44:14,147 DEBUG       android:         adb -s HIKEY960_1 shell pm grant com.google.android.exoplayer2.demo android.permission.INTERNET
2368 2018-10-04 13:44:14,214 DEBUG       android:         Cannot grant android.permission.INTERNET
2369 2018-10-04 13:44:14,215 DEBUG       android:         adb -s HIKEY960_1 shell pm grant com.google.android.exoplayer2.demo android.permission.READ_EXTERNAL_STORAGE
2370 2018-10-04 13:44:14,278 DEBUG       android:         adb -s HIKEY960_1 shell echo 'if [ -e '\''/sdcard/devlib-target/128kbps_Tchaikovsky_-_Romeo_and_Juliet_Ouverture_-_Antal_Dorati_(1959).ogg'\'' ]; th     en echo 1; else echo 0; fi' | su
2371 2018-10-04 13:44:14,352 DEBUG       android:         adb -s HIKEY960_1 shell settings get system user_rotation
2372 2018-10-04 13:44:14,415 DEBUG       android:         adb -s HIKEY960_1 shell settings put system accelerometer_rotation 0
2373 2018-10-04 13:44:14,478 DEBUG       android:         adb -s HIKEY960_1 shell settings put system user_rotation 0
2374 2018-10-04 13:44:14,635 DEBUG       android:         adb -s HIKEY960_1 logcat -c
2375 2018-10-04 13:44:17,730 DEBUG        signal:         Sending error-logged from <wa.utils.log.ErrorSignalHandler object at 0x7f29ff39e850>
2376 2018-10-04 13:44:17,731 DEBUG        signal:         Disconnecting <bound method Executor._error_signalled_callback of executor> from error-logged(<Sender: Any>)
2377 2018-10-04 13:44:17,730 ERROR        signal:           File "/mnt/data/src/lisa/tools/workload-automation/wa/framework/job.py", line 134, in setup
2378 2018-10-04 13:44:17,730 ERROR        signal:             self.workload.setup(context)
2379 2018-10-04 13:44:17,730 ERROR        signal:           File "/mnt/data/src/lisa/tools/workload-automation/wa/workloads/exoplayer/__init__.py", line 185, in setup
2380 2018-10-04 13:44:17,730 ERROR        signal:             self.monitor.start()

There are 2 calls to logcat -c but they aren't close enough :-/ I am starting to doubt a bug in adb v40..

If you try adding the extra command into your testing loop do you see the same behaviour?

I'll try and let you know

There are 2 calls to logcat -c but they aren't close enough :-/ I am starting to doubt a bug in adb v40..

It does look like it might be v40. I've tried rolling back my adb version and do not appear to have the same issue any more, update back to v40 and it's back.

If you try adding the extra command into your testing loop do you see the same behaviour?
doing adb logcat -c; adb shell dumpsys power; adb logcat -c; in a loop didn't generate any errors for me. So something about the way environment from which these commands run is creating a problem.

It does look like it might be v40. I've tried rolling back my adb version and do not appear to have the same issue any more, update back to v40 and it's back.

Are you referring to adbd running on the device or adb on the host machine? I usually run with adb v36 on my host and it fails; tried v40 on the host and didn't help. Didn't try downgrading adbd though; if that's what you did, how did you do that so I can give it a go too?

Doing adb logcat -c; adb shell dumpsys power; adb logcat -c; in a loop didn't generate any errors for me. So something about the way environment from which these commands run is creating a problem.

Ah ok it does appear that way, although I have not manage to find a reliable reproducer as of yet.

Are you referring to adbd running on the device or adb on the host machine? I usually run with adb v36 on my host and it fails; tried v40 on the host and didn't help. Didn't try downgrading adbd though; if that's what you did, how did you do that so I can give it a go too?

That's interesting, sorry this was just my host version I tried changing that seem to have that effect, I have not tried to change my from running adbd (v39) either.

I test with this workload and I see it at least once every run
 workloads:
    - name: jankbench
      iterations: 6

I test with this workload and I see it at least once every run
 workloads:
    - name: jankbench
      iterations: 6

Ok that does seem to produce the error at least once but within that I haven't found a pattern of when it will occur. I guess it must be some sort of race condition somewhere although I'm not sure what could be causing that.
Have you spotted any correlations?

No. I tried disabling the logact polling thread but this didn't seem to help. It might be the threading library causing the weird side effect. I'll keep looking as a lower priority background investigation; in the meantime I'll use my workaround to carry on with my work.
By the way I noticed that uiautomation base does call logcat as well - that might explain the exoplayer failure as python and uiautomation 'might' be doing some logcat operation simultaneously. But I think this is unlikely in the normal flow of operation. I don't know.

I've tried moving the calls to clear logcat into the setup method of the workload and this still appears to give the same problem so it doesn't appear to be the threading library, rather just the multiple calls themselves which I shall continue to investigate.
I've taken a look through the exoplayer workload and this also does logcat polling in the background so would make sense that your seeing the same issue there, although that appears to be done via devlib as well so should still be semi protected from concurrent operations.

Interestingly when running the jankbench workload; running ps -ax | grep adb returns interesting results
19549 ?        Ssl    0:01 adb -L tcp:5037 fork-server server --reply-fd 5
22253 pts/5    S+     0:00 adb -s HIKEY960_1 logcat
24356 pts/5    S+     0:00 adb -s HIKEY960_1 logcat
26498 pts/5    S+     0:00 adb -s HIKEY960_1 logcat
27040 pts/18   S+     0:00 grep --color=auto adb

Note that was while the test was in the 5th iteration. Iteration 2 failed with the usual logact -c error. At the time there was only one 'ghost' adb logcat process running.
It looks like something that interacts with logcat doesn't terminate correctly between runs and potentially might be causing the occasional logcat -c errors.
I couldn't figure out who spawns this logcat process yet.

May be you can display the parent pid ?

I'm seeing similar duplicated processes on my end so will track down what is spawning each one. We do not have a generic java automation so it will be coming from python somewhere.
I have also found this issue https://github.com/google/mobly/pull/501 which indicates that the failure to clear logcat is a known android  bug at least on Android O so the duplicated processes may be making hitting this bug more frequent rather than the root cause.

Silly me. Jankbench run() method executes adb logcat and doesn't do a proc.kill() once it's done. Let me try that to see if it hopefully solves the problem

Preliminary results of this change look promising
diff --git a/wa/workloads/jankbench/__init__.py b/wa/workloads/jankbench/__init__.py
index d4c60cef..5ccc6214 100644
--- a/wa/workloads/jankbench/__init__.py
+++ b/wa/workloads/jankbench/__init__.py
@@ -228,6 +228,8 @@ class JankbenchRunMonitor(threading.Thread):
                     if self.regex.search(line):
                         self.run_ended.set()

+        proc.terminate()
+
     def stop(self):
         self.stop_event.set()
         self.join()

I'll test with this for the rest of the day and see if I encounter more problems.
Worth noting that I have seen another problem that might be related to this where jankbench tests 'hangs' in the middle of a long test suite run that includes multiple workloads and a lot of iterations. I think that's when I noticed that there were too many adb processes in my system - but sadly didn't investigate that instance properly. I'm hoping it's another side effect of this problem - let's see.

Ah excellent, its looks like its solved the issue on my end as well so yes I'm happy to push the fix for that.
Have you been having any more problems with exoplayer?

No everything was happy as the test suite runs jankbench, exoplayer, pcmark and geekbench for many iterations and they all ran to completion as far as I see.
FWIW, here's my comment log in case it helps to document the issues I've encountered
https://github.com/qais-yousef/workload-automation/commit/6c1b8ff4d948aa89b36c4efc33842257f1af8dbd

Ok, thank you very much for your help debugging, please me know if you run into any more issues.


plugin-lighthouse: fix form factor field not working

Issue
When triggering an audit :

The "Emulated Form Factor" field does not work if the lighthouse-audit-service backend is up to date (version 1.0.2).
This means that when selecting "Desktop", the audit will be emulating a mobile device and the report will show screenshots in mobile screen size.
Fix
This PR fixes the issue with lighthouse-audit-service 1.0.2 while still being compatible with previous versions.
:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
Added or updated documentation bugfix
[ ] Tests for new functionality and regression tests for bug fixes
Screenshots attached (for UI changes) no ui change
[x] All your commits have a Signed-off-by line in the message. (more info)

Just closing and reopening to trigger the builds!


Add markdown linting for freezing-* projects

I've found Markdown linting to be helpful before. We should adopt PyMarkdownLint or something similar and use it in our GitHub Actions lint setup. See the PyMarkDownLint lint action for reference.

Implemented in #352


Multi-npub support

As a nostr user who has more than 1 account, I would like to see the ability for multiple npubs to write to the relay.

we will not be adding this feature. Recommended approach is to have two instances of haven on different ports.


Error: Build failed

Describe the Bug
Getting errors when previewing out-of-the-box emails.
Error: Build failed with 3 errors: emails/notion-magic-link.tsx:11:7: ERROR: Could not resolve "@react-email/components" emails/notion-magic-link.tsx:26:4: ERROR: Could not resolve "react/jsx-runtime" emails/notion-magic-link.tsx:152:38: ERROR: [plugin: render-resolver] Failed trying to import `renderAsync` from either `@react-email/render` or `@react-email/components` to be able to render your email template. Maybe you don't have either of them installed?

Which package is affected (leave empty if unsure)
create-email
Link to the code that reproduces this issue

To Reproduce
Follow the instructions here https://react.email/docs/getting-started/automatic-setup
Expected Behavior
Have out-of-the-box emails preview successfully as shown in docs.
What's your node version? (if relevant)
v18.18.2

Thinking this might be because by yarn modern's pnp. Here's an issue someone else also had https://github.com/resend/react-email/issues/1422#issuecomment-2072310780

@gabrielmfern getting the same issue, there are no pnp files in my project.

following up, I had to install react and react-dom to my package. the project is a pnpm monorepo.

Not sure what's happening but I have this error in a personal project, works in one computer and not in another.
Also using pnpm workspaces, code in https://github.com/propsto/props.to/tree/auth-domain and by running in the root pnpm i and then pnpm dev:email you should be able to reproduce the issue... or not. Tried node v20 and v18.
Also tried including react and react-dom.

@iantanwx From the comment I mentioned, the yarn files were not on their project but rather on their home:

That was it! I didn't use pnp (never actually have so I'm not sure where it came from), but I had .pnp.cjs .pnp.loader.mjs in my home directory.

From what you mention @leog,

works in one computer and not in another

I am pretty sure this is the same problem as the comment mentions. Also, the repo you mention doesn't have the script nor does it have react-email installed 😅.

@gabrielmfern notice I shared a repo with a particular branch auth-domain, in it there is a package under packages folder called email

I see it now, but no issues on my side:

Here's a separate comment on an issue talking about this https://github.com/evanw/esbuild/issues/2647#issuecomment-1516574767. Thinking you have ran yarn on the home directory, or at some higher up directory, which causes esbuild to find it up all the way there and expect the modules' resolution to be done by it, not by pnpm.

Can't wait to go back to my other computer and find those bastard files braking my emal dev flow 😄
Thanks for the pointer @gabrielmfern, I'll let you know later today.

yeahp, found it at /Users/username//.pnp.cjs, fixed! thanks all!


[Question]: Can SQLTableRetrieverQueryEngine or NLSQLTableQueryEngine filter specific value ?

Question Validation

[ ] I have searched both the documentation and discord for an answer.

Question
I use LLM= gpt-3.5-turbo and llama_index to connect database SQL server but I don't see the document to filter value while query in database or pass some values to filter data.
I see llama_index will convert prompt from users to statement SQL but cannot put or pass values before that and the get result from database.
Can you suggest how I can filter data with using NLSQLTableQueryEngine or SQLTableRetrieverQueryEngine ?
Ref: https://docs.llamaindex.ai/en/stable/examples/index_structs/struct_indices/SQLIndexDemo/

Can you show example that easily I can modify code or modified code in https://docs.llamaindex.ai/en/stable/examples/index_structs/struct_indices/SQLIndexDemo/
?

You probably want to create a more custom text-2-sql query pipeline
https://docs.llamaindex.ai/en/stable/examples/pipeline/query_pipeline_sql/?h=querypipeline


Add Python 3.12 test

Codecov Report

:exclamation: No coverage uploaded for pull request base (build@5698741). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             build     #497   +/-   ##
========================================
  Coverage         ?   69.41%
========================================
  Files            ?       79
  Lines            ?     9159
  Branches         ?        0
========================================
  Hits             ?     6358
  Misses           ?     2801
  Partials         ?        0

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


change the kubernetes pod wait logic to use akka after pattern

instead of fabric8 client waitUntilReady. See discussion in #4812

Description

This allows more precision for config specified at whisk.kubernetes.timeouts.run
Related issue and scope

[x] I opened an issue to propose and discuss this change (#4812)

My changes affect the following components

[ ] API
[ ] Controller
[ ] Message Bus (e.g., Kafka)
[ ] Loadbalancer
[x] Invoker
[ ] Intrinsic actions (e.g., sequences, conductors)
[ ] Data stores (e.g., CouchDB)
[ ] Tests
[ ] Deployment
[ ] CLI
[ ] General tooling
[ ] Documentation

Types of changes

[ ] Bug fix (generally a non-breaking change which closes an issue).
[x] Enhancement or new feature (adds new functionality).
[ ] Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

[x] I signed an Apache CLA.
[x] I reviewed the style guides and followed the recommendations (Travis CI will check :).
[ ] I added tests to cover my changes.
[ ] My changes require further changes to the documentation.
[ ] I updated the documentation where necessary.

Codecov Report

Merging #4857 into master will decrease coverage by 6.22%.
The diff coverage is 8.33%.

@@            Coverage Diff             @@
##           master    #4857      +/-   ##
==========================================
- Coverage   82.33%   76.10%   -6.23%
==========================================
  Files         200      200
  Lines        9283     9290       +7
  Branches      381      380       -1
==========================================
- Hits         7643     7070     -573
- Misses       1640     2220     +580

Impacted Files
Coverage Δ

...re/containerpool/kubernetes/KubernetesClient.scala
20.78% <8.33%> (-0.27%)
:arrow_down:

...core/database/cosmosdb/RxObservableImplicits.scala
0.00% <0.00%> (-100.00%)
:arrow_down:

...ore/database/cosmosdb/cache/CacheInvalidator.scala
0.00% <0.00%> (-100.00%)
:arrow_down:

...e/database/cosmosdb/cache/ChangeFeedConsumer.scala
0.00% <0.00%> (-100.00%)
:arrow_down:

...core/database/cosmosdb/CosmosDBArtifactStore.scala
0.00% <0.00%> (-96.23%)
:arrow_down:

...sk/core/database/cosmosdb/CosmosDBViewMapper.scala
0.00% <0.00%> (-93.90%)
:arrow_down:

...tabase/cosmosdb/cache/CacheInvalidatorConfig.scala
0.00% <0.00%> (-92.31%)
:arrow_down:

...e/database/cosmosdb/cache/KafkaEventProducer.scala
0.00% <0.00%> (-77.78%)
:arrow_down:

...enwhisk/connector/kafka/KamonMetricsReporter.scala
0.00% <0.00%> (-76.20%)
:arrow_down:

...whisk/core/database/cosmosdb/CosmosDBSupport.scala
0.00% <0.00%> (-74.08%)
:arrow_down:

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9038017...b714ae7. Read the comment docs.


Program throws an error after quitting VALORANT

`Traceback (most recent call last):
File "urllib3\connection.py", line 175, in _new_conn
File "urllib3\util\connection.py", line 95, in create_connection
File "urllib3\util\connection.py", line 85, in create_connection
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "urllib3\connectionpool.py", line 710, in urlopen
File "urllib3\connectionpool.py", line 386, in _make_request
File "urllib3\connectionpool.py", line 1040, in _validate_conn
File "urllib3\connection.py", line 358, in connect
File "urllib3\connection.py", line 187, in _new_conn
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x0000019856628D08>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "requests\adapters.py", line 450, in send
File "urllib3\connectionpool.py", line 786, in urlopen
File "urllib3\util\retry.py", line 592, in increment
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='<IP_ADDRESS>', port=50902): Max retries exceeded with url: /chat/v4/presences (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000019856628D08>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "src\presence\presence.py", line 80, in init_loop
File "src\presence\presence.py", line 54, in main_loop
File "valclient\client.py", line 751, in fetch_presence
File "valclient\client.py", line 114, in fetch
File "requests\api.py", line 75, in get
File "requests\api.py", line 61, in request
File "requests\sessions.py", line 529, in request
File "requests\sessions.py", line 645, in send
File "requests\adapters.py", line 519, in send
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='<IP_ADDRESS>', port=50902): Max retries exceeded with url: /chat/v4/presences (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000019856628D08>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))
press enter to exit...`

The error isn't pretty, but it's intentional. How would it work if VALORANT was closed?

The error isn't pretty, but it's intentional. How would it work if VALORANT was closed?

Where is the feature that opens the game if the game itself is closed, may I ask?


Problem placing a sticker on image

Problem starts when I have to place a sticker on canvas/image: the sticker returns back to the center, it's like there is gravity.
Other project's dependencies colud create conflicts with this library?
Canvas matches parent's width and height, as you can see in the xml below:
<ja.burhanrashid52.photoeditor.PhotoEditorView
    android:id="@+id/photoEditorView"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:layout_marginBottom="?attr/actionBarSize">
</ja.burhanrashid52.photoeditor.PhotoEditorView>

Duplicate of #333


Compute velocity errors when using an effort command interface

This PR is in reference to the discussion in issue https://github.com/ros-controls/ros2_controllers/issues/673#issue-1760444492. The changes in https://github.com/ros-controls/ros2_controllers/commit/8e6697b introduced a bug in that the velocity error terms was not computed when using an effort command interface, resulting in a wobbly/jerky motion. This PR fixes this issue.

Do you have a good idea for adding a test for this? To avoid making this mistake in the future

Codecov Report

Merging #679 (2e0da5d) into master (e7f9962) will increase coverage by 0.75%.
The diff coverage is 34.91%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##           master     #679      +/-   ##
==========================================
+ Coverage   35.78%   36.53%   +0.75%
==========================================
  Files         189        7     -182
  Lines       17570      676   -16894
  Branches    11592      357   -11235
==========================================
- Hits         6287      247    -6040
+ Misses        994      134     -860
+ Partials    10289      295    -9994

Flag
Coverage Δ

unittests
36.53% <34.91%> (+0.75%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ontroller/test/test_load_diff_drive_controller.cpp
11.11% <0.00%> (ø)

diff_drive_controller/src/odometry.cpp
42.16% <11.11%> (ø)

diff_drive_controller/src/speed_limiter.cpp
46.55% <11.11%> (ø)

...ive_controller/test/test_diff_drive_controller.cpp
17.62% <12.08%> (ø)

...iff_drive_controller/src/diff_drive_controller.cpp
47.09% <46.88%> (ø)

...de/diff_drive_controller/diff_drive_controller.hpp
100.00% <100.00%> (ø)

...troller/include/diff_drive_controller/odometry.hpp
100.00% <100.00%> (ø)

... and 189 files with indirect coverage changes

The velocity error was calculated before, but it was done directly in the computeCommand as seen here.

The velocity error was calculated before, but it was done directly in the computeCommand as seen here.

I see, sorry. I thought I removed the redundant difference calculation only, but hadn't had the effort interface in mind.

I've written a test and made the PR #690 against jtc-features branch, where the effort tests are active already. It failed before I cherry-picked the commit from this PR. Let the maintainers decide if we merge the test directly to master or wait for jtc-features to be merged.

@destogl this is already included in #705 (I squashed the changes, this is why it shows new commits but no changes)

OK, I'll then close this, OK?


No need to add "x" (the average of aperture fluxes) for contrast calculation

Hi Markus,
I recently re-inspected the definition of the contrast curve in VIP and the definition in Mawet et al. (2014). After some head scratching, I came to the conclusion that there was a confusion between what the steps in the paper describe and how the actual contrast achieved in an HCI image should be calculated. In my interpretation, the article describes the steps needed to calculate the minimum flux a test aperture should have in a processed image to consider a significant detection there (for the chosen FPF), which involves adding "x" (the average of the fluxes measured in the apertures) in the last step.
However to get the contrast, "x" should not be added. Instead, I'd argue it should just be:
contrast = tau * s * sqrt(1 + 1/n_ap) / throughput / star_photometry
In general x ~ 0, so it usually doesn't matter much, except in some symptomatic cases where I noted this can make a significant difference, in particular for classical full-frame PCA RDI. When an imperfectly correlated PSF is provided, oversubtraction can lead to negative annular regions, where the estimated contrast then gets way too optimistic, as a negative x gets added. In the worst cases, x could even get larger in absolute value than s * sqrt(1 + 1/n_ap), and lead to negative contrasts.
Another way to see this is that the actual (throughput-affected) companion flux in processed images is its aperture flux minus the average of aperture fluxes at the same radius (bar{x1}-bar{x2} in the notations of the paper) - be it in symptomatic cases or not. Therefore adding "back" x (x2) is not necessary, when calculating the achieved contrast.
I remember you mentioned the missing "x" in a VIP issue, which prompted a change on our side. I have now reverted the change in the latest release of VIP. I thought I'd let you know here as well since this also concerns how Applefy calculates contrasts.
Cheers,
Valentin

Hi Valentin,
thanks for checking this again! An imperfect correction of the PSF in RDI is definitely a problem for the t-test. If you see negative structures in the images after PSF subtraction your residual noise is probably highly non-Gaussian and highly non-independent in which case the test is not really applicable. If there is not way to remove the systematic noise not adding back x might be a work around. But from the statistics point of view it is important.
The main reason for this is the choice of the test: A two sample t-test. The test takes two noisy samples X and Y. In our case the sample which contains the signal of the planet only contains a single value:
Y_1 = signal after PCA + noise at the position of the planet.
The second part "noise at the position of the planet" is important as the test accounts the uncertainty in both samples X and Y. If we would know the true noise-free signal of the planet we should rather use a one sample t-test and test if our noise sample X is significantly different from the true planet signal.
Given only the residual image we don't have the noise free planet signal, so a one sample t-test is not applicable. For contrast curve calculations it would be possible. But the test statistic of a one sample test is very different changing the whole statistic.
If we want our contrast curves to be consistent with test we apply to our residual images we have to add back x to make sure both samples are noisy (as assumed by the test).
But there are two additional aspects which might be important for your RDI scenario:

The placement of the apertures: How you place the apertures in your image has a large effect on the final contrast. Applefy accounts for this by placing apertures at various different positions.
We have to be careful how we compute the throughput: The throughput is the faction of the planet light that remains after PCA. This means that for the calculation of the throughput x has to be subtracted to get the attenuation of the signal in a noise free environment. If we don't subtract x in this step the curves are probably biased in cases like RDI with a bad PSF subtraction.

I hope these comments help. I'll keep the implementation of applefy unchanged. But I'm happy to discuss further.
Cheers,
Markus

Hi Markus,
Thanks for your detailed reply - and sorry for the delay in mine. I agree with an important point you raise: the assumptions to apply the t-test may not be met in some symptomatic cases, and this is definitely something to be cautious about. In some cases, the results may just be meaningless because of that.
That being said, one could in principle have the case of an algorithm leaving non-zero mean residual in concentric annuli, that still correspond to aperture fluxes that are i.i.d and normally distributed - just with a non-zero mu. I am not saying this is necessarily the case of PCA-RDI, but for the sake of better interpreting the role of "x", let us consider that case. The only conditions for the test to be valid are i.i.d and normally distributed samples, but there is no requirement for mu to be zero.
I agree that the addition of x, the estimator of mu, is required to calculate the flux an aperture should have to correspond to a significant detection for a given FPF threshold, considering the 2-sample t-test statistics. But my point is that this aperture flux is not the companion flux - the latter is rather the aperture flux minus mu (or well its estimator x for not knowing the exact value of mu), due precisely to the fact that the mean of the noise distribution could be non-zero.
In essence, I’m suggesting that for the same reason you said x should not be added back when estimating the throughput, I don't think it should be added back when it comes to the actual companion flux to be considered in the contrast definition - i.e. what needs to be considered for the ratio with the stellar flux. Don’t you think?
Cheers,
Valentin


Allow __cuda_array_interface__ in HIP

This should fix
cupy_tests/lib_tests/test_polynomial.py::TestPoly1dPolynomialArithmetic::test_poly1d_arithmetic[_param_8_{func=<lambda>, type_l='ndarray', type_r='poly1d'}]
cupy_tests/lib_tests/test_polynomial.py::TestPoly1dPolynomialArithmetic::test_poly1d_arithmetic[_param_20_{func=<lambda>, type_l='ndarray', type_r='poly1d'}]
cupy_tests/lib_tests/test_polynomial.py::TestPoly1dPolynomialArithmetic::test_poly1d_arithmetic[_param_32_{func=<lambda>, type_l='ndarray', type_r='poly1d'}]

tests in hip, we can actually use this but we have to resort to v2 to avoid issues with stream sync.
TODO: run tests with hip

/test jenkins

As discussed offline I prefer #6414 over this one.


Issues with custom code file (JSON file is invalid)

Hello, I am looking to create a new code file for an unsupported air con unit from Panasonic. I download a file from the repo and altered with new base64 codes extracted from the broadlink manager app on windows. Unfortunately, no matter what I try I can't seem to get the file to work.
Home Assistant version
2020.12.2
SmartIR version
1.13.2
Code file configuration
File
Describe the bug
JSON file isn't accepted by the home assistant istance.
Debug log
2021-01-05 22:58:28 ERROR (MainThread) [custom_components.smartir.climate] The device Json file is invalid
Add your logs here.

Additional context
I have checked the file agains YAML and JSON validator and saved in both the indented form as well as uglyfied.

Your file works well for me. Could you post your configuration?

Your file works well for me. Could you post your configuration?

Sure. I don't really know what you need to know, but I'm running HA in a docker container in a rpi 4 and installed smartir through hacs.
After doing that, I added the code file to custom_components/smartir/codes/climate and this snippet to the configuration.yaml
`climate:

platform: smartir
name: Bedroom AC
unique_id: bedroom_ac
device_code: 1027
controller_data: remote.rm3mini_remote`

Sure. I don't really know what you need to know, but I'm running HA in a docker container in a rpi 4 and installed smartir through hacs.
After doing that, I added the code file to custom_components/smartir/codes/climate and this snippet to the configuration.yaml
`climate:

platform: smartir
name: Bedroom AC
unique_id: bedroom_ac
device_code: 1027
controller_data: remote.rm3mini_remote`

Double-checked. It works ok here

Double-checked. It works ok here

Update on this: Endend up forking the repo and editing climate.py on my ha install so that it downloads the code file from my fork. Added my code file to my fork and it ended up working.
On another note, I'm having issues with the actual codes themselves. they function as a toggle (for example, if I change the mode or temperature while the ac is on, it will turn off). Any tips regarding this or how to actually record codes correctly? I ended up going up the temps and recording the on commands.

Update on this: Endend up forking the repo and editing climate.py on my ha install so that it downloads the code file from my fork. Added my code file to my fork and it ended up working.
On another note, I'm having issues with the actual codes themselves. they function as a toggle (for example, if I change the mode or temperature while the ac is on, it will turn off). Any tips regarding this or how to actually record codes correctly? I ended up going up the temps and recording the on commands.

Update: Some more testing. Quick info on my AC unit:
Model: CS-SA9CKP
Remote Model: A75C2284
Notes:

Any command sent using the power button on the remote is read as a toggle on the AC unit (when I turn the AC on, hide the remote and press the power button again, the AC turns on again).

Each command sent with any other buttons don't turn on the AC if it is already off (although it does beep).

Each command appears to send the full state (tested with the swing feature of the AC).

After all of this, I assume I need to relearn codes (where, instead of turning the AC on and off, I should instead press any other button and save the state it turns into).
If anyone can confirm the assumptions I made do let me know.

Update: Some more testing. Quick info on my AC unit:
Model: CS-SA9CKP
Remote Model: A75C2284
Notes:

Any command sent using the power button on the remote is read as a toggle on the AC unit (when I turn the AC on, hide the remote and press the power button again, the AC turns on again).

Each command sent with any other buttons don't turn on the AC if it is already off (although it does beep).

Each command appears to send the full state (tested with the swing feature of the AC).

After all of this, I assume I need to relearn codes (where, instead of turning the AC on and off, I should instead press any other button and save the state it turns into).
If anyone can confirm the assumptions I made do let me know.

Final Update: Finally got everything to work.
Here is the file I used:
file
This is compatible with the model above and should also be compatible with the following models:

CS-SA9CKP
CU-SA9CKP5
CS-SA12CKP
CU-SA12CKP5

@vassilis-panos if you'd like to add these codes to the repo. Tested on RM3 Mini.

Final Update: Finally got everything to work.
Here is the file I used:
file
This is compatible with the model above and should also be compatible with the following models:

CS-SA9CKP
CU-SA9CKP5
CS-SA12CKP
CU-SA12CKP5

@vassilis-panos if you'd like to add these codes to the repo. Tested on RM3 Mini.

Thanks!

Thanks!


could have better error message when training agent without data

When training agent with just creating the agent and not create sayings/training phrases etc, it just shows an error on conversation window:

I feel the error could be: "You have no training phrases for the agent to learn on" Or something like that.
Somebody who isn't familar with AI may just hit the train button thinking its needed because it says "never trained"
Also got an error in browser console:
TypeError: o is not a function[Learn More]

Step 1.7 on Break script.
Add to this: training keywords that aren't even used doesn't result in any message/notification as well. IMO, an agent shouldn't be trained unless the keywords are actually used.

Did you download a release or pull from Master? Which release?
21.1
What operating system are you running?
10 ori
Are you running locally or in the cloud?
local
Which browser are you using?
firefox

@dcalvom I thought we fixed this, but I can still get it to happen on master. Click training with a completely empty agent. Maybe throw an error with a message of Add some sayings before training your agent.

Closed by https://github.com/samtecspg/articulate/commit/4b83727cedf6e7edbadf239df2d7be715c0bc7af