Help Plz: Filter/remove some documents from a this.cursor() - call?

Hello,
i'm rewriting parts of my app using cottz-publish-relations and it's going great so far! :D
But now I've come across a situation where I'm stumped and don't know how to handle it.
The task is the following:
I've got an Activity Stream (think Facebook wall) and objects which are published (let's say "stories").
But some of those stories should be private, or maybe only visible for certain people / groups. To keep  this example simpler, let's say they're private and should only be visible for the user themselves, but not for others.
For the users themselves this is easy:
this.cursor(Activities.find({}, {sortBy: {timestamp: -1}})), function(activityId, activityDoc) {
  if (activityDoc.storyId) {
    this.cursor(Stories.find({activityDoc.storyId}));
  }
});

Or something similar to this, right? (Pseudocode, untested).
But what if I want to decide whether to publish an Activity only if the Story has some kind of flag set (let's say "public")?
How can I remove the activity from the Activities.find() - result while keeping everything reactive?
If I'd first query all activity IDs and then make a {_id: {$in: [storyIds]}} - query on the stories, new activities wouldn't be published after that, right?
And the other way round, if I'd first get all the stories to filter by those, I'd potentially have a million stories in my cursor and only 5 related to the activity stream, which could produce a big overhead I think.

Actually I had one idea now: I could "nerf" the Activities for the "private" stories by unsetting / setting to undefined the fields I don't want the client to receive (I don't want to publish the stories' ids if they are private).
But is there a straightforward way to do this? I couldn't think of any.
One other solution I could think of would be to combine cottz-publish-relations with some other server side reactivity package and use a tracker to rerun the publication when some criteria of them change - does anybody have a good pattern / experience with this?
Thanks and best wishes
Daniel

before help with a solution let me know if I understood what you need.
But some of those stories should be private, or maybe only visible for certain people / groups. To keep this example simpler, let's say they're private and should only be visible for the user themselves, but not for others.

Want to post activities with stories that are public?

But what if I want to decide whether to publish an Activity only if the Story has some kind of flag set (let's say "public")?

Want to post activities only if the stories are public?

Exactly. I want to post activities only for stories which are public.
Or in the abstract:

Publish from a Collection A only those documents where in a document from a Collection B some condition is true. A references B, like A.id_collectionB -> collectionB-Document._id . Can I reactively publish Collection A with some documents filtered out?

 It would be best to keep the public status within the activity and can solve the problem before moving to find stories.
If the above is not possible other solution could be to use this.observeChanges and send the changes manually only if you find stories, but you would have performance problems because even if you do not send some activities continue to have an observer on them, it would be best to solve the problem by checking the Activities document

Ah, I see, using this.observe / this.observeChanges inside of my publish lets me monitor a collection without publishing it? Yes, that'd probably what I need. I think it'll be efficient enough for my use case, at least it'd scale linearly with the number of activities and not exponentially with it.
Thanks for the clarification, I'll try it that way!


Fix Approvals tests

Fixes approvals test according to server changes.

@MilanPospisil Approvals tests have been flaky, lets make sure we fix the flakiness before re-enabling those tests.

@MilanPospisil Approvals tests have been flaky, lets make sure we fix the flakiness before re-enabling those tests.

I repaired some flakiness last week. Lets rerun this multiple time and we will see.

@MilanPospisil Approvals tests have been flaky, lets make sure we fix the flakiness before re-enabling those tests.

So it run about 5 times without error.


Synchronization of scenes in editor mode;

Scenes from the Scene Manager UI can be synchronized and added to the editor's build settings.
We throw an error if you try to load a scene that is not included in the scene manager on the current platform.
And we have a new platform "Editor", which allows you to add and work with scenes only in the context of the Editor.

Invalid scene colors:

Right now we use yellow when the scene is missing and when the scene is not added to the editor build settings, let' 's use yellow and red. (this PR)

When we need to display a new warning our window jumps. Let's display constant message if we have all ok or if we have problems, and let's also specify those problems (maybe le'ts ad folodout so it won't jump when more problems are adeed). For exmaple, we have more scenes in Editor build settings then in our config, some scenes are missing or the order is incorrect (PR-2)

Let's improve awareness. When we entering into the play  and we know our editor build settings out of sync, let's display a warning message, with an ability to mute those notifications. Mute notifications options should also be present in the scenes warning block. (PR-3)

Throw error when trying to loaded a scene that not present in the plugin configuration (PR-4)

Another Improvement PR

When we add or remove the scene from the plugin settings - this is an explicit action, so we can add or remove the scene from the build setting and display notification.

When Editor build settings is 100% in sycn we can just clean & rebuild it
When it's out of sync we will just add / remove the scene without carying for the order.

Another Improvement PR lets look for the duplicates.
And also please convert those PR requests to issues.


REST POST data to /oauth/token endpoint in body with application/json ContentType and disable HTTP Basic Auth

Hi guys,
I have a two questions, but it is strongly related to each other.
I am starting with Spring OAuth2. So far so good, I have secured my app with the configuration. But I have an issue here, I can not use HTTP Basic Authorization in my client (but it does support it).
Is there a way how to disable HTTP Basic Auth for the /oauth/token endpoint? I would like to send the client_id and client_secret in the JSON body as well with the username and password (or at least with x-www-form-urlencoded ContentType). Is there a way how to make it happen with configuration by annotation?
curl examples I would like to work:
curl -X POST -H "Content-Type: application/json" -d '{
"username": "user",
"password": "pass",
"grant_type": "password",
"client_id": "test-client-id",
"client_secret": "test-client-secret"
}' "http://localhost:9999/api/oauth/token"

OR the cliend_id and client_secret in the headers.
curl -X POST -H "Content-Type: application/json" -H "X-Client-ID: test-client-id" -H "X-Client-Secret: test-client-secret" -d '{
"username": "user",
"password": "pass",
"grant_type": "password"
}' "http://localhost:9999/api/oauth/token"

Thanks a lot.

I finally figure it out. The way how to disable the HTTP Basich Auth is to enable a form authentication for clients. Just add those lines to your configuration.
@Configuration
@EnableAuthorizationServer
protected static class OAuth2Config extends AuthorizationServerConfigurerAdapter {
  @Override
  public void configure(AuthorizationServerSecurityConfigurer oauthServer) throws Exception {
    oauthServer.allowFormAuthenticationForClients();
  }
}

Now I am able to send successful request to TokenEndpoint like this:
curl -X POST -H "Content-Type: application/x-www-form-urlencoded" -d 'username=user_username&password=user_password&grant_type=password&client_id=your_trusted_client_id&client_secret=your_trusted_client_secret' "http://localhost:8080/oauth/token"

But there is still remaining the issue that I want to send data in json format. So the ContentType of my request will be application/json. Anyone have a solution for this?

Looks like it only accepts ContentType as x-www-form-urlencoded. I too look to send as json but no luck so far.

Yes x-www-form-urlencoded is based on the OAuth2 specification. But I have figure it out with some help on StackOverflow. You can see the solution here:
http://stackoverflow.com/questions/38165131/spring-security-oauth2-accept-json/38284206#38284206

Is there any way to Get the response in application/json by default. Actuallly I've integrated OAuth2 in my application and inject a (jackson-dataformat-xml) dependecy to produce the result in xml.
I have also used "configureContentNegotiation" to set default Accept type header in json.
Now My oauth2 layer is above the controller layer. So I am unable to get the response in json by default. I is producing result in xml.
Let me know If anyonw have any idea about, how I can set the value of which will geneate the response in json by default.
Thanks in advance.

```shell
curl -X POST -H "Content-Type: application/x-www-form-urlencoded" -d 'username=user_username&password=user_password&grant_type=password&client_id=your_trusted_client_id&client_secret=your_trusted_client_secret' "http://localhost:8080/oauth/token"

On Using :
oauthServer.allowFormAuthenticationForClients();
I am getting :
2{"error":"unauthorized","error_description":"Error creating bean with name 'scopedTarget.clientDetailsService' defined in org.springframework.security.oauth2.config.annotation.configuration.ClientDetailsServiceConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.security.oauth2.provider.ClientDetailsService]: Factory method 'clientDetailsService' threw exception; nested exception is java.lang.UnsupportedOperationException: Cannot build client services (maybe use inMemory() or jdbc())."}


Recursive load error running doom develop branch on alpine linux 37

Anybody tried running doom develop branch on alpine linux 37 and emacs 26 ?
Observed behavior
vagrant@alpine ~/m/doom> .emacs.d/bin/doom -p .doom.d/ install Recursive load: "/usr/share/emacs/26.1/lisp/jka-compr.el.gz", "/usr/share/emacs/26.1/lisp/jka-compr.el.gz", "/usr/share/emacs/26.1/lisp/jka-compr.el.gz", "/usr/share/emacs/26.1/lisp/jka-compr.el.gz", "/usr/share/emacs/26.1/lisp/jka-compr.el.gz", "/usr/share/emacs/26.1/lisp/emacs-lisp/subr-x.el.gz", "/home/vagrant/m inecraft/doom/.emacs.d/core/core-lib.el", "/home/vagrant/minecraft/doom/.emacs.d/core/core.el", "/home/vagrant/minecraft/doom/.emacs.d/init.el", "/home/vagrant/minecraft/doom/.emacs.d/bin/doom"
Expected behavior
Running .emacs.d/bin/doom -p .doom.d/ install
and Start install o refresh
Steps to reproduce
`.emacs.d/bin/doom -p .doom.d/ install
System information

doom develop branch
alpine linux 37 on Virtualbox
emacs-nox 26
chemacs with profile

Perhaps Emacs can't decompress jka-compr.el.gz. Do you have gzip installed on that system? And if not, does installing it fix it?

Thanks for your reply.
Installed gzip but not fix it.

I found similar errors in other emacs distribution. Seems like a bug of linux.
prelude issue
After add this line in .emacs.d/init.el
(setq load-prefer-newer nil)
And run
doom refresh
I got
⋊> ~/m/cleandoom .emacs.d/bin/doom -p .doom.d/ refresh                                                                                21:51:50
Initializing modules
Wrong type argument: hash-table-p, nil

instead of using doom refresh
make quickstart
make install

in .emacs.d directory started successfully the package install

I've pushed a fix for the hash-table-p error, but this load-prefer-newer fix is very interesting. Perhaps the built-in elisp libraries were byte-compiled with the wrong/older version of Emacs? Very strange.

Thanks for the quick fix

Hmm, I can't say I understand the problem, but I'll consider this issue resolved and keep the load-prefer-newer solution in mind. Thank you for debugging this and sharing your findings with me!

For reference: The current Emacs 26.3 build on Alpine Linux is broken and causes the above error (at least on my system) - to the person reading this, if you have the same trouble, you should try doing:
git clone --depth 1 git://git.alpinelinux.org/aports
cd aports/community/emacs
export REPODEST=$HOME/.packages
abuild -r
sudo apk add --repository $REPODEST/community emacs


Attributes collision using BelongsToMany + cursor()

Laravel Version: 5.5.21
PHP Version: 7.1
Database Driver & Version: mysql 5.7

Description:
When looping through a belongsToMany association using cursor(), the models aren't correctly hydrated: they get filled with values of the pivot table. If the pivot table have columns with the same name as columns in the related model table, pivot values are used to hydrate the related model.
Steps To Reproduce:
DB setup

users table: id, name, timestamps
groups table: id, name, timestamps
group_user table: id, name, timestamps, user_id, group_id, approved (boolean)

Relationship defined as such:
class User extends Model
{
    public function groups()
    {
        return $this->belongsToMany(Group::class)
            ->withTimestamps()
            ->withPivot('name', 'approved');
    }
}

Actual code showing bug:
$john = User::create(['name' => 'John']);
sleep(1);

$group = Group::create(['name' => 'subscribers']);
sleep(1);

$john->groups()->attach($group, ['name' => 'john_subscription', 'approved' => true]);

foreach ($john->groups as $group) {
    dump(get_class($group));
    dump($group->getAttributes());
}

foreach ($john->groups()->cursor() as $group) {
    dump(get_class($group));
    dump($group->getAttributes());
}

Outputs:
"App\Group"
array:4 [
  "id" => "1"
  "name" => "subscribers"
  "created_at" => "2017-11-20 11:17:51"
  "updated_at" => "2017-11-20 11:17:51"
]
// ^ using $john->groups as $group produces a correct $group model.

"App\Group"
array:7 [
  "id" => "1"
  "name" => "john_subscription" // <-- wrong, that's the pivot name
  "created_at" => "2017-11-20 11:17:52" // <-- wrong, that's the pivot creation date
  "updated_at" => "2017-11-20 11:17:52" // <-- wrong, that's the pivot update date
  "user_id" => "1" // <-- should be in pivot, not in $group attributes
  "group_id" => "1" // <-- should be in pivot, not in $group attributes
  "approved" => "1" // <-- should be in pivot, not in $group attributes
]
// ^ using $john->groups()->cursor as $group produces a corrupted $group model.

I had actually proposed a PR to fix this (#22126), but it was rejected because: "breaking change". So instead of attempting to fix it myself, I figured I would post a bug report :)

Related:
https://github.com/laravel/internals/issues/347
https://github.com/laravel/framework/issues/16704
https://github.com/laravel/framework/issues/21774
https://github.com/laravel/framework/pull/22127

A dirty workaround:
foreach ($john->groups()->cursor() as $group) {
    $group->id = $group->attributesToArray()['group_id'];
    dump($group->getAttributes());
}

@iceheat I don't understand why you ref this issue in https://github.com/laravel/framework/pull/25015 if your PR is not related to #22144?
@PGBI has already reported an issue with BelongsToMany & cusor here with the steps to reproduce.
Could you try it on a fresh install? :)
Best

@sylouuu My apologies for referencing this issue on PR that does not fix it. I blindly assumed that the issue you were complaining about was about hasManyThrough and cursor() since you were asking about it in a pr addressing hasManyThrough and chunk().
I had a quick look at the code for BelongsToMany. I believe I have a grasp of what is causing the issue. I will try to dedicate some time over the evenings/weekend to write a few integration tests before introducing changes because this can easily introduce breaking changes.
Cheers,

@staudenmeir Feel free to cherry-pick my commits and resubmit the PR. Given how my PR was rejected without the slightest start of explanation, I'm not willing to submit it again.

We just ran into this issue and fortunately in our case, we didn't modify related models' attributes whilst using a cursor. But should one do so, it could very well update nothing or completely unrelated records.
I don't understand why the original PR was rejected by @taylorotwell, because while it is technically a BC breaking change, the current behaviour is not only broken, it is outright dangerous and this should be considered a critical issue.
I've set up a quick and dirty example of this in a PHPUnit test case; the test case actually tests the current, broken behaviour on purpose. See https://github.com/Werelds/laravel-broken-cursor-demo
I'm going to create a fresh PR for this, because this has to be fixed. In its current implementation, one can not use cursors on at least BelongsToMany because it is inherently broken. It seems like it was fixed on some of the other relationship types, but for my own peace of mind I will test this against any relationship type that uses pivot tables.

This seems to be fixed in the current release. Thanks @mpyw and @PGBI for the original pr.


Allow to execute crud actions without redirect

I'm making a form where users can batch create multiple resources at once. This change lets me reuse existing actions, but wait with notification/redirect until all resources are created.
It's a bit annoying to get redirected say ten times in a couple of seconds:)

I like it. But the name silent doesn't speak to me. Could you rename it to redirect and make it default to true instead?
Also, why the change on crudGetList?

Awesome. Renamed to redirect.
The change on crudGetList allows me to fetch multiple resources in one go. Also something I need in my custom batch create.
Sorry I forgot to mention it in the original request. Do you want me to make it a separate PR?

Yes, please make atomic PRs.

Moved change to crudGetList to #391

Thanks!


Test development

get_study_est_sspse():
added functionality to pass arbitary number of sspse::poteriorsize() arguments from get_study_est_sspse() to sspse::posteriorsize().

Looks good @till-tietz . Can you also document the new argument in roxygen skeleton and document the package and pull again?

Perfect. Documented the new argument with devtools::document(). Does the documentation need to be updated anywhere else?


Upgrade actions/setup-java v3.13.0 -> v4.0.0

This PR contains the following updates:

Package
Type
Update
Change

actions/setup-java
action
major
v3.13.0 -> v4.0.0

Release Notes

actions/setup-java (actions/setup-java)
v4.0.0
Compare Source
What's Changed
In the scope of this release, the version of the Node.js runtime was updated to 20. The majority of dependencies were updated to the latest versions. From now on, the code for the setup-java will run on Node.js 20 instead of Node.js 16.
Breaking changes

Update Node.js runtime to version 20 by @aparnajyothi-y in https://github.com/actions/setup-java/pull/558

Non-breaking changes

Adding support for microsoft openjdk 21.0.0 by @ralfstuckert in https://github.com/actions/setup-java/pull/546
Update @actions/cache dependency and documentation by @IvanZosimov in https://github.com/actions/setup-java/pull/549
Implementation of the cache-dependency-path option to control caching dependency by @itchyny in https://github.com/actions/setup-java/pull/499

New Contributors

@ralfstuckert made their first contribution in https://github.com/actions/setup-java/pull/546
@itchyny made their first contribution in https://github.com/actions/setup-java/pull/499

Full Changelog: https://github.com/actions/setup-java/compare/v3...v4.0.0

[ ] If you want to rebase/retry this PR, check this box

Suggested commit message:
Upgrade actions/setup-java v3.13.0 -> v4.0.0


Directory service should use HTTP temporary redirect (307)

Motivation
HTTP redirections in discovery service component need to be done with 307 code instead of 302, as the broker is already doing.
When redirecting with 302, the HTTP method gets converted into a GET in the subsequent requests, when originally it might have been a POST or a PUT request.

@merlimat this fix has caught a bug in test-case as well:
Failed tests:
  DiscoveryServiceWebTest.testRiderectUrlWithServerStarted:99 expected [Cannot set replication on a non-global namespace] but found [Cannot get the replication clusters for a non-global namespace]

Can we change assert-string as : Cannot set replication on a non-global namespace

Yes, the test was sending a POST and comparing with the response for the GET operation. I'll fix that

👍


added carousel

Related Issue
Closes #807
Description
Added carousel to the contributors section of the about page.
Screenshots

https://www.loom.com/share/f5a05c6916db467390f37dfe98aecb2b?sid=f1e81d06-ff2c-45f7-b8e1-5a14713c7fcd
[Video Preview]
Checklist

[x] My code adheres to the established style guidelines of the project.
[x] I have included comments in areas that may be difficult to understand.
[] My changes have not introduced any new warnings.
[x] I have conducted a self-review of my code.

Add  "npm install react-slick slick-carousel" to package json to make it work!

@SiddhantPawar03 review please


Added Optimised Solutions for Solving Anagram problems

Hello Myself Yatin,
I just started to code using java.
I saw your repo it's amazing, very kind solutions were noted down.
I changed one of your solution to a better and optimized solution that will help a newbie to learn and understand the code faster and help to solve anagram problems easily.
Please except my pull request as i did it very sincerely and honestly for well being for newbie.
Have a nice Day!!

Hello Myself Yatin,
I just started to code using java.
I saw your repo it's amazing, very kind solutions were noted down.
I changed one of your solution to a better and optimized solution that will help a newbie to learn and understand the code faster and help to solve anagram problems easily.
Please except my pull request as i did it very sincerely and honestly for well being for newbie.
Have a nice Day!!


Strip dimensions

Still doing some testing, but so far this looks like it's stripping out the dimensions properly and having Cloudflare resize based on the original image.

Looks like the regex is fine and I do not see any issues. However due to your editor or whatever happened there is a horrible conflict and I cannot merge this PR.
Please fix the code (remove the horrible spacing) and make a new PR and I will merge it.
Many thanks.


[BUG]

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.
[X] I have thoroughly looked through the available Wiki articles and could not find a solution to my problem.
[X] I am using the latest version of Bloxstrap.
[ ] I did not answer truthfully to all the above checkboxes.

What problem did you encounter?
I was typing, which could be a part of it, but when I was typing in a game, my monitor temporarily blackscreened for no reason, and during that time, roblox crashed without the normal roblox crash message. When roblox disappeared during the black screen, other apps stayed open, so its likely on roblox or bloxstraps end. I sadly can't tell the difference between it being roblox or bloxstrap, as theres no evidence that would show what caused it.
Bloxstrap Log
N/A

Has this happened multiple times, or only a one-time occurrence? Seems like a GPU issue, but I could be wrong.

but when I was typing in a game, my monitor temporarily blackscreened for no reason, and during that time, roblox crashed without the normal roblox crash message

That kind of sounds like just you ran out of memory. Does this happen consistently or did this just happen one time?

one time

Closing as not a Bloxstrap issue


Contributing instructions in README.md could be clearer, fixes #77

@houli This should fix Mac OS issues and also clarifies things per what you added in https://github.com/exercism/purescript/issues/77.
I don't have a Mac OS to test this out, so if you have a moment, please clone this:
https://github.com/icyrockcom/xpurescript
checkout macos-bower branch and let me know if it works - specifically, running bin/check-bower.sh, bin/test.sh and bin/update-bower.sh.

@houli Just checking if you had a chance to check this update, to see whether it works on your end.

I've not tested this on OSX, shall we delay merging?

@lpil Sure, I'm fine with waiting. No particular rush, just don't want to have it stay in pending for long.
If @houli has a chance to test, that would be great. Wait another couple of weeks perhaps and merge then, what do you think?

Looks good to me

Great, thanks both!


Code Generation

Need to create some code generation scripts to ensure the pattern is followed.  Consider defining the schema using Json or Yaml

Need to create some code generation scripts to ensure the pattern is followed. Consider defining the schema using Json or Yaml
Look at this
https://github.com/neozhu/CleanArchitectureCodeGenerator

@neozhu thanks for the link.


Individual components fail to load

Asks for the same relative modules every single time no matter which component I try to load individually. Here is a clear request. I also followed the documentation's example.

Thanks @luisDanielRoviraContreras
Solved by installing Vue CLI 3's pre-processors
For future reference:
https://cli.vuejs.org/guide/css.html#pre-processors


Node.isNearby is not reliable

DataLayerAppHelper class contains a method to get connected nodes. This method filters out nodes that are "nearby" internally.
val nearbyNodes = connectedNodes.filter { it.isNearby }

I find this method not reliable and as a result I'm not getting my connected nodes. When debugging I've noticed that both for my Pixel Watch and Watch emulator isNearby returns false.
What's the purpose of this filter? Without it I'm able to get and communicate with all my nodes.
Library version: 0.5.18

Just to let you know that we will make a release next week, planning for Tue, will keep you posted of any changes.

Should be fixed in 0.5.21


[tools revamp] add bq sql query for timeline

add callback function that gets bq sql query for timeline

thanks for the review


fix query param for testnet estimate fee

You commits have been merged into main has part of #151. This PR is now a duplicate PR


[ML] fix(next-pylint): Resolve W0602, W0611, W0707, W1414, W2301 and W4905

Description
This pull request is part of a effort to ready the Azure ML Python SDK for an impending bump to the versions of pylint and azure-pylint-guidelines-checker that is used in our CI.
This pull request specifically resolves some pylint messages under the "W" code.
Validated with:
pylint==2.15.8
azure-pylint-guidelines-checker=0.0.0
pylint --rcfile=../../../eng/pylintc --disable=all --enable=W azure/

autoflake configuration used

Version: 2.1.1
Config:# pyproject.toml
[tool.autoflake]
exclude = ["_vendor","_generated","_restclient","samples","examples","test","tests","doc",".tox"]

Invocation (from sdk/ml/azure-ai-ml/): autoflake -ir azure/

tryceratops configuration used

Version: 2.1.1 (this latest version I was able to get autofix working with)
Config:[tool.tryceratops]
exclude = ["_vendor","_generated","_restclient","samples","examples","test","tests","doc",".tox"]
# Only want to autofix TRY200
ignore = ["TRY201", "TRY400"]

Patches:

Changed the default exception name used to fix TRY200 from ex to e : https://github.com/guilatrova/tryceratops/blob/e22c7f38c64f4f7aca0d99813d688a8be6ac7984/src/tryceratops/fixers/exception_block.py#L30
Changed exclude logic to be exact match instead of substring match: https://github.com/guilatrova/tryceratops/blob/e22c7f38c64f4f7aca0d99813d688a8be6ac7984/src/tryceratops/filters.py#L80

Invocation (from sdk/ml/azure-ai-ml/): tryceratops --autofix azure/

All SDK Contribution checklist:

[x] The pull request does not introduce [breaking changes]
[x] CHANGELOG is updated for new features, bug fixes or other significant changes.
[x] I have read the contribution guidelines.

General Guidelines and Best Practices

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[ ] Pull request includes test coverage for the included changes.

API change check
API changes are not detected in this pull request.


file-import does not support --server option

The file-import option does not recognize the --server option:
$ ./riot file-import --server localhost:6380 --file https://git.io/fjxPs
Unknown options: '--server', 'localhost:6380'

Redis connection parameters come before the command (file-import in your case). Try this:
$ ./riot --server localhost:6380 file-import --file https://git.io/fjxPs

Thanks.


Clean up our OWNERS files.

Drop people who have not contributed to the Crosswalk Project in a long
time. When they were the sole OWNERS in a given directory, the file
itself was removed.

@baleboy @darktears @halton @lincsoon

LGTM, thanks

of coz meaning PR!


Add Contains Ignore Case option to Seek Row functionality

We currently have Seek Row functionality in grids for searching for text within a column. There are only a few options there, we should add the Contains Ignore Case option as well.

Context: https://illumon.slack.com/archives/CAUG7MC2D/p1683398883944149

These should use human labels -- "Equals (ignore case)" or whatever we've done elsewhere.

@ethanalvizo you can create a table with a string column to test this feature. From our docs (https://deephaven.io/core/docs/reference/table-operations/create/stringCol/):
from deephaven import new_table
from deephaven.column import string_col

result = new_table([
    string_col("x", ["Deephaven", "DEEPHAVEN", "deephaven", "illumon", "ILLUMON"])
])

Then in the table that appears, open the table sidebar and click the "Go to" option to display the Go to feature:


testing ads are not shown for rewarded video

Thanks for your help guys, I just moved from admob-pro and I am very happy with your solution!

Actually the issue is not critical and can be fixed manually by providing test ID manually. The problem is that passing isTesting causes lots of troubles in rewarded video.

There is a similar issue with prepare methods for both interstitial and rewarded. For example, try to pass config directly in prepare method and autoShow: false will be ignored =)

I would like to make some input, please add this instructions which will save hours of other developers:

Make sure that additional libraries are installed via "Android Studio", "SDK Manager" in "SDK Tools":
Android Support Repository
Google Play services-ads
Google Repository

Manually add this line to platforms/android/project.properties:
cordova.system.library.3=com.android.support:support-core-utils:26.+

Make sure that play-services-ads has SAME version as firebase-core (if you are using firebase plugin), simply change both to "11.+" or "15.+"

With best regards
Yehuda

Did you have any solution ?

Re: [ratson/cordova-plugin-admob-free] testing ads are not shown for rewarded video (#243)
Hello Oğuzhan,
Sorry for late reply, I was trying to find solution with another issue (freezing app while sending first request).
As for the solution - I simply hard code in app replacing my key with public test keys... also sending empty key works for rewarded videos I think but does not work for interstitial =)
I can go deep and debug addon (js + java) but I have no clue how can I do that. When I tried to make some changes in local files stores in plugins/cordova-plugin-admob-free folder, all modifications were overwritten with next cordova run command. If you can guide me how can I do that then I will search and fix the bug.
Waiting for your reply
Yehuda
Friday, June 29, 2018, 10:37:23 PM, you wrote:
Did you have any solution ?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

I have problems i try get reward in video reward but freeze after vídeos end. I m use ter cordova 6.5.0 Android api 26 admobpro last version.


Dlib 194 points

My error "The full_object_detection must use the iBUG 300W 68 point face landmark style or dlib's 5 point style."
How do I use more points in the compute_face_descriptor?

You don't, and there isn't any point in trying to do so.

Why does not it make sense? Or how to understand this?


Integrate s3 coordinator with helm

Make helm honor s3 coordinator as values option.
Enable incremental sync, verified that this stuff worked.
Fix bug with missed cp-init
Fix bug with missed operationID passing to activation.

:shipit:


FXAA mess brightness when using World Environment Tone mapping

Godot version:
3.2.4 Beta 1
OS/device including version:
Windows 10
Issue description:
FXAA mess your scene brightness when using World Environment Tonemap other than default settings.

Steps to reproduce:
See above
Minimal reproduction project:
TesteFXAA.zip

Are you using GLES2 or GLES3?

looks GLES3 according to driver name on top right corner of gif.

Are you using GLES2 or GLES3?

GLES3

Same bug is in master and 3.2. I think I know what the issue is.


Feat: Created Videos page

Reference:
https://www.loom.com/share/03c236101f3c4cea81bd5eb70002488a?sid=581c0fc1-d340-47b0-b80f-03914dfe423d
Fixes:
#509
Close #509

Hey @sanjay-kv,
I would really appreciate if you would tag this PR as level 3, as I have worked with all HTML, CSS and JS.
Thank You


CAMEL-18127: added adapter auto-configuration for AWS2 Kinesis

Demo (to be added to the examples repository)

How are the adapters discovered? Some kind of classpath discovery? If so we have the existing FactoryFinder that we use for everything in Camel ala components, languages, health checks, and other things.
The manifest files are stored in META-INF/services/org/apache/camel/...
I noticed that in this PR that it may not be the location for this?


Pass CLANG_VERSION_* cmake flags to the LLVM build.

In a normal swiftless clang build, if it doesnt know CLANG_VERSION_,
clang's cmake falls back on LLVM_VERSION_. We pass a CLANG_COMPILER_VERSION
to swift, but it is not necessarily what clang wants to know.
The intention of this CLANG_VERSION_* flag in Clang is to know
what Swift calls clang_user_visible_version. This is the non-specific
version of Clang. For example, clang-900 - this is 9.0.0, while Swift
would like clang_compiler_version to be the specific version that the
driver will print out.
Flag naming!
Fix tests

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

@swift-ci test and merge

@swift-ci Please smoke test and merge


Bump js-debug to 1.78

A pipeline issue prevented publishing js-debug at the normal time, but we should take the update in order to have fixes and tests for the release.

Issues not on nightly as a result of the same issue https://github.com/microsoft/vscode-js-debug/compare/1fc48e57096379620125f6231e4c32c39dc51723...v1.78.0

Thanks, LGTM


Simplify API of data-managing widgets a bit, and document

Another step toward #21 and #13.

LGTM, thanks! Merging.


We need reference bank ...

This will be easier soon! http://build-me-the-docs-please.readthedocs.org/en/latest/Using_Sphinx/UsingBibTeXCitationsInSphinx.html right now, this is a really young package, but we can test it out

It looks like using: https://sphinxcontrib-bibtex.readthedocs.org/en/latest/usage.html could be a good option. Citations are then :cite:`citekey` . It doesn't yet have a nice author, year inline citation: it uses the author's last name and year mushed together (it would use [WH88] for Ward & Hohmann, but it can be customized). I am giving it a try on the "citation" branch.

closed with #50


[WIP] Rework the packs pack (more packs-related ChatOps!)

Aliases for the packs pack:
— pack install for installing a pack or multiple packs (just a minor rework here);
— pack search to query the index;
— pack show to display information about an installed pack, including HEAD location in the git tree (N commits ahead/behind);
— pack show available to fetch information about a pack from the index.

Most of packs.deploy is now obsolete, so I'm removing it and the helper actions for now, but I would appreciate @jjm finding some time to talk it over with me. A thing or two that was possible with packs.deploy might not be covered by our new pack management, and if that's the case, I'm fine with keeping the deploy workflow or breaking it down into multiple actions.

There was a linux pack directory and an action. Inside the packs pack. Apparently, for quite some time, too (I remember @enykeev noticing it). I removed it. Sorry, not sorry.

Some minor cleanups (this PR started as a minor cleanup, actually, but then things just got more and more interesting).

WIP! Like, really WIP. Don't test it yet, there's still some work to be done.

Btw, pack show available <pack> shows content count if it's present in the index, but pack show <pack> doesn't, because we don't have helper functions for content counting yet. @Kami promised to get it done :)

Current coverage is 76.55% (diff: 100%)

Merging #2982 into master will increase coverage by 0.12%

@@             master      #2982   diff @@
==========================================
  Files           423        423
  Lines         21689      21689
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits          16576      16602    +26
+ Misses         5113       5087    -26
  Partials          0          0

Powered by Codecov. Last update f3cc783...cc598c6

@emedvedev Yes, I think packs.deploy and it's associated actions can be removed. In fact GitHub pack deployment_event workflow that I added earlier in October does not call packs.deploy and calls packs.install directly.
I think with the release of Stackstorm-exchange the bitbucket rule post_receive_webhook.yaml

In fact GitHub pack deployment_event workflow that I added earlier in October does not call packs.deploy and calls packs.install directly.

Once we make the move, that rule and workflow will become a bit simpler, too!

      input:
        packs: [  <% $.ssh_url %>=<% $.deploy_ref %> ]

I'll skim through all the packs to make sure all the packs.install and packs.download references are updated when the packs are moved.

So it's fixed & Polished & Acceptance tested in Slack.
The new list of ChatOps Action Aliases is adjusted to conform with CLI commands for consistency.

!pack install (screenshot) - Install pack(s)
!pack get (screenshot) - Get info about locally installed pack
!pack search (screenshot) - Search for pack(s) in StackStorm Exchange
!pack show (screenshot) - Show pack info from StackStorm Exchange

The Aliases automated tests as well as Integration Tests (e2e, robo) are in my TODO list and will be added, but It's good to merge now to see things in pre-production (packages).

Additionally, I'll need staging-unstable packages to make sure that it works in EL6 (outdated git).

:heart: :heart: :heart: :heart:
LGTM.

enykeev	[10:30]
!pack show st2

stuffBOT	[10:30]
@enykeev: Getting back to you about the `st2` pack:
Here's the full index entry at StackStorm Exchange:
{author: st2-dev, description: StackStorm pack management, email<EMAIL_ADDRESS>    name: st2, repo_url: 'https://github.com/StackStorm-Exchange/stackstorm-st2',
    version: 0.1.2}

To install the pack: `pack install st2`
If the pack is already installed, I will upgrade it to the latest version, keeping the config.

I'm kinda missing hubot alias before the command, though I'm not sure we can get it now in any sane way.
Also, https://github.com/StackStorm/st2/pull/3051 to prevent different kinds of output formatting for different packs.
Other than that, looks good to me.

I'm kinda missing hubot alias before the command, though I'm not sure we can get it now in any sane way.

Yeah, @enykeev I just verified, and there is no hubot alias variable in context. I bet none could think that it might be useful to have in context.  Might be something good to adjust in future.

Nice work everyone!


chore(typings): fix Strategy-All typings

closes #187
Atm, StrategyState.array uses syntax that causes build issue with tsc, so tweak the typings to something less ideal, but should enable folks to continue to build without any issues. While this could break folks, it should be really minor as: the release is still new, and it's internal
cc @DrSammyD @fkleuver @EisenbergEffect
(I'm opening this instead of #189 since the other one failed for some unclear reasons)
Edit: also pushed another changes removing const enum so folks who use typescript together with babel loader wont have any issues closes #188

Since we removed the const enum, should we move the StrategyState to use numbers for keys instead of referencing the enum? Or do we want to take maintainability over a what is probably a pretty minor perf difference? @fkleuver

The perf difference would be negligible for this usage. I'd vote for maintainability


RFC: Release 0.19

@mne-tools/contributors we are trying to push out 0.19 in the next week or so. I've gone through and tagged issues and PRs that I think should go in 0.19. Feel free to also have a look and comment if there are any you think should and realistically can make it in during that time.

Closing as 0.19 should be doable this week, all that is left is whatever is remaining for deprecation in #3893 plus a bit of docs in #6767


Embed webpages

Looking to move to mf. The only feature I would strongly miss from Thunderbird is opening webpages with the reader's interface (most importantly, keyboard shortcuts!) being available.
An example with Nitter (feed URL: https://n.c7.ee/thepine64/search/rss?f=tweets)

Miniflux, normal. Problem is that there is no context:

Miniflux, original. The formatting is quite off an hard to understand. With 3-tweet threads this is ok, but with 20-tweet threads, having the original webpage is very very useful:
I noticed, that different USB adapters charge my #Pinepone at vastly different speeds, all between 0.5W (barely at all) or up to 4W (relatively rapidly). Is there a preferred rate? Or more so, can too rapid/slow damage my battery?

1

0

1
Ping @thepine64

1

0

1
Replying to @JayVii_de

Your PinePhone can charge at up-to 15W

7:50 AM · Jul 31, 2020

1

0

0

It is possible to open the webpage on miniflux, but this gets bad:
Thunderbird: Press t to mark as read and next
mf: Press v to open in new tab, press Ctrl+w to close tab, press → to load next item.
A macro might help, but the loading times for one tweet/page get very long. This is not good with >150 new tweets per day, where on Thunderbird I might go through a page per 2-3s.

Sorry, I'm not planning to implement that kind of feature. It's out of the scope of this software.


add close button to error boundary

Fixes https://issues.redhat.com/browse/RHOAIENG-2650
Description

Adds a close button to the error boundary. Clicking the close button resets the internal error state causing the error boundary to unmount. When this occurs, the app will resume loading as if no error occurred by rendering the contents of the current URL.

How Has This Been Tested?

Cause an uncaught error to occur.
Through the web console, add a breakpoint to a react component. Once the break point is hit, edit a variable such that an exception will be thrown attempting to access a property of an undefined variable
Test Impact

None
Request review criteria:

Self checklist (all need to be checked):

[x] The developer has manually tested the changes and verified that the changes work
[x] Commits have been squashed into descriptive, self-contained units of work (e.g. 'WIP' and 'Implements feedback' style messages have been removed)
[x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[x] The developer has added tests or explained why testing cannot be added (unit or cypress tests for related changes)

If you have UI changes:

[ ] Included any necessary screenshots or gifs if it was a UI change.
[ ] Included tags to the UX team if it was a UI/UX change (find relevant UX in the SMEs section).

After the PR is posted & before it merges:

[ ] The developer has tested their solution on a cluster by using the image produced by the PR to main

This works with debugging
/lgtm


improve vcpkg support for linux.

TLDR:
When using vcpkg, machine architecture should change  x86_64  to x64 not only on macos but also on linux .
Detail:
Run ./vcpkg help triplet
it show that the valid options of target machine architecture are as follows:
Available architecture triplets:
  arm-uwp
  arm-windows
  arm64-uwp
  arm64-windows
  x64-linux
  x64-osx
  x64-uwp
  x64-windows
  x64-windows-static
  x86-uwp
  x86-windows
  x86-windows-static

also refer to https://vcpkg.readthedocs.io/en/latest/users/triplets/#VCPKG_TARGET_ARCHITECTURE.

Thanks!


a little explanation needed about new mod option v5.6, 5.7.

is you removed the auto encode and add the conversion after the desired duration finished ?
that take time to process maybe ?..
the mode auto or manual referes to the track when account is online or it's for ffmpeg real time conversion ? or you just removed that real time conversion as it reason for destroyed "pixelated , boxed" video output ?
, can be an option linke , keep recordings at .flv and the user manually convert them later ? in case of setting low duration like 5 minutes it keep remake new files after 'like instance 5 minutes' and immediately start new file as keep with the stream without waiting the flv file to convert ?

The flv to mp4 conversion is instant.
I removed the recording via ffmpeg because I felt it was unnecessary


feat: expose secondary_storage and additional_security_group_ids in worker_pool variable

Issue: https://github.ibm.com/GoldenEye/issues/issues/11574
Description

Release required?

[ ] No release
[ ] Patch release (x.x.X)
[x] Minor release (x.X.x)
[ ] Major release (X.x.x)

Release notes content

Run the pipeline
If the CI pipeline doesn't run when you create the PR, the PR requires a user with GitHub collaborators access to run the pipeline.
Run the CI pipeline when the PR is ready for review and you expect tests to pass. Add a comment to the PR with the following text:
/run pipeline

Checklist for reviewers

[ ] If relevant, a test for the change is included or updated with this PR.
[ ] If relevant, documentation for the change is included or updated with this PR.

For mergers

Use a conventional commit message to set the release level. Follow the guidelines.
Include information that users need to know about the PR in the commit message. The commit message becomes part of the GitHub release notes.
Use the Squash and merge option.

/run pipeline

/run pipeline

:tada: This PR is included in version 4.2.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Prevents upgrade of puppetlabs-java module to 2.0.0

Currently puppet-jenkins allows puppetlabs-java only to be < 2.0.0, but 2.0.0 has been released. Upgrading puppetlabs-java to the latest version is therefore not possible at the moment.

I, too, wonder!

The same is true for puppetlabs-apt... what are the plans for the next release? Especially related to #772

Any news here? :(

PR requested.

see GH-801


Complete test suite using for ion-schema-tests submodule

There are still remaining tests for some of the constraints defined in ion-schema-test submodule. Add all the test files in constraints folder to ion-schema-test-runner.

Fixed with #132


Added decline reason to enums.py

Added decline reason 'AUTHENTICATION_REJECTED_BY_CARDHOLDER' to 'Undocumented in Monzo API docs' after running into error running code

Thanks for this! Hopefully this will no longer be an issue with https://github.com/pawelad/pymonzo/commit/7f330acebb8601bb2e02017412776a1216a76890 🤞


App crashes when scroll down from Intro section

Open a tN resources(.tsv file)
Scroll down from the introduction section
as soon as it reaches the next Block the App crashes.
Blank screen.zip

This is happening for OT books only. NT books are loading fine.
Seems like when the Parallel scripture section comes up the App crashes

Test comment

Tested in the Test Build at https://develop--glts.netlify.com./.
The App does not crash while loading Parallel Scriptures ion OT books anymore, But when we switch the book from Menu, the Intro changes to the new book but the Scripture part does not get refreshed to the new one but shows the previous book instead.

@elsylambert Do you mind if we either close this issue and create a new one with the new behavior? They aren't directly related. We can rename this issue if you'd like.

App no longer crashes on scrolling down to Parallel scripture viewer.


feat(deps): Add a command to install external deps

Command to install external dependencies of a project
As sfpowerkit is getting deprecated, sfpowerscripts
will feature a command to install external dependencies
Checklist
All items have to be completed before a PR is merged

[x] Adhere to Contribution Guidelines
[x] Updates to Decision Records considered?
[x] Updates to documentation at DX@Scale Guide considered?
[x] Tested changes?
[x] Unit Tests new and existing passing locally?

Codecov Report
Base: 61.14% // Head: 61.14% // No change to project coverage :thumbsup:

Coverage data is based on head (6aae463) compared to base (67ed4d2).
Patch has no changes to coverable lines.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #1208   +/-   ##
========================================
  Coverage    61.14%   61.14%
========================================
  Files           59       59
  Lines         2015     2015
  Branches       436      436
========================================
  Hits          1232     1232
  Misses         733      733
  Partials        50       50

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Allow Laravel 7

Laravel 7 is coming next week, so updated this to prepare/test with L7

Released on v4.23.0. Thanks!


Change/allow empty string

I know this is not joi compatible, but there is also not much sense is explicitly disallowing empty strings, empty strings are still strings, if we want to enforce a minimum length we should do so through min.

I agree with the sentiment of this PR and, as there are few users of this library outside of Beam, it's OK. I'd just add a note in compatibility.md for situations like this where we have the same rule that behaves differently than Joi.


Multiple prompts for non-managed devices

Adding additional text to explain OAuth Refresh Token are per application and do not share MFA state.  This can cause multiple MFA prompts if the device does not have an identity in AAD to pass a PRT.

@dakoer : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


Unexpected behavior of "+" operator

Describe the bug
There seems to be a bug with an expression like
A (B)+ @C
(reversing through an expression in brackets with a "+" operator)
To Reproduce
Steps to reproduce the behavior:
Go to https://dopamin.averbis.intern/iruta/notebooks/user/klumpp/Ruta-Bug-23-08-21.ipynb or see screenshots below.
Expected behavior
In the example on iRuta above, I would expect MyAnnotation on "a".
Screenshots
Unexpected behavior:

Correct behavior (without brackets):

Thanks for reporting this problem. It will be fixed in the upcoming release.


did not showing anchor

i changed code only location in hellorArExample. but did not showing anchor.
how can i use?
I only changed the longitude and latitude

Hi @hhong0326 . Do you get any errors in logcat that could help us diagnose your issue?


Integrate local SIFT features from OpenIMAJ library

Use the latest implementation of OpenIMAJ library in order to integrate the SIFT feature extractors both Gaussian and Dense regarding accuracy and degradation factors.
Update: This issue created back in 2015-12-17 those imported in github manually.

Closing issue #5 working on f0c0a5c, upgrading to version 1.1.4.
Update: This issue closed back in 2016-02-28 those added in github untimely.

Closing issue #5 working on f0c0a5c, upgrading to version 1.1.4.
Update: This issue closed back in 2015-12-21 those added in github untimely.


linux_wallet_x64.deb cannot start

linux_wallet_x64.deb cannot start

$maxcoin-qt
maxcoin-qt: error while loading shared libraries: libssl.so.1.0.0: cannot open shared object file: No such file or directory

On the desktop did not appear a shortcut of this Wallet

OS: Debian 9/64

Why does no one answer? So the project is in a state of stagnation??

Hi Abdula, I hope you realised you only posted your comment/issue only 22 hours ago, and we are busy working on other projects, and we may not get to requests as soon as you are expecting.
Thanks for letting us know your issue, we will look into it

sudo apt install libssl1.0.0

sudo apt install libssl1.0.0
The libssl1.0.0 package is not available,
2) So this is a bad method.
**libssl1.0.0** must be put inside **linux_wallet_x64.deb** so as not to search for it

libssl1.0.2 I have installed originally (libssl1.1 too).
Alas, it does not help :(

give me some more info so I can help...
here's what I have on my laptop:
$ dpkg -l | grep libssl
ii  libssl-dev:amd64                                            1.0.2g-1ubuntu13.5
and everything works well. what's your output of
$sudo apt-cache search libssl
did you try 'make clean' then 'make' after updating the ssl deps?
what errors are you getting? I don't know configuration of your OS so I can only help if you show me what's failing...and which libs you have installed.

Please,
$ dpkg -l | grep libssl
ii  libssl1.0.2:amd64                      1.0.2l-2+deb9u3                      amd64        Secure Sockets Layer toolkit - shared libraries
ii  libssl1.1:amd64                        1.1.0f-3+deb9u2                      amd64        Secure Sockets Layer toolkit - shared libraries

$sudo apt-cache search libssl
libssl1.0.2 - инструментарий Secure Sockets Layer — разделяемые библиотеки
libssl1.1 - инструментарий Secure Sockets Layer — разделяемые библиотеки
cl-plus-ssl - Common Lisp interface to OpenSSL
dcmtk - OFFIS DICOM toolkit command line utilities
libdcmtk-dev - OFFIS DICOM toolkit development libraries and headers
libdcmtk8 - OFFIS DICOM toolkit runtime libraries
libssl-ocaml - OCaml bindings for OpenSSL (runtime)
libssl-ocaml-dev - OCaml bindings for OpenSSL
libssl-dev - Secure Sockets Layer toolkit - development files
libssl-doc - Secure Sockets Layer toolkit - development documentation
libssl1.0-dev - Secure Sockets Layer toolkit - development files
perl-openssl-defaults - version compatibility baseline for Perl OpenSSL packages
r-cran-openssl - GNU R toolkit for encryption, signatures and certificates based on OpenSSL

"make clean," then "make" I did not. Because I do not know what it's need for and how to do it.

Developers, do not you want to help solve the problem? Well, I'll have to remove your stupid wallet.

default libssl location: /lib/x86_64-linux-gnu/libssl.so.1.0.0
libssl1.0.0 is available as a default package on your OS: https://packages.debian.org/search?keywords=libssl1.0.0
in case you installed libssl into custom location, please make symlink to it from default expected location.
manual install can be done by getting it from https://packages.debian.org/jessie/libssl1.0.0 and install like dpkg --install libssl1.0.0_1.0.1t-+deb8u6_amd64.deb as root

we now have some AppImages, even for rude people


not an issue

but is this the right way to congratulate you on becoming a doctor?

it was either here or parkin/CU-Bus-Guide since i cant find your email anymore

haha thanks! Closing this issue is definitely the best way to celebrate defending. I hope you are doing well @smsaladi!


bigquery: jsdoc optional paramteres for query

Not all parameters are required when doing bigquery.query(options).

Coverage decreased (-14.6%) to 85.356% when pulling bf249688700883abe1c4987bc0f9f8067b8c56a6 on mentos1386:mentos1836-fixes into<PHONE_NUMBER>2ce0f6fa5fdecc5d4bf8f181b73d3b on GoogleCloudPlatform:master.

Thank you @mentos1386. We document and list the options argument itself as optional. It is implied that everything it accepts within it is also optional. Some of our methods accept an argument named config, which is required, but not all of the properties within it are required. We make sure to mark the optional items in those instances.
The callback is indeed optional when you are using promise mode, even though it isn't marked as such. However, it will take a big sweep of our library to change the documentation to be consistent there, and I think this would be covered by #1699.


Add enter() and exit() functions

Parsers such as babel have visitors which define enter() and exit() callbacks. Adding enter()and exit() would enable plugins to register callbacks on both pre-order and post-order traversal.
{
  decl: {
     enter(node, result) {
        // do something when entering a declaration
     },
     exit(node, result) {
        // do something when exiting a declaration
     }
  }
}

Thanks for looking into this, @andyjansson.
Right now, the walker first visits the current node before it walks its children.
It looks like I just need to make that first visitation the enter() method, which allows the object-less shorthand. Then, after it walks the children, run the exit() method. I’ll add this.

Resolved via https://github.com/jonathantneal/postcss-visitor/commit/ac327ae8b7b7038d318ef97584e5e5dfbc2ddf56


[RFC] Query Planning and Rewriting

Problem statement
In traditional database systems, query planner and optimizer chooses the right indices and join order to ensure efficient execution of query. Opensearch does not have any such component causing degraded performance for queries not written well. The profiler can provide insights into the latency breakdown of each phase, but does not automatically optimize the queries for customers. The impact of manually rewriting the query has been confirmed both by github issues and during managed service customer engagements:

Github Issues:

Date histogram performance can significantly benefit by rewriting the aggregation into range query filter wherever possible (Github issue - https://github.com/opensearch-project/OpenSearch/issues/9310)
Dynamic pruning for cardinality aggregation - https://github.com/opensearch-project/OpenSearch/issues/11959

Other rewrites:

Stored fields/_source to docvalue_fields - This optimization has resulted in significant performance improvement for some of the managed service customers
Map/Global Ordinals execution hint - One of the managed service customer was able to achieve more consistent performance by providing map hint instead of default global ordinals

High Level Approach
There are primarily two approaches for solving this problem. One approach relies on cost estimation for planning the query similar to traditional database systems and other is more dynamic in nature by collecting feedback from the performance of rewritten queries and accordingly, enabling/disabling and tuning the parameters for query rewriting.
Query Cost Estimation Based
The key aspect of this approach is the query cost estimation component. Given any query, it is able to guess the query cost well allowing the query rewriter to compare multiple query plans. Accordingly, it can choose the most efficient form for query execution.

Pros:

Low cost, insignificant overhead. Need not worry about workload changes
Handles >1 rewrite for single query easily, by generating multiple plans with those rewrites

Cons:

Estimation process is complex due to sheer number of cases, might rely on clause execution time feedback
Hard to come up with the accurate estimates

Rewritten Query Execution Based

This approach starts with the query rewriting in shadow mode. For every query, it checks if the query is rewrite eligible and samples such queries, executing them asynchronously (shadow mode) and comparing the execution time for the original vs rewritten query. Along with the execution time, every rewrite logic emits the value of tunable rewrite parameters for the query rewrites. Taking date histogram and doc values rewrite as example, we can expect following data after few executions:
Date Histogram ParametersRewrite efficiency:directly proportional to document hit countinversely proportional to bucket count

Original Time
Rewrite Time
Bucket Count
Hit Count

500
50
20
1M

300
30
50
500k

40
60
500
500

20
40
1000
100

Doc Values Rewrite ParametersRewrite efficiency:directly proportional to total fields indexedinversely proportional to field requested

Original Time
Rewrite Time
Fields Requested
Total Fields Indexed

500
50
3
1000

300
30
1
500

350
500
70
100

250
400
150
200

Using the above data, query insights plugin will be able to help detect the right parameter values for each query rewrite type. And once it has sufficient confidence, it can operate in reverse shadow mode where the original query is run occasionally to detect any changes in workload for that particular type of rewrite.Pros:Makes query rewriting pluggable as and when new rewrites are identified. Each rewrite needs to plugin itself by providing the list of parameters on which it can be tunedActually running the queryCons:Cannot be done for every query to limit the overhead, relies on samplingNeed to handle caching for getting right execution timesDifficult to handle >1 rewrite for single query easily, as every rewrite has costNeed to run even original query to detect any workload changesAdditional resource consumption for async execution of rewritten / original queryMitigations:Reduce the subset by limiting to expensive queries > 100/200msTry the query rewrite on few shards for limiting performance impactThreshold the query execution time based on actual/rewritten query, maybe 2x (Query Sandboxing / Hard Cancellation)
Related component
Search:Performance
Describe alternatives you've considered
No response
Additional context
No response

review step 4 :
https://github.com/opensearch-project/sql/blob/main/docs/dev/intro-architecture.md

[Triage - attendees 1 2 3 4 5]
@jainankitk Thanks for filing this rfc

Most existing query engines don't run optimized queries in shadow mode - They optimize the plan and execute it.  I would consider requesting customers to opt-in when we launch the feature initially.  Later, once we have tuned the system, we can make query planning the default and let customers turn it off.

How Good are Query Optimizers, Really? (https://www.vldb.org/pvldb/vol9/p204-leis.pdf) might be relevant reading.

How Good are Query Optimizers, Really? (https://www.vldb.org/pvldb/vol9/p204-leis.pdf) might be relevant reading.

Thanks @kkmr for sharing this. I went through this paper and this is mostly around cardinality estimation and join order both of which are not directly relevant for Opensearch. Few observations from this publication:

Real vs benchmark data can have very different characteristics as the uniformity and correlations can be much more different. While the estimation is pretty accurate for TPC-H generated data, not the case for JOB (IMDB) data
Cost estimation is hard problem, but improvement can be significant as called out in Section 5.4 when the estimates were replaced by actual cardinalities for planning the join

Most existing query engines don't run optimized queries in shadow mode - They optimize the plan and execute it. I would consider requesting customers to opt-in when we launch the feature initially.

I am also leaning towards introducing query rewriting as feature which can be tuned or turned off using cluster setting. This will allow the front-loading of core value proposition, and the framework for tuning those settings/parameters can be worked upon in later milestones.

Later, once we have tuned the system, we can make query planning the default and let customers turn it off.

While I hope, this would be eventually possible, I feel that tuning for query planning will be continuous exercise for different types of customer workloads

Recently, Opensearch customers have shown strong interest for more performant and resource efficient Join operation. One approach for such "Join" is making it first class citizen within Opensearch and building a query planner more aware of OpenSearch shard and Lucene index architecture. Both logical and physical plan to execute a query could be optimized over existing SQL plugin architecture for “Join“, which is based on executing OpenSearch queries, gathering all data and performing Join operation over them. It doesn’t deal or make use of intricacies of OpenSearch architecture but works more like an application sitting on top of OpenSearch. Very simple join operation across 2 indices in OpenSearch could look something like below:
{
    "join_query": {
        "queries": {
            // all single index query or nested join_query
            "left": {
                "index": "index1",
                "query": {
                    // regular single query
                },
                "fields": ["f1, f2"]
            },
            "right": {
                "index": "index2",
                "query": {
                    // regular single query
                },
                "fields": ["f1, f2"]
            },
            "derived_field": [
                {
                    "name": "f3"
                    "value": "left.f1 + right.f1"
                }
            ]
        },
        "join" : {
            "type": "inner/left/right/semi",
            "query": {
                "boolean": {
                    "must": [
                        {
                            "match" : {
                                "left.f1": "emit(right.f1)"
                            }
                        },
                        {
                            "range" : {
                                "left.f2" : {
                                    "gte" : "emit(right.f2)"
                                }
                            }
                        }
                    ]
                }
            }
        },
        "agg": { // aggregation on joined index
            // regular agg on single index
        },
        "fields": []
    }
}

Note: Extensive Query DSL grammar and query engine related changes will be covered in much more depth as part of separate RFC by @rishabhmaurya. This RFC is specifically focused on the Query Planning aspects around this Join operation.

High level design for Query Planner within OpenSearch could look something like below:

Note: More details regarding each component to follow soon

At a high level this looks good. I would like to battle-test this proposal against vector search because it has some native parts, maybe @navneet1v could take a look?

love this idea. some thoughts

Execution Perspective: Should we consider developing the New-QueryEngine to primarily serve SQL first, and then gradually migrate DSL? This approach offers three key benefits:
1.1. It aligns with our goal of moving SQL to the core.
1.2. It has less impact on existing DSL users, while encouraging them to migrate to SQL.
1.3. It avoids the need to introduce new JOIN grammar in DSL.
Building New-QueryEngine: I suggest we construct the New-QueryEngine using battle-tested components. There's a relevant paper titled Assembling a Query Engine from Spare Parts that could provide valuable insights.

@penghuo thanks for providing feedback. I'm little hesitant in moving SQL support to core as it could be a significant overhead in maintenance and I like the plugin model better for supporting a new query language frontend.
However, I'm totally aligned on implementing functionality which SQL supports and OpenSearch query DSL doesn't, like JOIN, in the core if its in demand among our users.
I would like to understand more on why do we have "It aligns with our goal of moving SQL to the core."  as a goal?
Tagging other members and maintainers to take their input as well @andrross @msfroh @reta @dblock @smacrakis

Do look the cratedb implementation of joins (https://cratedb.com/docs/crate/reference/en/latest/concepts/joins.html).  CrateDB is derived from elastic- so it uses Lucene as the storage.  It also ports over postgres query engine code to Java (Postgres QP is reputed to be pretty good).  So some of the ideas in the cratedb code might translate well to this effort.

At a high level this looks good. I would like to battle-test this proposal against vector search because it has some native parts, maybe @navneet1v could take a look?

I took a high level look. In terms of vector search, as long as query is coming via Lucene Searcher interface vector search won't have any issue. @dblock is there any specific thing you want me to look at.
Because after reading the doc, what I can see on high level is we are trying to rewrite the queries to optimized queries and not sure how vector queries can make use of it because vector search queries doesn't have complexities like Histogram aggs etc.
Please correct me if my understanding is wrong.

Because after reading the doc, what I can see on high level is we are trying to rewrite the queries to optimized queries and not sure how vector queries can make use of it because vector search queries doesn't have complexities like Histogram aggs etc.

Thanks for chiming in! I wanted to make sure that the cost estimation will be able to (eventually) account for native parts of search (faiss, etc.).

Thanks for putting it together, @jainankitk ! I would really like to see Query Cost Estimation Based implementation (although I :100: agree this is hard problem). On the bright side, we could engage with Apache Lucene folks to sketch out a "fair" query cost estimation model per index (== OS shard).

Because after reading the doc, what I can see on high level is we are trying to rewrite the queries to optimized queries and not sure how vector queries can make use of it because vector search queries doesn't have complexities like Histogram aggs etc.

Thanks for chiming in! I wanted to make sure that the cost estimation will be able to (eventually) account for native parts of search (faiss, etc.).

@dblock
Now I understand your ask. Thanks for clarification.
To know what will be the cost, I would like to know more details on how cost estimation is working. I see we have sprinkled this information little bit in the GH issue, but those might not work for every type of query. For vector search we cannot estimate just like this. There are other(algorithm related) parameters that defines the scope of search. I think it will require more discussion.

To know what will be the cost, I would like to know more details on how cost estimation is working. I see we have sprinkled this information little bit in the GH issue, but those might not work for every type of query. For vector search we cannot estimate just like this. There are other(algorithm related and query related) parameters that defines the scope of search. I think it will require more discussion.

@navneet1v - Thank you for providing your feedback from the knn perspective. While we are working on ironing out more details around cardinality/cost estimation and overall long term approach, I am wondering if there are specific knn queries that can benefit from Query Planning. For example - Join operation can significantly benefit from the order in which it is executed, are there similar rewriting optimizations that can be done for knn queries resulting in noticeable performance improvement?


Improv

Checklist

[ ] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.
Here's what I've got...
For recipe:

requirements: build: go >= 1.13 should not contain a space between relational operator and the version, i.e. go >=1.13

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


CouchDB 2.3.1: Reduce function raised exception (new TypeError("template.metaData is undefined", "undefined", 10))

NOTE: # Using CouchDb 2.3.1 - Getting TypeError in "Reduce function (custom)" while fetching child fields e.g. _doc.metaData.SomeDateField, however works fine in "Map" Function. Same reduce function works fine in CouchDB 1.6
Description
We have recently upgraded to CouchDb 2.3.1 from CouchDb 1.6 and facing issue with one of the view which uses reduce function (custom), however same view works as expected in CouchDb 1.6.
In the view's reduce function, when we are fetching second level child fields, we get the following error in the couch log:
[info] 2019-05-03T06:01:46.097000Z couchdb@localhost <0.2402.0> -------- OS Process #Port<0.8540> Log :: function raised exception (new TypeError("template.metaData is undefined", "undefined", 10))
Steps to Reproduce

Create a view

Define map function as below:
function(doc) {  if (doc.$doctype !== 'someTemplate')      return;  emit([doc.metaData.fieldId1,//this line works i.e. getting fieldId1 from metaData field          doc.metaData.fieldId2,          doc.field3,          doc.templateTypeX,          doc.templateName.toLowerCase()  ], doc); }

Define reduce function as below:
`function(keys, values, rereduce) {
var mostRecentTemplate = null;
values.forEach(function (template) {
if (mostRecentTemplate == null
|| Date.parse(mostRecentTemplate.templateDate) < Date.parse(template.metaData.fieldId2)) {
mostRecentTemplate = {
'templateType': template.templateTypeX,
'templateName': template.templateName,
'templateDate': template.metaData.fieldId2  //this line causing issue
};
}
});
if (mostRecentTemplate && mostRecentTemplate.someAppState === -1) {
mostRecentTemplate = null;
}
return mostRecentTemplate;
}`

Run the view query

No document returned received null for each key.

Expected Behaviour
Return the document template created in custom "reduce" function for the view.
LOCAL Dev Machine

CouchDB Version used: 2.3.1
Browser name and version: Chrome
Operating System and version: Windows 10

Additional context
Here is how couch document looks like:
{   "_id": "sdfsfsfsfsfsfs",     "$doctype": "someDocumentTemplate",   "templateName": "AK",   "templateTypeX": "AllCategoriesTemplate",     "someKey": "989878799",   "metaData": { 	     "fieldId2": "2018-06-14T12:27:03.2204746Z" //Error fetching this field in the custom reduce function   },     "someAppState": 0   }

This appears to be because you don't handle the case where rereduce=true. The result from your reduce function is return in the 'values' parameter in that case, and, in the rereduce=false case, your 'mostRecentTemplate' object does not contain the metaData attribute.

This custom reduce function doesn't work in 1.6.1 either, you've just had too few documents in your database for couchdb to call you with rereduce=true.


Work around multiprocessing bug with ProcessError exceptions

Multiprocessing on Python 2.7.8 fails to deal properly with some
exception types, leading to a hang if they are raised.  See
https://bugs.python.org/issue9400
Work around this by explicitly wrapping all exceptions raised in
multiprocessed functions in a custom ParallelFailure exception type.
Add some extra code so that util.UserErrors are reraised properly.
Fixes gh-261

Build failure unrelated (apparently conda network failure when trying to do Conda.match)

Planning to merge soon, as the issue is triaged.


Consider memoizing serialization of fields

I had this in mind for a while and today I realized that Java implementation actually does this: https://github.com/protocolbuffers/protobuf/blob/8d3a7327606b115dda871be54c8b8cb66d41ff90/java/core/src/main/java/com/google/protobuf/LazyFieldLite.java#L110-L115
We could generalize the idea in the implementation above for JSON serialization as well.
This is somewhat related to #713. If I read a message but don't deserialize a field of it yet, I should be able to directly copy the bytes for the field to serialize, without having to deserialize it only to serialize it back.
One use case is when my app reads a large message, uses a few fields (maybe updates them, or just reads), and relays the message to some other app(s). For the message-fields that I don't modify I can directly copy the encoding to the relayed messages.
The main cost is memory to store the encodings, and an extra field to every field that can have memoized encoding.

The main cost is memory to store the encodings, and an extra field to every field that can have memoized encoding.

Perhaps that could be encoded in a bitfield...


Revert "Testing tryjob triage"

Reverts flutter/flutter#47893
I have confirmed tryjob ingestion is preforming as intended. This can be reverted now.

Cirrus isn't reporting correctly here. Confirmed this is good to go. :)


Add default text to untitled blocks

JIRA: https://openedx.atlassian.net/browse/MA-938

@ehmadzubair please review

:+1:


Deep layer size

According to the Librispeech AST config, a deep layer is used prior to output projection (https://github.com/eske/seq2seq/blob/master/config/LibriSpeech/AST.yaml#L42) and that layer size is 512 (https://github.com/eske/seq2seq/blob/master/config/LibriSpeech/AST.yaml#L45). However, running ./seq2seq.sh config/LibriSpeech/AST.yaml --train -v --purge shows that the actual deep layer size is 128 (the target embedding size). In https://github.com/eske/seq2seq/blob/master/translate/models.py#L776, decoder.pred_deep_layer_size is None so decoder.embedding_size is used. Would you be able to confirm which size is used for https://github.com/eske/seq2seq/blob/master/config/LibriSpeech/model-outputs.tar.xz ? Thanks in advance.

You're absolutely right. Thanks for bringing this to my attention.
It seems like the attribute name is pred_deep_layer_size, and I have been using deep_layer_size in the configuration files.
In the models provided on the README page (which were used to generate the model outputs), the actual deep layer size was indeed 128 (I checked in the log files).


Android: Unread counts aren't being updated on some installs

See https://getsatisfaction.com/newsblur/topics/in-main-list-unread-not-updating?topic-reply-list[settings][filter_by]=all&topic-reply-list[settings][page]=1#reply_15138655. Been getting quite a few reports about unread counts being stuck. Know why this would happen? Anyway, this should be the top priority right now.

Take a look at : https://getsatisfaction.com/newsblur/topics/every-thing-shows-no-worries-but-individual-feeds-do-have-unread-stories-inside-them?topic-reply-list[settings][filter_by]=all&topic-reply-list[settings][page]=1#reply_15234652
User says it was his SD card.

Ah, yeah, it would make sense that if storage were flaky, things might go awry.  Android has a bad (good?) habit of obscuring filesystem failures from apps.  This is why the vast majority of new devices don't have an SD slot anymore.

Looks like it's not the SD card: https://getsatisfaction.com/newsblur/topics/android-not-syncing-feeds-incorrect-unread-count

Another: https://getsatisfaction.com/newsblur/topics/android-unread-counts-stuck

Need a username to investigate.  Feel free to email if they don't want it on the issue log.

Another username
https://getsatisfaction.com/newsblur/topics/android-2vg87b6ayhm27?utm_source=notification&utm_medium=email&utm_campaign=new_reply&utm_content=reply_button&reply[id]=15266161#reply_15266161

Logged in as user po******ck.
/reader/feeds/?update_counts=true' included feed id 453413with"nt": 1`. Main activity UI showed 1 unread for that feed.
Tapped into said feed and scrolled through results provided by /reader/feed/453413?order=newest&page=1&f=453413&read_filter=all and confirmed seeing no unreads in UI list.
Switched unread filter to Unread Only and confirmed /reader/feed/453413?order=newest&page=1&f=453413&read_filter=unread returned zero results and UI showed no stories.
Hit mark-all-read button and invoked /reader/mark_feed_as_read/ with args feed_id=453413. Received "code": 1, "result": "ok" response.
Back in Main activity, refreshed feed/folder list.  /reader/feeds/?update_counts=true again returned data block for feed 453413 with "nt": 1, causing UI to show 1 unread.
As far as I can tell, the UI is reflecting the data provided by the API.  Am I missing something subtle?

This was the same situation observed when logged in as the Ca*******ds user, as reported via email where feed 181186 had 3 of these ghost unreads that won't go away.
One commonality I noticed is that all reports of this issue so far have come from non-premium users.

There's just more non-premiums than premiums. There is next to no difference otherwise. The river works differently, but when caching stories the premium/non-premium condition doesnt have an effect.
I would expect the API to be correct. It's probably their local devices not updating unread counts.

I would expect the API to be correct. It's probably their local devices not updating unread counts.

See #issuecomment-71089710 above; the app is, to the best of my understanding, exactly displaying what the API returns in both of the repro cases we have seen so far.

Ditto. I haven't been able to meaningfully use the app in weeks. Android, with SD card.

Closing this in lieu of #672, since a few minor versions have been launched since.


Auto updater

After reading https://medium.com/@robbyrussell/d-oh-my-zsh-af99ca54212c I think two simple additions could increase engagement. One is that when you update fisherman or a plugin, a short description of the changes  (commit messages/ what files have changed) could be shown (#168). Second, fisherman could automatically update every other week to make sure that people stay up to date with the latest and greatest (this issue).

@domoritz Rather than adding this functionality right into Fisherman, I'd rather create a plugin that only users willing to let Fisherman update itself can run. What do you think?

I think a lot of people will not update then. I think it would help to have an auto updater that updates even very infrequently.

What do @jethrokuan @orangecms @pickfire @herrbischoff @rolyreyes think?
I don't like the idea to update automatically, maybe just show a notification "there is an update available"?
What do other plugin/pkg managers do?

I don't like the idea to update automatically, maybe just show a notification "there is an update available"?

Definitely. Never update without the user's consent. I was thinking about "There is an update available. Do you want to install it? [Y/n]"

Updating Fisherman should only happen on the user's demand, preferably through their system's package manager (apt, pkg, zypper, portage, pacman, you name it) as they are used to that.

Updating Fisherman should only happen on the user's demand, preferably through their system's package manager (apt, pkg, zypper, portage, pacman, you name it)

Then the preferred way should be to install fisherman through a package manager. Right now the recommended way to to run curl -sL get.fisherman.sh | fish. Also, how would that work on OSX and Windows?

OSX has Homebrew, and on Windows I think there is cyg-apt or something like that.

I think a lot of people will not update then.

What's the issue with that? Just because you (or anyone) feels that everyone should use the latest version, does not automatically make it the right approach for everybody. Certainly this is no reason to annoy people into updating. Besides, we're talking about shell-savvy people here, not mainstream customers who are getting notified to update to the latest iOS — and even that is a one-time affair, despite the looming badge many still don't update right away, although eventually.
I agree that an official Homebrew package may be beneficial although that needs to be updated for every release and will do very little besides running the install script. Another point of irritation (for me at least) would be installing a framework via a package manager.

Yes, the system's package manager should always be preferred. OSX has Homebrew, and for Windows there is apt-cyg. The only thing is that people need to write the packages.

Should this point to using the Homebrew infrastructure to install not only Fisherman but also its plugins, I'm strongly against it. This would be overkill, besides that fact that you would need to have a separate tap that in turn would be under the control of the Homebrew team. This effectively means giving away the plugin management and therefore all decisions about what gets included. Not to speak of the added burden of creating a Ruby-based formula to install a shell extension. I don't think this is reasonable.

@bucaran, I don't support auto update. The update process is fast currently and can run in parallel. The user can just use a timer to set auto update.

@herrbischoff If Homebrew is that weird and complicated, that's a fair point. I thought it was more evolved. The benefit would be a clean home dir and install/uninstall procedure without (what package managers have been invented for anyway), which is why I created the AUR package, done easily. I also write packages for portage (Gentoo's port/package manager), which is also straightforward. Is it really that hard with Homebrew?

I think auto updating can come as a plug in,  so it's something users can
opt in to have. It's not something I'd use personally either.
On 30 Mar 2016 3:23 pm, "Daniel Maslowski"<EMAIL_ADDRESS>wrote:

@herrbischoff https://github.com/herrbischoff If Homebrew is that weird
and complicated, that's a fair point. I thought it was more evolved. The
benefit would be a clean home dir and install/uninstall procedure without
(what package managers have been invented for anyway), which is why I
created the AUR package, done easily. I also write packages for portage
(Gentoo's port/package manager), which is also straightforward. Is it
really that hard with Homebrew?
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/fisherman/fisherman/issues/169#issuecomment-203286509

@orangecms: Here's the installation formula for fish for comparison. It's not exactly hard since you run brew create <URL> and get a template. It's just that updating this every time you make a small edit and would need to release a new minor version adds significant overhead. Plus, the pull request has to be made to the Homebrew repository for each change. While it is possible to have HEAD-only formula, those are not updated automatically when running brew update --all and need to be updated manually through brew reinstall --HEAD <formula>.
I support auto updating as a plugin, should anyone feel the need to actually create it. I wouldn't use it though.

I agree with @herrbischoff on this one, as a plugin, that is, an official plugin that would be accepted into the index, it would not bother me at all, so if anyone wants to take a shot at it, ping me directly on the Slack channel and I'll gladly help out the best I can.
Now, as a core feature, I vote :-1:

@herrbischoff That looks just as easy as I expected, so with the proper Makefile that would be possible with fisherman as well. It's by design that software hits repos after the package maintainers' review, so if users really want to update from HEAD (where a package maintainer cannot be blamed) it becomes the user's own responsibility when something breaks, which is exactly the same as when they use the shell script. So if we just provide a proper Makefile, the Homebrew formula wouldn't need any maintenance at all (that'd be the same for any package manager actually). Users always benefit from a clean installation state.
So writing an updater would just mean extra work (though basically git fetch/merge etc., which again might fail if users do stuff with the cloned repo, move it, or whatever). As a plus, dependencies can be dealt with properly, like git currently (where the user would in case of an outdated version just see a hint and maybe be pissed), or fish later when the conf.d feature hits the next release and we want to support it. They'd want/need an older/fixed version of Fisherman then in case the don't want to/cannot upgrade fish for whatever reason (it might cause them other stuff to break, or depend on the sysadmin's/organization's policies etc.).

I think that just showing a notification would be the best approach, since it doesn't disturb user and they can update when ever they feel like.

Even then, I feel the notification should not be run automatically. To make that configurable and stay within the fish philosophy, it should be an optional plugin, as we already seem to have agreed on.

Alright, thanks @domoritz for bringing this up and thanks everyone for their comments.


Action requires node12, but it is deprecated

When using action I receive a warning:

The following actions uses node12 which is deprecated and will be forced to run on node16: SierraSoftworks/setup-grcov@v1. For more info: https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/

So node should be updated to node16

Now it says to update to node20:

Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: SierraSoftworks/setup-grcov@v1. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.


Windows are considered doors

A window entity with device_class: window is being included in the *_doors groups.  No *_windows groups are being created.
HASS Version | 0.118.3
HASS Type | Home Assistant Core
Magic_Areas version | 1.1.0

That shouldn't be happening. Can you post the attributes (from developer tools) of both the area_door_* and the device with device_class: window that is going into that aggregation?

I've run into the same issue. I turned on debugging but no Window aggregates are created.
Attributes of my binary_sensor.openclose_77 (a Sonoff DS01 contact switch over Deconz):
on: true
friendly_name: Bedroom Window
device_class: window

binary_sensor.area_bedroom:
presence_sensors: binary_sensor.bedroom_occupancy, binary_sensor.presence_25, binary_sensor.openclose_32, binary_sensor.openclose_34, binary_sensor.openclose_77, binary_sensor.withings_in_bed_jake, switch.area_presence_hold_bedroom
active_sensors: binary_sensor.openclose_32, binary_sensor.withings_in_bed_jake
lights: light.nightstands, light.dresser
climate:
clear_timeout: 600
update_interval: 15
on_states: on, home, playing, open
exterior: false
automatic_lights: disabled
friendly_name: Area (Bedroom)
icon: mdi:texture-box
device_class: occupancy

Settings:
    bedroom:
      clear_timeout: 600
      control_lights: false
      control_climate: false
      include_entities:
        - binary_sensor.withings_in_bed_jake
      exclude_entities:
        - binary_sensor.master_bedroom_fan_occupancy
        - light.extended_color_light_41
        - light.extended_color_light_44
        - light.bedroom

Versions:
Home Assistant: 2021.1.1
Type: Supervised
magic_areas: v.1.1.2

I've run into the same issue. I turned on debugging but no Window aggregates are created.
Attributes of my binary_sensor.openclose_77 (a Sonoff DS01 contact switch over Deconz):
on: true
friendly_name: Bedroom Window
device_class: window

binary_sensor.area_bedroom:
presence_sensors: binary_sensor.bedroom_occupancy, binary_sensor.presence_25, binary_sensor.openclose_32, binary_sensor.openclose_34, binary_sensor.openclose_77, binary_sensor.withings_in_bed_jake, switch.area_presence_hold_bedroom
active_sensors: binary_sensor.openclose_32, binary_sensor.withings_in_bed_jake
lights: light.nightstands, light.dresser
climate:
clear_timeout: 600
update_interval: 15
on_states: on, home, playing, open
exterior: false
automatic_lights: disabled
friendly_name: Area (Bedroom)
icon: mdi:texture-box
device_class: occupancy

Settings:
    bedroom:
      clear_timeout: 600
      control_lights: false
      control_climate: false
      include_entities:
        - binary_sensor.withings_in_bed_jake
      exclude_entities:
        - binary_sensor.master_bedroom_fan_occupancy
        - light.extended_color_light_41
        - light.extended_color_light_44
        - light.bedroom

Versions:
Home Assistant: 2021.1.1
Type: Supervised
magic_areas: v.1.1.2

Customizing is not respected since it looks like it gets applied after MA is loaded. What device class does this sensor have if you don't customize it?

Customizing is not respected since it looks like it gets applied after MA is loaded. What device class does this sensor have if you don't customize it?

I guess that makes sense. I think I ran into the same problem with a "switch" as I had to create a dummy "light" template to make that work... seems there's an opportunity for improvement here?

I guess that makes sense. I think I ran into the same problem with a "switch" as I had to create a dummy "light" template to make that work... seems there's an opportunity for improvement here?

Yes this is a known issue, currently investigating how to get around it.

Yes this is a known issue, currently investigating how to get around it.

All customize related issues are being fixed with rework #74, coming up on release 2.0.0.

I think I ran into the same problem with a "switch" as I had to create a dummy "light" template to make that work... seems there's an opportunity for improvement here?

Switches will not be automated by Magic Areas since they can be anything. The recommended approach for switch entities that control lights is to use https://www.home-assistant.io/integrations/light.switch/

All customize related issues are being fixed with rework #74, coming up on release 2.0.0.

I think I ran into the same problem with a "switch" as I had to create a dummy "light" template to make that work... seems there's an opportunity for improvement here?

Switches will not be automated by Magic Areas since they can be anything. The recommended approach for switch entities that control lights is to use https://www.home-assistant.io/integrations/light.switch/

Ohh, that's a new one for me. Thank you!!

Ohh, that's a new one for me. Thank you!!

Possibly fixed by #74, released on 2.0.0.
Can you folks test it out and let me know if the issue is gone?

Possibly fixed by #74, released on 2.0.0.
Can you folks test it out and let me know if the issue is gone?


Terminal Editors: Some important commands work don't work in terminal editors

Copy and paste for example apply to the active terminal in the terminal view
Repro:

Create a terminal
Create a terminal editor
Try copy and paste in the terminal editor

Let's cover split routing to the editor in this


High Memory Consumption on Barebone WAMP Component

Using memory_profile  I verified that a empty barebone WAMP component has a high memory consumption. Bellow the examples for Twisted and Asyncio with Cpython3, and Pypy3
Twisted code:
from os import environ
from autobahn.twisted.wamp import ApplicationSession, ApplicationRunner

class MyComponent(ApplicationSession):

    def onJoin(self, details):
        print("joined")
        pass

if __name__ == '__main__':
    runner = ApplicationRunner(url=u"ws://localhost:8888/ws",
                               realm=u"RE_acct")

    runner.run(MyComponent, auto_reconnect=False)

Python3

~35Mb of RAM
PyPy3

~85Mb of RAM
Asyncio:
from os import environ
import asyncio
from autobahn.asyncio.wamp import ApplicationSession, ApplicationRunner

class MyComponent(ApplicationSession):
    def onJoin(self, details):
        print("joined")
        pass

if __name__ == '__main__':
    runner = ApplicationRunner(url=u"ws://localhost:8888/ws",
                               realm=u"RE_acct")

    runner.run(MyComponent)

Python3

~18Mb of RAM
PyPy3

~75Mb of RAM

We made this comparison above to see who is responsible for the rather large memory footprint of a WAMP component. From image 3 ( Cpython + asyncio) and the fact, that an empty python program takes about 12Mb we conclude autobahn takes only 5Mb for itself. So the major memory consumers seem to be pypy3 runtime and twisted.
Do you have any further suggestions on how to reduce the memory consumption of WAMP components. Our use case is to use WAMP components as dockerized microservices in a scale-out architecture, so we want to have them as small as possible.

Our own conlusion from the above is to use Cpython + asyncio for microservices that are mostly idle and use pypy3 + asyncio for services that are heavily used.

Our own conclusion from the above is to use Cpython + asyncio for microservices that are mostly idle and use pypy3 + asyncio for services that are heavily used.

This makes sense and is what I would have expected.
PyPy is JITting code (which is why the memory footprint is increasing for a longer time than CPy).
asyncio is a completely new codebase, and has less footprint than Twisted (but also less features, knobs, etc - but those are usually not needed nowerdays with AB Py clients).
The 18MB of a CPy/asyncio base AB Py component: I would say this is quite good. You simply cannot expect to get this down to say the size possible with AB C++.
A AB C++ client is <3MB in footprint (I measured that). Since this was with a completely statically linked executable, this also runs in a Docker Null container (FROM scratch) of the same size (3mb).

So we should document above insights and considerations ..