[HUDI-911] Add Blog about Hudi-Spark decoupling and Flink integration…

…design
Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contributing.html before opening a pull request.

What is the purpose of the pull request
Add Blog about Hudi-Spark decoupling and Flink integration desigin
Brief change log
(for example:)

Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@vinothchandar @yanghua @leesf please take a look when free


[MINOR] Adding swagger rest Diagram #267

closes #267

When you have time, don't forget to add it the documentation.

When you have time, don't forget to add it the documentation.

I did it


Dl/theme support for view render

This allows the views to be overridden by the theme.
The order of searching is now going to search the theme for each defined plugin:

get_stylesheet_directory()/$plugin/$filepath.php
views/$filepath.php

Thanks, this seems like a useful feature! Also good to see child theme support (get_stylesheet_uri).
Why is the subdir of the theme named "Plugin" in the config? Could it be something more intuitive like 'mvc_theme_view_subdir'? As I understand it, this is a path underneath a theme, and has nothing to do with plugins...

Based on how I interpreted wp-mvc loading plugins that extend it, I wanted to ensures plugins didn't clobber other plugins if they had the same views. The Plugin configuration option is generated by wp-mvc on load, and so it is only the plugins that are "registered" with wp-mvc.
To my knowledge, this is how other plugins (that don't use wp-mvc) allow views to be overridden as well. I'm happy to change it, but I just wanted to make sure we could allow multiple plugins to co-exist without having any potential issues.

Thanks, this makes sense! So the plugin 'foo' that implements wp-mvc as its framework could have a theme override in template 'bar' at bar/foo/model_name/method_name.php. Is that the idea?

Yes, that's how it works in my installation.

Many thanks! This has been merged into GitHub and will go out with the next release to the WP Repository.


Indent xml

We outdent snippets as much as we can - but in this case we couldn't outdent anything, because the source code was already outdented.
This was we end up with an odd XML file, but it's still valid.

Self-merging, assuming fine.

LGTM


Points tracker

Initially it was working fine but I just had a deathless 400 invo raid which usually tells me around 10-13% (if using the wiki calc it shows similar) but this raid was showing 2.2% at the end? 23,000 points no deaths when on screenshots i have from earlier raids when the feature was released it was showing me as 4.5% at only 11k points so I'm not sure what's happened there.
provided a screenshot, not sure how/if it'll help.
https://gyazo.com/88565022799ca972baa88e21d48b7f0e

This looks like it calculated your drop chance on a 0 raid level. I released a new version of plug in a few hours ago that addresses this issue. Can you check whether you are on the most up to date version?
If your client is still open, you can check by opening the plugin hub panel and seeing if there is an update button for Tombs of Amascut.

Yeah my client was open for a while, I restarted it and saw the plugin address the updates and it seems to be working again now, thanks for the amazing work.


Catch 2-4-stable up with Master

This merges in the changes that have occurred on Master over the past few months.
Also, I do believe that line_items.digital? should report as digital if the master variant is digital. That way, a user can be issued download links from the master variant, regardless of if their purchased variant is digital or not.
This would allow you to send out any common downloads via the master variant, and then send out variant-specific downloads with each variant.

@halo or @JDutil, mind having a look?

The changes look okay to me, but I haven't manually tested things out.

@JDutil pushing to staging app now, will report back

@JDutil works like a charm on my end


GM: Add support for vehicles with manual parking brakes

Description
The message EPBStatus is only present in vehicles with an electronic parking brake - many truck still use manual parking brakes.
The newly discovered VehicleIgnitionAlt.ParkBrake signal appears to be present in all vehicles regardless of Electronic vs Manual parking brake
Verification
Confirmed that the message is present in all fingerprints; tested on Suburban (Manual) and Bolt (EPB).
Actuating the parking brake doesn't appear to be part of the typical test route. I've done as much validation as I can reasonably accomplish.
@sshane has offered to help with signal regression testing

Ran some data the new signal, looks like a good change!
The new signal does look like it rises slightly later, but if this unifies the signal for all current supported vehicles and upcoming ports, this should be good.
All local GM data last 180 days
Platform: CHEVROLET VOLT PREMIER 2017
Total segments checked: 25644
Segments with  parking brake != 0: 524
Segments with any mismatches: 261
Segments with mismatches > 10 (at 100Hz): 7

Platform: CADILLAC ESCALADE ESV 2016
Total segments checked: 1942
Segments with  parking brake != 0: 26
Segments with any mismatches: 7
Segments with mismatches > 10 (at 100Hz): 0

Platform: GMC ACADIA DENALI 2018
Total segments checked: 2171
Segments with  parking brake != 0: 0
Segments with any mismatches: 0
Segments with mismatches > 10 (at 100Hz): 0

The 7 Volt segments with mismatches > 10: https://imgur.com/a/yIn1M5q

Thanks for the PR!


Explain what the "enter a name for the environment" means

Note: If your feature-request is regarding the AWS Amplify Console service, please log it in the
official AWS Amplify Console forum

Is your feature request related to a problem? Please describe.
When I run amplify init for the first time, I am asked "Enter a name for the environment" as the second question. This doesn't make a whole lot of sense to me.
Should I write production, because my app is going to end up in prod at some point? Should I write test or dev, because that's the first place I'm going to deploy? Or should I write "earth" because I feel clever?
Describe the solution you'd like
I'd love a better description for this option, or perhaps it could just be set by default. This would help me not feel confused as a beginner.

Hi @selbekk, you can find more information for the environment here

Hi @selbekk, you can find more information for the environment here

Thanks! This makes sense - but I would love to automatically understand this without reading through all the docs 😙

Hi @selbekk - this is good feedback! What do you think of these options:
Option 1 - Provide examples in the prompt
Enter a name for the environment (ex: dev, prod):

Option 2 - Provide a default value "dev"
Enter a name for the environment: (dev)

Thanks! How about giving some context about what the environment is?
Examples are good, but they don’t explain what the context of the question is.
Could we add a paragraph explaining the context of the question?

We try to minimize the content in the init flow as much as possible. So I'd rather adjust the sentence to make it clearer than adding too much description.
@selbekk - what if we just remove environment all together from the init flow and just initialize it to dev in the background?
When you actually need to add additional environments through amplify env add is probably where you actually care about it. There, we can add additional description. Wdyt?

Sounds great!
That being said - why do you think it’s important to keep explanations through the init flow to an absolute minimum?

To be clear: it should be an absolute minimum in which the customer still understands what's happening. I'll take this back to the team and how to progress from here. Thanks for the fast feedback loop!

Yeah I understand - but perhaps the premise that "it should be an absolute minimum" isn't correct? I love info while going through a process like this. If possible, a link to more details would also be a great addition.

@selbekk after some more discussion with other customers, I've submitted a PR to make "dev" the default environment. #4201
Thanks for the feedback. We'll keep iterating on this.

I've got a similar experience. I'm setting up an environment for a real project for the first time, and I have no idea what this question is asking. Maybe if you don't want to have more explanation, you could include a link to the place in the docs that has enough information for me to answer the question? I don't even know if this environment is for my front end or back end or both. And what best practices are, etc. Thanks for what you do. I love Amplify.

On reading more, "dev" seems like a very suitable default. And I understand you not wanting an overly chatty interface.


[Feature Request] Improve the help command

Please tick all applicable boxes.

[x] I am using Python 3.5.3 or higher (run python --version on the command line)
[x] I have followed the official guides to install the bot for my system
[x] I have updated my dependencies to the latest version using the appropriate update script

Which version are you using?

[ ] The latest master version (release-260419)
[ ] The latest review version

What type of issue are you creating?

[ ] Bug
[x] Feature request
[ ] Question

It would be much nicer if the help list was cleaner. Since sometimes it's not easy to spot some commands, people get confused of why there is not a command, but really they don't see the command do to the ugliness of the help list.
It would be much easier to use the bot if the help list was clean, it would also be more enjoyable.

Idk if it'll help but try #1856 out

Those PR that are based on the cog PR should have help list that commands are divided into categories.

Duplicate of #1613


Visualization ambiguity

How did you map the sentiment neuron activation to colors?
Were the max and min values generated globally? If so what are they?
Do you just scale the min and max activation of a single sample?
Do you assume the min is -1 and max is -1?
I assumed the min and max and generated the following:

vs
from blog:

https://codepen.io/lapolonio/pen/wmNgKK?editors=1111

Hey, Can you forward the code that you used to generate the heatmap.

I forked the repo https://github.com/lapolonio/generating-reviews-discovering-sentiment
I used html for the heatmap.

Look at generate_html.py

Ok thanks


Add variables

Instead of having duplicates per node, add a variable list that can be re-used.

Added in de34e3c3af25746618feca03b0e670d833fb6d9e


Read keys from config file instead of environment variables

Read keys from config file instead of environment variables

just use: https://pypi.python.org/pypi/python-dotenv

so you can create .env file with all configs


[DoNotMerge] [Microsoft.Maintenance] Generated code from 2023-04-01 API version

Contributing to the Azure SDK
Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

Please review PR, do not merge PR.

@KalpeshChavan12 Track 1 SDK is deprecated, please update Azure.ResourceManager.Maintenance instead.

@live1206 we need track1 sdk for powershell module, we cant use track2 sdk


1015 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1016 http fetch GET 304 https://registry.npmjs.org/amplify-go-function-template-provider 960ms (from cache)
1017 http fetch GET 304 https://registry.npmjs.org/amplify-console-hosting 1832ms (from cache)
1018 http fetch GET 304 https://registry.npmjs.org/amplify-dotnet-function-template-provider 1768ms (from cache)
1019 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 1024ms
1020 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1021 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 1896ms
1022 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1023 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 1816ms
1024 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1025 http fetch GET 304 https://registry.npmjs.org/amplify-codegen 1976ms (from cache)
1026 http fetch GET 304 https://registry.npmjs.org/amplify-java-function-template-provider 656ms (from cache)
1027 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 2024ms
1028 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1029 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 688ms
1030 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1031 http fetch GET 304 https://registry.npmjs.org/amplify-go-function-runtime-provider 1328ms (from cache)
1032 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 1344ms
1033 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1034 http fetch GET 304 https://registry.npmjs.org/amplify-python-function-template-provider 232ms (from cache)
1035 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 232ms
1036 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1037 http fetch GET 304 https://registry.npmjs.org/aws-sdk 192ms (from cache)
1038 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 248ms
1039 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1040 http fetch GET 304 https://registry.npmjs.org/chalk 208ms (from cache)
1041 http fetch GET 304 https://registry.npmjs.org/ci-info 176ms (from cache)
1042 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 216ms
1043 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1044 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 184ms
1045 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1046 http fetch GET 304 https://registry.npmjs.org/amplify-java-function-runtime-provider 1472ms (from cache)
1047 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 1480ms
1048 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1049 http fetch GET 304 https://registry.npmjs.org/cli-table3 168ms (from cache)
1050 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 176ms
1051 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1052 http fetch GET 304 https://registry.npmjs.org/ejs 160ms (from cache)
1053 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 160ms
1054 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1055 http fetch GET 304 https://registry.npmjs.org/colors 176ms (from cache)
1056 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 176ms
1057 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1058 http fetch GET 304 https://registry.npmjs.org/enquirer 144ms (from cache)
1059 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 160ms
1060 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1061 http fetch GET 304 https://registry.npmjs.org/env-editor 136ms (from cache)
1062 http fetch GET 304 https://registry.npmjs.org/esm 112ms (from cache)
1063 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 152ms
1064 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1065 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 120ms
1066 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1067 http fetch GET 304 https://registry.npmjs.org/execa 144ms (from cache)
1068 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 144ms
1069 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1070 http fetch GET 304 https://registry.npmjs.org/figlet 136ms (from cache)
1071 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 136ms
1072 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1073 http fetch GET 304 https://registry.npmjs.org/fs-extra 208ms (from cache)
1074 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 216ms
1075 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1076 http fetch GET 304 https://registry.npmjs.org/global-prefix 192ms (from cache)
1077 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 208ms
1078 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1079 http fetch GET 304 https://registry.npmjs.org/graphql-transformer-core 264ms (from cache)
1080 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 320ms
1081 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1082 http fetch GET 304 https://registry.npmjs.org/amplify-nodejs-function-runtime-provider 1464ms (from cache)
1083 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 1496ms
1084 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1085 http fetch GET 304 https://registry.npmjs.org/amplify-nodejs-function-template-provider 1408ms (from cache)
1086 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 1456ms
1087 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1088 http fetch GET 304 https://registry.npmjs.org/gunzip-maybe 328ms (from cache)
1089 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 352ms
1090 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1091 http fetch GET 304 https://registry.npmjs.org/hidefile 312ms (from cache)
1092 http fetch GET 304 https://registry.npmjs.org/amplify-python-function-runtime-provider 1344ms (from cache)
1093 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 344ms
1094 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1095 http fetch GET 304 https://registry.npmjs.org/amplify-util-import 1352ms (from cache)
1096 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 1376ms
1097 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1098 http fetch GET 304 https://registry.npmjs.org/amplify-provider-awscloudformation 1552ms (from cache)
1099 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 1432ms
1100 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1101 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 1624ms
1102 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1103 http fetch GET 304 https://registry.npmjs.org/ini 368ms (from cache)
1104 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 432ms
1105 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1106 http fetch GET 304 https://registry.npmjs.org/inquirer 440ms (from cache)
1107 http fetch GET 304 https://registry.npmjs.org/lodash 392ms (from cache)
1108 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 464ms
1109 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1110 http fetch GET 304 https://registry.npmjs.org/node-fetch 368ms (from cache)
1111 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 416ms
1112 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1113 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 392ms
1114 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1115 http fetch GET 304 https://registry.npmjs.org/amplify-util-mock 1608ms (from cache)
1116 http fetch GET 304 https://registry.npmjs.org/open 424ms (from cache)
1117 silly pacote version manifest for<EMAIL_ADDRESS>fetched in 1680ms
1118 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1119 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 472ms
1120 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1121 http fetch GET 304 https://registry.npmjs.org/ora 496ms (from cache)
1122 http fetch GET 304 https://registry.npmjs.org/parse-json 432ms (from cache)
1123 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 512ms
1124 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1125 http fetch GET 304 https://registry.npmjs.org/progress 408ms (from cache)
1126 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 440ms
1127 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1128 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 432ms
1129 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1130 http fetch GET 304 https://registry.npmjs.org/promise-sequential 328ms (from cache)
1131 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 344ms
1132 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1133 http fetch GET 304 https://registry.npmjs.org/semver 360ms (from cache)
1134 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 368ms
1135 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1136 http fetch GET 304 https://registry.npmjs.org/tar-fs 440ms (from cache)
1137 http fetch GET 304 https://registry.npmjs.org/folder-hash 1400ms (from cache)
1138 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 448ms
1139 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1140 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 1400ms
1141 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1142 http fetch GET 304 https://registry.npmjs.org/update-notifier 464ms (from cache)
1143 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 472ms
1144 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1145 http fetch GET 304 https://registry.npmjs.org/which 416ms (from cache)
1146 http fetch GET 304 https://registry.npmjs.org/winston 400ms (from cache)
1147 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 432ms
1148 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1149 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 416ms
1150 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1151 http fetch GET 304 https://registry.npmjs.org/node-emoji 560ms (from cache)
1152 http fetch GET 304 https://registry.npmjs.org/rimraf 560ms (from cache)
1153 http fetch GET 304 https://registry.npmjs.org/strip-ansi 568ms (from cache)
1154 http fetch GET 304 https://registry.npmjs.org/graphql 568ms (from cache)
1155 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 576ms
1156 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1157 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 576ms
1158 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1159 http fetch GET 304 https://registry.npmjs.org/xcode 584ms (from cache)
1160 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 592ms
1161 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1162 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 600ms
1163 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1164 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 600ms
1165 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1166 http fetch GET 304 https://registry.npmjs.org/yargs 592ms (from cache)
1167 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 616ms
1168 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1169 http fetch GET 304 https://registry.npmjs.org/graphql-config 472ms (from cache)
1170 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 488ms
1171 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1172 http fetch GET 304 https://registry.npmjs.org/graceful-fs 216ms (from cache)
1173 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 232ms
1174 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1175 http fetch GET 304 https://registry.npmjs.org/universalify 232ms (from cache)
1176 http fetch GET 304 https://registry.npmjs.org/jsonfile 232ms (from cache)
1177 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 240ms
1178 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1179 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 240ms
1180 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1181 http fetch GET 304 https://registry.npmjs.org/graphql-request 312ms (from cache)
1182 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 320ms
1183 silly resolveWithNewModule<EMAIL_ADDRESS>checking installable status
1184 http fetch GET 304 https://registry.npmjs.org/graphql-import 336ms (from cache)
1185 silly pacote range manifest for<EMAIL_ADDRESS>fetched in 392ms
1186 warn deprecated<EMAIL_ADDRESS>GraphQL Import has been deprecated and merged into GraphQL Tools, so it will no longer get updates. Use GraphQL Tools instead to stay up-to-date!
If it still gives you trouble, you can use the compiled Amplify CLI as a workaround. Windows command: curl -sL https://aws-amplify.github.io/amplify-cli/install-win -o install.cmd && install.cmd
For more information, see https://docs.amplify.aws/cli/start/install

Hi @gersondi, can you please upgrade to the latest version of npm and try again?
If it still gives you trouble, you can use the compiled Amplify CLI as a workaround. Windows command: curl -sL https://aws-amplify.github.io/amplify-cli/install-win -o install.cmd && install.cmd
For more information, see https://docs.amplify.aws/cli/start/install

Thanks @jhockett the workaround using
curl -sL https://aws-amplify.github.io/amplify-cli/install-win -o install.cmd && install.cmd
installs Amplify CLI

Thanks @jhockett the workaround using
curl -sL https://aws-amplify.github.io/amplify-cli/install-win -o install.cmd && install.cmd
installs Amplify CLI

Note that this fails with Node 14.16.1, latest LTS with npm 6.14.12
Works in node v15.14.0 (npm v7.7.6)
Probably a bit too eager to use latest technology. :)


face changing possibility

What will happen if you use this method to change face?

The current version does not work for face changing.
It might work if you tune the model on face-changing data.


Update to Laravel 6.5 + PHPUnit 8.4 + PHP CS Fixer

Original PR #80 by @TheFrankman and #84 @michaelaguiar

Why did you added "illuminate/http": "5.1 - 6.5" instead of "5.1 - 5.8|^6.0"? Laravel moved to SemVer when it reached 6.0.

Interesting point. I've followed the upgrade guide and there is no mention about using correct tag. Also I've used listed tags and it advised to use 6.5 (which I've installed in test repo and everything worked): https://github.com/pionl/laravel-chunk-upload-example/blob/master/composer.json
Maybe I can remove the strict to prevent updating the repo all the time. Thanks for the quick feedback.


(GH-531) Implemented build provider support for github actions

This pull request adds a new build provider that will be used when running Cake.Recipe on github actions.
This build provider should be updated once Cake.Recipe supports 0.36.0+ to use Cakes included BuildSystem support.
A small basic workflow file was also added to be able to verify build isn't breaking anything on gh actions.
Unfortunately only windows could be enable until Cake.Recipe supports Cake 0.37.0+
resolves #531
/cc @gep13

This is now ready for review.
There are 2 known problems, if the branch contains a / then the provider won't show parse the correct branch (minor setback IMO).
The other problem is that Cake.Recipe detect GitHub Actions as a Local Build even after adding this provider, this unfortunately due to the version of Cake that Cake.Recipe currently supports do not have any implementation for GitHub Actions.

As it currently stands, I personally am targeting 1.0.0 for the next version of Cake.Recipe, since there is a bug fix in there that will be required for ReportGenerator to run properly as a global tool.  I would be happy to bump the supported version to 0.38.4 for the time being, if that helps move this PR forward.

@gep13 the first problem I mentioned would still be a problem, and whether the build is detected as a local build could probably be worked around anyhow.
I didn't consider the first problem as much of a problem, as the main branches (master/develop) is still detected correctly.
Give me a sec to see if there is anything I workaround regarding GH Actions being detected as a local build.

Added the workaround now, didn't want to bump Cake just yet as there is other things that need to be taking into consideration when doing that (like build system providers being used that are deprecated).
Just waiting on the workflow file to confirm if the workaround worked or not.

and of course, I mixed up and/or operators 😢

@AdmiringWorm said...
and of course, I mixed up and/or operators

So glad to hear that this isn't just me!

Confirmed that the workaround... uhm... worked: https://github.com/cake-contrib/Cake.Recipe/pull/538/checks?check_run_id=854341376#step:3:962

@AdmiringWorm your changes have been merged, thanks for your contribution 👍


--runonce with agent updates

It would be nice if --runonce didn't consider an agent update as a "run", but instead only counted actual jobs run by the server.
Consider that you may be running builds within a temporary environment that exists only for the duration of a job run.  For example: running within a docker container inside Kubernetes.
If the docker container was built with azure-pipelines-agent version [x], and it's asked to auto-update to version [y], it will download that new update and exit immediately:
Generate and execute update script.
Agent will exit shortly for update, should back online within 10 seconds.

This means that it's done no actual work.  Since this is a docker container, the actual update won't propagate to future runs, which will start with a new instance of the container.  So now your entire cluster would be out of service as they can only update themselves then stop.
If, however, it processed the update then listened for a job to run, they would be operational (albeit a bit slower with the unnecessary software updates).
If allowable, it would further be nice if I could simply not update my agents automatically.  I may be willing to risk not having the features in a new agent for some period of time while I update my docker image manually.

Also note that for end-users, it's rather confusing that some releases are marked as "pre-release", although you'll be automatically upgraded to them.  I downloaded the latest "actual" release from that page and was promptly auto-upgraded as soon as it connected to Azure Pipelines.  I may not understand the distinction between pre-release and release versions, but I would expect that if I was always using the latest "release" that I wouldn't encounter auto-updates.

Sorry, I missed this on stderr the first time.  I noticed in the run.sh that it would try to restart itself, but it's failing:
/agent/run.sh: line 47: /agent/bin/Agent.Listener: No such file or directory

Indeed, raising my delay to 15 seconds fixes this issue for me.  Alas.


Restic v0.11.0

https://github.com/restic/restic/releases/tag/v0.11.0
Hey all,
this is a ticket to

upgrade the included restic version
go throught the changelog and discuss whether changes affect us or could be of further interest.

Let's discuss

From a first look, I do not see any changes which could break resticker. Should be a smooth upgrade I guess.
We could think about supporting the new repository-file option, but I am not sure if this is interesting for anyone.

Done in #59


error while installing [err] syncing dependencies

    [ok] checking for go binary
    [ok] checking GOPATH
    [ok] installing dependency manager
   [err] syncing dependencies

failed cmd:
         glock sync 'github.com/itsabot/abot'

bash: glock: command not found

Interesting.  It seems your $GOBIN isn't configured. Can you share the output of echo $GOBIN?
AFAIK a default installation of Go should point it toward /usr/local/go/bin

Hi, thanks for the reply.
I didn't have my GOBIN set. I did set it in my .bashrc. but before I did I installed glock manually using go get github.com/robfig/glock and now it works.
I don't know if the glock installation step was necessary or not. so I don't know if this is just a matter of $GOBIN or a missing dependency.
If there is anything I can do to help figure this out, please let me know. (I'm a noob in go so I wouldn't know on my own).

oh and by the way, it does work.

The glock installation step is handled automatically in cmd/setup.sh, but it looks like it was a GOBIN problem.   Glad to hear it's sorted out!

Thank you! :)


fixes permanently down after refresh

root cause:
cached bean trying to re-register
resolution summary:
get re-created bean from context to re-register
Fixes #2972

I'll port fix to master soon

Don't port, we merge forward if accepted.

Codecov Report

Merging #3018 into 1.4.x will increase coverage by 7.2%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##              1.4.x    #3018     +/-   ##
===========================================
+ Coverage     69.38%   76.59%   +7.2%
+ Complexity     1923     1360    -563
===========================================
  Files           240      167     -73
  Lines          8489     5391   -3098
  Branches       1053      755    -298
===========================================
- Hits           5890     4129   -1761
+ Misses         2196      970   -1226
+ Partials        403      292    -111

Impacted Files
Coverage Δ
Complexity Δ

...ix/turbine/EurekaBasedTurbineClustersProvider.java

...eureka/server/EurekaServerMarkerConfiguration.java

...d/netflix/turbine/TurbineAggregatorProperties.java

...cloud/netflix/turbine/EurekaInstanceDiscovery.java

...ud/netflix/eureka/metadata/ManagementMetadata.java

...lix/eureka/EurekaDiscoveryClientConfiguration.java

...ework/cloud/netflix/sidecar/SidecarController.java

...tflix/metrics/spectator/SpectatorMetricReader.java

...eka/http/RestTemplateTransportClientFactories.java

...mework/cloud/netflix/eureka/CloudEurekaClient.java

... and 60 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2df8103...c473f5a. Read the comment docs.

can't get why it's failing, locally works well and changes should not affect test functionality which is failing...

@ryanjbaxter can we port it to master branch?

I merged it forward, you might want to double check the merge because it wasnt pretty.  All the tests were passing for me locally.

I see some unrelated changes to my PR at https://github.com/spring-cloud/spring-cloud-netflix/commit/13c2500d0ea77734223f3bcac4bdf0863a0fd3b5 - you want me to polish them?

The unrelated changes are coming from other PRs that were merged forward


Scrolling/panning doesn't / can't work

Describe the bug
"To pan, hold CMD and the scroll wheel, then drag. To zoom, hold CMD and scroll."
My laptop doesn't have a scroll wheel. It's not clear how I would pan; I'm not aware of a way to create middle-click-and-drag events, and most applications use modifier keys instead.
Scrolling is a valid touch gesture, of course, but CMD-scroll also has no effect; the cursor size changes instead.
To Reproduce
Steps to reproduce the behavior:

Buy a macbook
Go to https://zero01101.github.io/openOutpaint/
Attempt to zoom

Expected behavior
Functioning zoom.
Desktop (please complete the following information):

OS: OSX 12.5.1
Browser: Chrome / Safari
Version: Current as of 2022-12-23

Additional context
"Buy a macbook" is an unfortunate debugging instruction, and this will probably work if I connect an external mouse... I hope. However, maybe you could add a "pan" tool? I think "click m, click-drag" would be equally fast anyway.

That's a good point. Notebook users would have some difficulties with a track pad. We can probably allow panning with the right mouse button as well. Would that help in your case?

Sadly, no. Macbooks only have one "button". I can right-click by clicking with two fingers, but I can't right-click-and-drag.
The mac-native way to do that is to hold the Option modifier key, which is roughly equivalent to Alt.

Will see what I can do... I don't think it would be too hard, but it would definitely not be easy from how we handle mouse inputs.

This is more of an enhancement request than a bug per se.

While I'm here... the UI seems to react to scroll events, without considering the size of those events. Touchpad scrolling has much higher granularity than mouse scrolling.
In particular, the smallest possible finger movement leads to scrolling all the way from 512x512 to 2048x2048 and then back through 512x512 at least once. 🤣
What I think you'd need to do is look at WheelEvent.deltaY, as well as deltaMode, accumulate deltas and only treat the event as 'real' once it's scrolled enough to match one mouse-wheel movement. ...or preferably more than one. The touch pad is harder to control.

Well, knew that would come to bite us sometime!
We actually changed that because due to incompatibility between browsers, sometimes scroll would simply not do anything because of too tiny deltaY values.
I guess we will have to implement this properly now.
Will search for the commit regarding the mouse changes and update here.
During the next few days it will be a bit hard for me to implement these due to the holidays. The drag modifier specially will require quite some structural change to the input lib. The scroll fix should be a bit easier.

This is the previously referred commit in which mouse wheel handling was changed: aee812b70a7cb51c3415147031eb64faab77bfc4
A new commit (3d29847) has been pushed to the testing branch with an attempt to fix smooth scrolling devices on chrome. In chrome, smooth scrolling devices send A LOT of events, differently from Firefox, which accumulates smooth scrolling data until a threshold of a line is reached, and sends a single event.
@Baughn if you could give this fix a look to see if it fixes your touchpad scrolling issues  (You would need to clone this repo and run git checkout testing to check).
@zero01101 Would it be possible to create a new deployment in github-sites for the testing branch? (something like https://zero01101.github.io/openOutpaint-testing/)

Values are based on my tests:
Snapping Mouse Wheel on Firefox:

Snapping Mouse wheel on Chrome:

Smooth Touchpad on Firefox (Admittedly, would not be smooth):

Smooth Touchpad on Chrome:

Would it be possible to create a new deployment in github-sites for the testing branch? (something like https://zero01101.github.io/openOutpaint-testing/)

hmm, apparently not without just dumping the testing branch in an entirely different repo (https://github.com/orgs/community/discussions/21582, https://stackoverflow.com/questions/32629223/github-pages-dev-branch) but can offer this in penance as a hacky workaround:
https://raw.githack.com/zero01101/openOutpaint/testing/index.html

"Buy a macbook" is an unfortunate debugging instruction

but it's hilarious and luckily(?) i have a macbook provided by my employer, i just never use it for anything stable diffusion related... i can "confirm" that you CAN INDEED zoom by holding cmd and using the two finger gesture on the trackpad on a 2021 m1 macbook pro running a pretty outdated macOS, but god yeah it's an awful experience isn't it
i could respond to "Buy a macbook" with "buy a real mouse" but that seems petty ;)

Not everyone have a middle button. I'm voting for just a RMB for panning.

By @w84death on https://github.com/zero01101/openOutpaint-webUI-extension/issues/9.
For now,

We can't really simply RMB as quick RMB drags would also be counted as clicks, which would just erase whatever is under them on some tools.

I'm considering (Alt/option) + LMB Drag as an alternative for panning, which could be a reasonable compromise for both Macbook/non-middle click users.

yeah, i'm personally more inclined to go with left-click over right-click myself for this as well; i definitely prefer ctrl over alt just to stick with the "camera modifier key" paradigm just as a gut feeling :)

Hi, me again. I have a macbook, but I use an external usb mouse and keyboard. Pan or zoom isn't working for me.

So, Ctrl + mouse wheel is not doing anything?

should probably be cmd+scrollwheel on mac i believe...

....PEBCAK, ctrl is doing it. false alarm.

oh, it is ctrl on mac?  huh, will have to update docs

He is using an external keyboard. Not sure if an apple one.

K, so it's not my fault at least, the docs did say command.

It is an apple keyboard.

Okay, so Ctrl it is!

lmao i coulda SWORN it was cmd on my macbook, i'll have to check that

I can confirm that zooming work now, and brush size / render size selection feels comfortable. Panning doesn't work, but I don't think it was expected to.
Zoom would probably feel more comfortable if it was centered on wherever the cursor is at the moment, instead of always centering on the center of the canvas. (And, I'll admit, that would also work as a semi-functional replacement for panning.)

Hi, @Baughn! Could you test our solutions (allowing LMB drag using control, and mouse wheel fix) on the testing branch? (git checkout testing on the openOutpaint directory, or app directory without webui restart, if using the extension)

It is now on main, actually. Just update openOutpaint and test the changes.

Hmm. Well, nothing seems to have changed? It's still zooming into the center, and ctrl-drag (or cmd-drag) does nothing.
This is on the extension, in case it matters. Assuming I don't need to switch the branch on that.

Right... let's see.

Zooming in/out now zooms in/out on where the cursor is, which is great!
Unfortunately, it does so whether or not I'm holding ctrl. 🤣
I don't think this is new, but if I turn on 'sync cursor size' then the scroll wheel alters the stable diffusion resolution setting, but not the cursor size. It also does so at super high speed; the new scroll logic doesn't seem to apply.
After I turn it back off, the cursor size remains fixed; the scroll wheel now only zooms.

Are mouse controls still working properly?

Which ones? (I thought I covered everything.)

Oh no, general mouse controls! Just to see if I didn't break anything!

Those seem to work.

Trying it again today, the zoom-always-happens issue appears to have disappeared. I haven't updated the extension in the meantime, so I can only blame the browser cache... it might be a good idea to put a version number (hash?) in the filenames, if they don't have that already.

it might be a good idea to put a version number (hash?) in the filenames, if they don't have that already.

Agreed, as some errors seem to be derived from caching on other issues as well. Have been thinking on how to do it in a way that is sort of seamless for developers though. Git hooks are the most obvious solution, but we cannot really verify if a developer actually set up the hooks correctly.
GitHub actions alone would be fine, but would pollute the commit history quite a lot (one extra commit for each push to branch/pull request).
An approach that combines both, and only creates a correction commit when run on GitHub actions with incorrect hashes would be optimal.

It doesn't necessarily have to be part of the filenames, if you're using static file serving.
"foo.js?42" is a distinct resource from "foo.js?43", and won't be cached. So it should work if you put the current git hash in the query fragment, say.

Still, it would be pretty gruesome to have to update values for every resource indexed! Have sent a pull with some scripts, hooks and GitHub action entries at #130! Will hopefully not be a problem anymore going forward.

You can now give a look at  testing; should fix some sync issues.

Well, my implication was you'd generate the URLs on the fly. Possibly by scraping the hash from git, as in--

...er, I still haven't gotten used to that ability. ^^;
That way you wouldn't need to update anything manually. That said, the conversation has gone wildly off-topic for this bug... and I might just send you a PR instead. Is there an IRC channel or Discord server I should be in?

Not really, actually. We could ask @zero01101 to create one!
Also, we can't really generate anything on the fly because this is a fully frontend-only application with static file serving, and we plan to keep it this way. :)
Using timestamps is also non-optimal, as it would basically never cache.
So this is the most we can do to perform some type of proper cache busting. Still, I think this is a good compromise on developer ease and reliability.

lol ok so i uh... have used discord about 4 times before? i'm clearly old; every time i log in i think "man this should've just been a forum or something" lmao
i've got a local homelab IRC server idiotically exposed to the internet at irc dot djabacus dot net on port 6667 (annoying url to deter bot scraping like that'll do anything at all) and just started an #openOutpaint channel, but if all the cool kids truly prefer discord i guess i can probably look into what that entails ;)

Sorry to butt in but discord is super easy. It had private messages but it
is sorted by servers which are hosted by discord, and a server is divided
into channels. No ports or anything needed
On Thu, Dec 29, 2022 at 6:32 PM tim h @.***> wrote:

lol ok so i uh... have used discord about 4 times before? i'm clearly old;
every time i log in i think "man this should've just been a forum or
something" lmao
i've got a local homelab IRC server idiotically exposed to the internet at
irc dot djabacus dot net on port 6667 (annoying url to deter bot scraping
like that'll do anything at all) and just started an #openOutpaint
channel, but if all the cool kids truly prefer discord i guess i can
probably look into what that entails ;)
—
Reply to this email directly, view it on GitHub
https://github.com/zero01101/openOutpaint/issues/116#issuecomment-1367637151,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A4TQTDCEYHIOOA7QPYNDF7DWPYNSBANCNFSM6AAAAAATH6NWBQ
.
You are receiving this because you commented.Message ID:
@.***>

¯\_(ツ)_/¯
https://discord.gg/tCPTr3pu

Hi, @Baughn! Just to confirm if there is anything else that should be fixed!

Just one bug left, though it's an odd one.
If I click 'sync cursor size', whether to disable or enable, then the scroll wheel stops working to change cursor size until I ctrl-drag to move the canvas. Zooming the canvas works, but does not unstick it.

When you do this, is the reticle of the dream tool blueish? If so, it's more that the checkbox is focused, so clicking anywhere, including the canvas, should restore cursor size wheel handling.
For the future, I guess that we can automatically blur focus on checkboxes on value change event...

It doesn't visibly change appearance, but yes, that seems to be the cause.

From what I can test here, clicking on a checkbox does indeed make the reticle/crosshair turn bluish in appearance. Still, here is a fix for the full checkbox issue:

https://github.com/zero01101/openOutpaint-webUI-extension/issues/11
Did your testing branch break the main branch?

Will be closing this issue for now, as all pending problems seem to be resolved.


python CombineGlobally().with_fanout() cause duplicate combine results for sliding windows

not only there are more than 1 result per window, results for each window got duplicated as well.
here is some code I made to reproduce the issue, just run it with and without *.with_fanout*
if running with Dataflow runner, add appropriate *gs://path/* in *WriteToText*

import apache_beam as beam
from apache_beam.transforms import window
from apache_beam.utils.timestamp import Timestamp

class ListFn(beam.CombineFn):
  def create_accumulator(self):
    return []

 def add_input(self, mutable_accumulator, element):
    return mutable_accumulator + [element]

 def merge_accumulators(self, accumulators):
    res = []
    for accu in accumulators:
      res = res + accu
    return res

  def extract_output(self, accumulator):
    return accumulator

p = beam.Pipeline()

(
    p
    | beam.Create([
      window.TimestampedValue(1, Timestamp(seconds=1596216396)),

     window.TimestampedValue(2, Timestamp(seconds=1596216397)),
      window.TimestampedValue(3, Timestamp(seconds=1596216398)),

     window.TimestampedValue(4, Timestamp(seconds=1596216399)),
      window.TimestampedValue(5, Timestamp(seconds=1596216400)),

     window.TimestampedValue(6, Timestamp(seconds=1596216402)),
      window.TimestampedValue(7, Timestamp(seconds=1596216403)),

     window.TimestampedValue(8, Timestamp(seconds=1596216405))])
    | beam.WindowInto(window.SlidingWindows(10, 5))
    | beam.CombineGlobally(ListFn()).without_defaults().with_fanout(5)
    | beam.Map(repr)

   | beam.io.WriteToText("py-test-result", file_name_suffix='.json', num_shards=1))

p.run()

Imported from Jira BEAM-10617. Original Jira may contain additional context.
Reported by: leiyiz.

If this is still true, then it should be very easy to reproduce. @robertwb or @tvalentyn or @pabloem do you know?

@BjornPrime can you try to reproduce this ?

I've successfully reproduced the issue.
Compare with fanout: https://paste.googleplex.com/5670550667001856 and without: https://paste.googleplex.com/6400888045305856

Thanks, I'll add it to our interrupts tracker.

I feel like the labels P1 and good first issue should be mutually exclusive?

it may be neither actually.

It sounds like data loss, which should certainly be P1

Sure, we can investigate as P1. P2 was based on the suspicion that this behavior has been like this from initial implementation.
As next step, I think we should identify whether with_fanout by nature of its implementation only works for particular classes of combiners/accumulators, for example:  Top or Mean where repeating elements twice doesn't affect the end result, or whether this is a bug that can and should be fixed.

If we cannot quick fix, we should remove with_fanout until we can fix it, or at least warn somehow, and add to release notes for released SDKs.

I mean, disable with_fanout when sliding windows.
BTW is this on all runners?

OK, the problem is that we re-window to avoid stacked accumulating mode:
https://github.com/apache/beam/blob/release-2.40.0/sdks/python/apache_beam/transforms/core.py#L2742
This re-duplicates the windows for sliding windows WindowFns. (It will do the wrong thing for sessions as well.) What we need to do is set the new windowing to be the same thing without accumulating mode, but without applying the windowfn.

If we can't get someone to fix this we should at least throw an error ASAP, before the next release, e.g. rejecting all but known good windows (globals and fixed windows).

@tvalentyn any updates here ?

started working on this today, no PR yet.

This re-duplicates the windows for sliding windows WindowFns. (It will do the wrong thing for sessions as well.)

Does not seem to repro with sessions.  Passing test:
  def test_combining_with_session_windows_and_fanout(self):
    import logging
    class ListFn(beam.CombineFn):
      def create_accumulator(self):
        return []

      def add_input(self, mutable_accumulator, element):
        return mutable_accumulator + [element]

      def merge_accumulators(self, accumulators):
        res = []
        for accu in accumulators:
          res = res + accu
        return res

      def extract_output(self, accumulator):
        return accumulator

    options = PipelineOptions()
    options.view_as(StandardOptions).streaming = True
    with TestPipeline(options=options) as p:
      def has_expected_values(actual):
        from hamcrest.core import assert_that as hamcrest_assert
        from hamcrest.library.collection import contains_exactly
        from hamcrest.library.collection import only_contains

        hamcrest_assert(ordered, contains_exactly([0, 1, 2, 3], [5, 6, 7, 8]))

      result = (
              p
              | beam.Create([
        window.TimestampedValue(0, Timestamp(seconds=0)),
        window.TimestampedValue(1, Timestamp(seconds=1)),
        window.TimestampedValue(2, Timestamp(seconds=2)),
        window.TimestampedValue(3, Timestamp(seconds=3)),

        window.TimestampedValue(5, Timestamp(seconds=5)),
        window.TimestampedValue(6, Timestamp(seconds=6)),
        window.TimestampedValue(7, Timestamp(seconds=7)),
        window.TimestampedValue(8, Timestamp(seconds=8))])
              | beam.WindowInto(window.Sessions(2))
              | beam.CombineGlobally(ListFn()).without_defaults().with_fanout(5)
      )

Yea, it probably has to do with any windowfn that duplicates an element into multiple windows

OK, the problem is that we re-window to avoid stacked accumulating mode:

Can we re-window only if we detect accumulating mode? See: https://github.com/apache/beam/pull/23828
Alternatively, we could not rewindow if we detect slidng windows. disabling sliding windows is straightforward too if we want to go that route for now.

This is the only issue currently blocking the 2.43.0 release without an existing cherry-pick request.
Can we push this to the next release ?

I discussed the fix yesterday w/ Robert and we will pursue a slightly different fix, which should be ready shortly. however, this bug has been there from the very first commit implementing with_fanout, so I wouldn't delay this specific release on it.

Ack. Removing this from the milestone for now.

current fix caused an OOM  in one of customers' pipelines , planning to revert and investigate further.

Is it possible that the OOM is unavoidable because of the fix? This is a pretty seriouis data corruption problem, no? I suppose it is not a regression but I would very much like 2.44.0 to have correct results in this situation.

Is it possible that the OOM is unavoidable because of the fix?
it's unlikely

This is a pretty seriouis data corruption problem, no? I suppose it is not a regression
that's correct. it has been there since when with_fanout was added.

but I would very much like 2.44.0 to have correct results in this situation.
I am planning to come back to it next week when I am on a rotational duty again.

OK given this context I am going to remove the release milestone. We can document the range of releases for which it does not work.

Actually simply disabling it would be a reasonable way to protect user data and we should do that in the immediate term.

The mitigation has been cherrypicked, so I am removing it from the 2.44.0 milestone. The bug remains open.

If it isn't actively being worked on, I suggest downgrading and unassigning.

any progress on this ?

we had a tentative fix but it has caused a performance regression and was rolled back; since then it remains in the backlog.

I need the combiner because group-by is ineffective. And i needed it to further fanout as its streaming kafka input at quite a high rate.
any possible workaround approaches ?

Ack. this feedback would help w/ prioritization.
I'm not sure I can answer based on limited information about the usecase, but for experimentation processes you could apply https://github.com/apache/beam/pull/23828 locally. These changes only apply at job submission.


Field validation not triggered after reset

Very easy to reproduce:
https://codesandbox.io/s/o22j1lOZB
Modify age, then reset, then modify age, then submit.
Field validation on Username and Email are bypassed.
UPDATE_SYNC_ERRORS is not triggered after reset like it is when a validated field is dirty.
It looks like the conditions updateSyncErrorsIfNeeded are incorrects.
This is a major issue IMO. I have no easy workaround.
@erikras I know you must be terribly busy but can you have a look?

Hello there @bbenezech!
Where did u got that sandbox from ?
I certainly can reproduce the behavior on it, but I can't do it on the codesanbox provided in the docs:
https://codesandbox.io/s/PNQYw1kVy
perhaps someone screwed up while setting it up ?

@irvingv8 The only difference is the absence of validation on age.
If the modified field is validated, it works, if not, it doesn't. Sorry I should have made that clearer.
I looked at the conditionals in updateSyncErrorsIfNeeded, not sure I can do it, I have no clear idea what is going on here.

Ok! I Believe I fixed it! I'll submit a pull request, will be my first contribution to OSS <3! hopefully I did it well, even though this was an easy one

@erikras is there any guide/steps on how to contribute to redux-forms? It was little bit hard for me to get to test what i modified, I find the rebuild examples script, but it was going over all of the examples taking a LOT of time to rebuild each of them and I just wanted to test a single one.
I ended up deleting all the other examples, running the script and testing...
Maybe the script could take the name of a example to rebuild ?

I actually ran into this exact issue myself a while back. Really appreciate the PR, and I'm willing to help get this moved along!
You probably want to add a test here. Specifically, something like it should not erase sync errors if they exist. The test structure is pretty self-explanatory.
One question I have: I'm wondering about the order of things happening.

Validation seems to happen when new props are received. This function checks to see if values are deeply equal, and skips validation if so.
Your PR seems to be simultaneously setting the values to initialValues, and syncErrors to the existing syncErrors. Wouldn't these errors be outdated, so to speak?
The issue is only happening when "Age" is the only field populated when the form is reset. In other words, validation is being run after resetting when "Name" or "Email" field values change upon reset. It's not that errors aren't being properly persisted.

It seems the correct fix would lie in making either updateSyncErrorsIfNeeded or validateIfNeeded in the main file more robust.

Ty for the guidance @timhwang21, now that you mention it, it totally makes sense, I went straight to the fact that syncErrors was out of the state right after the reset, but you are right, there is something more to it as it only happens when the non validated field is the one populated.
I'll take another shot at it in a couple of hours and see if I can get to culprit of the trouble.
And about:

Your PR seems to be simultaneously setting the values to initialValues, and syncErrors to the existing syncErrors. Wouldn't these errors be outdated, so to speak?

values -> initialValues was already there.
And sorry but i didn't understand the "Wouldn't these errors be outdated, so to speak?", problem i found was that after the reset syncErrors wasn't in state anymore, so I'm not setting syncErrors to the existing ones (cause they don't exist), but I might be wrong it's my first time looking at the core.

Sure, what I meant is that values and errors are being populated at the same time in reset. Validation happens on the new values AFTER reset. The errors you are propagating are the errors from BEFORE reset. Hence any new errors caused by the reset won't be saved. Though, in the edge case at play here, if the reset causes errors then set validation would probably be called anyways. So it might work but strikes me as an indirect solution.
There was a PS from the last release that changes accessing errors from props to a property set on this, I had to stop digging but it seems this may be a regression due to that fix.

Good news, guys. I found the bug. This was a tricky one.

Would be great If you could share some details, I'd like to get started helping people on this repo, sorry for the Pull Request, It was my first time looking at the core. Is there any way I can contact you if I have any doubts about it ?

Published fix as v6.7.0.
@irvingv8 Twitter or email (see package.json)

This is still not working, no action for UPDATE_SYNC_ERROR is being dispatched & no validation occurs.

I confirm, the problem is not solved yet, after the reset, verification is not carried out..

@arman-mukatov could you provide a codesanbox sample to test ? I can no longer reproduce this issue with 6.8.0

I have this same problem using version 7.0.3

I won't re-open this bug because the original reproduction sandbox + repro steps do not work against the same sandbox upgraded to 7.0.3. If you are still experiencing an issue like this, please create a new formal issue report.

I am having same problem... validation does not work after reset form...


Translated into Ukrainian hello-world.md file

Added Ukrainian translation for hello-world.md file

@klymenkoo Please review changes.

@ithompson4 дякую за ваш переклад 😀


New devices and support for manufacturer / model specification

Added support for:

Diivoo Dual Smart Water Timer
Generic Tuya Smart Water Timer
Specification of Manufacturer & Model in yaml configuration

Bugfixes:

Execution during startup for creation of task not on main thread.

it might be worth looking also at my own PR: https://github.com/make-all/tuya-local/pull/851


do not require a C++ compiler

By default, CMake assumes both a C and C++ compiler are required.
This is not needed for mraa because it only provides C++ header
files.
Signed-off-by: Pieterjan Camerlynck<EMAIL_ADDRESS>

Thanks, Yeah sure why not!
Im concerned that it disables CXX even when its needed for the Swig compiles aspects of mraa. See the failed Travis build.
Conditionally adding CXX language feature to the project would probably satisfy this.
https://cmake.org/cmake/help/v3.0/command/enable_language.html?highlight=languages

Oops, built without the examples. Should be fixed now.

Great that looks like it might be it! It would be best as a single commit really.
I will give it a bit of testing in the morning!

It seems that @arfoll committed the patch (https://github.com/intel-iot-devkit/mraa/commit/a87a104efeb83d4e762992e5be3b8908e6d2bf40), so I'm closing the pull request.

Sorry I forgot to get back to you once pushed. Thanks for closing & the patch!


CORRELATION  interrupted

I use two stereo satellite images to generate DEM, and the dense matching algorithm I choose is asp_sgm.
When I use a stereo satellite image with a smaller area, I can successfully generate a better quality DEM, but when I use a satellite image with a larger area, the program breaks at the correlation stage.
Through the information output from the terminal, I found that the program is interrupted after processing all sub-blocks. Open the last subfolders in the folder such as "1-68640_54080_1385_1686", there are a total of 22 files in it.
The "Too many open files (code = 4)" error had previously appeared for larger areas of satellite imagery, but was resolved by using the latest version of ASP. Do you have any good suggestions for interruptions in the "correlation" phase?
Thank you very much.
The asp version I am using is as follows：
(base) hipeson@hipeson-Super-Server:~$ /home/hipeson/StereoPipeline-3.0.1-alpha-2022-04-14-x86_64-Linux/bin/parallel_stereo  --version
NASA Ames Stereo Pipeline 3.0.1-alpha
Build ID: e8809e32
Build date: 2022-04-14
Built against:
NASA Vision Workbench 3.0.0
Build ID: 63a66938
USGS ISIS 5.0.1
Boost C++ Libraries 106800
GDAL 2.4.1 | 20190315
The commands and parameters I use are as follows:
/home/hipeson/StereoPipeline-3.0.1-alpha-2022-04-14-x86_64-Linux/bin/parallel_stereo  fwd.tif  nad.tif   result/1  --threads 16  -t  rpc  -s 300-SGM-stereo.default
PREPROCESSING
alignment-method affineepipolar
ip-detect-method 1
ip-per-tile 200
individually-normalize
CORRELATION
cost-mode  4
stereo-algorithm 1
corr-kernel 7 7
corr-tile-size 2080
corr-memory-limit-mb  6000
The error message of the terminal is as follows:
[ 2022-Apr-17 14:32:27 ] : CORRELATION FINISHED
stereo_corr: elapsed=37:35.09 ([hours:]minutes:seconds), memory=5762104 (kb)
--> Setting number of processing threads to: 8
Using stereo file SGM-stereo.default.
Writing log info to: try1/1-12480_54080_2080_1686/12480_54080_2080_1686-log-stereo_corr-04-17-1407-23970.txt
Using session: rpc.
Loading camera model: block-fwd.tif
Loading camera model: block-nad.tif
Distance between camera centers in meters: 12616.5.
--> Using no pre-processing filter with stereo algorithm: asp_sgm
Stage 1 --> CORRELATION
--> Full-res search range based on D_sub: (Origin: (-1237, -341) width: 2637 height: 1441)
Kernel size:    Vector2(7,7)
Search range:   (Origin: (-1237, -341) width: 2637 height: 1441)
Cost mode:      4
Writing: try1/1-12480_54080_2080_1686/12480_54080_2080_1686-D.tif
--> Correlation :[................................................] 0%
--> Correlation :[................................................] 2%
……
--> Correlation :[***********************************************.] 99%
--> Correlation :[*******************************************] Complete!
[ 2022-Apr-17 14:32:36 ] : CORRELATION FINISHED
stereo_corr: elapsed=24:53.89 ([hours:]minutes:seconds), memory=3997736 (kb)
--> Setting number of processing threads to: 8
Using stereo file SGM-stereo.default.
Writing log info to: try1/1-14560_54080_2080_1686/14560_54080_2080_1686-log-stereo_corr-04-17-1408-24313.txt
Using session: rpc.
Loading camera model: block-fwd.tif
Loading camera model: block-nad.tif
Distance between camera centers in meters: 12616.5.
--> Using no pre-processing filter with stereo algorithm: asp_sgm
[ 2022-Apr-17 14:08:04 ] : Stage 1 --> CORRELATION
--> Full-res search range based on D_sub: (Origin: (-1237, -341) width: 2637 height: 1441)
Kernel size:    Vector2(7,7)
Search range:   (Origin: (-1237, -341) width: 2637 height: 1441)
Cost mode:      4
Writing: try1/1-14560_54080_2080_1686/14560_54080_2080_1686-D.tif
--> Correlation :[................................................] 0%
--> Correlation :[................................................] 2%
……
--> Correlation :[***********************************************.] 99%
--> Correlation :[*******************************************] Complete!
[ 2022-Apr-17 14:32:36 ] : CORRELATION FINISHED
stereo_corr: elapsed=24:35.27 ([hours:]minutes:seconds), memory=2202424 (kb)
Traceback (most recent call last):
File "/home/hipeson/StereoPipeline-3.0.1-alpha-2022-04-14-x86_64-Linux/libexec/parallel_stereo", line 1013, in
spawn_to_nodes(step, settings, parallel_args)
File "/home/hipeson/StereoPipeline-3.0.1-alpha-2022-04-14-x86_64-Linux/libexec/parallel_stereo", line 522, in spawn_to_nodes
asp_system_utils.generic_run(cmd, opt.verbose)
File "/home/hipeson/StereoPipeline-3.0.1-alpha-2022-04-14-x86_64-Linux/libexec/asp_system_utils.py", line 480, in generic_run
raise Exception('Failed to run: ' + cmd_str)
Exception: Failed to run: parallel --will-cite --env ASP_DEPS_DIR --env PATH --env LD_LIBRARY_PATH --env ASP_LIBRARY_PATH --env PYTHONHOME -u -P 10 -a /data/image/150/tmp_nftxqkn "/home/hipeson/StereoPipeline-3.0.1-alpha-2022-04-14-x86_64-Linux/bin/python /home/hipeson/StereoPipeline-3.0.1-alpha-2022-04-14-x86_64-Linux/libexec/parallel_stereo block-fwd.tif block-nad.tif try1/1 -t rpc -s SGM-stereo.default --skip-low-res-disparity-comp --processes 10 --threads-multiprocess 8 --entry-point 1 --stop-point 2 --work-dir /data/ image /150 --isisroot /home/hipeson/StereoPipeline-3.0.1-alpha-2022-04-14-x86_64-Linux --tile-id {}"

Your search range is outrageous, meaning the last two numbers on this line:
Full-res search range based on D_sub: (Origin: (-1237, -341) width: 2637 height: 1441)
Likely you have either high mountains, or clouds, or some outlier somewhere, or bad alignment. You can try opening both aligned images in stereo_gui like this:
stereo_gui --single-window result/1-L.tif result/1-R.tif
and toggle one of them on and off, and see just how different they are. If you see little changes in some places but huge changes in other places that means the alignment did the job but it was not enough. Then you can try using mapprojected images and some options to reduce the search range, per this: https://stereopipeline.readthedocs.io/en/latest/tutorial.html#handling-clouds and https://stereopipeline.readthedocs.io/en/latest/next_steps.html#mapproj-example.
(BTW, in the last days or so I managed to introduce a crash in in the --ip-filter-using-dem option which I caught a bit later, so if you have the bad luck or running into it you'd need to either not use this with your build or update it to the latest, which is 2022-04-17-daily-build .)

glad to have your advice.
I do have high mountains in the right area of my image. Due to some characteristics of my experimental images, vertical disparity cannot be eliminated as well as traditional satellite imagery through image alignment. Therefore, on the epipolar image, the same pixel has horizontal disparity and vertical disparity.
The traditional dense matching method based on epipolar 1D search doesn't work for my images, but ASP's automatic estimated search range works well, which I think is great.
Based on your suggestion, the next step is to use the mappproject tool to geo-project the image, use the ASP of 2022-04-17-daily-build, and add the --ip-filter-using-dem option. Then generate a DEM to see if it can be successful
In addition to that, I would like to ask you some conjectures.

Since the whole image is not well aligned, is it possible to use the local_epipolar alignment method while increasing the value of ip-per-tile to get more matching points. Does this make the search range smaller and more precise?

Change corr-seed-mode to 2, which is Low-resolution disparity from an input DEM. Use the Copernicus 30 m DEM or a better resolution open source DEM. Does this method narrow the search range compared to corr-seed-mode 1?

Thank You~

When I use the mapproject tool for image projection, the program breaks. My image resolution is about 1m, so I set mpp to 1, and the open source DEM is Copernicus 30m DEM. According to the error message, it should be because the image is too large. How to successfully project in this case?
The command entered in the terminal is as follows：
mapproject  -t rpc  --mpp 1  COP30.tif  nad.tif  map-nad.tif  --tif-compress  LZW --threads  20
The error message of the terminal is as follows：
Image box: (Origin: (0, 0) width: 181523 height: 91054)
Output image size:
(width: 181523 height: 91054)
Writing: map-nad.tif
[.........................................................................] 0%terminate called after throwing an instance of 'vw::ArgumentErr'
what():  Refusing to allocate an image larger than 79999 pixels on a side (you requested 108651 x 280)

Since the whole image is not well aligned, is it possible to use the local_epipolar alignment method while increasing the value of ip-per-tile to get more matching points. Does this make the search range smaller and more precise?

This should work in moderately steep terrain. For very steep terrain the mapprojection option should work better.

Change corr-seed-mode to 2, which is Low-resolution disparity from an input DEM. Use the Copernicus 30 m DEM or a better resolution open source DEM. Does this method narrow the search range compared to corr-seed-mode 1?

The --corr-seed-mode 2 option is good when you cannot get a good initial disparity, maybe because of  noise in images. But if your terrain is truly steep, rather than having clouds which just artificially increase the range, this option cannot reduce the search range more than what the terrain says it should be.
So mapprojection still looks like the better option.

mapproject -t rpc --mpp 1 COP30.tif nad.tif map-nad.tif --tif-compress LZW --threads 20
The error message of the terminal is as follows：
Image box: (Origin: (0, 0) width: 181523 height: 91054) Output image size: (width: 181523 height: 91054) Writing: map-nad.tif [.........................................................................] 0%terminate called after throwing an instance of 'vw::ArgumentErr' what(): Refusing to allocate an image larger than 79999 pixels on a side (you requested 108651 x 280)

I spent quite a bit of time trying to reproduce this, and with no luck. I tried even bigger data than yours, and each time the software was breaking up the images into small tiles, rather than failing as above. Here's what I think the tool should be doing:
mapproject -t rpc --mpp 0.01 copernicus.tif left.tif left.map.tif --tif-compress LZW --threads 20
mapproject_single --query-projection copernicus.tif left.tif  left.map.tif -t rpc --mpp 0.01 --tif-compress LZW --threads 20
--> Setting number of processing threads to: 20
Using session: rpc.
Loading camera model: left.tif
Image box: (Origin: (0, 0) width: 1826875 height: 1364328)
Output image size:
(width: 1826875 height: 1364328)
Query finished, exiting mapproject tool.
Output image size is 1826875 by 1364328 pixels.
Splitting into 357 by 267 tiles.
Can you try to type " mapproject --version", and see what you get? Hopefully it is a recent date. If the date is recent, you can also try adding '--tile-size 2048' to the command, but normally such a thing should happen automatically for recent versions of the software.

Sorry I was using ASP2.7 before, so I got the previous error.
When I read the ASP documentation, the parameter "--mpp" is interpreted as "Set the output file resolution in meters per pixel". My image resolution is about 1m, so I set it to "--mpp 1", but I see the suggested command you gave is "--mpp 0.01". Am I misunderstanding, or are you testing the feasibility of outputting a very large pixel image?
I used the 2022-04-17 version of ASP and successfully geoprojected the image using the mapproject tool. But there are still some small problems such as missing image on the right side of the image.
The command entered in the terminal is as follows：
/home/hipeson/StereoPipeline-3.0.1-alpha-2022-04-17-x86_64-Linux/bin/mapproject  -t rpc --mpp 1  COP30.tif  block-nad.tif  map-nad.tif --tif-compress  LZW --threads  20
The error message of the terminal is as follows：
0ERROR 4: ./map-nad_tif_tiles/tile_20480_0_25600_5120.tif' not recognized as a supported file format. Warning 1: Can't open ./map-nad_tif_tiles/tile_20480_0_25600_5120.tif. Skipping it ..ERROR 4: ./map-nad_tif_tiles/tile_15360_5120_20480_10240.tif' not recognized as a supported file format.
Warning 1: Can't open ./map-nad_tif_tiles/tile_15360_5120_20480_10240.tif. Skipping it
.10ERROR 4: ./map-nad_tif_tiles/tile_15360_10240_20480_15360.tif' not recognized as a supported file format. Warning 1: Can't open ./map-nad_tif_tiles/tile_15360_10240_20480_15360.tif. Skipping it ..ERROR 4: ./map-nad_tif_tiles/tile_15360_15360_20480_20480.tif' not recognized as a supported file format.
Warning 1: Can't open ./map-nad_tif_tiles/tile_15360_15360_20480_20480.tif. Skipping it
.20.ERROR 4: ./map-nad_tif_tiles/tile_15360_20480_20480_25600.tif' not recognized as a supported file format. Warning 1: Can't open ./map-nad_tif_tiles/tile_15360_20480_20480_25600.tif. Skipping it ..ERROR 4: ./map-nad_tif_tiles/tile_10240_25600_15360_30720.tif' not recognized as a supported file format.
Warning 1: Can't open ./map-nad_tif_tiles/tile_10240_25600_15360_30720.tif. Skipping it
30.ERROR 4: ./map-nad_tif_tiles/tile_10240_30720_15360_35840.tif' not recognized as a supported file format. Warning 1: Can't open ./map-nad_tif_tiles/tile_10240_30720_15360_35840.tif. Skipping it ..ERROR 4: ./map-nad_tif_tiles/tile_10240_35840_15360_40960.tif' not recognized as a supported file format.
Warning 1: Can't open ./map-nad_tif_tiles/tile_10240_35840_15360_40960.tif. Skipping it
40.ERROR 4: ./map-nad_tif_tiles/tile_5120_40960_10240_46080.tif' not recognized as a supported file format. Warning 1: Can't open ./map-nad_tif_tiles/tile_5120_40960_10240_46080.tif. Skipping it ERROR 4: ./map-nad_tif_tiles/tile_10240_40960_15360_46080.tif' not recognized as a supported file format.
Warning 1: Can't open ./map-nad_tif_tiles/tile_10240_40960_15360_46080.tif. Skipping it
..50ERROR 4: ./map-nad_tif_tiles/tile_5120_46080_10240_51200.tif' not recognized as a supported file format. Warning 1: Can't open ./map-nad_tif_tiles/tile_5120_46080_10240_51200.tif. Skipping it ..ERROR 4: ./map-nad_tif_tiles/tile_5120_51200_10240_56320.tif' not recognized as a supported file format.
Warning 1: Can't open ./map-nad_tif_tiles/tile_5120_51200_10240_56320.tif. Skipping it
.60ERROR 4: ./map-nad_tif_tiles/tile_5120_56320_10240_61440.tif' not recognized as a supported file format. Warning 1: Can't open ./map-nad_tif_tiles/tile_5120_56320_10240_61440.tif. Skipping it ..ERROR 4: ./map-nad_tif_tiles/tile_0_61440_5120_66560.tif' not recognized as a supported file format.
Warning 1: Can't open ./map-nad_tif_tiles/tile_0_61440_5120_66560.tif. Skipping it
ERROR 4: `./map-nad_tif_tiles/tile_5120_61440_10240_66560.tif' not recognized as a supported file format.
Warning 1: Can't open ./map-nad_tif_tiles/tile_5120_61440_10240_66560.tif. Skipping it
.70...80...90...100 - done.
gdal_translate -co compress=lzw -co bigtiff=yes -co TILED=yes -co INTERLEAVE=BAND -co BLOCKXSIZE=256 -co BLOCKYSIZE=256 ./map-nad_tif_tiles/mosaic.vrt map-nad.tif
Input file size is 181523, 91054
0...10...20...30...40...50...60...70...80...90...100 - done.
Removing: ./map-nad_tif_tiles/
Wrote: map-nad.tif
Finished in 3144.875788450241 seconds.

Yes, I set --mpp 0.01 for testing.
I don't know why you see pieces missing, and why there are tiling artifacts. It could be a filesystem thing on your system. You can try rerunning the problematic part with mapproject with the option --t_projwin. The bounds can be found as described here: https://stereopipeline.readthedocs.io/en/latest/tools/stereo_gui.html#finding-pixel-values-and-region-bounds.
If the small part works by itself,  it can be merged into the big maprpojected images with dem_mosaic. For the future, maybe you should use fewer processes on your machine, or maybe there are some storage issues, not sure. This is is a big dataset, so maybe your system has issues completing all of it.

The missing part of the projected image before may be related to the performance of the computer. When switching to another computer, a complete projected image can be obtained.
I used the image processed by mapproject to generate DEM, but the program was interrupted by an error at the "Stage 1 --> LOW-RESOLUTION CORRELATION" step.The main reason is that the search range is too large, which leads to the failure of low-resolution disparity map generation.
I also used the stereo_gui tool to select a small area for DEM generation, but the program still reported an error and interrupted because the search range was too large.
I checked the projected geolocation and both images are the same and correct. I also checked the ".match" file and the matching points generated are sufficient, correct and evenly distributed. Compared with the two unprojected images, the projected images are more similar, and the image distortion is further weakened.
I thought the search range would be reduced after projection, but the situation is not as expected. What are the reasons that may cause the search range of the projected image to become larger? Thank you so much!
The command used is as follows:
/home/hipeson/StereoPipeline-3.0.1-alpha-2022-04-17-x86_64-Linux/bin/parallel_stereo  map-fwd.tif  map-nad.tif
result/1  --threads-multiprocess 15  --threads-singleprocess 15
-t  rpcmaprpc  -s SGM-stereo.default     COP30.tif
The content of the default file is as follows:
ip-detect-method 1
ip-per-tile 200
individually-normalize
left-image-crop-win 53174  9455  9907  9979
right-image-crop-win<PHONE_NUMBER>7 31092 30369
cost-mode  4
stereo-algorithm 1
corr-kernel 7 7
corr-tile-size 2080
corr-memory-limit-mb  5000
The error message is as follows(use a small area):
[ 2022-Apr-22 16:52:32 ] : Stage 1 --> LOW-RESOLUTION CORRELATION
* Loading images for IP detection.
No IP file found, computing IP now.
* Detecting interest points.
--> Matching interest points using homography.
Looking for IP in left image...
Using 200 interest points per tile (1024^2 px).
Detecting IP
Removing IP near nodata with radius 4
Found interest points: 18869
Recording interest points to file: result/1-L.vwip
Looking for IP in right image...
Using 200 interest points per tile (1024^2 px).
Detecting IP
Removing IP near nodata with radius 4
Found interest points: 180288
Recording interest points to file:result/1-R.vwip
--> Adjusted uniqueness threshold: 0.8
Matching: [********************.] 99%
Matched points: 9354
Inlier threshold:                     200
RANSAC iterations:                    100
* Writing match file: result/1-L__R.match
--> Using interest points to determine search window.
* Loading match file: result /1-L__R.match
Removed 0 outliers based on percentiles of differences of interest points with --outlier-removal-params.
Refined search range using a histogram and --outlier-removal-params: (Origin: (438.509, 216.057) width: 19791.8 height: 19955.5)
Search width of 19791 is greater than desired limit of 4000, retrying with more aggressive IP filter.
Refined search range using a histogram and --outlier-removal-params: (Origin: (438.509, 216.057) width: 19791.8 height: 19955.5)
Search width of 19791 is greater than desired limit of 4000, retrying with more aggressive IP filter.
Refined search range using a histogram and --outlier-removal-params: (Origin: (438.509, 216.057) width: 19791.8 height: 19955.5)
Search width of 19791 is greater than desired limit of 4000, retrying with more aggressive IP filter.
Refined search range using a histogram and --outlier-removal-params: (Origin: (438.509, 216.057) width: 19791.8 height: 19955.5)
D_sub search range: (Origin: (-204.125, -228.125) width: 2471.25 height: 2491.25) px
Warning: Unable to compute valid search ranges for SGM input.
Warning: Unable to compute valid search ranges for SGM input.
Warning: Unable to compute valid search ranges for SGM input.
Writing: result /1-D_sub.tif
--> Low-resolution disparity:[] Complete!
Filtering outliers in D_sub based on --outlier-removal-params.
Empty disparity.
 Anything over 200 pixels is not good. Can you open your result/1-L.tif and result/1-R.tif in stereo_gui and put them on top of each other (from the menu, choose to view in single window).

I used "stereo_gui --single-window" to visualize the image, 1-L.tif in the red box and 1-R.tif in the blue box, the result is shown below. The two images do not overlap, which seems to be the reason for the excessive search range.

I use "gdalinfo" to view the information of 1-L.tif and 1-R.tif, the results are as follows：
gdalinfo 1-L.tif
Driver: GTiff/GeoTIFF
Files: 1-L.tif
Size is 9907, 9979
Corner Coordinates:
Upper Left  (  78.3686616,  30.2853299) ( 78d22' 7.18"E, 30d17' 7.19"N)
Lower Left  (  78.3686616,  30.0164012) ( 78d22' 7.18"E, 30d 0'59.04"N)
Upper Right (  78.6356499,  30.2853299) ( 78d38' 8.34"E, 30d17' 7.19"N)
Lower Right (  78.6356499,  30.0164012) ( 78d38' 8.34"E, 30d 0'59.04"N)
Center      (  78.5021557,  30.1508655) ( 78d30' 7.76"E, 30d 9' 3.12"N)
Band 1 Block=256x256 Type=Float32, ColorInterp=Gray
$ gdalinfo 1-R.tif
Driver: GTiff/GeoTIFF
Files: 1-R.tif
Size is 31092, 30369
Coordinate System is:
Corner Coordinates:
Upper Left  (  78.3647898,  30.2872702) ( 78d21'53.24"E, 30d17'14.17"N)
Lower Left  (  78.3647898,  30.0144609) ( 78d21'53.24"E, 30d 0'52.06"N)
Upper Right (  78.6440940,  30.2872702) ( 78d38'38.74"E, 30d17'14.17"N)
Lower Right (  78.6440940,  30.0144609) ( 78d38'38.74"E, 30d 0'52.06"N)
Center      (  78.5044419,  30.1508655) ( 78d30'15.99"E, 30d 9' 3.12"N)
Band 1 Block=256x256 Type=Float32, ColorInterp=Gray
NoData Value=-32768
I also use arcgis software to open 1-L.tif and 1-R.tif, the two images overlap very well, the color is 1-R.tif, and the black and white is 1-L.tif.

The geolocation of both images is correct, but the "stereo_gui --single-window" in ASP is wrong, I think the possible reason is the number of pixels. The size of 1-L.tif is 9907, 9979, and the size of 1-R.tif is 31092, 30369. The relationship between the two is about three times, which is consistent with the image displayed by stereo_gui.
When two images have different pixel counts, or different image resolutions, is it difficult for ASP to deal with it? I don't know if my guess is correct, please advise. Thank you very much！

Actually you did share before the text stereo printed on screen. My
apologies, I forgot.
I think the problem is the following, and I ran into it as well. Your two
images must be projected at the same resolution. So, the value of --tr to
be passed to the mapproject command should be the same. This is discussed
here:
https://stereopipeline.readthedocs.io/en/latest/next_steps.html#resolution-of-mapprojection
.
So you can delete your mapprojected images and try again. Since your images
are large, I will suggest mapprojecting first onto only a small area, using
the --t_projwin option of mapproject. How to get the window to pass to this
option is described here:
https://stereopipeline.readthedocs.io/en/latest/tools/stereo_gui.html#finding-pixel-values-and-region-bounds
.
I hope you can inspect in the gui your mapprojected images before running
stereo. They will hopefully have similar size and similar level of detail.
On Sat, Apr 23, 2022 at 8:17 AM Oleg Alexandrov @.***>
wrote:

This shows that image alignment failed. Can you share the text which the
program prints on the screen as it runs? It will likely suggest the issue.
And maybe you can respond to me only, not to the whole list. This is
becoming a little too detailed to make sense for a larger audience.
On Sat, Apr 23, 2022, 03:10 赵林博 @.***> wrote:

I used "stereo_gui --single-window" to visualize the image, 1-L.tif in
the red box and 1-R.tif in the blue box, the result is shown below. The two
images do not overlap, which seems to be the reason for the excessive
search range.
[image: 3]
https://user-images.githubusercontent.com/67795058/164890008-2018402d-a66a-4c50-8168-373acd6008b3.png
I use "gdalinfo" to view the information of 1-L.tif and 1-R.tif, the
results are as follows：
gdalinfo 1-L.tif
Driver: GTiff/GeoTIFF
Files: 1-L.tif
Size is 9907, 9979
Corner Coordinates:
Upper Left ( 78.3686616, 30.2853299) ( 78d22' 7.18"E, 30d17' 7.19"N)
Lower Left ( 78.3686616, 30.0164012) ( 78d22' 7.18"E, 30d 0'59.04"N)
Upper Right ( 78.6356499, 30.2853299) ( 78d38' 8.34"E, 30d17' 7.19"N)
Lower Right ( 78.6356499, 30.0164012) ( 78d38' 8.34"E, 30d 0'59.04"N)
Center ( 78.5021557, 30.1508655) ( 78d30' 7.76"E, 30d 9' 3.12"N)
Band 1 Block=256x256 Type=Float32, ColorInterp=Gray
$ gdalinfo 1-R.tif
Driver: GTiff/GeoTIFF
Files: 1-R.tif
Size is 31092, 30369
Coordinate System is:
Corner Coordinates:
Upper Left ( 78.3647898, 30.2872702) ( 78d21'53.24"E, 30d17'14.17"N)
Lower Left ( 78.3647898, 30.0144609) ( 78d21'53.24"E, 30d 0'52.06"N)
Upper Right ( 78.6440940, 30.2872702) ( 78d38'38.74"E, 30d17'14.17"N)
Lower Right ( 78.6440940, 30.0144609) ( 78d38'38.74"E, 30d 0'52.06"N)
Center ( 78.5044419, 30.1508655) ( 78d30'15.99"E, 30d 9' 3.12"N)
Band 1 Block=256x256 Type=Float32, ColorInterp=Gray
NoData Value=-32768
I also use arcgis software to open 1-L.tif and 1-R.tif, the two images
overlap very well, the color is 1-R.tif, and the black and white is 1-L.tif.
[image: arcgis]
https://user-images.githubusercontent.com/67795058/164890069-72ca769f-4cc0-47ac-9060-c738d9553670.jpg
The geolocation of both images is correct, but the "stereo_gui
--single-window" in ASP is wrong, I think the possible reason is the number
of pixels. The size of 1-L.tif is 9907, 9979, and the size of 1-R.tif is
31092, 30369. The relationship between the two is about three times, which
is consistent with the image displayed by stereo_gui.
When two images have different pixel counts, or different image
resolutions, is it difficult for ASP to deal with it? I don't know if my
guess is correct, please advise. Thank you very much！
—
Reply to this email directly, view it on GitHub
https://github.com/NeoGeographyToolkit/StereoPipeline/issues/363#issuecomment-1107446421,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKDU3HK35CTJPNLTB6AH5LVGPD7XANCNFSM5TT44QNQ
.
You are receiving this because you commented.Message ID:
@.***>

In fact, the two images I use have different resolutions, so when using "mapproject", I set the "--mpp" parameter differently.
I was reading the ASP documentation and found "If these two images have rather different auto-determined resolutions, it is suggested that the smaller one be used for both."
I think, if the lower resolution image is also projected to the higher resolution, will it affect the quality of the final DEM?
But for now, I can only do this experiment first. I will project both images and keep them at the same resolution. If there are new progress or problems in the follow-up, I will tell you here in time. Thanks a lot for your help.

Given that your two input images have different native resolutions, I am not fully sure what the optimal mapprojection resolution should be. You can try, in addition to using the highest resolution (that is, smallest ground sample distance), also something in between. For a fair comparison, the resulting point clouds should be made into DEMs using the same value for the --tr parameter rather than counting on it determining one automatically for each. Then the DEMs can be examined and see which has more detail and less artifacts.

I projected the two images to the same ground resolution and passed it into ASP to successfully generate a DEM. The integrity and accuracy of the DEM have achieved the expected results.
Then I use Stereo_GUI to open 1-L.tif and 1-R.tif. When I used "single-window", I found that the two images did not overlap with a large dislocation. But when I use "zoom all images to same region" first, and then use "single-window", I find that the two images overlap very well.
There may still be some hiccups, but now I get the correct results, which I think is a big improvement. The current success would not have been possible without your help, and I would like to express my sincerest thanks again for your continued help!

Your 1-L.tif and 1-R.tif represent the same ground, so when you "zoom all images to same region" they agree. Without that they don't agree because on the ground they have different extent, just like the input images, so one image may cover more terrain than another.
That is not a problem in stereo, as the constant shift among them gets compensated for. The big problem was the different resolution, as that resulted in a scale change and a big search range.
It can be convenient to cut both mapprojected images to the same domain using "gdal_translate -projwin" before doing stereo, and then likely the software will run a little faster and there will be no shift between L and R. But this is not a crucial thing.


Fixed browser page titles

https://bugzilla.redhat.com/show_bug.cgi?id=1201097

@romanblanco This looks good, but there is one place not calling the helper, that's in app/views/layouts/login.html.haml, line 7:
= "#{_('ManageIQ')}<EMAIL_ADDRESS>
Can you please change this to call the helper so the login screen will have the correct title as well?
Can just be:
= h(title_from_layout(@layout))

@dclarizio changed.

@romanblanco I don't see a change to the app/views/layouts/login.html.haml file . . . maybe it didn't get pushed?

@dclarizio My bad. Should be OK now.

Verified fix.


⚠ Bofrost.lu has degraded performance

In 0b5608d, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1696 ms

Resolved: Bofrost.lu performance has improved in 5ca0109 after 14 minutes.


error: This file was generated by an older version of protoc

Build fails for opencv-contrib-python-<IP_ADDRESS>:
In file included from /usr/ports/graphics/py-opencv-contrib-python/work-py39/opencv-contrib-python-<IP_ADDRESS>/opencv/modules/dnn/misc/tensorflow/attr_value.pb.cc:4:
/usr/ports/graphics/py-opencv-contrib-python/work-py39/opencv-contrib-python-<IP_ADDRESS>/opencv/modules/dnn/misc/tensorflow/attr_value.pb.h:17:2: error: This file was generated by an older version of protoc which is
#error This file was generated by an older version of protoc which is
 ^

All headers generated by protobuf should be regenerated during build. They can't be reused.

OpenCV uses own instance of Protobuf to handle ONNX, Caffe and some other DNN formats. Your error message means that you have yet another Protobuf instance in your build configuration. Please disable OpenCV DNN or use the same version everywhere.

All headers generated by protobuf should be regenerated during build. They can't be reused.

BUILD_PROTOBUF=OFF (to use external version) is almost useless without UPDATE_PROTO_FILES=ON option.
There are multiple similar questions in the upstream repository.

BUILD_PROTOBUF=OFF (to use external version) is almost useless without UPDATE_PROTO_FILES=ON option.

Maybe you should make this behavior default?

Adding -DBUILD_PROTOBUF=ON -DUPDATE_PROTO_FILES=ON to cmake arguments in setup.py doesn't fix the error This file was generated by an older version of protoc which is

-DBUILD_PROTOBUF=ON -DUPDATE_PROTO_FILES=ON

Wrong usage.
The first is the default value.
The second must be OFF in that case.
Don't touch these parameters if you don't want to use system's protobuf.
We have no idea that you are doing as we don't see any reproducer.

Dump compiler command line with all parameters.
Add -H option to show used headers.
Ensure that there is no mess with system's protobuf headers.

The flag was renamed. Current version uses -DPROTOBUF_UPDATE_FILES=ON. Added the case to FAQ on Wiki.


Make ERT PDAs and give them to the right presets.

About the PR

This PR gives the ERT the correct IDs and Job titles.

Why / Balance

I just didn't like that the CBURN title was "Central Commander" so I did something about it.
Technical details

Media

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Changelog

:cl:

tweak: ERT have received updated credentials.

this doesnt need a cl