Add concurrent read/write stress test

Here's a test that will reliably reproduce the deadlock in #45 for me.
I also bumped the timeout on the Extreme.execute call to make the deadlock a bit more apparent. I can revert that if you ever end up wanting to merge the test.
I'm curious if you're able to reproduce the deadlock as well.
With the timeout change and the default config of the EventStore, this test should result in the Extreme process stopping with reason :tcp_closed. This is the EventStore closing the connection due to a heartbeat timeout, presumably because Extreme did not respond to its heartbeat ping. If you then increase the EventStore heartbeat timeout, the test will produce GenServer timeouts on the Extreme.execute call.
# /etc/eventstore/eventstore.conf
ExtTcpHeartbeatTimeout: 30000
ExtTcpHeartbeatInterval: 60000

Hi @tonyvanriet,
Sorry for waiting. I did try your example and yes, it times out because of heartbeat. Problem is that read_events fetches 4096 events and it takes more time for Process to process them. Lower max_count to 128 for example and your test will pass!
Please let me know if that works for you

Thanks for checking on that, Milan.
It's true that this particular test doesn't fail with the smaller fetch size, but I'm not sure that helps to explain the failure. When the failure occurs, it's not a big slow read that happens to be long enough to trigger the EventStore heartbeat timeout. Both the read and write are completely locked up and there's zero activity for several seconds. If I increase the GenServer timeouts and EventStore heartbeat timeout, the deadlock will persist until the timeout occurs.
Also, when this occurs for us in production, there's very light traffic that would not fill up a 4096 event fetch.
As I mentioned, we've mitigated the issue by using multiple Extreme workers. We will have to get back to this issue and find root cause. When we do, I'll let you know what we find.
Thanks for all your help.

This pull request should fix this issue, #47

Fixed by #47


Not compatible with browsers of earlier versions

version：2.1.2
code:

error:
Error.stack: SyntaxError: Unexpected token ."

@rzdong upgrade v2.1.3

@jaywcjlove  Is there also a need to modify。: )
It might be handy to use babel

@rzdong upgrade v2.1.4

thanks a lot～～～～ : ）


/lib64/ld64.so.2: version GLIBC_2.22 /lib64/libc.so.6: version GLIBC_2.18'

Hello,
I would to check what are the requirements of rust (libc) and if the execution / installing are issue or just is requirement / limitation.
Please advise if there is some workaround as static compile glibc (as possible on x86_64 musl).
On Red-Hat 7.4 glibc 2.17 is used as in x86_64 (Red-Hat 7.3).
[root@powerpc64le tmp]# ldd ./nv
./nv: /lib64/ld64.so.2: version GLIBC_2.22' not found (required by ./nv) ./nv: /lib64/libc.so.6: versionGLIBC_2.18' not found (required by ./nv)
linux-vdso64.so.1 => (0x00007fffb0de0000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fffb0d20000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fffb0ce0000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fffb0ca0000)
libc.so.6 => /lib64/libc.so.6 (0x00007fffb0ab0000)
/lib64/ld64.so.2 (0x00007fffb0e00000)
[root@powerpc64le tmp]# ./nv
./nv: /lib64/ld64.so.2: version GLIBC_2.22' not found (required by ./nv) ./nv: /lib64/libc.so.6: versionGLIBC_2.18' not found (required by ./nv)
On x86_64:
[root@x86_64 ~]# ldd --version
ldd (GNU libc) 2.17
On powerpc64le:
[root@powerpc64le ~]# ldd --version
ldd (GNU libc) 2.17
[root@name ~]# curl https://sh.rustup.rs -sSf | sh
info: downloading installer
/tmp/tmp.pPXmBEil5N/rustup-init: /lib64/ld64.so.2: version GLIBC_2.22' not found (required by /tmp/tmp.pPXmBEil5N/rustup-init) /tmp/tmp.pPXmBEil5N/rustup-init: /lib64/libc.so.6: versionGLIBC_2.18' not found (required by /tmp/tmp.pPXmBEil5N/rustup-init)
docker pull ppc64le/centos;
docker create --name rust_centos -it ppc64le/centos;
docker start rust_centos;
docker exec -it rust_centos bash;
[root@c769e981bada /]# curl https://sh.rustup.rs -sSf | sh;

info: downloading installer
/tmp/tmp.awWf7OI1vO/rustup-init: /lib64/ld64.so.2: version GLIBC_2.22' not found (required by /tmp/tmp.awWf7OI1vO/rustup-init) /tmp/tmp.awWf7OI1vO/rustup-init: /lib64/libc.so.6: version GLIBC_2.18' not found (required by /tmp/tmp.awWf7OI1vO/rustup-init)

Thank you,
Yoni

The CI builders in upstream rust are careful to use older glibc and gcc for ABI compatibility.  Perhaps rustup could reuse the exact same docker containers to produce its binaries?

No, It failed. It's not possible to install rust there from the same reason.

That's not what I meant. The installer is one and the same; it comes from here, and has ABI issues. But the actual compiler binaries should be fine, as I used them to bootstrap our builds for RHEL. I'll check again tomorrow to make sure this hasn't regressed.
You might like to know that the EPEL repository has rust and cargo packages, which should work for both CentOS 7 and RHEL 7. Red Hat also has "rust-toolset-7" in SCL form.
Still, I do want to get the ABI fixed here in rustup, because it's handy for trying out different versions, testing with nightly, etc.

I think I hit this and am not sure how to proceed.
I am trying to build a binary in a docker container for amazonlinux using https://github.com/lambci/docker-lambda. AWS lambda apparently uses an old glibc. While I can build musl, it would be nice to build a smaller binary targeted specifically at amazonlinux as it could save money.

I think this should be fixed by https://github.com/rust-lang-nursery/rustup.rs/pull/1536 which uses CentOS 5 is as base system for i686 and amd64.

For those that are struggling with this issue, I created a gist to bootstrap rust using the binaries for RHEL/CentOS 7: https://gist.github.com/matthiasdiener/335fefd33be707c532a919a2971d6b86

@matthiasdiener Are you saying that the PR linked above still doesn't fix this for CentOS 7?

@kinnison that PR fixed it for i686 and amd64 as I said. Other architectures are still hit by this bug.

Yes, this is still an issue on RHEL7 ppc64le with the current Rustup version. I also tried all older Rustup versions that I could find, and they also do not run on this system.

I still think the right answer is to reuse rust-lang/rust's images in src/ci/docker.

Another option would be static linking to musl but I don't know how well it works for non x86 architectures.

OK, thank you for that report.  @alexcrichton does the suggestion of reusing the rust-lang/rust docker images make sense to you?

I think I tied that and hit another / same issue.

@yoni386 Why did you close this issue? This seems to be still a problem for ppc64le

Sorry it was by mistake.
I found out on different projects that rust as a new language is still
“weak” on all the compilation. On the positive side, when it do work it is
very easy but when you need different platform or fully static or something
more “custom” then you will probably hit a barrier and you will be stuck no
matter how much time you spent. On my future project I will definitely try
it again for now you can open it. Please remove me from tracking it.
On Sat, 2 Mar 2019 at 9:54 Matthias Diener<EMAIL_ADDRESS>wrote:

@yoni386 https://github.com/yoni386 Why did you close this issue? This
seems to be still a problem for ppc64le
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rust-lang/rustup.rs/issues/1339#issuecomment-468897697,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALrs2me9I4CYXF-OJIm-WZesEZbZub6Oks5vSi47gaJpZM4RlItv
.

--
Best Regards,
Yoni Shperling

@yoni386, only you can control whether you're subscribed to any particular issue, using the Notifications section in the web sidebar, or using the "mute" link in the email footer. Furthermore, only the issue submitter or someone with repo write access can open or close an issue.
I don't have such access, so instead I've opened a new issue: #1681.


Search result display

Describe the bug
the result of the search is display in the aside not in main windows
To Reproduce
Steps to reproduce the behavior:

Go to 'search and type a workd'
Click on 'seark'
And look the result

Desktop (please complete the following information):

OS: ubuntu 20.04 lts
Browser Chromium
Version 86

fixed by kevin


[No QA] VBA Requestor Step (UI Only)

Details
Implements the UI portion of the VBA Requestor Step.
Fixed Issues

Fixes (partially) https://github.com/Expensify/Expensify/issues/166919
Tests / QA Steps

In ReimbursementAccountPage.js, apply the following diff:

-  const currentStep = this.getStepToOpenFromRouteParams() || this.props.reimbursementAccount.achData.currentStep;
+  const currentStep = CONST.BANK_ACCOUNT.STEP.REQUESTOR;

Go to http://localhost:8080/bank-account/
Look at the Requestor Information form and confirm it looks all great 🌟

Tested On

[x] Web
[ ] Mobile Web
[ ] Desktop
[ ] iOS
[ ] Android

Screenshots

Web

Mobile Web

Desktop

iOS

Android

Looks like there is a NAB from @marcaaron, so @NikkiWines I'll leave the decision to you to address that or self merge the PR.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging in version: 1.0.70-1🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production in version: 1.0.73-3🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


1488: Reset back to 10

Ticket
Addresses 1488
Changes
Sets back to the default. The previous size was too large.
This did fix a minor issue where defaults were apparently ignored in production.
Context for reviewers
It doesn't address any of the substance of 1488, unfortunately.

Merging despite test failures because the test failures are either addressed on a different branch (Bundle audit) or will be addressed separately (anchore-scan).


Revise docs on randomness

This is my attempt to solve #5496 and start working on https://github.com/substrate-developer-hub/knowledgebase/issues/15
I've moved the docs to the places I think they should go, but I'm not confident I've gotten everything right, and would appreciate careful review. In particular I'm not sure whether this warning applies to Collective Flip, Babe, or both
	/// WARNING: Hashing the result of this function will remove any low-influence properties it has
	/// and mean that all bits of the resulting value are entirely manipulatable by the author of
	/// the parent block, who can determine the value of `parent_hash`.

Also requesting review from @burdges

paritytech-ci added the B2-breaksapi label

I think that's incorrect

@rphmeier can you review this please?
@JoshOrndorff can you please merge master?

I've merged master.
I've also removed the breaks-api tag because I think ci incorrectly added it. Please add it back if it should be there.
From my perspective, there is one remaining open question before this is ready for merge. Does the following warning apply to collective flip, Babe, or all implementors of the Randomness trait?
/// WARNING: Hashing the result of this function will remove any low-influence properties it has
/// and mean that all bits of the resulting value are entirely manipulatable by the author of
/// the parent block, who can determine the value of `parent_hash`.

We've no worries about hashing randomness from BABE, only randomness from the low influence collective flip scheme.  We should advise that randomness-collective-flip itself exists for low security situations, like tests.

Thanks for confirming my suspicion @burdges :) I agree that the extra addendum to the randomness-collective-flip docs would be good in this PR.

@rphmeier I've addressed your comments and merged master.


chore: Release (0.4.1)

:robot: I have created a release beep boop
0.4.1 (2024-08-13)
Bug Fixes

Put workflow dispatch and push events in a separate group (#139) (f5f9909)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

v0.4.1
:sunflower:


Add an extension-specific locator that will wrap the low-level locators.

(Note that most of this PR is new tests.)

[x] Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR).
[x] Title summarizes what is changing.
~[ ] Has a news entry file (remember to thank yourself!).~
[x] Appropriate comments and documentation strings in the code.
~[ ] Has sufficient logging.~
~[ ] Has telemetry for enhancements.~
[x] Unit tests & system/integration tests are added/updated.
~[ ] Test plan is updated as appropriate.~
~[ ] package-lock.json has been regenerated by running npm install (if dependencies have changed).~
~[ ] The wiki is updated with any design decisions/details.~

Codecov Report

Merging #13780 into master will increase coverage by 0.05%.
The diff coverage is 89.85%.

@@            Coverage Diff             @@
##           master   #13780      +/-   ##
==========================================
+ Coverage   59.89%   59.94%   +0.05%
==========================================
  Files         682      682
  Lines       37935    38002      +67
  Branches     5462     5473      +11
==========================================
+ Hits        22721    22781      +60
- Misses      14038    14041       +3
- Partials     1176     1180       +4

Impacted Files
Coverage Δ

...ent/pythonEnvironments/discovery/locators/index.ts
87.93% <89.70%> (+2.21%)
:arrow_up:

src/client/common/utils/async.ts
87.50% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 329292c...77fc9dc. Read the comment docs.

Some linting issues (pipeline):


Add service for retrieving current vehicle status

A service should be added that retrieves the current overview of the vehicle. The normal call/update retrieves the overview stored in the backend (which the vechicle updates regularly). This call would wake up the vechicle to get a current overview.

Added button instead, in 0631167


Pty and terminal

pty is irresponsive and terminal may not run  in my system

Please try again with 1.80 which is due out later today. If you still have the problem, provide more info.
/needsMoreInfo

Thanks for creating this issue! We figured it's missing some basic information or in some other way doesn't follow our issue reporting guidelines. Please take the time to review these and update the issue.
Happy Coding!

/duplicate https://github.com/microsoft/vscode/issues/130320


Feature/dev 382 update ethminer

@dsarfati @mkjenkins

Not sure if it matters, but when reported to the console Ethminer is hitting stderr and not stdout
Log sample,
----  being reported by ethminer

12:30:17.781 > stderr: cu 12:30:17 cuda-0   Generated DAG + Light in 9,832 ms. 7.77 GB
left.

12:30:17.959 > stderr:  i 12:30:17 <unknown> Job: 1da16c6d... eth-us-west1.nanopool.org [<IP_ADDRESS>:9999]

12:30:18.232 > stderr: cu 12:30:18 cuda-1   Generated DAG + Light in 10,283 ms. 4.77 GB left.

12:30:21.158 > stderr:  m 12:30:21 <unknown> 0:00 A0 363.02 Kh - cu0 185.12, cu1 177.90
12:30:26.159 > stderr:  m 12:30:26 <unknown> 0:00 A0 57.75 Mh - cu0 30.50, cu1 27.24

@thedigitaldesign Right now it doesn't matter which st* stream the logs come in from. If I remember correctly these have always been messed up in ethminer. There are tons of info messages that come on sterror and vice versa.


Creating tmp files and not clearing them

Hello! thanks for this library, but I found a bug(?) If I pass uri files that are already in the cache, this action creates a copy of the files and when I close the screen it does not delete these copies.... As a result every opening = new file. You need to either add a check that the uri refers to a file already, or add a cleanup of the files created

I have the same problem. Cache directory keeps growing when these temp file increases.


Please add a contributor

@aldeed can you please add someone as a contributor to this project as well as mongo-object and message-box? I would be open to being that contributor.
There are some bugs and PRs that have been waiting for 2 months and you don't seem to be active on these projects.

@aldeed if you don't add a contributor within the next week I'm going to fork all of these projects and release new versions that I maintain because I am actively using this for my company.

Hi all, I'm forking this package and publishing it under my own scope. The new package will be `@clayne/simple-schema'.
@aldeed, thanks for all the hard work on this package. I'm open to merging my fork back in if you decide to add me as a contributor to this project.

Closed via https://github.com/aldeed/node-simple-schema/issues/31. Please see https://github.com/clayne11/node-simple-schema.

@clayne11 Please send me an email at aldeed at gmail. I do want to add some contributors but gotta be careful and discuss some things first.


Android memory leak

FastImageViewManager. requestManager might cause memory leak.

already settled.
I found that using this plugin a lot will cause Caton.


Documentation update

Describe the bug
The current documentation (https://docs.spring.io/spring-cloud-sleuth/docs/current/reference/html/project-features.html#project-features) contains incorrect or out-of-date code samples.
Sample
@Configuration(proxyBeanMethods = false)
    class MyConfig {
    @Bean ZipkinRestTemplateCustomizer myCustomizer() {
        return new ZipkinRestTemplateCustomizer() {
            @Override
            void customize(RestTemplate restTemplate) {
                // customize the RestTemplate
            }
        };
    }
}

It seems that this method was deprecated in 2.2.0 and removed in the next versions. New method was added instead:
	default RestTemplate customizeTemplate(RestTemplate restTemplate) {
		return restTemplate;
	}

Are you willing to help us out by updating the docs?

Hi @marcingrzejszczak
I guess this is highly critical task that should be done only by project members

You're saying that updating the docs is a highly critical task that should be done by project members?


Contact-Form

I have to upload this Dev project in hackotberfest'22
it is contact form
tech stack :- PHP, HTML,JS,CSS
please assign this issue.

I have a project in contact form related.

Please check my contribution in this project .


handling internal server error

I'm unable to handle the internal server error. So how to avoid it.
Like this
SUCCESS: {
code = 500;
message = "There was an error processing your request. It has been logged (ID 37cfc9da1aa3df00).";
}

You seem to be having trouble properly communicating with whatever backend you're using. Asking such a question on StackOverflow, tagging Alamofire and providing some code showing how your request is being issue with likely lead to better results. If you do find a bug with Alamofire, please open a new issue and provide a code sample that shows the issue.
Asking Questions
We don't use GitHub as a support forum. For any usage questions that are not specific to the project itself, please ask on Stack Overflow instead. By doing so, you'll be more likely to quickly solve your problem, and you'll allow anyone else with the same question to find the answer. This also allows maintainers to focus on improving the project for others.


Code coverage

ok thx @astorije . I am gonna work back on the test suite soon. starting to be urgent.


Fix order /create endpoint:

response status changed from 500 to 400 if request body is invalid

Pull request is related to issue #220

Thanks! Please just update the backlog :-)


Repo view table columns for offline repos

Seems like the state columns are accounted for in the table header even though they are not relevant for offline repos.

I think they are filled after clicking "import zip"

Ah makes sense. Then an empty  is probably missing.


fix: dialog taking up the entire screen and preventing other buttons …

fix issue 18

Thanks @LeetaoGoooo for fixing the issue #18 this update will be published with the next version


if you hold a flask that is exactly the same material then shift is skipped

I just discovered this mechanic
putting it here for lack of knowing where to write this discovery:
if you hold a flask that is exactly the same material then shift is skipped
example:  polymorphine shift with polymorphine in your flask
then this shift will be skipped & the next shift will execute instead, taking your flask of polymorphine into account if  there's a flask in source or destination.
let's do that in this seed: 805956690
with NO previous shifts done,  polymorphine will be transformed to blood (shift number 2)

That's very interesting! Thanks for posting this.
I guess a good place for this would be in the the Noita Wiki page about fungal shifts (https://noita.fandom.com/wiki/Fungal_Reality_Shift?#:~:text=New Game Plus.-,Holding a flask,-If the player)

additional notes:
-if one of the materials is a "group" it won't skip. (since there is no way to match a group with a group AFAIK)
-if you use noita cheat tools then it will accurately tell you whether next shift will be skipped
and the wiki, well this information only becomes relevant if you use noita seed tool or something
otherwise if you don't, the only way to know that a shift was skipped is to count the number of shifts & realize you got less than 20
So what I'm getting at is that : I have no idea how to rewrite that idea specifically for the wiki.


Recreate font from scratch

[ ] Find the keyboard layout I made
[ ] Possibly use a Unicode Private Use Area

Inkscape custom font tutorial

For compatibility, I would need to make versions of the font in different formats (e.g. .ttf, .woff, .svg)
https://stackoverflow.com/questions/24990554/how-to-include-a-font-ttf-using-css


Mention parse options in the documentation

Schema method .parse() accepts an optional second argument params.
type ParseParams = {
    path: (string | number)[];
    errorMap: ZodErrorMap;
    async: boolean;
};

I've found it very useful, especially errorMap property.
The usage is described in https://github.com/colinhacks/zod/blob/main/ERROR_HANDLING.md.
Unfortunately, in the .parse() documentation the params is not mentioned, see https://zod.dev/?id=parse

I think this just hide a very useful feature.
Would be nice to add params to .parse() documentation, or at least mention it and link ERROR_HANDLING.md

Hm yeah, it's definitely an oversight to not include errorMap anywhere in the main README. I think I was hesitant to document error maps too visibly since they're a comparatively advanced feature and might feel intimidating to some users. I'll work on this.
The other two aren't really intended for direct use and should probably be hidden from the public API.

Thanks, errorMap is a super convenient feature.


[Microsoft.Automation/automationAccounts/jobSchedules]: Poor error message if resource name incorrectly formatted

Bicep version
Bicep CLI version 0.3.255
Describe the bug
I'm trying to create an automation account + runbook + schedules + link runbook and schedules. Every thing goes well but when I add automationAccounts/jobSchedules resource, I get a 404 error:

Here is the bicep file, that works perfect except when I add the jobSchedules resource:
https://raw.githubusercontent.com/joanteixi/az-bicep/master/start-stop-vm/automation.bicep
Thanks!

Moved to Azure types repo, renamed, and re-opened. We will follow up with the provider to improve the error message.


tap_action always toggles menu

config:
      - type: 'custom:radial-menu'
        entity: light.living_room
        style:
          left: 33%
          top: 27%
        tap_action:
          action: call-service
          service: light.toggle
          service_data:
            entity_id: light.living_room
        hold_action:
          action: toggle-menu
        items: ...

When trying to override the tap_action the menu still opens instead of just running the tap_action. The light toggles, but so does the menu. I'm trying to only have the menu open on hold. Am I missing something?

Setting default_dismiss to false "fixes" that problem, but that still seems a bit strange that it needs to be set to handle the initial tap_action

should be fixed in 1.4.5


Add BANP conformance tests for .Spec.Ingress and .Spec.Egress fields

This is same as done in https://github.com/kubernetes-sigs/network-policy-api/pull/99 but for BANP.
Since we don't have BANP Pass action, we don't need that for the tests here.
Sample output as run on OVN-K cluster:
   --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressSCTP (13.04s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressSCTP/Should_support_an_'allow-egress'_policy_for_SCTP_protocol;_ensure_rule_ordering_is_respected (0.24s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressSCTP/Should_support_an_'allow-egress'_policy_for_SCTP_protocol_at_the_specified_port (3.22s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressSCTP/Should_support_an_'deny-egress'_policy_for_SCTP_protocol;_ensure_rule_ordering_is_respected (6.26s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressSCTP/Should_support_a_'deny-egress'_policy_for_SCTP_protocol_at_the_specified_port (3.29s)
    --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressTCP (13.07s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressTCP/Should_support_an_'allow-egress'_policy_for_TCP_protocol;_ensure_rule_ordering_is_respected (0.23s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressTCP/Should_support_an_'allow-egress'_policy_for_TCP_protocol_at_the_specified_port (3.24s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressTCP/Should_support_an_'deny-egress'_policy_for_TCP_protocol;_ensure_rule_ordering_is_respected (6.32s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressTCP/Should_support_a_'deny-egress'_policy_for_TCP_protocol_at_the_specified_port (3.26s)
    --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressUDP (13.53s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressUDP/Should_support_an_'allow-egress'_policy_for_UDP_protocol;_ensure_rule_ordering_is_respected (0.43s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressUDP/Should_support_an_'allow-egress'_policy_for_UDP_protocol_at_the_specified_port (3.52s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressUDP/Should_support_an_'deny-egress'_policy_for_UDP_protocol;_ensure_rule_ordering_is_respected (6.26s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyEgressUDP/Should_support_a_'deny-egress'_policy_for_UDP_protocol_at_the_specified_port (3.30s)
    --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressSCTP (13.23s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressSCTP/Should_support_an_'allow-ingress'_policy_for_SCTP_protocol;_ensure_rule_ordering_is_respected (0.23s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressSCTP/Should_support_an_'allow-ingress'_policy_for_SCTP_protocol_at_the_specified_port (3.24s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressSCTP/Should_support_an_'deny-ingress'_policy_for_SCTP_protocol;_ensure_rule_ordering_is_respected (6.29s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressSCTP/Should_support_a_'deny-ingress'_policy_for_SCTP_protocol_at_the_specified_port (3.44s)
    --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressTCP (13.10s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressTCP/Should_support_an_'allow-ingress'_policy_for_TCP_protocol;_ensure_rule_ordering_is_respected (0.27s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressTCP/Should_support_an_'allow-ingress'_policy_for_TCP_protocol_at_the_specified_port (3.22s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressTCP/Should_support_an_'deny-ingress'_policy_for_TCP_protocol;_ensure_rule_ordering_is_respected (6.31s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressTCP/Should_support_a_'deny-ingress'_policy_for_TCP_protocol_at_the_specified_port (3.28s)
    --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressUDP (13.33s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressUDP/Should_support_an_'allow-ingress'_policy_for_UDP_protocol;_ensure_rule_ordering_is_respected (0.36s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressUDP/Should_support_an_'allow-ingress'_policy_for_UDP_protocol_at_the_specified_port (3.24s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressUDP/Should_support_an_'deny-ingress'_policy_for_UDP_protocol;_ensure_rule_ordering_is_respected (6.32s)
        --- PASS: TestConformance/BaselineAdminNetworkPolicyIngressUDP/Should_support_a_'deny-ingress'_policy_for_UDP_protocol_at_the_specified_port (3.38s)
PASS
ok  	sigs.k8s.io/network-policy-api/conformance	193.009s
?   	sigs.k8s.io/network-policy-api/conformance/tests	[no test files]
?   	sigs.k8s.io/network-policy-api/conformance/utils/config	[no test files]
?   	sigs.k8s.io/network-policy-api/conformance/utils/flags	[no test files]
=== RUN   TestPrepareResources
=== RUN   TestPrepareResources/empty_namespace_labels
=== RUN   TestPrepareResources/simple_namespace_labels
=== RUN   TestPrepareResources/overwrite_namespace_labels
--- PASS: TestPrepareResources (0.00s)
    --- PASS: TestPrepareResources/empty_namespace_labels (0.00s)
    --- PASS: TestPrepareResources/simple_namespace_labels (0.00s)
    --- PASS: TestPrepareResources/overwrite_namespace_labels (0.00s)
PASS

/lgtm
/approve

Thanks @tssurya!


dsVideoPort L3 test procedure documentation

Update the each L3 test procedure documentation with Acceptance Criteria, Expected Results, User Inputs

merged to develop


Source code change

Source code change to have EPANET2.2 routing method, include mass balance and OPENMP.

Source codes of MSX are updated.

EPANET2.2 topological routing method is adopted.
MASS balance check is added for RATE species
OPENMP is added to solve reactions parallelly.

Make files were not done. The build folder needs to be restructured.

Thanks @fengshang1972 - this is a very large PR. A few initial thoughts and questions.
It would be great to have a better sense of what might change in the build tooling - whether OpenMP is a hard requirement, how to enable/disable it as a compile- (or run-)time option. Also how you think the build folder needs to be restructured. I see there are platform subdirectories, so I imagine you are thinking to simplify that. I would suggest looking at CMake since that is a widely used standard build tool.
Is this code buildable as-is, and on what platforms?
Do you have a sense of the practical speedup of parallel execution, or benchmarks to share?
Looks like you added some solver structs - are these new/different solvers, or just making the existing ones easier to manage?
Can you link us to information on the new topo routing method? I know that's buried somewhere in the EPANET project threads, or maybe there's a good writeup in the EPANET docs...
And finally, I think there are some code readability issues (inheriting from the original codebase, like single-line if statements) that might benefit from adopting a code formatting convention. Formatting perhaps should be a separate issue though.
I'm excited to see what other people notice in the PR, and where the conversation will go!

Sam
As we have discussed, this PR is for development branch. I just wanted to reactivate the development of MSX.
Last night, Lew provided the make files for MSX, for WINDOWS. He is always so fast. Kind of the same approach for EPANET 2.2. I noticed Jim had a make file for Linux. We will try to see if that still work.
I tested for the chloramine example, 40% computational time is cut. But my computer is really low end. But OPENMP will not have a huge reduction. GPU is the future. But much more work is needed.
EPANET 2.2's WQ routing is different from 2.0. The routing is topological and mass balance is improved. The new MSX code adopted this approach and mass balance is reported as in EPANET 2.2.
The solver struct change is for better management and openmp. They are not new.
Best regards
Feng
From: Sam Hatchett<EMAIL_ADDRESS>Sent: Thursday, February 25, 2021 10:37 AM
To: OpenWaterAnalytics/epanet-msx<EMAIL_ADDRESS>Cc: Shang, Feng<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [OpenWaterAnalytics/epanet-msx] Source code change (#6)
Thanks @fengshang1972https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ffengshang1972&data=04|01|shang.feng%40epa.gov|08163769ff4a425850d808d8d9a344ce|88b378b367484867acf976aacbeca6a7|0|0|637498642548366713|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=YPO4u%2F6wcuQ2wM0bhLm5wREiktziNNCvqOnNMqUM67Y%3D&reserved=0 - this is a very large PR. A few initial thoughts and questions.
It would be great to have a better sense of what might change in the build tooling - whether OpenMP is a hard requirement, how to enable/disable it as a compile- (or run-)time option. Also how you think the build folder needs to be restructured. I see there are platform subdirectories, so I imagine you are thinking to simplify that. I would suggest looking at CMake since that is a widely used standard build tool.
Is this code buildable as-is, and on what platforms?
Do you have a sense of the practical speedup of parallel execution, or benchmarks to share?
Looks like you added some solver structs - are these new/different solvers, or just making the existing ones easier to manage?
Can you link us to information on the new topo routing method? I know that's buried somewhere in the EPANET project threads, or maybe there's a good writeup in the EPANET docs...
And finally, I think there are some code readability issues (inheriting from the original codebase, like single-line if statements) that might benefit from adopting a code formatting convention. Formatting perhaps should be a separate issue though.
I'm excited to see what other people notice in the PR, and where the conversation will go!

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOpenWaterAnalytics%2Fepanet-msx%2Fpull%2F6%23issuecomment-785991767&data=04|01|shang.feng%40epa.gov|08163769ff4a425850d808d8d9a344ce|88b378b367484867acf976aacbeca6a7|0|0|637498642548366713|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=UWpLFZ7tFDGVCa%2BePtbZE32QXvT1%2B2C3vdouv%2FsZfmk%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAMKPK5Q6VDUJO2IWTA5NIILTAZVAVANCNFSM4YDHZ5GA&data=04|01|shang.feng%40epa.gov|08163769ff4a425850d808d8d9a344ce|88b378b367484867acf976aacbeca6a7|0|0|637498642548376666|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=KtyhTLuCXnBJ1rG1Q2ANyURcdqWmxfewGXdIz3s8NhY%3D&reserved=0.

Hey Feng,
First off, a big thanks from me also for re-starting development of this code base. After 15 years, I project that it is just now starting to come into its prime and will be increasingly used.
I haven't looked at your PR. I gather from the discussion that the ingestion of it may be slower, just because of its size, and perhaps also because there wasn't extensive discussion about what was going to be done, within the issues -- I may be wrong about that; sorry in advance if I am. But if there's a lesson for the future, it's that collaborative programming is enabled by communication, facilitated through the repo issues tracking, and made smoother by smaller PRs done in chunks (wherever possible). I note that this is not how Lew has done it. But Lew is not doing collaborative code development. One has to choose what model to follow.
Having said that, I want to say again, thanks for re-starting development of this code base ;-).
Finally, as some smaller particular ideas,

I recall there being a couple of issues on MSX that would be considered bugs. I think they are on Github, but I'm lacking specifics and am too lazy to just check myself. Just be aware cause I think they were valid - for example I recall some issue where maybe the model wasn't be correctly parsed.

The 40% speedup is pretty pitiful, but maybe you're only using a couple of cores on your crappy machine? It would be cool to use GPUs, but before that is there a better way than OPENMP to access multiple cores on windows, linux, macos?  I know it would be machine specific solutions... but for something like MSX there is really such a huge impact of efficiency (hence your reason for doing this work), and I personally would think it might be valid to look more into ways to selectively compile for particular architecture... anyway I'm talking beyond the limits of my knowledge now so I'll stop.

Thanks!
-Jim
Jim Uber, Ph.D.
Director, Drinking Water Network Optimization
Xylem - Digital Solutions
<EMAIL_ADDRESS>M:<PHONE_NUMBER>

From: fengshang1972<EMAIL_ADDRESS>Sent: Thursday, February 25, 2021 11:05 AM
To: OpenWaterAnalytics/epanet-msx
Cc: Subscribed
Subject: Re: [OpenWaterAnalytics/epanet-msx] Source code change (#6)
Sam
As we have discussed, this PR is for development branch. I just wanted to reactivate the development of MSX.
Last night, Lew provided the make files for MSX, for WINDOWS. He is always so fast. Kind of the same approach for EPANET 2.2. I noticed Jim had a make file for Linux. We will try to see if that still work.
I tested for the chloramine example, 40% computational time is cut. But my computer is really low end. But OPENMP will not have a huge reduction. GPU is the future. But much more work is needed.
EPANET 2.2's WQ routing is different from 2.0. The routing is topological and mass balance is improved. The new MSX code adopted this approach and mass balance is reported as in EPANET 2.2.
The solver struct change is for better management and openmp. They are not new.
Best regards
Feng
From: Sam Hatchett<EMAIL_ADDRESS>Sent: Thursday, February 25, 2021 10:37 AM
To: OpenWaterAnalytics/epanet-msx<EMAIL_ADDRESS>Cc: Shang, Feng<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [OpenWaterAnalytics/epanet-msx] Source code change (#6)
Thanks @fengshang1972https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ffengshang1972&data=04|01|shang.feng%40epa.gov|08163769ff4a425850d808d8d9a344ce|88b378b367484867acf976aacbeca6a7|0|0|637498642548366713|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=YPO4u%2F6wcuQ2wM0bhLm5wREiktziNNCvqOnNMqUM67Y%3D&reserved=0 - this is a very large PR. A few initial thoughts and questions.
It would be great to have a better sense of what might change in the build tooling - whether OpenMP is a hard requirement, how to enable/disable it as a compile- (or run-)time option. Also how you think the build folder needs to be restructured. I see there are platform subdirectories, so I imagine you are thinking to simplify that. I would suggest looking at CMake since that is a widely used standard build tool.
Is this code buildable as-is, and on what platforms?
Do you have a sense of the practical speedup of parallel execution, or benchmarks to share?
Looks like you added some solver structs - are these new/different solvers, or just making the existing ones easier to manage?
Can you link us to information on the new topo routing method? I know that's buried somewhere in the EPANET project threads, or maybe there's a good writeup in the EPANET docs...
And finally, I think there are some code readability issues (inheriting from the original codebase, like single-line if statements) that might benefit from adopting a code formatting convention. Formatting perhaps should be a separate issue though.
I'm excited to see what other people notice in the PR, and where the conversation will go!

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOpenWaterAnalytics%2Fepanet-msx%2Fpull%2F6%23issuecomment-785991767&data=04|01|shang.feng%40epa.gov|08163769ff4a425850d808d8d9a344ce|88b378b367484867acf976aacbeca6a7|0|0|637498642548366713|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=UWpLFZ7tFDGVCa%2BePtbZE32QXvT1%2B2C3vdouv%2FsZfmk%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAMKPK5Q6VDUJO2IWTA5NIILTAZVAVANCNFSM4YDHZ5GA&data=04|01|shang.feng%40epa.gov|08163769ff4a425850d808d8d9a344ce|88b378b367484867acf976aacbeca6a7|0|0|637498642548376666|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=KtyhTLuCXnBJ1rG1Q2ANyURcdqWmxfewGXdIz3s8NhY%3D&reserved=0.

You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/OpenWaterAnalytics/epanet-msx/pull/6#issuecomment-786012451, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AASMACLJ2G42LBHKEJHA55LTAZYKZANCNFSM4YDHZ5GA.
CONFIDENTIALITY NOTICE: This e-mail, including any attachments and/or linked documents, is intended for the sole use of the intended addressee and may contain information that is privileged, confidential, proprietary, or otherwise protected by law. Any unauthorized review, dissemination, distribution, or copying is prohibited. If you have received this communication in error, please contact the original sender immediately by reply email and destroy all copies of the original message and any attachments. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Xylem Inc.

Hey @fengshang1972. Congratulation on your PR. Nice work! :smile:

@jamesuber
It is kind of hard for me to divide the implementation of topological routing into small steps. I have never done coding collaboration and it will take some time for me to figure the most efficient way. Maybe we have to strike some balance between the productivity and the rules. I am sure corporate people (like you, haha) have these kind of struggle.
Performance is big issue for sure. Lew's real time compiling improved the speed significantly. In terms of parallelization, I have little knowledge and basic OPENMP is all I know.
I think I run into some bugs in my previous job. Maybe they are the same as the ones you mentioned. Anyway, we surely will continuously fix the bugs.

@michaeltryby Soon we will need QA/QC for MSX.

@fengshang1972 happy to help out however I can.

?I understand Feng. Possibly the main takeaway may be less in form/volume of a PR, and more the form/volume of prior discussions related to the objectives of the PR.
I say this as someone who loves being by himself so much, that pandemic isolation has been a dream come true. It's hard. For all of us.
Jim Uber, Ph.D.
Director, Drinking Water Network Optimization
Xylem - Digital Solutions
<EMAIL_ADDRESS>M:<PHONE_NUMBER>

From: fengshang1972<EMAIL_ADDRESS>Sent: Thursday, February 25, 2021 12:26 PM
To: OpenWaterAnalytics/epanet-msx
Cc: Uber, James - Xylem; Mention
Subject: Re: [OpenWaterAnalytics/epanet-msx] Source code change (#6)
@jamesuberhttps://github.com/jamesuber
It is kind of hard for me to divide the implementation of topological routing into small steps. I have never done coding collaboration and it will take some time for me to figure the most efficient way. Maybe we have to strike some balance between the productivity and the rules. I am sure corporate people (like you, haha) have these kind of struggle.
Performance is big issue for sure. Lew's real time compiling improved the speed significantly. In terms of parallelization, I have little knowledge and basic OPENMP is all I know.
I think I run into some bugs in my previous job. Maybe they are the same as the ones you mentioned. Anyway, we surely will continuously fix the bugs.

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/OpenWaterAnalytics/epanet-msx/pull/6#issuecomment-786069872, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AASMACLIVN2YVVOAXBLHTZ3TA2B3FANCNFSM4YDHZ5GA.
CONFIDENTIALITY NOTICE: This e-mail, including any attachments and/or linked documents, is intended for the sole use of the intended addressee and may contain information that is privileged, confidential, proprietary, or otherwise protected by law. Any unauthorized review, dissemination, distribution, or copying is prohibited. If you have received this communication in error, please contact the original sender immediately by reply email and destroy all copies of the original message and any attachments. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Xylem Inc.

I just wanted to clarify some things mentioned in the previous comments:

In my view Feng's PR had a very clear and simple goal which was to update MSX to make it consistent with the updated WQ routine in EPANET 2.2 that improves mass conservation.
I supplied Feng with Cmake scripts, derived from those used for Epanet2.2, that should work for all platforms.
I used the scripts to successfully build an executable for Feng's code on Windows (VS-2017)
I tested the executable on the net3 msx files that are part of the current EPA distribution and they all ran successfully.
I can't see where OpenMP is included in Feng's update (I searched for a reference to "omp.h" without success). Maybe he used it in another test version of the code. Regardless, OpenMP seems to have it's speed-up peter out after about 6 or so cores are used, but it's better than nothing.
@jamesuber I guess you weren't paying attention to my (ongoing) collaborative efforts on OWA/EPANET.

I feel that Feng's PR should be accepted into dev, and we can start to build more project infrastructure from there (the Cmake scripts, automated testing, etc.).

@LRossman @jamesuber I agree with Lew's comment about multiprocessing having some finite speed-up potential. EPANET/MSX is going to run into communication or race conditions at some point because the problems just aren't broken up to really solve across processors. Molecular Dynamics software uses 2 timesteps to solve two different sized problems across processors, but I have never thought of a way to allow EPANET to do something similar (or if it would even produce useful results if we did). But If it offers the potential to get any speed up, it is probably worth thinking about (at the user's choosing).

@LRossman Never realized I did not inculde omp.h! Not sure how that can still work,
I turned the openmp flag on in VS project properties if I want openmp, and used #ifdef _OPENMP tin the codes.

?I'm not sure why these race conditions would occur? Maybe the code has been changed since the inception, but the previous algorithm boiled down to a step where you just had a bucket of separable initial value problems to integrate, one for each segment. I assume that's what the OpenMP is being used to do, but since I don't actually know how OpenMP works... Anyway, I'd think that a good parallel strategy to execute that loop should be able to use near to 100% of each thread. That's all that I had meant.
Jim Uber, Ph.D.
Director, Drinking Water Network Optimization
Xylem - Digital Solutions
<EMAIL_ADDRESS>M:<PHONE_NUMBER>

From: Jonathan Burkhardt<EMAIL_ADDRESS>Sent: Thursday, February 25, 2021 3:09 PM
To: OpenWaterAnalytics/epanet-msx
Cc: Uber, James - Xylem; Mention
Subject: Re: [OpenWaterAnalytics/epanet-msx] Source code change (#6)
@LRossmanhttps://github.com/LRossman @jamesuberhttps://github.com/jamesuber I agree with Lew's comment about multiprocessing having some finite speed-up potential. EPANET/MSX is going to run into communication or race conditions at some point because the problems just aren't broken up to really solve across processors. Molecular Dynamics software uses 2 timesteps to solve two different sized problems across processors, but I have never thought of a way to allow EPANET to do something similar (or if it would even produce useful results if we did). But If it offers the potential to get any speed up, it is probably worth thinking about (at the user's choosing).

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/OpenWaterAnalytics/epanet-msx/pull/6#issuecomment-786168854, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AASMACNKUOLUGAHTLFDO2O3TA2VATANCNFSM4YDHZ5GA.
CONFIDENTIALITY NOTICE: This e-mail, including any attachments and/or linked documents, is intended for the sole use of the intended addressee and may contain information that is privileged, confidential, proprietary, or otherwise protected by law. Any unauthorized review, dissemination, distribution, or copying is prohibited. If you have received this communication in error, please contact the original sender immediately by reply email and destroy all copies of the original message and any attachments. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Xylem Inc.

A Zoom call is probably better than this. Maybe the leadership should call one?
@LRossman no disrespect meant; I hope you know how highly I regard the contributions you make (and actually I read every one of your OWA messages).
Nobody here should feel bad or unappreciated, cause you all are making choices about your time, to make valuable contributions to the community. IMO code writing is single-threaded, and the collaborative part that's hard involves deciding what to do and roughly how to go about it. That's all I meant. ?
Jim Uber, Ph.D.
Director, Drinking Water Network Optimization
Xylem - Digital Solutions
<EMAIL_ADDRESS>M:<PHONE_NUMBER>

From: Lew Rossman<EMAIL_ADDRESS>Sent: Thursday, February 25, 2021 2:53 PM
To: OpenWaterAnalytics/epanet-msx
Cc: Uber, James - Xylem; Mention
Subject: Re: [OpenWaterAnalytics/epanet-msx] Source code change (#6)
I just wanted to clarify some things mentioned in the previous comments:

In my view Feng's PR had a very clear and simple goal which was to update MSX to make it consistent with the updated WQ routine in EPANET 2.2 that improves mass conservation.
I supplied Feng with Cmake scripts, derived from those used for Epanet2.2, that should work for all platforms.
I used the scripts to successfully build an executable for Feng's code on Windows (VS-2017)
I tested the executable on the net3 msx files that are part of the current EPA distribution and they all ran successfully.
I can't see where OpenMP is included in Feng's update (I searched for a reference to "omp.h" without success). Maybe he used it in another test version of the code. Regardless, OpenMP seems to have it's speed-up peter out after about 6 or so cores are used, but it's better than nothing.
@jamesuberhttps://github.com/jamesuber I guess you weren't paying attention to my (ongoing) collaborative efforts on OWA/EPANET.

I feel that Feng's PR should be accepted into dev, and we can start to build more project infrastructure from there (the Cmake scripts, automated testing, etc.).

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/OpenWaterAnalytics/epanet-msx/pull/6#issuecomment-786159866, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AASMACISDI5AM7TNNMZHBULTA2TFDANCNFSM4YDHZ5GA.
CONFIDENTIALITY NOTICE: This e-mail, including any attachments and/or linked documents, is intended for the sole use of the intended addressee and may contain information that is privileged, confidential, proprietary, or otherwise protected by law. Any unauthorized review, dissemination, distribution, or copying is prohibited. If you have received this communication in error, please contact the original sender immediately by reply email and destroy all copies of the original message and any attachments. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Xylem Inc.

@jamesuber Maybe race condition was the wrong term. I was just thinking that if you had two chemical species and solved for one on one core, and the other on a separate core, the slower of the two would always limit your solution time. Splitting the problem could still gain some speed up, but you're limited to some gain. If you split the network up, things get trickier because pipes that cross sub-network boundaries need to talk to each other. You have more potential gains in speed, because you could break the problem up more, but then you need to reconcile all the sub-networks at each step. Even that solution will still be limited by the worst performing subnetwork, as everything else has to wait for it to be solved.

@jamesuber We could solve hydraulics on one core, and solve WQ on a separate core (or cores) because hydraulics should always be faster than WQ, and thereby gain the speed-up by effectively eliminating hydraulics.

More clarification on how OpenMP is implemented in MSX. It's used when chemical reactions are being evaluated in each pipe, since all computed variables are local to a particular pipe. Thus if you had 100 pipes and four processors OpenMP by default assigns 25 pipes (in order) to each processor to update their WQ at each time step. The issue that Jonathan was alluding to is when the allocation of the work to each processor becomes unbalanced, like if the pipes assigned to processor 1 take much longer to compute their reactions than those assigned to processor 2, resulting in processor 2 sitting idle while the system waits for processor 1 to finish. OpenMP has some options to avoid this condition plus we might be able to implement some type of intelligent processor assignment in code. Note that we can't parallelize on WQ species since they don't behave independently.

Why not do the parallel processing at the segment level?  If you randomly assigned segments to processors, I'd think the thread work would be very well balanced. That's what I thought was happening, and so we'd be able to achieve near 100% utilization.
Jim Uber, Ph.D.
Director, Drinking Water Network Optimization
Xylem - Digital Solutions
<EMAIL_ADDRESS>M:<PHONE_NUMBER>

From: Lew Rossman<EMAIL_ADDRESS>Sent: Thursday, February 25, 2021 4:37 PM
To: OpenWaterAnalytics/epanet-msx
Cc: Uber, James - Xylem; Mention
Subject: Re: [OpenWaterAnalytics/epanet-msx] Source code change (#6)
More clarification on how OpenMP is implemented in MSX. It's used when chemical reactions are being evaluated in each pipe, since all computed variables are local to a particular pipe. Thus if you had 100 pipes and four processors OpenMP by default assigns 25 pipes (in order) to each processor to update their WQ at each time step. The issue that Jonathan was alluding to is when the allocation of the work to each processor becomes unbalanced, like if the pipes assigned to processor 1 take much longer to compute their reactions than those assigned to processor 2, resulting in processor 2 sitting idle while the system waits for processor 1 to finish. OpenMP has some options to avoid this condition plus we might be able to implement some type of intelligent processor assignment in code. Note that we can't parallelize on WQ species since they don't behave independently.

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/OpenWaterAnalytics/epanet-msx/pull/6#issuecomment-786244534, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AASMACM5HULTI62YI2KLE3LTA27JPANCNFSM4YDHZ5GA.
CONFIDENTIALITY NOTICE: This e-mail, including any attachments and/or linked documents, is intended for the sole use of the intended addressee and may contain information that is privileged, confidential, proprietary, or otherwise protected by law. Any unauthorized review, dissemination, distribution, or copying is prohibited. If you have received this communication in error, please contact the original sender immediately by reply email and destroy all copies of the original message and any attachments. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Xylem Inc.

Yes, that should help. But we'd have to retool the data structures used for segments. There's no big bucket of segments -- each pipe has its own linked list of segments which simplifies the transport step of species along the pipe (after reaction occurs). If we put all segments in one big array we'd have to find a way to access them in the correct order (along the length of their pipe), plus account for them being both added and removed, so that transport could be implemented efficiently.

Just catching up with this thread - hi everyone! Great to have such engagement.
There seems to be several things this PR is attempting to address.

updated routing (found the epanet PR: https://github.com/OpenWaterAnalytics/EPANET/pull/297)
struct-ifying / reorganizing solver code and basic formatting changes
adding OpenMP (did we determine that we are missing an include?)
calling updated epanet (2.2) api functions
anything else?

In a perfect world, each of these might have been dealt with in separate issues, and then separate PRs, as each item on the list is likely to generate opinions and possibly uncover great ideas. Dealing with this as a single PR/ single commit is tricky as we can see by the diverse multithreaded conversation above. This sort of thing catches people off-guard.
Take Jim's "bucket of segments" idea - a decent idea - which just shows that the ultimate parallel execution model that best serves the project is perhaps non-obvious. Accepting a PR that implements parallel execution into the community project without first discussing the approach with that community might tend to discourage these kinds of ideas from being shared, and prevent innovative solutions from being discovered in the first place.
To move this forward, I think the less efficient way would be to start the process over, break the concerns apart into separate discussions, and have those debates prior to merging anything. The (possibly) more efficient way is to bulk-import Feng's changes and debate what to keep and what to change after the fact - which might set a bad precedent for project governance and development flow. I truly hope that folks on this thread can understand the conundrum.
Those are my thoughts for now. I would really like to hear from you all as to the preferred way forward.

@jamesuber @LRossman @samhatchett
I did not call any functions defined in omp.h. So it is OK not including the header filer. The parallelization is done through #pragma omp parallel.
I put an example of openmp. For net3-bio.msx. Time reported in the report files are 33 seconds (with openmp) vs 70 seconds (without openmp). See the results at the end of the message.
I am a practical engineer, not a perfectionist. Having something that work and our peer researchers and engineers can start to test and comment is better than no activities for a decade. I am not a very good programmer and open to any criticisms and suggestions to modify and improve the codes. And I hope we can have more people get interested in the water quality modeling in water distribution system and start to contribute, from both research and software point of view. But I do not like getting stuck with some bureaucratic rules and unnecessary arguments. Commercial software can have endless versions and patches. Here we are just talking about a development branch. I find it kind of unbelievable. I would rather spend the time reading and writing research papers and developing codes. Anyway now I am a researcher, not a software engineer anymore. I do not rush for anything.
No intention to offend anybody. I see you as my mentors and friends.
Best regards
With OPENMP
Analysis begun Thu Feb 25 20:10:22 2021
Processing MSX input file net3-bio.msx
Page 1                                               EPANET-MSX 1.1

                 E P A N E T  -  M S X                     *

              Multi-Species Water Quality                  *

              Analysis for Pipe  Networks                  *

                      Version 1.1                          *

Two-Source Biofilm Model
Water Quality Mass Balance: TL (MG)
Initial Mass:       0.00000e+00
Mass Inflow:        1.57592e+08
Mass Outflow:       1.55226e+08
Mass Reacted:       0.00000e+00
Final Mass:         2.36600e+06
Mass Ratio:         1.00000
Water Quality Mass Balance: CL2 (MG)
Initial Mass:       0.00000e+00
Mass Inflow:        1.07712e+09
Mass Outflow:       6.34586e+08
Mass Reacted:      -4.28713e+08
Final Mass:         1.38196e+07
Mass Ratio:         1.00000
Water Quality Mass Balance: S (MG)
Initial Mass:       0.00000e+00
Mass Inflow:        4.53595e+08
Mass Outflow:       4.29667e+08
Mass Reacted:      -1.09884e+07
Final Mass:         1.29392e+07
Mass Ratio:         1.00000
Water Quality Mass Balance: Xb (UG)
Initial Mass:       0.00000e+00
Mass Inflow:        8.97599e+06
Mass Outflow:       6.77147e+07
Mass Reacted:       8.44616e+07
Final Mass:         2.57228e+07
Mass Ratio:         1.00000
Water Quality Mass Balance: Xa (UG)
Initial Mass:       0.00000e+00
Mass Inflow:        0.00000e+00
Mass Outflow:       0.00000e+00
Mass Reacted:       1.61812e+08
Final Mass:         1.61812e+08
Mass Ratio:         1.00000
Analysis ended Thu Feb 25 20:10:55 2021
Without OPENMP
Analysis begun Thu Feb 25 20:12:59 2021
Processing MSX input file net3-bio.msx
Page 1                                               EPANET-MSX 1.1

                 E P A N E T  -  M S X                     *

              Multi-Species Water Quality                  *

              Analysis for Pipe  Networks                  *

                      Version 1.1                          *

Two-Source Biofilm Model
Water Quality Mass Balance: TL (MG)
Initial Mass:       0.00000e+00
Mass Inflow:        1.57592e+08
Mass Outflow:       1.55226e+08
Mass Reacted:       0.00000e+00
Final Mass:         2.36600e+06
Mass Ratio:         1.00000
Water Quality Mass Balance: CL2 (MG)
Initial Mass:       0.00000e+00
Mass Inflow:        1.07712e+09
Mass Outflow:       6.34586e+08
Mass Reacted:      -4.28713e+08
Final Mass:         1.38196e+07
Mass Ratio:         1.00000
Water Quality Mass Balance: S (MG)
Initial Mass:       0.00000e+00
Mass Inflow:        4.53595e+08
Mass Outflow:       4.29667e+08
Mass Reacted:      -1.09884e+07
Final Mass:         1.29392e+07
Mass Ratio:         1.00000
Water Quality Mass Balance: Xb (UG)
Initial Mass:       0.00000e+00
Mass Inflow:        8.97599e+06
Mass Outflow:       6.77147e+07
Mass Reacted:       8.44616e+07
Final Mass:         2.57228e+07
Mass Ratio:         1.00000
Water Quality Mass Balance: Xa (UG)
Initial Mass:       0.00000e+00
Mass Inflow:        0.00000e+00
Mass Outflow:       0.00000e+00
Mass Reacted:       1.61812e+08
Final Mass:         1.61812e+08
Mass Ratio:         1.00000
Analysis ended Thu Feb 25 20:14:09 2021

@LRossman @jamesuber Just to clarify, I was just proposing some possible challenges and reasons why there might be a limited improvement in parallelization, not trying to say that was how the current implementation handled things. Sorry for any confusion that may have caused. Given Lew's response related to how it works, we'd get speed for each step for segment step, but still potentially suffer as those processes have to reconcile at the nodes across cores.
We can definitely test out some scenarios, with and without parallelization and against single species EPANET to better quantify speed-up. That would be worth the effort at some point if parallelization was pursued. As I said initially, any speed up might be worth it.  Testing on a variety of different sized networks, and different complexity problems might help identify where a bottleneck existed.
@samhatchett -- As to the general state of the code. I think 2 or 3 of your bullets listed were really the same thing of "just bringing the code up-to-date to function with 2.2". The second side of this discussion is that I think the people that may have actually looked at this codebase in the last 8 years are all currently on this chain. Any new person is going to have to dig pretty deep into this code to make sense of it with or without a logical version control history, and probably wouldn't be in a place to comment on its functionality in the near term anyways. For that reason, I think it makes sense to accept this version and work from there as a dev branch, making sure to address the issues that you highlighted initially before it ultimately gets merged into the main branch. As Feng/Lew discussed, this is a working branch with potential improvements in mass balance vs the original MSX. The community needed a version that worked with 2.2, and this version does that.

Hey Jonathan - On the parallel processing, I don't believe we were thinking you were trying to explain the current code or modifications; at least I was not, so no worries from my end on that.
I think the first approach (and likely what Feng did) was to focus only on parallelizing the solution of the ODE systems for all the segments. This was the main computational drawback of MSX, cause you're likely to have thousands of such segments that require integration during each WQ time step, and depending on the solver and the problem characteristics, that load can take some time. It was terrible I remember before Lew added the capability to dynamically compile and link the code for evaluating the ODEs, as otherwise those expressions needed to be interpreted by a stack processor.
Before we'd get too far we should obviously profile the code (with the compilation option) to confirm, but I'm guessing there are people here who have done that.
In the above scenario, the requirement to "reconcile at the nodes across cores" is outside of the segment parallel processing - it comes after and is done serially (it could be parallelized...). So really, the question is "how fast can you perform N integrations of identical ODE systems, with different initial conditions, across M processors?" (A detail is that the tank ODEs are different.) That will depend on the variation in computational characteristics of the ODE systems, but since the only difference is initial conditions and this is a physical system, there might not be that much change across the segments. And if there were, a randomization of segment assignment to processors and the law of large numbers, might help smooth things out.
Thanks for the engagement. -Jim
Jim Uber, Ph.D.
Director, Drinking Water Network Optimization
Xylem - Digital Solutions
<EMAIL_ADDRESS>M:<PHONE_NUMBER>

From: Jonathan Burkhardt<EMAIL_ADDRESS>Sent: Friday, February 26, 2021 8:25 AM
To: OpenWaterAnalytics/epanet-msx
Cc: Uber, James - Xylem; Mention
Subject: Re: [OpenWaterAnalytics/epanet-msx] Source code change (#6)
@LRossmanhttps://github.com/LRossman @jamesuberhttps://github.com/jamesuber Just to clarify, I was just proposing some possible challenges and reasons why there might be a limited improvement in parallelization, not trying to say that was how the current implementation handled things. Sorry for any confusion that may have caused. Given Lew's response related to how it works, we'd get speed for each step for segment step, but still potentially suffer as those processes have to reconcile at the nodes across cores.
We can definitely test out some scenarios, with and without parallelization and against single species EPANET to better quantify speed-up. That would be worth the effort at some point if parallelization was pursued. As I said initially, any speed up might be worth it. Testing on a variety of different sized networks, and different complexity problems might help identify where a bottleneck existed.
@samhatchetthttps://github.com/samhatchett -- As to the general state of the code. I think 2 or 3 of your bullets listed were really the same thing of "just bringing the code up-to-date to function with 2.2". The second side of this discussion is that I think the people that may have actually looked at this codebase in the last 8 years are all currently on this chain. Any new person is going to have to dig pretty deep into this code to make sense of it with or without a logical version control history, and probably wouldn't be in a place to comment on its functionality in the near term anyways. For that reason, I think it makes sense to accept this version and work from there as a dev branch, making sure to address the issues that you highlighted initially before it ultimately gets merged into the main branch. As Feng/Lew discussed, this is a working branch with potential improvements in mass balance vs the original MSX. The community needed a version that worked with 2.2, and this version does that.

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/OpenWaterAnalytics/epanet-msx/pull/6#issuecomment-786646201, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AASMACJAWWLPKAYWIQJEKVDTA6OLZANCNFSM4YDHZ5GA.
CONFIDENTIALITY NOTICE: This e-mail, including any attachments and/or linked documents, is intended for the sole use of the intended addressee and may contain information that is privileged, confidential, proprietary, or otherwise protected by law. Any unauthorized review, dissemination, distribution, or copying is prohibited. If you have received this communication in error, please contact the original sender immediately by reply email and destroy all copies of the original message and any attachments. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Xylem Inc.

Jim - Just wanted to clarify, but I'm glad we're on the same page regarding where I was coming from.
I appreciate the insight into the ODE aspect of this, and I think we're thinking the same thing from two different sides. As you say, some level of profiling is likely in order if we want to invest effort seeking additional performance gains. My guess is that the serialized step of reconciling updates after segments are calculated will be the major bottleneck that might not have an easy solution. We'd need to ensure that two links weren't trying to add mass to a node at the exact same time if trying to do that step in parallel. There may be parallel programming approaches to do that that I'm not aware of, but that type of problem would cause some issues that might not manifest the same each time or at different parallelization levels.
That said, I'm just putting those thoughts out there for a longer term discussion about performance improvements, not necessarily for this specific code.

?OK Jonathan - I'll just say though, that I do not think the node mass balance step is going to be the computationally demanding part. It's already done serially after the ODE updates, and I doubt it is a bottleneck in terms of CPU usage.
Jim Uber, Ph.D.
Director, Drinking Water Network Optimization
Xylem - Digital Solutions
<EMAIL_ADDRESS>M:<PHONE_NUMBER>

From: Jonathan Burkhardt<EMAIL_ADDRESS>Sent: Friday, February 26, 2021 9:05 AM
To: OpenWaterAnalytics/epanet-msx
Cc: Uber, James - Xylem; Mention
Subject: Re: [OpenWaterAnalytics/epanet-msx] Source code change (#6)
Jim - Just wanted to clarify, but I'm glad we're on the same page regarding where I was coming from.
I appreciate the insight into the ODE aspect of this, and I think we're thinking the same thing from two different sides. As you say, some level of profiling is likely in order if we want to invest effort seeking additional performance gains. My guess is that the serialized step of reconciling updates after segments are calculated will be the major bottleneck that might not have an easy solution. We'd need to ensure that two links weren't trying to add mass to a node at the exact same time if trying to do that step in parallel. There may be parallel programming approaches to do that that I'm not aware of, but that type of problem would cause some issues that might not manifest the same each time or at different parallelization levels.
That said, I'm just putting those thoughts out there for a longer term discussion about performance improvements, not necessarily for this specific code.

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/OpenWaterAnalytics/epanet-msx/pull/6#issuecomment-786666962, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AASMACPV4FKABCRLMMAYWVTTA6TBXANCNFSM4YDHZ5GA.
CONFIDENTIALITY NOTICE: This e-mail, including any attachments and/or linked documents, is intended for the sole use of the intended addressee and may contain information that is privileged, confidential, proprietary, or otherwise protected by law. Any unauthorized review, dissemination, distribution, or copying is prohibited. If you have received this communication in error, please contact the original sender immediately by reply email and destroy all copies of the original message and any attachments. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of Xylem Inc.

@fengshang1972 - would you comment on why this was closed, and what you intend by this action? thanks!

MSX update is in EPA's STRAP research plan and we have deadlines to meet. At this stage, I think I should focus on the task and deliver what we have promised in the research plan. We will ask for the community for beta testing and feedback.
From: Sam Hatchett<EMAIL_ADDRESS>Sent: Monday, March 1, 2021 1:58 PM
To: OpenWaterAnalytics/epanet-msx<EMAIL_ADDRESS>Cc: Shang, Feng<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [OpenWaterAnalytics/epanet-msx] Source code change (#6)
@fengshang1972https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ffengshang1972&data=04|01|shang.feng%40epa.gov|1e6b3a2397a9448414fa08d8dce3e718|88b378b367484867acf976aacbeca6a7|0|0|637502218706946418|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=%2FC7d8l8SHOOxyvInsXA3WHLzaT1R9a2%2FOkbzSFjuEU0%3D&reserved=0 - would you comment on why this was closed, and what you intend by this action? thanks!

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOpenWaterAnalytics%2Fepanet-msx%2Fpull%2F6%23issuecomment-788188647&data=04|01|shang.feng%40epa.gov|1e6b3a2397a9448414fa08d8dce3e718|88b378b367484867acf976aacbeca6a7|0|0|637502218706946418|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=4YjPcoLrwNqggWdCAJuBs%2Bq7jS1fps6vTdngqM05H58%3D&reserved=0, or unsubscribehttps://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAMKPK5TG75ZYSJ6S5VDV5P3TBPPSRANCNFSM4YDHZ5GA&data=04|01|shang.feng%40epa.gov|1e6b3a2397a9448414fa08d8dce3e718|88b378b367484867acf976aacbeca6a7|0|0|637502218706956389|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=uowyCRS35PXZp98LkFPOVxT7IdLB5YxT0%2BdVQNfZb8A%3D&reserved=0.

Revisiting this PR, I can see that we have several good arguments for including the code as-is, into a development branch, for further review. Thanks all for your input!
I am a bit unclear as to the USEPA's involvement going forward. Almost get the impression that we are officially forked from here on out, but I'm open to other interpretations of the comments above. I think the community would welcome participation by all parties who wish to discuss the future of this codebase and I want to echo @jamesuber 's comments about collegiality and understanding that OSS development is a team sport. As such, we do need to understand that there will be many different opinions about development and we need a safe space to hammer out the details and find a path forward.


High-DPI not working

I'm trying to use the high-dpi mode of Squeak 6.0 but whenever I open or move SqueakJS on a high-DPI screen, the VM seems to scale up the entire canvas instead of informing the image about the new scale factor via primitiveScreenScaleFactor (which always fails right now).
Shouldn't pixelation already work? I tried something like
https://9090-codefrau-squeakjs-39cuvd00s49.ws-eu53.gitpod.io/run/#pixelated=true&image=/FreshTrunk.image
https://9090-codefrau-squeakjs-39cuvd00s49.ws-eu53.gitpod.io/run/#pixelated=false&image=/FreshTrunk.image

But there was no visible difference.

SqueakJS currently ignores window.devicePixelRatio which basically tells us how dense pixels actually are on the screen.
If we knew that the image supports highDPI settings, then we would scale the canvas by devicePixelRatio and I guess primitiveScreenScaleFactor should report that value.
So ... is there a way to tell if the image supports high DPI? Or is the call to that primitive the only indication? We can make that work, too.

Good question. Afaik there is no other information than whether the image is calling the primitive - but since there is no guarantee that it will use the result, this sounds also imperfect to me.
In the OSVM, newer versions of the VM just assume that the image will care about the DPI factor. But this has drawbacks regarding the display of older images, of course.
Would it be reasonable to use the image format for this decision?

Are there images that call the primitive but don't respect the result? This would sound like an image bug to me that maybe we can ignore in the VM?
All that would happen in that case is that everything looks tiny so you would have to manually adjust the preferences I think.

In the OSVM, newer versions of the VM just assume that the image will care about the DPI factor. But this has drawbacks regarding the display of older images, of course.

Not quite. We assume that the platform offers scaling if needed. Only macOS 12 dropped that support rather recently. Windows 11 is still fine. OSVM's Linux/X11 support never scaled and so didn't the Linux platform itself.
There are more than one attempts from me to explain that on vm-dev and/or squeak-dev during the last 2-3 months.

Would it be reasonable to use the image format for this decision?

Most recently, Eliot introduced an extra bit in the image header that we can set to indicate whether the VM supports high resolutions or not. However, it is still a lot of effort to support VM-based upscaling in case the platform (here: macOS) does not. Especially if we want to make it per-monitor aware. -.-"
When we find a good design to implement VM-based upscaling on demand for all desired platforms (macOS openGL/metal/codeGraphics, Windows DirectX/OpenGL, Linux X11/framebuffer/...), we might also re-release Squeak 6.0 with that flag set in the header. For now, we do not.

Are there images that call the primitive but don't respect the result? This would sound like an image bug to me that maybe we can ignore in the VM?

See above. I was really surprised by Apple's move here. It's puts even more pressure on our small community. :-( Older images do not try to call that primitive. Only Squeak 6.0 and 6.1alpha do that.
Squeak 5.3 is now also bundled with the latest OSVM release and scales up fine on Windows 11 because there it is a matter of what is in the .manifest file. Windows takes care of the scaling. macOS 12 just ignores its known flags from the Info.plist regarding upscaling. But they now have a "notch compatible full screen mode" ... wohoo ...

Yes, the image format changed in Squeak 6.0 because we enabled SistaV1. You might want to use this for now as a hack that indocates high-DPI awareness.

I just tried out-of-the-box Squeak 6.0 on a Mac screen that has PlatformScaleFactor = 2.0 and when I use the menu to change the scale from the default 200% to 100% the host window's size shrinks by half, making every pixel smaller. Is that the intended behavior?
I find that confusing. Can you describe how a VM should behave that cannot change the visual size of the host window, but could support "small" or "big" pixels on a high-dpi screen?
Here's what I think SqueakJS could do:

add a "small pixel / high-dpi mode" flag meaning the display bitmap the VM creates would use devicePixelRatio times as many pixels for the given window size, disabled by default
add primitiveScreenScaleFactor to answer either 1.0 (if in big-pixel-mode) or devicePixelRatio (in small-pixel-mode)
if the image is a 6.0 image (as indicated by the Sista format bit), enable small-pixel-mode by default
optionally, we could add a VM option to "force" small-pixel-mode for older images, e.g. by adding highdpi=true to the URL

How does that sound?

Is that the intended behavior?
I find that confusing.

Yes, that behavior mimicks the way platforms treat multi-monitor setups when moving an application window from, for example, a low-resolution monitor to a high-resolution monitor. The effective screen real estate should remain the same. That's why one also changes the host-window size.
If you refer to "100% should mean what the screen thinks is okay", there is a "Show relative scale factor" preference for that. Yet, comparing it to Windows and Linux, it is a rather macOS-specific way of communicating these things to the user.

Can you describe how a VM should behave that cannot change the visual size of the host window, but could support "small" or "big" pixels on a high-dpi screen?

Well, that's how Squeak's regular full-screen mode works. No change of host-window size, meaning still "full screen". Only the size of pixels change.

How does that sound?

Splendid. 😄

Okay, so commit c6288043b59db529901c9aa8e0b0bf89ac1298d7 adds a highdpi mode via url option and 54a8539549f58afbdec326093f4f6f3c550db577 adds primitiveScreenScaleFactor.
Appears to work, see e.g. https://squeak.js.org/run/#url=https://freudenbergs.de/vanessa/squeakjs&zip=[Squeak5.3-19452-64bit.zip,SqueakV50.sources.zip]&wizard=false&highdpi=true (note highdpi option in url)
I'm somewhat hesitant now to make this the default, even for 6.0 images, because with 4x the number of pixels to draw everything does become slower. What do we think?

Agreed. I think it's enough to make it easy to do. SqueakJS still has a speed perception  issue.


nodejs-react devfile based components fails to start when pushed to cluster

/kind bug

Which area this bug is related to?

/area registry

What versions of software are you using?
Operating System: Linux
Go Pkg Version: n/a
Bug Summary
Describe the bug:

Components created using registry devfile and starter project should be pushed without error and modification in source code.
nodejs react component is not starting after pushing it to the cluster.
To Reproduce:

Create component using nodejs react devfile
Push it to cluster
Open route in browser

url is not accessible
Expected behavior

should open starter project index page.
Any logs, error output, screenshots etc? Provide the devfile that sees this bug, if applicable.

Last 20 lines of the component's log:
time="2022-07-14T01:15:28Z" level=info msg="try to start program" program=devrun
time="2022-07-14T01:15:28Z" level=info msg="success to start program" program=devrun
ODO_COMMAND_RUN is npm run start
Changing directory to ${PROJECT_SOURCE}
Executing command cd ${PROJECT_SOURCE} && npm run start
npm ERR! Missing script: "start"
npm ERR!
npm ERR! Did you mean one of these?
npm ERR!     npm star # Mark your favorite packages
npm ERR!     npm stars # View packages marked as favorites
npm ERR!
npm ERR! To see a list of scripts, run:
npm ERR!   npm run

npm ERR! A complete log of this run can be found in:
npm ERR!     /opt/app-root/src/.npm/_logs/2022-07-14T01_15_28_333Z-debug-0.log
time="2022-07-14T01:15:29Z" level=debug msg="wait program exit" program=devrun
time="2022-07-14T01:15:29Z" level=error msg="program stopped with error:exit status 1" program=devrun
time="2022-07-14T01:15:29Z" level=info msg="Don't start the stopped program because its retry times 0 is greater than start retries 0" program=devrun
To get the full log output, please run 'odo log'

Pushing devfile component "nodejs-react-starter"

Additional context

Any workaround?

Rename 'dev' script to 'start' in package.json, or change start command in devfile to node start dev
Suggestion on how to fix the bug

Based on the error, it looks like this problem should be addressed by https://github.com/devfile/registry/pull/117

@dgolovin This was caused from me updating the react starter project so I could pass the tests on the registry. To my knowledge,, I don't think there is anyway around this because our tests in the registry require an up to date starter project which creates a temporary mismatch between the starter project and the devfile.

@schultzp2020 that sounds like right conclusion. @elsony just pointed out in chat that introduction of multiple stack versions support should address that, "but devfiles in the registry has not take advantage on that yet".


[PROPOSAL] - Application Creator System

Describe Your Proposal
While in flux, proposal in HackMD
Discord: https://discord.com/channels/937858703112155166/1184979120409559131/1191823150120452267
When settled there, this issue will be filled out here.

Desired outcome
We design around desired outcomes. What benefit to the TBD community to you expect to see if this proposal is fulfilled?
Current Gaps
We can always improve. What needs aren't being met which this proposal looks to address?
Implementation Suggestions (optional)
Are there strategies you've found useful to employ? Ideas for us to vet first? Sometimes a more comprehensive analysis is helpful, and you can link to a Forum discussion on this below.
Forum Discussion (optional)
Some proposals benefit from community discussion before work digs in. If so, link that thread here!
Discord Discussion (optional)
Maybe your idea stemmed from a discussion on Discord - if so, link here!

Meeting this week on Examples Repo Structure: https://discord.com/channels/937858703112155166/1184979120409559131/1201640721493524581


MobileStyleGAN Checkpoint converted to ONNX generates grey images

Hi!
Thank you for an amazing repository.
I successfully converted my StyleGAN2-ada rosinality checkpoint, by running the following line:
python convert_rosinality_ckpt.py --ckpt {path_to_rosinality_stylegan2_ckpt} --ckpt-mnet output/mnet.ckpt --ckpt-snet output/snet.ckpt --cfg-path output/config.json
I tested the checkpoint with demo.py and it produces images as expected.
I then converted it to ONNX by running
python train.py --cfg output/config.json  --export-model onnx --export-dir onnx-2
and tried to use the converted checkpoint in MobileStyleGAN web demo (https://github.com/cyrildiagne/mobilestylegan-web-demo).
It produces uniform grey images for all seeds.  The web demo works fine with the authors' ffhq checkpoint so it seems to be an issue with the converted model.
Do you have any thoughts on what might be causing this?

it seems the onnx has problem with fusedleakyrelu -
related - https://github.com/rosinality/stylegan2-pytorch/issues/322
am using this same function in own project -
https://github.com/bes-dev/MobileStyleGAN.pytorch/blob/a9776ff8f05a868b2d3b637bda14eca4c074d2a3/core/models/modules/ops/fused_act.py#L22
i have this class that spits out onnx -
the opsets are very important as lower versions dont have the support
https://github.com/johndpope/IMF/blob/main/onnxconv.py#L153
but this didn't resolve things for me yet.
still digging....
wait a second maybe this recent upstream fix on pytorch solves things -
https://github.com/pytorch/pytorch/issues/125753
need to test
pip install onnxscript -U


UI - Button does not keep his position, structure nested in different scenes [.NET]

Tested versions
v4.2.1.stable.mono.official
System information
Windows 10 - Godot Engine v4.2.1.stable.mono.official.b09f793f5 - Vulkan API 1.3.260 - Forward+ - Using Vulkan Device #0: NVIDIA - NVIDIA GeForce RTX 3060
Issue description
The following scenario:

Picture 1 Structure of the scene "Startmenu" // Is a container node
Startmenu still contains a scene StartmenuIcon
Startmenu is linked to the Desktop scene.

Picture 1 - Setup in desktop scene:

Picture 2 - The white box is the button in the top left-hand corner. Is in project execute mode.

Picture 3: - In the StartmenuIcon scene, the position is in the correct place.

Problem
The button in StartmenuIcon always jumps to the upper left corner only in the desktop scene. The position setting is not correctly adopted by the main scene.
Or is this due to the structure of the scenes?
I have already tried 10 times to adjust the position data in each scene. But the button always goes to Pos 0, 0. The position of the button is reset to Pos 0, 0 in the desktop scene after project execute mode.
Steps to reproduce
Please see pictures
Minimal reproduction project (MRP)
TestProjectUI.zip

Please add a minimum reproduction project, this is very specific and is hard to test without that as very little of the actual details are available, such as properties on the different nodes

A small Godot project which reproduces the issue, with no unnecessary files included. Be sure to not include the .godot folder in the archive (but keep project.godot).
Having an MRP is very important for contributors to be able to reproduce the bug in the same way that you are experiencing it. When testing a potential fix for the issue, contributors will use the MRP to validate that the fix is working as intended.
If the reproduction steps are not project dependent (e.g. the bug is visible in a brand new project), you can write "N/A" in the field.
Drag and drop a ZIP archive to upload it (max 10 MB). Do not select another field until the project is done uploading.
Note for C# users: If your issue is not C#-specific, please upload a minimal reproduction project written in GDScript. This will make it easier for contributors to reproduce the issue locally as not everyone has a .NET setup available.

@AThousandShips Okay thank you. What is MRP and how do I get that?

Read the instructions right there :)

@AThousandShips I have now added a test project with the UI.
To be honest, the structure is very strange (because we are still beginners). But we will definitely design the structure of the UI differently and build it better. But maybe I have found a bug that could have an effect on other parts. Or there are other ideas to prevent certain things, display a warning or something similar. We will see. Thanks for your support.

There are a few issues with your project. The StartmenuIcon control does not define a minimum size, so when you add it to a Container node, it gets collapsed and occupies no space. The inner button is anchored to be at the center of the area, but since there's no area at all, it implicitly means it will be moved to the corner. Container nodes will resize your nodes arbitrarily, so if you want a minimum size, you need to specify it additionally:
https://github.com/godotengine/godot/assets/6501975/4d26c215-7fef-4206-abfb-cfbf43a3dea8
Another issue is that there's a lot of small changes on Editable Children enabled nodes. This makes it difficult to troubleshoot such issues as it overrides any changes on the source scene. I had to disable and re-enable all of these and delete the .godot folder in order for changes in the original scenes to propagate to other nodes, so I'd avoid relying too much on this feature.
Lastly, do not mix Control and Node2D nodes, as they are fundamentally different. In this context, Node2D objects will not move/scale properly with changes in the UI, and cannot "ask" for minimum size either. For displaying assets, the equivalent in your button would be TextureRect:

Thank you for your help and quick support.
The issue has been resolved and I am now closing it.


vNext: Update coding style guidelines

Re: use of WINDOWS_UWP vs UNITY_UWP, etc

I don't think this is in vNext yet


Fix false positive in some games with AntiKick

Do you happen to know if the game has any plugins

Its most likely a plugin/script in the game causing the kick.

Do you happen to know if the game has any plugins

Sadly not, I'm not a developer of that game nor do I know anyone who is, it's possible though. /shrug

This is already fixed by:

#959 This issue fixed :unincognito
#778 This issue fixes invalid workspace names

But the first one isn't committed to the Adonis release, meaning it's fixed but an update hasn't been released.
The second issue is fixed by #778 pull request but it's not merged by Sceleratis yet.

So #778 fixes probably what the issue is.
What the issue the pull fixes is if the workspace name is blank. Ie. ""
So before Sceleratis merges #778 you can set Workspace name to something else than a blank string!

So #778 fixes probably what the issue is.
What the issue the pull fixes is if the workspace name is blank. Ie. ""
So before Sceleratis merges #778 you can set Workspace name to something else than a blank string!

I don't think it will, I got this screenshot from one of the people who told me about this:
They're not an exploiter, so I don't get how this would even fire off for them.

What method did it show? Like method1, method2, method3?

So #778 fixes probably what the issue is.
What the issue the pull fixes is if the workspace name is blank. Ie. ""
So before Sceleratis merges #778 you can set Workspace name to something else than a blank string!

I don't think it will, I got this screenshot from one of the people who told me about this:
They're not an exploiter, so I don't get how this would even fire off for them.

~What method did it show? Like method1, method2, method3?~ Edit: Nvm it's method2 i can see the screenshot

I just asked them anyway, they said it initially says method1 then changes instantly to method2 after the first kick message appears.

Is the workspace name a blank string. Or does it become a blank string?

So #778 fixes probably what the issue is.
What the issue the pull fixes is if the workspace name is blank. Ie. ""
So before Sceleratis merges #778 you can set Workspace name to something else than a blank string!

I don't think it will, I got this screenshot from one of the people who told me about this:
They're not an exploiter, so I don't get how this would even fire off for them.

~What method did it show? Like method1, method2, method3?~ Edit: Nvm it's method2 i can see the screenshot

I just asked them anyway, they said it initially says method1 then changes instantly to method2 after the first kick message appears.

Is the workspace name a blank string. Or does it become a blank string?

Got a friend to go in with Dex, it's definitely Workspace. By the way, the user corrected themselves after I asked about if they saw method 3 on the kick messages, and guess what; they did after all. I think it's safer to disable these detections for now until we find the root cause, as we could theoretically be causing games to lose players if this is more widespread than we think it is.

By looking at the messages it appears that the Roblox namecalls do not error. Instead of disabling the checks you should only disable the ones which use namecalls.
I still haven't been able to reproduce the issue in the game.
Also are you sure that are there any scripts that might set workspace name? Check if workspace name is something different

Just asking for coriosity. What devices are they using. Are they using a Windows beta app?
Also ask what version of the Roblox client are they using. Ask them to send a screenshot of this \/

Both my friend and the user having the issue are using the latest version

Can you show me the percise issue they are getting. I haven't been able to reproduce this in that game.

Both my friend and the user having the issue are using the latest version

Oops I meant can you show me the direct version of Escape Menu > Help > Version container

I can't reproduce this issue and I didn't find anyone else having this issue in the game, in the group or in their discord server (or at least they haven't reported so)

Was this an april fools joke? (Just asking)

Actually nevermind this is a bug on Robloxes side

The only way this can be fixed is if Roblox fixes this.


Adding related repositories to README

Coverage remained the same at 55.357% when pulling cf405a12875c88dc49a471110128c4bbcb4a2bf2 on mtlynch:related-repos into 7697e2551d2199947d7891744083460d7e8dff86 on JeetShetty:master.


protocolVersion on isvc should be an array?

/kind feature
Describe the solution you'd like
The protocolVersion field on ServingRuntime is an array, should this also be an array on InferenceService as for example kserve python model server now supports both v1 and v2(Open Inference Protocol) ?
Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Related Issues: #3674