[Bug] flutter build web / flutter build bundle copies files to assets which aren't assets

I created a demo for this bug: https://github.com/jld3103/flutter_assets_bug
When you use flutter build web / flutter build bundle files which aren't in the assets, but have the same name. The are copied to build/web/assets / build/flutter_assets/.
This needs to be fixed very soon.

Hi @jld3103
could you please provide your updated  flutter doctor -v ?
Thank you

[✓] Flutter (Channel master, v1.12.3-pre.49, on Linux, locale en_US.UTF-8)
    • Flutter version 1.12.3-pre.49 at /home/jld3103/flutter
    • Framework revision 21e6fe1ddf (13 hours ago), 2019-11-17 23:39:51 -0500
    • Engine revision 3261324fab
    • Dart version 2.7.0

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
    • Android SDK at /home/jld3103/Android/Sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-29, build-tools 29.0.2
    • ANDROID_HOME = /home/jld3103/Android/Sdk
    • Java binary at: /home/jld3103/.local/share/JetBrains/Toolbox/apps/AndroidStudio/ch-0/192.5994180/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b4-5784211)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at google-chrome

[✓] Android Studio (version 3.6)
    • Android Studio at /home/jld3103/.local/share/JetBrains/Toolbox/apps/AndroidStudio/ch-0/192.5994180
    • Flutter plugin version 41.2.3
    • Dart plugin version 192.7402
    • Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b4-5784211)

[✓] IntelliJ IDEA Ultimate Edition (version 2019.3)
    • IntelliJ at /home/jld3103/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/193.5233.12
    • Flutter plugin version 40.2.5
    • Dart plugin version 193.4386.6

[✓] Connected device (3 available)
    • SM G389F   •<PHONE_NUMBER>f5b36f • android-arm    • Android 6.0.1 (API 23)
    • Chrome     • chrome           • web-javascript • Google Chrome 78.0.3904.97
    • Web Server • web-server       • web-javascript • Flutter Tools

• No issues found!

Thanks for the repo @jld3103, I'm taking a look now

So the issue here is that this code is triggering our variant detection logic. Normally variants are handled such that all of the following assets are bundled with a single a.png asset listed in the manifest.
2x/a.png
1x/a.png
a.png

There isn't any restriction on what the variant should be called though, which leads to us picking up the random txt file.
For reference: https://github.com/flutter/flutter/blob/ad1d67e0a1ebc444ec9e89573d6df97a2fdb2e41/packages/flutter_tools/lib/src/asset.dart#L499

Duplicate of https://github.com/flutter/flutter/issues/45075.


Add store#keys to Caching object requirements

A NoMethodError exception was raised when I tried expiring the cache while using a PORO Cache <-> DB adapter that did not implement #keys.
As an aside, sorry if it was inappropriate to submit the PR without creating an issue first, but it seemed unnecessary to create an issue for such a small change solely to documentation. I'm interested if you feel the same way, as I'm relatively new to contributing in the wild.

Thanks for this! (And no, an issue is not necessary.)
So, #keys is not currently listed as required because it's only needed for expiring all keys (not individual keys), which, at the time I wrote the docs, didn't seem like core cache functionality. Whether it is or isn't, I agree that it should be mentioned. What do you think about listing it as optional, and/or mentioning that it's only needed for expiring the whole cache?

This makes me wonder if it were optional, there should be some code that wraps the cache expire methods, checking first if keys exists, so it can fail gracefully.
But that also may just be covering an error up that you would need to see, in order to know something is amiss. I feel like since the initial objects expect the cache store to respond to #keys, it should be required. And, thankfully, it's probably no more troublesome to implement than the get method anyway.
I'm not sure what the best direction is, and right now I'm only thinking in the context of a rdbs adapter, where select keys is no biggie.

It is actually quite troublesome to implement in a system like memcached which cannot list the keys. That's why it's not required. If anything, I think expire(:all) should fail with a more useful error message. But clear(er) documentation would also be sufficient, for now.

Okay, I better understand now and agree that it shouldn't be marked as required. In the area describing the anatomy of a caching object, perhaps have it be notated as optional with a short description as to when it's used, and perhaps just a quick reminder that your caching object should respond to #keys in the section that talks about how to expire :all... "If your cache object responds to #keys you can expire :all keys..." or some such? Too much blah blah?
Feel free to add the changes to the docs as you best see fit, or I can add another commit tomorrow after getting more feedback from you to confirm this direction.
Thanks!

I've tried to keep the documentation simple while still getting the information across. I've also added information at runtime if you try expiring :all without implementing :keys.
I wasn't sure if the best place was right before the call for #keys or immediately after calling expire(:all). I felt :all made more sense, so the error message could be more helpful, and specific to calling :all.
Let me know if you'd like this further improved. Thanks!

This looks good, thanks. Just two things:

Because error messages can come from deep within code you didn't write, it'd be helpful to clarify where the exception comes from. So maybe a message like:
"The Geocoder cache store must implement #keys in order for expire(:all) to work"

I think the logic would be clearer if there was a multi-line if/else/end structure: if keys is implemented, call it, else raise an exception. Make sense?

Other than that, this looks good to me.

Okay, requested updates are pushed up.
Can I ask you -- given this back and forth and minor text fixes across three commits -- what is the typically desired submission process? It's nice to have the individual commits and the associated discussion between the iterations, but for the actual project, it should obviously be squashed.
Do contributors typically squash the series of commits after these resolutions and submit a new PR? Or force push to the branch? Or just let the maintainer deal with it? Thanks for the feedback.

Thanks! Looks great. As for the process (it's so nice that you asked), I don't have a standard way of handling it. Some people rebase as they go and others force push new commits. If there's something substantive to discuss about the commits themselves I'll usually start a discussion about it, but usually I just squash/merge (and lightly edit the commit message). I think that makes the most sense in this case. Thanks again!


Make use of a directory provider

Is your feature request related to a problem? Please describe.
Abstraction of the System.IO.Directory class to enable better testability.
Describe the solution you'd like
Make it happen.
Describe alternatives you've considered
None.
Additional context
Same implementation as the IFileProvider and FileProvider can be applied.

@VeganMeatGuy thx for the hint.


[BUG] - Just cannot fix nosystemD for month and months?

Again same old same old I want to compile without systemd preconfiguring the project with:
I run:
./scripts/gen-cabal-nosystemd.sh
It creates:
cabal.nosystemd.project
I run:
cabal configure -O0 -w ghc-8.10.2
It returns:
`cabal: Could not resolve dependencies:
[__0] trying: Win32-network-<IP_ADDRESS> (user goal)
[__1] trying: base-<IP_ADDRESS>/installed-<IP_ADDRESS> (dependency of Win32-network)
[__2] trying: lobemo-scribe-systemd-<IP_ADDRESS> (user goal)
[__3] next goal: libsystemd-journal (dependency of lobemo-scribe-systemd)
[__3] rejecting: libsystemd-journal-1.4.5 (conflict: pkg-config package
libsystemd==209 || >209, not found in the pkg-config database)
[__3] rejecting: libsystemd-journal-1.4.4 (conflict:
base==<IP_ADDRESS>/installed-<IP_ADDRESS>, libsystemd-journal => base>=4.6 && <4.13)
[__3] skipping: libsystemd-journal-1.4.3, libsystemd-journal-1.4.2,
libsystemd-journal-1.4.1, libsystemd-journal-1.4.0, libsystemd-journal-1.3.4,
libsystemd-journal-1.3.3, libsystemd-journal-1.3.1, libsystemd-journal-1.3.0,
libsystemd-journal-1.2.0, libsystemd-journal-1.1.0, libsystemd-journal-1.0.0
(has the same characteristics that caused the previous version to fail:
excludes 'base' version <IP_ADDRESS>)
[__3] fail (backjumping, conflict set: base, libsystemd-journal,
lobemo-scribe-systemd)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: base, libsystemd-journal,
lobemo-scribe-systemd, Win32-network
Try running with --minimize-conflict-set to improve the error message.
`
Seriously fuck systemd and it's degenerate users and followers.

I have same problem...
Step 19/47 : RUN scripts/gen-cabal-nosystemd.sh
 ---> Running in 79567ac754e7
++++ realpath scripts/gen-cabal-nosystemd.sh
+++ dirname /usr/src/cardano-node/scripts/gen-cabal-nosystemd.sh
++ dirname /usr/src/cardano-node/scripts
+ root=/usr/src/cardano-node
+ cd /usr/src/cardano-node
+ sed -e '/plugins\/scribe-systemd/d' cabal.project
+ echo ''
+ echo 'flags: -systemd'
+ '[' -e cabal.project.freeze ']'

and then
Step 23/47 : RUN cabal install cardano-cli       --project-file=cabal.nosystemd.project       --installdir=/usr/local/bin       --install-method=copy       -f -external-libsodium-vrf
 ---> Running in e98a3ce0488c
cabal: Could not resolve dependencies:
[__0] trying: Win32-network-<IP_ADDRESS> (user goal)
[__1] trying: base-<IP_ADDRESS>/installed-<IP_ADDRESS> (dependency of Win32-network)
[__2] trying: cardano-tracer-0.1.0 (user goal)
[__3] next goal: libsystemd-journal (dependency of cardano-tracer)
[__3] rejecting: libsystemd-journal-1.4.5 (conflict: pkg-config package
libsystemd>=209, not found in the pkg-config database)
[__3] rejecting: libsystemd-journal-1.4.4 (conflict:
base==<IP_ADDRESS>/installed-<IP_ADDRESS>, libsystemd-journal => base>=4.6 && <4.13)
[__3] skipping: libsystemd-journal-1.4.3, libsystemd-journal-1.4.2,
libsystemd-journal-1.4.1, libsystemd-journal-1.4.0, libsystemd-journal-1.3.4,
libsystemd-journal-1.3.3, libsystemd-journal-1.3.1, libsystemd-journal-1.3.0,
libsystemd-journal-1.2.0, libsystemd-journal-1.1.0, libsystemd-journal-1.0.0
(has the same characteristics that caused the previous version to fail:
excludes 'base' version <IP_ADDRESS>)
[__3] fail (backjumping, conflict set: base, cardano-tracer,
libsystemd-journal)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: libsystemd-journal, cardano-tracer,
base, Win32-network
Try running with --minimize-conflict-set to improve the error message.

The command '/bin/sh -c cabal install cardano-cli       --project-file=cabal.nosystemd.project       --installdir=/usr/local/bin       --install-method=copy       -f -external-libsodium-vrf' returned a non-zero code: 1
markstopka@mark-zenbook-s:~/Work/github.com/PERLUR/docker-cardano-images/cardano-node/1.35.0>

with 1.35.0 node version.

I can confirm that RUN zypper install -y systemd-devel fixes the issue...


Set up distributive-related config-ownership

Apache run on defferent systems under different users,
e.g. "apache" (RedHat) or "www-data" (Debian), and
this user must own configuration file of zabbix web interface.
Otherwise user can see an error: "Unable to create the configuration
file".

Hi @envrm
Travis is failing:
TASK [ansible-zabbix-server : Create zabbix-web directory] *********************
fatal: [zabbix-server-mysql-centos]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'apache_user' is undefined\n\nThe error appears to have been in '/home/travis/build/dj-wasabi/ansible-zabbix-server/tasks/main.yml': line 73, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: \"Create zabbix-web directory\"\n  ^ here\n"}
fatal: [zabbix-server-pgsql-centos]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'apache_user' is undefined\n\nThe error appears to have been in '/home/travis/build/dj-wasabi/ansible-zabbix-server/tasks/main.yml': line 73, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: \"Create zabbix-web directory\"\n  ^ here\n"}

Can you please take a look at it?
Kind regards,
Werner


Usability of Lasair-ZTF

Review web interface and functionality for Lasair-ZTF

I don't think we want to do this. Our focus now should be how we evolve to Lasair-LSST, not what we can do further for Lasair-ZTF

I would like to undertake some sort of overview of the Lasair web interface. There's a lot of squirrelly bits that I would hate to see grandfathered in to our new system.


DuckyIcons/Fonts is not working / set-up correctly

Seems special characters are now invalid; like å.
Possibly related / fixable at the same time.

This was closed by #18


Poedit crashes upon opening a PO translation file

Windows 7
Poedit version 2.2.4
Anytime I try to open a PO translation file it opens for a split second before crashing and causing the crash report dialog box to appear. I have sent many crash reports for this issue but to no avail. I have tried uninstalling and re-installing PO Edit but that didn't seem to help.
I thought that perhaps it could've been a space issue as my C drive storage was becoming full  but I have cleared some space it and Poedit still crashes. The Poedit application itself opens but not when I try to open specific files. Other users opening the same specific files aren't having issues.
Any help would be greatly appreciated.

Please read this document (which GitHub asked you to do when you created this issue), follow the instructions therein and provide the missing detailed information. Thanks!
In particular, please do note the very first bullet point in there.

Apologies for my above error. I have sent an email to<EMAIL_ADDRESS>with my crash report xml and dmp file along with more details on my actions before the crash occurs.

I do not understand why you make such a mess - if you file a bug report, be prepared to follow through on it, instead of turning to a different channel :(
Therefore deleting this  issue as useless.


Specify yarnpkg version explicitly

This avoid the failure by changing yarn.lock format by yarnpkg change.
Drawbacks
By this change, we need to specify the version of yarnpkg.
It increases our maintenance burden.

This change is

bors r+


It doesn't go over from "Building wheels for collected packages:" in Colab

I've tried to run !pip install flash_attn --no-build-isolation but it doesn't move over 5 minuites.
With Issue#304 Colab's nvidia cuda version  and pytorch cuda version is also same.
I also tried other version (eg 1.0.7, 1.0.9 and even 1.0.3) but always do not goes on.

I have similar error as you with higher versions. Version 1.0.8 works fine for me now.

Any updates on how to resolve this issue?

@GuangxingHan  @apcameron Still even I tried 1.0.8 version using !pip install flash_attn==1.0.8 --no-build-isolation but it doesn't work 😢

@tridao Is there a workaround or fix for this?

We recommend the Pytorch container from Nvidia, which has all the required tools to install FlashAttention.


EmrEtlRunner: placeholder for deciding what to do with file-related no-ops

One of the powerful features of EmrEtlRunner is that it is very good at detecting no-op scenarios that should lead to a graceful termination of the run.
In particular:

If no data is found in raw for moving to staging, then do not proceed (nothing to do)
If data is found in staging, enriched or shredded, then do not proceed (previous run is ongoing or has failed pending manual fix)

In a world where all S3 file manipulation has moved to EMR, and in a world where EmrEtlRunner is having the dataflow logic extracted from it (ready for the move to Dataflow Runner), then we need to decide:

Do we still need these no-op detections, or is there a better way of accomplishing the same thing?
Where should these no-op detections live / how should they be implemented?

One idea is to move these checks into individual jobflow steps (run on master), and make them terminate the cluster if they don't pass. Then it's on EmrEtlRunner (and later Dataflow Runner) to realize that these steps terminating the cluster means a benevolent no-op, not a job failure per se... Thoughts @BenFradet, @chuwy ?

The added value from the current checks I see is the short-circuiting of the launch of the EMR cluster.
Once the EMR cluster is up, I don't think those checks matter, it has virtually no impact whether to run the no-ops steps or not, unless I'm mistaken?
And since we want to move everything to EMR, the cluster would be up no matter what.

You are right - we are losing the ability to short-circuit the launch of the cluster; that's never coming back.
However:

We have to terminate the cluster if the previous run has failed/is ongoing - the pipeline is not currently setup to successfully work through a subsequent load if the previous one failed
(Less important) there may be an advantage in shortcutting a no-data run early (vs just letting the pipeline run its course with no data). It may even be that a downstream component will break if there's no data to process - needs testing.

From what I gathered/experimented with today, we could:

implement the different checks (empty dirs and even number of lzo files) through simple bash scripts submitted as emr steps as you said earlier
use s3distcp for staging and archiving. However, the cleaning up of the filenames performed in s3_tasks seems a bit too involved for an s3distcp step. As a result do you think we can do without or that we should rely on something else entirely to do the staging step?

Thanks @BenFradet -

Implement the empty dirs checks through simple bash scripts submitted as emr steps sounds good - the challenge is we need to figure out how to capture return codes from those bash scripts
Note that the LZO files check will be gone soon (#2740)
S3DistCp for archiving raw is coming soon (#1977)
S3Distcp for archiving enriched/shredded will be tackled by @chuwy (#1777)
the cleaning up of the filenames performed in s3_tasks seems a bit too involved for an s3distcp step - agree; if we consider that we have four collector formats to handle (cloudfront, clojure, lzo sink, Urban Airship), my hunch is that 2 or 3 of these can be handled by S3DistCp with a bit of hacking; 1 or 2 of these will need a dedicated script/binary which runs on the EMR cluster

the challenge is we need to figure out how to capture return codes from those bash scripts
Isn't a fail/succeed step enough?

I'll keep on investigating the last point

Isn't a failed/succeeded step enough?

Yes, I think you will end up with:

Succeeding/failing the step
Writing custom code in EmrEtlRunner to interpret certain jobflow steps failing as a no-op with specific return code from EmrEtlRunner, versus currently where any jobflow step failing means that EmrEtlRunner failed.

I guess I don't really see the difference between:

now:

a no-op which results in the termination of both the cluster (or avoid starting it but this goes away no matter what as we said earlier) and eer (through a DirNotEmptyError or a NoDataToProcessError), for me this does seem like a kind of failure
the failure of a step like enrich which results in the same thing (through EmrExecutionError)

and what would be (as I imagine it):

no-ops and failures both result in the termination of both the cluster and eer (through EmrExecutionError)

Since EmrExecutionError invites the user of eer to check the console, she/he will see why things failed (no-op or "real" failure).
I think this simplify things quite a bit actually regarding the transition to dataflow runner compared to having to inspect the status of a particular set of steps.
Would love to hear your thoughts.

The difference is that any DirNotEmptyError or a NoDataToProcessError should not be escalated to a engineer for investigation - they should not generate an incident waking up a team member in the middle of the night to verify that they are dealing with a no-op versus a failure requiring manual intervention.

So here is how I've managed to mimic the current behavior of eer without relying on Sluice based on two scripts that check whether a directory is empty:

script step checking that the processing is empty, if not failure of the step which is then picked up by eer and results in a DirNotEmptyError
staging s3distcp step (still has to be coded)
script step checking that the processing has been filled, if not -> failure of the step -> NotDataToProcessError
script steps checking that shred and enrich are empty, if not -> failure of the steps -> DirNotEmptyError

I think this sounds good @BenFradet. Maybe split this into two tickets:

One covering (both) DirNotEmptyError scenarios
One covering the NoDataToProcessError scenario

@alexanderdean Ok, I'll try to split the commit in two.

created #3130 and #3131 :+1:

Closing and de-scheduling this given we have specific tickets now...


Update pricing.md

Added pricing examples for Calls with at least 1 PSTN leg

@dassabya15 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@dassabya15 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Closing at author request - changes have been moved to private repo.


Feature/7 create user

Clinic admins can create users now.

Not covered lines in userController relate to server errors


Support marking all tasks in a note as done/undone

One may have some checklists which contain many tasks/items. One may mark the tasks in a checklist file as done one-by-one. However, for the next, one may want to start from the beginning again. Therefore, it would be great if one can mark all tasks as undone with a command and/or hotkeys. Similarly, one can also mark all tasks in a note as done.

Let's take the following two examples:
## Tasks
- [x] one
  - [x] sub-task 1
- [x] two
  - [x] sub-task 2
- [ ] three
  - [x] sub-task 3
- [ ] four
  - [ ] sub-task 4

"Mark all tasks done" would complete three and four.
"Unmark all tasks" would clear the completion status for all 8 tasks.

This is what you're asking for?
Using Move completed tasks:
## Tasks
- [ ] three
  - [x] sub-task 3
- [ ] four
  - [ ] sub-task 4
## Log
- [x] one
  - [x] sub-task 1
- [x] two
  - [x] sub-task 2

Given the above, if you "Move completed tasks" again. sub-task 3 would move to the completed area.
"Complete all tasks" would then complete 3, 4, and sub-task 4.
"Move all completed tasks" would then move all of those to the log section
"Unmark all tasks" could clear the task completion data, but wouldn't be able to recreate/move incomplete tasks back up into the Tasks section.

What do you think should be done here?
Should an "Unmark all tasks" command ignore things in the completed area (if configured)?

I think the behaviour depends on specific use cases/examples.
Example 1:

This is the feature request I mainly refer to, and the two behaviours you described are exactly what I am asking for.
As I explained early, such a use case may be checklists which may use repeatedly/regularly, e.g. during GTD Weekly Review.

However, such repeatedly used checklists may not be put in pages/notes like daily notes.
Therefore, for this use case, may be it is not necessary to have a Log section for accumulating all completed tasks.

Example 2:

This may represent tasks in project pages, daily notes, etc.
If this is the case, then the question I am interested is that in which case(s) one would like to "Unmark all tasks".

If the answer is that it can be used for things like managing recurring tasks, then maybe one can first re-name the section "Log" to something else, and then Unmark all tasks, if needed.

To sum up:

For Example 1, one can simply mark all tasks as done or undone.
For Example 2, there can be some user-configured Section that the "Unmark all tasks" command would ignore.

If the user would like to unmark certain tasks in the completed area, then one can do things like re-naming the section name. I, personally, would probably rarely use this, though.

For Example 1, is it possible to also support mark all tasks in embedded notes on the current page as done/undone? For instance, the Note A may consists of the following:
- [ ] Task A
- ![[Note B]]

Note B may contain additional tasks.
There can be a separate command for marking all tasks and tasks in embedded notes as done/undone.


Reduce cache.Get lock conflict

Please provide a description of this PR:
For #40744

@hzxuzhonghu: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
57c48be2fc0fb2e2cc6185432945eaeaa01025e8
link
true
/test unit-tests-arm64

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
57c48be2fc0fb2e2cc6185432945eaeaa01025e8
link
true
/test unit-tests-arm64

release-notes_istio
57c48be2fc0fb2e2cc6185432945eaeaa01025e8
link
true
/test release-notes

unit-tests_istio
57c48be2fc0fb2e2cc6185432945eaeaa01025e8
link
true
/test unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
57c48be2fc0fb2e2cc6185432945eaeaa01025e8
link
true
/test unit-tests-arm64

release-notes_istio
57c48be2fc0fb2e2cc6185432945eaeaa01025e8
link
true
/test release-notes

unit-tests_istio
57c48be2fc0fb2e2cc6185432945eaeaa01025e8
link
true
/test unit-tests

lint_istio
57c48be2fc0fb2e2cc6185432945eaeaa01025e8
link
true
/test lint

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Yes, i see the ut raced.

It is not evict, but c.evictList.MoveToFront(ent)

looks like this should be fixed in upstream

That repo hasn't seen a commit in > 1.5 year. If we want the change we probably have to contribute it ourselves.
I wonder if we're hitting the ceiling (XDSCacheMaxSize) of the LRU in practice.

It's really not clear from the issue reported in golang-lru what the problem is, or from this PR. Happy to have this fixed in golang-lru, especially if you know what's wrong and can PR it.

I don't think its a bug in golang-lru at all; by nature of the implementations Get() in the LRU mutates data (to evict data/record use), so using a read lock only (rather than write lock) is not sufficient on Get().

Ah, OK, a misunderstanding. I will close the upstream bug then.


Conform WebSocketHTTPHandler to RFC 6455 section 4.2.1

RFC 6455 section 4.2.1 prescribes which headers are mandatory for a WebSocket handshake request to be correct. It also details what each of these headers should contain. This PR brings WebSocketHTTPHander up to spec (which has a typo by the way, but I didn't change that to keep this PR focussed).
I have documented the new code in a way that makes it easy to match up each check with its corresponding RFC rule (see WebSocketHTTPHander.verifyHandshakeRequestHeaders).
There is one conundrum that I ran into while implementing the checks, which is that rule 1 mandates that websocket connections must begin as GET requests, however in FlyingFox the method for websocket endpoints is up to users of the framework. We could add some kind of check to appendRoute, or we could just leave it given that developers will probably realise pretty quickly that their websocket won't work as a POST endpoint. Another option would be creating a separate method for hosting websocket endpoints such as appendWebSocketRoute which takes only a path and doesn't let developers specify the method.

I also added a test for the new checks, but it would probably be better if I split it into a test per rule (and updated it to use validateHandshakeRequestHeaders now that I've refactored through checks into their own function).

RE: the requirement for Sockets requiring the GET HTTPMethod.  I do think it is probably OK to allow developers to choose the HTTP Method, even if it is not compliant. We could use HTTPLogger to output a warning.
One use case I have for FlyingFox is to assist within testing. WebSocketHTTPHandler* should be as close to RFC compliant as possible, but I see some utility in allowing any HTTPHander to switch to a web socket connection.

And I have renamed the handler here.

Looks great. Thank you for these RFC fixes.
A month ago I had no idea how websockets actually worked so this really helps 🙏🏻.

 While reading through the RFC I also found out that I had no idea how it actually worked hahah, there's certainly a lot to learn by reading the RFC documents, but I will admit that they can be quite bland sometimes. I never knew that websocket connections started as HTTP requests and then switched, the more you know.

RE: the requirement for WebSockets requiring the GET HTTPMethod. I do think it is probably OK to allow developers to choose the HTTP Method, even if it is not compliant. We could use HTTPLogger to output a warning.
One use case I have for FlyingFox is to assist within testing. WebSocketHTTPHandler* should be as close to RFC compliant as possible, but I see some utility in allowing any HTTPHandler to switch to a web socket connection.

I agree with that reasoning, I don't see any issues with straying from the RFC in that regard 👍


Fix tools/parser/BUILD.bazel

Hi. Thanks for taking the time to look at this.  Unfortunately this bit of code is left over from earlier experiments; my apologies for not removing it. It will be removed from the next alpha release.   The build code for re2c is in //external/re2c.


businessSectorID won't display

The Industry field or the businessSectorID won't display on the bullhorn JSON Feed after I put the businessSectorID inside app.json.
The support already added the businessSectorID to the publicRESTJobAPIFieldList System and  still won't work.
Steps to replicate behavior

insert businessSectorID to app.json  under fields
ng serve

Hello,
businessSectorID is not a field in Bullhorn.  business sector is a to-many association that will likely need to be formatted like businessSector(id)
Entity Reference Documentation for reference.


Pass in a collection from resolved data

I'm resolving some data in my route and I'd like to pass it to the directive's collection instead of the directive making a second request when the view is loaded. Is this possible?

Sorry, I lost track of this issue. Have you found a way around the problem in the meantime?

I wasn't able to spend too much more time to figure out a way around the issue, so I just let the directive handle querying the collection. If I have some bandwidth to experiment and find a viable solution, I'll definitely let you know.


Cannot use jest --watch

Do you want to request a feature or report a bug?
bug
What is the current behavior?
When I run yarn test --watch , it's never going to work. and reports many errors.

My package.json file:
"scripts": {
    "build": "webpack",
    "dev-server": "webpack-dev-server",
    "test":"jest"
  },

I followed someones' guide that run yarn test -- --watch still not working.
It's my first time using a testing tool, is there anything ignored?
If the current behavior is a bug, please provide the steps to reproduce and either a repl.it demo through https://repl.it/languages/jest or a minimal repository on GitHub that we can yarn install and yarn test.
What is the expected behavior?
Please provide your exact Jest configuration and mention your Jest, node, yarn/npm version and operating system.
Jest : 21.2.1
node : 8.5.0
yarn : 1.2.1
system: macOS High Sierra 10.13

Looks like you don't run in git/mercury repository? cc @aaronabramov

@thymikee yep, --watch works after repository initialized。

Duplicate of #4419


Run command for only one user

Hello,
I was wondering if there is a way to run this command on only one or just a few of our enterprise users as a test run. This would help make sure it will be ok when we run it on all of our enterprise users.
Thanks for the help.

Unfortunately not; this tool was purpose-build to update everybody in the enterprise. I would recommend one of:

Create a developer enterprise with a non-edu account. Create an app in the developer enterprise and run the tool with that token.
Run this against a test edu enterprise if you have one.

If you're comfortable with C# (or even Java), you could also change the code to accept a command line argument for a single username and pass that username to the Box user query as a filterTerm the form "username@".

Thanks, John. I'll give this a try.


Resampler: IPython 5.0 triggers wrong FutureWarning

With the latest IPython:
In [1]: rng = pd.date_range('20130101', periods=10, freq='T')
   ...: ts = pd.Series(np.arange(len(rng)), index=rng)

In [3]: r = ts.resample('2T')

In [4]: r
Out[4]: c:\users\vdbosscj\scipy\ipython-joris\IPython\utils\dir2.py:65: FutureWa
rning: .resample() is now a deferred operation
use .resample(...).mean() instead of .resample(...)
  canary = getattr(obj, '_ipython_canary_method_should_not_exist_', None)
DatetimeIndexResampler [freq=<2 * Minutes>, axis=0, closed=left, label=left, con
vention=start, base=0]

In [5]: import IPython

In [6]: IPython.__version__
Out[6]: '5.0.0'

In [7]: pd.__version__
Out[7]: '0.18.1+184.g15c2a3a'

while using IPython 4.2:
In [1]: rng = pd.date_range('20130101', periods=10, freq='T')

In [2]: ts = pd.Series(np.arange(len(rng)), index=rng)

In [3]: r = ts.resample('2T')

In [4]: r
Out[4]: DatetimeIndexResampler [freq=<2 * Minutes>, axis=0, closed=left, label=l
eft, convention=start, base=0]

In [5]: import IPython

In [6]: IPython.__version__
Out[6]: '4.2.0'

In [7]: pd.__version__
Out[7]: '0.18.1+184.g15c2a3a'

@takluyver This seems to be related to https://github.com/ipython/ipython/pull/9289 (previously, it was first checked if method is a callable, and only then it was checked if it could retrieve a non-existing attribute, while now the callable check is moved to the end of get_real_method).
The problem is that the deprecation warning is only triggered once you "do" something with it, and now it is triggered by displaying its repr

That's unfortunate. But if our check for the canary attribute triggers this, won't our checks for _repr_pretty_, _repr_json_, etc. also trigger it?

Apparently not. Where are those checks in the IPython code?

IPython.core.formatters - they're somewhat hidden in infrastructure, but if you search for print_method in that module, it should be possible to work out what's going on.

Ah, I was in a terminal, so I suppose there only the plain text displayer will be used?
If I try it in a notebook, but using IPython 4.2, the same issue already shows up

I thought the same machinery would check for _repr_pretty_ even in the terminal, but maybe it's done a bit differently.

@takluyver We solved it at our side! (at least, @chris-b1  did :-) )


blackbox: fix writeData mkdir permissions

It's clear that the intent of this mkdir was to use permissions 0755 (drwxr-xr-x.) and not 755 (d-wxrw---t).

cc @gmcculloug @jrafanie @blomquisg @kbrock

@miq-bot add_label fleecing
@miq-bot add_label bug

:+1:


Issue #3780 Documentation

Fixes #3780
Motivation
Update the documentation to more accurately reflect the condition of the queue when the availablePermits value is negative. The current documentation indicates that condition means the consumer is ready to receive more messages, when, in fact, it is NOT.
Modifications
Updated the http://pulsar.apache.org/docs/en/admin-api-non-persistent-topics.html#get-stats page
Verifying this change

[ ] Make sure that the change passes the CI checks.

This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): NO
The public API: NO
The schema: NO
The default values of configurations: NO
The wire protocol: NO
The rest endpoints: NO
The admin cli options: NO
Anything that affects deployment: NO

Documentation

Does this pull request introduce a new feature?


Make Subscription isValid check more robust

The documentation for a Subsriptions's isValid method claims to check if subscription is "active". However, looking over the source code, it appears to just check that the Subscription object was initialized with a non-null ConnectionImpl. This private conn class variable doesn't even to be set to null upon unsubscribe. As a minimum, conn should be set to null on unsubscribe. Ideally, it would check more state on the connection to ensure messages can actually make it to the subscription.

It looks like the subscription's conn is already being set to null: https://github.com/nats-io/java-nats/blob/1.0/src/main/java/io/nats/client/ConnectionImpl.java#L1527-L1528

We are working on a new version of the client that should resolve this issue, I am leaving the issue open for now, but if you are interested in this feature please try the new library in the version2 branch. Feedback is appreciated.

Fixed in version 2.


Expose Timeout,Workers to shield-daemon config

Here is a small pull request to resolve #88
I proposed that the variables are always written to the config file of shield-daemon so that the (default) values are easily visible to the operator. The alternative would have been the vars not to be there, in which case the operator should look the shield codebase to see the default values.
Not sure if the description Max number of concurrent tasks in running state is accurate.
Thx

I am not sure they concourse is failing.
Are there some pipelines that I can run against concourse-lite to verify the pull requests?

@karampok I'll investigate the CI failure. The PR support is something I'm working on. Can you confirm that your PR is based on master/HEAD?

@karampok Yep, if you can do the CI system a favour and rebase your branch against upstream master branch, then CI should work:

yes, no problem. Now it looks better!

Don't forget to add release notes when you make changes...

Ahh sorry I forgot to look for them in the PR before merging. I'll add them manually.

From: James Hunt<EMAIL_ADDRESS>Sent: Sunday, August 20, 2017 9:39:26 PM
To: starkandwayne/shield-boshrelease
Cc: Dr Nic Williams; State change
Subject: Re: [starkandwayne/shield-boshrelease] Expose Timeout,Workers to shield-daemon config (#90)
Don't forget to add release notes when you make changes...
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHubhttps://github.com/starkandwayne/shield-boshrelease/pull/90#issuecomment-323579662, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAAAbHXXtBUkRNayAVU3BcwXPNyr7WEQks5saBrtgaJpZM4O7Til.


Implement BrokerHostUsage using java

Motivation
Issue #81
I've made a first go a this, I believe it's working ok, please review :blush:
Modifications
I've removed support for usage script and made BrokerHostUsage calculate metrics from system data.
Current implementation only supports Linux, we should probably find a way to implement the same functionallity for other OS.
Result
Broker can collect usage metrics without need for a script.

CLA is valid!

@merlimat do these last changes seem ok? I've reused loadManagerExecutor to schedule usage calculations, and addressed most of the comments I believe.

@merlimat pushed a generic implementation, could you review this again please?

@merlimat tests are green now, can you give this a last review?


Update SopsSecretGenerator to 1.3.2

I just learnt, that the used SopsSecretGenerator doesn't use an installed sops binary, but instead has sops compiled in! And the version we are using (1.3.0) has sops 3.5.0 compiled in.
So we should update the SopsSecretGenerator to 1.3.2 which is using sops 3.6.1

fixed with #114


Product icon for debug-stackframe-dot is not overwritten by designated icon in product icon theme

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.87.2
OS Version: Windows_NT x64 10.0.22621 (Windows 11 Professional)

Steps to Reproduce:

Create a product icon theme that replaces the product icon with identifier debug-stackframe-dot
Load product icon theme
Start debug session in a file/language with debugger support (e.g. Python)
The icon is not replaced. In developer tools I can see that the CSS property for content is incorrect (it uses the default VSCode product icon font and the font glyph content ID) as well as that the font-family property is not set.

I am in the middle of making a product icon theme and I happened to notice the described behaviour.
Either the documentation at https://code.visualstudio.com/api/references/icons-in-labels is (no longer?) incorrect or the logic to replace this icon is not present. All the other debug and debug-stackframe icons seem to be working just fine.

cc @hbons who has been working on icons. This might not be a trivial change for design to make though. Possibly @aeschli can advise re: icon theme logic?

@connor4312 I've found some two more icons that are not (properly) replaced. Would you like me to edit the issue or create a new issue? Would it be of any value to provide the icon theme replacing the icons in question and a small repo to generate it for further validation?

The problem is here: https://github.com/microsoft/vscode/blob/c115d644bb7ae916d19d9ff9e6d4bb39b7261b0d/src/vs/workbench/contrib/debug/browser/media/debug.contribution.css#L39
this hardcoded fontCharacter prevents the icon from being themable.

Reopened as the fix is reverted

This was re-merged

This doesn't work yet as the the fix uses circleSmallFilled instead of debugStackframeDot
I created #211334, PR #211335


TestNG @Before configuration method @Steps show in test

When executing a test which includes @BeforeMethod or @BeforeClass and a method is executed which includes @Step, the logged step is included in the @Test.  In this situation it should not log the steps from configuration methods as they are not part of the test.  I can provide example code if necessary.

dup #706


Playlist - auto jump to next (method);

When the song is over the, the next song should start automatically;

Next song start automatically now. 0b14a90


Make default _auto_monitor choice configurable by user

@tacaswell The PV read timeout problem happens with EpicsSignal and EpicsSignalRO objects, as well.  Can we make a @classmethod to change the default auto_monitor=True for EpicsSignalBase?
Originally posted by @prjemian in https://github.com/bluesky/ophyd/pull/922#issuecomment-719356998
Similar to what has been done for timeout as referenced in #917

Implemented in #926


Update LDAP URIs

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

Run Seed Job


Error: Aborted because 145 is not accepted

I was doing some debugging in VS Code 1.13.1 and the error below keeps coming up every time I make a code change while debugging.
Is code changing not allowed while debugging in VS Code or am I missing something?

This is a duplicate of https://github.com/facebookincubator/create-react-app/issues/1880.
It's not intentional. I think when Webpack authors switched to using Promise.reject() there they didn't realize it would trigger debuggers.
I'm not sure what the solution could be. We asked people to think about workarounds in https://github.com/facebookincubator/create-react-app/issues/1880 but so far nobody has been interested in investigating this.

@gaearon @cr101 See solution for VS Code here, https://github.com/Microsoft/vscode-chrome-debug/issues/458#issuecomment-315646523

You can also check if https://github.com/webpack/webpack/issues/5175#issuecomment-312803913 helps. If it does it would be nice to submit a PR with it to Webpack.

I hope this https://github.com/webpack/webpack/issues/5175#issuecomment-352999475 may help


HTTP/2 field values aren't properly validated

HTTP/2 field values aren't properly validated
RFC 9113 Section 8.2.1 says this:

A field value MUST NOT start or end with an ASCII whitespace character (ASCII SP or HTAB, 0x20 or 0x09).

Envoy does not enforce this check.
To reproduce

Configure Envoy with the following config file, where PROXY_BACKEND_PLACEHOLDER is the address of a server you control:

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: <IP_ADDRESS>, port_value: 80 }
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          codec_type: AUTO
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match: { prefix: "/" }
                route: { cluster: some_service }
          http_filters:
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  clusters:
  - name: some_service
    connect_timeout: 0.25s
    type: STRICT_DNS
    lb_policy: ROUND_ROBIN
    load_assignment:
      cluster_name: some_service
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: PROXY_BACKEND_PLACEHOLDER
                port_value: 80

Send it the following HTTP/2 request:

PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00E\x01\x05\x00\x00\x00\x01\x00\n:authority\tlocalhost\x00\x05:path\x01/\x00\x07:method\x03GET\x00\x07:scheme\x04http\x00\x05test1\x03\ta\t

Envoy should forward the following HTTP/1.1 request to your server:

GET / HTTP/1.1\r\nhost: localhost\r\ntest1: \ta\t\r\nx-forwarded-proto: http\r\nx-request-id: 35852947-d952-4cf5-b660-550a85ac0188\r\nx-envoy-expected-rq-timeout-ms: 15000\r\n\r\n

Note that this request contains a header value that both begins and ends with \t.

Proposed fix
Respond 400, then close the stream when you receive an HTTP/2 request with a field-value that begins or ends with ASCII whitespace, as the standard requires.

CC @yanavlasov

Some HTTP servers, including Cesanta Mongoose and HAProxy, until recently, treat an empty header name and value as the end of the headers. The combination of this bug with that one could be used to do request smuggling.
This is sort of an obscure scenario, but I think it demonstrates that it's moderately important for this bug to be patched.

I'll start working on a patch for this.

I tried the following patch, but it doesn't seem to affect the problem.
diff --git a/source/extensions/http/header_validators/envoy_default/header_validator.cc b/source/extensions/http/header_validators/envoy_default/header_validator.cc
index e492dd1ee8..ae46282331 100644
--- a/source/extensions/http/header_validators/envoy_default/header_validator.cc
+++ b/source/extensions/http/header_validators/envoy_default/header_validator.cc
@@ -239,6 +239,10 @@ HeaderValidator::validateGenericHeaderValue(const HeaderString& value) {
   const auto& value_string_view = value.getStringView();
   bool is_valid = true;

+  if (!value_string_view.empty()) {
+    is_valid &= value_string_view.front() != ' ' && value_string_view.front() != '\t' && value_string_view.back() != ' ' && value_string_view.back() != '\t';
+  }
+
   for (auto iter = value_string_view.begin(); iter != value_string_view.end() && is_valid; ++iter) {
     is_valid &= testCharInTable(kGenericHeaderValueCharTable, *iter);
   }

I assume this is because of whitespace stripping somewhere. Does anyone have any insight into what's going on here?

Sorry for the delay. This is definitely a non compliant behavior on the part of the current codec nghttp2.
The code you have changed is not yet active in main Envoy build, but will be EOY. I think it makes sense to fix it in the new header validator and not backport into nghttp2, as I do not think this behavior is a security concern.
You can file an issue in the nghttp2 repository and see if its maintainer will consider fixing this bug.
Also, please let us know if this bug is causing production issues for you and we can address it earlier if posible.


Browser Closes Automatically

I am just wondering if there is a way to stop the browser from closing automatically.
Currently, even if I do not call $browser->close(); the browser will close.

This is due to the idle_timeout option being set to 60 by default. You can set it to a higher value, which is recommended if you want to change this value for a production environment, or you can also set it to 0 if you want to disable the timeout, however this should be used only for development environments.

        $puppeteer = new Puppeteer(['idle_timeout' => 3600]);
        $browser = $puppeteer->launch(['headless' => false]);
        $page = $browser->newPage();
        $page->goto('https://example.com');

The browser will still close immediately after it is opened.

This is perfectly normal, PHP ends its script once the page is loaded. If you want to keep the browser open, you will have to keep PHP running, this could be done, for example, with the sleep() function.

Added a while (true); at the end of script hehe.


Wearable: fixes for feedback notification.

Moved "Rate this session" action from notification Activity's onClick to the notification Action - I think this is more intuitive and in line with Wear platform; additionally shows how to add actions to the Wear notification.
Set custom size preset for the feedback notification (SIZE_LARGE). Before that, in case of session title in two lines, room number and speaker name were not visible.

Closing for now. Feel free to re-open.


[v1.2.x] Add static builds

Necessary for https://github.com/conda-forge/libkml-feedstock/issues/2

Thank you very much @h-vetinari , please merge when you wish

@h-vetinari , heroic effort here, amazing.  I'm sorry I am not any help and wasn't able to do this task when you asked for it to be done :(  Do you want this work ported to main branch as well once completed?  I'd like to certainly "borrow" your cleanups here

No worries. I thought it's easier to show by example rather than to try to explain what I mean by static outputs. However, it turned into a bit of a longer exercise, not least because of the many traps for the unwary (like me) in CMake's DSL 🙈
You can of course pick the clean-ups into the main branch (including the conversion into a multi-output recipe if you want; that'll make the other commits rebase more easily), but for now I think we should hold off on adding static builds to 4.0.
libkml is an extreme example of a package stuck on an ancient minizip; as long as that doesn't happen elsewhere, we should be able to just keep moving packages forward as minizip releases new versions.


Config Schema Migrations

migrating multiple packs to config.schema
Status

pack extension, complete

Description
Migrate packs to use new config.schema.yaml configuration
Some packs have unused config.yaml files - these have been removed.
Checklist (tick everything that applies)

[X] Metadata: pack.yaml, icon, structure (required for new packs)
[X] Version bump (required for changed packs)
[X] Code linting (required, can be done after the PR checks)

Nice work - thanks a lot for this!
I will merge this into master and fix some small things directly in master.


Make decorators on variables available within CiceroMark

It would be very useful for transformations to have access to all the decorators placed on variable Concerto types -- allowing them to customise the output based on metadata specified on the variable types.
For example, this could be used by signature blocks to ensure that variables with the @SignatureVariable() decoration are rendered as white text in PDF, and are not editable in the Slate editor.
See: https://github.com/accordproject/cicero/issues/557
/**
 * A Signature block
 */
asset SignatureClause extends AccordClause {

  @SignatureVariable()
  --> AccordParty party

  @DocuSignTab("Text")
  @SignatureVariable()
  o String favoriteColor

  @DocuSignTab("Title", "optional")
  @SignatureVariable()
  o String title

  @DocuSignTab("SignHere")
  @SignatureVariable()
  o String signature

  @DocuSignTab("FullName")
  @SignatureVariable()
  o String name

  @DocuSignTab("DateSigned")
  @SignatureVariable()
  o String date
}

Describe the solution you'd like
All decorators on a Concerto type to be copied into the typed CiceroMark variable node.
Describe alternatives you've considered
None.
Additional context
To access decorators:
const val = myField.getDecorators();

ouch... I completely forgot about this. I think this will have to happen in a future release.

Implemented.


cms-lib/address-book component Refactor

cms-lib/address-book/address-card/address-card.component.scss

Remove SASS variables by transferring its values to the CSS properties.
Add CSS variables and set previous values as fallbacks.
Simplify CSS selector names
Update HTML markup to reflect selector names  (keep cx-prefix)
Remove unneeded/redundant classes.


StringPool#clear

Allow clearing a StringPool, resetting the buffers to their initial capacity

At this point you might as well make a new stringpool, right?

At this point you might as well make a new stringpool, right?

Probably, but that's probably less performant than clearing the existing instance.

Yes, but not if the string pool is a constant.
On Wed, Jun 19, 2019, 13:43 Sijawusz Pur Rahnama<EMAIL_ADDRESS>wrote:

At this point you might as well make a new stringpool, right?
Probably, but that's probably less performant than clearing the existing
instance.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/crystal-lang/crystal/pull/7900?email_source=notifications&email_token=AABL6TXKCX55FN54PXTR5QDP3ILXDA5CNFSM4HZIEBEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYBTBLA#issuecomment-503525548,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABL6TQ2E4XMEJSTJTZUQV3P3ILXDANCNFSM4HZIEBEA
.

This is exactly the same as creating a new StringPool: it creates new HEAP allocations and loses references that must be garbage collected. Maybe it could be interesting to keep the HEAP allocations and only the size was reset to 0 instead.
But, if the only use case is to modify a constant, then wouldn't a class/module property be a better fit?

In our code it's difficult/impossible to pass around the pool. But in some
cases we need to reclaim the memory if the pool has grown too large,
clearing it is a very crude way of doing it. A size limit, and a way throw
away the least used strings would've been ideal..
On Wed, Jun 19, 2019, 13:59 Julien Portalier<EMAIL_ADDRESS>wrote:

This is exactly the same as creating a new StringPool: it creates new HEAP
allocations and loses references that must be garbage collected. Maybe it
could be interesting to keep the HEAP allocations and only the size was
reset to 0 instead.
But, if the only use case is to modify a constant, then wouldn't a
class/module property be a better fit?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/crystal-lang/crystal/pull/7900?email_source=notifications&email_token=AABL6TSQD6VVPP2TRRGK5NDP3INT7A5CNFSM4HZIEBEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYBUHSI#issuecomment-503530441,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABL6TU5HAFCJZ5L46XWQMTP3INT7ANCNFSM4HZIEBEA
.

Also, I created StringPool exclusively for the compiler but poorly made it public, then I though about using it in JSON... so the API is not the best one. Just some thoughts on this.
(this is kind of similar to what happened with Markdown)

@carlhoerberg I meant to replace an App::POOL = StringPool.new definition to:
module App
  class_property pool = StringPool.new
end

Then change calls to POOL for App.pool, which behaves the same, except that the later but can be changed (unlike constants, which IMO should be thought as immutable).

Fair enough


[ENH] use async instead of multi processing

won't fix in the short term, but if someone sends a PR we'll accept it


Add erc20 support to the chain service

This PR is extends https://github.com/statechannels/statechannels/pull/2641.

How do you think we move from here to support a third or fourth kind of asset type? For example, an application may have its own custom asset type it wants its users to use. Will it need to edit the code of the chain service? Perhaps the methods to provide an asset holder address and ABI should be public on the service?

How do you think we move from here to support a third or fourth kind of asset type? For example, an application may have its own custom asset type it wants its users to use. Will it need to edit the code of the chain service? Perhaps the methods to provide an asset holder address and ABI should be public on the service?

The asset holder address (or addresses) is an argument in both fund channel and register channel APIs. It's a good question on how to specify an ABI for those addresses. To be honest, I haven't given that much thought yet...


python312Packages.gardena-bluetooth: 1.4.2 -> 1.4.3

Diff: https://github.com/elupus/gardena-bluetooth/compare/refs/tags/1.4.2...1.4.3
Changelog: https://github.com/elupus/gardena-bluetooth/releases/tag/1.4.3
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 339458 run on x86_64-linux 1

5 packages built:

home-assistant-component-tests.gardena_bluetooth
python311Packages.gardena-bluetooth
python311Packages.gardena-bluetooth.dist
python312Packages.gardena-bluetooth
python312Packages.gardena-bluetooth.dist


Unable to get mTLS working for my services

I have been migrating the popular guestbook example to run on istio mesh.  The current set up is redis runs out side of mesh and guestbook runs inside of the mesh.  I also created guestbook-v2 to modernize the app to talk to a watson service via a new service called analyzer to add machine learning capability.
The issue I have is when mTLS is enabled for the mesh, I can't get guestbook working, even if I put annotation to disable mTLS on both the guestbook and analyzer service.
https://github.com/linsun/examples/tree/master/guestbook-go
The only way I can get it working is to disable mTLS for the mesh.
Since mTLS annotation is supposed to be specified on the target service, I'm expecting to get this working without the need to turn off mTLS per service.
The traffic flow is guestbook-> redis or guestbook->analyzer
I'm using istio 0.7.1

@wattli could you help look into this or let me know what data i need to collect to help you debug this?  cc @elevran

If possible, please provide the reproduce steps. That would be useful for diagnose

and feel free to ping me on slack :)

Actually can you try a newer version, I tried some simple demo with auth policy yesterday and it works fine. The version i used is: https://gcsweb.istio.io/gcs/istio-prerelease/daily-build/release-0.8-20180424-09-15/

Sure!  The instruction is in readme, basically you deploy redis outside of the mesh and guestbook-v1 in the mesh, when mTLS is enabled in the mesh.
Q for you @wattli, in this case, do I need to disable mTLS for the guestbook service?
I'll try the newer build.

Assigned back to Lin, and feel free to close it.

Created another issue to better track this problem: https://github.com/istio/istio/issues/5296, close this one now.


Fix issues with native extension initialization skipped steps

This fix solves native extension initialization by allowing some steps (from InitializationLevel) to be skipped. This happens in builds where there are no editor for example, I which case the INITIALIZATION_LEVEL_EDITOR is skipped.
I our case, it prevented us to load native extensions when compiling in release mode.
This solves the issues the following way:

Adds a static constant skippable array to the NativeExtension class to define which initialization steps are skippable, and check that no non-skippable step is skipped when calling initialize_extensions.
Replace the current level of initialization tracker (the level variable) by the initialized_levels array. This array contains the list of all initialized steps, and uses the initialized_level_index to keep track of the last initialized level index.

This does fix the crash we have in release with a GDExtension.

FYI this maybe should not be merged yet, there seems to be more going on, or at least another issue that impacts this as well, we may need to investigate this more.

Superseded by #58629.


Port defaults to 2015 when using tls self_signed

1. What version of Caddy are you using (caddy -version)?
0.10.2
2. What are you trying to do?
Use Caddy with tls self_signed, just the domain as the address
3. What is your entire Caddyfile?
example.com {
    ...
    tls self_signed
}

4.
Dockerized
5. Please paste any relevant HTTP request(s) here.
N/A
6. What did you expect to see?
Caddy listening on ports 80 and 443
7. What did you see instead (give full error messages and/or log)?
Caddy listens only on port 2015
8. How can someone who is starting from scratch reproduce the bug as minimally as possible?
See Caddyfile above

In the docs, there's this line:

The default port is 2015 (unless the site qualifies for automatic HTTPS, in which case it's changed to 443).

The workaround is to prepend the hostname with https://.
I would argue that Caddy should consider self_signed as "automatic HTTPS", for the purposes of port assignments, if no port is specified. I'm pretty sure the same behaviour can be observed when using an existing cert and key. I think it's inconsistent/confusing behaviour that it works like this. Is there a specific reason why it only defaults to 80 and 443 on automatic TLS and not on other modes?

I figure the simplest description of "automatic" is ... automatic: it just works, you don't do anything. Once you start configuring it, it's not automatic. If you are configuring it, you "must" know what you're doing, so we hand the reins over to the user. This is simpler than drawing a line halfway in the spectrum and saying, "this and this and this is automatic, but this and this and this isn't."
If you touch it, it's not automatic anymore.
Nice question!

Fair enough re: definition of automatic, but is there a reason why other TLS modes don't default to port 80->443 if https:// is not included? I just think that's a poor default if TLS is on at all.

is there a reason why other TLS modes don't default to port 80->443 if https:// is not included?

Some people don't want to run TLS on port 443. Or 2015. We have to choose a port, might as well just keep it simple and say the default port is 2015.

I'm gonna leave my original message intact below, just because I'd feel bad deleting it all after having typed it up 😛
I figured out that the -port cli arg lets me do exactly what I wanted, replace the default port from 2015 to 443. I think this could be better documented here, so I'll probably make a PR for that later.

Alright. Regarding your following point:

If you are configuring it, you "must" know what you're doing, so we hand the reins over to the user.

That's what I think when I expect the default to be 443. That's the default HTTPS port, I feel like it makes more sense to default to the "default", and let users who actually want a different port to just pick it in that case.
A bit of an aside, is there a reason for the default port being 2015 and not 80? I don't see any mention of that in the docs, I feel like that could use some explanation somewhere.
Wait - never mind, disregard the above question, I think I guessed the answer: 2015 is default because usually low ports can't be bound if the user doesn't have root. Follow-up to that, could we default to 80/443 if Caddy does have the low ports capability?
To clarify a bit more why I'm bringing this up - I use environment variables to configure my application. Caddy runs in Docker, I expose ports 80 and 443 in Docker. I allow configuration of TLS via import, if you remember. I just have the expectation that Caddy will bind to ports 80/443 regardless of the TLS config.
Another possible idea, a command line arg to force it to default to 80/443 unless specified otherwise?

I totally expected Caddy to listen for both 80, 443 when using self signed. I was migrating from nginx to caddy on our local dev-env but this just made it not "automatic" and not "just works". I have it running on docker too.

Even worse, caddy defaults to :2015 if given a valid certificate manually (using tls /path/to/cert.crt /path/to/key.key). This is completely unexpected, and a pain to work around (You have to split the blocks to http:// and https:// and manually set up the http -> https redirect)
https should always default to 443

Why this issue closed ...

Sure, but you kinda can't avoid configuring it! As soon as you start creating your first caddy configuration file. Anyway this mess could be avoided if the autoredirect was easier to add as an option. One configuration key and one sane default value would make it so much better.
For them being we went back to nginx due to the way issues are handled in this project.
Hopefully that gets resolved as the project mission is nice!

@Zakay

Sure, but you kinda can't avoid configuring it!

I was referring to the HTTPS-related configuration, such as the site's TLS settings. I suspect the vast majority of users never have to touch that configuration (although telemetry will tell us for sure).

For them being we went back to nginx due to the way issues are handled in this project.

What would you like to see different about it?

@mholt imo the problem is really this:
With auto https (letsencrypt etc), it's super easy to set up a site properly with http redirection
Without it, it's a nightmare, you need 2 separate website blocks, a manual redirect, all the ports set manually.
imo this block:
example.com {
  tls /cert /key
  path /path
}

Should "just work" as expected, as if the tls command wasn't there (same with self-signed)

@mholt The current behavior is really annoying, why should the use of self-signed certificates disable the automatic HTTP->HTTPS redirection?
Consider following scenario: You manage a webserver and use a separate staging server to test the configuration before deploying it to the production server. The production server is accessible from the internet, but the staging server not (could be a local virtual machine). So for the production server, Caddy should obtain certificates from Lets Encrypt, but on the staging server it must use a self-signed certificate.
This setup is currently not possible with Caddy, because as soon as you add tls self_signed to the Caddyfile, HTTPS->HTTP redirection is disabled and port is changed to 2015! So the behavior of production and staging setup is completely different, just because of the tls self_signed statement.

@mholt

I was referring to the HTTPS-related configuration, such as the site's TLS settings. I suspect the vast majority of users never have to touch that configuration (although telemetry will tell us for sure).
And I was referring to the kind of thinking regarding this matter. Changing the delivery method of certification should not affect other features and default settings such as auto-redirect and port config.

What would you like to see different about it?

For on thing: Things like these, the expected behaviour is very far from what you get. Like this cert thing.  Doing something totally unexpected should require good motivation from the person suggesting it. Instead people are being asked to motivate and give examples on why they want something to work as expected. Default behaviour is port being 443,
Secondly: When we experience issues with php-fpm and caddy, things weren't investigated by the team at caddy. Motivation behind it was something along "we don't use php". And of course I get it, if you don't have any production environment with php, its going to be difficult for you to reproduce the error. But I would have loved if I got debug binaries or anything to help us find the issue and solve it. I would gladly help, as I love the core ideas of this project. I still do, but currently can't use it.

Here is how I do it with custom ssl.  Specify both ports in the hostname part, then add that redir if they aren't on ssl.  That way you don't have to do two blocks.
*.example.com:80, *.example.com:443 {
  root /app/html
  errors stderr
  log stdout
  cors
  gzip
  redir 301 {
    if {scheme} not https
    / https://{host}{uri}
  }
  tls {
    load /app/ssl
  }
  proxy / api {
    transparent
    except /app
  }
}

thanks @dusty, certainly better than how i've been doing it but I still think it that should be the default

I agree here, default http -> https should be kept on auto_signed, here is how i'm doing:
<IP_ADDRESS>:80, <IP_ADDRESS>:443

redir 301 {
    if {scheme} not https
    / https://{host}{uri}
}

tls self_signed
root /data/public
index index.php
gzip

fastcgi / app:9000 php

rewrite {
    to {path} {path}/ /index.php?{query}
}

log stdout
errors stdout

@mholt Please reconsider this. I like how simple it is to just have it do the redirect and everything. I don't like having to manually set that up.


[fabric 1.20.1] ui bug when using vivecraft

whenever i interact with the alloy forge while using vivecraft it displays whatever i last looked at as a still image, with me only being able to look at the bottom of the screen

it works normally whenever i disable vr mode.

I suggest reporting this to ViveCraft, as I do not know how to even begin debugging VR without owning it.


Make clock settable for testing date suppliers

Unit testing validation rules using absolute dates is not straightforward because they are provided by LocalDateSuppliers with static methods.
This PR adds methods to LocalDateSuppliers for overriding system clock to be used in date suppliers.

LGTM


Added .idea to gitignore for IntelliJ

Closing because of lack of activity. Feel free to reopen or create a new PR for this.


update docker and docker-compose versions

update docker and docker-compose versions in presetup-mac

Hi @gaboesquivel. These updates are already in place in the develop branch (https://github.com/blinkreaction/boot2docker-vagrant/blob/develop/presetup-mac.sh).
I will cut a new RC soon, once some other work is finished in the develop branch.


Comments in between if statement branches are not read or not printed

See e.g. XSL0301_script.lua

Fixed in 6fe27fca0be64271fda21e24a45af8a6ab163fb7


Rename access methods for Godot 4 Beta 15 and onwards

This fixes a breaking change in beta 15 renaming a number of key functions.

Seeing beta 15 is out, should we merge this?

Certainly. Beta 15 looks to be quite good, so I doubt anyone will be sticking with an earlier beta.


Notification center - origin

Description
This pull request is the origin of the Notification center. It allows to launch an Express server from the index.ts file located at the route of src/ to the desired port, providing 3 routes :

/updateNotificationPreferences to create a new user in the MongoDB database (using Mongoose module and the function tools created inside database/index.ts) according to the following shema :

    // Define the schema for the user collection
    const userSchema = new mongoose.Schema({
    id: { type: String, unique: true },
    channels: {
        telegrams: [String],
        emails: [String],
    },
    });

from the model.ts file.

/sendNotification is used to send a notification to the services referred by the user via his handles stored in the database to the above request. (notification.ts)

Everything that concerns the management of services is located in the plugin/ folder. In this PR only the Telegram plugin is functional :
telegram.ts get the ids of the chats corresponding to the handles informed by the user by parsing his messages and then send to these ids the message of the notification.
⚠ Don't forget to set the telegram API_KEY inside .env file (check .env.exemple)

/ping which just allows to check the status of the server

Changes
This PR is the origin of the Notification Center.
Checklist
Everything was done according to the issues rules. Still needs to be dockerized and isValidPayload has been commented and will wait for the watcher implementation to be fully tested correctly.
Requirements
Secrets :
DB_URL=<mongodb://ip:port/app>
PORT=<PORT_WHERE_THE_SERVER_SHOULD_RUN>
MAX_TIME_DIFF=<PAYLOAD_DEADLINE>
TELEGRAM_TOKEN=<TELEGRAM_API_TOKEN>

Closes https://github.com/LedgerHQ/ledger-fresh-management/issues/37
Closes https://github.com/LedgerHQ/ledger-fresh-management/issues/38
Closes https://github.com/LedgerHQ/ledger-fresh-management/issues/39
Closes https://github.com/LedgerHQ/ledger-fresh-management/issues/40


Package up test suite

Currently, our test suite relies on a specific version of gcc (5.4.0). The properties cbat is checking is based on binaries and not C programs, so we should package up these test cases.

Closed with #281.

Closed with #281.


AvalancheForecastZoneCard: render 'general information' better

Similarly to 'no rating', we just want the words here, not the numerical value.
Signed-off-by: Steve Kuznetsov<EMAIL_ADDRESS>Follows up on #58

@floatplane second commit in here did some bash surgery to fix goofy quotation breakages


Automated Package Upgrade

:tada: This PR is included in version 12.4.132 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Documentation fix:

I couldn't find the documentation in this github repo, or a repo that had the documentation, so please redirect me there if it's online somewhere:
in https://docs.keydb.dev/docs/first-test/ ... I think all occurances of redis have been replaced with keydb... but this causes sentences such as:
If you have never used KeyDB or KeyDB before but want to play around with some commands, install KeyDB then run the server.
Which are clearly wrong

Thanks, I updated the lines you pointed out.  We do plan on reviewing in greater detail, and this  repo will be growing substantially in the coming months. I have made an official repo for KeyDB-docs where you can see and edit the markdown documents used on the website. Feedback and pull requests are welcome and appreciated
https://github.com/benschermel/KeyDB-docs


Add workspace optimizer

Misses a lot of documentation but should be basically feature complete.

Open problems:

Removing the executable bit does not seem to work
The guest writed unwanted data to the host:     contracts/cw1-subkeys/contract_artifacts/
     contracts/cw1-whitelist/contract_artifacts/
     contracts/cw1-whitelist/foobar/
     contracts/cw20-base/contract_artifacts/
     contracts/cw20-escrow/contract_artifacts/

Docs missing


Spark JVM Memory tuning

We're getting OOM'd on workers because of excessive memory usage. By default, spark uses total RAM - 1GB; on a c3.8xlarge, it computes this as 51200M for the worker. This doesn't leave enough for python processes/OS. Take this down to 47200M by setting the SPARK_WORKER_MEMORY environment variable, as documented here

Not sure if that's enough, we'll see

Okay, I'll get on this right now!

@wearpants okay check it out! Please let me know if this works and I'll close this issue. :) Thanks!
https://github.com/opensyllabus/osp-pipeline/commit/83f9d1f92abe7c17a4a80db021934eb73c6638d8

Yeah lets doument. Actual value Should probably live in same place we set instance type& number of nodes, and then propogate through Ansible variable

Uh oh having some issues but we're getting to the bottom of it...

Should probably revert or comment out in light of #99

eh, let's leave it, a little free ram won't hurt


Add option to invoke callout prior to each instruction

This pull request introduces a new debugging feature for the ubpf virtual machine, allowing a debug function to be registered and invoked before each instruction execution. The most important changes include adding the debug function registration and invocation, updating the VM state structure, and creating a new test to verify this functionality.
Debugging Feature Enhancements:

Added a new typedef ubpf_debug_fn and the function ubpf_register_debug_fn to ubpf.h to allow the registration of a debug function that gets invoked before each instruction. (vm/inc/ubpf.h)
Updated the ubpf_vm structure to include the debug function and its context. (vm/ubpf_int.h)
Modified ubpf_exec_ex to call the registered debug function before executing each instruction. (vm/ubpf_vm.c)
Implemented the ubpf_register_debug_fn function to handle the registration and validation of the debug function. (vm/ubpf_vm.c)

Testing and Documentation:

Created a new test source file ubpf_test_debug_function.cc to verify that the debug function is correctly registered and invoked, capturing the VM state before each instruction. (custom_tests/srcs/ubpf_test_debug_function.cc)
Added a test description file ubpf_test_debug_function.md to document the purpose and functionality of the new test. (custom_tests/descrs/ubpf_test_debug_function.md)
Included a test input file ubpf_test_debug_function.input to provide the necessary data for the new test. (custom_tests/data/ubpf_test_debug_function.input)

Summary by CodeRabbit

New Features

Introduced debugging capabilities within the U-BPF virtual machine, allowing users to register debug functions that can inspect the VM state before each instruction execution.
Added a new test utility to verify debug call-out functionality.

Documentation

Added a "Test Description" section to clarify the purpose of the new debugging test.

Bug Fixes

Enhanced the execution process to incorporate debugging functionality, improving overall testing accuracy.

@coderabbitai full review


enable rustls on windows

bump up crate version (patch)

done

Maybe it will fail to compail due to:
https://github.com/infinyon/future-aio/blob/e97957ef12d294ef5dad06e8cfd0812a83bbeb1f/src/rust_tls.rs#L107 ?

Maybe it will fail to compail due to:
https://github.com/infinyon/future-aio/blob/e97957ef12d294ef5dad06e8cfd0812a83bbeb1f/src/rust_tls.rs#L107
?

Maybe I fixed it?

probably should add windows to CI, doing it now

bors r+

closes https://github.com/infinyon/future-aio/issues/207

@sehz could you check that dependency update?

bors r+

closes https://github.com/infinyon/future-aio/issues/108

bors r+

bors cancel

bors r+

bors r+

bors r+


Invalid OAuth token?

Starting yesterday, I noticed my app wasn't getting track information anymore. When I debugged my app, I saw that Track from a GetStatus was null. Tried it with the sample app and noticed I was getting the following response when calling SpotifyAPI.Local.RemoteHandler.GetNewStatus():
"[ {\n\"error\": {\n  \"type\": \"4102\", \n  \"message\": \"Invalid OAuth token\"\n}, \n\"version\": 9, \n\"client_version\": \"<IP_ADDRESS>.g58007b4c\", \n\"running\": true\n} ]"
Anyone else seeing something similar? Wondering if it's something just on my side.

Did you change you're antivirus software ? :)

Did you try restarting both, the spotify client and the web helper? I came across a similiar problem but a restart fixed it.

Yep, tried that. Also tried rebooting. Even tried it on a different computer on a different network and still got the same result. Will keep investigating when I get a little more time though.

I've spent some more time trying to debug and no luck. I keep getting the same errors. Next thing I'll try is with a different user account to see if it's somehow specific to my Spotify account. If anyone has any ideas, I'm all ears. @JohnnyCrazy, would you prefer I close this issue or leave it open in case anyone else runs into it or I end up figuring it out?

@gusper if its not fixed or doesn't have any explanation, you should leave it open :wink:
Sadly enough i'm unable to do tests myself with the local player since my antivirus blocks connection.

@gusper We can leave it open I guess
Sadly, I can't reproduce it. Could you give me the values of csfr-token and oauth-token? They should be both inside the RemoteHandler Query Method.

Appendix: Instead of writing the values, you can also just confirm they look like regular tokens :smile:

Here's what they look like:
OauthKey: "NAowChgKB1Nwb3RpZnkSABoGmBAByAEBJeIG9VYSFA6cvYavFSCObbTOWg1HHR2vks5E"
CfidKey: "09e93d471595dd0564aff816cfef58aa"
If you want the exact values, I can email them to you. Do they look as you'd expect?

Well, somewhat good news I guess. Both of the machines I was experiencing the problems on (which were on entirely different networks) all of a sudden started working again today. The OauthKey looks like it's different than I had previous while the CfidKey is the same I had.
Going to go ahead and close this out now. Let me know if you have any questions though that I could help answer...

@gusper  Are the OAuth tokens both of the same "structure"? Like same length etc.?

Yes. Actually, parts of it are exactly the same (e.g., the beginning of the token) while other parts are completely different.

Mh, then it's probably not the oauth tokens... Maybe it's just the mood of the spotify player :wink:

Will try to be nicer to it going forward now that I know it's a little sensitive. :)
Thanks for the help!


Updating tests to handle nutrient Components

Sub-task of #14
As part of #14 the unit tests for UNIFHY need to be updated to expect nutrient components before any development on UNIFHY to add these components takes place.

There is some question on whether or not to make the nutrient components mandatory or not for the tests (and thus for the user).
As UNIFHY allows NullComponents, which essentially remove the component from the Model, I think for now we will require all 6 components be provided for a Model, as the user will have the option of making the nutrient components NullComponents if they do not wish to model nutrients.

I am in the process of adapting this from hand-written notes so it's a bit of a mess right now, this will change...
The following changes need to be made to the tests for them to require 6 components instead of 3:
test_model.py:

Simulator.from_scratch(): Add 3 extra arguments defining the 'category' of the nutrient Components in the function declaration and to all calls of the function, and edit the body of the function appropriately to process these [BE MORE SPECIFIC]
Simulator.setup_model(): Add 3 extra arguments defining the nutrient Component 'kinds' to the function declaration and function calls, and add 'get-dummy. component' calls for nutrients Components as for the WaterEnergy Components to the body of the function
Add nutrient component folders in unifhy/tests/tests/components for nutrient components, like for the WaterEnergy components. Each folder needs to contain an
init-.. py file with just from. dummy import Dummy in it.
a component_name.py file that replicates the equivalent WaterEnergy Component in all but name (except perhaps for the dummy C and dummy Fortran bits which can be done later?)
The class that Dummy subclasses (one of the NutrientSurface, NutrientSubsurface or NutrientOpenwater classes) will not exist yet causing the test to fail as expected.

Add some WE4N transfers into the dummy component files.
(or some for later in the name of keeping the changes as
small as possible in line w/unit-testing Pandian
Create dummy data to fit the tests. I e. w/the inputs
specified in the dummy nutrients components. Copying the
existing files e adding/removing/changing the names of the data
variables as needed should do the trick.
Add these/any new input variables to the 'science ishdriving-
data-daily too. no Modify calls to to unde arguments
that are the
nutrients components, e the
names identifier to include these as well.
unity. Model
Extend
clean-up. dump-files C)e clean-up. record-files () in test. M py to
include nutrients components directories files
3
Add simulator-2. model. nutrient layer
initialise-states-from-dump lines (triggering an expected
FAIL in
tests pinup-dump. init spiny
)
Add nutrient model comps (as model. nutrallayer) in
check-fuel.
conditions
add transfers tobop over transfers in
check-exchangers
transfers
The transfers add blown components.
0 add are
any you
involving the new nutrient components, defied in tests/tests/
components/layernamel dummy. py
Add nutrients components to you files in tests
lionfigurations, copying the time e space setup from the WE
comps, and the transfers/datasets/ params ete. from the
dummy comps.
we
Add in c d n in selector for the 3 nutrient components e the
N comps to ideepcopy following loop, add a sub est, calls I add
assertive compare states test for N comps in
Advanced Test Model.test-set up, spiny, simulate-resumeall(
still in test-d. py).
In the Basket Model vision, add in 3 dee.pespyknes.se
compare-states lies for the mutant comps.
In Busiest Model. check-fat cords 0 add in the 3 N comps to
theloop over components
Busiest Model. check-records 4In
do the same.
Add u c d n in doe selector
N comps for 3 N comps i add to subtlest callfor
Add 'Python, "Fortran, 'c' for 3 N comps is doe selector, add
them to subsequent for loop e add to subtlest call for
test-setup, simulate ()
test-setup, simulate vinous-sources C)
test-component. PyIn the various dictionaries, add entries for
the 3 N components
Matching the setup of the WE comps should do the job for now,
though . might want to build an additional test for doffs blown
matching comps of Weed in the future. Might need to create
new dummy-subsurface-parameter.... or similar, files.
Add unphy. component. nutrient compname to the sibling
class loop intest. instantiation-datacomponentCausing an
expected FAIL as
these won't exist get
Same
test. instantiation-mleomponettfor
test-clock.py
Add the 3 mutant comps to the
test-clock.int C)of s e to the assert Equal statements below. this
will cause an expected FAIL as Clock will not have been set up
to expect N comps.start
Repeat for
Repeat for
test-clock iteration ()test-clock-incompatible,time-domains ()
Repeat for
test-clock-incompatible,dumping-frequency (
Ed-a
{
lap, boot like a e z
correspond to the test resolutions set up in test-time. py. If testtime-
py is changed, will need to change the above variables
here
tests record PY
add in the expected records for the snow nut. comps for the
same-t e diff-t tests, like for the Wt comps. This will depend on
the values in the input files easy processing in the dummy
comps

Regarding the structure of the new dummy components. I have settled upon the setup shown in the diagram below. The key features are:

Replication of the transfers between WaterEnergy Components in the Nutrient Components, with different names assigned to avoid ambiguity and errors
The addition of two transfers between the WaterEnergy set of Components and the Nutrient set of Components.

The two transfers added are:

From the Nutrient SurfaceLayer Component to the WaterEnergy SurfaceLayer Component (transfer_h)
From the WaterEnergy OpenWater Component to the Nutrient OpenWater Component (transfer_p)

To accommodate these transfers the following edits were made to the WaterEnergy Components:

OpenWater Component: transfer_p set to first division and current timestep of state_a
SurfaceLayer Component: new output_y created which is transfer_h multiplied by the current timestep of state_a

The Nutrient Components are copies of the WaterEnergy components with only the transfer names changed as shown in the diagram below. The edits then made to accommodate the two new transfers added are:

OpenWater Component: transfer_p added to output_y
SurfaceLayer Component: transfer_h set to first timestep of state_a multiplied by ancillary_e

Hi Matt,
For the dummy files, you may find these scripts useful (I hope): https://github.com/hydro-jules/data-generators
As you will have noticed, I tried to keep the "equations" in the dummy components as simple as possible while using each input/transfer/parameter/state/constant at least once. This meant that the time series the components were producing were mostly made of integers or at least rational numbers (see https://github.com/unifhy-org/unifhy/blob/4105932ed7dfec34d428c1f2d2f85ec25ea522ed/tests/tests/test_utils/test_record.py#L10C6-L59 and https://github.com/hydro-jules/data-generators/blob/a4667021f7f0e4c9edf32eacd40f0d3394ce71c9/generate_dummy_cf_substitute_data.py#L35-L66). This was to avoid having to deal with floating point precision when comparing what was produced by the components vs. what I was expecting on paper. Since you are necessarily adding transfers between WaterEnergy and Nutrient, you will have to increase the "complexity" of these "equations", so you may not be able to avoid ending up with irrational numbers, but I guess it is not too bad.

Thanks Thibault :)
I was aware of this issue of possible irrational numbers, so I tried to keep the extra calculations as simple as possible but didn't actually check what the records would now come out as.
That's my next job xD
It looks like I don't have access to the data-generators repo - I'm just getting a 404

Addding in the expected records for the 3 nutrient Components for the
same_t and diff_t tests to test_record.py took a long time due to the need to calculate 16 model timesteps by hand, with 6 components.
To ease this process for future development I intend to create a script that users can trust to calculate these values for them once they input dummy components. See issue

Test updates (finally!) complete.
Next step is to analyse which of these changes will result in the errors when the full framework is run and make changes to the framework to allow the tests to pass.

The following changes need to be made to the tests when adding a new Component to the framework:
test_model.py:

[x] Simulator.from_scratch(): Add an extra argument defining the 'category' of the new Component in the function declaration and to all calls of the function, and edit the body of the function appropriately to process these
[x] Simulator.setup_model(): Add an extra argument defining the new Component 'kind' to the function declaration and all function calls, and add a get_dummy_component() call for the new Component as for the other Components to the body of the function
[x] Modify all calls to unifhy.Model() to include a new argument that is the new Component, and modify the identifier (within the function body?) to include this as well.
[x] Extend clean_up_dump_files()  and clean_up_record_files() to include the new Component directories and files
[x] Add  simulator_2.model**.xxxxxyyyyy**.initialise_states_from_dump lines to the test_spinup_dump_init_spinup() test
[x] Add the new Component (as model.xxxxxyyyyy) in check_final_conditions()
[x] Add transfers to loop over in check_exchanger_transfers(). The transfers to add are any added between Components involving a new dummy Component defined in tests/tests/components/componentname/dummy.py
[x] In AdvancedTestModel.test_setup_spinup_simulate_resume_run(): add in 'c', 'd' and 'n' in the 'doe' (design of experiment) selector variable for the new Component and the following loop, add the new Component to subtest and deepcopy calls, and add assertTrue() to compare states tests for the new Component.
[x] In BasicTestModel.test_setup_spinup_simulate_resume_run(), add in a deepcopy line and compare_states line for the new Component.
[x] In BasicTestModel.check_records() add in the new Component to the loop over Components
[x] In AdvancedTestModel.test_setup_simulate(): add 'c', 'd' and 'n' to the 'doe' selector variable for the new Component and add to subTest call
[x] In AdvancedTestModel.test_setup_simulate_various_sources(): add 'Python', 'Fortran', 'c' for the new Component in the 'doe' selector variable, add them to subsequent for loop and add to subTest call

test_component.py

[x] In the various dictionaries, add entries for the new Component.
[x] Add unifhy.component. **XxxxxYyyyy** to the substituting class loop in test_instantiation_datacomponent()
[x] Do the same for test_instantiation_nullcomponent()

test_clock.py

[x] Add td_x, exp_x and exp_idx_x to the start of the TestClock class definition, corresponding to the time domains of the new dummy Components, where 'x' is a currently unused letter of the alphabet in the other similar variables
[x] Change the equivalent variables for the dumping to have the 'y' suffix (i.e. td_y etc.), where 'y' is the next letter in the alphabet to the suffix used in the above bullet point, making sure to also change all occurences of them (in can also just be any unused letter of the alphabet)
[x] Add the new Component to the start of test_clock_init() and to the assertEqual statements below, using the variables created in the above 2 steps.
[x] Repeat for test_clock_iteration(), change/add to the letters in the loop over clock, and the corresponding if and append statements
[x] Repeat for test_clock_incompatible_time_domains()
[x] Repeat for test_clock_incompatible_dumping_frequency()
Note: td_{a,b,c,d,e,f,x,y} are derived from the get_dummy_timedomain function in test_time.py . If the function is changed, then td_a,b,c,d,e,f,x,y and thus exp_bool_a,b,c,d,e,f,x,y and exp_idx_a,b,c,d,e,f,x,y variables will need to be changed accordingly.

test_record.py

[x] Add in the expected records for the new Component for the
same_t and diff_t tests, like for the other Components. This will depend on the values in the input files and any processing in the dummy nutrient Components.

Dummy component files

[x] Add in a new Component folder in unifhy/tests/tests/components like for the other Components. Each folder needs to contain:

[x] an __init__.py file with just from .dummy import Dummy in it.
[x] a component_name.py file with some dummy transfers in it. The class that Dummy subclasses is the new Component class introduced in the unifhy/component.py
[x] a dummyfortran folder with blank __init__.py file and a similar to the other Components
[x] a dummyc folder with blank __init__.py file and a Makefile & setup.py files similar to the other Components

[x] Do the same for the Fortran versions, which involves:

[x] a dummy.f90 file in the dummyfortran folder which:

[x] has separate variables for each state timestep
[x] x, y and the outputs are also in the subroutine definitions and declared within the subroutines
[x] All other inputs used in them declared at the start of the subroutines
[x] The same calculations as done in the python version (without the UnifHy routing function if it is used in the python version)

[x] Adding a Makefile in the dummyfortran folder, this is identical across Components
[x] Adding DummyFortran classes within the new Component dummy.py file. These pass the inputs and inwards to and receive the outputs and outwards from the Fortran subroutines, pass these to/from the exchanger, and perform any routing that needs doing, which can only be done from a python file.

[x] And for the C versions which involves:

[x] A dummy.c file containing the C code for the initialise/run/finalise functions
[x] A dummy.h file that just contains the declaration statements for the functions
[x] A dummy.pyx file that wraps the C initialise/run/finalise functions in the dummy.c file in python/cython code
[x] Adding DummyC classes within the new Component dummy.py file which call the initialise/run/finalise functions defined in the dummy.pyx files.

[x] Add the dummy Fortran and C components to the Makefile in unifhy/tests/tests/components
[x] Add new Components to the yaml files in tests/configurations, copying the transfers/datasets/params etc. from the dummy Components.

Dummy data files
There also needs to be:

[x] dummy data to fit the tests. I e. the inputs/outputs specified in the new dummy Components.
[x] Substitute data for when Components are tested as Data Components. These files need to have the outward transfers of the new Component in them. All the temporal and spatial resolution permutations also need to be covered, these are shown at the top of the test_component.py file. Thibault's data-generators scripts can generate the data files once the values and permutations are input.

unifhy.model

[x] Add

xxxxxyyyyy: tests.components.xxxxxyyyyy.dummy

to the docstring of the from_yaml method
xxxxxyyyyy  is the name given to the component by the name of the folder the dummy.py file resides in