Add new rule comment-style

This rule enforces eslint-disable-line and eslint-disable-next-line comments to be line comments instead of block comments.

Hi @remcohaszing!
Since this repo is unmaintained, you might want to re-open this PR in the @eslint-community fork https://github.com/eslint-community/eslint-plugin-eslint-comments
For more info about why we created this organization, you can read https://eslint.org/blog/2023/03/announcing-eslint-community-org

Closed in favor of https://github.com/eslint-community/eslint-plugin-eslint-comments/pull/132


runtime, cmd/compile: possible bad pointers with map[struct{}]struct{}

Many places in the hashmap implementation, we calculate key/val pointers that might point at the end of the bucket if both key and value have zero size.
Given how rare this is, I'd hate to complicate the map routines with checks everywhere. One possible fix is to have special mapfast routines just for maps with zero-sized keys and values, which would then generally be dead-code eliminated.
Other ideas?
@randall77

Maybe it'd suffice to set maptype.valuesize to 1 for such maps.

special map fast routines sounds good. Since this should be very rare they will not usually end up in the binary and they dont complicate existing code.
I was also looking into making other special zero sized elem methods for other make methods since they are rare and we can slim down the common code paths:

growslice
makechan (for zero sized elems and queue)
In both situations we have simpler overflow tests too since there are less multiplications involved.
But i am currently working to slim down and get the current methods work with overflow correctly before splitting or duplicating them.

So specializing map functions sounds good.

So I totally forgot that we pushed the overflow pointer to the end of the bucket exactly to solve this sort of problem, and the one in #21459 .

Oh right. Yay! For future reference, the original bug was #9384 and fix was CL 1869.


Add swipe commands

Proposed changes
Adding native commands to swipe the center of element to the edge of the window
Checklist

[ ] Checkstyle and unit tests pass locally with my changes by running gradle check chrome_headless firefox_headless command
[ ] I have added tests that prove my fix is effective or that my feature works
[X] I have added necessary documentation (if appropriate)

Since it's just using actions under hood - looks like it can be just a custom command not a part of regular code base.
You can see example here

agree with @rosolko. Also I believe that such swipe won't work in all cases - so it is project specific


The app doesn't properly on android 8.0 Oreo

The current version of the app doesn't work properly on android 8.0 Oreo or higher since it doesn't have permission to install apps from unknown sources.
It's a new permission that was added at android 8.0.
To fix this problem, you have to add the following tag to the manifest:
<uses-permission android:name="android.permission.REQUEST_INSTALL_PACKAGES" />

I've downloaded the source code and added this tag and managed to run correctly this app on an android tv 8.0 emulator using android studio.
I hope you upload a new release for this app with this fix, It would be wonderful!

I dont think anyone is working on this anymore.
@mataniss I tried to disassemble it with apktool and add the line you mentioned in the manifest, but it will not re-build, even without adding the line. Do you have a working copy I could get from you?
@Fleker I have a few apps that do not show, like yatse, but yet I can create a lin to them in Kodi or something and they run fine. I really would like to be able to create the shorcuts and hope someone can fix this or something else soon!

@Wanderer189
Download Android studio and GitHub desktop and either download the project from the browser or use GitHub desktop to sync a copy to you pc and launch Android studio to recompile the app with the change

Could anyone provide a working apk here for people like me who don't know what manifest is and how to build and compile apps on their own? Would be awesome!
Right now some apps worked but most for most of them tvapprepo fast closes on me after selecting "create shortcut".

Could anyone provide a working apk here for people like me who don't know what manifest is and how to build and compile apps on their own? Would be awesome!
Right now some apps worked but most for most of them tvapprepo fast closes on me after selecting "create shortcut".

Mod with fix https://cloud.mail.ru/public/8tnX/fAPVpV1A6

Thx for the effort, but upon creating a shortcut the app still fast closes
:(
oldboy1988<EMAIL_ADDRESS>schrieb am So., 3. März 2019, 22:17:

Mod with fix https://cloud.mail.ru/public/8tnX/fAPVpV1A6
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ITVlab/TvAppRepo/issues/71#issuecomment-469065217,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Ahiel5rv0YV67-FI2_PrAt_TTpkQrr26ks5vTDvVgaJpZM4aHnKN
.

In mi box 3 work fine

^ Only works on mi box 3 then.

Yeah, I can't get any results with it on Shield either

Mod with fix https://cloud.mail.ru/public/8tnX/fAPVpV1A6
Link is not available. Could you please share it again? TIA.

Here is another link to the apk that works with oreo:
https://drive.google.com/file/d/1jE4P5QsCFjD6l-IM8iLeRaK2bohd9pNw/view?usp=sharing

Here is another link to the apk that works with oreo:
https://drive.google.com/file/d/1jE4P5QsCFjD6l-IM8iLeRaK2bohd9pNw/view?usp=sharing

This still doesn't work on a Nvidea Shield

not working. working apk but not working homescreen shortcut function.

Yeah, this hasn't worked since android 8 I believe and no one seems or has attempted to fix it...

Yeah, this hasn't worked since android 8 I believe and no one seems or has attempted to fix it correctly...

https://play.google.com/store/apps/details?id=dxidev.sideload.channel.launcher i'm trying this app...
worked fine but i cant fix app's image yet.
i must add manually app images. it will be long time for me.
the best app is TVAppRepo :D for us..


Why should I have dependencies under charts/

When I am specifying the dependency chart name, version and repo (the mandatory details to identify and download a chart) details in the requirements.yaml why should I have to add the tgz charts in the charts\ folder. Why not helm read that requirements.yaml file and do the magic on the fly.
dependencies:
- name: grafana
  version: 1.12.0
  repository: https://kubernetes-charts.storage.googleapis.com/

Hi @gkarthiks
helm dep build should populate your charts dir with the charts from requirements file.


Stop mixing camelCase and snake_case in code!

nice work btw 😉

Ah yes. The two functions I copied from stackoverflow. Forgot to change their cases :P Send a pull request

That code was unused anyway. In the latest commit (c0a0d10), it's removed.

👍


rename doesn't work for built in types

Getting the following warning

Warning: REAL(dp) not supported

adding the following line to the config.i file doesn't change the behavior or the warning.

%rename real(dp) real(c_double)

calling

~/code/fortwrap/fortwrap.py -i config.i  test.f90

The rename configuration directive is for renaming either a function or derived type in the generated wrapper code, so it wouldn't have any effect here.
How are you defining dp?  Right now FortWrap will only recognize it if it is a literal integer constant, e.g.:
INTEGER, PARAMETER :: dp = 8

Thank you so much for the reply.
I see I  clearly misunderstood the use case. Dp is mapped to c_double. I was able to just add that in the proper location in the type dictionary. However I still haven’t gotten it to generate wrappers. I need to mess with some more to figure out what I am doing wrong. The code I am working with makes Heavy use of more modern Fortran in some parts.


Unable to deploy vm's without trusted launch.

Why am i not able to deploy vm's without the trusted launch option using powershell,
Tried editting the parameters in de tempaltespecs file and also in the confguration my function app but unfortunately the only way for me to do this will be using trusted launch and not enabeling the vtpm and secure boot options. this works but that means a bunch of image rewriting..

Same issue. My images are build with Standard boot (No Trusted or confidentials boot).
When i hit review & create, i got this error:
{"code":"InvalidTemplate","message":"Deployment template validation failed: 'The value for the template parameter 'SecureBootEnabled' at line '94' and column '26' is not provided. Please see https://aka.ms/arm-create-parameter-file for usage details.'."}

@ankurvvvv  @nandosie012  -- are you both using portal-ui deployment from this repo? I saw that it returns the value of the checkboxes but those are hidden when standard mode is selected. Could be fixed there but I was able to fix by setting a default value in the file:
deploy\arm\DeployAVDSessionHostReplacer.json [arm template].
I added "defaultValue": false around line 98 for securebootenabled and tpmenabled.
"SecureBootEnabled": {       "type": "bool",       "defaultValue": false     },     "TpmEnabled": {       "type": "bool",       "defaultValue": false     },
I'll reach out to Willy

Fixed in v.0.2.8


https://github.com/allinurl/goaccess is crashing

From https://github.com/inlang/inlang/issues/749#issue-1708944036

Message: "The reference language "en" must be included in the list of languages"

Additional information
The po plugin doesn't seem to include the reference language in the list of languages.

@jannesblobel we discussed it already but the po plugin needs to create "virtual" resources for the reference language. yes, annoying that those resources can be edited but as we both figured out .po is a bad format (little we can do about it)

Yep, I'm going to change the plugin. It should also be ready tomorrow. But then we will have the problem with jsdelivr for a week: the version has to be updated that takes a week.

@jannesblobel you can purge JSDelivr https://www.jsdelivr.com/

Quick update: the problem is the file structure of this repo. The Reflanguage is "en", but the name of the reference file is 'goaccess.pot" instead of 'en'.  I am working on a good solution.


Parallelize BBC listing removal in clear-bounties

This can be done pretty easily with a lib.discordUtil.BasicScheduler, check out some existing examples to find out how.

must have done this a while ago


when fingerprinting fails use last successful fw fingerprint for VIN

idea to better handle people taking their car in for service and openpilot doesn't work when they leave to due to updates applied to their vehicle
example: https://github.com/commaai/openpilot/issues/24444
when we process qlogs we should automatically open PRs to add missing fw versions if the fingerprint was not an exact match
needs tests and probably more logging if this is something we actually want to do

Closing this since I don't think this will be necessary with 100% reliable fuzzy fingerprinting, but noting in #26612.


known issue: copying images often puts data:image/png;base64 on clipboard

From @0n0mat0po3ia:

Comment: I noticed that selecting Copy from the modal window creates a new clip in the top stack, displaying the base64 encoding of the image (e.g., data:image/png;base64,iVBORw0KGgoAAAANSU...) rather than the image itself. Is this the expected behavior?

blocked by (in a forcing function sense): https://github.com/cablehead/stacks/issues/46


Rudimentary support for multi F# projects

Related to #1975.
This PR adds simple support for multiple F# projects in a solution. It makes sure that ReferencedProjects field in FSharpProjectOptions is populated correctly. All referenced projects are registered with Roslyn; however, there is no real integration with Roslyn workspaces yet.
Ideally most of the changes in ProjectSitesAndFiles.fs should be done in the F# project system i.e. figuring out referenced projects directly from FSharpProjectNode. I need some pointers to do it in a better way. This is currently done similar to VFPT where we rely on Project and VSProject to find referenced projects.
After this PR, go to definition seems to work for those symbols defined in a referenced project.

I hope somebody finally wakes up at Microsoft and merge it, after that I can check that inline rename, navigate to and find refs work solution wide.

@dungpa I'm afraid it does not fix Navigate To since it requires all projects to be registered in the Workspace.

Project.ProjectReferences is always empty. For this solution:

Library2 project has no references, however it should reference Library1:

This is awesome, I constantly work with solutions that have multiple F# projects (VS2015) and having the global renaming, navigate-to (sometimes already works) and inline-renaming work cross-project is invaluable (even if that requires VS2017, which I think is the case here).

@vasily-kirichenko I have tried to wire project references. Could you try e2cf8b9  on your inline rename branch?
We still need some way to do this for all projects in a solution. Do you have a suggestion?

@dungpa I'll try it later today.
Maybe we should subscribe of "solution loaded" VS event (as we did in VFPT if I'm not wrong)?

If both the project are registered in the Solution, then it works:

Positive :).
My only concern is that it would be very slow to register every project when we just start opening a solution.
We didn't actually register all projects on solution load in VFPT. If a feature requiring multiple projects (i.e. rename, find all references, navigate to), we enumerate all F# projects in the solution, calculate dependent projects, and register them.

Lazy projects loading sounds good, especially considering truly asynchronous nature of the new API, for example, inline rename is gathering affected documents in background, while the user is entering new symbol name. It should work great.

This is huge work, @dungpa, really excited to see this come through.  I agree with @vasily-kirichenko that lazy-loading seems like a great approach.  Deferred project loading is something we're doing/investigating for VS all-up.

@dungpa
This looks great, we are going to have someone from the roslyn team take a look at the approach to ensure that this is the best way to achieve this, it will however, take a few days, because "priorities" and stuff.
Thanks for looking at this, when we have it in, it will make a huge difference to larger solution development.
Kevin

I've tagged @jasonmalinowski to take a look when he can.

@dotnet-bot test this please

@dungpa, in response to your comment:

My only concern is that it would be very slow to register every project when we just start opening a solution.
I agree wholeheartedly, and it's been something that I've been meaning to raise as a separate issue entirely. When a solution has a large number of F# projects (say 100), solution load times are tremendously slow compared to the same number of C# projects.

I'm concerned that if we're adding more work to the 'solution load'/'project load' events that this time is going to grow even larger. The current slowness is already impeding our work.

When a solution has a large number of F# projects (say 100), solution load times are tremendously slow compared to the same number of C# projects.

@saul  Points well made. In the times of F# 3.x, my current project was separated in 25 or so subprojects, which each having their corresponding test projects. With utility projects this came to about 50 or so and impossible to deal with.
I have made it a habit to split a large solution into smaller solutions but this is a maintenance nightmare. Or I merge projects together (10x project load of 1 file vs 1x project load of 10 files makes all the difference).
Not having to worry too much about this would be very beneficial. Though F# loads significantly slower than C# of similar size, C# is not without size problems itself. Anybody ever trying to maintain a 50+ project solution in C# knows that Visual Studio comes down to a crawl, crashes, disappears etc.
(I am aware that VS2017 is supposed to solve this with an option for large projects for C# / C++, would be great if we could join that)

@dotnet test this please

@saul Although this isn't shipping for vs2017, one of our highest priorities for F# is to get it on the Roslyn Project System (see #1193 for more information) after vs2017 ships.  Although this won't address all performance problems, it will help.

@dotnet test this please

@dungpa merge with master, please.

@vasily-kirichenko Done.

@dungpa
Thanks for doing this,
Kevin

@KevinRansom So I rebased to current master. I limit the purpose of this PR to populate project references correctly. This is already an improvement to make Go To Definition working correctly.
There are two big questions that I leave to future PRs:

Populate current project and all projects that use it. This is needed for Find all references and Rename to
return correct results.

Populate all F# projects in the solution. This is important for Navigate To.

That's great ... I was going to merge and pull it today anyway so you have saved me some work.
Thanks
Kevin

@dotnet-bot test this please --- rebuild with latest bits

@dungpa
Thank you for taking care of this.
Kevin


Implemented signalling for chat, applied styling

Please check if the PR fulfills these requirements

[x] You've followed the contributing guidelines
[x] You've adheared the code of conduct

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

[ ] Bug Fix
[x] New Feature
[ ] Documentation Update
[ ] Other

If bug fixes or new features

What is the current behavior? (You can also link to an open issue here)

There is no chat

What is the new behavior (if this is a feature change)?

A chat using signalling, provided by the Vonage Video API. Currently just shows text, not who sent it, and doesn't distinguish between the current user and others.

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

No

Other information:

None

This PR also changes some styling from #11 to make room for the chat


Create sgd-darmstadt.txt

Informatiker Studium 437

@Formano lib/domains/de/sgd-darmstadt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

Thank you for help i changed the file path


Align Header Text and Icon for Better Aesthetics #38

Updated CSS :--

Is it merged ? or is it closed?


Add testing for cl_d3d10.h, cl_d3d11.h, and cl_dx9_media_sharing.h

Adds missing testing for cl_d3d10.h, cl_d3d11.h, and cl_dx9_media_sharing.h.

This PR has been open for a while too.  Are any additional changes needed?  If not, can it be approved and merged?  Thanks!

This PR has been open for a while too.  Are any additional changes needed?  If not, can it be approved and merged?  Thanks!

I've rebased this after the layer's change.  I think this can go in anytime.  Thanks!


🌐 Add Traditional Chinese translation for docs/zh-hant/docs/async.md

Thank you! @ILoveSorasakiHina :rocket:
And thanks for your (previous) review @mattwang44 :nerd_face:


Support engine kwarg in dd.to_parquet

The engine kwarg is already supported in dd.read_parquet, but it would be nice to also support different engines for writing.

I suspect that we're waiting on Arrow folks for this.  cc @wesm
On Fri, Oct 13, 2017 at 7:36 PM, Jim Crist<EMAIL_ADDRESS>wrote:

The engine kwarg is already supported in dd.read_parquet, but it would be
nice to also support different engines for writing.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/2783, or mute the thread
https://github.com/notifications/unsubscribe-auth/AASszFtMGsTQhBclPlcLtp79wiQ5G9Poks5sr_P-gaJpZM4P5Kku
.

Since this is already supported in pandas and we've been testing cross-compatibility of the metadata between engines, adding Arrow support in Dask shouldn't be a large task. It may produce some additional requirements for writer options, so it would be good to get a list of what those are so they can be prioritized in upcoming releases (or at least fail over to reasonable default behavior)
cc @jreback @cpcloud


system_modes: 0.2.3-1 in 'foxy/distribution.yaml' [bloom]

Increasing version of package(s) in repository system_modes to 0.2.3-1:

upstream repository: https://github.com/micro-ROS/system_modes.git
release repository: https://github.com/microROS/system_modes-release.git
distro file: foxy/distribution.yaml
bloom version: 0.9.7
previous version for package: 0.2.1-2

Holding for sync.

Sync is out.


Fix path of dev_setup.sh in process-agent README

I believe that we're getting rid of this file.

@kevinconaway Thanks for pointing this PR


No windowed mode on Galaxy Note (10.1, 2014)

Please search to check for an existing issue (including closed issues, for which the fix may not have yet been released) before opening a new issue: https://github.com/k9mail/k-9/issues?q=is%3Aissue
Expected behavior
Being able to create a resizable window in order to share the screen with other apps (multitasking).
Actual behavior
K9 opens up maximised instead, thus minimising other apps currently on screen
Steps to reproduce

Open up an app
Pull up the multitasking panel (long-press the return button)
Tap K9

Environment
K-9 Mail version: latest
Android version: 5
Account type (IMAP, POP3, WebDAV/Exchange):

Tell Samsung to update your system to Android 7. k9mail will work in both split screen and floating window modes that are implemented using the GENUINE multi-display modes built into Android. I doubt that anybody is all that eager to work on implementation for a samsung hack.

Virtually every decent app I’ve used to date has supported that mode. K9 is obviously an outsider ;)

I don't see much harm in supporting it as long as it's relatively isolated and maintainable.
I'm going to tag this as help wanted as you need a Samsung phone to work on this and I assume none of the active devs have one.
For reference we already have an issue regarding Android multiwindow support - it's a matter of testing and bumping the target API (which means updating various bits and prices) - it works right now but you get a warning.

@philipwhiuk Samsung has abandoned their proprietary multi-window mode and is now using AOSP. OP is using android 5, which is already ancient history. Supporting this isn't worth one single line of code.

We still support Android 4.4 I believe (API 19). Android 5 was released in November 2014. It's not ancient history. I think it's more than reasonable to add 1 line of code for this.

It isn't android 4 or 5. Its SAMSUNG. A bastardized fork of android.

"Samsung and LG now account for 78 percent of all Android sales in the US | Android Central"
Yeah, just Samsung, no big deal.


⚠ ClamAV has degraded performance

In 1b1f52b, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 78 ms

Resolved: ClamAV performance has improved in a303bdb.


M1 gemfile updates - Removal

Description
Removing M1 Changes until better tested
Issues Resolved
Will resolve current build issues.
Check List

[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.

Not needed


Non-maximal suppression in ImageAI

Hello,
Does ImageAI do non-maximal suppression to get rid of multiple bounding boxes for an object??

yep, it is implemented here

How to apply Non-maximal suppression, when using the detectCustomObjectsFromImage


(Fix) Approve transaction modal closes

This PR closes #1474, by:

adding an equality function to the useAction hook for the featuresEnabled selector

Travis automatic deployment:
https://pr1477--safereact.review.gnosisdev.com/app

Travis automatic deployment:
https://pr1477--safereact.review.gnosisdev.com/app


add litellm for using many LLM APIs, refer to #173

Description:
Add litellm to support llm in APIs
Reference Issues: #173 (XX is the issue number you work on)

sounds good ! this PR looks good


Encrypt and Decrypt async storage

The async storage is currently unencrypted.
The application should have a way to encrypt the data being placed into the storage and decrypted after retrieval.
It is recommended to look at ways of securing the application.
An article can be found here as some form of reference - https://docs.expo.dev/versions/v46.0.0/sdk/local-authentication/
The bare minimum would be a password that the user must access the application.
For further reading about encrypting and decrypting with Expo - https://blog.logrocket.com/encrypted-local-storage-in-react-native/

Thank you for the Input.
The first option looks interesting, it would be nice to include FaceId or TouchId as a quick way to log in.

Also, found that react native has an extensive resource on security. I highly recommend reading through - https://reactnative.dev/docs/security


docs(autocomplete): Rename parameter action to suggestionAction

It appears that the parameter action has been renamed to suggestionAction so the docs were outdated; this commit fixes that

Thank you @f-person 🙏


Add main queue child contexts of the main queue MOC

Summary of Changes
Gives the API consumer the ability to create an NSManagedObjectContext with MainQueueConcurrencyType as a child of the Main Queue Context.
Addresses
#127

:eyes: @jobot


Known Salt Regression in v1.0.4

In an internal testsuite I discovered the following regression regarding known salts in the v1.0.4 release:
Testcase
public function testKnownSaltRegression() {
    $hash = password_hash("foobar", PASSWORD_BCRYPT, array("cost" => 04, "salt" => "OyuaiuF.Bcw7mNCak/3Au7c="));
    $this->assertEquals('$2y$04$T3l1YWl1Ri5CY3c3bU5DYOiIKqbbPpdcIiqpnkLoRwJkELw/uCaUO', $hash);
}

Resultmatrix

PHP
v1.0.3
v1.0.4

5.5.14
✔
✔

5.4.39
✔
✗

It's not really an issue for me because in the real application, a known salt is never used.
I suspect the provided salt OyuaiuF.Bcw7mNCak/3Au7c= is simply malformed/not valid and the issue is related to https://github.com/ircmaxell/password_compat/pull/38.
Because this is not a real issue for me feel free to close this issue.
Nevertheless I would love to understand what's the root cause of this regression and what was wrong with the salt in the first place.
I also fully understand that providing custom salts is a bad idea and will most probably remove that from my internal interfaces completely.

That's correct. The change was applied from #39, but I never made it upstream.
That's irrelevant now that salts are going to be deprecated (I will deprecate them in this library as well). So :-)


feat: add theorems about isLUB and isGLB for Finset.sup and Finset.inf

feat: add theorems regarding isLUB and isGLB for Finset.sup and Finset.inf
theorems were previously missing stating that Finset.sup produces a least upper bound and Finset.inf produces a greatest lower bound. This commit adds those theorems as well as the corresponding one for Finset.sup' and Finset.inf'.

Can you please also provide IsLeast and IsGreatest versions for LinearOrders? The lemma [Finset.inf'_mem will come in handy for proving it I believe.

Finset.isGLB_iff_isLeast should do all the work there.

Oops, I didn't know we had Finset.isLeast_min'. Indeed, I didn't even know we had Finset.min'.
It's not clear to me why we have these in addition to Finset.inf', but at least you can ignore my earlier suggestion.

Ok, I followed Eric's suggestion. If I understand correctly, Jireh, since Finset.min' and Finset.max' exist and have IsLeast and IsGreatest, no other change is requested of me.
If that's wrong, just let me know.


[Doc-Bug] App setup policy for Guest users

Hi,
Following was mentioned in documentation:

But if we add app to app setup policy under installed apps, then app is installing for Guest users. So, the statement in documentation is outdated: Setup policies don't apply to guest users.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0735d322-f6f5-7cc8-9994-1420e3fb7baa
Version Independent ID: f9b25b51-eb9c-63fa-b30e-e1490c8d31b6
Content: Teams apps behavior for non-standard users - Microsoft Teams
Content Source: Teams/non-standard-users.md
Service: msteams
GitHub Login: @cichur
Microsoft Alias: v-cichur

Thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Based on the outcome let me know if it is something that can be called out in the docs.
Please follow this link to contact support for business products:https://docs.microsoft.com/office365/admin/contact-support-for-business-products
Please keep us posted here on the resolution so we can feed whatever information you discover into the content.


Typo in pyproject.toml

https://github.com/medspacy/medspacy/blob/9d422d1ed418882e8a515ea4d461f472b989210d/pyproject.toml#L19
Should be requirements

@jianlins  It seems like we should take this change, right?  Otherwise, it seems to me that this line does not do anything (i.e., does not bring in the requirements from requirements.txt.
Do you agree?  Also, do you think this could explain any build related issues we've had since we started using the .toml file?

It's done with the last pull.


Fixing value overflow issue when using ModSimVectors with integers

This fix converts each element of a vector to a float when the Vector function is used to create a ModSimVector.
When ModSimVectors are created using the Vector function with large integer arguments, subsequent operations on that vector can result in unexpected values due to an apparent integer overflow in numpy.dot. This issue was discovered when using the vector_mag and vector_hat functions, and the problem was found to originate in np.dot.
This error did not occur if the vector values were cast to floats, presumably because np.dot defaults to using a data type for floats that allows higher values.

Thanks for this suggestion.  I am undecided about whether this is a good change.
On one hand, it seems like a Vector that contains integers is useful and should not be prohibited.  If the user wants floats, they can create the Vector with floats.
On the other hand, the intent of this class is to represent physical quantities, which would most appropriately be floats.
Can you tell me more about the use case that causes the error?

This is a problem that several first-year students taking ModSim have encountered while working on the  orbital mechanics notebook. 1 AU expressed in meters, or 1.49e11, is large enough to cause an overflow issue when it is squared (as happens when finding the magnitude of the vector.)
My fix may not be ideal, but it seems like it would be good to address it in some way. Currently it gives unpredictable results to students who are not necessarily familiar with Python, and in the case that the result is positive, it's hard to tell where anything went wrong.
One thing which reduced the appearance of this error is that, when a vector with units is created using the syntax:
r = Vector(x, y) * units
Even if x and y are integers, the multiplication with units causes them to be converted to floats. So, in some cases, the conversion happens anyway. The values will stay as integers if this syntax is used:
r = Vector(x*units, y*units)
If you want to allow integers to be used, I can think of a couple alternatives.

Conversion to a float would happen by default, but the user could pass a 'type' parameter to the Vector function, specifying a data type recognized by numpy to be used for the vector values. Perhaps the vector would be stored as a numpy array of the specified type, with associated units. Then, at least, the user has to be explicit about what type they want, and are more likely to catch and understand any error that occurs.

Vector functions which might trigger an overflow could check the initial type and value, and give a warning if the value was likely to exceed the size limit. This isn't my favorite, but at least the user will know what went wrong and might have an opportunity to fix it while testing their code.

On the other hand, the ModSimPy library is intended for modeling real-world situations, and Vectors are for real-world quantities, so it may make sense for these to be float values. If a user knows they need integer values, there are other tools they can use, such as numpy arrays.

I am still confused about the use case. 1.49e11 is a float, so where are the integers coming from?
Allen
On Sat, Nov 16, 2019, at 2:56 PM, jsieving wrote:

This is a problem that several first-year students taking ModSim have encountered while working on the orbital mechanics notebook. 1 AU expressed in meters, or 1.49e11, is large enough to cause an overflow issue when it is squared (as happens when finding the magnitude of the vector.)

My fix may not be ideal, but it seems like it would be good to address it in some way. Currently it gives unpredictable results to students who are not necessarily familiar with Python, and in the case that the result is positive, it's hard to tell where anything went wrong.

One thing which reduced the appearance of this error is that, when a vector with units is created using the syntax:
r = Vector(x, y) * units
Even if x and y are integers, the multiplication with units causes them to be converted to floats. So, in some cases, the conversion happens anyway. The values will stay as integers if this syntax is used:
r = Vector(x*units, y*units)

If you want to allow integers to be used, I can think of a couple alternatives.

Conversion to a float would happen by default, but the user could pass a 'type' parameter to the Vector function, specifying a data type recognized by numpy to be used for the vector values. Perhaps the vector would be stored as a numpy array of the specified type, with associated units. Then, at least, the user has to be explicit about what type they want, and are more likely to catch and understand any error that occurs.

Vector functions which might trigger an overflow could check the initial type and value, and give a warning if the value was likely to exceed the size limit. This isn't my favorite, but at least the user will know what went wrong and might have an opportunity to fix it while testing their code.

On the other hand, the ModSimPy library is intended for modeling real-world situations, and Vectors are for real-world quantities, so it may make sense for these to be float values. If a user knows they need integer values, there are other tools they can use, such as numpy arrays.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/AllenDowney/ModSimPy/pull/54?email_source=notifications&email_token=AAOLP3JORWJ7EPCTCHBHRGLQUBF5DA5CNFSM4JOCSOH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEHZHGY#issuecomment-554668955, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOLP3J74EVNZP4OL5LJLXTQUBF5DANCNFSM4JOCSOHQ.

Sorry, I wrote it in scientific notation for brevity, which was really unclear since in Python that would be a float. When people type it out as<PHONE_NUMBER>00, then there's a problem. Some students are in the habit of typing out long numbers instead of using scientific notation. I realize it's a small use case, but I'm sure this will come up in projects involving planetary motion, or in future iterations of the course. Of course, the answer could be to tell people not to use integers, but it seems like there are ways the function could work that would save people a few hours of confusion, especially because value overflows aren't something people expect to see when debugging Python.

Running into this error seems like a learning opportunity. The book doesn't talk much about integers and floating-point, but at some point it's good to know that integers are for counting and floats are for measuring. Expressing a large quantity as an integer is a bad idea, in part because it might cause overflow, but also in part because it's not the right type to express a measurement.
On Sat, Nov 16, 2019, at 10:04 PM, jsieving wrote:

Sorry, I wrote it in scientific notation for brevity, which was really unclear since in Python that would be a float. When people type it out as<PHONE_NUMBER>00, then there's a problem. Some students are in the habit of typing out long numbers instead of using scientific notation. I realize it's a small use case, but I'm sure this will come up in projects involving planetary motion, or in future iterations of the course. Of course, the answer could be to tell people not to use integers, but it seems like there are ways the function could work that would save people a few hours of confusion, especially because value overflows aren't something people expect to see when debugging Python.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/AllenDowney/ModSimPy/pull/54?email_source=notifications&email_token=AAOLP3KLB7CLKMPDDDHMVJTQUCYE3A5CNFSM4JOCSOH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEIAMCQ#issuecomment-554698250, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOLP3N3UTRP43YRGB6EMQ3QUCYE3ANCNFSM4JOCSOHQ.

While I think that's a good point, it's hard to use it as a learning opportunity if the person encountering the problem never figures out what's wrong. The issue was originally solved by one of the professors, and I don't think they had an explanation for it. Although I'm familiar with the technical cause, even I didn't think "this issue occurs because I used an integer for a measurement, and that's not appropriate" until you put it that way. So, if you think this is a good time for students to learn that distinction, maybe there should be a small section in the book around Chapter 22, or a link in the orbital mechanics notebook to an explanation that they should read.
This is one of those things that if you don't understand the root cause, finding the solution seems as random as crossing your fingers while making seemingly trivial changes to your code. If this isn't going to be treated as a learning opportunity, then it should at least be changed so that students don't spend hours fruitlessly checking over their code and ultimately turning it in incomplete.

Ok, let's do it.  I'll merge first and run the test suite later, because that's the kind of software engineer I am.

All tests passing.


[Review] Sprint 13

Review

Issues:

Timeline: https://cernbox.cern.ch/index.php/s/sWuZHrXKHkC7Hfc

UI
Migrator kit
Q&A

Do we need to keep files in the records? Does it happen that the record has a file, other than thesis ?
Discuss links to records with hashed PIDs (/document/ABCD-UUZ)
We should clarify if only users associated with CERN(having a cern email) can loan books.

@kprzerwa yes, we need to be able to attach files for the standards: we attach the standard to the record. It might happen, but much more rarely, for proceedings and books

1 file only per electronic item (no files on Item/Documents)


Issue #248 Implement schedule clear command

Closes #248

Test cases are up (thankfully coverage has increased) @charmainehly and @LeeEnEn may start your review any time you are free! Thank you 😄

@LeeEnEn and @charmainehly I will add test cases tomorrow morning! Don't need to review yet! Thank youuuu

Hello @LeeEnEn and @lzan98 y'all can start to review! I think I'll add in more testcases in v1.4 or tonight if time permits


export model pickle overflow error

Hi, while exporting models larger, then 4 gigabytes there is a pickle overflow error. This is usually easily fixed with protocol=4 parameter in pickle.dump. Please consider updating the source code for the export method. Thank you!

Note that we are not calling pickle.dump but torch.save. I can make that argument surface if helpful.

Yes, that would be definitely helpful and even making it a default setting will benefit everyone imho.


fix race condition for SetFormatter and SetReportCaller

Unfortunately https://github.com/sirupsen/logrus/issues/1233 didn't really fix the race condition, the test just wasn't triggering the race condition to be detected...
*Entry.write just needs a lock for the whole function.

@sirupsen Any chance of merging this in? Thanks!

<3 thx for the bump @r0mant
ye @sirupsen  or @dgsb let me know if you want to see anything changed before merging

@sirupsen it's really annoying to have random pipeline failures over this every so often :/

@sirupsen @dgsb Bumping this again guys, any chance to get this merged in? Thanks!


Issue @ getting started, attached npm-debug.log

React Native Version: react-native-cli: 1.0.0
react-native: n/a - not inside a React Native project directory
OS: Mac OS X El Captain
npm-debug.txt

What are you trying to do ? Can you provide more information ?

After longtime, wanted to try creating an React Native application. So went to "Getting started" and followed the instruction. Installed Homebrew, flow, watchman & ReactNative-cli.
executed "react-native init PlaybookPreMobile" in command line and i got below error in log.
log:

This will walk you through creating a new React Native project in /Users/yogeshofit/Desktop/React/NativeWorkspace/PlaybookPreMobile
Installing react-native package from npm...
npm ERR! Darwin 15.0.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "--save" "--save-exact" "react-native"
npm ERR! node v6.2.2
npm ERR! npm  v3.9.5
npm ERR! Invalid Version: 0.0.1alpha1
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     https://github.com/npm/npm/issues
npm ERR! Please include the following file with any support request:
npm ERR!     /Users/yogeshofit/Desktop/React/NativeWorkspace/PlaybookPreMobile/npm-debug.log
npm install --save --save-exact react-native failed

Regards,
Yogesh Surendran alias CarreraPHP

Any update for the issue?

Everything works fine on my side on OS X and node v6.2.2.
Can you try again?

Hi there! This issue is being closed because it has been inactive for a while.
But don't worry, it will live on with ProductPains! Check out its new home: https://productpains.com/post/react-native/issue-getting-started-attached-npm-debuglog
Product Pains has been very useful in highlighting the top bugs and feature requests:
https://productpains.com/product/react-native?tab=top
Also, if this issue is a bug, please consider sending a pull request with a fix.


fix jovo deploy broken link

Proposed changes

Types of changes

[ X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[X ] My code follows the code style of this project
[ ] My change requires a change to the documentation
[ ] I have updated the documentation accordingly
[ ] I have added tests to cover my changes
[ ] All new and existing tests passed

Thank you @zeuslawyer! I think this PR is going to your repo's master branch, not the jovo-framework repo's. Could you merge into your master and then create a PR to our repo?
Thanks!
Jan

After poking around a bit, I was able to create a PR from your branch. Thanks a lot!  Forgot how it works 😂
On Tue, Nov 3, 2020, 02:51 Jan König<EMAIL_ADDRESS>wrote:

After poking around a bit, I was able to create a PR from your branch.
Thanks a lot! :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zeuslawyer/jovo-framework/pull/1#issuecomment-720556071,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB5FCAOVMW5TF4IUPPSSHI3SN3IQZANCNFSM4TE3NP5A
.


Rancher cross-host networking - missing iptables rules

Rancher v1.1.3
Cattle  v<IP_ADDRESS>
User Interface  v1.1.12
Rancher Compose v0.8.6
Docker 1.12.1
Ubuntu 16.04.1 LTS
ProxMox KVM hosts with bridged network on primary interface eth0 (<IP_ADDRESS>/24)
Single Node Rancher on <IP_ADDRESS>
First host on <IP_ADDRESS>
Second host on <IP_ADDRESS>
Catle environment
Setup the rancher host, and add two agents using a specific host IP (otherwise the network agent will detect the docker0 interface as the primary interface).
Result: the hosts appear in the infrastructure/hosts view, and appear to work.
Setup any stack that requires cross-host communication, and behold: the load balancer remains in 'initializing' state. It seems that monit process causes this.
Exec into one of the network agent shells, and try to ping the other agent at 10.42.x.x: 100% packet loss. Pinging the other host over the docker subnet works though.
sudo iptables -L -n --line-numbers -t nat
Shows a lot of these lines:
280  MARK       all  -- !<IP_ADDRESS>/16         <IP_ADDRESS>      MAC 02:64:FE:CD:2F:31 MARK set 0x315e8
But does not seem to produce the correct NAT rules.
sudo iptables -F -t nat and then a restart of the network agent causes the following to appear:
Chain POSTROUTING (policy ACCEPT)
num  target     prot opt source               destination
1    MASQUERADE  udp  --  <IP_ADDRESS>           <IP_ADDRESS>           udp dpt:4500
2    MASQUERADE  udp  --  <IP_ADDRESS>           <IP_ADDRESS>           udp dpt:500

Chain CATTLE_POSTROUTING (0 references)
num  target     prot opt source               destination
1    SNAT       all  -- !<IP_ADDRESS>/16         <IP_ADDRESS>      mark match 0x315e8 to:<IP_ADDRESS>

Chain CATTLE_PREROUTING (0 references)
num  target     prot opt source               destination
1    MARK       all  -- !<IP_ADDRESS>/16         <IP_ADDRESS>      MAC 02:64:FE:CD:2F:31 MARK set 0x315e8

Chain DOCKER (0 references)
num  target     prot opt source               destination
1    DNAT       udp  --  <IP_ADDRESS>/0            <IP_ADDRESS>/0            udp dpt:4500 to:<IP_ADDRESS>:4500
2    DNAT       udp  --  <IP_ADDRESS>/0            <IP_ADDRESS>/0            udp dpt:500 to:<IP_ADDRESS>:500

Expected:
I would have expected to see DNAT rules for IPSEC traffic on the 10.42.X.X network? Anyway: cross-host communications keep failing, and adding containers etc. does not change anything. All agents were started with the CATTLE_AGENT_IP option. I believe this issue is also the reason why my convoy-nfs agents have been stuck in initialising since they started.

@daankets did you manage to sort this? Sometimes i find the order of firewall rules being programmed prevents docker communication. i typically restart iptables, then docker. Please do reopen if this is still an issue.

@aemneina No, I didn't succeed. I've tried restarting iptables, docker in various orders, but without a breakthrough. In the end, I've shutdown the VMs, and started over using the latest RancherOS image. I've in the meantime deleted the old VMs.


Could not list contents of /path/to/project/.buckd/sock when generating signed apk

[x] Provide a minimal code snippet / rnplay example that reproduces the bug.
Only when I was generating signed apk.
[x] Provide screenshots where appropriate

When executing ./gradlew assembleRelease:
...
:app:generateReleaseAssets UP-TO-DATE
:app:mergeReleaseAssets
:app:generateReleaseResValues UP-TO-DATE
:app:generateReleaseResources UP-TO-DATE
:app:mergeReleaseResources UP-TO-DATE
:app:bundleReleaseJsAndAssets

FAILURE: Build failed with an exception.

* What went wrong:
Could not list contents of '/Users/wildcat/WebstormProjects/queshu/.buckd/sock'.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

With --debug:
...
22:19:46.365 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.store.CachedStoreFactory                                                      ]                                            Resolved configuration cache closed. Cache reads: 0, disk reads: 0 (avg: 0.0 secs, total: 0.0 secs)
22:19:46.365 [DEBUG] [org.gradle.api.internal.artifacts.ivyservice.resolveengine.store.ResolutionResultsStoreFactory                                           ]                                            Deleted 2 resolution results binary files in 0.002 secs
22:19:46.365 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] The daemon has finished executing the build.
22:19:46.623 [DEBUG] [org.gradle.launcher.daemon.client.DaemonClientInputForwarder                                                                             ]                                            Dispatching close input message: CloseInput[id=8f55d3bc-926c-4bc3-b50c-023086203b88.2]
22:19:46.623 [DEBUG] [org.gradle.launcher.daemon.client.DaemonClientConnection]thread 11: dispatching class org.gradle.launcher.daemon.protocol.CloseInput
22:19:46.624 [INFO] [org.gradle.launcher.daemon.client.DaemonClient] Received result CommandFailure[value=org.gradle.launcher.exec.ReportedException: org.gradle.internal.exceptions.LocationAwareException: Could not list contents of '/Users/wildcat/WebstormProjects/queshu/.buckd/sock'.] from daemon DaemonInfo{pid=42671, address=[b889bab8-f701-4b94-9282-b08c1c4e171a port:55820, addresses:[/0:0:0:0:0:0:0:1, /<IP_ADDRESS>]], idle=true, context=DefaultDaemonContext[uid=b59af716-c9d2-453b-97fc-0401a12e14eb,javaHome=/Library/Java/JavaVirtualMachines/jdk1.7.0_79.jdk/Contents/Home,daemonRegistryDir=/Users/wildcat/.gradle/daemon,pid=42671,idleTimeout=10800000,daemonOpts=-XX:MaxPermSize=256m,-XX:+HeapDumpOnOutOfMemoryError,-Xmx1024m,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant]}.
22:19:46.625 [DEBUG] [org.gradle.launcher.daemon.client.DaemonClientConnection]thread 1: dispatching class org.gradle.launcher.daemon.protocol.Finished
22:19:46.626 [DEBUG] [org.gradle.launcher.daemon.client.DaemonClientConnection]thread 1: connection stop

[x] What's the version of React Native you're using?
0.28.0
[x] Does this occur on iOS, Android or both?
Only Android
[x] Are you using Mac, Linux or Windows?
Mac OS X 10.11
Additional Info:
When I execute buck in my project direction:

➜  queshu git:(master) ✗ buck
BUILD FAILED: Failed to read NDK version from /Users/wildcat/Library/Android/android-ndk-r11c

I had the same issue... Just delete lock file and buildings works. I think we need there some kind of filter to ignore lock files.
I'll look into it tomorrow.

@K-Leon Thanks.
I initialised a news project, coping the codes to the project, problem solved.

I've recently been getting this error a lot. What were the steps to fix this? I've had to remove the project and re-clone to get it to work, but then the error will randomly popup sometimes again and I have to repeat the process.

rm -rf ./buckd also worked for me 👍


⚠ Processors has degraded performance

In 8eed920, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 143 ms

Resolved: Processors performance has improved in 4b27915.


Hiding then archiving prgma or prgmc bricks the calculator

Hiding then archiving prgma or prgmc bricks the calculator.
Steps to reproduce:

Create prgma or prgmc, with a lowercase letter for the name. This is left as an exercise for the reader.
Find prgma/prgmc in Cesium, then press alpha to open the properties menu.
Select hidden, then archived.
Exit Cesium.
Enjoy your paperweight.

Hiding then archiving prgma or prgmc bricks the calculator.
Steps to reproduce:

Create prgma or prgmc, with a lowercase letter for the name. This is left as an exercise for the reader.
Find prgma/prgmc in Cesium, then press alpha to open the properties menu.
Select hidden, then archived.
Exit Cesium.
Enjoy your paperweight.

Bricks? So it cannot boot again or do you have to re-install the OS?

Fixed with 475c4dd


Installing on OSX: The operation can’t be completed because one or more required items can’t be found. (Error code -43)

Platform: Darwin MacBook-Pro.local 17.4.0 Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64 x86_64
Kap Version: 2.0.0
Steps to reproduce Drag
Download .dmg, double-click and drag Kap to Applications folder
Current behaviour
Popup appears saying: The operation can’t be completed because one or more required items can’t be found. (Error code -43)
Expected behaviour
Installing
Workaround
None

https://apple.stackexchange.com/questions/139105/error-the-operation-can-t-be-completed-because-one-or-more-required-items-can

We suspect this issue is upstream @annapowellsmith. Will close this issue shortly if we don't hear back from you.

Yes, it was upstream. (I did Google!) Thanks, closing this.


Route struct not exported?

Question
Where is the Route struct exported in master? On v0.2.0 its at tide::Route.

I don't think it is. I can't speak to why it was removed. However, if you can share more about your use case, I'd like to see every viewpoint represented as work to revamp the routing layer continues.

Route is exported now!


Modifier is incorrectly added to all damage rolls

Taking the Ankheg as an example...
Large monstrosity, unaligned

Armor Class 14 (natural armor), 11 while prone
Hit Points 39 (6d10 + 6)
Speed 30 ft., burrow 10 ft.

STR	DEX	CON	INT	WIS	CHA
17 (+3)	11 (+0)	13 (+1)	1 (-5)	13 (+1)	6 (-2)
Senses darkvision 60 ft., tremorsense 60 ft., passive Perception 11
Languages --
Challenge 2 (450 XP)

Actions
Bite. Melee Weapon Attack: +5 to hit, reach 5 ft., one target. Hit: 10 (2d6 + 3) slashing damage plus 3 (1d6) acid damage. If the target is a Large or smaller creature, it is grappled (escape DC 13). Until this grapple ends, the ankheg can bite only the grappled creature and has advantage on attack rolls to do so.
Acid Spray (Recharge 6). The ankheg spits acid in a line that is 30 feet long and 5 feet wide, provided that it has no creature grappled. Each creature in that line must make a DC 13 Dexterity saving throw, taking 10 (3d6) acid damage on a failed save, or half as much damage on a successful one.

...the imported Bite and Acid Spray abilities have an incorrect +3 to their acid damage rolls:

It looks like setDamageRolls should check the value of damagemod1 and damagemod2 capture groups against the relevant ability score before appending + @mod to the damage.

Thanks for pointing that out. Fixed in 1.4.5


Display change in StockTwits overview

For instance if the score has gone up or down, or if this ticker is new for that category, compared to last post.
Use the same methods as for assets, use 🌟 if the symbol is new.

Do it the same as assets for users, have only 1 message in the channel and display the change compared to the last message sent

Could also use custom emoji's to indicate change in ranking

Probably easier to save the last StockTwits Ranking data instead of reading the old embed


Support message delivery through amazon cloud

Amazon has a pair of offerings within AWS that appear to do the same thing as firebase. Given the general availability of AWS, it would be valuable to have a uproxy cloud service using that cloud.
In particular these are:
SMS - simple messaging service, lets you push messages for delivery to an HTTP endpoint, an email address, a mobile SMS, or into an internal service queue (the most likely)
SQS - simple queue service, lets clients watch queues for received messages.
SQS allows for long-polling of the queues by an HTTP client (up to 20 seconds), which is the main thing we need for low-traffic use by uproxy. see: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ReceiveMessage.html
http://aws.amazon.com/sqs/details/
The free tier pushes 1 million requests / month, which corresponds to about 1 continuously active polling user. Subsequent cost would be $0.50 / active user / month.
This seems like it could potentially get quite pricy. If you were able to hook into the other message delivery options like watching email, sms, or an HTTP hook, price would be much much cheaper.

Closing as this is not a priority right now.


[Storage][DataMovement] Refactor download transfers

This change refactors download transfers to hopefully improve performance and simplify the code:

Remove the eventing for DownloadChunkHandler as it wasn't needed. The job part now simply adds directly to the channel.
Removed "early bird" chunks entirely. Chunks were written in order to disk and chunks that came out of order were saved to a temporary file. Now chunks are written in whatever order they come in by seeking to the proper location in the file for each chunk.

Further testing needs to be done to see if this is performant enough. Further improvements could also be made like somehow leaving the file stream open for the duration of that job part transfer, so we don't continuously open and close the file.

Other minor refactoring.

API change check
API changes are not detected in this pull request.

Perf results - https://dev.azure.com/azure-sdk/internal/_build/results?buildId=4345659&view=logs&j=42ded549-05ee-5cdb-7ba7-7a948a0cc056&t=1a18f1cb-d016-5e13-bec4-a5dc2ac35679

/azp run net - storage - datamovement - tests

/azp run net - storage - datamovement - tests


env.sh is chainging CWD

Currently env.sh is changing the current working directory (like invoking cd - I assume).
This is confusing and considered as bug.

env.sh is dead and history. In devon CLI the problem is solved.


Allow changing labels description

The GitHub API allows to change labels description:
https://docs.github.com/en/rest/reference/issues#update-a-label

Coverage decreased (-0.07%) to 94.564% when pulling 56ae6ff0a7ee7ccac15d55334dd4fa081b5e3640 on smortex:label-description into 02ec883ee7038385559854c1081d235cb2dd1245 on piotrmurach:master.

Coverage decreased (-0.07%) to 94.564% when pulling 56ae6ff0a7ee7ccac15d55334dd4fa081b5e3640 on smortex:label-description into 02ec883ee7038385559854c1081d235cb2dd1245 on piotrmurach:master.


fix: Let SSE client reconnect after EOF

Features and Changes
Fixes leaking go routine - github.com/ian-ross/sse/v2.(*Client).readLoop
119 @ 0x446d76 0x40f00e 0x40ebbd 0x168247e 0x47b3c1
#	0x168247d	github.com/ian-ross/sse/v2.(*Client).readLoop+0x25d

Calling Unsubscribe in the OnDisconnect callback causes the errorChan in the patched sse package to never be consumed. Causing readLoop go routine to never end.
This allows the sse client to reconnect. A custom ReconnectStrategy is used to ensure it reconnects indefinitely.

Closes (add link to issue here)

Dependencies

Testing

Screenshots

@ian-ross @huantt Any movement on this? The goroutine leak is quite serious, and we had to fork this library to use this PR in order to fix the issue.

@ian-ross @huantt Any movement on this? The goroutine leak is quite serious, and we had to fork this library to use this PR in order to fix the issue.

Sorry I can't help there, @atombender, I'm no longer working for GrowthBook.

@ian-ross @huantt Any movement on this? The goroutine leak is quite serious, and we had to fork this library to use this PR in order to fix the issue.

Sorry I can't help there, @atombender, I'm no longer working for GrowthBook.

Do you know if anyone is? Is there someone to ping? We're seriously considering adopting growthbook but looking how little attention it receives from maintainers is concerning. Especially that there are multiple pull request proposals to address the issues and they are just hanging stale.


Enable https

what

Add support for self-signed TLS certificates
A few minor improvements.

why

To have an end-to-end secure connection. e.g. HIPAA compliant.
This PR is related to the issue https://github.com/cloudposse/terraform-aws-jenkins/issues/21

@ivan-pinatti btw, check out the updated description. Generally, we use this convention:
## what
* thing that changed

## why
* justifications for change

@osterman all changes done.

What is your proposition then @aknysh ?
Because the URL is required to generate the self-signed certificate during the build.

@ivan-pinatti
thank you again for your contribution.
I'll review it little bit later, we should come up with a good way to solve the issue.

No problem @aknysh.
I reviewed and I think this is the only way to give the user the option (via parameter in the module) to use their own URL and to make Jenkins to auto-generate the self-signed certificate.
The other possible solution would be somehow manage to change the Jenkins build process to retrieve a valid certificate from somewhere else, perhaps AWS S3 for example.

@ivan-pinatti
thanks
it was busy few weeks for us, but I'll get to this shortly and we'll discuss the best solution

@aknysh no problem, take your time. :)
This is a new feature, no need to rush it.

@aknysh @ivan-pinatti any plans to address this? I know it's been a while.

@aknysh friendly reminder :)


adding xrt_teem

Adding xrt_teem.py only

[pre-commit.ci] autofix

pre-commit.ci autofix

@jslavin If possible, add documentation to the section of Getting Started - XRTpy Objects for users to get more information about xrt_teem before the squash and merge.

I've not taken a close look at this, but one high level suggestion would be to make the two input XRT images sunpy map objects rather than accepting two data header pairs. This would vastly simplify the metadata handling as SunPy already includes code for handling XRT headers. For example, the parsing of the filter wheel combination is already done on the XRT map source in SunPy and can be accessed as a property.
I would also suggest that the return type of the emission and temperature also be SunPy maps. That way, a user can continue to take advantage of all the usual operations (e.g. cropping, visualization) on their derived temperature maps.

Hi Will,
I have been wondering about the best types to require for inputs. Other
possibilities that I've considered are HDUList objects and filenames/paths.
The latter has the advantage that it is very simple for someone new to
solar physics. So SunPy map objects are created via map =
sunpy.map.Map(path), right? So if the path to an XRT fits data file were
provided then a map object could be created internally in xrt_teem.
One issue with returning Sunpy maps is how to set some of the metadata.
Actually that also applies to returning an HDUList. In particular I'm
thinking of things like the filters and observation date that really only
apply to one of the two images used. I suppose I could design my own
metadata for the map object, though that'd be some more work.
Jon
On Thu, Jan 26, 2023 at 12:33 AM Will Barnes @.***>
wrote:

I've not taken a close look at this, but one high level suggestion would
be to make the two input XRT images sunpy map objects rather than accepting
two data header pairs. This would vastly simplify the metadata handling as
SunPy already includes code for handling XRT headers. For example, the
parsing of the filter wheel combination is already done on the XRT map
source in SunPy and can be accessed as a property.
I would also suggest that the return type of the emission and temperature
also be SunPy maps. That way, a user can continue to take advantage of all
the usual operations (e.g. cropping, visualization) on their derived
temperature maps.
—
Reply to this email directly, view it on GitHub
https://github.com/HinodeXRT/xrtpy/pull/89#issuecomment-1404581381, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAE5OY27HQ2PW67OE2CM3HLWUIEDPANCNFSM6AAAAAASO4RUVQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

@wtbarnes it looks like the parsing of filter wheel combos is done via the measurement attribute of the xrt map object. However, the format is not consistent with that required as input to create a TemperatureResponseFundamental object in XRTpy. For example for Be-thin/Open, the xrtmap.measurement value is 'Be thin-Open' whereas the format of the input needs to be 'Be_thin' or 'Be-thin'. So until this is changed, it would require more parsing to use the xrtmap object.

Hi Will, I have been wondering about the best types to require for inputs. Other possibilities that I've considered are HDUList objects and filenames/paths. The latter has the advantage that it is very simple for someone new to solar physics. So SunPy map objects are created via map = sunpy.map.Map(path), right?

Yes, in principle one need only pass the file to Map. From a design perspective, I think it makes sense to allow as few  input types as possible. In this case, it is very easy for a user to create a map themselves just using the filepath.
Furthermore, I think that Map objects are a fairly standard data container for 2D image data (with accompanying metadata) and most Python users in the solar physics community are comfortable using/creating Map objects. In the docstring of the function, there could be a link (e.g. using the intersphinx notation that @namurphy mentioned above) to the sunpy map object showing how to do this.
aiapy takes this approach. All functions that operate on AIA images take a map as an input and return a map as an output (e.g. the correct_degradation function https://gitlab.com/LMSAL_HUB/aia_hub/aiapy/-/blob/main/aiapy/calibrate/prep.py#L113).

In particular I'm thinking of things like the filters and observation date that really only apply to one of the two images used. I suppose I could design my own metadata for the map object, though that'd be some more work.

I'm not sure I understand here. Why doesn't the metadata exposed by Map already meet these needs? I would suggest avoiding a custom metadata object both because, as you say, it involves more work and also it would reduce interoperability.

it looks like the parsing of filter wheel combos is done via the measurement attribute of the xrt map object. However, the format is not consistent with that required as input to create a TemperatureResponseFundamental object in XRTpy. For example for Be-thin/Open, the xrtmap.measurement value is 'Be thin-Open' whereas the format of the input needs to be 'Be_thin' or 'Be-thin'. So until this is changed, it would require more parsing to use the xrtmap object.

Yes, you're correct. I also recently came across this. On the sunpy side, we can definitely adjust the format here if that is desirable. I believe that the choice was arbitrary when the XRT source was created. In the meantime, you can use the following code snippet to convert between the two formats,
import sunpy.extern.parse as parse
filter_key_sunpy = xrt_map.measurement
filter_key_xrtpy = parse('{filter}-open', filter_key_sunpy.lower()) or parse('open-{filter}', filter_key_sunpy.lower())
filter_key_xrtpy = '-'.join(filter_key_xrtpy['filter'].split())

@wtbarnes with regards to my comments about the custom metadata, I was referring to returning map objects as the output of xrt_teem as you suggested. The outputs are temperature and emission measure maps and the errors on each of those. The question would then be what the metadata for those maps should be since they are created from two images, each of which have their own metadata. In my example notebook, I just used the metadata (fits header) from one of the two input images to create a map for the purposes of plotting, though that was arbitrary and the map thus lacks some of the information about how the data was created.

@wtbarnes with regards to my comments about the custom metadata, I was referring to returning map objects as the output of xrt_teem as you suggested. The outputs are temperature and emission measure maps and the errors on each of those. The question would then be what the metadata for those maps should be since they are created from two images, each of which have their own metadata. In my example notebook, I just used the metadata (fits header) from one of the two input images to create a map for the purposes of plotting, though that was arbitrary and the map thus lacks some of the information about how the data was created.

Ah I see. Sorry, I initially misunderstood. Yes, that's an interesting question. I would suggest using the make_fitswcs_header function in SunPy to create a new header that you can then use to create the output map. The docs for that can be found here. You can use the reference pixel, reference coordinate, scale, data shape, and rotation matrix from the input map (they should be essentially the same for both input maps) and then fill in the units appropriately for temperature and emission measure.

Hi Will,
My current plan is to have the outputs from xrt_teem to be SunPy maps. At
this point I plan to still do the rebinning within xrt_teem to preserve the
same functionality as in the IDL code. Mark wants us to maintain fairly
close correspondence of the XRTpy code with the IDL code.
By the way, I ran into some odd bugs at first when trying to make plots
from the map outputs. For some reason, getting the date attribute to a map
produced errors when I included the 'timesys' keyword in the metadata. In
XRT data that has the value 'UTC (TBR)'. I found that there was no problem
if I included the entire metadata from the image, which I don't want to do
since much of it is irrelevant to the temperature/EM images and some only
applies to one of the images used to produce the data. However if I remove
certain keywords, and I haven't determined exactly which yet, I get the
error:
File
/export/slavin/anaconda3/envs/xrtpy/lib/python3.11/site-packages/astropy/time/core.py:526,
in TimeBase._init_from_vals(self, val, val2, format, scale, copy,
precision, in_subfmt, out_subfmt)
524 if scale is not None:
525     if not (isinstance(scale, str) and scale.lower() in
self.SCALES):
--> 526         raise ScaleValueError(
527             f"Scale {scale!r} is not in the allowed scales "
528             f"{sorted(self.SCALES)}"
529         )
531 # If either of the input val, val2 are masked arrays then
532 # find the masked elements and fill them.
533 mask, val, val2 = _check_for_masked_and_fill(val, val2)
ScaleValueError: Scale 'utc (tbr)' is not in the allowed scales ['local',
'tai', 'tcb', 'tcg', 'tdb', 'tt', 'ut1', 'utc']
Removing the 'timesys' keyword prevents the error from occurring. Thought
that might be of interest. I could create an issue for SunPy if you want,
though it seems like a fairly specialized case.
Regards,
Jon
On Thu, Feb 2, 2023 at 11:48 AM Will Barnes @.***>
wrote:

@.**** commented on this pull request.
In xrtpy/response/xrt_teem.py
https://github.com/HinodeXRT/xrtpy/pull/89#discussion_r1094798261:

data1 = data1.astype(float)

data2 = data2.astype(float)
data1 = np.ma.masked_where(mask, data1)
data2 = np.ma.masked_where(mask, data2)
if binfac > 1:

   s = data1.shape

   ns = (s[0] // binfac, s[1] // binfac)

   rbs = (ns[0], binfac, ns[1], binfac)

   # sums the data in binfac x binfac sized regions

   d1 = data1.reshape(rbs).sum(-1).sum(1)

   d2 = data2.reshape(rbs).sum(-1).sum(1)

   # this makes a pixel masked if any of the summed pixels is masked

   # if we want to mask only if all the pixels are masked then we could

   # use prod in place of sum here

   msk = mask.reshape(rbs).sum(-1).sum(1)

   # need to convert back to bool after summing

   msk = msk.astype(bool)

   # This restores the image to the size of the original images as in the

   # IDL code:

   d1tmp = np.zeros_like(data1)

   d2tmp = np.zeros_like(data2)

   mtmp = np.zeros_like(mask, dtype=bool)

   for i in range(binfac):

       for j in range(binfac):

           d1tmp[i::binfac, j::binfac] = d1[:]

           d2tmp[i::binfac, j::binfac] = d2[:]

           mtmp[i::binfac, j::binfac] = msk[:]

   data1 = d1tmp

   data2 = d2tmp

   mask = mtmp

I've looked into the superpixel method. It seems like a good option. One
thing I found annoying about it is that it requires that the dimensions
argument have units even though they have to be integers. Is that really
necessary (or useful)?
In sunpy, all inputs and outputs must have units, which includes pixels.
If the superpixel method is called within xrt_teem, you could always
accept a unitless argument to xrt_teem and then add u.pix when calling
superpixel. The advantage of using superpixel is that xrtpy doesn't have
to worry about maintaining a custom rebin method that may potentially
behave differently than other rebin methods.
The question of including that in xrt_teem gets to the question of the
extent to which we want the XRTpy version to be consistent with the IDL
version.
Fair point. I guess I am not aware to what extent the XRT team would like
to adhere to the style/interface of the original IDL code. There of course
is an argument for wanting to be consistent with what people have used in
the past.
Also, people may not need a binned version of their image, but for the
purposes of deriving the temperature and emission measure, the extra
signal/noise might be useful. I'm inclined to leave the option to rebin in.
Fair point though I think it is not unreasonable to expect users to do
this themselves before passing the map into this function, particularly
given that it is doable using a single method on the Map object. This
also means fewer arguments to the xrt_teem function.
—
Reply to this email directly, view it on GitHub
https://github.com/HinodeXRT/xrtpy/pull/89#discussion_r1094798261, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAE5OY5A7RDIEH44HSMVMZTWVPQNFANCNFSM6AAAAAASO4RUVQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hi Will,
I have now implemented returning Maps from xrt_teem. However I identified
one behavior that's different between the IDL routine and the
XRTpy routine. Using the superpixel method for a masked image, the summing
is done treating the masked pixels as 0s. The mask itself is not summed, so
the net effect is that any summed pixel will not be masked unless all the
pixels that make it up are masked. The IDL routine on the other hand makes
any binned pixel masked if it has any of its pixels masked. One could argue
either way about the proper way to do that, though certainly you're not
getting a properly summed pixel if some of the original pixels are masked.
To change what is returned by superpixel would require modification of the
code.
Jon
On Thu, Feb 2, 2023 at 3:33 PM Jonathan Slavin @.***> wrote:

Hi Will,
My current plan is to have the outputs from xrt_teem to be SunPy maps. At
this point I plan to still do the rebinning within xrt_teem to preserve the
same functionality as in the IDL code. Mark wants us to maintain fairly
close correspondence of the XRTpy code with the IDL code.
By the way, I ran into some odd bugs at first when trying to make plots
from the map outputs. For some reason, getting the date attribute to a map
produced errors when I included the 'timesys' keyword in the metadata. In
XRT data that has the value 'UTC (TBR)'. I found that there was no problem
if I included the entire metadata from the image, which I don't want to do
since much of it is irrelevant to the temperature/EM images and some only
applies to one of the images used to produce the data. However if I remove
certain keywords, and I haven't determined exactly which yet, I get the
error:
File
/export/slavin/anaconda3/envs/xrtpy/lib/python3.11/site-packages/astropy/time/core.py:526,
in TimeBase._init_from_vals(self, val, val2, format, scale, copy,
precision, in_subfmt, out_subfmt)
524 if scale is not None:
525     if not (isinstance(scale, str) and scale.lower() in
self.SCALES):
--> 526         raise ScaleValueError(
527             f"Scale {scale!r} is not in the allowed scales "
528             f"{sorted(self.SCALES)}"
529         )
531 # If either of the input val, val2 are masked arrays then
532 # find the masked elements and fill them.
533 mask, val, val2 = _check_for_masked_and_fill(val, val2)
ScaleValueError: Scale 'utc (tbr)' is not in the allowed scales ['local',
'tai', 'tcb', 'tcg', 'tdb', 'tt', 'ut1', 'utc']
Removing the 'timesys' keyword prevents the error from occurring. Thought
that might be of interest. I could create an issue for SunPy if you want,
though it seems like a fairly specialized case.
Regards,
Jon
On Thu, Feb 2, 2023 at 11:48 AM Will Barnes @.***>
wrote:

@.**** commented on this pull request.
In xrtpy/response/xrt_teem.py
https://github.com/HinodeXRT/xrtpy/pull/89#discussion_r1094798261:

data1 = data1.astype(float)

data2 = data2.astype(float)
data1 = np.ma.masked_where(mask, data1)
data2 = np.ma.masked_where(mask, data2)
if binfac > 1:

   s = data1.shape

   ns = (s[0] // binfac, s[1] // binfac)

   rbs = (ns[0], binfac, ns[1], binfac)

   # sums the data in binfac x binfac sized regions

   d1 = data1.reshape(rbs).sum(-1).sum(1)

   d2 = data2.reshape(rbs).sum(-1).sum(1)

   # this makes a pixel masked if any of the summed pixels is masked

   # if we want to mask only if all the pixels are masked then we could

   # use prod in place of sum here

   msk = mask.reshape(rbs).sum(-1).sum(1)

   # need to convert back to bool after summing

   msk = msk.astype(bool)

   # This restores the image to the size of the original images as in the

   # IDL code:

   d1tmp = np.zeros_like(data1)

   d2tmp = np.zeros_like(data2)

   mtmp = np.zeros_like(mask, dtype=bool)

   for i in range(binfac):

       for j in range(binfac):

           d1tmp[i::binfac, j::binfac] = d1[:]

           d2tmp[i::binfac, j::binfac] = d2[:]

           mtmp[i::binfac, j::binfac] = msk[:]

   data1 = d1tmp

   data2 = d2tmp

   mask = mtmp

I've looked into the superpixel method. It seems like a good option. One
thing I found annoying about it is that it requires that the dimensions
argument have units even though they have to be integers. Is that really
necessary (or useful)?
In sunpy, all inputs and outputs must have units, which includes pixels.
If the superpixel method is called within xrt_teem, you could always
accept a unitless argument to xrt_teem and then add u.pix when calling
superpixel. The advantage of using superpixel is that xrtpy doesn't have
to worry about maintaining a custom rebin method that may potentially
behave differently than other rebin methods.
The question of including that in xrt_teem gets to the question of the
extent to which we want the XRTpy version to be consistent with the IDL
version.
Fair point. I guess I am not aware to what extent the XRT team would like
to adhere to the style/interface of the original IDL code. There of course
is an argument for wanting to be consistent with what people have used in
the past.
Also, people may not need a binned version of their image, but for the
purposes of deriving the temperature and emission measure, the extra
signal/noise might be useful. I'm inclined to leave the option to rebin in.
Fair point though I think it is not unreasonable to expect users to do
this themselves before passing the map into this function, particularly
given that it is doable using a single method on the Map object. This
also means fewer arguments to the xrt_teem function.
—
Reply to this email directly, view it on GitHub
https://github.com/HinodeXRT/xrtpy/pull/89#discussion_r1094798261, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAE5OY5A7RDIEH44HSMVMZTWVPQNFANCNFSM6AAAAAASO4RUVQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

Mark wants us to maintain fairly close correspondence of the XRTpy code with the IDL code.

Understood

By the way, I ran into some odd bugs at first when trying to make plots from the map outputs. For some reason, getting the date attribute to a map produced errors when I included the 'timesys' keyword in the metadata. In XRT data that has the value 'UTC (TBR)'. I found that there was no problem if I included the entire metadata from the image.

I suspect the issue is because, in the XRT map source, there is a fix for the non-standard TIMESYS entry "UTC (TBR)"
https://github.com/sunpy/sunpy/blob/a846ce09322cd6ded64da35ab0dc3ec90d970b44/sunpy/map/sources/hinode.py#L63-L67. When you include all of the metadata from the original image, Map recognizes this as an XRT map and thus handles the key appropriately. However, when you only use subset of the metadata, specifically when you don't set the INSTRUME key to "XRT", Map does not know how to handle this TIMESYS key since the fix is only applied for XRT maps. When you remove the TIMESYS key, map falls back to the default which is UTC https://github.com/sunpy/sunpy/blob/main/sunpy/map/mapbase.py#L815
I agree with you re: not including all the original metadata as it no longer applies to the resulting EM/temperature maps. My suggested fix would be to set the TIMESYS keyword to "UTC". Note also that if you use the header helper function (see my comment above), this will be handled for you.

I have now implemented returning Maps from xrt_teem. However I identified one behavior that's different between the IDL routine and the XRTpy routine. Using the superpixel method for a masked image, the summing is done treating the masked pixels as 0s. The mask itself is not summed, so the net effect is that any summed pixel will not be masked unless all the pixels that make it up are masked. The IDL routine on the other hand makes any binned pixel masked if it has any of its pixels masked. One could argue either way about the proper way to do that, though certainly you're not getting a properly summed pixel if some of the original pixels are masked. To change what is returned by superpixel would require modification of the code.

That's an interesting point. In sunpy, the logic of masked pixels in superpixel is handled by NumPy masked arrays which set masked pixels to zero when summing. Would you mind creating an issue about this on the sunpy repo? I would be interested to hear other's opinions on this behavior and whether there is room for flexibility.
Note that you could you could recreate the kind of mask you're talking about by passing the mask attached to your original Map to reshape_image_to_4d_superpixel and then summing over the appropriate dimensions.

pre-commit.ci autofix

@jslavin — I made a few edits directly to this pull request to try getting the docstring to show up in the documentation preview for xrt_teem, and to ignore a linter error regarding substitutions.  It'll be necessary to do a git pull to get the updates.
There seems to be a problem with the reStructuredText references to other packages not forming links.  This may be an issue with how intersphinx is set up, though I wasn't able to find any problems.  I'll have to look into this a bit more to figure out what's happening here.  I think there was also just a new release of Sphinx, which might(?) have something to do with it.  In any case, that's outside the scope of this pull request, so it shouldn't hold this up.

Hi Nick,
The documentation looks good. Thanks for your work in getting that done.
Jon
On Mon, Feb 13, 2023 at 6:36 PM Nick Murphy @.***>
wrote:

@jslavin https://github.com/jslavin — I made a few edits directly to
this pull request to try getting the docstring to show up in the documentation
preview for xrt_teem
https://xrtpy--89.org.readthedocs.build/en/89/api/xrtpy.response.xrt_teem.xrt_teem.html#xrt-teem,
and to ignore a linter error regarding substitutions. It'll be necessary to
do a git pull to get the updates.
There seems to be a problem with the reStructuredText references to other
packages not forming links. This may be an issue with how intersphinx is
set up, though I wasn't able to find any problems. I'll have to look into
this a bit more to figure out what's happening here. I think there was also
just a new release of Sphinx, which might(?) have something to do with it.
In any case, that's outside the scope of this pull request, so it shouldn't
hold this up.
—
Reply to this email directly, view it on GitHub
https://github.com/HinodeXRT/xrtpy/pull/89#issuecomment-1428862674, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAE5OY3EXUIHHCIHSD53H5LWXLAPHANCNFSM6AAAAAASO4RUVQ
.
You are receiving this because you were mentioned.Message ID:
@.***>


Enhancement | Add the enhancements category to our release template l…

What?

Add the enhancements category to our release template (https://github.com/binbashar/le-tf-infra-aws/blob/master/.github/release-drafter.yml) like TF does for its releases (eg: https://github.com/hashicorp/terraform/releases/tag/v1.3.0-beta1):

categories:
  - title: '🚀 Features'
    labels:
      - 'feature'
  - title: '✨ Enhancements'
    labels:
      - 'enhancement'

Why?

✅ Improve release clarity with an specific category intended for the upgrades of terraform providers/modules and those related things that are not really features (or fixes)

@diego-ojeda-binbash :)


Insert statements within remote replica transactions fail with transaction timed-out

Insert statements that are executed within concurrent transactions initiated from a remote replica connection seem to hang indefinitely until the transaction times out and the statements fail. Below is a minimal reproducer for the same:
use libsql::{Builder, Connection, Result};

#[tokio::main]
async fn main() {
    let db_url = "http://localhost:8080";
    let replica = Builder::new_remote_replica(
        "/tmp/embedded_transaction.db",
        db_url.to_string(),
        String::new(),
    )
    .build()
    .await
    .unwrap();
    let replica_conn_1 = replica.connect().unwrap();
    let replica_conn_2 = replica.connect().unwrap();
    let replica_conn_3 = replica.connect().unwrap();

    setup_db(replica_conn_1.clone()).await.unwrap();

    // This works as expected.
    let replica_task_1 = db_work(replica_conn_1).await;
    assert!(replica_task_1.is_ok());

    // If we execute concurrently, two tasks
    let replica_task_2 = tokio::task::spawn(async move { db_work(replica_conn_2).await });
    let replica_task_3 = tokio::task::spawn(async move { db_work(replica_conn_3).await });

    let (task_2_res, task_3_res) = tokio::join!(replica_task_2, replica_task_3);
    let replica_task_2_res = task_2_res.unwrap();
    let replica_task_3_res = task_3_res.unwrap();

    if replica_task_2_res.is_err() {
        eprintln!("Task 2 failed: {:?}", replica_task_2_res);
    }
    if replica_task_3_res.is_err() {
        eprintln!("Task 3 failed: {:?}", replica_task_3_res);
    }

    // One of these concurrent tasks fail currently. Both tasks should succeed.
    assert!(replica_task_2_res.is_ok());
    assert!(replica_task_3_res.is_ok());
}

async fn db_work(conn: Connection) -> Result<()> {
    let tx = conn.transaction().await?;
    // Some business logic here...
    tx.execute("INSERT INTO test (name) VALUES (?1)", ["somename"])
        .await?;
    tx.commit().await?;
    Ok(())
}

async fn setup_db(conn: Connection) -> Result<()> {
    conn.execute(
        "CREATE TABLE IF NOT EXISTS test (id INTEGER PRIMARY KEY, name TEXT)",
        (),
    )
    .await?;
    Ok(())
}

A bit simplified reproducer:
use libsql::{Builder, Connection, Result};

#[tokio::main]
async fn main() {
    let db_url = "http://localhost:8080";
    let replica = Builder::new_remote_replica(
        "/tmp/embedded_transaction.db",
        db_url.to_string(),
        String::new(),
    )
    .build()
    .await
    .unwrap();
    let replica_conn_2 = replica.connect().unwrap();
    let replica_conn_3 = replica.connect().unwrap();

    setup_db(replica_conn_2.clone()).await.unwrap();

    // If we execute concurrently, two tasks
    let replica_task_2 = db_work(replica_conn_2);
    let replica_task_3 = db_work(replica_conn_3);

    let (replica_task_2_res, replica_task_3_res) = tokio::join!(replica_task_2, replica_task_3);

    if replica_task_2_res.is_err() {
        eprintln!("Task 2 failed: {:?}", replica_task_2_res);
    }
    if replica_task_3_res.is_err() {
        eprintln!("Task 3 failed: {:?}", replica_task_3_res);
    }

    // One of these concurrent tasks fail currently. Both tasks should succeed.
    assert!(replica_task_2_res.is_ok());
    assert!(replica_task_3_res.is_ok());
}

async fn db_work(conn: Connection) -> Result<()> {
    let tx = conn.transaction().await?;
    // Some business logic here...
    tx.execute("INSERT INTO test (name) VALUES (?1)", ["somename"])
        .await?;
    tx.commit().await?;
    Ok(())
}

async fn setup_db(conn: Connection) -> Result<()> {
    conn.execute(
        "CREATE TABLE IF NOT EXISTS test (id INTEGER PRIMARY KEY, name TEXT)",
        (),
    )
    .await?;
    Ok(())
}

Works just fine without embedded replica:
let replica = Builder::new_remote(
        db_url.to_string(),
        String::new(),
    )

It seems that one transaction hangs on commit while the other hangs on execute.

One issue it might be is read_your_writes.
The first transaction is waiting for embedded replica to sync up to the write the transaction did and this is why commit takes that long.
The second transaction is blocked by the first.
One plot I got was:
Starting db work
Starting db work
Executing db work
Executing db work
Committing db work
Task 2 failed: Err(RemoteSqliteFailure(2, 0, "Transaction timed-out"))
Task 3 failed: Err(Replication(Client(Status { code: Unknown, message: "connection closed before message completed", source: Some(hyper::Error(IncompleteMessage)) })))
thread 'main' panicked at src/main.rs:33:5:
assertion failed: replica_task_2_res.is_ok()

Building the db with .read_your_writes(false) should help for read your writes issue but there is apparently another issue as this does not make the code to finish successfully.
let replica = Builder::new_remote_replica(
        "/tmp/embedded_transaction.db",
        db_url.to_string(),
        String::new(),
    )
    .read_your_writes(false)
    .build()
    .await
    .unwrap();

Here's a deterministic reproducer:
    eprintln!("Start tx1");
    let tx2 = replica_conn_2.transaction().await.unwrap();
    eprintln!("Start tx2");
    let tx3 = replica_conn_3.transaction().await.unwrap();
    eprintln!("Execute tx1");
    tx2.execute("INSERT INTO test (name) VALUES (?1)", ["somename"])
        .await.unwrap();
    eprintln!("Execute tx2");
    tx3.execute("INSERT INTO test (name) VALUES (?1)", ["somename"])
        .await.unwrap();
    eprintln!("Commit tx1");
    tx2.commit().await.unwrap();
    eprintln!("Commit tx2");
    tx3.commit().await.unwrap();
    eprintln!("Done");

Start tx1
Start tx2
Execute tx1
Execute tx2
Commit tx1
thread 'main' panicked at src/main.rs:44:24:
called `Result::unwrap()` on an `Err` value: RemoteSqliteFailure(2, 0, "Transaction timed-out")

Here's a deterministic reproducer:
    eprintln!("Start tx1");
    let tx2 = replica_conn_2.transaction().await.unwrap();
    eprintln!("Start tx2");
    let tx3 = replica_conn_3.transaction().await.unwrap();
    eprintln!("Execute tx1");
    tx2.execute("INSERT INTO test (name) VALUES (?1)", ["somename"])
        .await.unwrap();
    eprintln!("Execute tx2");
    tx3.execute("INSERT INTO test (name) VALUES (?1)", ["somename"])
        .await.unwrap();
    eprintln!("Commit tx1");
    tx2.commit().await.unwrap();
    eprintln!("Commit tx2");
    tx3.commit().await.unwrap();
    eprintln!("Done");

Start tx1
Start tx2
Execute tx1
Execute tx2
Commit tx1
thread 'main' panicked at src/main.rs:44:24:
called `Result::unwrap()` on an `Err` value: RemoteSqliteFailure(2, 0, "Transaction timed-out")

Ok this is a bit too simplistic because Execute tx2 has to wait for Commit tx1 so it's a deadlock

Deterministic reproducer this time without deadlock:
    eprintln!("Start tx1");
    let before = Instant::now();
    let tx1 = replica_conn_2.transaction().await.unwrap();
    let after = Instant::now();
    eprintln!("Finished in {:?}", after.duration_since(before));
    let before = Instant::now();
    eprintln!("Start tx2");
    let tx2 = replica_conn_3.transaction().await.unwrap();
    let after = Instant::now();
    eprintln!("Finished in {:?}", after.duration_since(before));
    let before = Instant::now();
    eprintln!("Execute tx1");
    tx1.execute("INSERT INTO test (name) VALUES (?1)", ["somename"])
        .await.unwrap();
    let after = Instant::now();
    eprintln!("Finished in {:?}", after.duration_since(before));
    let before = Instant::now();
    eprintln!("Execute tx2 and Commit tx1 concurrently");
    let exec_fut = tx2.execute("INSERT INTO test (name) VALUES (?1)", ["somename"]);
    let commit_fut = tx1.commit();
    let (exec, commit) = join!(
        exec_fut,
        commit_fut);
    let after = Instant::now();
    eprintln!("Finished in {:?}", after.duration_since(before));
    exec.unwrap();
    commit.unwrap();
    let before = Instant::now();
    eprintln!("Commit tx2");
    tx2.commit().await.unwrap();
    let after = Instant::now();
    eprintln!("Finished in {:?}", after.duration_since(before));
    eprintln!("Done");

Start tx1
Finished in 17.477393ms
Start tx2
Finished in 22.61203ms
Execute tx1
Finished in 25.201501ms
Execute tx2 and Commit tx1 concurrently
Finished in 5.368149362s
thread 'main' panicked at src/main.rs:63:12:
called `Result::unwrap()` on an `Err` value: RemoteSqliteFailure(2, 0, "Transaction timed-out")

After adding more logging it seems that exec for tx2 finishes before commit for tx1 so it seems that exec for tx2 blocks commit for tx1.
    eprintln!("Start tx1");
    let before = Instant::now();
    let tx1 = replica_conn_2.transaction().await.unwrap();
    let after = Instant::now();
    eprintln!("Finished in {:?}", after.duration_since(before));
    let before = Instant::now();
    eprintln!("Start tx2");
    let tx2 = replica_conn_3.transaction().await.unwrap();
    let after = Instant::now();
    eprintln!("Finished in {:?}", after.duration_since(before));
    let before = Instant::now();
    eprintln!("Execute tx1");
    tx1.execute("INSERT INTO test (name) VALUES (?1)", ["somename"])
        .await.unwrap();
    let after = Instant::now();
    eprintln!("Finished in {:?}", after.duration_since(before));
    let before = Instant::now();
    eprintln!("Execute tx2 and Commit tx1 concurrently");
    let exec_fut = async {
        let res = tx2.execute("INSERT INTO test (name) VALUES (?1)", ["somename"]).await;
        let after = Instant::now();
        eprintln!("Exec finished in {:?}", after.duration_since(before));
        res
    };
    let commit_fut = async {
        let res = tx1.commit().await;
        let after = Instant::now();
        eprintln!("Commit finished in {:?}", after.duration_since(before));
        res
    };
    let (exec, commit) = join!(
        exec_fut,
        commit_fut);
    let after = Instant::now();
    eprintln!("Finished in {:?}", after.duration_since(before));
    exec.unwrap();
    commit.unwrap();
    let before = Instant::now();
    eprintln!("Commit tx2");
    tx2.commit().await.unwrap();
    let after = Instant::now();
    eprintln!("Finished in {:?}", after.duration_since(before));
    eprintln!("Done");

Start tx1
Finished in 20.555148ms
Start tx2
Finished in 17.230765ms
Execute tx1
Finished in 20.49774ms
Execute tx2 and Commit tx1 concurrently
Exec finished in 5.005591167s
Commit finished in 5.0059767s
Finished in 5.005993392s
thread 'main' panicked at src/main.rs:73:12:
called `Result::unwrap()` on an `Err` value: RemoteSqliteFailure(2, 0, "Transaction timed-out")

It seems that the lock in ProxyService::execute is being held during the execution of the query even though it is just protecting the creation of the Libsql connection before the execution of the query. This could lead to a situation when a transaction keeps lock but is block by another transaction and then the other transaction can't proceed because the first transaction keeps the lock.

It seems that the lock in ProxyService::execute is being held during the execution of the query even though it is just protecting the creation of the Libsql connection before the execution of the query. This could lead to a situation when a transaction keeps lock but is block by another transaction and then the other transaction can't proceed because the first transaction keeps the lock.

Is this lock only held during execution of a query on an embedded replica connections? Since, txns on remote connections are working fine. Or, is the ProxyService only used for embedded replicas and not remote connections?

It seems that the lock in ProxyService::execute is being held during the execution of the query even though it is just protecting the creation of the Libsql connection before the execution of the query. This could lead to a situation when a transaction keeps lock but is block by another transaction and then the other transaction can't proceed because the first transaction keeps the lock.

Is this lock only held during execution of a query on an embedded replica connections? Since, txns on remote connections are working fine. Or, is the ProxyService only used for embedded replicas and not remote connections?

I believe ProxyService is only used for embedded replicas.
Here's a fix: https://github.com/tursodatabase/libsql/pull/1339