Examine using the XML data available from the courts?

I just discovered that there is an XML version of the data that this scraper is ingesting. It's not wonderful XML, but it's decent. If we ever rewrite this code, we should see if using it is easier/more reliable.
Here's an example link:
https://ecf.cod.uscourts.gov/cgi-bin/CourtInfo.pl?output=xml
I think we could probably get these links for pretty much every jurisdiction, if we wanted.

Probably sturdier, but I doubt it will make the scraping much faster. Might as well implement it and see, though.

Yeah, I'd expect speed to be nearly identical, but maybe the data is more structured/easier to work with?

So I tried implementing this just now, but the only problem is that I don't have links to the individual courts' /CourtInfo.pl page.
I only have the general PACER courtinfo URLs that look like this: https://www.pacer.gov/psco/cgi-bin/courtinfo.pl?court=E_ALMDC&output=xml
That's because I'm scraping this page to get the list of courts: https://www.pacer.gov/psco/cgi-bin/links.pl

I think you could use the domains from each of the URLs on the page that lists the courts. For example, Alabama links to:
https://ecf.almd.uscourts.gov/
If you just tack /cgi-bin/CourtInfo.pl?output=xml on the end, it seems to work:
https://ecf.almd.uscourts.gov/cgi-bin/CourtInfo.pl?output=xml
Could that work?

That could definitely work.
Unfortunately, I don't think the XML version has any info that the web page doesn't, for example for the U.S. Bankruptcy Court for the Western District of North Carolina. the go live date is missing on the web page and is also missing from the XML: https://ecf.ncwb.uscourts.gov/cgi-bin/CourtInfo.pl?output=xml

Feels like something to do later, if this is ever rewritten from scratch.

The XML is definitely missing valuable information. Today I observed:
https://ecf.dcd.uscourts.gov/cgi-bin/CourtInfo.pl

Case Number Format
O:YY-TY-#####-INI-RIN example: 1:18-cv-00374

RSS FeedDocket entries of type: all
Last 24 hours' entries - Internet

but https://ecf.dcd.uscourts.gov/cgi-bin/CourtInfo.pl?output=xml:
<opt>
  <CaseNo>O:YY-TY-#####-INI-RIN <i>example:</i> 1:18-cv-00374
</CaseNo>

Also, just to record my note from Twitter this afternoon: https://twitter.com/johnhawkinson/status/965717770120855552

I guess I sort of wish @audiodude's scraper https://court-version-scraper.herokuapp.com/  tracked historical versions and also presented the entire XML. Was this the change from rss_outside.pl to readyDockets?

Although I guess I should have written s/XML/HTML/


Multiple schema support

I am on version 1.0.0 of graphql on Android. Can i still use multiple schema.json files in my single app? Or do i need to upgrade to 1.3x?

Hi! 1.0.0 is pretty old and uses the Groovy gradle plugin. If I remember correctly, you can have multiple schema.json files if you organise them like this:
src/main/graphql/service1/schema.json
src/main/graphql/service1/query.json
src/main/graphql/service2/schema.json
src/main/graphql/service2/query.json

I would recommend updating to a more recent version that allows to define schemas explicitely though.

Thanks @martinbonnin For the prompt reply. Also how do I read these schema.json files in my build.gradle?

If I remember correctly, you don't need to configure the location of the schema.json files in your build.gradle files. The plugin had some logic to find them automatically. But again, this was a long time ago and this logic was pretty complex. It's one of the reason we rewrote the gradle plugin in Kotlin. So updating to a more recent version is certainly your best bet.

Hey @martinbonnin thanks again! We will move to a newer version soon, but for sometime more, we will have to stick to this old version. Hence these doubts.
I made changes as per your suggestions, but the build kept failing because the model classes were not getting generated. Any help here?

If you make a small reproducer somewhere I can take a quick look but 1.0.0 is not something we typically support.
Another option that might be easier until you update is to make a separate module for your second schema.

@martinbonnin - Some clean builds and a new issue popped up. There are 2 schemas - schema1.json and schema2.json.
Schema is not configured for subscriptions -> this error has been coming for the schema2.json. In the same folder as schema2.json, i have a subscription.graphql file present too.
any help here?

Invalid or incomplete schema, unknown type: SubscriptionHasThrowable. Ensure that a full introspection query is used in order to build a client schema.
I have one IntrospectionQuery.graphql in my project.

Invalid or incomplete schema, unknown type: SubscriptionHasThrowable. Ensure that a full introspection query is used in order to build a client schema

That's an error from the old node cli. Not really sure what this is. An "introspection query" is used to get schema.json from your backend. Usually it's not something you would add to the queries you use in your project so I don't think you need any IntrospectionQuery.graphql

@martinbonnin - Hey Martin, I made a different sample project in which I bumped up the version to 2.3.1 and used the 2 different schemas and everything worked fine. Project compiled, built and run successfully. Model classes generated properly.
Then I simply bumped down the version back to 1.0.0 (and made the necessary changes, specified outputPackageName etc), but here the model classes were not generated at all. Not sure why.

Hi! I'm sorry I won't be able to help much more there.
Too many things have changed since 1.0.0 and updating your project to a more recent version is the way forward. As said above, if you really really really want to keep 1.0.0, make two gradle modules for your two different schemas.

Thanks for all the help!


运行或打包时出现build failed

你好，
请问下，我在编译或者打包的过程中，有几率会出现编译失败的情况，但这个情况不是每次都出现，而且只要多运行几次就能编译过去了，现在用的是最新的版本

Task :app:transformClassesWithDebounceForRelease
Wrote file status to file:///Users/zhangtao/project/YBJK_Android/app/build/outputs/debounce/logs/release/files.txt
====> COST: 18147 msavaResource

Task :app:outputMappingForRelease FAILED
Wrote TXT report to file:///Users/zhangtao/project/YBJK_Android/app/build/outputs/debounce/logs/release/classes.txt

FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':app:outputMappingForRelease'.

Cannot invoke method hasDebouncedMethod() on null object

Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

Get more help at https://help.gradle.org

BUILD FAILED in 1m 40s

Hi,
I also notice this sometimes, cleaning the project and making the project again helps
@SmartDengg I also noticed that I had to do Invalidate Caches / Restart... whenever I make changes to the exclusions specified, for the new exclusions to be effective
@SmartDengg thanks for this Gradle plugin, I've been looking for something like this for a very long time

I still get this error with the latest release:

Task :app:outputMappingForDebug FAILED
Wrote TXT report to file:///Users/derrickcroes/developer/openpyn-nordvpn-juiceSSH/app/build/outputs/debounce/logs/debug/classes.txt

Execution failed for task ':app:outputMappingForDebug'.

Cannot invoke method hasDebouncedMethod() on null object


feat: donate with crypto on the bottom right

Closes #196

The space between [FAQ, Donation Policy] and [Donate with Crypto..] does not seems to be ok, they should be equal.

@martinbianchi
Paddings fixed :D
I also made the frequency popover disappear when you select a freq, take a look


[iOS] App lifecycle methods (OnResume, OnStart, OnSleep...) are not triggered while popup is open

Replication Procedure:

Open a popup

Try to trigger an App lifecycle method (OnResume, OnStart, OnSleep...) while the popup is open

Actual behavior :
The App is not executing the code inside of the App lifecycle methods.
Expected behavior:
The App executes the code inside of the App lifecycle methods.
Additional note
It is working as expected on Android.

Do you have any sample for this? I cannot reproduce this issue, it works for iOS and Android for me

I could solve it adding this code in Platforms/iOS/AppDelegate.cs
        public override void OnActivated(UIApplication application)
        {
            (application.ConnectedScenes.AnyObject as UIWindowScene)?.Windows.FirstOrDefault()?.MakeKeyWindow();
            base.OnActivated(application);
        }

        public override void OnResignActivation(UIApplication application)
        {
            (application.ConnectedScenes.AnyObject as UIWindowScene)?.Windows.FirstOrDefault()?.MakeKeyWindow();
            base.OnResignActivation(application);
        }


ENM is not creating AP on Pi 3 + ESP8266

Tried pushing ENM to Raspberry Pi 3, Resin OS 2.3.0+rev1(dev), ESP8266, but the Pi is not creating AP at all.

Have you created a dependent application and set the RESIN_HOST_TYPE=esp8266 Fleet Configuration?

I got it working! I was setting RESIN_HOST_TYPE=esp8266 and RESIN_SUPERVISOR_DELTA=1 to the host application, not on dependent side! Thanks!


What security should be required for the /health endpoint?

In the security section of  the document it says
HTTP Digest Auth MUST be one supported authentication mechanism
HTTP Digest Auth SHOULD be the default algorithm for the HTTP protocol binding
HTTP digest auth is optional for JEE so why would want to require it here?  Instead, since JWT will be included in Microprofile 1.2 shouldn't we require using JWT?  It seems it would be no extra effort to do so.
Also, since both CDI and JWT are both present, a HealthCheck implementation should be able to inject @Principal and @JsonWebToken.  Wouldn't we want to declare this in the spec?

If an MP-JWT implementation co-exists with an MP-HEALTH implementation, it would be automatic that one can inject the MP-JWT principal and token claims. It could be referenced from the HealthCheck spec, but I don't see that there is anything to declare.
In terms of requiring MP-JWT auth for the HealthCheck endpoints, I can see that this would be potentially be more difficult than supporting Digest Auth as MP-JWT is a custom authentication scheme not even mentioned by the servlet spec, while Digest Auth is mentioned as something that should be supported.
And, even if it is supported by the underlying container, it may be an issue to configure the HealthCheck clients to support obtaining a bearer token.
While we should define interop of the MP features, I think we want to keep the 1.0 release rather simple and build up requirements as we go.

So, what do you suggest, Scott?  That we remove all authentication
requirements for the /health endpoint in this first release?  Seems to
leave this quite open...  Do you think not specifying any authentication is
sufficient?  And, leave it to an "implementation detail"?
Since both JWT and HealthCheck are proposed for MP 1.2, then specifying a
dependency on JWT might be okay.  Kind of like our previous dependencies on
CDI.
Following up on some of your comments, do you see HealthCheck being used
outside of MicroProfile and that's why you don't want to have the
dependency on JWT?
Thanks, Kevin
On Thu, Aug 17, 2017 at 7:14 PM, Scott M Stark<EMAIL_ADDRESS>wrote:

If an MP-JWT implementation co-exists with an MP-HEALTH implementation, it
would be automatic that one can inject the MP-JWT principal and token
claims. It could be referenced from the HealthCheck spec, but I don't see
that there is anything to declare.
In terms of requiring MP-JWT auth for the HealthCheck endpoints, I can see
that this would be potentially be more difficult than supporting Digest
Auth as MP-JWT is a custom authentication scheme not even mentioned by the
servlet spec, while Digest Auth is mentioned as something that should be
supported.
And, even if it is supported by the underlying container, it may be an
issue to configure the HealthCheck clients to support obtaining a bearer
token.
While we should define interop of the MP features, I think we want to keep
the 1.0 release rather simple and build up requirements as we go.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/eclipse/microprofile-health/issues/47#issuecomment-323226912,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHXkL8lSRGv6Ldirgs3mFs8z3yfNDyOMks5sZNdXgaJpZM4O63dv
.

I agree with Scott:

While we should define interop of the MP features, I think we want to keep the 1.0 release rather simple and build up requirements as we go.

The specs  should be interopable yes, but I am against hard dependencies (i.e. making JWT auth a mandatory element in the health spec). That said and in response to @kwsutter 's concern

we remove all authentication
requirements for the /health endpoint in this first release?  Seems to
leave this quite open...

Some authentication mechanism would be good to enforce in 1.0 and digest is the mandatory requirement in the current spec. The JEE argument brought forward by @rlcundiff is less of a concerns to me. Even if it's optional in JEE, I think basically all HTTP server implementations support it. Digest strikes a good balance between out-of-the-box experience (think setup and configuration) and actual security.
Payload constraints
An alternative to actual authentication in 1.0 could be working around payload constraints. Think of it: authentication is really just required if something vulnerable or secret is exposed. IMO this is the response payload of HealthCheck procedures. Hence I believe an alternative, much more simple approach to bypass authentication needs would be toggles that suppress the response payload, i.e.:
**@Health(sensitive=true)**
@ApplicationScoped
public class SecuredCheck implements HealthCheck {

    public Response call() {
        return Response.named("successful-check").up();
    }
}

In the above pseudo code example (we don't yet have the @Health annotation), the usage of @Health(sensitive=true) indicates that the response payload will be suppressed and only the outermost protocol status codes (i.e. 503 and 200 used to signal the system state).
Some general thoughts on security
If you think on a whole-system level (i.e. including the health consumers) then authentication becomes quite a different twist. The spec says health checks are primarily intended to be used for machine-to-machine communication and could be used by human operators, but that's a secondary concern.
I believe the typical cloud platform use cases is machine-to-machine and significantly constraints the ability to introduce authentication on this level of interaction. For instance I don't know of any cloud platform that leverages health checks that actually supports authentication (someone please correct me here).
From this point of view authentication really just addresses the use cases that the spec currently declares as secondary concerns (usage by human operators).

@kwsutter, no, the concern is not HeathCheck is used outside of MP environments, the concern is what @heiko-braun brings up with the point that the most common user of the health check are containerized/cloud environments where there is some question as to whether a custom bearer token auth mechanism like JWT could be easily introduced.
What I would propose is that we say security SHOULD be supported the the /heath endpoint, and that DIGEST and MP-JWT SHOULD be supported, but not go beyond that. As @heiko-braun mentions, I expect that most cloud environments will be using the trusted domain feature to bypass a security check altogether.

So what is the feeling about updating the security section of the HC spec to the following:

A producer MUST enforce security on all health check invocations if the context associated with the endpoint has been configured as secure.

IMHO, we should not go this route.  Environments like AWS explicitly do not support authentication.  So this would be a show stopper for me.   If we want to make security an option, that is fine.  But I would avoid the issue fault tolerance and just say that by default no authentication, however you can configure authentication via some means, possibly via the MP Config API.

@johnament We thought of an alternative with the following characteristics:

All response payload would be suppressed by default for non-authenticated requests
This removes the need for authentication all together as we would not be exposing sensitive data to un-authenticated requests
Meta data (i.e. @Health(sensitive=false)) could be used to override the default behaviour per health check

Does this sounds like a reasonable balance between out-of-the-box experience and security needs?

Sure, but I'm assuming for v1 you're only going to mention the first bullet?

I think we can do all three bullets. The annotation is alteady there

I would be against bullet 3 in the current form.  If I'm providing a library that enables health out of the box, I wouldn't want my library to determine whether or not a check is considered sensitive.
bullet 2 isn't a spec item.

@starksm64 @johnament Let's move this to 1.1 It requires further thought.

I would suggest the following for 1.0:
Change the spec to recommend auth protocols, rather than requiring them and making it explict that neither of this is enforced, but up to the implementation.
All of this require much further thought, especially considering that practically no cloud platforms support auth with health checks

@rlcundiff I'll leave this open and created #98 for 1.0

For reference, Spring supports authentication for the health check endpoints generated by their actuators. As noted, it acts as a filter on the returned payload (it has no bearing on which checks are executed). Unauthenticated requests get the sparest of answers {"status":"UP"} or {"status":"DOWN"}. You only get the details if you're logged in. To me, that is the difference between use by automation (up/down) and use by a human (what exactly is going on with this server...).

Thanks @ebullient , that works for me too. Either no payload at all for un-authenticated requests or just the status attribute, but maybe along with the name of the check.

I am closing this (question) as resolved. It seems we know how we want to address this problem and moving forward wi'll use  #2 to track progress. I will carry over the relevant parts to #2


Consents: add admin field text representation of a consent term

When displaying all consents for a given person, we're missing a (possibly) short form text describing what the consent stands for.
For example, may-contact should have a corresponding textual representation of May contact. Note that it's not always the same as changing from hyphenated to title case.

@elichad I think you can work on this if you have some time.
The use case is for example here: https://github.com/carpentries/amy/pull/2335/files#diff-a97c97acc5cd76b1c48999bacdc2b399b62f678806ab48b559dcc0944c366536R278
Currently we hard-code the consent label as key in this dictionary.
A different approach is used in instructor_dashboard: https://github.com/carpentries/amy/pull/2335/files#diff-58d69c1b96f35b50169a3e31901cfa9f3c71936c62e835196ee4a0a68a817792R146
Here a consent.term.key (https://github.com/carpentries/amy/pull/2335/files#diff-f2062243d2135c5b8d1a5580222ec6977b34f42980c40798cb487c6ca65ca83bR94) is used as dictionary key to retrieve it in the template. The actual label displayed to the user is hardcoded in the template: https://github.com/carpentries/amy/pull/2335/files#diff-cc32e702a6a8e899fe39ede7eb631a9346697a634935472f71e991432e0f2137R40
I would consider adding the label displayed in instructor dashboard to the term model, too.


AWS lambda layer version retention

Currently creating a lambda layer uses the layerVersion function. This deletes all previous versions of the layer. This could cause issues with shared common layers between lambdas if not all of the lambdas are updated at once.
Below could be an example of how to call the function with version retention - taken from an AWS CDK pull request to solve the same issue:
const lambdaLayer = new aws.lambda.LayerVersion("lambda_layer", {
    compatibleRuntimes: ["python3.8"],
    code: new pulumi.asset.FileArchive("./layers/"),
    layerName:  "lambda_layer_test",
    removalPolicy: aws.lambda.LayerVersion.RemovalPolicy.RETAIN
});

Also tracked upstream in https://github.com/hashicorp/terraform-provider-aws/issues/17668


read: restore aggregations

If reduce by has a missing field, we take the field out of the aggregation.
But if it's reduce by -every, we need to put it back for future batches.
Fixes #129
Rework of #130

@davidvgalbraith

+1


Add cn-north-1 to valid check

Signed-off-by: jhaohai<EMAIL_ADDRESS>Add cn-north-1 to the valid check because the aws driver supports this region in fact.

Current coverage is 51.22%

Merging #1660 into master will decrease coverage by -6.67% as of f76c622

@@            master   #1660   diff @@
======================================
  Files          122     122
  Stmts        10683   10684     +1
  Branches       840     747    -93
  Methods          0       0
======================================
- Hit           6185    5473   -712
+ Partial        840     747    -93
- Missed        3658    4464   +806

Review entire Coverage Diff as of f76c622

Uncovered Suggestions

+0.30% via .../driver/s3-aws/s3.go#733...764
+0.28% via .../driver/s3-aws/s3.go#269...298
+0.22% via .../driver/s3-aws/s3.go#885...907
See 7 more...

Powered by Codecov. Updated on successful CI builds.

LGTM


align with models 0.3.1

What: Changes to align with models openapi.yaml version 0.3.1
Need to account for format and model names that conflict with generated classes ending in List.

@ajbalogh wouldn't SnappiIter (base class) and ObjIter (derived class) be more appropriate ? We also have lesser chances of seeing Iter in node names.

I agree - let me open an issue and align with that naming convention.


PLEASE UPDATE THIS PROJECT.. PROJECT BUILD NOT WORKING

project runs using npm start.. but whenever I create a build for this project, it doesn't work.. kindly update the project along with dependencies

I will make this happen. thanks for reporting.

error: package "@mui/x-tree-view" with tag "https://pkg.csb.dev/mui/mui-x/commit/1f23b33d/@mui/x-tree-view" not found, but package exists


Uncaught Error: Unable to build data connection: Operation not permitted in events.js:160

Uncaught Error: Unable to build data connection: Operation not permitted
events.js:160
Hide Stack Trace
Error: Unable to build data connection: Operation not permitted
    at makeError (/Users/sboettc1/.atom/packages/Remote-FTP/node_modules/ftp/lib/connection.js:1067:13)
    at Parser.<anonymous> (/Users/sboettc1/.atom/packages/Remote-FTP/node_modules/ftp/lib/connection.js:113:25)
    at emitTwo (events.js:106:13)
    at Parser.emit (events.js:191:7)
    at Parser._write (/Users/sboettc1/.atom/packages/Remote-FTP/node_modules/ftp/lib/parser.js:59:10)
    at doWrite (_stream_writable.js:307:12)
    at writeOrBuffer (_stream_writable.js:293:5)
    at Parser.Writable.write (_stream_writable.js:220:11)
    at TLSSocket.ondata (/Users/sboettc1/.atom/packages/Remote-FTP/node_modules/ftp/lib/connection.js:273:20)
    at emitOne (events.js:96:13)
    at TLSSocket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:176:18)
    at TLSSocket.Readable.push (_stream_readable.js:134:10)
    at TLSWrap.onread (net.js:543:20)

Hi, you try IPv6?
Maybe related #949


Filesystem (local share) Not fully syncing, missing files

What version of Music Assistant has the issue?
2023.6.0b0
The problem
I have a rather small music library, 402 files, mostly MP3s and a couple FLAC. I had MusicBee, my desktop PC's media library, place these tracks in the local media folder for Home Assistant. The file structure is Media\<Artist>\<Title>. I then added a file system media provider to Music Assistant and told it to use the HA Media folder.
It only detected 167 of the 402 files. If I browse and attempt to view a folder where a missing track is located, MA will not open the folder. No error message is displayed, the folder just doesn't open. The animation for having clicked on it works.
I can use the default Media service in Home Assistant to find and play these tracks fine, so it isn't an issue with the files themselves.
The only relevant log output appears to be that MA can't find the album artist tag on these tracks and that's screwing it up, but that seems....silly to me if that's what's going on? I haven't gone through all the logs to make sure that only the missing tracks have these log entries, but it appears so.
How to reproduce
I suppose I could share my music library, but otherwise I don't know how to reproduce this issue.
Relevant log output
2023-06-29 12:01:56 ERROR music_assistant.websocket_api  -- [139920709261776] Error handling message: CommandMessage(message_id=59, command='music/browse', args={'path': 'filesystem_local://Bon Jovi'})
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/music_assistant/server/providers/websocket_api/init.py", line 237, in _run_handler
result = await result
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/music_assistant/server/controllers/music.py", line 242, in browse
return await prov.browse(path)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/music_assistant/server/providers/filesystem_local/base.py", line 246, in browse
elif track := await self.get_track(item.path):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/music_assistant/server/providers/filesystem_local/base.py", line 393, in get_track
return await self._parse_track(file_item)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/music_assistant/server/providers/filesystem_local/base.py", line 654, in _parse_track
raise InvalidDataError("missing ID3 tag [albumartist]")
music_assistant.common.models.errors.InvalidDataError: missing ID3 tag [albumartist]
Additional information
No response
What version of Home Assistant Core are your running
2023.06.4
What type of installation are you running?
Home Assistant OS
On what type of hardware are you running?
Generic x86-64 (e.g. Intel NUC)

I have verified that all of my tracks are tagged with an AlbumArtist tag in Picard.

Just confirming have you actually retagged them all with Picard or just inspected them? Please retag them if you haven’t.
Try removing the Bon Jovi folder and see if everything works ok then. If so there is a problem with something in that folder. Slowly add the tracks back until you find the offending one.

You can also change the setting in the file system provider advanced settings to not skip the track but it is better if you can tag it properly

You can also change the setting in the file system provider advanced settings to not skip the track but it is better if you can tag it properly

When I set it to use track artists, the number of tracks that MA is willing to play went from 167 down to 12.

Just confirming have you actually retagged them all with Picard or just inspected them? Please retag them if you haven’t.
Try removing the Bon Jovi folder and see if everything works ok then. If so there is a problem with something in that folder. Slowly add the tracks back until you find the offending one.

I verified the tags.  I should have mentioned that the Bon Jovi folder was just an example, I have hundreds of errors of this type being thrown.

You can also change the setting in the file system provider advanced settings to not skip the track but it is better if you can tag it properly

When I set it to use track artists, the number of tracks that MA is willing to play went from 167 down to 12.

Just confirming have you actually retagged them all with Picard or just inspected them? Please retag them if you haven’t.
Try removing the Bon Jovi folder and see if everything works ok then. If so there is a problem with something in that folder. Slowly add the tracks back until you find the offending one.

I verified the tags.  I should have mentioned that the Bon Jovi folder was just an example, I have hundreds of errors of this type being thrown.

There is something wrong with your tags. Retag all of your tracks with Picard. We hav also seen issues with artwork embedded with iTunes which used a non-standard approach. If you retag with Picard all should be well.

Ok, so I deleted everything from my media share and set it as the output directory for Picard, verifying the tags were saved and that Picard was the only thing updating and saving the media. It's now in a different file system format, but it works fine.
However, Music Assistant still only shows 149 tracks. It shows 158 artists, and 184 albums.
I no longer see the errors for missing AlbumArtist but I still am missing many many tracks. An example of a track that is physically on the disk but is not showing on the list of tracks:

Try browsing to that particular track and see what happens. Check the log after doing so.

Also what if you navigate to the album Narrow Stairs. Can you see the track there?

I logged in to HA/MA today to this test, and it is now showing all 415 tracks.
Is it possible it just needed several days to build the database?

Not necessarily days but it does take a little time to scan everything although 415 tracks is not much. Many people have thousands. But sounds like everything is working now so I will close this.


BUG FIX: Breakpoint select

In the breakpoint selection method was a bug. It was not selecting the right size.

@nolimits4web
Could you approve my changes?

How to replicate this issue?

Closing as incompatible with current version of Swiper


ARROW-15454: [Python] Try to make CSV cancellation test more robust

Use signal.set_wakeup_fd to wait reliably for the signal to get to the Python interpreter.

@github-actions crossbow -g python

@github-actions crossbow submit -g python

@kszucs Perhaps this can allow us to remove the workarounds added in #12243

Since it only occurred in the M1 wheel verification tests I think it's fine to keep the workaround for 7.0 and ship this with 8.0.

@github-actions crossbow submit wheel-macos-big-sur-cp38-arm64

@github-actions crossbow submit wheel-macos-big-sur-cp38-arm64

Looks like the changes here weren't enough to fix the test on macOS M1.

I've removed the Crossbow tasks changes and will merge if green.

Note the Python MacOS build was cancelled because the dependency install phase took too long.


Terminal quick fixes: exitStatus flag is confusing

Testing #166762
The behavior I see is that with "exitStatus": true, the provider only runs when the exit code was 0. With "exitStatus": false, it only runs when non-0. Is that right? I think there should be an option for my provider to run regardless of exit code.

Or if you want the user to specify the exit code exactly, because the quick fix is targeted at a specific command and knows its behavior, that's ok. But the name and description of that property needs to be more clear.

With "exitStatus": false, it only runs when non-0. Is that right?

That's right

I think there should be an option for my provider to run regardless of exit code.

It was done this way to prevent accidents, a typical quick fix will almost always want to run on either success (0) of failure (non-0). Originally we had the actual exitCode, but that's not portable across OS'. 👍 to improving docs

I thought exitStatus did the opposite and was stuck until I found this issue 😛
Maybe the flag should be matchWhen: /* 0 */ "succeeded" | /* non-zero */ "error" | /* any */ "exited"

@connor4312 yeah that would probably be better, I would still exclude the "exited" one though as we want to help extension authors not be bad actors by running these regex's on successful commands unless they absolutely need to.

I was confused by exitStatus too, but like this proposal:
matchWhen: /* 0 */ "succeeded" | /* non-zero */ "error"

nit on my own suggestion: maybe matchOn: success | error -- "succeeded" is a state and "error" is not, "errored" is but it doesn't sound nice 😛

I still prefer "exitStatus" though as matchOn/When doesn't strongly indicate it's about the how the command exited.

matchOnCommandResult: success | error?


feat: add average time

This PR attempts to track average time per website #19
Implementation
This is done by storing the performance on every page with an array of performance which will be used by the popin.
The popin displays the graph of time performance of the page in the orange color, below it in the green color the average time of this page.
Going further

possibility to reset that data store (chrome may through an exception because too many things are stored)
possibility to track only a / some website by adding an option page by example
possibility to group the tracking (by domain for instance)

sorry, I decided not to implement this https://github.com/alex-vv/page-load-time/issues/19#issuecomment-726800695


Refactor config file parsing, add support for more robust config file format(s)

The current config file format is overly simplistic. Support for lists and booleans was added as an after-thought and works only for very simple use-cases, but doesn't generalize.
This issue coalesces Issue #10 and other format-related Issues (#16, #19).
I think the right way to move forward on this will be:

step 1 - refactor / extract out config file-format-aware code into a separate class that can be overriden more easily.
step 2 - implement the class to provide a more robust alternative to the current format.

Any suggestions and/or pull requests will be appreciated.

@bw2 FWIW, I have forked the repo and am actively working on adding support for HOCON.  I agree with you on the general plan--I will be abstracting out the file-format-aware code as part of my PR (although my priority is HOCON support because that's my main use case that isn't yet addressed).

I'm closing this and re-opening a new issue for creating a better config file format.


Re-enable previously failing benchmarks

Updates JAX, PyTorch and TFLite model artifacts and re-enables previously failing workloads.
Removes Tensorflow benchmarks since it is becoming a large maintenance burden and artifact generation failed.

A100 benchmarks are passing again. PTAL.


Improve description of arguments and errors when using them incorrectly

Bug description
I think the options described when doing espup install -h could be improved a bit:
-e:
This description ends abruptly after ESP-IDF version to install. If empty, no esp-idf is installed. Version format: without actually specifying the possible formats. They are listed here, but they don't show up when running said command in a terminal. Also, the part If empty, no esp-idf is installed. made me think that I need to choose a version, in order to be able to use std. Perhaps the documentation could specify explicitly that this option can be left empty, and that esp-idf-sys will automatically take care of things (without having to install it in some other step). After reading this documentations a few times, l understand this now, but perhaps it could be made a bit more clear.
-m:
A few more details describing what the minimal installation entails would be nice.
-f:
Perhaps this is just me not interpreting things correctly, but the description Destination of the generated export file made me think that I could specify the directory, where the file should be placed. Turns out, that you need to specify the full path of the file, however.
This brings me to my next point, though. When specifying an invalid argument for -f, I would like a better error message. Currently, when specifying a directory instead of a file path, the program just fails with an Access denied. (os error 5). It would be nice if we could indicate that the specified path does not make for a valid file.
Further, it may be good to check the validity of the arguments earlier. Currently, the above mistake results in a partial installation and requires manual deletion of the installed toolchain before the installation can be restarted with a correct argument. This could be circumvented by validating the arguments before starting the installation.
Would you like to work on a fix? [y/n]
I don't have a lot of time to do spend on this right now, but I am willing to take a look, if nobody else wants to do it :)
To Reproduce
Steps to reproduce the behavior:

run espup install -h.
run espup install -f <DIR> where <DIR> is the path to a directory.

(Tested on the main branch)

Environment

OS: Windows 10 21H2
espup version: 0.2.5-dev

Thanks for opening an issue with so many details! I will be working on improving all the mentioned issues

You are very welcome! Thanks for the quick response, and for maintaining this project. It makes for a very pleasant experience when setting up the environment for ESP-development :)

A small comment regarding:

This description ends abruptly after ESP-IDF version to install. If empty, no esp-idf is installed. Version format: without actually specifying the possible formats. They are listed here, but they don't show up when running said command in a terminal.

-h is not the same as --help, -h only shows a summary, that's why it was only showing the first line. This is still an issue as it ended abruptly and it made no sense.

Oh, I wasn't aware of that. Thanks for pointing it out.

Just created #122 fixing all the issues you commented. Can you review them @LeCyberDucky?

Fantastic! I read through the changes, and they look great. Thanks for the super quick update!


[libgnutls] Compile on windows

[ ] Changes comply with the maintainer guide
[ ] SHA512s are updated for each updated download
[ ] The "supports" clause reflects platforms that may be fixed by this new version
[ ] Any fixed CI baseline entries are removed from that file.
[ ] Any patches that are no longer applied are deleted from the port's directory.
[ ] The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
[ ] Only one version is added to each modified port's versions file.

@dg0yt Thank you! that was helping!


Fix to generate yamls for release

Signed-off-by: Paul S. Schweigert<EMAIL_ADDRESS>Changes
This PR makes a few fixes to the generate-yamls hack script, so that we actually generate yamls and artifacts when releasing. It also renames the configmap yaml as that is not crio-specific.

LGTM

/lgtm

/cherry-pick release-0.1


Improve download-latest.sh script: integrate apple-silicon binary

Fixes multiples issues: https://github.com/meilisearch/meilisearch/issues/3044, https://github.com/meilisearch/meilisearch/issues/3027, https://github.com/meilisearch/meilisearch/issues/2613, https://github.com/meilisearch/meilisearch/issues/3014
Improvement/Addition

https://github.com/meilisearch/meilisearch/issues/3044
https://github.com/meilisearch/meilisearch/issues/3027

Simplification

With https://github.com/meilisearch/meilisearch/issues/3014, we removed the complex part to get the latest version of Meilisearch we can now get directly with the GitHub API

Bug fixes:

Should remove the problem the users encountered in this issue (https://github.com/meilisearch/meilisearch/issues/2613) because the related part has been removed from the script

@dureuill I put you as reviewer if you are curious since this is a change you spotted we had to do after v0.30.0 is released 😉

For your information, I don't know why I cannot link the issues to the PR right now, looks like there is a bug on GitHub side, I will try later

@irevoire can you test on Linux? 😇

@irevoire can you test on Linux? 😇

Hey @irevoire @curquiza,
Since I had a bit of time I tested under archlinux and was able to get a working Meilisearch 0.30.0 by using the script.

Just discussed with @irevoire internally, he told me it's ok to merge
bors merge


Downgraded syntax

Downgraded syntax in two classes:

ZCL_AGS_SERVICE_GIT->BRANCH_LIST()
ZCL_AGS_SERVICE_REST->READ_BLOB()

looks good, thanks :smile:


Update multinode

fixed error:
fatal: [ct1]: FAILED! => {"msg": "The conditional check 'inventory_hostname in groups[item.value.group]' failed. The error was: error while evaluating conditional (inventory_hostname in groups[item.value.group]): 'dict object' has no attribute 'prometheus-haproxy-exporter'\n\nThe error appears to be in '/usr/local/share/kolla-ansible/ansible/roles/prometheus/tasks/config.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n---\n- name: Ensuring config directories exist\n  ^ here\n"}

Thank you for your contribution!
This GitHub repository is just a mirror of https://opendev.org/openstack/kolla-ansible, where development really happens. Pull requests proposed on GitHub are automatically closed.
If you are interested in pushing this code upstream, please note that OpenStack development uses Gerrit for change proposal and code review.
If you have never contributed to OpenStack before, please see:
https://docs.openstack.org/contributors/code-and-documentation/quick-start.html
Feel free to reach out to the First Contact SIG by sending an email to the openstack-discuss list with the tag '[First Contact]' in the subject line. To email the mailing list, you must first subscribe which can be done here:
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss


Feature request: A Layer object wrapping multiple Layer objects

tf.contrib.seq2seq.BasicDecoder module allows provision for a single output layer, to modify the output of the RNN before proceeding to further sample.
In my particular application, I require application of multiple layers.
This is not possible as far as I have checked.
A MultiLayer class could be implemented, similar to the tf.contrib.rnn.MultiRNNCell.
This class could wrap multiple Layer objects and apply them on call.

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

Have I written custom code - N/A
OS Platform and Distribution - N/A
TensorFlow installed from - N/A
TensorFlow version - 1.8
Bazel version - N/A
CUDA/cuDNN version - N/A
GPU model and memory - N/A
Exact command to reproduce - N/A

Nagging Assignee @bignamehyp: It has been 17 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

We will consider it, thanks.

Nagging Assignee @bignamehyp: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @bignamehyp: It has been 31 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @bignamehyp: It has been 46 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @bignamehyp: It has been 61 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @bignamehyp: It has been 76 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @bignamehyp: It has been 91 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @bignamehyp: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

@varshiths,
Sorry for the delayed response. Can you please let us know if adding StackedRNNCells as an Argument, Output_Layer in the BasicDecoder fulfills your request? Thanks!


Register v0.7.0

@JuliaRegistrator register

Registration pull request created: JuliaRegistries/General/52503
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.7.0 -m "<description of version>" 10d6a56e4aeb9d16c949e810444bf47517297c26
git push origin v0.7.0


chore: use tarpaulin-install action to install tarpaulin.

Description
Updates the ci.yaml workflow file to use the new github action created to install the cargo tarpaulin.
Related to https://github.com/kubewarden/kubewarden-controller/issues/667

I've closed this PR in favor of #701 . I need to use the main branch to test other things. Therefore, I've reopen this PR from another branch.


[Feature]: Upload A Selected File or Section In Shopify Theme

What area(s) will this request affect?
Theme
What type of change do you want to see?
New feature
Overview
In some cases of developing themes, our current directory is associated with two environments (test environment and production environment). Although they correspond to the same directory, the online files of the two environments are used differently. We only want to use some files in the test environment, and some in the production environment. Therefore, we often need to upload different or specific code files to different environments. Therefore, if it is possible to support uploading specific files to the online environment, just like theme deploy + specific file path in theme kit. This feature is very helpful for my development, and it is also an important reason why I can't give up theme kit and use Shopify theme cli!
Motivation
This feature is very helpful for my development, and it is also an important reason why I can't give up theme kit and use Shopify theme cli and I think this is also the reason why many people are still using theme kit !

I would love to try to add this functionality to a Shopify theme if possible!

I see this more as a 'process' issue than an issue with CLI.
This seems like it could simply be overcome by having two GIT branches? We also have files that are specific to production and testing but this is sorted by way of simply having different git branches.
In addition, CLI does support prod/test environment flags.
In any case I see no reason anyone should be using ThemeKit in 2024 - CLI is a much more streamlined and safer approach to theme development.
I wouldn't see any benefit in this feature.

Hey @easonduan1!
I don't know your full workflow so please feel free to correct me if I've missed something, my initial impression is that the simplest approach to achieve what you are asking for is to follow @rcasimmons' suggestion of maintaining a test and prod branch separately.
This will prevent you from needing to manually configure specific paths that you want to push and should be a less error-prone approach.
If you do, however, want to maintain a single branch, might I suggest looking into the theme push command? You can specify specific files to upload using --only, --ignore, and you can specify the theme via flags as well.
Another alternative would be to configure two different shopifyignore files and switch them out as you like before pushing your theme.
Let me know if that helps, or if you need more clarification!
Since I believe what you are asking for is already achievable, I'm going to close this issue for the moment, but feel free to open it again with clarification if you disagree!
Thanks for helping improve the CLI!


How to use Agrume Offline with SDWebImage

How to use Agrume Offline with SDWebImage

The README section Custom Download Handler outlines a way. You can set a global handler through AgrumeServiceLocator.shared.setDownloadHandler – doesn't matter what library/code you use here as long as it calls the completion with a UIImage in it you're good to go.


Update hello-world.md

Please do add this

Hello, and thank you for your pull request. This is intended to be a very minimal example, add making the port pull from the environment will just add confusion to new users who are trying to see it work.

Thank You for response


feat: Refactor CLI

The current CLI works, but does not provide an API.
Here, we will refactor the CLI (rewrite) provide some kind of api for generating services, commands and subscriptions.
Feature-request:

make file naming configurable (kebab-case as an example instead of camelCase)

see: #181


Integration with vocabsieve and support for European languages

Hello,
I'm the author of vocabsieve, which aims to simplify the process of creating cards by expediting the step of looking up words and retrieving pronunciation. It is designed to serve a purpose similar to Yomichan (though with more limited scope and features), but aimed at European languages rather than only Japanese. Currently I recommend using mpvacious or immersive (both mpv extensions), but they are rather difficult to set up for less technically skilled users. It would be great if asbplayer can be used as an alternative to these.
I see two relatively simple ways of integrating them:

Using vocabsieve to add the cards on a separate window, as with mpvacious. This can be easily achieve with the current ability to copy subtitles (thought it would be great to be able to auto-copy subtitles on change to save one keystroke for the user). There is also a need for the ability to add the screenshot and audio to the last added note (supported by mpvacious as one of the possible workflows), since vocabsieve is unable to capture the audio and image from the browser. For languages using spaces, it would also be a lot more convenient if there is a feature to directly select words from the subtitle with a mouse click or something similar. Then, in order for vocabsieve to load the correct values, the following json-string should be copied to clipboard, in UTF-8 encoding:

{
"sentence": "<sentence>",
"word": "<word>"
}

Requesting local vocabsieve API for definitions in asbplayer. This would be preferable since it would allow users to fullscreen the player which is a more immersive experience. This would also require a feature to allow word selection, either with mouse or keyboard. This would be easier than the alternative of implementing dictionary support in asbplayer itself since vocabsieve supports a number of dictionary formats (stardict, migaku, json key-value pairs), as well as online dictionaries like Wiktionary, Google dictionary, and Google translate without an API key.

Links

Blog post on combining mpvacious with vocabsieve
Description of vocabsieve local API

Hi @1over137 , your suggestions sound like great possible additions to asbplayer. I would like to point out that updating the last note with media is implemented in the Anki export dialog via the 'update last card' button. If that's insufficient we can look at improving that feature in addition to exploring the deeper integrations that you suggest.

Oh. I overlooked that. That seems to work just fine. Anyways, the easiest enhancements would be:

Auto copy subtitle on change as an option
Activating the update last note feature with a single key combination or button
With these it would have feature parity with mpvacious for this usage.
However, the process would still be a quite bit nicer with deeper integrations, especially with word selection and lookup either from the Anki dialog or from the player interface itself by clicking on words. The overall goal would be to be able to add a card within just 2-3 clicks or keypresses to minimize interruption to immersion

Oh. I overlooked that. That seems to work just fine. Anyways, the easiest enhancements would be:
Auto copy subtitle on change as an option
Activating the update last note feature with a single key combination or button

These two features are implemented (not released).
I am planning to add dictionary support to asbplayer. Likely I will integrate ssmtool API as a first option.

The API doesn't exist as of this writing and the other proposals were implemented, so closing.

From what I can see, there is still the "copy subtitle and word as json" that's not implemented. The Refold guide calls for using Vocabsieve to add the card and using asbplayer to update it, and it seems like there is some room for improvement on getting the info from asbplayer.
For languages with spaces a good majority of the words a learner wants to learn is probably a single word, so this should work reasonably well. My suggestion is to (maybe behind an option) make either the words in the subtitles or on the sidebar clickable, and clicking (or double click, you decide) on one of the words should result in the json being copied (just the entire line as sentence would be good enough). This would save the user the clicks to select and copy text on asbplayer and then click on the word. Saving 2-3 clicks per card is a pretty sizable improvement for a simple feature like this.

You're right. I had overlooked the request to use the proposed JSON.


expired token

hi,
script was working fine in the last weeks...
but now i got error message..
Traceback (most recent call last):
File "./pcomfortcloud.py", line 5, in
main.main()
File "/opt/scripts/python-panasonic-comfort-cloud-master/pcomfortcloud/main.py", line 202, in main
session.login()
File "/opt/scripts/python-panasonic-comfort-cloud-master/pcomfortcloud/session.py", line 99, in login
self._create_token()
File "/opt/scripts/python-panasonic-comfort-cloud-master/pcomfortcloud/session.py", line 132, in _create_token
raise ResponseError(response.status_code, response.text)
pcomfortcloud.session.ResponseError: Invalid response, status code: 401 - Data: {"message":"New version app has been published","code":4106}
can you check this?
THX

Hi, version 0.0.22 should fix this

BIG THxs!
its working!
br
Flo


"View Decklist" not working

The "View Decklist" Button whilst being clickable is not working as long as you have more then 0 cards selected for the deck or sidedeck.
When at exactly 0 cards the button functions as intended.

I have the same issue.

Thanks for reporting this, although I'm not seeing this behavior in my tests. What set and format (draft or sealed) are you using?

We made an multiplayer sealed game (3 people) withe the set Theros Beyond Death, 10 (or maybe 12) packs and the default Cardpool. After we alle three maid our decks, none of us could open the "View Decklist" option to export the deck.


Drivers: sensors: Add support for AMS AS621x series

Added support for the AMS AS621x series of temperature sensors as a
variant of the TI TMP108 temperature sensor.
Signed-off-by: Jared Baumann<EMAIL_ADDRESS>

@gmarull @MaureenHelm @mbolivar rather than submitting a new driver for this chip, we wanted to reuse the TMP108 code I had submitted already since 90% of the functionality of AMS AS621x is the same. (just different registers) We weren't sure what the best way to do this was, but thought this route made sense. Please let us know what you think.

Please rebase and remove the merge commit

Done

How similar are these two sensors, beyond the fact that they both have a single config register and a result register?

The sample rates aren't all the same
The output data format is not the same
AS621 has no hysteresis settings
TMP108 has no consecutive fault settings

You mention Enable driver for the TMP108 temperature sensor and it's variant the AMS AS621, but the changes seem larger than a variant to me (I would expect accuracy changes across a variant, not different functionality).

I'd go for two separate drivers given the details presented so far.

I haven't looked at the details but either alternative (1 driver that supports 2 compatibles or 2 drivers) are fine with me depending on whatever makes the most sense to the sensor folks.

@MaureenHelm and @mbolivar-nordic clean up and requested changes have been implemented, please review when you have some time. Thank you

@MaureenHelm and @mbolivar-nordic clean up and requested changes have been implemented, please review when you have some time. Thank you

@MaureenHelm @gmarull @mbolivar-nordic ping

Please rebase and check if the CI errors go away.

No, the CI failures here are due to an invalid change in this PR.

@uLipe please revisit


Getting a ModuleNotFoundError: No module named 'trainers' (although 'trainers' is present in SitePackages)

Rasa version:0.7.2
Python version: 3.6.8
Operating system (windows, osx, ...):Windows 10
Issue: Getting a ModuleNotFoundError:  for trainers but in my Site PAckages that folder is already present
Trying to train the ML model run. (following the https://github.com/bhavaniravi/rasa-site-bot)
python -m rasa_nlu.train -c AI-engine/config_spacy.json --data AI-engine/data/sitebot-data.json
from trainers.spacy_sklearn_trainer import SpacySklearnTrainer
ModuleNotFoundError: No module named 'trainers'
trainers is present in Site packages.
C:\Python3.6\Lib\site-packages\rasa_nlu\trainers
Content of configuration file (config.yml):

Content of domain file (domain.yml) (if used & relevant):

Thanks for raising this issue, @paulaWesselmann  will get back to you about it soon.

Hey @yendada,
please ask this question in the forum.
As this is a usage issue and rasa version 0.7.2 is very outdated and not supported anymore I'll close this issue here.
You might be able to fix your issue by upgrading to a newer version of rasa_nlu.
For the newest version of rasa_nlu for example run pip install rasa_nlu==0.14.6.
Information on how to migrate to newer versions can be found here


Rename to parity-scale-codec and revert version to 1.0

Fixes: https://github.com/paritytech/parity-codec/issues/80

And now it is failing because of the benches. :upside_down_face:

ahhh! xD

@DarkEld3r now! :D


Sharing power platform ai hack entry

Category

[x] New Sample

What's in this Pull Request?
Sharing Power Platform AI Hack Entry - NFP Case Management

Hello @marcos-abrematia,
Thank you for your submission! We value your time and contribution.
To move forward with the review and merging of your pull request, we kindly request that you update your PR in accordance with the Contribution Guidelines. Please ensure that the README file is updated following the provided template, and add the source code as instructed.
Additionally, make sure to update your branch to reflect the latest changes from the main branch.
Thank you!
Regards,
Katerina Chernevskaya


Added Protection to all the routes

Checklist:

[ ] Have you followed the guidelines in our Contributing document?
[ ] Have you checked to ensure there aren't other open Pull Requests for the same update/change?
[ ] Have you successfully ran tests with your changes locally?
[ ] Have you added an explanation of what your changes do and why you'd like us to include them?

Issues Solved
Fixes #{99}

And please resolve conflicts.

Working on it bro

@aashutoshrathi lines 70-72 will be uncommented after creating a form that the user selects various options for modifying parameters  on the backend

@rossoskull Please update the score for this PR. This is an easy one.


EventHubProducerClient stops sending batch and gives wrong maximumSizeInBytes in Exception

Describe the bug
EventHubProducerClient stops sending batch and gives different maximumSizeInBytes in Exception then what is supposed to be.
Exception or Stack Trace
20:33:45 [main] WARN  c.a.c.a.i.RetryUtil - Error is not a TimeoutException nor is it a retryable AMQP exception.
Size of the payload exceeded maximum message size: 256 kb, errorContext[NAMESPACE: mmp-streams.servicebus.windows.net, PATH: billing, REFERENCE_ID: 6d0df19120c4489a8f405b86a7b5ded1_G31, LINK_CREDIT: 300]
20:33:45 [main] ERROR a.e.ProtoProducer - Exception:
com.azure.core.amqp.exception.AmqpException: Size of the payload exceeded maximum message size: 256 kb, errorContext[NAMESPACE: mmp-streams.servicebus.windows.net, PATH: billing, REFERENCE_ID: 6d0df19120c4489a8f405b86a7b5ded1_G31, LINK_CREDIT: 300]
 at com.azure.core.amqp.implementation.ReactorSender.send(ReactorSender.java:217)
 at com.azure.messaging.eventhubs.EventHubProducerAsyncClient.lambda$send$6(EventHubProducerAsyncClient.java:437)
 at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:118)
 at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1705)
 at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:241)
 at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:121)
 at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2267)
 at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.request(FluxMapFuseable.java:162)
 at reactor.core.publisher.MonoFlatMap$FlatMapInner.onSubscribe(MonoFlatMap.java:230)
 at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onSubscribe(FluxMapFuseable.java:90)
 at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:54)
 at reactor.core.publisher.Mono.subscribe(Mono.java:4110)
 at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:199)
 at reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53)
 at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:48)
 at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:150)
 at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1705)
 at com.azure.core.amqp.implementation.AmqpChannelProcessor.subscribe(AmqpChannelProcessor.java:220)
 at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:55)
 at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52)
 at reactor.core.publisher.FluxRetryWhen.subscribe(FluxRetryWhen.java:87)
 at reactor.core.publisher.MonoRetryWhen.subscribeOrReturn(MonoRetryWhen.java:50)
 at reactor.core.publisher.Mono.subscribe(Mono.java:4095)
 at reactor.core.publisher.Mono.block(Mono.java:1663)
 at com.azure.messaging.eventhubs.EventHubProducerClient.send(EventHubProducerClient.java:218)
 at app.eh.ProtoProducer.publish(ProtoProducer.java:67)
 at app.workers.Processor.lambda$deductFunds$6(Processor.java:216)
 at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
 at java.base/java.util.stream.ReferencePipeline$Head.forEach(Unknown Source)
 at app.workers.Processor.deductFunds(Processor.java:208)
 at app.workers.Processor.process(Processor.java:263)
 at app.Main.main(Main.java:43)
 Suppressed: java.lang.Exception: #block terminated with an error
  at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99)
  at reactor.core.publisher.Mono.block(Mono.java:1664)
  ... 8 common frames omitted
Caused by: java.nio.BufferOverflowException: null
 at org.apache.qpid.proton.codec.WritableBuffer$ByteBufferWrapper.ensureRemaining(WritableBuffer.java:152)
 at org.apache.qpid.proton.codec.BinaryType.fastWrite(BinaryType.java:83)
 at org.apache.qpid.proton.codec.EncoderImpl.writeBinary(EncoderImpl.java:535)
 at org.apache.qpid.proton.codec.messaging.FastPathDataType.write(FastPathDataType.java:126)
 at org.apache.qpid.proton.codec.messaging.FastPathDataType.write(FastPathDataType.java:36)
 at org.apache.qpid.proton.codec.EncoderImpl.writeObject(EncoderImpl.java:734)
 at org.apache.qpid.proton.message.impl.MessageImpl.encode(MessageImpl.java:740)
 at org.apache.qpid.proton.message.impl.MessageImpl.encode(MessageImpl.java:696)
 at com.azure.core.amqp.implementation.ReactorSender.send(ReactorSender.java:210)
 ... 31 common frames omitted
Caused by: java.lang.IndexOutOfBoundsException: Requested min remaining bytes(437) exceeds remaining(314) in underlying ByteBuffer: java.nio.HeapByteBuffer[pos=261829 lim=262143 cap=262144]
 at org.apache.qpid.proton.codec.WritableBuffer$ByteBufferWrapper.ensureRemaining(WritableBuffer.java:148)
 ... 39 common frames omitted

To Reproduce
Steps to reproduce the behavior:
Start sending messages with no maximumSizeInBytes value supplied (Library automatically fills it as 1022 KB) and simulate low traffic where it takes 30-35 mins to fill up a batch. Code being used is the same as shown here: https://github.com/Azure/azure-sdk-for-java/tree/master/sdk/eventhubs/azure-messaging-eventhubs#create-an-event-hub-producer-and-publish-events and library version is 5.0.3
Code Snippet
try {
            if (!eventDataBatch.tryAdd(new EventData(message.toByteArray()))) {
                eventHubProducerClient.send(eventDataBatch);

                eventDataBatch = eventHubProducerClient.createBatch();

                // Try to add that event that couldn't fit before.
                if (!eventDataBatch.tryAdd(new EventData(message.toByteArray()))) {
                    log.error("Event is too large for an empty batch. Max size: " + eventDataBatch.getMaxSizeInBytes());
                }
            }
        } catch (Exception e) {
            eventDataBatch = eventHubProducerClient.createBatch();
            log.error("Exception: ", e);
        }

Expected behavior
If the message is too big then instead of the Exception, the already created batch should first be sent to EventHub and then a fresh batch is supposed to be created in which the message should be inserted. Instead the message payload size large exception is being thrown.
Additional context
This is also only happening during low traffic when it takes a lot of time to fill up a batch (with default size being taken as 1022KB). This has occured twice around the same time frame where traffic is low and successive eventHubProducerClient.send(eventDataBatch) had > 30 mins time gap.
Also want to point out that when I am not giving a maximumBatchSize in bytes (while creating a EventDataBatch), its automatically taken as 1022 KB and suddenly in the stack trace, the size is given to be 256 KB instead of 1022 KB.
From what I could figure out, the maximumLinkSize is taken as maximumSizeInBytes. Is it the case that if link is broken, it'll fallback to 256 KB and since the already created batch size is greater than 256 KB, we're getting this error?
I got this from this method in EventHubProducerAsyncClient class' following method:
public Mono<EventDataBatch> createBatch(CreateBatchOptions options) {
        if (options == null) {
            return FluxUtil.monoError(this.logger, new NullPointerException("'options' cannot be null."));
        } else {
            String partitionKey = options.getPartitionKey();
            String partitionId = options.getPartitionId();
            int batchMaxSize = options.getMaximumSizeInBytes();
            if (!CoreUtils.isNullOrEmpty(partitionKey) && !CoreUtils.isNullOrEmpty(partitionId)) {
                return FluxUtil.monoError(this.logger, new IllegalArgumentException(String.format(Locale.US, "CreateBatchOptions.getPartitionKey() and CreateBatchOptions.getPartitionId() are both set. Only one or the other can be used. partitionKey: '%s'. partitionId: '%s'", partitionKey, partitionId)));
            } else {
                return !CoreUtils.isNullOrEmpty(partitionKey) && partitionKey.length() > 128 ? FluxUtil.monoError(this.logger, new IllegalArgumentException(String.format(Locale.US, "Partition key '%s' exceeds the maximum allowed length: '%s'.", partitionKey, 128))) : this.getSendLink(partitionId).flatMap((link) -> {
                    return link.getLinkSize().flatMap((size) -> {
                        int maximumLinkSize = size > 0 ? size : 262144;
                        if (batchMaxSize > maximumLinkSize) {
                            return FluxUtil.monoError(this.logger, new IllegalArgumentException(String.format(Locale.US, "BatchOptions.maximumSizeInBytes (%s bytes) is larger than the link size (%s bytes).", batchMaxSize, maximumLinkSize)));
                        } else {
                            int batchSize = batchMaxSize > 0 ? batchMaxSize : maximumLinkSize;
                            Objects.requireNonNull(link);
                            return Mono.just(new EventDataBatch(batchSize, partitionId, partitionKey, link::getErrorContext, this.tracerProvider, link.getEntityPath(), link.getHostname()));
                        }
                    });
                });
            }
        }
    }

Library version is:
<dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-messaging-eventhubs</artifactId>
            <version>5.0.3</version>
        </dependency>

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[x] Bug Description Added
[x] Repro Steps Added
[x] Setup information Added

I can mitigate the issue if I remake the EventHubProducerClient once the AmqpException happens.
Following is the code I am using:
public void publish(Message message) {

        toBePushed.add(billing);
        try {
            if (!eventDataBatch.tryAdd(new EventData(message.toByteArray()))) {
                eventHubProducerClient.send(eventDataBatch);
                toBePushed = new ArrayList<>();

                markBatchMetrics(eventDataBatch);
                eventDataBatch = eventHubProducerClient.createBatch();

                // Try to add that event that couldn't fit before.
                if (!eventDataBatch.tryAdd(new EventData(message.toByteArray()))) {
                    log.error("Event is too large for an empty batch. Max size: " + eventDataBatch.getMaxSizeInBytes());
                }
            }
        } catch (AmqpException e) {

            log.error("AmqpException: ", e);

            eventHubProducerClient = new EventHubClientBuilder()
                    .connectionString(connectionString)
                    .buildProducerClient();
            eventDataBatch = eventHubProducerClient.createBatch();

            for (Message message1: messages) {
                if (!eventDataBatch.tryAdd(new EventData(message1.toByteArray()))) {
                    eventHubProducerClient.send(eventDataBatch);
                    toBePushed = new ArrayList<>();

                    eventDataBatch = eventHubProducerClient.createBatch();

                    // Try to add that event that couldn't fit before.
                    if (!eventDataBatch.tryAdd(new EventData(message1.toByteArray()))) {
                        log.error("Event is too large for an empty batch. Max size: " + eventDataBatch.getMaxSizeInBytes());
                    }
                }
            }
            eventHubProducerClient.send(eventDataBatch);
            eventDataBatch = eventHubProducerClient.createBatch();
        } catch (Exception e) {
            eventDataBatch = eventHubProducerClient.createBatch();
            log.error("Exception: ", e);
        }

Is there a config that would let me control for how long the EventHubProducerClient stays connected?

Thanks for reporting this issue @shubhambhattar.  @srnagar can you please investigate?
/cc @conniey

I've created a small sample code to reproduce it. The above shared code is not complete and cannot be run as it is. This one can be run as it is just by including the dependency (version 5.0.3).
import com.azure.messaging.eventhubs.EventData;
import com.azure.messaging.eventhubs.EventDataBatch;
import com.azure.messaging.eventhubs.EventHubClientBuilder;
import com.azure.messaging.eventhubs.EventHubProducerClient;

import java.time.Instant;

public class ProducerTest {

    private static EventHubProducerClient eventHubProducerClient;
    private static EventDataBatch batch;

    private static void addToBatch() {
        try {
            for(int i = 0; i < 45000; i++)
                batch.tryAdd(new EventData("test".getBytes()));
        } catch (Exception e) {
            System.out.println(e.getMessage());
        }
    }

    private static void send(EventDataBatch batch) {

        try {
            eventHubProducerClient.send(batch);
        } catch (Exception e) {
            System.out.println(e.getMessage());
        }
    }

    public static void main(String[] args) {

        eventHubProducerClient = new EventHubClientBuilder()
                .connectionString("Endpoint=sb://mmp-click-streamsbackup.servicebus.windows.net/;SharedAccessKeyName=rw-dummy;SharedAccessKey=0+S5tott7+phI19maOga6nQsbDvld1HIcTzSiDr1ucQ=;EntityPath=dummy")
                .buildProducerClient();

        batch = eventHubProducerClient.createBatch();
        addToBatch();
        System.out.println("MaxSize = " + batch.getMaxSizeInBytes() / 1024.0d + " KB, msgSize = " + batch.getSizeInBytes() / 1024.0d + " KB");
        send(batch);
        long timePushed = Instant.now().getEpochSecond();

        batch = eventHubProducerClient.createBatch();
        addToBatch();

        while(true) {

            // wait till 31 mins
            if (Instant.now().getEpochSecond() - timePushed > 31 * 60) {
                System.out.println("MaxSize = " + batch.getMaxSizeInBytes() / 1024.0d + " KB, msgSize = " + batch.getSizeInBytes() / 1024.0d + " KB");
                send(batch);
                break;
            }
        }
    }
}

The corresponding logs are:
19:50:43 [main] INFO  c.a.m.e.EventHubClientBuilder - connectionId[MF_fc8b42_1588602043945]: Emitting a single connection.
19:50:43 [main] INFO  c.a.m.e.i.EventHubConnectionProcessor - connectionId[some-namespace.servicebus.windows.net] entityPath[dummy]: Setting next AMQP channel.
19:50:43 [main] INFO  c.a.c.a.i.ReactorConnection - connectionId[MF_fc8b42_1588602043945]: Creating and starting connection to some-namespace.servicebus.windows.net:5671
19:50:44 [main] INFO  c.a.c.a.i.ReactorExecutor - connectionId[MF_fc8b42_1588602043945], message[Starting reactor.]
19:50:44 [single-1] INFO  c.a.c.a.i.h.ConnectionHandler - onConnectionInit hostname[some-namespace.servicebus.windows.net], connectionId[MF_fc8b42_1588602043945]
19:50:44 [single-1] INFO  c.a.c.a.i.h.ReactorHandler - connectionId[MF_fc8b42_1588602043945] reactor.onReactorInit
19:50:44 [main] INFO  c.a.c.a.i.AzureTokenManagerProvider - Creating new token manager for audience[amqp://some-namespace.servicebus.windows.net/dummy], resource[dummy]
19:50:44 [single-1] INFO  c.a.c.a.i.h.ConnectionHandler - onConnectionLocalOpen hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_fc8b42_1588602043945], errorCondition[null], errorDescription[null]
19:50:44 [single-1] INFO  c.a.c.a.i.h.ConnectionHandler - onConnectionBound hostname[some-namespace.servicebus.windows.net], connectionId[MF_fc8b42_1588602043945]
19:50:47 [single-1] INFO  c.a.c.a.i.h.ConnectionHandler - onConnectionRemoteOpen hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_fc8b42_1588602043945], remoteContainer[68897ad4abc542519d69b6e61a1579ba_G18]
19:50:47 [single-1] INFO  c.a.c.a.i.h.SessionHandler - onSessionRemoteOpen connectionId[MF_fc8b42_1588602043945], entityName[dummy], sessionIncCapacity[0], sessionOutgoingWindow[2147483647]
19:50:47 [single-1] INFO  c.a.c.a.i.ReactorConnection - Setting CBS channel.
19:50:47 [single-1] INFO  c.a.c.a.i.ReactorConnection - Emitting new response channel. connectionId: MF_fc8b42_1588602043945. entityPath: $cbs. linkName: cbs.
19:50:47 [single-1] INFO  c.a.c.a.i.RequestResponseChannel<cbs-session> - connectionId[MF_fc8b42_1588602043945] entityPath[$cbs]: Setting next AMQP channel.
19:50:47 [single-1] INFO  c.a.c.a.i.h.ReceiveLinkHandler - onLinkLocalOpen connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], localSource[Source{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}]
19:50:47 [single-1] INFO  c.a.c.a.i.h.SessionHandler - onSessionRemoteOpen connectionId[MF_fc8b42_1588602043945], entityName[cbs-session], sessionIncCapacity[0], sessionOutgoingWindow[2147483647]
19:50:47 [single-1] INFO  c.a.c.a.i.h.SendLinkHandler - onLinkRemoteOpen connectionId[MF_fc8b42_1588602043945], linkName[cbs:sender], remoteTarget[Target{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null}]
19:50:47 [single-1] INFO  c.a.c.a.i.h.ReceiveLinkHandler - onLinkRemoteOpen connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], remoteSource[Source{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}]
19:50:48 [single-1] INFO  c.a.c.a.i.ActiveClientTokenManager - Scheduling refresh token task. scopes[amqp://some-namespace.servicebus.windows.net/dummy]
19:50:49 [single-1] INFO  c.a.c.a.i.h.SendLinkHandler - onLinkRemoteOpen connectionId[MF_fc8b42_1588602043945], linkName[dummy], remoteTarget[Target{address='dummy', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null}]
MaxSize = 1022.0 KB, msgSize = 1021.982421875 KB
MaxSize = 1022.0 KB, msgSize = 1021.982421875 KB
20:06:58 [parallel-3] INFO  c.a.c.a.i.ActiveClientTokenManager - Refreshing token. scopes[amqp://some-namespace.servicebus.windows.net/dummy]
20:06:59 [single-1] INFO  c.a.c.a.i.ActiveClientTokenManager - Authorization successful. Refreshing token in 971000 ms. scopes[amqp://some-namespace.servicebus.windows.net/dummy]
20:21:05 [single-1] INFO  c.a.c.a.i.h.SendLinkHandler - onLinkRemoteClose connectionId[MF_fc8b42_1588602043945], linkName[dummy], errorCondition[amqp:link:detach-forced], errorDescription[Idle link tracker, link dummy has been idle for 1800000ms TrackingId:4baeefe9-6ae0-4635-8d4c-d963c589cb41_G18, SystemTracker:client-link13935138, Timestamp:2020-05-04T14:51:05]
20:21:05 [single-1] INFO  c.a.c.a.i.h.SendLinkHandler - processOnClose connectionId[MF_fc8b42_1588602043945], linkName[dummy], errorCondition[amqp:link:detach-forced], errorDescription[Idle link tracker, link dummy has been idle for 1800000ms TrackingId:4baeefe9-6ae0-4635-8d4c-d963c589cb41_G18, SystemTracker:client-link13935138, Timestamp:2020-05-04T14:51:05]
20:21:05 [single-1] ERROR c.a.c.a.i.ReactorSender - [dummy] Error occurred in sender error handler.
Idle link tracker, link dummy has been idle for 1800000ms TrackingId:4baeefe9-6ae0-4635-8d4c-d963c589cb41_G18, SystemTracker:client-link13935138, Timestamp:2020-05-04T14:51:05, errorContext[NAMESPACE: some-namespace.servicebus.windows.net, PATH: dummy, REFERENCE_ID: 68897ad4abc542519d69b6e61a1579ba_G18, LINK_CREDIT: 299]
20:21:05 [single-1] INFO  c.a.c.a.i.ReactorSession - linkName[dummy]: Error occurred. Removing and disposing send link.
20:21:05 [single-1] INFO  c.a.c.a.i.h.SendLinkHandler - onLinkLocalClose connectionId[MF_fc8b42_1588602043945], linkName[dummy], errorCondition[amqp:link:detach-forced], errorDescription[Idle link tracker, link dummy has been idle for 1800000ms TrackingId:4baeefe9-6ae0-4635-8d4c-d963c589cb41_G18, SystemTracker:client-link13935138, Timestamp:2020-05-04T14:51:05]
20:26:05 [single-1] INFO  c.a.c.a.i.h.SessionHandler - onSessionRemoteClose connectionId[dummy], entityName[MF_fc8b42_1588602043945], condition[Error{condition=null, description='null', info=null}]
20:26:05 [single-1] INFO  c.a.c.a.i.h.SessionHandler - onSessionRemoteClose closing a local session for connectionId[MF_fc8b42_1588602043945], entityName[dummy], condition[null], description[null]
20:26:05 [single-1] INFO  c.a.c.a.i.ReactorConnection - connectionId[MF_fc8b42_1588602043945] sessionName[dummy]: Complete. Removing and disposing session.
20:26:05 [single-1] INFO  c.a.c.a.i.ReactorSession - sessionId[dummy]: Disposing of session.
20:26:05 [single-1] INFO  c.a.c.a.i.h.SendLinkHandler - onLinkRemoteClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:sender], errorCondition[amqp:connection:forced], errorDescription[The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05]
20:26:05 [single-1] INFO  c.a.c.a.i.h.SendLinkHandler - processOnClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:sender], errorCondition[amqp:connection:forced], errorDescription[The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05]
20:26:05 [single-1] WARN  c.a.c.a.i.RequestResponseChannel<cbs-session> - Retry #1. Transient error occurred. Retrying after 4511 ms.
The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05, errorContext[NAMESPACE: some-namespace.servicebus.windows.net, PATH: $cbs, REFERENCE_ID: cbs:sender, LINK_CREDIT: 98]
20:26:05 [single-1] INFO  c.a.c.a.i.h.ReceiveLinkHandler - onLinkRemoteClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], errorCondition[amqp:connection:forced], errorDescription[The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05]
20:26:05 [single-1] INFO  c.a.c.a.i.h.ReceiveLinkHandler - processOnClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], errorCondition[amqp:connection:forced], errorDescription[The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05]
20:26:05 [single-1] INFO  c.a.c.a.i.h.SessionHandler - onSessionRemoteClose connectionId[cbs-session], entityName[MF_fc8b42_1588602043945], condition[Error{condition=null, description='null', info=null}]
20:26:05 [single-1] INFO  c.a.c.a.i.h.SessionHandler - onSessionRemoteClose closing a local session for connectionId[MF_fc8b42_1588602043945], entityName[cbs-session], condition[null], description[null]
20:26:05 [single-1] INFO  c.a.c.a.i.ReactorConnection - connectionId[MF_fc8b42_1588602043945] sessionName[cbs-session]: Complete. Removing and disposing session.
20:26:05 [single-1] INFO  c.a.c.a.i.ReactorSession - sessionId[cbs-session]: Disposing of session.
20:26:05 [single-1] INFO  c.a.c.a.i.h.ConnectionHandler - onConnectionRemoteClose hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_fc8b42_1588602043945], errorCondition[amqp:connection:forced], errorDescription[The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05]
20:26:05 [single-1] INFO  c.a.m.e.i.EventHubConnectionProcessor - Channel closed.
20:26:05 [single-1] INFO  c.a.m.e.i.EventHubReactorAmqpConnection - connectionId[MF_fc8b42_1588602043945]: Disposing of connection.
20:26:05 [single-1] INFO  c.a.c.a.i.ReactorConnection - connectionId[MF_fc8b42_1588602043945]: Disposing of ReactorConnection.
20:26:05 [single-1] INFO  c.a.c.a.i.AmqpExceptionHandler - Shutdown received: ReactorExecutor.close() was called., isTransient[false], initiatedByClient[true]
20:26:10 [parallel-2] INFO  c.a.c.a.i.RequestResponseChannel<cbs-session> - Retry #1. Requesting from upstream.
20:26:10 [parallel-2] INFO  c.a.c.a.i.RequestResponseChannel<cbs-session> - connectionId[MF_fc8b42_1588602043945] entityPath[$cbs]: Connection not requested, yet. Requesting one.
20:26:10 [parallel-2] ERROR c.a.c.a.i.ReactorConnection - connectionId[MF_fc8b42_1588602043945]: Connection is disposed. Cannot get CBS node.
20:26:10 [parallel-2] INFO  c.a.c.a.i.ReactorConnection - Emitting new response channel. connectionId: MF_fc8b42_1588602043945. entityPath: $cbs. linkName: cbs.
20:26:10 [parallel-2] INFO  c.a.c.a.i.RequestResponseChannel<cbs-session> - connectionId[MF_fc8b42_1588602043945] entityPath[$cbs]: Setting next AMQP channel.
20:27:05 [single-1] INFO  c.a.c.a.i.ReactorExecutor - Unable to acquire dispose reactor semaphore within timeout.
20:27:05 [single-1] INFO  c.a.c.a.i.h.SendLinkHandler - onLinkLocalClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:sender], errorCondition[amqp:connection:forced], errorDescription[The connection was inactive for more than the allowed 300000 milliseconds and is closed by container 'LinkTracker'. TrackingId:68897ad4abc542519d69b6e61a1579ba_G18, SystemTracker:gateway5, Timestamp:2020-05-04T14:56:05]
20:27:05 [single-1] INFO  c.a.c.a.i.h.ReceiveLinkHandler - onLinkLocalClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], errorCondition[null], errorDescription[null]
20:27:05 [single-1] INFO  c.a.c.a.i.h.ConnectionHandler - onConnectionLocalClose hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_fc8b42_1588602043945], errorCondition[null], errorDescription[null]
20:27:05 [single-1] INFO  c.a.c.a.i.h.ConnectionHandler - onConnectionUnbound hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_fc8b42_1588602043945], state[CLOSED], remoteState[CLOSED]
20:27:05 [single-1] INFO  c.a.c.a.i.h.SendLinkHandler - onLinkFinal connectionId[MF_fc8b42_1588602043945],  linkName[dummy]
20:27:05 [single-1] INFO  c.a.c.a.i.h.SessionHandler - onSessionFinal connectionId[MF_fc8b42_1588602043945], entityName[dummy], condition[null], description[null]
20:27:05 [single-1] INFO  c.a.c.a.i.h.SendLinkHandler - onLinkFinal connectionId[MF_fc8b42_1588602043945],  linkName[cbs:sender]
20:27:05 [single-1] INFO  c.a.c.a.i.h.ReceiveLinkHandler - onLinkFinal connectionId[MF_fc8b42_1588602043945],  linkName[cbs:receiver]
20:27:05 [single-1] INFO  c.a.c.a.i.h.SessionHandler - onSessionFinal connectionId[MF_fc8b42_1588602043945], entityName[cbs-session], condition[null], description[null]
20:27:05 [single-1] INFO  c.a.c.a.i.h.SendLinkHandler - onLinkFinal connectionId[MF_fc8b42_1588602043945],  linkName[cbs:sender]
20:27:05 [single-1] INFO  c.a.c.a.i.h.ReceiveLinkHandler - onLinkFinal connectionId[MF_fc8b42_1588602043945],  linkName[cbs:receiver]
20:27:05 [single-1] INFO  c.a.c.a.i.h.SessionHandler - onSessionFinal connectionId[MF_fc8b42_1588602043945], entityName[cbs-session], condition[null], description[null]
20:27:05 [single-1] INFO  c.a.c.a.i.ReactorConnection - connectionId[MF_fc8b42_1588602043945] sessionName[cbs-session]: Complete. Removing and disposing session.
20:27:05 [single-1] INFO  c.a.c.a.i.ReactorSession - sessionId[cbs-session]: Disposing of session.
20:27:05 [single-1] INFO  c.a.c.a.i.h.ConnectionHandler - onConnectionFinal hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_fc8b42_1588602043945], errorCondition[null], errorDescription[null]
20:27:05 [single-1] INFO  c.a.c.a.i.h.SendLinkHandler - onLinkLocalClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:sender], errorCondition[null], errorDescription[null]
20:27:05 [single-1] INFO  c.a.c.a.i.h.ReceiveLinkHandler - onLinkLocalClose connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], errorCondition[null], errorDescription[null]
20:27:05 [single-1] INFO  c.a.c.a.i.ReactorExecutor - connectionId[MF_fc8b42_1588602043945], message[Processing all pending tasks and closing old reactor.]
20:27:05 [single-1] INFO  c.a.c.a.i.h.ReceiveLinkHandler - onLinkLocalOpen connectionId[MF_fc8b42_1588602043945], linkName[cbs:receiver], localSource[Source{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}]
20:27:05 [single-1] INFO  c.a.c.a.i.ReactorExecutor - connectionId[MF_fc8b42_1588602043945], message[Stopping the reactor because thread was interrupted or the reactor has no more events to process.]
20:31:11 [main] INFO  c.a.m.e.i.EventHubConnectionProcessor - connectionId[some-namespace.servicebus.windows.net] entityPath[dummy]: Connection not requested, yet. Requesting one.
20:31:11 [main] INFO  c.a.m.e.EventHubClientBuilder - connectionId[MF_65d28d_1588604471389]: Emitting a single connection.
20:31:11 [main] INFO  c.a.m.e.i.EventHubConnectionProcessor - connectionId[some-namespace.servicebus.windows.net] entityPath[dummy]: Setting next AMQP channel.
20:31:11 [main] INFO  c.a.c.a.i.ReactorConnection - connectionId[MF_65d28d_1588604471389]: Creating and starting connection to some-namespace.servicebus.windows.net:5671
20:31:11 [main] INFO  c.a.c.a.i.ReactorExecutor - connectionId[MF_65d28d_1588604471389], message[Starting reactor.]
20:31:11 [single-1] INFO  c.a.c.a.i.h.ConnectionHandler - onConnectionInit hostname[some-namespace.servicebus.windows.net], connectionId[MF_65d28d_1588604471389]
20:31:11 [main] INFO  c.a.c.a.i.AzureTokenManagerProvider - Creating new token manager for audience[amqp://some-namespace.servicebus.windows.net/dummy], resource[dummy]
20:31:11 [single-1] INFO  c.a.c.a.i.h.ReactorHandler - connectionId[MF_65d28d_1588604471389] reactor.onReactorInit
20:31:11 [single-1] INFO  c.a.c.a.i.h.ConnectionHandler - onConnectionLocalOpen hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_65d28d_1588604471389], errorCondition[null], errorDescription[null]
20:31:11 [single-1] INFO  c.a.c.a.i.h.ConnectionHandler - onConnectionBound hostname[some-namespace.servicebus.windows.net], connectionId[MF_65d28d_1588604471389]
20:31:14 [single-1] INFO  c.a.c.a.i.h.ConnectionHandler - onConnectionRemoteOpen hostname[some-namespace.servicebus.windows.net:5671], connectionId[MF_65d28d_1588604471389], remoteContainer[fa9aa3015eb84990a62817ad60423545_G8]
20:31:14 [single-1] INFO  c.a.c.a.i.h.SessionHandler - onSessionRemoteOpen connectionId[MF_65d28d_1588604471389], entityName[dummy], sessionIncCapacity[0], sessionOutgoingWindow[2147483647]
20:31:14 [single-1] INFO  c.a.c.a.i.ReactorConnection - Setting CBS channel.
20:31:14 [single-1] INFO  c.a.c.a.i.ReactorConnection - Emitting new response channel. connectionId: MF_65d28d_1588604471389. entityPath: $cbs. linkName: cbs.
20:31:14 [single-1] INFO  c.a.c.a.i.RequestResponseChannel<cbs-session> - connectionId[MF_65d28d_1588604471389] entityPath[$cbs]: Setting next AMQP channel.
20:31:14 [single-1] INFO  c.a.c.a.i.h.ReceiveLinkHandler - onLinkLocalOpen connectionId[MF_65d28d_1588604471389], linkName[cbs:receiver], localSource[Source{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}]
20:31:14 [single-1] INFO  c.a.c.a.i.h.SessionHandler - onSessionRemoteOpen connectionId[MF_65d28d_1588604471389], entityName[cbs-session], sessionIncCapacity[0], sessionOutgoingWindow[2147483647]
20:31:14 [single-1] INFO  c.a.c.a.i.h.SendLinkHandler - onLinkRemoteOpen connectionId[MF_65d28d_1588604471389], linkName[cbs:sender], remoteTarget[Target{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null}]
20:31:14 [single-1] INFO  c.a.c.a.i.h.ReceiveLinkHandler - onLinkRemoteOpen connectionId[MF_65d28d_1588604471389], linkName[cbs:receiver], remoteSource[Source{address='$cbs', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}]
20:31:14 [single-1] INFO  c.a.c.a.i.ActiveClientTokenManager - Scheduling refresh token task. scopes[amqp://some-namespace.servicebus.windows.net/dummy]
20:31:14 [single-1] WARN  c.a.c.a.i.RetryUtil - Error is not a TimeoutException nor is it a retryable AMQP exception.
Size of the payload exceeded maximum message size: 256 kb, errorContext[NAMESPACE: some-namespace.servicebus.windows.net, PATH: dummy, REFERENCE_ID: dummy, LINK_CREDIT: 0]
Size of the payload exceeded maximum message size: 256 kb, errorContext[NAMESPACE: some-namespace.servicebus.windows.net, PATH: dummy, REFERENCE_ID: dummy, LINK_CREDIT: 0]

Process finished with exit code 0

This will probably be helpful.
I've simulated the case where it takes more than 30 mins (used 31, just to be safe) to completely fill up the batch and send it. The difference between the time when the batch is first initialized and the time the batch completely fills up has to be > 30 mins.
Do note the logs at the end which says its not a TimeoutException but rather says that the size is greater than 256 KB which shouldn't be the case given the max batch size when the batch was created is assumed to be 1022 KB.

@shubhambhattar I am looking into this issue. Thanks for providing a reproducible sample. I will have an update soon.


[dagster-pipes-test] improve test_message_report_asset_check

Summary & Motivation
Add severity assertion to ensure severity is set correctly by the implementation. And fix the Java test in PipesTest class to add severity when calling reportAssetCheck.
How I Tested These Changes
Make java implementation tests passed by running uv run pytest in java directory.
Changelog

Hey @wingyplus, could you pull from main please? I just added integration tests to CI

Hey @wingyplus, could you pull from main please? I just added java integration tests to CI

@danielgafni Done. 🙏


Add a screenshot.

This is currently a featured package, would be nice to provide prospective users with a screenshot.

Thanks, Kudos! Added a screenshot.


Improve items card

this pr contains enhancements to the items card
Changes:

better styling
responsive design

Not responsive


Add asserts to DefaultTextStyle.inherit

The context and child arguments are required.  We should assert that
they're non-null.
Fixes #3843


Scrubbing and Fix Atomacos

This PR resolves https://github.com/MLDSAI/OpenAdapt/issues/47.
Features:

Scrubs before Visualization only.
Scrubs all Screenshots with a accurate spacy model (en_core_web_trf).
Scrubs the dictionary and the data whose keys are in config.SCRUB_KEYS_HTML
Scrubs all the fields of "state" field.
Offers code reusability by making changes.
Scrubs "text" and "canonical_text" both with and without hyphenated form.

Newly Added Features:

Added a field in configure_logging called SCRUB_LOGGING that logs if it is True or not.
Scrubs during logging in visualize (by default SCRUB_LOGGIN is True).
Scrubs task_description in record.py logging. But it ensures that the task_description put in db is not changed/scrubbed (for now).
Scrubs "text" both in separated and non-separated form. And also ensures to return the scrubbed_text in the same form.
If text was scrubbed then, it ensures that all of its children key_chars, key_vk etc. all are also scrubbed! See: https://github.com/MLDSAI/OpenAdapt/pull/211#issuecomment-1572835364
Scrubs text in diff images as well. See: https://github.com/MLDSAI/OpenAdapt/pull/211#issuecomment-1572727834 It scrubbed my name as it detected as PII.
Also, Fixes 2 TODOs:
- utills.py
- utils.py

Important Note:

The current of scrubbing ignores scrubbing for "text" and "canonical_text" whose value starts with the config.TEXT_PREIX or ends with config.TEXT_SUFFIX. Eg. It ignores scrubbing of , ,  + c, etc.

Also look into modifying configure_logging to scrub data before it is logged? If it's a lot of work we can leave it as a TODO for now.

This PR also resolves: https://github.com/MLDSAI/OpenAdapt/pull/211/files#r1213492837
If this is merged we can close the the above mentioned PR.

Now, scrubbing also scrub the children key_chars if the text or canonical_text was scrubbed.

@abrichr Ready for Final Review!

@0dm has a Toggle button for Scrubbing before Visualization.
But this PR, ensures to SCRUB the screenshots and the data in the visualization. There is no toggler currently.
BUT, what we can toggle in Scrubbing is just Scrubbing during Logging (printing in console).
Is there anything I need to address here so that it fit our requirements?
My Opinion: Scrubbing before Visualization should not be in the hands of User. We should scrub always.

@0dm Any thoughts on it ?
@abrichr ^^

We can enable / disable it by default, but I think it's a good idea to give the user an option to use scrubbing. Some users may want it off if they know that there's nothing to scrub and want higher performance. We can redefine to visualize(scrubbing=True) and call with visualize(false) if scrubbing has been toggled off.

Related: https://github.com/MLDSAI/OpenAdapt/issues/218 (Just keep it in the loop) :)

@abrichr  Ready for Merging!

@abrichr Ready for Merging! :)

Here are some screenshots of the html (after Visualization):

Actual:

Actual: