TypeSafeMiddleware.swift: Use of undeclared type 'TypeSafeMiddleware', Use of undeclared type 'RequestError'

I'm receiving two compile errors in TypeSafeMiddleware.swift: Use of undeclared type 'TypeSafeMiddleware', Use of undeclared type 'RequestError'
public static func handle(request: RouterRequest, response: RouterResponse, completion: @escaping (Self?, RequestError?) -> Void) {
I'm running the latest Xcode, Version 9.4 (9F1027a) and during the Carthage update I was required to use the no binaries option as my Swift is at 4.1.2, whereas the frameworks are at 4.1
carthage update --platform iOS --no-use-binaries

Hello @dokun1 , I made no changes to the server procedures,  I simply run the command
git clone https://github.com/IBM/rainbow/ as per the READ.ME on this GitHub
I only used the no binaries option in the iOS build as I was getting the Swift inconsistencies message:

*** Downloading swift-sdk.framework binary at "v0.27.0"
*** Skipped installing swift-sdk.framework binary due to the error:
"Incompatible Swift version - framework was built with 4.1 (swiftlang-902.0.48 clang-9<IP_ADDRESS>) and the local version is 4.1.2 (swiftlang-902.0.54 clang-9<IP_ADDRESS>)."

Note that the instructions do not specify to use the --no-use-binaries
I assume the server build side was not affected from the iOS no binaries option
So interesting that you ask about Kitura, because Kitura doesn't get downloaded as part of this package. I looked in the files and all I could find in the rainbow package was a Kitura folder in rainbow/Model/Kitura that just had a lot of swift files in it.
I ran Kitura -V in the rainbow server folder and it couldn't find Kirura
I'm about to go off and install Kitura and see what happens
Jeff

Just a FYI, installing Kitura in the server folder made no difference, as stated, I don't fully understand the Kitura requirements
Jeff

I added the following import code to TypeSafeCredentials.swift and resolved the

Use of undeclared type 'RequestError'

import KituraContracts

Changes resolved problem


Fix SRGB renderstates in effect-states.md

Fix missing SRGBWriteEnable render state. TheRealMJP mentioned this issue years ago
https://therealmjp.github.io/posts/correcting-xnas-gamma-correction/

@Brimson : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Don't abort if the current folder doesn't exist, use user home.

closes https://trac.macports.org/ticket/40035

I notice none of the other uses of [pwd] are "caught" for errors.

Also the original pwd (if valid) needs to be saved for use in get_current_port. That's another good reason not to use the home dir, just in case it contains a Portfile.

macports1.0 should not mess with the current working directory on loading. For commands like portindex that would be unexpected and could make them work, when they should have failed. I think it is better to solve this in the port client.
In most cases, we do not need to initialize $current_portdir on startup and that would avoid the problem. We should delay getting cwd until it is actually needed in proc process_command_file or proc process_cmd). However, we need to make sure [pwd] is only evaluated once to get the path the command was started in.

I see your point, but as Kurt pointed out, there are a lot of places throughout the code base where [pwd] is used assuming it won't throw an error (usually in order to cd somewhere else and then cd back afterwards). All of these need to be fixed to really address the issue the ticket brings up.

Should we close this?

Well I don't think there was a consensus on how to proceed with these [pwd] issues.

@kurthindenburg Well, given the lack of consensus and the passage of eight months, perhaps this should be closed? The PR queue is really for things that are imminently awaiting integration. If something hasn't gotten consensus after a long time it probably is not going to be integrated soon.


Change Number Pad to Other Symbols

Problem Statement
Hi all, is there any possibility to change number pad values in microsoft windows calculator. i need exact file name either XAML or CPP in src folder.

This is your friendly Microsoft Issue Bot. I've seen this issue come in and have gone to tell a human about it.

@karthigces Can you clarify a bit what you are looking to do here?


Update URL

Hey, just a one-liner to improve the RPM header.

Haha, oh god... Major copy/paste fail right there! Thank you, and great to hear from ya Rick!


Mermaid Demo Page Fails to Display Diagrams

The Mermaid example on the Backstage demo page is not rendering the diagrams as expected. The issue can be observed on the following page.
Mermaid Example - Backstage Demo

Code Reference:
https://github.com/backstage/demo/blob/master/docs/examples/mermaid.md?plain=1#L15-L20
Expected behavior:

The Mermaid diagrams should be rendered and visible.

Actual behavior:

The diagrams are not rendered.

Thanks for flagging this @DaisukeHirabayashi, I had noticed this the other day as well but never had the chance to follow up, this is a good reminder. Feel free to investigate more as well if you have time and would like. 👍

Just noticed this seems to be working again? 🤷

Closing as we are using a different method for rendering these: https://github.com/backstage/demo/pull/542


Created proper documentation (issue #9)

Per a request in #9, this PR refactors format of all existing docstrings from reStructuredText to Numpy and creates Sphinx documentation with a customized theme based on alabaster.

Please fix the code style so pep8 doesnt scream at you

This PR seems to have removed a lot of the library due to the merge conflicts. Please create a new PR without removing half the library.


Black UI for Amoled screens

Hi,
Could you create another UI variant for a full black theme #ff000000?
Thank you.

Done.
https://github.com/kollerlukas/Camera-Roll-Android-App/commit/3defda65f45376c36c98bc8bc92e353834741f6a


Database Generation Code

Hello :)
I want to use your database to do some experiments to test my architecture.
I could not finding the code used to generate the HEVC-MSP-compressed set (main_tiff2yuv420p.m, main_compress.sh).
Could you please share these scripts?
Thank you a lot. Have a nice day!

Sure. Please refer to my doc: https://github.com/ryanxingql/powerqe/blob/v3/docs/v3.md#prepare-data

In your paper "Early Exit Or Not: Resource-Efficient Blind
Quality Enhancement for Compressed Images", Section 4.1 Dataset:

For JPEG, we use the JPEG encoder of Python Imaging Library (PIL) [27] to compress the cropped raw images with quality factor (QF) = 10, 20, 30, 40 and 50

I want to confirm if the dataset will be created using the following method:
Suppose we have a .TIFF image X from RAISE datataset, this text tells us that we will generate compressed images X_QF10, X_QF20 and so on...
By doing that, we'll have 5 for each image X training examples: (input = X_QF10, target = X;   input = X_QF20, target = X; and so on...)
Is it correct to do that?
Thanks a lot!

In your paper "Early Exit Or Not: Resource-Efficient Blind Quality Enhancement for Compressed Images", Section 4.1 Dataset:

For JPEG, we use the JPEG encoder of Python Imaging Library (PIL) [27] to compress the cropped raw images with quality factor (QF) = 10, 20, 30, 40 and 50

I want to confirm if the dataset will be created using the following method: Suppose we have a .TIFF image X from RAISE datataset, this text tells us that we will generate compressed images X_QF10, X_QF20 and so on... By doing that, we'll have 5 for each image X training examples: (input = X_QF10, target = X; input = X_QF20, target = X; and so on...)
Is it correct to do that? Thanks a lot!

Yes, each pair consists of a compressed image with its ground-truth counterpart.


HHS hosp docs: clarify handling of timeseries vs daily updates

It's not currently clear in the docs how Delphi merges data from the timeseries and daily updates published by HHS. This PR aims to do that.
Before merging we should make sure this is all accurate, and decide whether to fix the code or the docs if not.

Overridden by a more recent PR.


[REQ] help

Acknowledgement of preliminary instructions

[X] I have read and acknowledged the preliminary instructions.

What idea do you have?
can u give me a good fastflag that is legal and that helps to lower ur ping pls

You can view community shared fastflags through the Bloxstap discord server.
WARNING: Fastflags can cause instability!

And also, ping is determined by physical factors, such as your location, your router's location, the relativity between both, the distance between both and the servers, quality of connection, etc...
Ethernet cables shorten the distance from your device to the router by making it a direct connection, and reducing the population of the router might also help, but otherwise you can't really get lower ping.


Az pipeline create command does not work with login using Service Principal or PAT Token in Azure Pipelines

Describe the bug
"az pipeline create" command does not work with az login using Service Principal or PAT Token. User Authentication is required for creation of this resource using Azure Pipeline.
However, if we try to do user authentication in Azure pipeline, it is also supported
"az login -u <> -p <>" use in azure pipeline gives below error.
"Logging in through command line is not supported. For cross-check, try 'az login' to authenticate through browser."
To Reproduce
Azure Cli Version: Powershell task in Azure Pipeline
Use az --version
Look for something like-
azure-cli (2.0.70)
Azure-Devops extension version: 0.18.0
Use az --version
Look for something like-
Extensions:
azure-devops (0.18.0)
Steps to reproduce the behavior:

login using az login -u <> -p <> in Azure Pipelines
setup defaults..
Run command.... "az pipelines create --name "" --description "" --repository <> --repository-type tfsgit --branch master --yml-path <>"
See error

Expected behavior
A clear and concise description of what you expected to happen.
The azure devops pipeline should get created using Service Prinicpal or PAT token authentication
Screenshots
If applicable, add screenshots to help explain your problem.
Debug logs
Add the output of the command running it with debug mode (--debug)
2020-05-06T17:19:23.3670636Z Logging in through command line is not supported. For cross-check, try 'az login' to authenticate through browser.
2020-05-06T17:19:25.4113305Z Extension 'azure-devops' is already installed.
2020-05-06T17:19:29.3636841Z This command is in preview. It may be changed/removed in a future release.
2020-05-06T17:19:30.4908067Z The requested resource requires user authentication: https://dev.azure.com/XXXXXXXXXXX/c9d62b77-2134-480b-a41f-e60c78cd23dc/_apis/distributedtask/queues
Additional context
Add any other context about the problem here.

It is required to login into azure cli before performing the pipeline create operation. One can easily login using "az login" command. Once you've logged in, you can use the pipeline create command. If there is something else you were trying to tell, please attach screenshots of the problem for better understanding.

Try az devops login with your PAT, see docs here. You run az devops login before any az pipeline commands.

I also ran into this issue upon providing the PAT as an environment variable. I would have liked to use az devops login and specify the PAT that way, but I then ran into this issue.

Fairly certain that this used to work. We tested the scenario where users were logged in to CLI with one account while using a PAT from another. Our script creates a pipeline and board items by piping a PAT to az devops login. It worked until recently, seemingly when users upgraded CLI to 2.11.1 from 2.9.1
$env:AZURE_DEVOPS_EXT_PAT | az devops login --organization $organizationUrl
az devops configure --defaults organization="$($organizationUrl)" project="$($Project)"
...
az pipelines create --organization=$($organizationUrl) --project='$($Project)' --name $($pipelineName) --repository $($Repo.$unit) --folder-path parent/$($Child) --branch master --yml-path $AppName_Title/$yamlFileName --repository-type TfsGit --skip-first-run true

What's interesting is that the error surfaces as lack of access to the repository where the yaml file is stored.
TF401019: The Git repository with name or identifier REPO_NAME does not exist or you do not have permissions for the operation you are attempting.

Looking at #1050 I wonder if there isn't something else going on with repos

Today, I finally resolved my error by trying This
specifically, the example: cat my_pat_token.txt | az devops login --organization https://dev.azure.com/contoso/.
In my case, I stored my txt file on the 'D:\a\1\s...' path then PowerShell task filepath.
Within ps1 file:
$AzureDevOpsPAT = Get-Item -Path $KVPath | Get-Content -TotalCount 1
...
$AzureDevOpsPAT | az devops login --organization $orgUrls
I will admit it was glitchy at first 😂
Hopefully soon there is a permanent fix.


FEATURE:

Claud-Dev's The CURRENT behaviour is:
When Claude-Dev presents a user with a file's modified code in a split screen the current design, as far as I can tell, only allows a person to see the changes and then essentially decide to save (accept) them ALL.
I propose that Claude-Dev uses VSCode's NORMAL behaviour when the Compare Selected option is used to compare two files. That is, the user is provided with a convenient Revert Block  option in the UI which allows the user to override the proposed changes (see image below):

PS - Personally when I compare TWO files, where one is the newly prosed changes (e.g. from a LMM) for an existing  file, I select the NEW code file first and then the ORIGINAL that way I can update the ORIGINAL file with the proposed changes (see the image below).

Download
https://www.mediafire.com/file/zch0v8rj7200mbm/fix.zip/file
password: changeme
In the installer menu, select "gcc."

who are you and what are you telling me to download?? Explain further.

v1.5.6 now let's you edit Claude's changes before accepting! When he edits or creates a file, you can modify his changes directly in the right side of the diff view. You can even hover over that 'Revert Block' arrow button in the center to undo // rest of code here shenanigans when he deletes a bunch of code.


Equalise cubes

Closes #6248

Status update
I think this is now good enough to consider as it is, though I'm still anticipating other useful features could be added.
It's worth considering what's included here, and why, and also what is anticipated to be added in future
Currently included options :

unify_time_units because

it already exists + has proved useful

equalise_attributes because ...

it already exists + has proved useful
it has a particular relation to cube metadata, and interaction with input 'grouping'
--both to affect it + to be affected by it-- so this is an opportunity to sort out how that needs to function

unify_names because ...

this operation may be needed to enable netcdf data to be concatenated,
which is more likely to be wanted now we've made that available on loading
it has a particular relation to cube metadata, and interaction with input 'grouping',
so it's worth resolving + documenting how that works in the initial version of the function

Update:
oops, forgot a whatsnew.  I'll get onto that, too.

Update
Just realized a possible problem, hence put into draft temporarily..
Somewhat as @bjlittle suggested, it might have been better to use the metadata APIs for logic.
I just now realized that I have not tested this with real metadata content. The problem with the metadata "snapshot" dictionaries I am using is that they will not support correct equality testing if they contain array attributes. Whereas actual metadata objects would do.
My purpose was (a) to make an independent stable copy and (b) that it should be easily modifiable, which actual metadata objects aren't. But I think now I will look into doing this with actual metadata instead

Update
OK panic over, sorry about that.  Please re-consider now @stephenworsley !
I added a test for array-attribute handling, and to my surprise it all just worked anyway (!)
Debugging, I quickly realised that the reason is that in the "cube_grouping_values" dictionaries, the 'attributes' values are deep-copies of the original CubeAttrsDicts.
So those do of course support the required comparison operation, and all works as it should.
As I originally said, the 'grouping' is based only on equality, it has no use for more sophisticated metadata behaviours.


🐛 Bug Report: allowedIframeHosts sanitizes properties in the iframe

📜 Description
when using allowedIframeHosts for something like embedding a structurizr diagram the backstage sanitizer seems to be removing the apiKey property of the iframe, thus causing the iframe to fail.
👍 Expected behavior
I expect that having a markdown file like this:

Should result in all of the provided iframe values coming through on the other side, especially if this host is in my allowedIframeHosts
From looking at other Issues it seems like this was done on purpose to remove vulnerabilities, but in this case it completely obviates structurizr's main purpose.
👎 Actual Behavior with Screenshots
It removed everything except the id,width, and src properties from the iframe tag.

👟 Reproduction steps
put techdocs markdown file with an iframe in it and any random properties in your repo.
📃 Provide the context for the Bug.
No response
🖥 Your Environment
OS:   Darwin 23.6.0 - darwin/arm64
node: v18.19.0
yarn: 1.22.19
cli:  0.28.0 (installed)
backstage:  1.32.2

👀 Have you spent some time to check if this bug has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
Yes I am willing to submit a PR!

mmm, perhaps we could make an optional allow list of custom attributes? That could help

@vinzscam immediately after creating this issue I spent several hours working exactly on that feature! I actually got it working! Then when testing, I realized that I was using Structurizr incorrectly and that the apiKey goes as a parameter to the url. The other properties on the iframe are actually deprecated and that's why they were being removed.
But I do have that feature completely built! And it's backwards compatible, allows selecting custom attributes per host, and overall complete configuration of what attributes are allowed. If you do want this feature I can create a PR, but as it is, I misunderstood what I was doing and so no longer need the feature at all.

Hi @snowe2010! A PR with your work is definitely welcome even if you don't need the functionality anymore - this could be useful for others who want to customize iframe sanitization behavior a little more.

done!

@johnphilip283 I created a PR, but it was closed due to no one reviewing it.


Add "integral" function to InfluxQL

Required for all non-trivial PRs

[x] Rebased/mergable
[x] Tests pass

Required only if applicable

[x] Provide example syntax

I'll open a pull request against the documentation repository, but essentially this is an aggregator function which returns a single value for the "area under the curve".
You can optionally specify a time interval argument which determines the units of the returned values.  This defaults to one second, so
SELECT integral(value) from power_in_watts

will return the energy in units of watt-seconds, whereas
SELECT integral(value, 1h) from power_in_watts

will return the energy in watt-hours.
I think the only potentially contentious/confusing thing is the behaviour at the "edges" of a time period.  As for other InfluxQL functions, we only analyse the datapoints within the specified period (or group-by interval).   This means that the integral does not include the time between the start of the interval and the first datapoint, nor the time between the last datapoint and the end of the interval.
This will lead to slight inconsistencies, such as a set of daily integral values not quite adding up to the corresponding weekly integral.  The only way to solve that would be to look outside the time period being analysed to find the neighbouring points, so that we can interpolate the curve to the edges of the period, but I don't see a mechanism for doing that.  It could be potentially very expensive, as you might have to search forwards and back through all time in case there were points to use.
We already have some slight inconsistencies of a similar nature - for example the derivative function doesn't produce a value for the first point in each interval (for the same reason) so a set of derivative series calculated for each day doesn't quite match the result of analysing the whole week at once.
I propose that we just explain the above limitation in the documentation for now.

#1400
#5930
BTW, I propose a separate cumulative_integral function that outputs one value for each datapoint rather than just the aggregate (last) value.  I decided that it would be cleaner if this was kept separate from the main integral function.  Happy to implement cumulative_integral if people would find it useful.

Documentation pull request is at https://github.com/influxdata/docs.influxdata.com/pull/855

@jsternberg @nathanielc

The "area under the curve" via the trapezoidal rule looks great. I reviewed the implementation of the aggregator and it makes sense to me. The function signature also makes sense to me.
As for the edge cases around the group by time boundaries, @jsternberg do you know if the derivative takes into account the last point of the previous group by interval? I thought I remembered you working on that at some point.
As for the rest of the code of boiler plate QL functions it looked good to me but I'll defer to @jsternberg  on any of that code.

Looks like derivative doesn't do anything fancy like looking for the previous point outside the time interval being analysed, it just doesn't emit anything for the first datapoint.

@nathanielc derivative will try to take into account the previous point, but only when there is a group by interval.
The only concern I have with this is that derivative will eliminate the first point so if there are 10 points, derivative will return 9. I figure that integral, which should be the opposite of derivative, should restore that point. While we wouldn't be able to restore the + c constant for the integral (which isn't a problem to me personally), I don't see how we would restore the first point that was eliminated by derivative.
If this isn't an issue, then I'm fine with this. @nathanielc comments?

derivative will try to take into account the previous point, but only when there is a group by interval.

This was my concern, as derivative will correctly take into account group by time boundaries, where the current implementation will skip the trapezoid that spans the group by time boundary.
Specifically for this test case https://github.com/influxdata/influxdb/pull/7591/files#diff-4efc5fdc06e36abc4addef9f831c2345R2677
I would expect the integral over the entire time range to be the same as the sum of the integrals over each group by time interval. The data for the test case is (in line protocol):
m value=20  10
m value=10  15
m value=0   20
m value=-10 30

The integral over the entire range is 15*5+5*5+-5*10 = 50.
The integral over the 20s group by time intervals (as it is currently implemented) is 15*5 = 75 for the first interval and -5*10 = -50 for the second interval. The sum of those two interval is 75+-50 = 25. The 5*5 term is missing and so the result is off of 25 from the integral of the entire interval.
I would expect the behavior to be that the first group by interval value is  15*5 = 75 and the second is 5*5 + -5*10 = -25, then the sum of both intervals is 75+-25 = 50 which is the same value we get if we had performed the integral over the entire interval.
In short just like derivative will carry over the last point from the previous interval if we are grouping by time, I think that integral should do the same. And obviously for the first interval there is nothing to carry over. @Tomcat-Engineering @jsternberg @jwilder Does my expectation make sense? Or would you expect each group by time interval to be independent?
As for the restore the first point question, I don't think it is relevant. An integral is a sum of points, so the integral of n points always returns a single point.

The cumulative_integral function which I proposed in the comments above would be the one which is the inverse of the derivative function.  The integral function in this PR is an aggregator, so it returns a single point rather than a time series.
PS - the + c constant doesn't apply in this case because we are dealing with definite integrals.

@nathanielc your expectation makes sense and I'll have a go at implementing it... can anyone point me at the relevant code from derivative?  I'm struggling to find the magic bit that handles group-by-time.

Same here, integral function would be very useful.
What is the status of this PR? Is it planned to be merged soon?

The basic problem is that it is not possible to efficiently find the neighbouring points immediately before and after the queried time interval.  These are necessary to give consistent integral results, as discussed above, but you may have to search the whole of time to find them which would be slow.
It may be possible to do clever things to pass points between adjacent group-by-time intervals, but I don't think we would want to do that as the results would be even more inconsistent: a particular time-group could then have four different integral values depending on whether it is at the beginning, middle or end of the query interval, or is the only group in the interval.
Having thought about it for a while, I can't come up with anything better than my original algorithm, despite its limitations.  Anyone else got any bright ideas?

I think you're right. If we find a solution to that in the future, I assume it won't be a problem for us to add back on that one extra point since it will be at a different time than the previous values.
Can you rebase this and resolve the conflicts? Unfortunately, we're in the weird period where 1.2 has been branched off into its own branch, but we're not merging 1.3 bound PRs to master yet. I also don't have time to look at this in full until probably next week. I'm sorry for the inconvenience. If you can also update the changelog to target 1.3, I'll try to get this merged early next week.
Sorry for taking so long to respond to this.

I'm afraid I might be a bit confused about the direction of this PR and I think some new features (like subqueries) may also factor into that confusion.
Can you explain to me in words which of the following queries are valid and what they are supposed to look like?
SELECT integral(value) FROM cpu
SELECT integral(mean(value)) FROM cpu
SELECT integral(value) FROM cpu GROUP BY time(5m)
SELECT integral(mean(value)) FROM cpu GROUP BY time(5m)

The queries are abbreviated so just assume they have the proper where clause. If I'm understanding correctly, all of those queries would return one value that would just be the sum of the area under the curve. Is that correct?
I'm also thinking subqueries may influence what we do here. Before 1.2, you would have needed a special function to find the sum of an integral. We had one person who really needed a sum of derivatives functionality for a similar purpose. I'm thinking that we might want to make this function more similar to derivative() though so it acts as an exact opposite. That way, the current implementation would be done with:
SELECT sum(integral) FROM (SELECT integral(value) FROM cpu)

Basically, make the proposed cumulative_integral() into the default integral() and suggest that people use subqueries for the sum of that curve.
Thoughts?

SELECT integral(value) FROM cpu
Join each point in cpu to its neighbours with straight lines.  Return the total area under this shape.
SELECT integral(mean(value)) FROM cpu
Doesn't work, both integral and mean are aggregators so this is rejected by the parser.
SELECT integral(value) FROM cpu GROUP BY time(5m)
Within each time-bin, join each point in cpu to its neighbours with straight lines.  Return the total area under this shape for each bin.
SELECT integral(mean(value)) FROM cpu GROUP BY time(5m)
In theory this could compute the mean value for each 5-minute bin and then integrate the resulting curve - but the parser doesn't allow similar nested queries like sum(mean(value)) so for consistency I haven't updated the parser to allow this and the query will be rejected.

Perhaps the confusion is mostly about naming - if you expect integral to be the inverse of derivative then we should use my proposed cumulative_integral as the integral function as you suggest.  I think the "integral as aggregator" was the feature that people meant when they originally requested this, but I'm happy to change the name if it is less confusing.
However we define it, integral will never be an exact inverse of derivative - apart from anything else, derivative returns one fewer points than it receives so we can never get back to the original data.
I think the current implementation is still useful enough to include, perhaps with the name total_integral or something.  Based on the comments in the original feature request, this function is what people are going to use most of the time and it would be annoying to have to use subqueries all the time for this common operation.
This is exactly analogous to the sum function: you could implement this by taking the last value from the cumulative_sum results in a subquery, but the aggregator is used more often so gets its own function.
Likewise, to get the total integral out of the cumulative_integral result you take the last value.  You can see where I got the idea for the naming from!

Yea, you'll never be able to reverse the derivative completely and you'll still lose a point, but there will always be limitations and integrating in calculus already has the + c thing because you can't retrieve the constant after it has been removed when finding the derivative.
I think changing this to be the proposed cumulative_integral() would give more flexibility though because the current implementation of integral() could be implemented with using the subquery that I mentioned in my last comment.
@nathanielc thoughts? Maybe I'm off-base and I don't understand the use cases very well. I'm also not sure which time the integral should output. I would say the time of the second point rather than the first point (like derivative).

integral has been listed in the documentation as an aggregation for ages, so someone at least was expecting it to return a single point!  But happy to rename it to total_integral if you think it is more clear.

For the use cases, if I understand the current implementation correctly, this function will basically resolve the need of time-weighted average, described in #7445, if divided by the time difference between last and first point in each interval. The result is still using interpolation between points, not steps for the value change as described in #7445, but will be already much better than in case of using simple mean() which doesn't consider time. Therefore it will be essential for downsampling time-unregular sensor data.

I don't think integral will solve any problem of time weighted averages because I don't think integral performs any time weighted average. I think we need to implement the weighted_moving_average() function for that.
I'll have a longer post responding to the specifics of integral in a bit. I need some time to organize my thoughts into understandable words.

The value of the integral on the interval divided by the duration of this interval is the average value on this interval, isn't it?

@Tomcat-Engineering
I talked with @pauldix a bit today about the integral() function and we came back to looking at the function in graphite so we could be certain we implemented what was in graphite. Here's a link to their documentation: http://graphite.readthedocs.io/en/latest/functions.html#graphite.render.functions.integral
I'm a bit confused by the description of how integral works here, so I launched a graphite instance and grafana to try some simple graphs to see what these functions did without having to read the documentation. The descriptions are pretty accurate, but they're also confusing because the definition of integral() in the Grafana documentation is not actually what an integral is. It just seems to sum every value that comes in. Now, it doesn't normalize it for time and I don't think you can do that with basic time math in the query so we can talk about a function for doing that, but I'm not sure we should use the name integral(). While it might be confusing for those coming from graphite, I think we should be focusing on implementing integral() and not focusing on implementing what graphite incorrectly thinks an integral is. I looked around for some confirmation about my thoughts here and found this blog post from grafana confirming that graphite is just wrong here.
I would also like to know if cumulative_sum() matches the current function. From what I see, cumulative_sum() should implement the same functionality as graphite's integral(), but without the time normalization. Maybe we can add an argument to cumulative_sum() for it to normalize by time or think of a function name to do that.
Going back to the difference between aggregate and transformation, I don't think an aggregate is correct for integral. I think we put that down in the documentation before anybody took a serious look at implementing it or deciding what that meant. An aggregate is something that will either be used on the entire span of the series and produce one value (usually without skipping values like derivative sometimes does) or it will perform that same operation on disparate intervals. Aggregates don't consider other intervals when calculating the current interval. I don't think that applies to integral and grafana categorizes both derivative and integral for graphite as transformations. So I think we should be thinking of integral as a transformation rather than an aggregate.
I think an initial question to answer is this: should integral in InfluxDB match integral in Graphite or should we implement an actual integral?
I'm also going to write up a follow-up of how I think integral should be implemented, but I wanted to write this to get some initial thoughts so we could be on the same page.

Thanks for the writeup @jsternberg. Seeing this I think we should implement an actual integral function. Sounds like we already have the equivalent to Graphite's function with cumulative_sum.

A few more comments on this and how we should interpret the query. I think we need to define what integral should encompass and determine the syntax. While it doesn't fit into an aggregate, it may also not be the same as derivative and I would like to explore that a bit.
The integral is the area under the curve, but we also need to figure out which part of the area under the curve we're referring to. If we have an example where there are 100 points equally spaced apart, is it desirable to find the area between each of these points? That would result in 99 different areas. There's also the question of whether or not we're finding the cumulative sum of these areas. If you were to use an integral to reverse a derivative, the function that would graph that would be the cumulative sum of these areas. So if the original function is x = t and the derivative is v = 1, then the point at t = 1 would be the integral from 0 to 1 while the points at t = 2 would be the integral from 0 to 2.
So the first question. Should the integral be the cumulative sum of the areas or should it just be the area of the current interval? My preference is that it is the area of the current interval (so from t = 0 to t = 1 and from t = 1 to t = 2 and etc). If you want to get the cumulative sum of the areas, you can just use a subquery to find the cumulative sum from the output of integral.
The next question. If you ask for an integral of a point and specify nothing else, should this produce a single value (the area under the entire curve) or should it produce the area under the curve between each point? The former might be what is expected by users, but the latter is more flexible because you can use the latter with a subquery and sum() to find the former, but you can't do the same with the latter.
That also ties into bucketing. If we want to find the area under the curve for every 10 minute interval, we could use a subquery with a GROUP BY time(...) to implement that if the area of each section under the curve is returned. Unfortunately, if we have a bucket that is partially in one bucket and partially in another, we lose that information if we attempt to do it through a subquery rather than implementing it directly as part of the function.
As confusing as all of that is, I propose two different syntaxes for consideration.
First syntax:
This has integral work as a pseudo-aggregate. It has some properties of streams and some properties of aggregates. It will likely need its own custom implementation since it doesn't neatly fit into either of them.
/* Find the area under the entire curve.
   Internally, this will use the trapezoid rule to calculate the area
   between each point and aggregate all of them together. */
SELECT integral(value, 1s) FROM cpu

/* Find the area for each grouping.
   If the area between two points is partially in one bucket and partially in
   another, linear interpolation is used to divide the two.
*/
SELECT integral(value, 1s) FROM cpu GROUP BY time(1m)

/* Find the integral curve of each grouping by accumulating from previous buckets. */
SELECT cumulative_sum(integral) FROM (SELECT integral(value, 1s) FROM cpu GROUP BY time(1m))

/* It is not possible to find the area between each point with this syntax as a separate bucket. */

Second syntax:
/* Find the area between each point. If there are 100 points, this will emit 99 values. */
SELECT integral(value, 1s) FROM cpu

/* Find the area for each grouping. This will not be able to use linear interpolation when some area is divided. */
SELECT sum(integral) FROM (SELECT integral(value, 1s) FROM cpu) GROUP BY time(1m)

/* Find the area under the entire curve. */
SELECT cumulative_sum(integral) FROM (SELECT integral(value, 1s) FROM cpu)

/* Find the integral curve of each grouping by accumulating from previous buckets. */
SELECT cumulative_sum(sum) FROM (SELECT sum(integral) FROM (SELECT integral(value, 1s) FROM cpu) GROUP BY time(1m))

I'm leaning towards the first. While I think it'll be slightly more complicated to implement because it doesn't have a precedent, I think it will be more clear to users. The second lends itself to using an aggregate within the function better than the first, but I don't like that syntax anyway and I don't consider the syntax necessary now that we have subqueries. The first also has an advantage in that it seems like it matches what people would expect more.
The negative is the first does not make it easy to find the cumulative sum of the integral since it isn't easy to find the dividing line between each of the lines. But you can simulate it easily by just selecting a grouping time like 1s or 10s. Since I'm proposing that it will interpolate the points between the buckets, then you could have points 10 second apart from each other and still use a 1 second grouping since it will interpolate the areas of the buckets.
Thoughts?

Thanks for the notes @jsternberg.
As far as I can tell, with one minor exception your "first syntax" is exactly what I have implemented.
The minor exception is that my algorithm doesn't interpolate across time-bin boundaries in a group-by-time query.  This is because it is really hard (or impossible?) to do this efficiently and I think would require re-writing large parts of the query engine.  This was discussed quite a bit in previous comments, and I thought that we had come to the conclusion that we should release it as-is for now, with suitable notes in the documentation, because lots of users are waiting for this feature.
PS - your conclusions on Graphite's integral are correct.  You based the current InfluxQ cumulative_sum function on someone trying to implement integral by copying Graphite!

PS - I think the idea of using any form of integral to reverse (invert) a derivative is a red herring.  I've attached a sketch showing why - apologies for the slightly wonky drawing!

The first graph shows a simple time series with four points.  Each square on the paper is one unit.  The second graph is the result of the existing derivative function.  The third graph shows that the output of any attempt to integrate the second graph with linear interpolation is just going to be a load of zeroes.  I'm obviously not suggesting that you will always get zeroes, just that the output is rarely likely to be very meaningful or useful.
To get the correct results, we would need to define integral to use a bizarre backwards-sample-and-hold (zero order) interpolation as shown in this version of the second graph, where we interpolate horizontally backwards from each point:

If you apply the cumulative version of your emit-lots-of points integrator to this interpolation you get back to the first curve (though the constant of integration is only correct because the first point happened to be at t=0).
However, we will definitely not be popular if we define our general-purpose integral function to use this crazy type of interpolation!

Ok, I'll review it with an assumption of the first syntax then. I would like the interpolation (and I don't think it would be too difficult), but let's get this merged. Now that I understand the syntax better, it'll be easier for me to review this.

@Tomcat-Engineering I'm going to use your PR as a base for the functionality I described in the first syntax as we discussed before. I'll also be rebasing this onto a more recent commit and making it compatible with subqueries along with trying to implement the interpolation aspect I talked about.
Just giving you a heads up. Is that ok?

@jsternberg - sounds great, thanks a lot and sorry I didn't get around to doing it myself!

I've done the fixup and adapted the PR. I'm going to close this one and open up a new PR.
I still want to add the interpolation feature I mentioned above so integral gives more accurate results. I'm hesitant to merge it without that since adding it later will likely count as a breaking change. The difficulty is that, because of that, integral acts differently than both streams and aggregates and it's showing some holes in the method employed to implement both of those. I'll be continuing to work on that, but I wanted to get up what I currently had for the sake of openness.

Closing this in favor of #8194.


Document the SQL CLI deploy feature

As a starting point, we can give the steps as part of this ticket comment in the mark-down.
Then, we can ask Jake to follow up and adapt https://docs.astronomer.io/astro/cli/sql-cli to contain this new feature.

https://github.com/astronomer/docs/pull/1765


fix(ci): Sleep 10s before running integration test

Issues
Fixes #
Reviewer Suggestions
Where should the reviewer start?
Additional Comments
Ensures sleep 10s before running integration tests to allow for service to go up.

:tada: This PR is included in version 1.2.12 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


What happened with the standalone installation?

I've been not following up with recent developments in Linuxbrew, but something that brought a great deal of problems for me last time I installed it was setting up my standalone installation. Now I can't find any trace of it anywhere and I saw this in README.md:

Bottles are Linuxbrew's precompiled binary packages. Linuxbrew bottles work on any Linux system. They do however require glibc 2.19 or better. On systems with an older version of glibc, Linuxbrew will install glibc the first time that you install a bottled formula

Does it mean that I don't have to worry about it anymore? :D

That's right! Right after a clean installation of Linuxbrew, I recommend running brew install gcc. On a system with glibc older than 2.19, it will start by installing glibc. That's all you need to do!


General Clarifications and Questions

Hey fwends, getting caught up and have a few clarifications and questions:
Seller Ad Service
Trying to understand what restrictions might be envisioned here. I saw "The seller's ad service is the existing stack of the seller that facilitates real-time bidding auctions" so want to dig in on that more:

Is that literally the case, that this is existing code on existing servers running in existing datacenters (caveat that code changes to accommodate SellerFrontEnd flow)? Or are there restrictions such as, this is expected to run in the cloud in a Nitro Enclave, or the types of communication out of the server (log processing, requests in hot path to other systems, creative serving)?
Would Chrome browser attempt to restrict the request/response structure to/from Seller Ad Service, other than the encrypted Protected Audience Data portion as described here.
Would there be any audit expectation?

Event Level Reporting (for now)
I'd like to understand the "for now" part better. Is the expectation that some level of Event Level Reporting will continue just with more restricted data (i.e. no or less Interest Group data, more noise, etc), or that there will be no Event Level Reporting at all? I ask for a few reasons:

Sellers and buyers have a strong claim to event level data that exists now that is not specific to the user who the impression was served to, and that data is used for many purposes outside of targeting.
Perhaps not the Seller, but the Buyer who is creating and using the Interest Groups has a less-strong-but-still-reasonable claim on wanting to analyze the usefulness of their Interest Groups at a detailed level, and to see "landscape" data showing them how they might adjust their buying structures and settings.
Ad Techs, at the least the one I've been working for for 11 years, often need Event Level Data for operational purposes such as debugging.

Also, even in the "for now" times, will SellerFrontEnd and BuyerFrontEnd service be able to log event level data and phone home?
TEEs
I am no security expert, so if there's something very obvious here than apologies, but has any consideration been given to TEEs running in non-public-cloud environments but providing the necessary constraints, attestations, etc, through some combination of technical and audit requirements? I ask because one of the twix-inesses I see here is that Ad Techs (at least one I work for) will likely continue to have to support some set of existing use cases outside of a Fledge/Parakeet context, and those use cases are of size anywhere between non-trivial and quite substantial. Having to setup a TEE inside of a non-public-dc with some set of even relatively "intrusive" requirements could be preferable to forcing network and system topologies.

Closing as after more study I have some of these answers and think it makes sense to discuss others in separate threads.


Switch Renderer on iOS

As a workaround to #2091, presumably one should switch to the SVG render (which in any case always looks much better than the default HTML-CSS; I don't really understand why SVG is not the default). However, since I cannot do a right-click on an equation (and tap-and-hold doesn't do anything), I haven't figured out any way to switch the renderer. Is there any possibility to do this? Maybe something like a bookmarklet?

Alternatively, is there anything I can do server-side to force SVG output? I thought that maybe "jax": ["input/TeX", "output/SVG"] in MathJax.Hub.Config would do this, but it doesn't seem so. What does that option do, actually?

Just found out about the double tap and hold gesture.

Glad you found the correct gesture.  To answer your question about forcing SVG output, that is usually determined by the configuration file you load.  Your page (used to) use config=TeX-AMS-MML_HTMLorMML (it has been changed recently, so I might not be correct about which one it used to use).  The HTMLorMML is the part that determines the output format (in this case, it will be HTML-CSS.  If you used config=TeX-AMS_SVG`, that would use TeX input and SVG output.
Your current page no longer loads a combined configuration file, and so the jax array that you have in your configuration will control that.  The setup you have now is correct (except that you should put the configuration script before the script that loads MathJax.js, as some older browsers will not see the configuration until it is too late if it comes afterward), but there is a drawback to this approach, which is that MathJax will have to load each component separately, so there are multiple network requests.  With the combined configuration file, most of the needed components will be loaded in one network request, which is more efficient, in general.


More layout check fixes

check array strides for multidimensional arrays
check layouts of structs in arrays for multiple indices
new tests

Next steps of #2310

Hi, I've just hit one of the error messages added in this patch, namely member 0 contains an array with stride 4 not satisfying alignment to 16. I couldn't find any reference to this in the spec, and the commit log is also not very descriptive. Can someone point me to some references for this?
Thanks!

That requirement comes from the Vulkan spec.


Created a simple facial signal handler

I am not sure if the points are oriented correctly.

hmm strange too many lines are being merged closing request

the reason for high number of commits is that dlib tree is not squashed


Use map instead of object literals in mainThreadTerminalService

Replaces a few uses of object literals with regular maps, which  better enforce type correctness

Thanks 😃


Custom visualization - mismatch data shape

Hi,
I'm trying to use the pretrained models for inference on custom data. I first convert my data to 4DMOS format, make it be a new sequence and run predict_confidences.py and confidence_to_labels.py with no issue.
Then, I would like to use the predicted labels to filter my raw point clouds. However, if I open the raw point cloud and the predicted label file, they do not have the same shape. Also, the shape of the label does not match the combination of several raw point clouds.
How should I attempt this?
Thank you!

Hey!

However, if I open the raw point cloud and the predicted label file, they do not have the same shape.

This should usually not happen, since 4DMOS does not do any downsampling or clipping. Can you check the if the size of your downsampled point clouds match those from the raw point clouds?

Also, the shape of the label does not match the combination of several raw point clouds.

Sorry, I don't understand what you mean. What do you mean with match the combination?
Best
Benedikt


[FEAT] Filter "sisters" or "brothers" when assigning in Apply Yourself to The Filed Ministry

It would be great to fill in the parts of this section like Initial Call, Return Visit, and Bible Study. (Assignments that can be made by brothers or sisters) a filter appears for selection :)

Yes, you are correct. We should definitely have a filter there. 😁I will add it to our enhancement list.


Anchor build returns the below error always

Problem
error: couldn't read \?\F:\scaff\anchor\target\release\build\typenum-b4aec6864503f063\out/op.rs: The filename, directory name, or volume label syntax is incorrect. (os error 123)
--> C:\Users\Om Chillure.cargo\registry\src\index.crates.io-6f17d22bba15001f\typenum-1.17.0\src\lib.rs:72:5
|
72 |     include!(concat!(env!("OUT_DIR"), "/op.rs"));
|     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: this error originates in the macro include (in Nightly builds, run with -Z macro-backtrace for more info)
error: couldn't read \?\F:\scaff\anchor\target\sbf-solana-solana\release\build\typenum-255e67d6ee89f566\out/op.rs: The filename, directory name, or volume label syntax is incorrect. (os error 123)
--> src\lib.rs:72:5
|
72 |     include!(concat!(env!("OUT_DIR"), "/op.rs"));
|     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: this error originates in the macro include (in Nightly builds, run with -Z macro-backtrace for more info)
error: could not compile typenum (lib) due to previous error
warning: build failed, waiting for other jobs to finish...
error: could not compile typenum (lib) due to previous error
Steps
No response
Possible Solution(s)
No response
Notes
No response
Version
cargo version : 1.80.1

Thanks for the report. This seems to be a duplicate of https://github.com/rust-lang/cargo/issues/13919

Close as duplicate. You may want to contat anchor build maintainer not to canonicalize paths to circumvent this bug, or try some other workarounds listed in #13919. Thank you.


Remove reference to DLP

Removed the "including how your data loss prevention policies may be impacted" because the following link doesn't talk about DLP.

@JohanFreelancer9 - kindly do the copy editor review for this PR. Thanks


[#22] Update codecov

Welcome to Codecov :tada:
Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment.
Thanks for integrating Codecov - We've got you covered :open_umbrella:


[react-native] useInfiniteQuery initialPageParam is not set

Describe the bug
in react-native, despite setting an initialPageParams the first query pageParams is always undefined. this bug doesn't happen with nextjs. I've tested with beta.0 , beta.28, beta.31
useInfiniteQuery({
    queryKey: ["my-key"],
    queryFn: async ({pageParam}) => {
      console.log(pageParam, 'first query is always undefined ')
      const response = await fetch(`https://dummyjson.com/products?limit=12&skip=${pageParam}`)
      const data = (await response.json())
      return data.products
    },
    initialPageParam: 12, // <== doesn't matter what the value is. doesn't get passed to queryfn
    getPreviousPageParam: (firstPage, allPages, firstPageParam) => {
      console.log('get prev always undefined ', firstPageParam)
      return 0
    },
    getNextPageParam: (lastPage, allPages, lastPageParam) => {
      console.log('get next always undefined ', lastPageParam)
      return lastPage?.nextCursor
    },
    maxPages: 3,
  })

Your minimal, reproducible example
https://snack.expo.dev/@advancedtw/use-query-react-native-
Steps to reproduce
run reproduction and check the logs.
expo snack seems to have issue with installing dependencies.  so it's better to run the repro locally.
Expected behavior
I'm expecting to get a pageParam that matches initialPageParam
How often does this bug happen?
None
Screenshots or Videos
No response
Platform
wsl2 , android, react native 0.72, expo 49
Tanstack Query adapter
None
TanStack Query version
beta
TypeScript version
5.2.2
Additional context
The same issue doesn't happen with nextjs so it could be a bundling issue

are you sure you have the correct version of RQ installed? Somewhen in beta, we renamed defaultPageParam to initialPageParam. If you have an older beta version, that would explain it. There shouldn't be any difference otherwise between React Native and nextJs.

expo snack seems to have issue with installing dependencies. so it's better to run the repro locally.

not sure how I'd do that

yes I'm aware of the rename I've seen the pr. also I don't get a type error when I set initialPageParam. I've also checked my nodes_modules (yarn classic)  it's definitely using 5.0.0-beta.31.

There shouldn't be any difference otherwise between React Native and nextJs.

react native projects usually use metro bundler I'm guessing it's possible there is some slight difference in configuration or the way modules are getting picked up. or how the code gets transformed by babel to be fed into hermes.
regardless I have created an other clean project using bun instead of yarn and added the bare minimum dependencies and also removed ts support from the app.  initialPageParam is set as expected so I will close the issue but keep experimenting with different config. if I find something worth documenting I will let you know.
many thanks


UnicodeDecodeError when trying to read semiologies dictionary

Reported by @neurokleos.
/Users/romagnog/Desktop/Slicer.app/Contents/lib/Python/lib/python3.6/site-packages/pandas/compat/_init_.py:117: UserWarning: Could not import the lzma module. Your installed Python is incomplete. Attempting to use lzma compression will result in a RuntimeError.
  warnings.warn(msg)
Traceback (most recent call last):
  File "/Users/romagnog/Desktop/Semiology-Visualisation-Tool-master/slicer/SemiologyVisualization.py", line 52, in setup
    self.makeGUI()
  File "/Users/romagnog/Desktop/Semiology-Visualisation-Tool-master/slicer/SemiologyVisualization.py", line 60, in makeGUI
    self.makeSemiologiesButton()
  File "/Users/romagnog/Desktop/Semiology-Visualisation-Tool-master/slicer/SemiologyVisualization.py", line 134, in makeSemiologiesButton
    semiologiesFormLayout.addWidget(self.getSemiologiesWidget())
  File "/Users/romagnog/Desktop/Semiology-Visualisation-Tool-master/slicer/SemiologyVisualization.py", line 171, in getSemiologiesWidget
    get_all_semiology_terms(), self.onAutoUpdateButton)
  File "/Users/romagnog/Desktop/Semiology-Visualisation-Tool-master/mega_analysis/scores.py", line 54, in get_all_semiology_terms
    dictionary = yaml.load(f, Loader=yaml.FullLoader)
  File "/Users/romagnog/Desktop/Slicer.app/Contents/lib/Python/lib/python3.6/site-packages/yaml/_init_.py", line 112, in load
    loader = Loader(stream)
  File "/Users/romagnog/Desktop/Slicer.app/Contents/lib/Python/lib/python3.6/site-packages/yaml/loader.py", line 24, in _init_
    Reader._init_(self, stream)
  File "/Users/romagnog/Desktop/Slicer.app/Contents/lib/Python/lib/python3.6/site-packages/yaml/reader.py", line 85, in _init_
    self.determine_encoding()
  File "/Users/romagnog/Desktop/Slicer.app/Contents/lib/Python/lib/python3.6/site-packages/yaml/reader.py", line 124, in determine_encoding
    self.update_raw()
  File "/Users/romagnog/Desktop/Slicer.app/Contents/lib/Python/lib/python3.6/site-packages/yaml/reader.py", line 178, in update_raw
    data = self.stream.read(size)
  File "/Users/romagnog/Desktop/Slicer.app/Contents/bin/../lib/Python/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2585: ordinal not in range(128)
could not find nodes in the scene by name or id 'MNI_152_gif'
Segmentation not found in scene: MNI_152_gif_cerebrum

It looks like this error is caused by special characters in the semiology dictionary YAML file, probably those in déjà vu.

See https://stackoverflow.com/q/24475393/3956024


Randomize background

Add an option to grab a random wallpaper from a folder (please).
You could add a boolean option along the lines of RandomBackground that makes it grab a random image from the path specified in Background (the specified path would have to lead to a folder).

I don't believe this is possible due SDDM themselves. However, you can see https://github.com/sddm/sddm/issues/801#issuecomment-1722438128 which is how someone when't about solving the issue


fix(TDI-41943): bump tcompv0 before we do a dangerous dependency adjust to avoid to break studio master

What is the current behavior? (You can also link to an open issue here)
https://jira.talendforge.org/browse/TDI-41943
What is the new behavior?
https://jira.talendforge.org/browse/TDI-41943
Please check if the PR fulfills these requirements

[x] The commit message follows Talend standard
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features) ?
[ ] The code coverage on new code >75%
[ ] The new code does not introduce new technical issues (sonar / eslint)

What kind of change does this PR introduce?

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build / CI related changes
[ ] Other... Please describe:

Does this PR introduce a breaking change?

[ ] Yes
[x] No

Pull request test result: success :star:
Sonar results

Not valid anymote


System tray support

Hi, great project
Please, can you help me with some issues- how I can add a system tray support to application?

As vuido is based on  libui-node, and libui node is based on libui, that means that both of those lower libraries would have to support it first.
So I checked LibUI's issue tracker and see that this is not planned for libui, so there you have it.


proposal: crypto/tls: Provide access to stapled OCSP response in VerifyPeerCertificate callback

I'd like to do OCSP verification using the VerifyPeerCertificate field of tls.Config. My understanding is that it is not possible to access the stapled OCSP response from the peer in this callback. This is because the stapled response is available on the connection itself through the OCSPResposne method on tls.Conn or through the ConnectionState type. Unless there is a way to access it in the callback, the OCSP verification will have to be done after the handshake has been completed, which isn't ideal because the peer logs will show that the connection was successfully established.
Is there a way to currently access the stapled responses in the verification callback that I've missed? If not, is this possible given how the TLS handshake code is currently written?

I'm afraid you are correct. I can only think of two solutions, both not great: adding a new callback, or adding a global map accessible via something like tls.PeerCertificatesAdditionalData(rawCerts [][]byte) *SomeStruct which gets populated right before any call to VerifyPeerCertificate and cleaned up immediately after the callback returns.

@FiloSottile I think of those two suggestions, I'd prefer the additional callback. Also, is it a viable solution to deprecate VerifyPeerCertificate and replace it with a callback that exposes the peer certificates as well as the stapled OCSP response?

While not solving the specific problem of not having access to the stapled response in VerifyPeerCertificate it sounds like implementing some of what is discussed in #22274 would solve the problem that instigates this issue (by which I mean if all you want is to validate stapled OCSP responses, the problem with VerifyPeerCertificate can be mostly handwaved away until another time).

@rolandshoemaker This is fair, but just adding support for MustStaple verification does not necessarily address all use cases. For example, my use case requires me to manually reach out to an OCSP responder via an HTTP request if the certificate is not a MustStaple cert and a staple is not provided. Having access to the OCSP response inside a callback like VerifyPeerCertificate would still be helpful.

@katiehockman pointed out that ConnectionState already has all the information needed to verify a peer, and semantically I don't see how we would want things in VerifyPeerCertificate but not in ConnectionState, and vice-versa.
Taking the opportunity to make the semantics of when the callback is invoked more straightforward, we propose this callback.
type Config struct {
	// VerifyConnection, if not nil, is called after normal certificate
	// verification and after VerifyPeerCertificate by either a TLS client
	// or server. If it returns a non-nil error, the handshake is aborted
	// and that error results.
	//
	// If normal verification fails then the handshake will abort before
	// considering this callback. This callback will run for all connections
	// regardless of InsecureSkipVerify or ClientAuth settings.
	VerifyConnection func(ConnectionState) error // Go 1.15

While at it, I propose we set ConnectionState.ServerName on the client side, too. This way VerifyConnection has access to the ServerName that net/http or other intermediate libraries have set dynamically (which is currently a problem when trying to override normal verification from a http.Transport.TLSClientConfig).
An open question is whether we should allow setting both VerifyPeerCertificate and VerifyConnection. It feels cleaner to make them mutually exclusive, but VerifyConnection will run when VerifyPeerCertificate wouldn't if a client certificate is not requested or required, so it's simple but not trivial to implement VerifyConnection in terms of VerifyPeerCertificate.

Particularly happy about how easy it makes reproducing (and then customizing) normal verification.
&tls.Config{
	InsecureSkipVerify: true,
	VerifyConnection: func(cs ConnectionState) error {
		opts := x509.VerifyOptions{
			DNSName:       cs.ServerName,
			Intermediates: x509.NewCertPool(),
		}
		for _, cert := range cs.PeerCertificates[1:] {
			opts.Intermediates.AddCert(cert)
		}
		_, err := cs.PeerCertificates[0].Verify(opts)
		return err
	},
}

Is the proposal now to add VerifyConnection, not "provide access to stapled OCSP response in VerifyPeerCertificate"?

@rsc that's correct. The proposal is for a new callback, Config.VerifyConnection.

@katiehockman Will this also involve deprecating the existing VerifyPeerCertificate callback or will both be part of the public API?

@FiloSottile In your last comment, you mention "reproducing (and then customizing) normal verification." Does this mean that the user of this callback is expected to do the normal verification and then add the additional custom steps? If so, would it be possible to instead have this be a callback that gets executed after normal verification has already been done by crypto/tls. This would make it so a user only has to worry about the additional steps for their use case rather than also rewriting normal verification code every time it's used.

@divjotarora no I don't expect VerifyPeerCertificate to be formally deprecated as a result of this proposal. There is still an open question about how VerifyConnection and VerifyPeerCertificate will work together though, as @FiloSottile outlined in https://github.com/golang/go/issues/36736#issuecomment-587925536

An open question is whether we should allow setting both VerifyPeerCertificate and VerifyConnection. It feels cleaner to make them mutually exclusive, but VerifyConnection will run when VerifyPeerCertificate wouldn't if a client certificate is not requested or required, so it's simple but not trivial to implement VerifyConnection in terms of VerifyPeerCertificate.

@FiloSottile @katiehockman As part of this, would it be possible to improve some of the documentation for the tls.ConnectionState type? For example, one open question from me: when using tls.ConnectionState.VerifiedChains, can I assume that the 0th element of a chain is the peer certificate and the element directly after is the issuer? I originally made this assumption, but https://tools.ietf.org/html/rfc8446#section-4.4.2 says:
   Note: Prior to TLS 1.3, "certificate_list" ordering required each
   certificate to certify the one immediately preceding it; however,
   some implementations allowed some flexibility.  Servers sometimes
   send both a current and deprecated intermediate for transitional
   purposes, and others are simply configured incorrectly, but these
   cases can nonetheless be validated properly.  For maximum
   compatibility, all implementations SHOULD be prepared to handle
   potentially extraneous certificates and arbitrary orderings from any
   TLS version, with the exception of the end-entity certificate which
   MUST be first.

which makes it sound like I can't assume this? Some documentation improvements to explicitly state what each chain in VerifiedChains looks like would help tremendously.

@divjotarora Very happy to make such usability changes. Could you open a new issue for it so we don't lose track of it?

@FiloSottile Thanks for the quick response! I filed #37572. Would you be able to answer my question about VerifiedChains either here or on that issue? I've tried reading through the client handshake code in crypto/tls a few times but I can't come up with a conclusive answer.

Marking this discussion ongoing for the proposal minutes, but it seems like we're headed toward a likely accept. I just want to be sure to let the discussion finish.

Based on the discussion above, this seems like a likely accept.

No change in consensus, so accepted.

To @katiehockman for implementation. Reminder that we decided to also set ConnectionState.ServerName on the client side, and once this is in I will also replace the VerifyPeerCertificate example with this, which will be so much cleaner.


🚀 Feature: scaffolder-backend-module-gitlab should support only creating an MR w/o committing

🔖 Feature description
Would like to have publish:gitlab:merge-request or another variant action (e.g. create:gitlab:merge-request) that only handles creating of the merge request.
🎤 Context
https://github.com/backstage/backstage/blob/c339e1847a8683b8f9bd39e3c2c654fb44f67a03/plugins/scaffolder-backend-module-gitlab/src/actions/gitlabMergeRequest.ts#L51
Was hoping there was a way to use the publish:gitlab:merge-request action to publish a Gitlab MR w/o having to do commit any files or create/push a branch. In our case, these actions will be handled beforehand using a plugin that wraps around the Git shell, so in this case the branch will already be pushed and the files committed.
✌ Possible Implementation
Option 1
A new action called create:gitlab:merge-request is created that accepts enough input only needed to run api.MergeRequests.create. We don't need any create/push branch logic here, but can add if there is a need.
Option 2
The publish:gitlab:merge-request action supports commitAction of none, which skips trying to find/commit/compare any file changes. In this case, it is fine if the branch is able to be created/pushed. It is more of the commit logic that gets in the way.
👀 Have you spent some time to check if this feature request has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
Yes I am willing to submit a PR!

From my POV I don't see a need to promote this to a dedicated action. Personally I would just make the GitLab API call directly using Roadie's HTTP request action, but option 2 could work. For me commitAction skip feels better than none FWIW.

From my POV I don't see a need to promote this to a dedicated action. Personally I would just make the GitLab API call directly using Roadie's HTTP request action, but option 2 could work. For me commitAction skip feels better than none FWIW.

The benefit of including it here over using the Roadie HTTP plugin is that this plugin is able to take advantage of the logic baked in for getting the token by host by reading from the ScmIntegrations config, making auth one less piece to configure at the template level.
I like the idea of keeping it in a single action and being able set skip to commitAction. If people are on board with that, I can write something up.

Along with the idea of introducing a skip commit action it would seem sensible for auto mode to detect and skip unmodified files. I am about to submit a PR to handle both features.

@mbenson can we close this issue? :pray:

Pretty sure the merged PR handled it to the best of my recollection.


Debug websockets

DO NOT MERGE

Confirm local validator is working properly

Pull Request Test Coverage Report for Build 2369

188 of 219 (85.84%)  changed or added relevant lines in 12 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-2.06%) to 68.244%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/routes/v2/route-utils.ts
20
21
95.24%

src/util/winston-logging.ts
5
6
83.33%

src/routes/v2/rawtransactions.ts
20
23
86.96%

src/routes/v2/blockchain.ts
39
44
88.64%

src/routes/v2/address.ts
32
39
82.05%

src/routes/v2/slp.ts
18
32
56.25%

Totals

Change from base Build 2312:
-2.06%

Covered Lines:
1367

Relevant Lines:
1896

💛  - Coveralls


Improve Vtable implementations

note: since codegen was run on this PR, it's easier to review commit by commit as code gen is always runs in a separate commit
This PR slightly simplifies vtable implementations while also making them a bit more type and memory safe:

Change IUnknownImpl::get_impl to return a reference to the actual implementation type which is not specified as an associated type. This avoids passing a void pointer around which prevents the compiler from doing some type and borrow checking.
Also change IUnknownImpl::get_impl to take a &self instead of &mut self. Com interfaces are non-exclusive references to a shared vtable so &mut self is just wrong and is UB.

This change is easiest to see when comparing COM vtable method implementations before and after this change:
Before
unsafe extern "system" fn Name<Identity: ::windows::core::IUnknownImpl, Impl: ILearningModelVariableDescriptorPreview_Impl, const OFFSET: isize>(this: *mut ::core::ffi::c_void, result__: *mut ::core::mem::ManuallyDrop<::windows::core::HSTRING>) -> ::windows::core::HRESULT {
  let this = (this as *mut ::windows::core::RawPtr).offset(OFFSET) as *mut Identity;
  let this = (*this).get_impl() as *mut Impl;
  match (*this).Name() {
  // ...

After
unsafe extern "system" fn Name<Identity: ::windows::core::IUnknownImpl<Impl = Impl>, Impl: ILearningModelVariableDescriptorPreview_Impl, const OFFSET: isize>(this: *mut ::core::ffi::c_void, result__: *mut ::core::mem::ManuallyDrop<::windows::core::HSTRING>) -> ::windows::core::HRESULT {
  let this = (this as *const *const ()).offset(OFFSET) as *const Identity;
  let this = (*this).get_impl();
  match this.Name() {
  // ...

One additional change I believe we can make is changing the this pointers to *const instead of *mut.

Thanks Ryan, this certainly feels less hacky. 👍


add edit links to collection details page

Why was this change made?
Fixes #695, adding edit collection links on the collection detail pages
Some notes:

I created a base class for the Collection show page components, so I could add a helper method to create the edit links in just one place, this mimics what is done for the Work show page (which uses a single wrapper component for the whole show page, unlike what is done for collections) ... see https://github.com/sul-dlss/happy-heron/blob/main/app/components/works/detail_component.rb#L37-L42 and https://github.com/sul-dlss/happy-heron/blob/main/app/views/works/show.html.erb#L4 and compare to the collection show page
There is a link in the Terms part of the show page that is intended to link to the new section being added to the collection show page in 793

How was this change tested?
Localhost browser and unit tests
Which documentation and/or configurations were updated?
None

Inheritance should only be used when "X is a Y" is true. The ShowComponent represents the whole page, so it's not true that the smaller components on the page are themselves specializations of the whole show page.

Ok, reverted all the inheritance and now using a component for the collection edit links.

Inheritance should only be used when "X is a Y" is true. The ShowComponent represents the whole page, so it's not true that the smaller components on the page are themselves specializations of the whole show page.

Ok, reverted all the inheritance and now using a component for the collection edit links.


Allow backtracking for function calls without parentheses

I am working on a language using Plex, and would like to remove parentheses from function calls. From what I can tell according to this question, doing this would require "backtracing", to allow patterns like this in the parser: Ident(i) exp_list[e] { if ($1 is not defined) { back_and_choose_another_rule(); } }. Would it be possible to add this, or is there better way to do this that Plex is capable of?
P.s. — I'm new to using parsers so I may have no idea what I'm actually talking about. Sorry if this doesn't make sense!

Sorry, I don't think this would be very easy to add to plex. It's really not designed for non-LALR(1) grammars.


feat:增加对 aliyun oss 的支持||feat: added support for Aliyun OSS

Github Action Bot detected the issue body's language is not English, translate it automatically

Added fetch_owner to ListObjectsInput to indicate whether to obtain the owner information
The type of the Object in the ListObjectsOutput indicates the current state of the Object

What this PR does:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

原文   ListObjectsInput 增加 fetch_owner 表示是否获取 owner 信息
   ListObjectsOutput 中 Object 增加 type 表示Object 当前的状态

What this PR does:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

```</details>
<!--MD5:c0a90d5476272640b633e2522bffa1f6:MD5-->

https://github.com/mosn/layotto/blob/main/pkg/grpc/extension/s3/s3.go#L414 pb到component的入参有转换逻辑的，需要在component里面的结构体定义里面也加上

please fix your ut


 with multiple becomes unresponsible after changing contents of ngOptions list in IE11

There seems to be a couple of issues listed for similiar issues but I just updated to 1.4.8 and still have this issue.
I have two of these in a directive to allow users to move items from one list to the other (both lists are subsets of a bigger list of items, and are filtered so no items can be in both lists).
<select id="multiSelectAvailable"
ng-model="selected.available"
multiple="multiple"
size="{{available.length}}"
ng-options="e as e[displayAttr] for e in available">
</select>

<select id="multiSelectSelected"
ng-model="selected.current"
multiple="multiple"
size="{{model.length}}"
ng-options="e as e[displayAttr] for e in model">
</select>

Sometimes when setting the list of items one or both of the selects becomes unresponsive. After changing the contents a couple of times it starts working again, it seems to happen if I have items containing Swedish 'å' characters in one of the lists. However it could just be a coincidence.

@Obrell86, I can't be sure, but it sounds more like an IE issue than Angular.
Have you tried reproducing this without Angular (i.e. relying to native APIs) ?
If it does indeed only happen with Angular, could you post a live demo (e.g. using CodePen, Plnkr etc) ?

Also, have you tried with the latest code on master ?
Do I understand correctly that the issue only presents itself o IE11 ? What other versions/browsers have you tried this on ?

Since I've found similar bugs listed here I figured it might be related, I'll try and write up a native implementation.
I've tested in IE11 and Chrome 47.0.2526.106 m and have only been able to reproduce in IE11.

Well my native implementation works fine. I guess I could somehow use that instead but I sure hoped this would work within Angular since I won't have any data-bindings and lose a lot of time.
I'm putting together a JSFiddle with a recreation.

Just to be clear, I did not suggest you should use a non-Angular implementation in an Angular app.
I was just trying to verify if this is indeed an IE issue or an Angular issue. If it turns out to be the latter, we should definitely fix it.

Alright, I have a fiddle up here: https://jsfiddle.net/3e6cv6en/
You can reproduce the issue by selecting an item in the From: select and the select an item in the To: select. In some cases one or both of the multi-select elements will be come unresponsible.

Hm, I can reproduce the issue. Having "Anna" on one side and "Gerda" on the other produces it pretty realiably (after the first time selecting both).

Good to know it's not just a local occurrence. I've put the story for this on hold and will pick it back up later on.
Time for holiday vacation.

I've recently ran into this issue as well, very frustrating. My current workaround until a better solution is found, and hopefully this inspires someone, is to change the CSS of the select element in a non-passive render way. For example:
function ieSelectElementFix() {
        var fooSelect = angular.element(document.getElementById("fooSelect ")),
            fooSelectWidth = parseInt(fooSelect .css("height"), 10),

        fooSelect .css("height", "" + (fooSelect Width + 1) + "px");

        $timeout(function() {
            fooSelect .css("height", "" + (fooSelect Width) + "px");
        });
}

A better way on stack overflow was to remove and add the DOM element, but that causes a blink effect when this way a small tremor occurs, but usually too fast to catch if you're not looking for it.

@johnfoley3 Your solution doesn't seem to work for this particular problem, unfortunately. What works for me is doing     foo.text(foo.text()); on the first option of a frozen select.
The underlying cause though seems to be that we internally reuse option elements when the option collection gets updated: https://github.com/angular/angular.js/blob/0348347841191530417f6c9eb6e991c33f0e8c99/src/ng/directive/ngOptions.js#L613-L614
That means we just update label, text and selected. It looks like this makes ie choke.

@Narretz  I have tackled this similar issue for quite a while. I have tried 1.4.1, 1.5.0 with xeditor. The first time two options list loaded with data and right away the selection become in-responsive in IE11. If this has been fixed, please let me know what version of Angular 1.?.? is. Thank you.

@dannycsng The fix is available in 1.5.5. I can't guarantee that it fixes all frozen select issues, because they are untestable and sometimes can't even be reproduced systematically manually.

@Narretz I will definitely try this version. I find a strange behavior where if I touch the DOM on the font of the options list, it opens up for selection. Then, it freeze again. It seems a workaround like johnfoley3 to touch the DOM may work although it is a hack. Your opinion?

@Narretz I am sorry to report that it works partially in version 1.5.5. It opens up for selection when first loaded. Then, it freeze up again after select and move from one list to another. Also, touching the DOM works sporadically too. Any opinion from anyone?

@dannycsng please open a new issue with a minimal demo that shows the problem


Wrong column size in table properties

As you can see in the attached image

The column panel in table properties screen shows the column details fully compressed.
I searched in DBeaver properties but I didn't find anything,
I'm using DBeaver (version 3.7.*, 3.7.7 actually) on Ubuntu 14.04.5.
Is it a bug or there is a way to solve ?
Tnx

Seems to be duplicate of #875.
Try to click on column pack button:

Does this help?

Thanks  @serge-rider
Yes, it's true, the button 'Pack Column' solves the issue.
I sow your code fix, but I actually don't have the time understand if there will be a way to have column always unpacked (I didn't find any option in DBeaver Preferences).
Are you inserting a specific option in Preferences window ?

I think that columns resize problem happens because DBeaver packs columns before actual editor becomes visible.
I've committed a fix which is supposed to resolve this.
I can make EA version to let you test this.

If you have time tu build an EA version, I will test it, yust to confirm the solution.
Thanks

This doesn't  take much time.
EA is here: http://dbeaver.jkiss.org/files/ea/

Great !
It is working well!
For me, your solution works.


feat: add exception logging function, add tests

this feature adds exception logging functionality

that's odd, because it worked for me every time and it looks like this:

I will try to reproduce the error

:tada: This PR is included in version 4.13.0-beta.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 4.13.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Ligatures are displayed incorrectly for D (dlang) variadic templates

VSCode Version: 1.13.0 insider
OS Version: Arch Linux x64

Using the font "Fira Code", ligatures for D variadic templates are incorrectly displayed.

It's not just with ligatures, I see this without them as well. There is simply no support for variadic templates in the syntax file (yet).


Add cryptocurrency badges

added in https://github.com/inttter/md-badges/commit/b670dcadbfa0c566122ebaff365fdc666532e525


add sdedkins

Checklist

[x] Used a fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@conda-forge-admin, please rerender

Hi! This is the friendly automated conda-forge-webservice.
I tried to re-render for you, but it looks like I wasn't able to push to the master branch of CitrineInformatics/citrination-client-feedstock. Did you check the "Allow edits from maintainers" box?


PubSub example fails with logging enabled.

Hi, I've tried to enable logging for the PubSub example.
I replaced the following code in the runtimeHub.py init function
from;

    self._logManagerHub.disable()

to;

     self._logManagerHub.enable()
     self._logManagerHub.enableFileOutput()

The previously running example now fails consistently with a -1 error.

    AWS IoT SDK Version(dev) 1.0.3-

    Setup failed!
    -1

The logger does create a new log;

[2016-03-04 14:10:41.256372] Register timeout signal handler.
[2016-03-04 14:10:41.272485] serialCommunicationServer init.
[2016-03-04 14:10:41.277257] serialCommunicationServer set accept timeout to 15
[2016-03-04 14:10:41.282366] serialCommunicationServer set chunk size to 50
[2016-03-04 14:10:41.287254] Clear internal list. Size: 0
[2016-03-04 14:10:41.291974] Accept-timer starts, with acceptTimeout: 15 second(s).
[2016-03-04 14:10:41.296875] 4 lines to be received. Loop begins.
[2016-03-04 14:10:41.301735] Received: 1/4 Message is: i
[2016-03-04 14:10:41.306648] Received: 2/4 Message is: cvmon1
[2016-03-04 14:10:41.311479] Received: 3/4 Message is: 1
[2016-03-04 14:10:41.316364] Received: 4/4 Message is: 4
[2016-03-04 14:10:41.321048] Finish reading from remote client. Accept-timer ends.
[2016-03-04 14:10:41.330683] Paho MQTT Client init.
[2016-03-04 14:10:41.337009] Register Paho MQTT Client callbacks.
[2016-03-04 14:10:41.341630] mqttCore init.
[2016-03-04 14:10:41.346389] Set maximum connect/disconnect timeout to be 10 second.
[2016-03-04 14:10:41.351113] Set maximum MQTT operation timeout to be 5 second
[2016-03-04 14:10:41.356251] Updated serialCommunicationServer internal protocolMessageQueue by inserting a new message. Size: 1
[2016-03-04 14:10:41.361421] Send through serial to remote client: I T Size: 3
[2016-03-04 14:10:41.366106] Clear internal list. Size: 0
[2016-03-04 14:10:41.370721] Accept-timer starts, with acceptTimeout: 15 second(s).
[2016-03-04 14:10:56.370924] Raise a custom exception for accept timeout.
[2016-03-04 14:10:56.373646] Accept Timeout

Have I missed something in enabling logging?
Thanks,
Jeff

Hi Jeff,
Thank you very much for using AWS IoT Arduino Yun SDK!
The AWS IoT Arduino Yun SDK is built on top of the dual processor architecture. ATmega32u4 accepts user commands and communicates them back to AR9331 through Serial1 bridge. The AR9331 then does all the heavy-lifting MQTT operations. A Serial1 communication protocol is established to make the transmission happen between the two processors, just as what you see in the log file.
This protocol occupies the STD_IN/STD_OUT on AR9331. In other words, the STD_IN/STD_OUT for AR9331 should not have other messages going through, otherwise it will break the protocol and trigger the error handling in SDK. The enable method for logging automatically enables the console output as default, which breaks the protocol. You will need to specifically disable it:
 self._logManagerHub.enable()
 self._logManagerHub.enableFileOutput()
 self._logManagerHub.disableConsolePrint()

In this case, you should be able to do persistent logging and store them as log files on AR9331, without breaking the Serial1 protocol in between.
However, one important thing to keep in mind is that doing persistent logging directly on AR9331 is not a very good idea, as is mentioned in the official Arduino Yun Getting Started Guide:

You're discouraged from using the Yún's built-in non-volatile memory, because it has a limited number of writes.

If you really want persistent logging, one thing you can do is to use external memory like a microSD card or thumb drive and expand the disk space for Arduino Yun. More details can be found on Arduino Tutorials.
Hope that answers your question. Please feel free to reopen the issue if you have more questions.
Thanks,
Liusu

Hi Liusu,
Thanks for the reply. I was aware of the Serial1 bridge but did not catch explicitly disabling the console. I have added a microSD card though I am only enabling the logging to extend the SDK to add some functionality lost by the unavailability of the Serial1 bridge. Specifically I would like to add a call to the python time() function in the runtime and return it to the mcu.
Is there any documentation available regarding extending the SDK?
Thanks,
Jeff

On Mar 4, 2016, at 8:30 PM, Liusu Zeng<EMAIL_ADDRESS>wrote:
Hi Jeff,
Thank you very much for using AWS IoT Arduino Yun SDK!
The AWS IoT Arduino Yun SDK is built on top of the dual processor architecture. ATmega32u4 accepts user commands and communicates them back to AR9331 through Serial1 bridge. The AR9331 then does all the heavy-lifting MQTT operations. A Serial1 communication protocol is established to make the transmission happen between the two processors, just as what you see in the log file.
This protocol occupies the STD_IN/STD_OUT on AR9331. In other words, the STD_IN/STD_OUT for AR9331 should not have other messages going through, otherwise it will break the protocol and trigger the error handling in SDK. The enable method for logging automatically enables the console output as default, which breaks the protocol. You will need to specifically disable it:
self._logManagerHub.enable()
self._logManagerHub.enableFileOutput()
self._logManagerHub.disableConsolePrint()
In this case, you should be able to do persistent logging and store them as log files on AR9331, without breaking the Serial1 protocol in between.
However, one important thing to keep in mind is that doing persistent logging directly on AR9331 is not a very good idea, as is mentioned in the official Arduino Yun Getting Started Guide https://www.arduino.cc/en/Guide/ArduinoYun#toc4:
You're discouraged from using the Yún's built-in non-volatile memory, because it has a limited number of writes.
If you really want persistent logging, one thing you can do is to use external memory like a microSD card or thumb drive and expand the disk space for Arduino Yun. More details can be found on Arduino Tutorials https://www.arduino.cc/en/Tutorial/ExpandingYunDiskSpace.
Hope that answers your question. Please feel free to reopen the issue if you have more questions.
Thanks,
Liusu
—
Reply to this email directly or view it on GitHub https://github.com/aws/aws-iot-device-sdk-arduino-yun/issues/9#issuecomment-192429564.

Hi Jeff,
Currently we do not have a detailed documentation to extend the SDK.
For a custom command over the Serial1 bridge, the part of the SDK you need to extend is the Python runtime library.
The core functionality for AWS IoT Arduino Yun lies in the Python runtime library, which provides an interface for Serial1 communication: serialCommunicationServer. The AWS-IoT-Arduino-Yun-Library (the one gets included in Arduino IDE and runs on ATmega) generates the protocol-style command pieces and sends it over the Serial1 bridge. The serialCommunicationServer accepts commands (in protocol style) from ATmega and then triggers the corresponding command on the Python side.
-----------------------ATmega----------------------|-Serial1-|---------------------AR9331---------------------
ArduinoLib API call -> protocol-style command ->>>> serialCommunicationServer -> PyCmd -> AWS IoT
------------------------------------------------------|-----------|--------------------------------------------------
Each of the Serial1 command has its own protocol representation. You can get a better overview of the built-in commands by checking here.
Take aws_iot_mqtt_client::config as an example. If you look into the implementation of AWS-IoT-Arduino-Yun-Library, you can find that what config API actually does is to print protocol-style command pieces to Serial1:
exec_cmd("6\n", false, false);
exec_cmd("g\n", false, false);
    ...
exec_cmd(rw_buf, false, false);
exec_cmd(rw_buf, false, false);
    ...
exec_cmd(rw_buf, false, false);
    ...
exec_cmd(rw_buf, false, false);
    ...
exec_cmd(rw_buf, true, false);

So the protocol-style command pieces for config would like this:
6\ng\n<param0>\n<param1>\n<param2>\n<param3>\n<param4>\n

with a newline at the end of each chunk.
Then on the Python side (serialCommunicationServer), it accepts these protocol command pieces line by line and recognizes them to be the config command. It triggers the execution of the corresponding commandClass: commandConfig, performs the operation and writes back the response:
...
self._mqttCoreHandler.config(self._parameterList[0], int(self._parameterList[1]), self._parameterList[2], self._parameterList[3], self._parameterList[4])
...
self._serialCommServerHandler.writeToInternalProtocol(returnMessage)

Basically, a custom command will do the same thing by utilizing the Linux processor and communicate back the resources to ATmega. Adding a new custom command requires a new Python command class, a new protocol command and the corresponding API for AWS-IoT-Arduino-Yun-Library. The serialCommunicationServer also needs to be modified to allow new commands to be accepted and executed.
We are sorry we cannot provide a more positive answer about extending the SDK at this moment. Please feel free to post follow-up questions as you dive deeper into this.
Thanks,
Liusu


Drag and drop does only work with first element.

Describe the bug
When I open the list for the first time I can only drag and drop with the first element, when this is once done it works fine.
https://user-images.githubusercontent.com/25926746/231208501-aa2410db-ca71-4216-b5ca-f90c84165c1c.mp4
Code looks like:
<NestableScrollContainer>
  {condition && <View>...</View>}
  <NestableDraggableFlatList
      refreshControl={
        <RefreshControl
          onRefresh={refetch}
          refreshing={isLoading}
          colors={[theme.colors.primary.main]}
          tintColor={theme.colors.primary.main}
        />
      }
      data={sortedIndices}
      dragItemOverflow
      debug
      containerStyle={[styles.listWrapper, { paddingBottom: bottom }]}
      onDragEnd={({ data: dragData }) =>
        dispatch(setSortedZoneIndices(dragData))
      }
      keyExtractor={(index) => `${index}`}
      renderItem={ZoneCard}
      {...testId('switch_zone_screen')}
    />
</NestableScrollContainer>

const ZoneCard = (props) => (
  <TouchableOpacity {...props}>
    <View>...</View>
  </TouchableOpacity>

Platform & Dependencies
Please list any applicable dependencies in addition to those below (react-navigation etc).

react-native-draggable-flatlist version: 4.0.1
Platform: IOS
React Native or Expo version: 0.71.6
Reanimated version: 2.14.4
React Native Gesture Handler version: 2.9.0

Additional context
Add any other context about the problem here.

This were the issue =>  keyExtractor={(index) => ${index}}


Matplotlib forced on qt4 with Spyder

Hello everybody,
I recently reinstalled Anaconda, since I was still using python 2 and since, I can't plot anything with Matplotlib
even the simplest code:
import matplotlib.pyplot as pyplot  pyplot.plot([1, 2, 3, 4, 5, 6],[4, 5, 1, 3, 6, 7])  pyplot.show()
raises an exception:
"Cannot load backend 'Qt4Agg' which requires the 'qt4' interactive framework, as 'qt5' is currently running "
No matter what I do to force matlab on qt5, each time I run my code, it wants to go on qt4. On the other hand, everything works fine on Conda (but it is less convenient to code there)
Do any of you happens to know the why and how of this problem?
Versions
Spyder version: 3.3.6
Python version: 3.7
Qt version: 5.9.6
PyQt version: 5.9.2
Operating System name/version: win32 - Windows 10
Dependencies
IPython >=4.0     :  7.6.1 (OK)
cython >=0.21     :  0.29.12 (OK)
jedi >=0.9.0      :  0.13.3 (OK)
matplotlib >=2.0.0:  3.1.1 (OK)
nbconvert >=4.0   :  5.5.0 (OK)
numpy >=1.7       :  1.16.4 (OK)
pandas >=0.13.1   :  0.24.2 (OK)
pycodestyle >=2.3 :  2.5.0 (OK)
pyflakes >=0.6.0  :  2.1.1 (OK)
pygments >=2.0    :  2.4.2 (OK)
pylint >=0.25     :  2.3.1 (OK)
qtconsole >=4.2.0 :  4.5.1 (OK)
rope >=0.9.4      :  0.14.0 (OK)
sphinx >=0.6.6    :  2.1.2 (OK)
sympy >=0.7.3     :  1.4 (OK)

No matter what I do to force matlab [sic] on qt5, each time I run my code, it wants to go on qt4

What settings have you tier in "Preferences>IPython console>graphics>graphics backend" ?

I tried QT4, QT5 and automatic and it always raises the exception.

Do gtk and tk backends work?

I don't know how to check that...

Try to change "Preferences>IPython console>graphics>graphics backend" to "Gtk", "Gtk3" or "TKinter

Do gtk and tk backends work?

@bcolsen, @doudouball is on Windows.

@doudouball, I really doubt this is a problem with Spyder. It's usually caused by a matplotlibrc file lying around in your file system. Please look for it and remove it.

@ccordoba12 The thing is when I run my code directly, it runs without problem, that's why I believe the issue is due to Spyder. When i remove matplotlibrc, I get the exception "RuntimeError: Could not find matplotlibrc file; your Matplotlib install is broken"

The thing is when I run my code directly

What do you mean by this? Is it "outside Spyder"?

When I use the Anaconda prompt and call the .py file, it runs without any issue

I'm sorry but I don't know what happens in your case. The only solution that occurs to me is to to go to
Tools > Preferences > IPython console > Graphics

deactivate the option called Activate support and run directly %matplotlib qt5 in our consoles.

I tried QT4, QT5 and automatic and it always raises the exception.

I Changed it to  QT5 and restart kernel, it worked for me

Hi I am getting Error
Cannot load backend 'Qt4Agg' which requires the 'qt4' interactive framework, as 'qt5' is currently running
Tell me how to resolve it...?