Dependencies not injected on classes without @inject using Python 3

Hello, documentation states following:
If you use Python 3 you can make Injector use constructor parameter annotations to determine class dependencies. The following code:
class B:
   @inject
   def __init__(self, a: A):
       self.a = a

can now be written as:
class B:
    def __init__(self, a:A):
        self.a = a

I can not get it working:
from injector import *

class A:
    pass

class B:
    def __init__(self, a: A):
        self.a = a

class MyModule(Module):
    @singleton
    @provider
    def provide_a(self) -> A:
        return A()

injector = Injector(MyModule)
injector.get(B)

If @inject is added to the B's constructor, then everything working fine.

The exception is:
TypeError: __init__() missing 1 required positional argument: 'a'

During handling of the above exception, another exception occurred:

injector.CallError: Call to B.__init__() failed: __init__() missing 1 required positional argument: 'a' (injection stack: [])

Hmm, the documentation is outdated, not sure why (https://readthedocs.org/projects/injector/builds/ says last build is 1 year 4 months old). @alecthomas help?
This piece of documentation has been removed in 5e435c00233be8d54e8dbf6ba17a699ca52528b5 as it wasn't even correct, @inject is required.

Could you clarify reasons why is this feature was dropped? It looks so convenient to me because of:

simplicity of use
one does not need to "pollute" business code (possibly pure python without dependencies) with implementation details like injector

Thanks!

Readability and documenting intent (if you see @inject you expect for at least some of the arguments to be provided by the dependency injection mechanism, not by the client code).
If you don't mind me asking - what's the situation that results in "pure python without dependencies" and Injector-provided arguments at the same time?

Sure. For example, i have a library that is written with pure python and have no third-party dependencies.
This library is packaged and used in several projects.
For example, there are a class A that has some public dependencies from the same library. Every client instantiates this class manually with providing an instantiated dependencies to it.
So the library is third-party dependency for other projects itself and can not depend on injector.  But the projects can use injector. So to build the A class one need to write a module with a factory function that creates an instance and then call injector.get(A) instead of just calling injector.get(A).
I hope i was accurate enough to describe the situation.

Yes, thank you, that was informative. I'd like to add one thing - decorating constructors with @inject doesn't make your library depend on Injector strictly speaking - yes, you need to be able to import inject and the injector package needs to be installed but the client code is free to keep constructing the objects manually. @inject doesn't prohibit manual instantiation.

it's a shame the decorator is required, you could presumably figure out the positional args at the point binder.bind is called.
I'd much prefer to register all my services with binder.bind instead of making modules take a dependency for the decorator.

The presence of the decorator is, at this point, a design decision meant to document that dependency injection is expected to take place, it's not intrinsically required and serves (see the implementation) as just a marker really. So whether it's a shame or not depends on a point of view.

when you say it's not required, it's not obvious how to get an Injector instance to be able to resolve even with explicit calls to binder.bind
from injector import Injector, ClassProvider

class Inner:
    def __init__(self):
        self.forty_two = 42

class Outer:
    def __init__(self, inner: Inner):
        self.inner = inner

def configure(binder):
    binder.bind(Outer, to=ClassProvider(Outer))
    binder.bind(Inner, to=ClassProvider(Inner))

injector = Injector(configure)

inner = injector.get(Inner)
print(inner.forty_two)

# Throws
outer = injector.get(Outer)
print(outer.inner.forty_two)

I've just started trying out injector on a new project and it looks quite interesting. But I would have to agree that requiring @inject is an unfortunate choice. Although it's a common choice for DI frameworks to make, IMO it is not a good choice. It's actually one of the first things I look at when I consider adopting a DI framework.
Here's some theoretical reasons why it doesn't make sense to require @inject:

"Dependency injection" is a general technique and does not imply the use of a DI framework. A framework can be used, but we can also create object graphs "by hand" - the two are functionally equivalent. As such, classes shouldn't need to broadcast their availability to DI frameworks.
From the perspective of the class using @inject, it makes no difference whether it is being created by a container or some other way. As long as the dependencies are provided, it shouldn't care how it got them. In other words, @inject is not something the class cares about, it's something the application cares about. The responsibilities here are all wrong.
One reason for other DI frameworks to require the equivalent of @inject is that the some other languages permit overloaded constructors. In that case @inject can serve to disambiguate which constructor to use. However, python does not have this issue.

There's also one really important practical reason to consider: it can actually hinder adoption of this library. In the past, I've had to search for DI frameworks for large projects. But we passed on a lot of - otherwise good - frameworks because they forced us to either add @inject to a lot of code, or to hand-write a bunch of providers, or both (we can't just modify 3rd party libraries). On top of that, if the annotations were non-standard, we would be effectively locked into using that specific framework. It can be hard to justify such things, despite any benefits we might derive. But when the framework just allows us to use a top-level injector without all the extra work, then it becomes a really attractive choice.

I should note that we are using injector in few large projects and many smaller ones (microservices). We are using DDD approach, so injector and it's @inject decorator is just a cross-cutting concern, just like logging, etc. With such approach @inject becomes a no problem from theoretical point of view.
It could be better to not to use that DSL to annotate constructor injection, but anyway it is working perfectly for us. Also, we are not defining or using binder for each of dependency, but just for specific implementations of abstract classes (interfaces) that can not be chosen automatically and provided from nowhere. The code stays clean and simple.

@kwvanderlinde In case I wasn't clear in the earlier comments, @inject's presence (together with @noninjectable etc.) is to document availability/expectation of DI using a DI framework to the programmers, not to the DI framework. The framework could work just fine without it. As such, while your points may apply in principle, they just represent one set of values (and design decisions) while others exist. I hope that you can see that in this light and in this context "hindering adoption of this library" is a non-argument.
That said, adding an opt-in way to assume @inject in every situation it's expected now would not be out of question I think.

I don’t think announcing that an ioc container is expected to be used is useful, the classes constructor announces it needs dependencies that is enough, a readme file cabinet explain to new comers that an ioc container is used.
The decorator is enough to prevent me making use of this project as it would essentially be required for every application service or infrastructure class I ever wrote, to at some point replace the ioc container would therefore require making changes to every single class in the project whereas with a more imperative style I would only have to go to a single place (or a couple of modules) to update the configuration.
I hope you don’t opt for assuming @inject for all constructors I think that would be a mistake, I just wish there was both an imperative and declarative way to configure based on the needs of a given project.

@netijoe, the logic for injecting dependencies is currently implemented in the @inject decorator. Does not that count as a technical reason?

Thank you for the response. I based my post on this one:

The presence of the decorator is, at this point, a design decision meant to document that dependency injection is expected to take place, it's not intrinsically required and serves (see the implementation) as just a marker really. So whether it's a shame or not depends on a point of view.

It seemed like it should work without @inject when binder.bind is used.


Changelog init

Start to write a changelog file

Tasks done :

[x] Create a "feature/initChangelog" branch
[x] Create and fill the "changelog.md" file
[x] Commit the "changelog.md" file


Consider adding API for enumerating all visible Inputs in PVW and/or PGM

I'm submitting a ...
[ ] bug report
[x] feature request (I'll code it and open a PR if we think its a good idea)
[ ] question about the decisions made in the repository
[ ] question about how to use this project

Summary
A common task is writing code which reacts to which Inputs are present in PGM and/or PVW. This is harder than it seems at first glance, because we can't simply use the previewInput/programInput property of the ME state object, because that wouldn't account for keyers and SuperSources. To figure out what Inputs are actually displaing in PVW/PGM right now, we have to loop through all keyers and SuperSources.

This is a common enough task that I think it should be provided by this library. This is my first thought at what this API would look like:
atem.on('stateChanged', state => {
    const me = state.video.getMe(0);

    // This includes the following Inputs:
    // 1. The current `programInput`
    // 2. If `programInput` is a SuperSource, then it also includes all inputs being used by that SS (recursively)
    // 3. Any inputs currently being shown as DSKs or USKs
    console.log(me.getVisibleInputs('program')); // => logs an array of numbers

    // Same as above but for PVW instead of PGM.
    console.log(me.getVisibleInputs('preview')); // => logs an array of numbers
});

I'm not completely married to this API design, and am very open to feedback and suggestions.
cc @baltedewit and @Julusian for their thoughts

I think having this in the library would make sense.
In your checks, you didnt account for if ME2 is referenced anywhere by ME1.
I don't think it should live on the ME object though, as the checks need to be able to access all of the state.
Perhaps a loose/importable helper function
function listVisibleInputs(state: AtemState, mode: 'program' | 'preview', me = 0): number[]

Some questions:

Do we include any sources used in an active/inactive dip/stinger/dve transition?
Does program contain all of preview while mid transition?

I agree with your points.

Do we include any sources used in an active/inactive dip/stinger/dve transition?

I think including sources being actively used in a transition makes sense. I guess I'm not sure what you mean by "inactive", or why inactive things would show up as PVW or PGM.

Does program contain all of preview while mid transition?

I think it should, because in my mind, "PGM" means "on the air in any capacity", which is true for the entirety of the transition. Y'all are more experienced than me though, so if that's not what the industry expects then I'm fine using a different heuristic.

Great idea, love it. I think this would be perfect as a method on the Atem class as part of the high level API. I don't think it would be useful to run this on a state that isn't the current state so passing the state as a parameter might be redundant.
I think we should attempt to keep this in line with tally output as displayed on the internal multiviewer.

I think we should attempt to keep this in line with tally output as displayed on the internal multiviewer.

Yes, you're totally right, that is what users will expect. Is this well-documented anywhere? If not, my ATEM is already set up in a rack in my living room so I can put in the time to figure that out.

Findings:

Everything involved in a transition is marked as being in PGM for the entire duration of that transition.

This inlcudes the current PVW and PGM sources, as well as any source involved in the transition via a DIP, WIPE, STRING, or DVE transition. This also includes both the key and fill of things like the DVE transition.

Yes, all items of the SuperSource show up as PGM while the SS is in PGM.

This recurses down into other MEs if they are one of the SS boxes.

Yes, taking one of the secondary MEs into ME1 as PGM will result in that secondary ME's current PGM source(s) being marked as being in ME1 PGM.

In short: the multiviewer and tally seem to exclusively care about what ME1 is doing, and will recurse through SuperSources and nested MEs all the way to the leaf nodes to figure out what all is in PVW and/or PGM. And during a transition, everything involved in that transition is marked as PGM for the entirety of said transition.
I think this is all I need to know, and I can begin implementing this algorithm now.

I'm making progress, but something I think we're missing is a lookup table of what input numbers correspond to what type of source. For instance, if my code sees that input 123 is in PGM, it needs to know if that ID corresponds to a normal input, supersource, ME, etc. Only then can it properly recurse the tree.
I thought I remembered us having something that did this mapping, but I can't find it. Does that only exist in LibAtem, and do I need to port it over here? @Julusian

I was just about to reply with InternalPortType :)

I think including sources being actively used in a transition makes sense. I guess I'm not sure what you mean by "inactive", or why inactive things would show up as PVW or PGM.

By inactive I meant transition properties for the next transition. I'm not sure why they would show up though. Probably best they dont.

@baltedewit I think this would be perfect as a method on the Atem class

Yeah, I forget that is somewhere we can put things. I can see that having it separate might be useful in future for something tsr/timeline (ie checking what we are about to do), but lets leave that until it is needed.
Re findings, that sounds like what I remember experiencing with an atem, and is what I would expect. Please make sure we are able to run the algorithm starting from ME2, as that would be really useful when each ME is used in isolation.

Please make sure we are able to run the algorithm starting from ME2, as that would be really useful when each ME is used in isolation.

Will do.

Oof okay, hit a new problem:
Say I see an Input with ID 10020. I can figure out that its an MEOutput via InternalPortType, but I don't know how to programmatically figure out which ME it corresponds to without hard coding a map like LibAtem has. Any thoughts?

This has been merged :)


[core] traverse_obj: Various fixes

IMPORTANT: PRs without the template will be CLOSED
Description of your pull request and other information

Implement utils.traverse_obj tests, update docstring
and reimplement the function completely to fix various issues.
Also allow the now stabilized dict key for public consumption.
Template

Before submitting a pull request make sure you have:

[x] At least skimmed through contributing guidelines including yt-dlp coding conventions
[x] Searched the bugtracker for similar pull requests
[x] Checked the code with flake8 and ran relevant tests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check one of the following options:

[x] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[ ] Fix or improvement to an extractor (Make sure to add/update tests)
[ ] New extractor (Piracy websites will not be accepted)
[x] Core bug fix/improvement
[ ] New feature (It is strongly recommended to open an issue first)

-@decorator
+@ decorator

Is this just fashion?

I didn't even notice. That was autopep8 doing (or rather not doing) it's work in b2b2af6a6e6339997287381f2deae74169e094b5. It is unrelated to the code.
It must have gotten messed up when autoformatting on save with invalid syntax? I am very sorry for providing an unclean commit like that.

I missed it during merge, but have reverted it in a latter commit

Auto-formatting tools are the correct solution to code styling ... except when they're not.


chore(filter_key): Avoid filter key creation if user does not have permission

SUMMARY
When loading a dashboard, Superset would always fire a request to /api/v1/dashboard/{{dashboard_id}}/filter_state?tab_id={{tab_id}}, regardless if the user has access to interact with this endpoint or not. If the user doesn't have permission, a 403 FORBIDDEN error is returned, resulting in a console error.
Even though the error message is expected, a console error is not ideal specially for Embedded implementations in which usually console errors are monitored for the hosting app, for proper troubleshooting.
This PR adds a validation so that this request is only triggered in case the user has access to it.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Before

After

TESTING INSTRUCTIONS

Enable Embedding (through the EMBEDDED_SUPERSET FF).
Embed a dashboard into a hosting app.
Access it.
Validate that there are no console errors.

Another potential test:

Create a Role without permission to create a dashboard permalink.
Publish a test dashboard.
Grant this role to a test account.
Make sure the test account has access to the published dashboard.
Access Superset using this account.
Validate that there isn't a console error for this endpoint.

ADDITIONAL INFORMATION

[ ] Has associated issue:
[x] Required feature flags: EMBEDDED_SUPERSET
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

hey @michael-s-molina @villebro @kgabryje I would really appreciate if you could share your thoughts about this change. I'm still getting familiar with Superset frontend so any feedback is really appreciated!

@Vitor-Avila This endpoint is used to store the filters state allowing a user to refresh the page without loosing inputed values. If a user has access to the filters panel, it should always have access to this endpoint. I believe this is a problem with our fragmented permissions, which leads to invalid combinations (ex: give a user access to SQL Lab but not the permission to execute queries 🤦🏼♂). The solution is to combine the permissions into one or just make sure users that have access to the filters panel also have the permission to this endpoint.

that makes a lot of sense, @michael-s-molina. Thanks for the insight. In this case, this def not feel like the correct way to fix this -- I'll close this PR.

@michael-s-molina I was syncing with @eschutho about this issue, and we were wondering if this change would still make sense specifically for the embedded use-case. Embedded users should view the filter modal, but they shouldn't have access to write a native_filter_key (or persist a filter combination) given these URL parameters are not loaded in the embedded mode.
Let me know your thoughts! Thank you!

It's possible. Just keep in mind that the host app will lose the ability to refresh without loosing the current filter selections. In that case, we should check for the permission before calling publishParameters and you should get user from Redux.


File upload not working

Hi, can I confirm that file upload should work?
I have created a stand alone FastAPI endpoint and can upload a file to it no problem, but when using this lib, it fails. Does not matter what file type I test with and the cURL calls are identical
curl --location 'http://localhost:8000/api/py' \
--form<EMAIL_ADDRESS>
Fast API version
from fastapi import FastAPI, UploadFile
app = FastAPI()

@app.post("/api/py/")
async def create_upload_file(file: UploadFile):
    # do something with the file
    return {"filename": file.filename}

But the Sveltekit version under routes/api/py/+server.py fails
from fastapi import UploadFile

async def POST(file: UploadFile):
     print(file.filename)
     return {"filename": file.filename}

The response I receive is:
{
    "detail": [
        {
            "loc": [
                "body",
                "file"
            ],
            "msg": "field required",
            "type": "value_error.missing"
        }
    ]
}

Turns out I was sending the request as "files" instead of "file" and the SveltKit version got stuck, while the FastAPI version did not.


[Question] WelcomeMessage not appearing automatically on Web Chat Channel

System Information (Required)

SDK Language: C#
SDK Version: <IP_ADDRESS>
Development Environment: LOCALHOST, AZURE_BOT_SERVICE, AZURE_APP_SERVICE

Issue Description
I cannot get welcome message. I have shamelessly copied the code from github, but cannot get it to work. The Welcome message never appears on Web Chat channel until I hit "hi" or something.
Code Example
            IConversationUpdateActivity update = message;
            var client = new ConnectorClient(new Uri(message.ServiceUrl), new MicrosoftAppCredentials());
            if (update.MembersAdded != null && update.MembersAdded.Any())
            {
                foreach (var newMember in update.MembersAdded)
                {
                    if (newMember.Id != message.Recipient.Id)
                    {
                        var reply = message.CreateReply();
                        reply.Text = Responses.WelcomeMessage;
                        client.Conversations.ReplyToActivityAsync(reply);
                    }
                }
            }

Steps to Reproduce

I get Welcome Message on emulator (though I have to wait for long)
I do not get Welcome Message in Web Chat channel (i'm using the preview version)

Expected Behavior
Welcome message to appear
Actual Results
The welcome message doesn't come on Web Chat channel until you hit "hi" or something.
Screenshots

Bot screenshot

Also, I can't get "Bot is Typing" on Web Chat channel. It appears on emulator but not on Web Chat channel.
Code snippet pasted below
` var connector = new ConnectorClient(new Uri(activity.ServiceUrl));
Activity isTypingreply = activity.CreateReply();
isTypingreply.Type = ActivityTypes.Typing;
            await connector.Conversations.ReplyToActivityAsync(isTypingreply);`

@ranmax123 try something like this.  It seems you are not casting your reply as an activity.
 if (member.Id == iConversationUpdated.Recipient.Id)
                        {
                            var reply = ((Activity)iConversationUpdated).CreateReply($"Hi! I'm a bot! Ask me things.");
                            await connector.Conversations.ReplyToActivityAsync(reply);
                        }

@JasonSowers , no it doesn't work still. Is it a bug in the web chat channel code?

@ranmax123  please try replacing the following code, I was able to replicate your error.  Also be sure to refresh the page if it does not work the first time.   I found that if the website with the web chat was auto-launched by my publishing the app I would have to refresh.  If I directly entered the web chat URL into my browser (as an end user would do)  it worked fine
if (newMember.Id != message.Recipient.Id)

with
if (newMember.Id == message.Recipient.Id)

@ranmax123 please let me know if you are still experiencing issues.  I am going to close this issue, but if this doesn't work, we can always reopen.

no worries, I'm using older version of web chat channel for now.

On  emulator welcome message is appear, But in Web Chat Channel message is appear when user write something.
Note: I am get member id from web url and get user information from our server.

Welcome message not showing in web chat channel but shows in emulator.

This works for me try this code exactly:
            else if (message.Type == ActivityTypes.ConversationUpdate)
            {
                // Handle conversation state changes, like members being added and removed
                // Use Activity.MembersAdded and Activity.MembersRemoved and Activity.Action for info
                // Not available in all channels
                                IConversationUpdateActivity iConversationUpdated = message as IConversationUpdateActivity;
                                if (iConversationUpdated != null)
                                {
                                    ConnectorClient connector = new ConnectorClient(new System.Uri(message.ServiceUrl));

                                    foreach (var member in iConversationUpdated.MembersAdded ?? System.Array.Empty<ChannelAccount>())
                                    {
                                        // if the bot is added, then
                                        if (member.Id == iConversationUpdated.Recipient.Id)
                                        {

                                            var reply = ((Activity)iConversationUpdated).CreateReply($"Hi! I'm a Spotify bot! Ask me about your favorite artist.");
                                            await connector.Conversations.ReplyToActivityAsync(reply);
                                        }
                                    }
                                }
            }


Prepare for new release of Bitbucket

Workflow links: null, null.

Actual builds:
JDK 8: https://github.com/atlassian-labs/atlassian-slack-integration-server/runs/2031870536
JDK 11: https://github.com/atlassian-labs/atlassian-slack-integration-server/runs/2031858467

Current Bitbucket plugin version 3.0.3 is marked as compatible with Bitbucket Server 7.10.1.


Link CVEs to Platform

Howdy,
I've noticed that some CVEs can be associated with software versions/platforms of devices.
Would it be possible to add referencing CVEs to platforms so you could see in the platform views what CVEs are associated?
Also, as a bonus, could you also use the platform association to populate the affected devices based on device/platform?

an environment may have multiple versions of that platform, so the end user should map several platforms one for each version

This is the way we do it in our environment. We are also using ansible to upgrade our devices (just switches for now) and to update NetBox with the new platform. I know associating a CVE to multiple Platforms might be tedious, but I think it might be the best way to go (and you might be able to use tools like Ansible to automate it).

Thanks for sharing the way you map the platforms.
For the NIST CVE API I'm using this doc as reference:
https://www.signalscorps.com/blog/2023/building-cve2stix-1-cpes-cves-nvd-api/
And the  GET request is built that way:
cpe:2.3::::
::::<sw_edition>:<target_sw:>:<target_hw>:
With that schema I need to be able split the platform =  from its version and using only the platform name isn't much reliable...
But it may be a good option to use platforms this way but calling custom fields from they.

Howdy,
Just for some clarity, this FR is to add assigning CVE's to Platforms manually to allow the basic hand-jamming associations.
The GET method is still very good, but I thing just falls a bit out of scope of this FR.
v/r
-Jake

Ohh I see @cyberndj, about the Vulnerability Assignments, in the next release it may be configuration based, so the users gonna able to assign vulnerabilities to any model


response-ratelimiting plugin appears broken in 0.5.3

I've got two instances of Kong running, 0.5.2 and 0.5.3, both spun up identically via the official AWS AMI.
The response-ratelimiting plugin works correctly in 0.5.2.
In 0.5.3, the X-RateLimit-Remaining header does not decrement (from the perspective from the API consumer) and the API is never limited.
Perhaps 0.5.3 requires different configuration than 0.5.2?  Perhaps the cassandra connection is messed up and the plugin usage statefulness is not getting stored?

This is weird since we didn't change anything in the response rate-limiting plugin in 0.5.3: https://github.com/Mashape/kong/compare/0.5.2...0.5.3
Do the error logs show anything weird? Maybe Cassandra is not incrementing the counters, thus they always remain the same?

2015/11/21 22:28:59 [error] 2738#0: lua entry thread aborted: runtime error: ...share/lua/5.1/kong/plugins/response-ratelimiting/log.lua:7: bad argument #1 to 'log' (string, number, boolean, or nil expec
ted, got table)
stack traceback:
coroutine 0:
	[C]: in function 'log'
	...share/lua/5.1/kong/plugins/response-ratelimiting/log.lua:7: in function 'increment'
	...share/lua/5.1/kong/plugins/response-ratelimiting/log.lua:15: in function <...share/lua/5.1/kong/plugins/response-ratelimiting/log.lua:11>, context: ngx.timer, client: <IP_ADDRESS>, server: 0.0.0
.0:443

It appears that in log.lua line 7, the :increment function on the DAO returns an error. Unfortunately the error is not a strong but a table (like all DAO errors), hence, the ngx.log function cannot log it. One needs to execute tostring(err) to output the content of the error in ngx.log. If this was corrected we could see the error properly in the logs.

I re-deployed the AMI and it's fine.  Sorry for the false alarm!


Calculating kurtosis from histogram

How would you suggest that I calculate the Kurtosis of the histogram?
Ideally I would have at least 1024 bin (10 bit adc resolution)
Would you be able to provide such an example?

1024 bins means you need 4K RAM to hold all bins if you use the Histogram class.
If you use Histogram16 you need 2K RAM to hold the bins.
Some googling showed there is quite some math involved.
These links might help you to get started.

https://en.wikipedia.org/wiki/Kurtosis
https://www.geeksforgeeks.org/how-to-calculate-skewness-and-kurtosis-in-python/?ref=lbp
https://www.johndcook.com/blog/skewness_kurtosis/

(from the last link, not Arduino code)
#ifndef RUNNINGSTATS_H
#define RUNNINGSTATS_H

class RunningStats
{
public:
    RunningStats();
    void Clear();
    void Push(double x);
    long long NumDataValues() const;
    double Mean() const;
    double Variance() const;
    double StandardDeviation() const;
    double Skewness() const;
    double Kurtosis() const;

    friend RunningStats operator+(const RunningStats a, const RunningStats b);
    RunningStats& operator+=(const RunningStats &rhs);

private:
    long long n;
    double M1, M2, M3, M4;
};

#endif

And here is the implementation file RunningStats.cpp.

#include "RunningStats.h"
#include <cmath>
#include <vector>

RunningStats::RunningStats()
{
    Clear();
}

void RunningStats::Clear()
{
    n = 0;
    M1 = M2 = M3 = M4 = 0.0;
}

void RunningStats::Push(double x)
{
    double delta, delta_n, delta_n2, term1;

    long long n1 = n;
    n++;
    delta = x - M1;
    delta_n = delta / n;
    delta_n2 = delta_n * delta_n;
    term1 = delta * delta_n * n1;
    M1 += delta_n;
    M4 += term1 * delta_n2 * (n*n - 3*n + 3) + 6 * delta_n2 * M2 - 4 * delta_n * M3;
    M3 += term1 * delta_n * (n - 2) - 3 * delta_n * M2;
    M2 += term1;
}

long long RunningStats::NumDataValues() const
{
    return n;
}

double RunningStats::Mean() const
{
    return M1;
}

double RunningStats::Variance() const
{
    return M2/(n-1.0);
}

double RunningStats::StandardDeviation() const
{
    return sqrt( Variance() );
}

double RunningStats::Skewness() const
{
    return sqrt(double(n)) * M3/ pow(M2, 1.5);
}

double RunningStats::Kurtosis() const
{
    return double(n)*M4 / (M2*M2) - 3.0;
}

RunningStats operator+(const RunningStats a, const RunningStats b)
{
    RunningStats combined;

    combined.n = a.n + b.n;

    double delta = b.M1 - a.M1;
    double delta2 = delta*delta;
    double delta3 = delta*delta2;
    double delta4 = delta2*delta2;

    combined.M1 = (a.n*a.M1 + b.n*b.M1) / combined.n;

    combined.M2 = a.M2 + b.M2 +
                  delta2 * a.n * b.n / combined.n;

    combined.M3 = a.M3 + b.M3 +
                  delta3 * a.n * b.n * (a.n - b.n)/(combined.n*combined.n);
    combined.M3 += 3.0*delta * (a.n*b.M2 - b.n*a.M2) / combined.n;

    combined.M4 = a.M4 + b.M4 + delta4*a.n*b.n * (a.n*a.n - a.n*b.n + b.n*b.n) /
                  (combined.n*combined.n*combined.n);
    combined.M4 += 6.0*delta2 * (a.n*a.n*b.M2 + b.n*b.n*a.M2)/(combined.n*combined.n) +
                  4.0*delta*(a.n*b.M3 - b.n*a.M3) / combined.n;

    return combined;
}

RunningStats& RunningStats::operator+=(const RunningStats& rhs)
{
        RunningStats combined = *this + rhs;
        *this = combined;
        return *this;
}

Additional links

https://forum.arduino.cc/t/kurtosis-calculations/395561
https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Higher-order_statistics

Found some time,
Made an experimental sketch + library based upon the above code.
It does not implement the adding of two Kurtosis objects.
Kurtosis.zip

Slightly faster skew() as a division is replaced by multiplication (to be verified).
    double skew()
    {
      return sqrt(double(_count)) * M3 * pow(M2, -1.5);
    };

@RobTillaart that is very kind of you to jump into this and put together that library!
Today my my code does something of this kind:
I have two histogram buffers, I increment a bin on each ADC value (running at 250Khz sample rate) and I swap buffers every 100ms, so that I can process one while the other is being filled.
RAM consumption is not of an issue as I am using a Teensy 4.1 with 1MB of internal RAM and 16MB of PSRAM,
I assume that in order to setup the same kind of architecture, I would need two histogram classes?

(Not a statistician, never worked with the kurtosis/skewness until today, so all disclaimers apply)
Thinking out loud,
Using two buffers sounds logical however swapping the buffers means (1) either you clear and start all over again or (2) you continue with the contents of that histogram.

case 1 you loose historical data which is good or not (for you to decide)
case 2 you miss 50% of historical data during analysis, which could introduce some interference if your dataset has a similar frequency in it. (e.g. if you are sampling a signal that changes every 100 ms too).

What you should do is to use three histogram objects(x,Y,Z),
two of them (X and Y)are filled alternatingly,
the third (Z) is the sum which adds the data, so you always work on all samples.
in pseudo code
task_filler()
{
  forever()
  {
    value = adc();
    if (flag == x) X[adc]++;
    else Y[adc]++;
  }
}

task_analyzer()
{
  Kurtosis K;
  forever()
  {
    K.reset();
    //  add the Y histo to the Z histo and add the new Z value to K.
    for (idx = 0; idx < max; idx++)
    {
      if (flag == x)  Z[idx] += Y[idx];
      else Z[idx] += X[idx];
      K.add(Z[idx]);
    }
    print(K.count + "\t");
    print(K.kurtosis());

    //  clear process historgram, "swap" X and Y, and start again.
    if (flag == x)
    {
      Y.clear();
      flag = y;
    }
    else
    {
      X.clear();
      flag = x;
    }
}

250 KHz samples every 100 ms means that you could have 25000 samples in one bin.
So the X and Y histogram could be a histogram16 type, where Z must be an histogram type

Created a new library - https://github.com/RobTillaart/Kurtosis (under development)
The calculation of the kurtosis of a histogram is more work as you need to add every entry, e.g. if bin 13 has a count of 123, you have to add 123 times the value bin 13 represents. I can implement an add(value, times) but the only gain is that you have no function call overhead. Might still be worth it
So it will easier to have one histogram and add all measurements to the Kurtosis object directly after making the measurement.

@david-res
Released the version 0.1.0 of the kurtosis library today.
Furthermore in the new 0.1.1 develop branch I have added add(double value, uint8_t times).
This is code under investigation and looks very promising.
There are some overflow risks that can be tackled by using uint64_t for the internal counter.
Another option is to refactor the code in an overflow preventing way.
This add(double value, uint8_t times).allows to add a single value multiple times in one call.
Current implementation is more efficient from times >= 3 and has a near constant runtime as far as tested.
So it is ideal for going through a histogram to determine the skewness and
(need to find time)

@RobTillaart that's amazing, thanks!!
Could you help me put together a simple example of a 1024 bin histogram (I saw you need to declare the bin list in the class, and it can't be set after - so how would I do over 1k bins without looping over b[] to assign a value to each bin?)
Then use the Kurtosis to calculate the value based on the histogram?
Once I have something basic in front of me, I can scale it up for the double or triple buffers etc

There might be a (time consuming) trick to handle this, I will dive into it right away

Trick failed, 😒, learned (again) that adding distributions is different from adding values.
This means that the add(value, times) will be removed from the 0.1.1 API.
So the only way is just add the values when they are produced by the ADC.
Sorry I could not solve your question. For me the investigations will stop for now as I am out of ideas and time.
I will finish the 0.1.1 version of the library asap.

@RobTillaart so today what I do is I fill a buffer(histogram) with a count increment based on the ADC value as each bin represents an ADC value, something like this:
`uint32_t histogramBuffer[1024] = {0};
void adc0_isr(void) {
uint16_t measurement = adc->adc0->analogReadContinuous();
// Increment the corresponding histogram bin
histogramBuffer[measurement] += 1;
}
//called after switching histogram buffers
void processData(){
for(int i=0;i>1024;i++){ //Loop over each bin
uint32_t count = histogramBuffer[i]; //get bin count value
for(int j=0;j<count;j++){
K1.add(i);
}
}
Serial.print("KURT:\t");
Serial.println(K1.kurtosis());
}`
My only concern is the Kurtosos is not aware of bins that have a value of 0 - would that not put off the calculation?

The for loop is doing that already. If count eq 0 then the loop won't run.
But then the mean of the histogram will be off if we are not reporting bins with a value of 0.
That's why its better that the add function have two argument - the bin number and it's value.
What do you think?

I see now zero bins are handled correctly.
What is a possible concern is that the inner loop uses an int for j which should be an uint32 to prevent overflow (if int is 16 bits it could happen)

Good catch, I missed that for int j.
But what about my comment regarding the mean calculation?
I think I could increment a variable each mine the bin value eq 0. The call K1.add(0) x variable count.
This will assure I have the right mean set.

Simple example
1024 bins, 1 measurements e.g. 654.
If you add zero for every bin that is zero the internal count would be 1024 and the average  0.65... while the average of the measurements is 654.
Bins with value 0 should therefor not be added as zero.

K1.add(0)
Means that you want to add the measurements with the value zero. (E.g the ADC reads 0Volts)

Right right, that makes total sense now.
I was looking at if from the aspect of values and not count of values

@RobTillaart I got the Kurtosis class working, thanks!
I have another question - how can I create a pointer array to two Histogram objects? This is so that I can swap between them every 100ms in my existing intervalTimer interrupt.

Tomorrow I might have some tome

I ended up creating two histograms object: H1 and H2
I them created a class pointer array:
Histogram *  hiatogram[2] = {&H1, &H2};
it compiles, but I have not run it yet

Looks like nice pointers which will work.
However if your application runs multitasking you have to take some precautions to make swapping them atomic.
It is easier to have an array of histograms and adjust the (byte size) index one may write to.. this index is either 0 or 1 and can only have one value, so atomic by design (note it should be byte sized).

Depending on board swapping two pointers can be made atomic enough.
You have one process that writes to a H1 and a process that reads H2.  When the reading process is done it clears H2 and changes the pointer if the write process to pint to H2. (Both pointers point to H2 now however the read process is dine).
The write process continues writing and the read process changes its pointer to H1.
So it is possible if the reader is in controll of the swapping.

//
//    FILE: hist_pointer.ino
//  AUTHOR: Rob Tillaart
// PURPOSE: demo histogram pointer
//     URL: https://github.com/RobTillaart/Histogram

#include "histogram.h"

//  boundaries does not need to be equally distributed.
float a[] = {
  0, 50, 125, 200, 250, 350, 500
};
float b[] = {
  0, 100, 200, 300, 325, 350, 375, 400
};

Histogram histA(7, a);
Histogram histB(8, b);
Histogram *phist;

void setup()
{
  Serial.begin(115200);
  Serial.println(__FILE__);
  Serial.print("HISTOGRAM_LIB_VERSION: ");
  Serial.println(HISTOGRAM_LIB_VERSION);
  Serial.println();

  phist = &histA;

}

void loop()
{
  phist->clear();
  for (int i = 0; i < 10000; i++)
  {
    int x = random(600) - 50;
    phist->add(x);
  }

  Serial.print(phist->count());
  for (uint16_t i = 0; i < phist->size(); i++)
  {
    Serial.print("\t");
    Serial.print(phist->frequency(i), 2);
  }
  Serial.println();

  //  swap
  if (phist == &histA) phist = &histB;
  else phist = &histA;
}

//  -- END OF FILE --

compiles and runs.
Sample output shows the different length of the 2 histograms.
...Histogram\examples\hist_pointer\hist_pointer.ino
HISTOGRAM_LIB_VERSION: 0.3.6

10000	0.08	0.08	0.12	0.13	0.08	0.16	0.25	0.09
10000	0.09	0.17	0.16	0.17	0.04	0.04	0.04	0.04	0.24
10000	0.09	0.08	0.13	0.13	0.08	0.16	0.25	0.09
10000	0.08	0.17	0.16	0.17	0.04	0.04	0.04	0.04	0.25
10000	0.09	0.09	0.12	0.12	0.08	0.17	0.25	0.09
10000	0.08	0.17	0.17	0.17	0.04	0.04	0.04	0.04	0.24
10000	0.08	0.08	0.13	0.13	0.08	0.17	0.25	0.08
10000	0.09	0.17	0.17	0.16	0.04	0.04	0.04	0.04	0.24
10000	0.09	0.08	0.13	0.12	0.08	0.17	0.25	0.09

Created develop branch with pointer and array example + reference to new kurtosis library.
Think the original issue of kurtosis math is solved now.

@david-res
Develop branch is merged, you may close the issue

Note to self:
thinking about the Kurtosis of a histogram.
(thinking out loud modus)
if a bin representing a value V has a count of C, does in mean it has a C values added in the range R = <V-delta, V+delta>
so when adding a bin to do some statistics me could / should / might add some noise so that we add C values in the range R.
So we do not add C times V but V +-delta, which averages to V but could represent the original values better.
Of course if a bin only handles integer values, one should only regenerate integer values within the range R.
As said before, no statistician, however the idea would recreate the possible spread of the original values.
It might be closer to reality.

Cheers @RobTillaart!
Already got the Kurtosis implemented with my current histogram.
For some reason when I use your histogram class it stops my UI from rendering, but everything else keeps working - will investigate it soon.

For some reason when I use your histogram class it stops my UI from rendering, but everything else keeps working - will investigate it soon.

Please open a new issue if it is a histogram issue.
FYI added a standalone kurtosis and skewness function to https://github.com/RobTillaart/kurtosis develop branch.
Works only if you have a full dataset available as it needs to go through it twice.


⚠ Bofrost.lu has degraded performance

In f492963, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 891 ms

Resolved: Bofrost.lu performance has improved in 94e366e after 7 minutes.


MudBlazor XAML

Feature request type
Other
Component name
No response
Is your feature request related to a problem?
No response
Describe the solution you'd like
We Can Fork MudBlazor Project And Create another project for maui xaml and have mudblazor like components
Have you seen this feature anywhere else?
No response
Describe alternatives you've considered
No response
Pull Request

[ ] I would like to do a Pull Request

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hi.
This is not on our roadmap. MudBlazor is exclusively focused on the advancement and development of Blazor technology.


Can openAPI ts remove type declarations after project construction to reduce packaging volume

Description
Can openAPI ts remove type declarations after project construction to reduce packaging volume

Hi @JL-Code, I don't understand the feature request here. Can you provide an example?


First task implementation.

The following work has been done:
• A graph loading and manipulation system has been created.
• Added the function of saving the graph in DOT format.
• Tests have been written for the system.

Вы сильно опоздали с дедлайном, поэтому только половина баллов.


⚠ Mobitopup has degraded performance

In e181106, Mobitopup (https://napcard.voz.tech) experienced degraded performance:

HTTP code: 200
Response time: 3349 ms

Resolved: Mobitopup performance has improved in c45bd68 after 7 minutes.


Move i18n resources (Messages_XX.properties) from DSpace/dspace-api-lang into DSpace/DSpace

Description
Currently, there is a difference in i18n message resource handling between DS backend and DS frontend.
In the DS frontend all message resources are kept in the same project. In contrast, DS backend uses the Maven dependency org.dspace:dspace-api-lang to get i18n resources from https://github.com/DSpace/dspace-api-lang.
With the replacement of the JSPUI by Angular SPA the length of Messages.properties (in the DS backend) has reduced dramatically. Currently, the file has ~130 lines. This brings me to the question if this additional complexity (two projects instead of one) is still needed. Is it better to move the i18n files from https://github.com/DSpace/dspace-api-lang/tree/main/src/main/resources to the DS Java base project and set the dspace-api-lang project in archive mode?
There was a discussion in Slack (https://dspace-org.slack.com/archives/C3SG47SGY/p1723134160720209). @tdonohue wrote:

It's an excellent question.  I'm not against moving Messages.properties back into DSpace/DSpace (and all translation files).  It's only still separate because of how dspace-api-lang was used in DSpace 6.x and below.
Overall though I'd want feedback from others on this idea.  Perhaps the best approach is to create a GitHub Issue ticket to discuss the pros/cons, and decide on whether to "retire" dspace-api-lang or not.

I'm going to add this to the Dev Mtg agenda for Aug 22, to see if others have opinions on this direction.  I'd like feedback from others on this.

There was general approval in today's DevMtg (Aug 22) for this issue ticket.  So, all Committers in attendance agreed that it'd be best to retire the DSpace/dspace-api-lang project and move all Messages_XX.properties files into DSpace/DSpace likely under here: https://github.com/DSpace/DSpace/tree/main/dspace-api/src/main/resources
This is just in need of a volunteer to help this process along.  (I may be able to help move this along.  But, I currently don't have time to create a PR.)   Moving this to the 9.0 board in the hopes we can get this done during that release.

@tdonohue , currently there are 6 open PRs in dspace-api-lang. Is it better to review (and hopefully merge) these PRs before we archive dspace-api-lang?

@saschaszott : The PRs in dspace-api-lang all appear to be for DSpace 6 or below (as they translate older JSPUI keys) with the exception of your two recent PRs: https://github.com/DSpace/dspace-api-lang/pull/40 and https://github.com/DSpace/dspace-api-lang/pull/41
I've gone ahead and merged your PRs and closed all others.

I've gone ahead and merged your PRs and closed all others.
Thanks, @tdonohue !


Off-by-one error in outer waitgroup

After completing, sqsmv panics with
panic: sync: negative WaitGroup counter
goroutine 1 [running]:
sync.(*WaitGroup).Add(0x4cd4f8, 0xc0000120f0)
	/usr/local/go/src/sync/waitgroup.go:74 +0x105
sync.(*WaitGroup).Done(0x9)
	/usr/local/go/src/sync/waitgroup.go:99 +0x25
main.main()
	/go/src/app/main.go:59 +0x582

This appears to be due to an extraneous defer wg.Done() on line 51
Removing this line allows sqsmv to complete successfully without panicking
Addressed by https://github.com/scottjbarr/sqsmv/pull/33

Thanks for catching the error in my PR!


fix: Add missing field when creating a SubscriberClient.Settings from another.

Found when looking at #10358

If you see this in time, could you fix the typo at https://github.com/googleapis/google-cloud-dotnet/blob/main/apis/Google.Cloud.PubSub.V1/docs/history.md?plain=1#L148 as well?


Update sbt-converter to 1.0.0-beta42+1-f593f499-SNAPSHOT

About this PR
📦 Updates org.scalablytyped.converter:sbt-converter from 1.0.0-beta37+33-45b800b3-SNAPSHOT to 1.0.0-beta42+1-f593f499-SNAPSHOT
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scalablytyped.converter", artifactId = "sbt-converter" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.scalablytyped.converter", artifactId = "sbt-converter" }
}]

labels: sbt-plugin-update, early-semver-pre-release, semver-spec-pre-release, commit-count:1

Superseded by #195.


follow along: sizeof

maybe i did this wrong, but when i tried to compile the program as they lay it out in the video, I got errors saying:
error: format specifies type 'int' but the argument has type 'unsigned long'
I had to make all the '%i' into %li and then it compiled. Not sure if there's anything we can do but... ya might mess with some people

i added a warning


Add support for Node.js require

Is your feature request related to a problem? Please describe.
We're looking to use the API layer from this library for SIP signalling in Node.js, but the ESM exports can't be imported due to the browser specific implementation of import used by Node.js. From your FAQ it seems you're open to having support for other platforms.
We'll be overriding the SessionDescriptionHandlers and Transport for Node.js, so we'd just need the library to support require() for it to work in Node.js for what we're doing.
Describe the solution you'd like
Locally I've made the following changes (mimicking Redux's package.json), and it works on Node.js 14:

Removed "type": "module" from package.json
Added a tsconfig.json for commonjs which extends src/tsconfig.json
Added a npm run build-cjs command, and added this to the build script
Compiled CommonJS output to a cjs/ folder, and referenced this in the "main" property of package.json

Node can then use the common.js provided in "main", and bundlers will pickup the existing "module" property for ESM support.
This is the method used by D3, apollo-client, redux and others; I can clean it up and submit a PR.
Describe alternatives you've considered
Our app doesn't use a transpiler, and I'd rather avoid adding one to support this module, but we could fork the repo and transpile it ourselves to get around this.
Additional context
If you have a preferred way of implementing this, let me know and I can raise a PR.
If you're not interested in having Node.js support out of the box, let me know and we'll look into alternatives.

We currently run sip.js in browser and Node.js environments with no alterations. We have a custom Transport for UDP interfaces, but that is not a hack, SIP.js is built to support it.
I'm curious if there are other alternatives out there for tying together old and new. With Node 10 becoming EOL in a few months, all supported Node versions will work with ESM.

We currently run sip.js in browser and Node.js environments with no alterations. We have a custom Transport for UDP interfaces, but that is not a hack, SIP.js is built to support it.
I'm curious if there are other alternatives out there for tying together old and new. With Node 10 becoming EOL in a few months, all supported Node versions will work with ESM.

You may want to investigate this: https://www.npmjs.com/package/esm
We used it while transitioning forward, but it's no longer needed. It's a one line fix.

You may want to investigate this: https://www.npmjs.com/package/esm
We used it while transitioning forward, but it's no longer needed. It's a one line fix.

We're using Node 14, but the ESM implementation in Node.js uses URL resolution, so if I try to import the module, it throws the following error:
Error [ERR_UNSUPPORTED_DIR_IMPORT]: Directory import '/home/mark/git/node/sip-tests/node_modules/sip.js/lib/api' is not supported resolving ES modules imported from /home/mark/git/node/sip-tests/node_modules/sip.js/lib/index.js
https://nodejs.org/dist/latest-v14.x/docs/api/esm.html#esm_mandatory_file_extensions
Are you using the --es-module-specifier-resolution=node to run this without alterations?

We're using Node 14, but the ESM implementation in Node.js uses URL resolution, so if I try to import the module, it throws the following error:
Error [ERR_UNSUPPORTED_DIR_IMPORT]: Directory import '/home/mark/git/node/sip-tests/node_modules/sip.js/lib/api' is not supported resolving ES modules imported from /home/mark/git/node/sip-tests/node_modules/sip.js/lib/index.js
https://nodejs.org/dist/latest-v14.x/docs/api/esm.html#esm_mandatory_file_extensions
Are you using the --es-module-specifier-resolution=node to run this without alterations?

No, but we use --experimental-specifier-resolution=node.

No, but we use --experimental-specifier-resolution=node.

Thanks for clarifying, it'll work if you enable that experimental flag and then use ESM throughout your project, which I've just tried and I can confirm. Unfortunately we use the default CommonJS module syntax, so this doesn't work in our case.
The solution I listed at the top works with CommonJS and bundlers, though it'd break your own setup if you're using the experimental flag to treat "main" as an ESM module in Node.js, so that's a no go.
In practice you'd need to provide .cjs files to enable CommonJS support, which would need something like rollup.
If you're open to using rollup for packaging I can look into creating a PR, but if not we can look at solving this on our end in a forked repo instead.

Thanks for clarifying, it'll work if you enable that experimental flag and then use ESM throughout your project, which I've just tried and I can confirm. Unfortunately we use the default CommonJS module syntax, so this doesn't work in our case.
The solution I listed at the top works with CommonJS and bundlers, though it'd break your own setup if you're using the experimental flag to treat "main" as an ESM module in Node.js, so that's a no go.
In practice you'd need to provide .cjs files to enable CommonJS support, which would need something like rollup.
If you're open to using rollup for packaging I can look into creating a PR, but if not we can look at solving this on our end in a forked repo instead.

We do have a webpack'ed version of our repo that we attach to our release page here: https://github.com/onsip/SIP.js/releases
Every release has a .js and .min.js. If you are going to further bundle, using the non-minified is recommended. Would this fix the situation?

We do have a webpack'ed version of our repo that we attach to our release page here: https://github.com/onsip/SIP.js/releases
Every release has a .js and .min.js. If you are going to further bundle, using the non-minified is recommended. Would this fix the situation?

Closing this as I hope the webpack version provides a reasonable workaround.

Closing this as I hope the webpack version provides a reasonable workaround.


mapValueObject on XML with multiple child nodes of same type

Hi,
I'm having a struggle with reading a XML and structuring it with mapValueObject.
I have the following xml structure. As you can see a JobSet has multiple Job nodes.
<JobList>
   <JobSet>
         <Job></Job>
         <Job></Job>
         <Job></Job>
    </JobSet>
</JobList>

$service->mapValueObject('{http://webservice.talentfinder.be}JobList', JobList::class);
$service->mapValueObject('{http://webservice.talentfinder.be}JobSet', JobSet::class);
$service->mapValueObject('{http://webservice.talentfinder.be}Job', Job::class);

It gets mapped correctly but if I inspect my php array I've got just one Job object in JobSet, instead of a bunch of them. Actually only the last is in the array. Probably this node gets overwritten while looping the XML?
Any idea what I could do to overcome this?
Thanks!

Can you share the JobSet class?

Great!


Broken aggregate function in main branch

What happened + What you expected to happen
If I use hierarchicalforecast v0.3.0 there's a problem with "unbalanced" time series, see: https://github.com/Nixtla/hierarchicalforecast/issues/189
However, if you exclude this specific issue, it works fine.
If I use the latest version from the main branch (maybe after this commit? https://github.com/Nixtla/hierarchicalforecast/commit/c1072171e074470e97d343e3fbb0188f68509f5d) I have huge issues.
I have unbalanced time series.
If I set the new "is_balanced" parameter to False (the default behavior), the returned dataframe has many missing dates (ds). For some unique_ids I expect many years of data but I just get 2 rows.
If I set the new "is_balanced" parameter to True, I get a reshaping error.
The same code works with version 0.3.0. I had to revert to the released 0.3.0 version because the version in the main branch is not usable for me.
Versions / Dependencies
hierarchicalforecast main/master branch
Python 3.10
Linux
pandas 2.0.2
Reproduction script
Y_df, S_df, tags = aggregate(Y_df, spec)
Unfortunately, I can't share my data set.
Issue Severity
None

Just to add some context:
I have time series with the same ending date but different starting dates.
I think it's ok for reconciliation to fill missing dates with zeros.
However, for model training data I don't want to add leading zeros to my time series.

Doesn't aggregate have a is_balanced parameter not a is_unbalanced? The function assumes the data is not balanced by default.

Doesn't aggregate have a is_balanced parameter not a is_unbalanced? The function assumes the data is not balanced by default.

Yes I misspelled in the message above. The parameter is called is_balanced and the default value is False. I edited the message above. The issue is the same.

I was looking into the source code for new version and old version
here I find some the order of code logic
new version with sparse (say snippet1 )
"    Y_bottom_df = Y_bottom_df.groupby(['unique_id', 'ds'])['y'].sum().reset_index()
Y_bottom_df.unique_id = Y_bottom_df.unique_id.astype('category')
Y_bottom_df.unique_id = Y_bottom_df.unique_id.cat.set_categories(S_df.columns)
"
here we are aggregating first and then getting column names as index for Y_bottom_df.
but is old version (say snippet2 )
"    Y_bottom_df.unique_id = Y_bottom_df.unique_id.astype('category')
Y_bottom_df.unique_id = Y_bottom_df.unique_id.cat.set_categories(S_df.columns)
Y_bottom_df = Y_bottom_df.groupby(['unique_id', 'ds'])['y'].sum().reset_index()
"
we were first creating all the unique_id's first that is helping us to get the complete time series for all the unique_id's
this small change has resolved this issue for me
just change (say snippet1 ) with (say snippet2 ) in new updated version
thanks


Filtering findings for generating reports with multiple accounts

While using this tool to generate a combined report for a number of accounts I would get this error (while using all and listing accounts manually):
jshodd > cloudmapper > python cloudmapper.py report --accounts product,test
* Getting resource counts
  - product
  - test
* Getting IAM data
  - product
  - test
* Getting public resource data
  - product
  - test
* Auditing accounts
Traceback (most recent call last):
  File "cloudmapper.py", line 72, in <module>
    main()
  File "cloudmapper.py", line 66, in main
    commands[command].run(arguments)
  File "/Users/jshodd/opt/tools/cloudmapper/commands/report.py", line 454, in run
    report(accounts, config, args)
  File "/Users/jshodd/opt/tools/cloudmapper/commands/report.py", line 322, in report
    count = findings_severity_by_account[finding.account_name][
KeyError: 'test'

Looking at the code It seems that the lsit findings contains a list of findings for all accounts being reported on, but we are looping over that list for each account and adding findings to findings_severity_by_account with the key being the account name on the finding. The issue there is that the account name key is only being added as the loop iterates over each account. So on the first account that gets iterated, it creates a key in findings_severity_by_account with it's account name and then loops over the findings. The findings loop adds findings to that dict using finding.acount_name which works fine for the first x number of findings, because they are all for the same first account, but then that loop encounters a finding where finding.account_name is equal to the next account and tries to do the following:
            count = findings_severity_by_account[finding.account_name][
                conf["severity"]
            ].get(finding.issue_id, 0)

but at this time, the second account has not been looped over yet, so findings_severity_by_account[finding.account_name] doesn't exist yet. My change uses a list comprehension to filter down the list findings so that we are only looping over a list of findings for that specific account so that this key error does not occur. After this change the operation works as expected:
 jshodd > cloudmapper > python cloudmapper.py report --accounts product,test
* Getting resource counts
  - product
  - test
* Getting IAM data
  - product
  - test
* Getting public resource data
  - product
  - test
* Auditing accounts
Report written to web/account-data/report.html

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@jshodd Thank you! I just noticed this issue last night and fixed it locally, but your fix is cleaner.  Nice work!


SPFx webparts fail when showing older version in page history

Target SharePoint environment
SharePoint Online
What SharePoint development model, framework, SDK or API is this about?
💥 SharePoint Framework
Describe the bug / error
When showing an item from the version history of a page in SharePoint Online, our custom webparts don't load anymore, and they throw an error:

ERROR:
Cannot read properties of undefined (reading 'webPartTag')
CALL STACK:
TypeError: Cannot read properties of undefined (reading 'webPartTag')
at https://res-1.cdn.office.net/files/sp-client/sp-pages-assembly_en-us_49d37ded9db4e12f19a385ada4e9abe3.js:70:1381319

Steps to reproduce

Create a SharePoint page.
Add a custom SPFx webpart to the page
Publish the page
Edit the page, and republish
Show the "old" version from the Version History

Expected behavior
The webparts should be loaded properly when opening an older version from the page.

@AJIXuMuK is anyone looking into this? Or is there anything you might share time-wise?

@NickSevens - sorry for the long delay.
I was trying to repro it multiple times and all seems to be working fine.

Could you please try repro it again? If that's still the case for you with the steps you provided above - is it possible to share the tenant/test account with me? You can use<EMAIL_ADDRESS>to share it.

@AJIXuMuK I've relayed the issue to my colleagues who are currently working for the customer. They will contact you and send you the necessary details.

@AJIXuMuK I've provided you the details (via email) regarding the login to the customer's tenant. Thank you for your time regarding this issue.

I was unable to repro it even on the shared tenant.
Closing for now.


Proposal: vendor restore --missing, --retry, and --continue options

A flaky internet connection has left me wanting three additional (related) options for the gb plugin vendor-restore:

--missing: only fetch vendor packages not present. By default everything is fetched, which makes sense given the vendor/src directories are but copies of the cloned working-copy, so they won't have revision information inside them, which makes an npm update equivalent statement impossible. However this option could work if:

the vendor's package src exists, assume it's the correct one (a bit crap, and possibly dangerous if always relied upon, but a stepping stone)
or the working copy's revision was somehow stored (a .gb/HEAD file anyone? :sob: or better leverage #316) it could check if the revision matches the manifest before restoring (although this would probably be better as the default behaviour, without options)

--retry n: should the first attempt fail (connection timeout, DNS resolution, etc), retry n times.
--continue: should fetching a vendor's package fail, carry on with the next.

Currently post pulling in a new revision/branch I run gb vendor restore and when my ISP is wigging out, I haven't been able to complete a full restore of more than half a dozen (or the 41) packages.
I know there has been plenty said about gb vendor restore (#175, #271) but the inclusion of #281 suggests if you're ideologically inclined then have a go with it.
Sure, I could use git subtree/submodule, and I'm not against this, but this is a manual task I will have to grapple separately, which doesn't leverage things like the vendor/manifest and gb vendor purge.
Caveat: I'm no veteran to gb, I've been using it on a converted project of mine for about a week, so maybe I'm missing a trick!

Thanks for raising this issue. I don't plan on adding these flags. If your
upstream is unstable then you should be vendoring those packages and
checking them into source control.
On Thu, 17 Dec 2015, 01:39 Steve Craig<EMAIL_ADDRESS>wrote:

A flaky internet connection has left me wanting three additional (related)
options for the gb plugin vendor-restore:

--missing: only fetch vendor packages not present. By default
everything is fetched, which makes sense given the vendor/src
directories are but copies of the cloned working-copy, so they won't have
revision information inside them, which makes an npm update equivalent
statement impossible. However this option could work if:

the vendor's package src exists, assume it's the correct one (a
bit crap, and possibly dangerous if always relied upon, but a stepping
stone)
or the working copy's revision was somehow stored (a .gb/HEAD
file anyone? [image: :sob:] or better leverage #316
https://github.com/constabulary/gb/issues/316) it could check if
the revision matches the manifest before restoring (although this
would probably be better as the default behaviour, without options)

--retry n: should the first attempt fail (connection timeout, DNS
resolution, etc), retry n times.
--continue: should fetching a vendor's package fail, carry on with
the next.

Currently post pulling in a new revision/branch I run gb vendor restore
and when my ISP is wigging out, I haven't been able to complete a full
restore of more than half a dozen (or the 41) packages.
I know there has been plenty said about gb vendor restore (#175
https://github.com/constabulary/gb/issues/175, #271
https://github.com/constabulary/gb/issues/271) but the inclusion of #281
https://github.com/constabulary/gb/pull/281 suggests if you're
ideologically inclined then have a go with it.
Sure, I could use git subtree/submodule, and I'm not against this, but
this is a manual task I will have to grapple separately, which doesn't
leverage things like the src/manifest and gb vendor purge.
Caveat: I'm no veteran to gb, I've been using it on a converted project
of mine for about a week, so maybe I'm missing a trick!
—
Reply to this email directly or view it on GitHub
https://github.com/constabulary/gb/issues/498.

@davecheney does not planning also mean not opposed? (Am I reading what I want to read?)
If so, happy to take a look.
Completely understand this is a roundabout solution to essentially upstream not being stable, and there are existing, precise solutions for that. Having said that, gb vendor restore exists, and these might be some nice tools for that toolbox.

I'd prefer not to add those features. Vendor restore was a mistake, I should have pushed back harder at the time.
If you don't want to vendor your code, that's fine, I also think vendoring is a crude solution, and am working on something much better, have a look at issue 12302, my comments towards the end of the issue. That's what I propose instead.

On 17 Dec 2015, at 22:28, Steve Craig<EMAIL_ADDRESS>wrote:
@davecheney does not planning also mean not opposed? (Am I reading what I want to read?)
If so, happy to take a look.
Completely understand this is a roundabout solution to essentially upstream not being stable, and there are existing, precise solutions for that. Having said that, gb vendor restore exists, and these might be some nice tools for that toolbox.
—
Reply to this email directly or view it on GitHub.

OK no worries. Glad to get a resolution on gb vendor restore at least!
Issue 12302?

@steevee I guess @davecheney refers to this: https://github.com/golang/go/issues/12302

@ngrilly cheers I found it but lost myself in the unfolding novel :smile:
For completeness I also found #350 which is very similar to this.
Appreciate vendor restore was a mistep, and presume @davecheney was referring to this comment, but perhaps he can clarify?
In the meantime, I'm going to vendor with git submodules/subtree. A method to leverage the manfiest (for my 41 external packages) would make a hell of a Christmas gift!

Yup, that's the comment.
Steve, why not use go vendor fetch ? You don't have to use th vendor at all, but if you do, this is the way the plugin was designed to work.

On 18 Dec 2015, at 01:38, Steve Craig<EMAIL_ADDRESS>wrote:
@ngrilly cheers I found it but lost myself in the unfolding novel
For completeness I also found #350 which is very similar to this.
Appreciate vendor restore was a mistep, and presume @davecheney was referring to this comment, but perhaps he can clarify?
In the meantime, I'm going to vendor with git submodules/subtree. A method to leverage the manfiest (for my 41 external packages) would make a hell of a Christmas gift!
—
Reply to this email directly or view it on GitHub.

@davecheney sorry if I'm missing your point, but I understand vendor fetch (the go get equivalent) fetches one src at a time, so I was using vendor restore to process the manifest in bulk.
My /vendor/manifest is committed and has dozens of dependencies, so I'm looking for ways to make management easier.
Vendoring in dependencies via git submodule/subtree is OK, or simply using manifest like I use package.json appeals even more to me. However, I'd like to see more tools in the gb vendor plugin shed to help either git submobule/tree creation, or manfest restoration.


[VL][CI] Use pre-installed celeborn to avoid download failure

Celeborn 0.3.2 & 0.4.0 have been pre-installed in CI docker.

LGTM. Thanks!


Module '"sip.js"' has no exported member 'Session'.

Prior to now, I've been able to use these properties:
{ Session, SessionInviteOptions, SessionState, UserAgent, UserAgentOptions, UserAgentState }
But installing sip.js today and there's an error showing no export member for those.
And another thing I observed is that the version for this is still the same as my old project.  "sip.js": "^0.20.0"

This is almost certainly a project configuration issue, particularly given that you have the same version for both (please note how you have the version declared does not guarantee they will be the same, but that happens to be the latest version). I'd compare your build configurations, this may be an ES6 or ES module issue.


When using this permission, the backing application of the service principal being created must in the local tenant

Error seen on an az aro create - no further context currently.
When using this permission, the backing application of the service principal being created must in the local tenant
Possibly need to catch and retry this if it happens more.

I think this must happen in def create_service_principal(self, app_id) - need a retry here

most of az just catches all exceptions on this call and does a sleep/retry loop here

I did hit it 3 times today. Here relevant output of az aro create with --debug:
msrest.http_logger : Request URL: 'https://graph.windows.net/7a1815fb-63ba-4af7-8f33-e7333e421980/servicePrincipals?api-version=1.6'
msrest.http_logger : Request method: 'POST'
msrest.http_logger : Request headers:
msrest.http_logger :     'Accept': 'application/json'
msrest.http_logger :     'Content-Type': 'application/json; charset=utf-8'
msrest.http_logger :     'accept-language': 'en-US'
msrest.http_logger :     'Content-Length': '49'
msrest.http_logger :     'User-Agent': 'python/3.8.2 (macOS-10.15.4-x86_64-i386-64bit) msrest/0.6.13 msrest_azure/0.6.3 azure-graphrbac/0.60.0 Azure-SDK-For-Python AZURECLI/2.5.1 (HOMEBREW)'
msrest.http_logger : Request body:
msrest.http_logger : {"appId": "17eb8791-1a44-4428-b350-94861a804d18"}
msrest.universal_http : Configuring redirects: allow=True, max=30
msrest.universal_http : Configuring request: timeout=100, verify=True, cert=None
msrest.universal_http : Configuring proxies: ''
msrest.universal_http : Evaluate proxies against ENV settings: True
urllib3.connectionpool : Starting new HTTPS connection (1): graph.windows.net:443
urllib3.connectionpool : https://graph.windows.net:443 "POST /7a1815fb-63ba-4af7-8f33-e7333e421980/servicePrincipals?api-version=1.6 HTTP/1.1" 403 284
msrest.http_logger : Response status: 403
msrest.http_logger : Response headers:
msrest.http_logger :     'Cache-Control': 'no-cache'
msrest.http_logger :     'Pragma': 'no-cache'
msrest.http_logger :     'Content-Type': 'application/json; odata=minimalmetadata; streaming=true; charset=utf-8'
msrest.http_logger :     'Expires': '-1'
msrest.http_logger :     'ocp-aad-diagnostics-server-name': 'oMHfAEIp1gHYFHdJ6hb0QGsjL3oM+bEEQZMKernWW0M='
msrest.http_logger :     'request-id': '0e6defe8-65c8-48d5-abfa-990bc8a1ca6a'
msrest.http_logger :     'client-request-id': '82d1f0c0-9514-11ea-83f5-38f9d3592049'
msrest.http_logger :     'x-ms-dirapi-data-contract-version': '1.6'
msrest.http_logger :     'ocp-aad-session-key': 'REDACTED'
msrest.http_logger :     'DataServiceVersion': '3.0;'
msrest.http_logger :     'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
msrest.http_logger :     'Access-Control-Allow-Origin': '*'
msrest.http_logger :     'X-AspNet-Version': '4.0.30319'
msrest.http_logger :     'X-Powered-By': 'ASP.NET'
msrest.http_logger :     'Duration': '2120017'
msrest.http_logger :     'Date': 'Wed, 13 May 2020 12:23:16 GMT'
msrest.http_logger :     'Content-Length': '284'
msrest.http_logger : Response content:
msrest.http_logger : {"odata.error":{"code":"Authorization_RequestDenied","message":{"lang":"en","value":"When using this permission, the backing application of the service principal being created must in the local tenant"},"requestId":"0e6defe8-65c8-48d5-abfa-990bc8a1ca6a","date":"2020-05-13T12:23:16"}}
msrest.exceptions : When using this permission, the backing application of the service principal being created must in the local tenant
cli.azure.cli.core.util : When using this permission, the backing application of the service principal being created must in the local tenant
When using this permission, the backing application of the service principal being created must in the local tenant
Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x111589ca0>]
az_command_data_logger : exit code: 1
telemetry.save : Save telemetry record of length 2629 in cache
telemetry.check : Negative: The /Users/mradchuk/.azure/telemetry.txt was modified at 2020-05-13 13:17:01.260508, which in less than 600.000000 s
command ran in 2.844 seconds.

@jim-minter Hi Jim, I encoutered the same issue while creating sp after creating an application, may i ask how didi you resolve it?
Much appreciate!
Maggie

Hi @liuke0712,
I believe that this issue was was fixed in Azure Red Hat OpenShift command #631 by adding retries to handle AAD sync delays.
If you still experience something similar while creating an Azure Red Hat OpenShift I encourage you to reach out to Microsoft support: it will enable us to investigate this further.
Kind regards,
Mikalai


Target Deployments from All Branches

In order to run the CI tool, we should provide a very good and intuitive readme.

[x] Provide an example of running the CI action against a deployment that is triggered by a merge request
[x] Provide an example of running the CI action against a pull request
[x] Provide an example of using GitHub to compute the preview URL's for common tools:

[x] Vercel
[x] Netlify
[x] Custom configuration of the URL with parameters from a commit, or deployment

Background
There are some useful tools that show how you can access the GitHub refs for deployment here: https://docs.github.com/en/free-pro-team@latest/actions/reference/events-that-trigger-workflows#deployment
Further, there is an API document that is in use for sending deployment information to GitHub: https://docs.github.com/en/free-pro-team@latest/rest/reference/repos#create-a-deployment-status
It would be ideal if the deployment would send the URL, and it was available in the GitHub action, having been triggered by the deployment e.g.
on:
  deployment:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Mozilla Observatory Github Action
        id: observatory
        uses<EMAIL_ADDRESS>        with:
          web_host: ${{github.deployment.environment_url}}
      - name: Create commit comment
        uses: peter-evans/commit-comment@v1
        with:
          body: ${{ steps.observatory.outputs.observatory-report }}

This is possible by accessing the event on the GitHub context. It will contain the parameters of the event which has triggered the workflow:
https://docs.github.com/en/free-pro-team@latest/actions/reference/context-and-expression-syntax-for-github-actions#github-context

closed by #73

Closed by #73


Feature: Add safe DetoursAttach (and friends) overloads

Fixes #176
I've also added a sample (a copy of the simple sample, but without the (PVOID&) casts) to validate the functionality.
Microsoft Reviewers: Open in CodeFlow

Would you mind writing some documentation for these new APIs?
Like: https://github.com/microsoft/Detours/wiki/Reference#apis-for-inserting-dlls-and-payloads-into-new-processes

@msftbot make sure @virtuald gets to approve

@msftbot forget everything I just told you

@sylveon sigh... msftbot went crazy :) would you mind addressing the feedback in a follow up PR?

As for the docs... I can't edit the wiki but I don't think making a new page is required, would be better to just add a second declaration or a note to the existing pages for DetourAttach and friends to make the overload known


How to disable the resize images?

How to disable the resize images? I want to view the real images not the resize images. Thanks

the demo use the picasso to resize the image in BitmapPlayerActivity.java
Picasso.with(getApplicationContext())
                .load(uri)
                .resize(callback.getMaxTextureSize(),callback.getMaxTextureSize())
                .onlyScaleDown()
                .centerInside()
                .memoryPolicy(NO_CACHE, NO_STORE)
                .into(mTarget);

And it has the difference max texture size limited according to the device.

Yes.... So what should I do?
On Aug 7, 2017 09:27, "Asha"<EMAIL_ADDRESS>wrote:

the demo use the picasso to resize the image in BitmapPlayerActivity.java
Picasso.with(getApplicationContext())
.load(uri)
.resize(callback.getMaxTextureSize(),callback.getMaxTextureSize())
.onlyScaleDown()
.centerInside()
.memoryPolicy(NO_CACHE, NO_STORE)
.into(mTarget);
And it has the difference max texture size limited according to the device.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ashqal/MD360Player4Android/issues/179#issuecomment-320553824,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AG3ddojUQ8bEfOwB2CirGxO_CtXTpWAgks5sVnYggaJpZM4OtXUA
.


Reshape all polygons to be the intersection of the polygon and our data domain

We keep encountering polygons that fail in the app because there's no data.  Instead of fixing this on an ad-hoc basis, I suggest that we reshape every polygon in the app to be the intersection of the polygon and our data domain (we can intersect with the IEM domain even though we know that won't include permafrost everywhere).
Xref #91

That makes sense - so like this example using the native corporations? Black lines are the original and the colored polygons are the intersection with the "IEM with AK Aleutians" domain.

I recommend closing this issue - our data domain is nebulous and ever-changing and full of odd gaps and holes. We have more sophisticated ways to deal with no-data in certain areas now.


Learning completion tracking for organisations

Great content on MS for skilling up my IT teams on Azure! But whats stopping me from widespread adoption is the ability to track what staff in my org have completed what modules? I heard its coming, can I have a ETA please, otherwise will need to look to use a paid training alternative which does this.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 26c7d914-e786-d705-977d-4f5fe050f420
Version Independent ID: 90e0d126-c3ff-00ec-b910-528c64a8c9db
Content: Introducing Microsoft.com/Learn
Content Source: teamblog/introducing-ms-learn.md
Product: team-blog
GitHub Login: @jeffsand
Microsoft Alias: jeffsand

@mbedford78 Thanks for taking the time to leave your blog feedback.
I've assigned your issue to a member of our Learn team for answers and a roadmap.
In the future, make sure to leave product or path feedback on the product page to ensure the fastest response possible.
Appreciate your contributions to improving the Microsoft docs community.


Auto-exposure fails to readjust after reaching min exposure unless sensor is covered

Required Info

Camera Model
D435

Firmware Version
<IP_ADDRESS>

Operating System & Version
Ubuntu 18.04

Kernel Version (Linux Only)
(e.g. 4.14.13)

Platform
PC/Raspberry Pi/ NVIDIA Jetson / etc..

SDK Version
2.28.0

Language
C++

Segment
Robot

Issue Description
When I expose the D435 RealSense to direct sunlight with autoexposure enabled, it correctly drops the gain and exposure of the camera to min settings; however, if the sunlight is removed, the camera does not adjust away from min settings unless it is placed in total darkness. If placed in reduced light, such as a closet away from any direct sun exposure, it is unable to update the exposure and gain to proper settings, but if I place my hand over the sensor for several seconds, it is able to adjust back to normal behavior and produces reasonable gain and exposure settings for the current environment.
Here are images of the sequence; see left terminal and ROS image:
Before min exposure

Min exposure from sun

Still at min exposure though away from sun

Adjusting from min exposure to max exposure due to hand covering (note changes in left terminal)

Proper exposure after hand covered sensor

I wonder if you may get better results if you use a feature called fractional exposure.
https://github.com/IntelRealSense/librealsense/issues/2875#issuecomment-446788177

Thanks, do you have more documentation on how to enable that feature?
No worries on timing, it's dark here anyway and the sun won't be up for another 12 hours.

My reading of the information provided by agrunnet is that there are two approaches to using fractional exposure.

Turn off auto-exposure and manually set the exposure to a lower value, which can go as low as '1'.

Or alternatively, 2.  Have auto-exposure enabled and set a Region of Interest (ROI). In the RealSense Viewer it can be done by starting the stream, clicking the 'Set ROI' button next to the "Enable Auto Exposure" option and dragging on the image to set a ROI.

None of those options lead to desirable behavior. I do want auto-exposure enabled, as I will be running in both low light and saturated light environments without prior knowledge of the transitions, and I do want the ROI to be the entire image. I am trying to use the IR image + its gain and auto-exposure from the RealSense to predict when the background IR will saturate other IR based sensors, as well as use the RealSense to produce a depth image.
To be blunt, I'm pretty sure the fact that the auto-exposure doesn't readjust is either a bug in the auto-exposure firmware code (I suspect this is the case) or there's something in hardware that puts the RealSense in a weird mode to protect the sensors which needs to be better documented so I can figure out how to avoid it.

There was a known issue earlier this year where the IR could freeze if the camera was directed at outdoor bright light such as the sun.  It was more likely to occur when USB 2.0 was being used.
https://github.com/IntelRealSense/librealsense/issues/4140
The most recent firmware release notes at the time of writing this (August 2019) say that a glitch referred to as "Auto-exposure (AE) for Color is not optimized in bright sunlight" is not fixed yet.  The issue's code name is DSO-10011,

Are auto-exposure for color and auto-exposure for IR controlled by the same system? If so, I can see that being the root cause.

Regarding #4140, the IR stream isn't freezing; it's just not updating its auto-exposure or gain when placed in a lower light environment.

I was sure I had seen a case similar to yours where covering the camera up resets it after it stops updating.  I just managed to finally track the link down.
https://support.intelrealsense.com/hc/en-us/community/posts/360033445713-D435-Sunlight-saturation-problem

Thank you very much for tracking that link down. Sounds like that the new firmware version will fix this issue. Do you know when that version of the firmware will be released or, if it's already released, how to get it?

The firmware version is a point of minor confusion.  When an Intel support member said it had been fixed in a firmware (<IP_ADDRESS>), this was apparently an internal firmware at Intel and a final firmware release had not been made to the public at the time that the statement was made.
There has not been a firmware 5.11.12 yet though.  The most recent release was <IP_ADDRESS>, the one that you are already using.  According to the developer notes for the next release of the SDK, the recommended firmware version will become 5.11.15, which is not on the public firmware download page yet.
https://github.com/IntelRealSense/librealsense/commit/c3102f5f82087766d3edb562ceeb6343f2900b09
I am not an Intel employee, so I do not have knowledge of whether the auto-exposure fix was put in <IP_ADDRESS> or will be in 5.11.15.  Hopefully a member of the Intel support team on this forum can clarify this.

Here are some additional inputs that may help. From the screenshot, it looks like you may be running in USB 2.1 mode. There was an issue with fractional row exposure control settings. We updated the settings from OVT in firmware <IP_ADDRESS> to correct the issue. More details: https://github.com/IntelRealSense/librealsense/issues/4140. Or make sure to use USB3.
In any event, if you update to firmware <IP_ADDRESS> or later versions, like @MartyG-RealSense  says, the AE should work properly in bright sunlight.

@agrunnet I had the device plugged into a port on my computer that supports USB3

However, I have noticed on several occasions that the realsense-viewer will report being connected to a USB2 port despite being connected to the exact same physical port; unplugging and replugging the cable from the RealSense and the computer seems to succesfully make the realsense-viewer report it is once again connected to a USB3 port. Do you have any insight into what is happening in that case?
Also, why are there significant differences in the behavior of the D435 by being plugged into a USB2 vs USB3 port? My understanding of the differences between USB2 and USB3 were with respect to throughput data rate; is there differences in power modes as well that's causing varying performance of the sensor? This is important to us as we would like to deploy these on mobile robots and if something non-transparently changes sensor performance (e.g. it decided to use USB2 instead of USB3 on a USB3 port), it's something we need to account for.

As a follow up to my above comment where I took the USB3.2 screenshot, I took the same setup, moved the cable around while ensuring both ends remained plugged into the computer and reopened realsense-viewer and it stated that I was connected to a USB2.1 port.

Intel give some technical insights into operation in USB2 mode in the blog post linked below.
https://www.intelrealsense.com/usb2-support-for-intel-realsense-technology/

In regard to the USB2 mis-detection problem, the camera may mis-detect due to the way it is inserted, with slow insertion increasing the chance of mis-detection and a quick and firm insertion reducing it.  I have also heard that if the cable starts to withdraw from the USB port (e.g accidentally jostling out), that may also trigger a USB2 mis-detection.
If you can confirm that inserting the cable quickly and firmly increases the chances of the camera being detected correctly, this may justify using a USB cable with screw-locks on the connector to keep the connector firmly in the USB port.  I once handled a case where a RealSense user with a mobile robot was having the camera disconnected due to the robot's motion, and I recommended a screw-lock cable in that case.

@MartyG-RealSense thanks for the screwlocks tip. That will hopefully solve the USB3 => USB2 problem. What is the best way to detect what mode (USB2 vs USB3) the RealSense is running in via the librealsense API? I failed to find anything but I might just be missing something.

rs-enumate-devices is a program that can retrieve information about connected devices.
https://github.com/IntelRealSense/librealsense/tree/master/tools/enumerate-devices
The sensor-control example can similarly get information about connected devices.
https://github.com/IntelRealSense/librealsense/tree/master/examples/sensor-control
In a simpler example, a RealSense user got the current USB detection state of a connected device by searching for it via the model 'PID' ID number of that particular model (a T265 in this example's case).
https://github.com/IntelRealSense/librealsense/issues/4331

As a followup to @agrunnet and @MartyG-RealSense, I tested the RealSense again in direct sunlight while in USB3 mode by being very careful with the wires (I also confirmed the device was in USB3 with realsense-viewer before and after this experiment took place). It correctly dropped to 1 exposure and 16 gain when I pointed the RealSense at the sun, and it then correctly increased the exposure when I turned it to face slightly away from direct sunlight.
Direct sun:

Slightly away from direct sun:

The critical piece here seems to be running in USB3 mode, so thank you so very much for that insight, I deeply appreciate it.
@MartyG-RealSense  Thank you for the links on USB3 vs USB2 detection. Do you know if there's a way to go so far as to disable USB2 mode and have the RealSense only try to maintain a USB3 connection? We're planning on deploying these sensors on robots for many hours at a time and I am worried we might have a problem of them constantly dropping to USB2 which seems to require a power cycle to restart; it would be ideal if the RealSense would just retry for a USB3 connection.

I haven't heard of a specific mode to disable USB2 in Librealsense.  I wonder if you could do a check for whether USB2 is detected and if it is, initiate a hardware_reset instruction to automatically reset the camera from within the script until USB3 is detected and the reset loop is halted until the next time that USB2 is detected.  Here's an example code snippet for hardware_reset:
https://github.com/IntelRealSense/librealsense/issues/2161#issuecomment-408779150

@MartyG-RealSense thank you for that snippit. I was worried I was going to have to figure out how to  kill power to the USB port then power it again.
If you find a way to disable USB2 mode, please let me know, but for now I think cable screws + power cycling to keep the camera in USB3 mode is a hack to get me off the ground.
Finally, @MartyG-RealSense I want to thank you for all your help in this issue, you've been extremely responsive and I am very impressed with your quality of support.

@kylevedder   A new SDK version is available now.
https://support.intelrealsense.com/hc/en-us/community/posts/360035827554

Hi @MartyG-RealSense - I'm tracking the same AE problem raised here and in #4140 with the limitation that I'm constrained to USB 2.0 only.
The internal support rep identified 5.11.12 as the fix version and now 5.11.15 has come out, but I can't see any mention of this ticket nor #4140 mentioned anywhere in the release notes.
Do you know if this issue has been addressed in the latest 5.11.12 firmware?
Thanks

The firmware release notes for <IP_ADDRESS> have been published but do not include auto exposure.
https://support.intelrealsense.com/hc/en-us/community/posts/360036665993-Real-sense-firmware-release-notes-updated-to-5-11-15-0

Thanks @MartyG-RealSense & @RealSenseCustomerSupport
We would still like to see this issue get fixed for USB 2.0 mode because our hardware constraints prevent us from being able to use USB 3.0. If the fix was indeed made in firmware version <IP_ADDRESS> then it wasn't pulled into the .15.0 release, so it should be a relatively simple change, no?
How would you suggest we proceed? Should we have this ticket re-opened or request to re-open #4140, or something else?
Thank you.

Just for reference, I pulled up the link to the mention of the auto exposure fix in <IP_ADDRESS>.
https://github.com/IntelRealSense/librealsense/issues/4140#issuecomment-507410089
As you didn't create this discussion thread and wouldn't be able to open it again, you could create a new issue and link to this one.


file: support ADV files in (File).SegmentFile(..)

Issue: https://github.com/moov-io/ach/issues/580

Codecov Report

Merging #582 into master will decrease coverage by 0.02%.
The diff coverage is 86.66%.

@@            Coverage Diff             @@
##           master     #582      +/-   ##
==========================================
- Coverage    90.7%   90.67%   -0.03%
==========================================
  Files          64       64
  Lines        5248     5256       +8
==========================================
+ Hits         4760     4766       +6
- Misses        352      354       +2
  Partials      136      136

looks good!  I forgot I made constants for all the transaction codes.  I think I may like C or D better!


BevelFilter rotation defaults to 0° instead of 45°

The constructor ignores the rotation value, but setting it after initialization works: https://jsfiddle.net/3anhs0pw/
Also, to be consistent, I think rotation values should be in radians like in Pixi, while angle values would be in degrees.

This bug isn't there with Pixi 7 and Filters 5: https://jsfiddle.net/buo96kmv/
I haven't built it and made sure that this is the problem, but It looks like it's a degrees-to-radians conversion bug. rotation is multiplied by DEG_TO_RAD once when it's initialized: https://github.com/pixijs/filters/blob/ca0c344fb8ea32c1b7f855dc2a64a78b698130dd/src/bevel/BevelFilter.ts#L82
and then again when Object.assign calls the setter: https://github.com/pixijs/filters/blob/ca0c344fb8ea32c1b7f855dc2a64a78b698130dd/src/bevel/BevelFilter.ts#L125 https://github.com/pixijs/filters/blob/ca0c344fb8ea32c1b7f855dc2a64a78b698130dd/src/bevel/BevelFilter.ts#L133-L137

Good catch.


Fix typo on feature description for version 1.1.0

Very tiny fix :). But happy to contribute.

Thanks :)


How to convert a crossing edge into a vertex?

Hi! I'm a musician doing a musical investigation on the circle of fifth and I'm new to graphonline. I want to make new nodes from crossing edges. Is it possible to do that?
I want to be able to analyze all the intersections of a complete K12 graph as musical notes of the circle and convert them into new nodes.
Any suggestions would be greatly appreciated.

Thanks for your question. Unfortunately it is impossible to make it easy. You need to remove edges and add node and connect nodes again.
But we will try to add this functionality in the future.


Change lazy_static for once_cell

This PR changes our lazy_static dependency for once_cell as the latter's API is more likely to be included in std: https://github.com/rust-lang/rfcs/pull/2788

Seems good to me :+1:
No point pulling in a dependency if it's been moved to the standard library.

I'm fine with it can you include a cargo tree output to show if this adds new dependencies until its in stable std.

Never knew that was a thing you could do with cargo 🤔
TIL

me neither. But once_cell has an optional dependency on parking_lot:
https://docs.rs/once_cell/1.4.0/once_cell/#minimum-supported-rustc-version
which is only used when a certain feature is enabled.


Would it be better to have current nft minting fee in common file (@.common:data.KSM.fee) or something like that?

wouldn't it be better to have current nft minting fee in common file (@.common:data.KSM.fee) or something like that?
Originally posted by @roiLeo in https://github.com/kodadot/nft-gallery/pull/1399#pullrequestreview-825002527

@yangwao this can be closed as I additionally implemented it in #1399 👍

ah I see! make sense