Implement t-Digest

We would like to implement t-Digest based on the following paper:
Ted Dunning, Otmar Ertl. Extremely Accurate Quantiles Using t-Digests
and the following implementation in Java:
https://github.com/tdunning/t-digest

Also unclear why stable sort is needed

There are two questions here. One is about the case of updates with weight >1 and the other is about stable sorting.
The first issue is inherent in adding samples with greater than unit weight. Unless the digest can split that sample into multiple updates, the invariant can easily be violated. In the test in question, a single sample at 1 is given with weight two. Inserting this as is results in a single centroid with weight 2 which violates the invariant which says that all samples must have proper scaled weight or have weight 1. If we were allowed to split this into two samples at the same point, the invariant can be preserved. The problem with that is that we don't really know if this is two samples at the same value or two samples with that mean value. That might be resolved by documenting that it should not be used to enter mean values.
Regarding the stable sort, this is also required to avoid violating the invariant. Take the case where you have a digest generated from a bunch of samples at the same point. These samples will be gathered into centroids as the invariant allows, but all centroids created that way will have the same mean value. If you add a bunch more samples and (stably) sort the new samples into the old centroids things will be fine if the order of the old samples is preserved since you can add unit weight centroids anywhere you like without changing the invariant. On the other hand, if you use an unstable sort you can move the big centroids away from the center of the distribution and thus violate the invariant.
These sorts of things are the biggest reason for having so many tests that focus on repeated values.

I should add that since you appear to be using cpp for this new implementation, you can sort entire structures. That can radically simplify your work since you can use the standard sorting utilities instead of sorting indexes.

As I understand, we should not even provide a public add() method with weight.
Of course I am going to call std::sort or std::stable_sort. I am afraid I do not quite get the importance of stable sort and what exactly the invariant is.

The need for stability comes from the need to preserve the t-digest
invariant which is defined by a scale function.
The scale function is a function that maps q (the quantile) to k (a scaled
version. Scale functions look like kind of like this:
[image: image.png]
The t-digest invariant says that the k-size of any centroid must be less
than one except if the weight of the centroid is 1.
The k=size of a centroid is k(q_right) - k(q_left) where q_left is the
value of q on the left side of the centroid and q_right is the value on the
right. Because the scale function is steep near q=0 and q=1, the k-size of
a cluster with constant weight is bigger if it is near the ends rather than
near the middle. Defining the bound in terms of the scale function is what
constrains the centroid sizes so that interpolation near these extremes
will give good results. The invariant also bounds the size of the t-digest
for suitable scale functions (see https://arxiv.org/pdf/1903.09921.pdf for
details).
This is all very simple when all of the samples that we use to create the
t-digest are distinct because that means all of the centroids will have
different means and sorting will work predictably.
But what if we have 1000 samples all at the same point? The sizes of the
centroids in the resulting t-digest near the center will be bigger than the
ones at the edge. That's good. But if we sort those centroids purely by
their means (which are all the same) with an unstable sort, they could get
all jumbled. That could put the biggest centroids near the edge where their
k-size would be much larger than it was near the center.
Does this make sense?
On Tue, Dec 12, 2023 at 10:42 AM Alexander Saydakov <
@.***> wrote:

As I understand, we should not even provide a public add() method with
weight.
Of course I am going to call std::sort or std::stable_sort. I am afraid I
do not quite get the importance of stable sort and what exactly the
invariant is.
—
Reply to this email directly, view it on GitHub
https://github.com/apache/datasketches-cpp/issues/409#issuecomment-1852608805,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAB5E6RBVBVGDRBCSZ6WVFDYJCQR7AVCNFSM6AAAAABANDLXNOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJSGYYDQOBQGU
.
You are receiving this because you commented.Message ID:
@.***>

Yes, thank you very much.

Those look pretty sweet!
Latency shows a nice hard upper bound and accuracy looks identical to the java reference implementation.
I should get you to finish my Julia implementation!

I checked in the code as a branch here: https://github.com/apache/datasketches-cpp/tree/tdigest
Not done yet: get_quantile(rank) method, merge(other) method
This is equivalent to MergingDigest with the default K_2 scaling function.
I am not sure whether we want other flavors and scaling functions.
Also I don't quite understand a couple of expressions in get_rank() (cdf() in Java) marked with "// ?"

Also I am not sure that the buffer needs to have weights since the buffered input values have weight of 1. Perhaps it is a tradeoff to avoid allocations during compressions (and, perhaps, merging t-digests, which I did not implement yet)

The buffer has weights to increase the symmetry in the code (everything has
weights). It makes every kind of sorting and merging look the same.
On Tue, Dec 19, 2023 at 12:57 PM Alexander Saydakov <
@.***> wrote:

Also I am not sure that the buffer needs to have weights since the
buffered input values have weight of 1. Perhaps it is a tradeoff to avoid
allocations during compressions (and, perhaps, merging t-digests, which I
did not implement yet)
—
Reply to this email directly, view it on GitHub
https://github.com/apache/datasketches-cpp/issues/409#issuecomment-1863461948,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAB5E6XGPMGOGG4O4AR4HG3YKH5UXAVCNFSM6AAAAABANDLXNOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRTGQ3DCOJUHA
.
You are receiving this because you commented.Message ID:
@.***>

characterization code:
https://github.com/apache/datasketches-characterization/pull/60

I am looking at the code to merge t-digests. I don't like that a modifying operation compress() is called on the input t-digests. I wonder whether this is avoidable. I would think this is done for simplicity to avoid dealing with unmerged buffer in the incoming t-digests, however the compress() method forces compression even if the buffer is empty.
There is a practice in C++ to pass const reference to the input object. Having a requirement of a mutable input is quite ugly. And forcing compression unnecessarily is even worse. I am not sure how to deal with this.

The compression on the inputs is not necessary. It may have a positive
effect on accuracy in a few cases, but I strongly doubt the effect will be
important. In practice, most of the digests being merged will previously
have been persisted and thus will have already been compressed.
Just allocate a buffer big enough for all the live centroids in all of the
merging digests, copy in the centroids and sort/compress.  A final copy to
a trimmed buffer is a nice touch, but only necessary if the intermediate
buffer is massive.
An alternative would be to simply pour the centroids in, one digest at a
time. Putting in part of a digest at a time will lead to situations where
the invariant may be violated transiently until you get the rest of the
data in. I haven't analyzed that scenario in detail so I can't say that it
will be completely safe. The case where you add an entire t-digest at a
time has been analyzed, however, and is safe. This extends to adding
multiple complete digests.
I like and sympathize with keeping arguments const and might well have
preferred that approach were I to do it over again.
On Fri, Dec 22, 2023 at 2:23 PM Alexander Saydakov @.***>
wrote:

I am looking at the code to merge t-digests. I don't like that a modifying
operation compress() is called on the input t-digests. I wonder whether
this is avoidable. I would think this is done for simplicity to avoid
dealing with unmerged buffer in the incoming t-digests, however the
compress() method forces compression even if the buffer is empty.
There is a practice in C++ to pass const reference to the input object.
Having a requirement of a mutable input is quite ugly. And forcing
compression unnecessarily is even worse. I am not sure how to deal with
this.
—
Reply to this email directly, view it on GitHub
https://github.com/apache/datasketches-cpp/issues/409#issuecomment-1868105273,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAB5E6RT7HJBQ7GLML3F6P3YKYB5BAVCNFSM6AAAAABANDLXNOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRYGEYDKMRXGM
.
You are receiving this because you commented.Message ID:
@.***>

I implemented merge without calling compress() on the input and one tdigest at a time. Here is what happens with rank error after merge. The plot is too crowded. I can try plotting some things separately later. The rank error seems to be a bit worse compared to single tdigest, but does not seem to get worse with increased number of tdigests (even improves a little). I tried 32, 64 and 128 (the number does not have to be a power of 2, just a habit).

And this is around rank 0.95

Now 32-way merge only

I would recommend a higher default compression factor than 100.
But otherwise, your results look good. The reason for the increased
accuracy in some cases is that by merging you are effectively keeping a lot
of extra centroids in all of the t-digests that you are merging together.
This was the motivation behind the way that the merging digest doesn't
fully compact the centroids in the buffer until persisting the digest. This
has a cost in terms of buffer space (and thus requires a few percent more
merges) but wins in accuracy.
On Wed, Jan 10, 2024 at 10:09 PM Alexander Saydakov <
@.***> wrote:

Now 32-way merge only
TDigest.100.rank.error.after.32-way.merge.0.99.uniform.random.input.png.png
(view on web)
https://github.com/apache/datasketches-cpp/assets/13126686/4d6489b3-119e-46fe-a529-da2f3ab1aef5
—
Reply to this email directly, view it on GitHub
https://github.com/apache/datasketches-cpp/issues/409#issuecomment-1886353997,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAB5E6XIHDSMQSR5ZPQC2TTYN5625AVCNFSM6AAAAABANDLXNOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBWGM2TGOJZG4
.
You are receiving this because you commented.Message ID:
@.***>

Well, I am not sure I understand this argument about increased accuracy due to keeping more centroids. If so, why single digest does better? Perhaps, I should measure with just a few tdigests.

Hmm... I misunderstood your scenario.
When you merge digests do you merge all of their centroids at once into a
single buffer?
On Wed, Jan 10, 2024 at 11:19 PM Alexander Saydakov <
@.***> wrote:

Well, I am not sure I understand this argument about increased accuracy
due to keeping more centroids. If so, why single digest does better?
Perhaps, I should measure with just a few tdigests.
—
Reply to this email directly, view it on GitHub
https://github.com/apache/datasketches-cpp/issues/409#issuecomment-1886436262,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAB5E6RJXGHKAFEYCMS6T4LYN6G7BAVCNFSM6AAAAABANDLXNOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBWGQZTMMRWGI
.
You are receiving this because you commented.Message ID:
@.***>

I measure like this: for each stream length (16 points per octave) spray the input in round-robin fashion across N sketches, then create a fresh sketch and merge all N sketches into it, measure rank error.
Here is the result for just 2 sketches

Merging 2 digests in Java has this unexpected result (I was even afraid that I misconfigured something, but I believe I double-checked)

added 4-way merge

these are fascinating (and depressing).
I would strongly recommend you shift to using δ=200 instead of 100 for
practical use.
But otherwise, you are digging up some interesting and surprising phenomena.
On Thu, Jan 18, 2024 at 10:09 PM Alexander Saydakov <
@.***> wrote:

Screenshot.2024-01-18.at.10.08.48.PM.png (view on web)
https://github.com/apache/datasketches-cpp/assets/13126686/8bae5147-f177-42b7-8e0a-5b32a3dffb2a
—
Reply to this email directly, view it on GitHub
https://github.com/apache/datasketches-cpp/issues/409#issuecomment-1899823416,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAB5E6TK7UG6GPXRNLG5TKDYPIEYTAVCNFSM6AAAAABANDLXNOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJZHAZDGNBRGY
.
You are receiving this because you commented.Message ID:
@.***>

@tdunning could you explain why forced compression is needed (if buffer is empty)?

I think we can close this issue now that it's merged?

Perhaps. There might still be a few things to decide. For instance, I did not change the default from 100 to 200 as @tdunning suggested. There was no explanation why. Maybe we should not have any default?
tdigest(100) is about 10KB in memory. tdigest(100) is about 20KB.

There is a typo in your last comment.
But, yes, the footprint with δ = 200 is twice the footprint for smaller values. The issue is that I changed the definition slightly a while ago and a default of 200 matches peoples accuracy expectations better than δ = 100 does. I prefer it if people who use the default aren't surprised negatively by accuracy so that's the default I recommend.
Note that the persisted size will be substantially smaller since persistence trims away the incoming buffer and does a final compression.

That was not a typo, but invisible &lt:float> and <double> because of angle brackets. I just fixed that.

Ahhhh!
Makes MUCH more sense.  In any case changing 100 to 200 will roughly double the memory footprint.  And the disk footprint (which will be much smaller).

I wonder whether we should have a default at all. What if we just let the user decide?

It's a reasonable approach. I wanted users to have a reasonable starting point if they didn't want to learn about the knobs, thus I gave a default.
YMMV in your own package. I think you will need to at least provide a recommendation.

Yes.
Exactly.
On Mon, Mar 4, 2024 at 2:38 PM Alexander Saydakov @.***>
wrote:

OK, let's keep the default. I changed it to 200, and therefore the
internal compression is 400. Is this what you had in mind?
—
Reply to this email directly, view it on GitHub
https://github.com/apache/datasketches-cpp/issues/409#issuecomment-1977590696,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAB5E6SKV44QKFVIFZU7JMTYWTZVXAVCNFSM6AAAAABANDLXNOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZXGU4TANRZGY
.
You are receiving this because you were mentioned.Message ID:
@.***>


Server 2019

Should Server 2019 be added to the supported RDS versions on this page?  Per this announcement:  https://docs.microsoft.com/en-us/DeployOffice/deploy-office-365-proplus-by-using-remote-desktop-services
Thank you,
Mike

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6da18a8b-9e96-0155-7b30-9c211070d793
Version Independent ID: 5a7e610a-bfcb-d9cc-03f7-589d970128d9
Content: Deploy Office 365 ProPlus by using Remote Desktop Services
Content Source: DeployOffice/deploy-office-365-proplus-by-using-remote-desktop-services.md
GitHub Login: @DHB-MSFT
Microsoft Alias: danbrown

@DHB-MSFT  The article is still not updated to include Windows Serve 2019. Please fix this ASAP, Microsoft licensing is confusing enough as it is, without this lack of information!

Thanks for bringing this up. We will clarify and update our documentation accordingly.

I've got word back, Microsoft 365 Apps for enterprise is supported on Windows Server 2019 running RDS since September 2019, so any build newer than version 1909 is supported. I will work it into the documentation soon.

Thanks a lot for getting back to us with that information!

I've incorporated this feedback into PR#811...


♿  Remove fixed height on Button

My first thought was to add this to @equinor/eds-labs – but we already have a button component there with the CSS spike. We could potentially rename that to something else… or publish this to @equinor/eds-core-react to a next release so they can test it in Envis 🤔
How to test:

Spin up a Codespace from this pr
Wait for the dust to settle
Run pnpm storybook:core-react in the terminal and open the preview in a browser
Change the root font size in the browser* and observe how the button size changes

*In Chrome, go to Settings → Appearance → Font size
Resolves #1858

I’ve left out the ghost icon button, since the icon size is specified in px – not sure how that fits with the needs in Envis 🤔 Also did the calculations manually for now to see how it works with first, which is why I’ve left some comments with the original values. So this is ready for review, but we will have to publish it to with a dist-tag to test it properly in Envis – so

What doesn’t scale is the border-width. Maybe it should? 🤔 Any thoughts on that @equinor/eds-ux? We could use 1px as the minimum width for example when users decrease the font-size, but use a relative width when users increase the font-size.
Border-radius is relative to the font-size, it’s 4px when the font-size is 14px as in Figma, but increases in parallell with the font-size.
The icon doesn’t scale. Maybe it should? 🤔 When the button is scaled up, it looks a bit weird when the icon doesn’t scale.

Not everyone uses Chrome... 😬 Which is a good thing.
Anyway without diving into the philosophy of pixels vs relative units and all that, I had a look at storybook and it seems to solve our issues at least:

Not everyone uses Chrome... 😬  Which is a good thing.

Absolutely! I used Chrome as an example since that’s what most people use. I used Firefox Developer Edition too for a while, but on Chrome OS it’s pretty much just Chrome – I haven’t bothered with installing anything else.

I think we should rename this PR to something that is more descriptive 👍  It goes far beyond a simple remove fixed height imho.

People on the mighty Internet suggest to be explicit about the calculations for rem/em, as a result it might be easier to understand what is going on without changing the mental model? Leaving it here as an example, but if you change the font size on the root things will be more complicated
font-size: calc(24 / 16 * 1rem); <- In this example, 24 is the value we are aiming for 24px

My last comment is about complexity. This obviously makes things a lot more complicated by using em and rem for all spacings, especially with a lot of decimals. I think it's hard to understand what is going on by just looking at the code, there's so much going on. This might or might not be what we want, but if we really want to go down this road we need to ensure that it makes sense in terms of a much better user experience. (because of the added complexity) People tend to have different opinions on which units to use for different use cases beyond typography. :)

It goes far beyond a simple remove fixed height

I think the other changes are side effects of removing the fixed height. Lets say you have a button with a border-radius of 5px and a font-size of 16px for example. If you increase the font-size to 32px then it looks like the border-radius is only 2.5px relative to the font-size. And the same goes for spacing. So the way the button behaves now, with the exception of border-width, is that when you increase the font-size, the button always looks the way it was designed in Figma, only larger. But if you take a screenshot of it, and scale that down, it should fit on top of the default one.
The reason I didn’t change the border-width, was because that would make the button look heavier as the font-size increases – which I don’t think is something we want necessarily.
In EnergyVision all the margins and paddings are dynamic based on the size of the viewport, so then I think it would look strange if the button didn’t follow the same principles 🤷🏻♂

be explicit about the calculations for rem/em

That’s a good idea, I’ll change that.

I think it's hard to understand what is going on by just looking at the code

I agree, so adding the calculations would make that easier. And using css custom properties makes it easier too. I revisited the calculations for the dynamic typography a couple of weeks ago in a Codepen and did it with custom properties instead of using the Power Apps calculator I made a while back. It’s still relatively complex, but I would argue that using custom properties makes it more readable. The --convertToRem variable is only used as a multiplier to convert px to rem.
And there’s always a debate over which units to use, as you say. I think relative units make it more responsive and accessible, but I use px for some things, mostly borders. The discussion about rem vs. em is another topic, but I think that depends on what you want. If border-radius, padding and font-size is harmonious and everything falls apart if you change one of them I then that wouldn’t be a very robust design in my opinion.
In the example you mentioned on Slack from that CSS course, I think the author has some valid points. But in his first example, I wouldn’t use 2ems in both of those styles when they don’t mean the same, I would use an em value that would translate to the same px value. And I also agree with the point about having less space for text if the padding is increased, but that doesn’t take large screen layouts into considerations, where you actually want more space – because the alternative would be narrow columns of content surrounded by white space which is precisely what we wanted to avoid.
I think we should get @BirteThornquist to take a look at it too, I can publish the component to npm and you can try it out, perhaps it’s easier to evaluate when you use the component in a web page – and I’ll use the calc() function to make values easier to reason about and start working on the table component.

…or alternatively I could make two quick examples in Codepen so we can compare the font-size only approach to the all-in approach.

From my point of view, it should definitely be considered whether the edge widths, the rounded corners, are also scaled when the buttons are increased. When the buttons get bigger and the rounded corners remain with a corner radius of 4, the shape will change character and the experience of proportions will be different.
This also applies to the line thickness of outline buttons, although it can withstand a larger scale before it changes expression. Icon scaling should also be considered.
In general, changing one single proportion in a molecule / template etc. will also give changes to the other "elements" if the goal is to give the same form experience.

From my point of view, it should definitely be considered whether the edge widths, the rounded corners, are also scaled when the buttons are increased. When the buttons get bigger and the rounded corners remain with a corner radius of 4, the shape will change character and the experience of proportions will be different.

That’s what I thought too, so the border-radius is relative to the font-size.

This also applies to the line thickness of outline buttons, although it can withstand a larger scale before it changes expression. Icon scaling should also be considered.

Yes, I think you’re right about that too, that the icons should also scale. But then there’s the issue of pixel snapping, so we may have to investigate that in a separate issue 🤔

In general, changing one single proportion in a molecule / template etc. will also give changes to the other "elements" if the goal is to give the same form experience.

I agree.

Note to self:
Do a visual check on components that use button:

[ ] Singleselect
[ ] Multiselect
[ ] Search
[ ] Popover
[ ] Pagination
[ ] Sidesheet

Published to npm as v0.18.0-envis.1


Issues in sync with APKAM changes

Describe the bug

[ ] Wavi app keys are created and synced to the cloud
[ ] Connected to another client which has enrolled to buzz namespace and Tried to run sync, by using the command sync:from. All the keys with wavi namespace are also synced
[x] Scan verb displays each key twice -- This issue is not reproducible with the latest changes

Steps to reproduce
Attaching the test steps in the ticket
test_logs_apkam_buzz_namespace.txt
Expected behavior
Only buzz keys should be synced
Screenshots
No response
Smartphones

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Were you using an atApplication when the bug was found?
No response
Additional context
No response

Setting priority to P1

Issue-3 - The (scan verb displays each key twice) is no longer reproducible with the latest trunk branch of the server.
"public:custom_hobbies.wavi@ alice🛠","public:email.wavi@ alice🛠","public:field_order_of_self.wavi@ alice🛠","public:firstname.wavi@ alice🛠","public:lastname.wavi@ alice🛠","public:phone.wavi@ alice🛠","public:pkaminstalled@ alice🛠","public:privateaccount.wavi@ alice🛠","public:pronoun.wavi@ alice🛠","public:publickey@ alice🛠","public:signing_publickey@ alice🛠"]

PR review pending

The changes are completed and PR is ready to merge. Currently, the functional tests are failing in at_server. Waiting for the functional test to pass.


Error when installed with setup.py

from flashfftconv import FlashFFTConv, FlashDepthWiseConv1d
File "/home/user/miniconda3/envs/common/lib/python3.12/site-packages/flashfftconv-0.0.0-py3.12.egg/flashfftconv/init.py", line 1, in
File "/home/user/miniconda3/envs/common/lib/python3.12/site-packages/flashfftconv-0.0.0-py3.12.egg/flashfftconv/conv.py", line 9, in
ModuleNotFoundError: No module named 'monarch_cuda'

Please install the CUDA kernels first:
cd flash-fft-conv
cd csrc/flashfftconv
python setup.py install
On Thu, Aug 22, 2024 at 8:24 AM Kyuu9TroppoFumare @.***>
wrote:

from flashfftconv import FlashFFTConv, FlashDepthWiseConv1d
File
"/home/user/miniconda3/envs/common/lib/python3.12/site-packages/flashfftconv-0.0.0-py3.12.egg/flashfftconv/
init.py", line 1, in
File
"/home/user/miniconda3/envs/common/lib/python3.12/site-packages/flashfftconv-0.0.0-py3.12.egg/flashfftconv/conv.py",
line 9, in
ModuleNotFoundError: No module named 'monarch_cuda'
—
Reply to this email directly, view it on GitHub
https://github.com/HazyResearch/flash-fft-conv/issues/28, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABDDIIUCCNHATLXWG3DIRTLZSX7BDAVCNFSM6AAAAABM6MO5GKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ4DCMBZG44TIMQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Yes, now it works, I missed the setup for csrc by mistake, thanks for your patience.


Farmware panel modifications

Coverage increased (+0.4%) to 74.931% when pulling 1145ef45f7b7d41e45031d06dcdad3598e93c4c4 on gabrielburnworth:master into 8ff8b669b3f944e3d27366cf9269f0ed52418ca7 on FarmBot:master.


Add Tier to tag setting

Cost analysis is a very important area in SaaS.
SaaS Boost supports various cost allocation tags.
However, the cost allocation tag for the tier is not supported.
Accurate cost measurement for each tier is an essential element in cost estimation.
For example, if a lower tier spends more money than a higher tier, it will adversely affect profitability. We should be able to identify this.

In the process of creating a tenant, a code was added to add the tier information to the tag and use it for cost estimation.
It's a simple fix, but I think it's necessary.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

Hi @PoeppingT
Thanks for the detailed explanation.
A1) No other reason. I think I left it out by mistake.
A2) I understand how to pass parameters to other resources in tenant-onboarding-app.yaml.
A3) I'll take a look at handleOnboardingBaseProvisioned again and understand.
I will request a review after making necessary changes and testing enough.
Thanks!! :)

Hi @PoeppingT
I modified the code. To pass the Tier tag as a parameter to the tenant-onboarding-app.yaml CloudFormation stack through handleOnboardingBaseProvisioned in services/onboarding-service/.../OnboardingService.java.
I used the sample application to complete the necessary tests. Tier tags are normally assigned to all resources (rds, efs, ecs, etc.) created by CloudFormation stack.


Questions about transfer learning on COCO with DetCon pre-train

Hi, @skoppula
Thanks for your effort and contribution to self-supervised learning. Your recent paper Efﬁcient Visual Pretraining with Contrastive Detection (DetCon) inspired me a lot. However, there are still some issues bothering me, hope you can reply to me in your free time.
When transferring the pre-trained model to COCO, the fine-tuning recipe of DetCon is different with standard 1x schedule:

Learning rate is 0.3 for DetCon, the standard 1x is 0.2 for batch size 16
Weight decay is 4e-5 for DetCon, while 1e-4 for standard 1x
training and testing resolution is 1024 for DetCon while 1333x800 for standard 1x

I've noticed that other methods are almost always fine-tuned with the standard 1x, and I wonder why DetCon uses different hyperparameters, and if it's fair to do so.
If we need to adjust the parameters during fine-tuning for different pre-training models, do you have any tips and suggestions for parameter tuning?
Thanks again for your contribution and hope you can reply soon!

Hi Ming,
Thanks for your interest in our work!
Indeed, we use the Cloud TPU Mask-RCNN implementation (https://github.com/tensorflow/tpu/tree/master/models/official/detection) when transferring to COCO, which differs from the detectron implementation in the ways you mention. However, we verified that our baselines (e.g. supervised and SimCLR pre-training) match those of published works (e.g. https://arxiv.org/pdf/2011.10043.pdf) so we don't think these implementation differences result in a significant change in performance.
All the best,
Olivier


⚠ Bofrost.de has degraded performance

In 96efb48, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1480 ms

Resolved: Bofrost.de performance has improved in 9e79181.


I don't see any ExportToExcelIGeneratedCustomExport interface

I don't think the ExportToExcelIGeneratedCustomExport interface is available in the application anymore. It seems to be replaced by OfficeIGeneratedWorkbookCustomExporter.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8848c893-613d-1271-9f00-306fda77eae3
Version Independent ID: 0c390ed8-b9a7-c636-4069-cc84551293d5
Content: Create Open in Excel experiences - Finance & Operations | Dynamics 365 | #MSDyn365FO
Content Source: articles/dev-itpro/office-integration/office-integration-edit-excel.md
Service: dynamics-ax-platform
GitHub Login: @ChrisGarty
Microsoft Alias: cgarty

@PalleAgermark, thanks for posting this feedback!
@ChrisGarty, please review and update the topic appropriately.

Duplicate


Python version <3.11 requirement

Requirements indicates python <3.11, >=3.7. Latest Anaconda distribution base environnement version is 3.11.
I can't recall why the <3.11 requirement was added. Does it still hold?

Hi Jean,
I had not seen this issue until today. Sorry for the elapsed time.
Here is the answer #2. It is because a dependency conflict. pymedphys 0.39 requires python <3.11


Skip tests that cannot be run

ref https://github.com/Mangopay/mangopay2-php-sdk/issues/85
When running the whole test suite, some scenarios fails because of incomplete setup. This cannot be addressed on the client side, so instead of making the suite fail, those tests are now skipped.
Before
/mangopay2-php-sdk$ cd cd tests/suites/
/mangopay2-php-sdk$ php all.php
all.php
1) Cannot test contesting dispute because there's no disputes that can be resubmited
in the disputes list. at [/mnt/data/dev/mangopay2-php-sdk/tests/cases/disputes.php
line 200]
    in test_Disputes_ResubmitDispute
    in MangoPay\Tests\Disputes
    in ../cases/disputes.php
    in MangoPay\Tests\All
E_NOTICE: Undefined offset: 0 in /mnt/data/dev/mangopay2-php-sdk/tests/cases/disputes.php
on line 347
Exception 1!
Unexpected PHP error [Undefined offset: 0] severity [8] in
[/mnt/data/dev/mangopay2-php-sdk/tests/cases/disputes.php line 347]
    in test_Disputes_GetRepudiation
    in MangoPay\Tests\Disputes
    in ../cases/disputes.php
    in MangoPay\Tests\All
E_NOTICE: Trying to get property of non-object in
/mnt/data/dev/mangopay2-php-sdk/tests/cases/disputes.php on line 347
Exception 2!
Unexpected PHP error [Trying to get property of non-object] severity [8] in
[/mnt/data/dev/mangopay2-php-sdk/tests/cases/disputes.php line 347]
    in test_Disputes_GetRepudiation
    in MangoPay\Tests\Disputes
    in ../cases/disputes.php
    in MangoPay\Tests\All
Exception 3!
Unexpected exception of type [MangoPay\Libraries\ResponseException] with message
[Not found. No HTTP resource was found that matches the request URI
'https://api.sandbox.mangopay.com/v2.01/sdk-unit-tests/repudiations/'.] in
[/mnt/data/dev/mangopay2-php-sdk/MangoPay/Libraries/RestTool.php line 331]
    in test_Disputes_GetRepudiation
    in MangoPay\Tests\Disputes
    in ../cases/disputes.php
    in MangoPay\Tests\All
FAILURES!!!
Test cases run: 16/16, Passes: 621, Failures: 1, Exceptions: 3

After
/mangopay2-php-sdk$ cd cd tests/suites/
/mangopay2-php-sdk$ php all.php
all.php
Skip: Cannot test creating dispute document because there's no dispute with expected status in the disputes list.
Skip: Cannot test creating dispute document page because there's no dispute with expected status in the disputes list.
Skip: Cannot test contesting dispute because there's no disputes that can be contested in the disputes list.
Skip: Cannot test contesting dispute because there's no disputes that can be resubmited in the disputes list.
Skip: Cannot test closing dispute because there's no available disputes with expected status in the disputes list.
Skip: Cannot test getting dispute's document because there's no dispute with expected status in the disputes list.
Skip: Cannot test submitting dispute's documents because there's no dispute with expected status in the disputes list.
Skip: Cannot test getting repudiation because dispute has no transaction.
OK
Test cases run: 16/16, Passes: 603, Failures: 0, Exceptions: 0

Thanks very much @nclavaud :-)


Fix issue #1879: allow failures to be recorded from multiple threads

Check List:

Fixes #1879
Unit tests : NA
Javadoc with a code example (on API only) : NA

Integrated thanks @unix-junkie!


Spell Corrector Added

Closes: #95

Title : Spell Corrector Model
Name: Garv Anand
Idenitfy yourself: Contributor@ GSSoC and SSoC

Describe the add-ons or changes you've made 📃
Made the complete model and also added heirarichal data format files of encoder and decoder
Type of change ☑

What sort of change have you made:

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Code style update (formatting, local variables)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested? ⚙

Checklist: ☑

[x] My code follows the Contributing Guidelines & Code of Conduct of this project.
[x] This PR does not contain plagiarized content.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly wherever it was hard to understand.
[x] My changes generate no new warnings.

Screenshots 📷

Attached is a video of the working model
https://github.com/user-attachments/assets/491bc4da-e20f-4929-933f-9e3180cf43b6

Uploading Spell Corrector - Made with Clipchamp.mp4…
Note to reviewers 📄

added the option in the website and integrated with the website and It is working absolutely fine and a small training and working of model is shown in the above video already

Always link your issue not the pr


⚠ Community has degraded performance

In ce07388, Community (https://community.wrk.com/health) experienced degraded performance:

HTTP code: 200
Response time: 164 ms

Resolved: Community performance has improved in 00a52f2.


Application layout

The application layout should resemble the original Evernote app, while featuring a minimalist look and feel with only the most important features (as per #2).

Done in https://github.com/fmoliveira/tantalus/commit/3778055ae8f28e5a9f313ad3e928da3a7cf865e3.
Minimal adjustments would be needed, but it's enough to satisfy this issue already.


Model View: Add tests for the controls and layouts

SQL Operations Studio Version:

Steps to Reproduce:

Closing older unfunded task work-items.


reconnect feature is unspported

for example:
a -> b
b -> c
I modified the output of node a to the input of node c, but it will not reconnect between a and c.

As is stated in the readme:

Note: To link node $A$ (data_0 in the above example) to node $B$ (the 1st Conv in the above example), it is suggested to edit the input of node $B$ to the output of node A, rather than edit the output of node $A$ to the input of node B. Because the input of $B$ can also be other node's output (Transpose  in the above example ) and unexpected results will happen.

You can try to rename the input of node c to the output of node a (rather than rename the output of node a to the input of node c ) and see whether it works.


Consider the addition of QEMU-3DFX

I came across this video ( https://youtu.be/j7liRXxGdXo ) which demonstrated (which i believe is) emulating a Voodoo card in QEMU, which obviously is UTM's emulation backend. This allows you to make use of the M1 GPU to play games like Unreal Tournament on Windows 98/XP. This may sound too optimistic but maybe  if this allows 3D acceleration on XP, we could open the door to possibly being able to have 3D acceleration for PowerPC macs.

Can you link to the source code?

https://github.com/kjliew/qemu-3dfx

Oh it’s the same as #3323
I’ve reopened that one since it has more context. However realistically it probably won’t be done as I’d rather spend my efforts on a full GPU solution that works across everything.

Oh it’s the same as #3323
I’ve reopened that one since it has more context. However realistically it probably won’t be done as I’d rather spend my efforts on a full GPU solution that works across everything.

+1 on full GPU solution, because this work he mentioned isnt even GPL legal plus doesnt even let you run Aero Glass on Windows. It's strictly for 3Dfx, Glide and very old OpenGL games in a certain context. Not that it would let you run AutoCAD on it, nope.

So then, how are you gonna deal with GPL legality issues? May I suggest you a lesson on "What GPL really means"?
Again, Google is Your Friend, I did just that and came up with
https://opensource.stackexchange.com/questions/52/what-can-should-i-do-when-i-see-a-violation-to-gpl-restrictions

It is not up to you to enforce the license of software for which you do not own the copyright. None of your rights are being infringed, so you have no standing to take legal action against the copyright infringement.

So if the corporates backing of QEMU and WineHQ do not have their lawyers going after the project for their rights being infringed, then the Sith will be laughing sinisterly on the verge of 64-bit legacy-free computing that Apple had demonstrated since macOS Catalina. Neither the answer poster nor I were lawyers, I presumed. If you reside in the countries & regions that deviate GPL interpretations, then you could try pursuing the case in the courts there.
Though it seems Master Yoda from another Parallel universe had telepathically taught the young Padawan well in the arts of absorbing & devouring the Sith Apprentice [⚡FORCE LIGHTNING⚡]...🤣🤣

UTM is simply a front end for QEMU. UTM bundles QEMU and their licenses are provided: https://mac.getutm.app/licenses/
If there were a modification of these GPL licensed components, we will license them as GPL as well. If that doesn't fall in their those parameters, we can simple re-license the project to that.


Unable to find any community operators like Strimzi or Opendatahub operator after a fresh install.

General information

OS: Linux / Fedora 32
Hypervisor: KVM
Did you run crc setup before starting it (Yes/No)? Yes
Running CRC on: Laptop

CRC version
crc version
CodeReady Containers version: 1.17.0+99f5c87
OpenShift version: 4.5.14 (embedded in binary)
CRC status
crc status --log-level debug
DEBU CodeReady Containers version: 1.17.0+99f5c87
DEBU OpenShift version: 4.5.14 (embedded in binary)
DEBU About to run SSH command with hidden output
DEBU SSH command succeeded
DEBU About to run SSH command:
df -B1 --output=size,used,target /sysroot | tail -1
DEBU SSH command results: err: , output:<PHONE_NUMBER>0<PHONE_NUMBER>6 /sysroot
CRC VM:          Running
OpenShift:       Running (v4.5.14)
Disk Usage:      14.76GB of 32.72GB (Inside the CRC VM)
Cache Usage:     25.68GB
Cache Directory: /home/rhui/.crc/cache
[rhui@MiWiFi-R1CM-srv Downloads]$
CRC config
crc config view

memory                                : 15700
pull-secret-file                      : ./pull-secret.txt

## Host Operating System
cat /etc/redhat-release
Fedora release 32 (Thirty Two)

### Steps to reproduce

  1. Just a fresh install, and there are redhat operator in the operatorhub but none of community operators
  2. See oc-command.txt and it seems that the catalog are fine.
  3. See the screen shot on operator hub and there are others just not community.

### Expected
   1. able to installed community operators like strimzi or opendatahub.

### Actual

oc-commands.txt

As per oc-commands I can see that the all the operator sources are succeed, sometime on the web console side it take some time to show it. Did you wait for around 2-3 mins and refresh the web console?

Hi Praveen,
Yea, I have waited for more than that, and even using a different browser and clear cookies..
In my CRC -> OperatorHub -> Side bar I only see
Provider Type
Certified(140)
Marketplace(81)
In another instance (quicklab) -> I see
Provider Type
Red Hat(50)
Certified(140)
Community(130)
Marketplace(81)
I wonder if there are some authentication or flags that I need to enable..
Thanks,
Roger

Hi Praveen,
I figured it out..
oc logs pod/packageserver-7b8f7b49d4-hhr47 -n openshift-operator-lifecycle-manager|grep -i community
time="2020-10-13T10:04:19Z" level=warning msg="error getting stream" action="refresh cache" err="rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp <IP_ADDRESS>:50051: connect: connection timed out"" source="{community-operators openshift-marketplace}"
oc get pods -n openshift-marketplace
I saw the community-operators and redhat-operators pods unable to connect to quay and restart many times.
I just delete the community-operators and redhat-operators pods and it pull the service, I now have all the operators in the operatorhub page.

I dig deeper to my own issue.. I find out that quay is slow and it takes longer to pull all the project down, the LivenessProbe and Readiness Probe is too short. Increasing that helps.
deployment.apps/community-operators


AWS Lambda support

I'm currently building a serverless architecture and was hoping to use elastic cloud as a one-stop repository for logs and metrics. Per elastic/apm-agent-python#320 it seems that Lambda is not a supported environment for Elastic-APM.
I'd be happy to put in some work to make this happen, but I'm not entirely clear on where the problems lie. Is there a reason why this is more complex than calling collect on the  metric repository and flush on the transport?
If I had a MetricRegistry with a collect_interval of 0ms, and an unthreaded transport that flushed only on demand, I could flush explicitly at the end of the lambda invocation. It would be simple to extend that to a decorator that flushed on exit from the wrapped function.
Is this an enormous simplification of the requirements?

Hi @bobthemighty. Yes, that would probably work, however it might incur quite a heavy overhead as basically every lambda invocation would result in an HTTP request to your APM Server. That might be ok if you have relatively long-running lambda functions, but for very short running functions (< 100ms or thereabout), the overhead might start to dominate the actual execution time.
I'm not very familiar with how exactly the life cycle of an invoked lambda function is, but if they shut down the interpreter cleanly after the invocation, you might not even have to flush the queue, as we use atexit to flush the queue on shutdown.

That goes in the general direction of what we think might be a workable solution, basically use CloudWatch or possibly SQS as a transport.
Note however that the events that the agent generates are not elasticsearch documents. They first need to be processed by the APM Server, which turns them into ES documents.
If you want to write your own transport, note that we just merged a pretty large change to the inner workings of transports (#411). This shouldn't make it much more complicated to implement such a transport, though (we might have to move the init_buffer inner function to a method though, so you can avoid the GZip compression).

Note however that the events that the agent generates are not elasticsearch documents. They first need to be processed by the APM Server, which turns them into ES documents.

Roger. Is the format you're writing identical between apm-node/python/go, or is there additional language-specific munging? I'm thinking that if we were to write to CloudWatch, and the format is the same, then it's not too hard to provide a library for parsing those logs out and forwarding them to an APM instance.

note that we just merged a pretty large change to the inner workings of transports (#411).

What Transport methods other than "send" are relied on by client code? I can see that there's a queue method that's public-looking, but not sure where it's actually in use. I'm wondering how much of the transport infrastructure is actually relevant to a stdout implementation.
Thanks for the prompt reply, btw. :+1:

The API between the agent and APM Server is standardized and all agents use the same (it's defined as JSONSchema here: https://github.com/elastic/apm-server/tree/master/docs/spec).
For implementing your own transport, ideally you'd only need to override send(data). data however is a gzip encoded blob, which is less than ideal for printing to stdout. So you'd either need to unzip it again, which is wasteful, or have a way to disable gzip compression in the first place. If it's ok with you, I'd recommend to unzip it for now, and if this turns out to be a workable solution, we can add a hook to disable gzip compression.

Ref https://github.com/elastic/apm/issues/352

We're going to try to get an MVP of this going soon -- it will not be using lambda extensions initially, but will just be sending directly to the configured APM server from the lambda function itself. We know this will not be useful in all cases as the overhead will be dramatic, especially for short-running lambda functions. But it will be a first step toward a better implementation using extensions.


[Argument cannot be null. Parameter name: baseStream] after opened for a hour

I was using aws sdk in unity game and use both facebook and cognito developer identity token for credential to call lambda function
But when I used developer token and opened my app ,which ping lambda every 30 seconds, for approximately one hour. This error was popped up
I was suspect it would be a problem about token expired
Expected Behavior
Should not have any error in this process
Current Behavior
This cause an error that was caught in the system
Steps to Reproduce (for bugs)

Generate cognito developer identity token from server
use that token as credential to LambdaClient in unity3D
Use lambda client to invoke every 30 seconds continuously for a hour
Catch the error

Your Environment
Unity3D
AWSSDK.Lambda, Version=<IP_ADDRESS>
AWSSDK.CognitoIdentity, Version=<IP_ADDRESS>

Could you share what exact APIs you used?  Also, what's the error that was returned by the service?  Please help us help you by providing more concrete detail such as sample code, exact error message, or http traffic capture.

@sstevenkang I suspect it most likely the AmazonLambdaClient.InvokeAsync my app use that function in almost everything.
Argument cannot be null. Parameter name: baseStream is most likely came from
https://github.com/aws/aws-sdk-net/blob/37b2546044248de151a6ce47641ba096728fb5cb/sdk/src/Core/Amazon.Runtime/Internal/Util/WrapperStream.cs#L45
But I don't know much detail about it from my app. Only log error from some client sometimes so it hard to track

Just to start, a stacktrace would help us narrow down how the exception surfaced.

@sstevenkang I can't get stacktrace because it really random behaviour that occur with unknown cause from aws sdk

I have same error for SnsClient.SubscribeAsync and SnsClient.CreatePlatformEndpointAsync
Responce Callback recieve Exception with this stacktrace:
Amazon.Runtime.Internal.Util.WrapperStream..ctor
Amazon.Runtime.Internal.Util.CachingWrapperStream..ctor
.at
Amazon.Runtime.Internal.Util.CachingWrapperStream:.ctor
Amazon.Runtime.Internal.Transform.XmlUnmarshallerContext..ctor
Amazon.Runtime.Internal.Transform.XmlResponseUnmarshaller.ConstructUnmarshallerContext
Amazon.Runtime.Internal.Transform.ResponseUnmarshaller.CreateContext
Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleException


[transducers] array type of flatten output is not flattend

Hi,
I realised thi.ng/iterators is deprecated and checked out transducers. But using it, some typescript errors came up.
When flatten has a multi-dimensional array as input, it returns the same, instead of a simple array. I.e. ot returns number[][], but it is probably better to use number[], otherwise one has to always use as number[] afterwards.
Here is a small codesandbox: https://codesandbox.io/s/admiring-poitras-mq03w?file=/src/index.ts
It contains this code:
import { flatten } from<EMAIL_ADDRESS>
let n: number[][] = [
  [0, 1, 3],
  [0, 1]
];

const myArray = [...flatten(n)];
// type of myArray should be number[], but is number[][]

console.log(myArray);
// [0, 1, 3, 0, 1]

Hi @dennemark - yes, this is one of those older cases where my TS type foo wasn't (still isn't?) strong enough, but quite obviously the current result type is wrong and at the very least should become any or unknown, at least by default... I also do believe the correct result type can't even be fully defined/automatically inferred in TS (but very happy to be corrected!). The correct return type would have to be expressed recursively and the deeper the nesting level of the (arbitrary) input data, the higher the potential for super complex and vast result types. Additionally, flatten() is just syntax sugar for flattenWith(), which also takes a custom predicate function to decide if a value could/should be flattened further and if so how. Because any logic is possible here, there's no way of expressing this (ahead of time) in a general way in TS (IMHO)...
For example, the correct type of the flattened sequence in the following (super contrived) case is much more complex:
interface Foo {
  attribs: Partial<Record<"pos"|"col", { stride: number; buf: Float32Array; }>>;
  indices: number[][];
}

flattenWith<Foo>(
  (x) => isPlainObject(x)
    ? pairs(x)
    : isNotStringAndIterable(x)
      ? x
      : null;
  [{ attribs: { pos: { stride: 3, buf: new Float32Array([1, 2, 3]) } }, indices: [[0, 1, 2]] }]
);
[ 'attribs', 'pos', 'stride', 3, 'buf', 10, 20, 30, 'indices', 0, 1, 2 ]

To express this as type would be something along these lines:
keyof Foo | keyof Foo["attribs"] | keyof Foo["attribs"]["pos" | "col"] | number
...and here we can be happy that Foo isn't any deeper and/or uses more varied leaf value types.
So with all that said, I'd propose to change the signatures of flatten and flattenWith to:
function flatten<A, B = any>(): Transducer<Iterable<A>, B>;
function flatten<A, B = any>(src: Iterable<A>): IterableIterator<B>;

function flattenWith<A, B = any>(fn: Fn<any, MaybeIterable<B>>): Transducer<Iterable<A>, B>;
function flattenWith<A, B = any>(fn: Fn<any, MaybeIterable<B>>, src: Iterable<A>): IterableIterator<B>;

Of course, this approach requires manually adding generics, but I can't see a straightforward reliable other way and at least the above is more correct than the current types...

Thanks for the long explanation. The flattenWith example is indeed a complex type. But it seems using generics is a great alternative even though it needs to be manually added!

FWIW I've just tried a kind of escape hatch approach to at least support nested array input types in a more automatic manner and it's promising. Maybe this could work, at least for flatten with its hardcoded predicate function (need to do more tests):
// attempt recursive lookup of array element types
type ElementType<T> = T extends unknown[]
    ? ElementType<T[0]>
    : T extends TypedArray
    ? number
    : T;

export declare function flatten<A, B = ElementType<A>>(src: Iterable<A>): IterableIterator<B>;

[...flatten([[1, 2], ["a", [{a: 1}, {b: 2}]]])]

// result type:
// (string | number | { a: number; b?: undefined; } | { b: number; a?: undefined; })[]

[...flatten([[], ["a", new Float32Array()]])]
// result type
// (string | number)[]


Update update-by-query.md

Updated the slices parameter description to be in-line with what's actually available.  You can specify the string 'auto' as well.  The text is pulled from the delete-by-query documentation to keep in sync.
Description
Update slices parameter definition
Checklist

[ X] By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Thanks for your contribution @seittema. I left a suggestion on how to fix.

@Naarcha-AWS Should I update the delete-by-query to match your proposed changes?  The grammer in this PR was pulled from current release documentation so that it was consistent.

@seittema: That would be super helpful! Thanks for calling that out.


Occasional crash on server startup

Describe the bug
Occasionally upon server startup the follow error is encountered.  It seems to go away if more time between server shutdown and startup is given:
[AUDIT   ] CWWKT0016I: Web application available (default_host): https://9f85390dd73a:9443/fhir-openapi/
[AUDIT   ] CWWKZ0001I: Application fhir-openapi started in 1.886 seconds.
[ERROR   ] CWWKZ0002E: An exception occurred while starting the application fhir-bulkimportexport-webapp. The exception message was: com.ibm.ws.container.service.state.StateChangeException: org.jboss.weld.exceptions.DefinitionException: Exception List with 1 exceptions:
Exception 0 :
java.lang.RuntimeException: javax.naming.NameAlreadyBoundException: com
at com.sun.jersey.server.impl.cdi.CDIExtension.initialize(CDIExtension.java:196)
at com.sun.jersey.server.impl.cdi.CDIExtension.beforeBeanDiscovery(CDIExtension.java:297)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:95)
at org.jboss.weld.injection.MethodInvocationStrategy$SpecialParamPlusBeanManagerStrategy.invoke(MethodInvocationStrategy.java:187)
at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:330)
at org.jboss.weld.event.ExtensionObserverMethodImpl.sendEvent(ExtensionObserverMethodImpl.java:123)
at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:308)
at org.jboss.weld.event.ObserverMethodImpl.notify(ObserverMethodImpl.java:286)
at javax.enterprise.inject.spi.ObserverMethod.notify(ObserverMethod.java:124)
at org.jboss.weld.util.Observers.notify(Observers.java:166)
at org.jboss.weld.event.ObserverNotifier.notifySyncObservers(ObserverNotifier.java:285)
at org.jboss.weld.event.ObserverNotifier.notify(ObserverNotifier.java:273)
at org.jboss.weld.event.ObserverNotifier.fireEvent(ObserverNotifier.java:177)
at org.jboss.weld.event.ObserverNotifier.fireEvent(ObserverNotifier.java:171)
at org.jboss.weld.bootstrap.events.AbstractContainerEvent.fire(AbstractContainerEvent.java:53)
at org.jboss.weld.bootstrap.events.AbstractDefinitionContainerEvent.fire(AbstractDefinitionContainerEvent.java:44)
at org.jboss.weld.bootstrap.events.BeforeBeanDiscoveryImpl.fire(BeforeBeanDiscoveryImpl.java:54)
at org.jboss.weld.bootstrap.WeldStartup.startInitialization(WeldStartup.java:407)
at org.jboss.weld.bootstrap.WeldBootstrap.startInitialization(WeldBootstrap.java:79)
at com.ibm.ws.cdi.impl.CDIContainerImpl$2.run(CDIContainerImpl.java:149)
at com.ibm.ws.cdi.impl.CDIContainerImpl$2.run(CDIContainerImpl.java:146)
at java.security.AccessController.doPrivileged(AccessController.java:678)
at com.ibm.ws.cdi.impl.CDIContainerImpl.startInitialization(CDIContainerImpl.java:146)
at com.ibm.ws.cdi.liberty.CDIRuntimeImpl.applicationStarting(CDIRuntimeImpl.java:448)
at com.ibm.ws.container.service.state.internal.ApplicationStateManager.fireStarting(ApplicationStateManager.java:51)
at com.ibm.ws.container.service.state.internal.StateChangeServiceImpl.fireApplicationStarting(StateChangeServiceImpl.java:50)
at com.ibm.ws.app.manager.module.internal.SimpleDeployedAppInfoBase.preDeployApp(SimpleDeployedAppInfoBase.java:550)
at com.ibm.ws.app.manager.module.internal.SimpleDeployedAppInfoBase.installApp(SimpleDeployedAppInfoBase.java:511)
at com.ibm.ws.app.manager.module.internal.DeployedAppInfoBase.deployApp(DeployedAppInfoBase.java:347)
at com.ibm.ws.app.manager.war.internal.WARApplicationHandlerImpl.install(WARApplicationHandlerImpl.java:65)
at com.ibm.ws.app.manager.internal.statemachine.StartAction.execute(StartAction.java:140)
at com.ibm.ws.app.manager.internal.statemachine.ApplicationStateMachineImpl.enterState(ApplicationStateMachineImpl.java:1266)
at com.ibm.ws.app.manager.internal.statemachine.ApplicationStateMachineImpl.run(ApplicationStateMachineImpl.java:881)
at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:239)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.lang.Thread.run(Thread.java:818)
Caused by: javax.naming.NameAlreadyBoundException: com
at com.ibm.ws.jndi.internal.ContextNode.createSubcontext(ContextNode.java:136)
at com.ibm.ws.jndi.internal.WSContext.createSubcontext(WSContext.java:245)
at com.ibm.ws.jndi.WSContextBase.createSubcontext(WSContextBase.java:141)
at org.apache.aries.jndi.DelegateContext.createSubcontext(DelegateContext.java:111)
at javax.naming.InitialContext.createSubcontext(InitialContext.java:500)
at com.sun.jersey.server.impl.cdi.CDIExtension$1.stepInto(CDIExtension.java:280)
at com.sun.jersey.server.impl.cdi.CDIExtension.diveIntoJNDIContext(CDIExtension.java:267)
at com.sun.jersey.server.impl.cdi.CDIExtension.createJerseyConfigJNDIContext(CDIExtension.java:273)
at com.sun.jersey.server.impl.cdi.CDIExtension.initialize(CDIExtension.java:192)
... 40 more
[AUDIT   ] CWWKT0016I: Web application available (default_host): https://9f85390dd73a:9443/fhir-server/api/v4/
[err] log4j:WARN No appenders could be found for logger (org.apache.cxf.common.logging.LogUtils).
[err] log4j:WARN Please initialize the log4j system properly.
[err] log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
[WARNING ] [ONCE] Create tablespace not supported in Derby
[WARNING ] [ONCE] Derby does support not multi-tenancy: SUBSTANCENUCLEICACID_STR_VALUES
[ERROR   ] Encountered an exception while bootstrapping the FHIR database
java.sql.SQLException: Table/View 'SUBSTANCENUCLEICACID_STR_VALUES' already exists in Schema 'APP'.
[AUDIT   ] CWWKZ0001I: Application fhir-server-webapp started in 29.948 seconds.
[AUDIT   ] CWWKF0012I: The server installed the following features: [appSecurity-2.0, batch-1.0, batchManagement-1.0, cdi-2.0, distributedMap-1.0, el-3.0, federatedRegistry-1.0, jaxb-2.2, jaxrs-2.1, jaxrsClient-2.1, jdbc-4.1, jndi-1.0, jsf-2.3, json-1.0, jsonp-1.1, jsp-2.3, ldapRegistry-3.0, localConnector-1.0, mpConfig-1.2, mpOpenAPI-1.0, oauth-2.0, openidConnectServer-1.0, servlet-4.0, ssl-1.0, transportSecurity-1.0, websocket-1.1].
To Reproduce
Sporadic problem
Expected behavior
Additional context

@elsalant can you share your server.xml? Please note it has passwords, please remove those before putting / uploading.  you can also ping the server.xml to @lmsurpre  or<EMAIL_ADDRESS>

Nothing special here - I just replaced the default password for the user
registry for user name fhiruser, and added an additional user, fhiruser1.
No other changes...
Regards, Eliot
===================================
Eliot Salant
IBM Haifa Research Labs
Haifa University, Mt. Carmel, Haifa, Israel
31905
Tel.    +972-48296-121
FAX:  +972-48296-113
From:   Paul Bastide<EMAIL_ADDRESS>To:     IBM/FHIR<EMAIL_ADDRESS>Cc:     elsalant<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Date:   26/09/2019 17:14
Subject:        [EXTERNAL] Re: [IBM/FHIR] Occasional crash on server
startup (#157)
@elsalant can you share your server.xml? Please note it has passwords,
please remove those before putting / uploading. you can also ping the
server.xml to @lmsurpre or<EMAIL_ADDRESS>?
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

HI @elsalant
please add to your server.xml the trace specification for JNDI. (add a logging node or append to the traceSpecification).
<logging traceSpecification="*=info:Naming=all:NamingService=all" traceFormat="BASIC"/>

When you notice this occurring again?  Please share the trace.log.
Take care
Paul

Okay, I got this error again.  Here is the trace.log file
Regards, Eliot
trace.log

Closed by mistake...

OK.  I see what's happening.  It doesn't fully make sense as to why.  What libraries are your custom persistence libraries injecting?
mvn dependency:tree -f <path-to-pom>
it shouldn't be injecting com/sun/jersey/config

I have a lot of persistence libraries...
Regarding jersey, these include:
jersey-client-1.9.jar
jersey-client-2.22.2.jar
jersey-common-2.22.2.jar
jersey-container-servlet-2.22.2.jar
jersey-container-servlet-core-2.22.2.jar
jersey-core-1.9.jar
jersey-guava-2.22.2.jar
jersey-guice-1.9.jar
jersey-json-1.9.jar
jersey-media-jaxb-2.22.2.jar
jersey-server-1.9.jar
jersey-server-2.22.2.jar
Thanks, Eliot

Interestingly, this error is now consistently happening when I try running the server with my persistence libraries from a Docker container

can you provide the dependency tree? it'll help me tell you how to setup exclusions. You have three Jax-RS libraries competing and registering ( in particular your jersey libraries are conflicting).  This common happens with Hadoop / HBase dependencies as they are embedded in a Web Application.
P.S. Feel free to ping me your pom.xml internally.

OK. I see what's happening. It doesn't fully make sense as to why. What libraries are your custom persistence libraries injecting?
mvn dependency:tree -f <path-to-pom>
it shouldn't be injecting com/sun/jersey/config

Eliot added <cdi12 enableImplicitBeanArchives="false"></cdi12> which seems to have gotten by the core issue (avoiding the multiple discovered jndi entries for Jaxrs)

Working with Eliot and Lee we've reached a few conclusions:
Problem
1 - Hadoop and other libraries are known to package Jersey dependencies.  These dependencies introduce into the environment a Jersey framework.
2 - The IBM FHIR Server webapp uses PARENT_FIRST as a class loader policy, as such, the Jersey framework in the Hadoop libraries in the userlib container (as bundled in a uber jar), contains Jersey.
3 - The IBM FHIR Server uses a different implementation of JaxRS, the one that is bundled with the OpenLiberty projects as a feature.
4 - CDI is automatically enabled in the J2EE profile in OpenLiberty, as such, the environment discovers Jersey's JNDI declarations are overrides the IBM FHIR Servers.
5 - The CDI leaves the IBM FHIR Server in an inconsistent state and may result in a crash (depending on the discovery classpath).
To solve this problem, add the <cdi12 enableImplicitBeanArchives="false"></cdi12> to the server.xml

Closing per agreement with Eliot


OCPBUGS-31088: Fix loopvar issue

Fix loopvar issue in a range loop.

@jparrill @bryan-cox @muraee @Patryk-Stefanski Could you please take a look ? Thanks.

/retest-required

/lgtm

/retest

/retest-required

/retest

/retest

/retest

/retest

@Patryk-Stefanski Done. Let's see if the Konflux thing will pass this time.

/lgtm

Okay Red Hat Konflux / hypershift-operator-hhlfs / hypershift-main-enterprise-contract failed again. It just won't work..

/retest

/retest

/override "Red Hat Konflux / hypershift-operator-wfjcb / hypershift-main-enterprise-contract"


Add support for setting serviceAccountName

Setting serviceAccountName is necessary when using PodSecurityPolicies that generally don't allow running privileged pods. In this case, a separate service account must be created and associated with a privileged PodSecurityPolicy.
This adds a small template change to allow setting this value from .Vaules.node.daemonset.serviceAccountName.

Cool! Thanks for this!
I wonder... should we also automate creating a service name as well?

Cool! Thanks for this!
I wonder... should we also automate creating a service name as well?

Probably. But in the famous words of @redhatrises, that'd be outside the scope of this PR ;)

Bumped to 0.10.0

It appears the version changed on main after this PR was opened. I've rebased this over the current main.

Thanks


Document how to use user-alertmanager

We get questions on how to set up alerts for application metrics. Let's document this.
Acceptance criteria:

compliantkubernetes.io / User Guide / Alerting contains information on how to create an alert for an application deployed by the user.

Fixed already: https://elastisys.io/compliantkubernetes/user-guide/alerts/


New version: Ryujinx.Ryujinx version 1.1.1202

Pull request has been created with komac v2.1.0 :rocket:
Microsoft Reviewers: Open in CodeFlow

Automatic Validation ended with:

No errors to post.

(Automated response - build 792.)

@wingetbot waivers Add Validation-Executable-Error


Dockers - HIP Backend: Unable to find code object for all current devices

I have built the mivisionx docker using this dockerfile .
rocminfo inside docker as follows:
root@xsjfislx31:/workspace# rocminfo
ROCk module is loaded
=====================
HSA System Attributes
=====================
Runtime Version:         1.1
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:<PHONE_NUMBER>3709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE
System Endianness:       LITTLE

==========
HSA Agents
==========
*******
Agent 1
*******
  Name:                    AMD EPYC 7F52 16-Core Processor
  Uuid:                    CPU-XX
  Marketing Name:          AMD EPYC 7F52 16-Core Processor
  Vendor Name:             CPU
  Feature:                 None specified
  Profile:                 FULL_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        0(0x0)
  Queue Min Size:          0(0x0)
  Queue Max Size:          0(0x0)
  Queue Type:              MULTI
  Node:                    0
  Device Type:             CPU
  Cache Info:
  L1:                      32768(0x8000) KB
  Chip ID:                 0(0x0)
  Cacheline Size:          64(0x40)
  Max Clock Freq. (MHz):   0
  BDFID:                   0
  Internal Node ID:        0
  Compute Unit:            32
  SIMDs per CU:            0
  Shader Engines:          0
  Shader Arrs. per Eng.:   0
  WatchPts on Addr. Ranges:1
  Features:                None
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: FINE GRAINED
      Size:                    263823792(0xfb9a1b0) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
Alloc Alignment:         4KB
      Accessible by all:       TRUE
    Pool 2
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    263823792(0xfb9a1b0) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
    Pool 3
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    263823792(0xfb9a1b0) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
  ISA Info:
*******
Agent 2
*******
  Name:                    AMD EPYC 7F52 16-Core Processor
  Uuid:                    CPU-XX
  Marketing Name:          AMD EPYC 7F52 16-Core Processor
  Vendor Name:             CPU
  Feature:                 None specified
  Profile:                 FULL_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        0(0x0)
  Queue Min Size:          0(0x0)
  Queue Max Size:          0(0x0)
  Queue Type:              MULTI
  Node:                    1
  Device Type:             CPU
  Cache Info:
    L1:                      32768(0x8000) KB
  Chip ID:                 0(0x0)
  Cacheline Size:          64(0x40)
  Max Clock Freq. (MHz):   0
  BDFID:                   0
  Internal Node ID:        1
  Compute Unit:            32
  SIMDs per CU:            0
  Shader Engines:          0
  Shader Arrs. per Eng.:   0
  WatchPts on Addr. Ranges:1
  Features:                None
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: FINE GRAINED
      Size:                    264208444(0xfbf803c) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
    Pool 2
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    264208444(0xfbf803c) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
    Pool 3
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    264208444(0xfbf803c) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
  ISA Info:
*******
Agent 3
*******
  Name:                    gfx908
  Uuid:                    GPU-20b160b85ec60c80
  Marketing Name:
  Vendor Name:             AMD
  Feature:                 KERNEL_DISPATCH
  Profile:                 BASE_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        128(0x80)
  Queue Min Size:          4096(0x1000)
  Queue Max Size:          131072(0x20000)
  Queue Type:              MULTI
  Node:                    2
  Device Type:             GPU
  Cache Info:
    L1:                      16(0x10) KB
    L2:                      8192(0x2000) KB
  Chip ID:                 29580(0x738c)
  Cacheline Size:          64(0x40)
  Max Clock Freq. (MHz):   1502
  BDFID:                   9984
  Internal Node ID:        2
  Compute Unit:            120
  SIMDs per CU:            4
  Shader Engines:          8
  Shader Arrs. per Eng.:   1
  WatchPts on Addr. Ranges:4
  Features:                KERNEL_DISPATCH
  Fast F16 Operation:      TRUE
  Wavefront Size:          64(0x40)
  Workgroup Max Size:      1024(0x400)
  Workgroup Max Size per Dimension:
    x                        1024(0x400)
    y                        1024(0x400)
    z                        1024(0x400)
  Max Waves Per CU:        40(0x28)
  Max Work-item Per CU:    2560(0xa00)
  Grid Max Size:<PHONE_NUMBER>(0xffffffff)
  Grid Max Size per Dimension:
    x<PHONE_NUMBER>(0xffffffff)
    y<PHONE_NUMBER>(0xffffffff)
    z<PHONE_NUMBER>(0xffffffff)
  Max fbarriers/Workgrp:   32
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    33538048(0x1ffc000) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       FALSE
    Pool 2
      Segment:                 GROUP
      Size:                    64(0x40) KB
      Allocatable:             FALSE
      Alloc Granule:           0KB
      Alloc Alignment:         0KB
      Accessible by all:       FALSE
  ISA Info:
    ISA 1
      Name:                    amdgcn-amd-amdhsa--gfx908:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE
      Profiles:                HSA_PROFILE_BASE
      Default Rounding Mode:   NEAR
      Default Rounding Mode:   NEAR
      Fast f16:                TRUE
      Workgroup Max Size:      1024(0x400)
      Workgroup Max Size per Dimension:
        x                        1024(0x400)
        y                        1024(0x400)
        z                        1024(0x400)
      Grid Max Size:<PHONE_NUMBER>(0xffffffff)
      Grid Max Size per Dimension:
        x<PHONE_NUMBER>(0xffffffff)
        y<PHONE_NUMBER>(0xffffffff)
        z<PHONE_NUMBER>(0xffffffff)
      FBarrier Max Size:       32
*** Done ***

When running canny example seeing below error.
root@xsjfislx31:/workspace# export PATH=$PATH:/opt/rocm/mivisionx/bin
root@xsjfislx31:/workspace# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/rocm/mivisionx/lib
root@xsjfislx31:/workspace# runvx /opt/rocm/mivisionx/samples/gdf/canny.gdf
runvx 1.0.0
OK: using AMD OpenVX 1.3.0
"hipErrorNoBinaryForGpu: Unable to find code object for all current devices!"
Aborted (core dumped)`

We are using MI100 gpu. From this, I see that the MI100 is supported gpu.

@anilmartha please set this environment variable (export HCC_AMDGPU_TARGET=gfx908) in the docker filer before running your test.

@asalmanp should we add this to docker instructions?

@asalmanp after adding export HCC_AMDGPU_TARGET=gfx908 in dockerfile, I was able to run the application successfully. Can we add this info in dockerfile or readme?

PR #791 should fix this issue - @asalmanp can you check this with Ubuntu-20.04

The fix for this merged into TOT - @anilmartha please reopen the issue if find problems with the updated dockerfiles.


Conv2D fails with sparse input

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):  Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):  Linux Centos7
TensorFlow installed from (source or binary):  Conda install
TensorFlow version (use command below):  2.5.0-rc3
Python version: 3.7.5

I'm not certain if this is a bug or missing feature?  Trying to run sparse data through convolutional layer, but error immediately results:
x = keras.Input(shape=(100,100,2), sparse=True)
C = keras.layers.Conv2D(16,(3,3))(x)

Error as follows:
TypeError: Failed to convert object of type <class 'tensorflow.python.framework.sparse_tensor.SparseTensor'> to Tensor. Contents: SparseTensor(indices=Tensor("Placeholder_1:0", shape=(None, 4), dtype=int64), values=Tensor("Placeholder:0", shape=(None,), dtype=float32), dense_shape=Tensor("PlaceholderWithDefault:0", shape=(4,), dtype=int64)). Consider casting elements to a supported type.

@Mastiff37
Could you please provide complete code/colab gist in order to expedite the trouble-shooting process.
also refer similar  issue and let us know if it helps.Thanks

All it takes to create the error is to enter those two lines, for example in an ipython terminal.  Do you need more than that?  Because I never got beyond that myself.  Thanks.

@Mastiff37
Could you please refer this,and let us know if it helps.Thanks

Thanks for the reply.  I don't see anything there that exactly applies.  Are you referring to the solutions where the input is manually converted to dense before use?

@jvishnuvardhan
I was able to replicate the issue reported.Please find the gist here.Thanks

@Mastiff37 I think sparse input is not supported. Some other layers like Dense supports sparse tensors. You could convert sparse tensors into dense tensors and pass it to the Conv layers. Thanks!

Understood.  The desired use case is one in which processing the non-sparse version is impractical.  But maybe this is a feature request rather than an issue.  Or maybe the issue is that a more helpful error message could be provided, indicating lack of support?

@Mastiff37 Recently keras team improved layers.preprocessing code. If the current preprocessing layers are not supporting, then please raise an issue in keras-team/keras. Please note that Keras development moved to another repository to focus entirely on only keras.
When you open a new issue, please provide a standalone code with some toy data to test and understand the issue. Thanks!

Agree with you on the error message. Will try to update it. Thanks!


[Question] How to grant permission from admin

@lzw-all-in commented on Sat Mar 14 2020
Hi teams,
I'm an intern at Microsoft. I want to build an application for others to get ADL access token. I already add ADL API to my app. But it need to be grant permission from admin. How can my app be authorized by admin. I don't know who are the admin I can send email to apply this permission.
Thanks !

文档详细信息
⚠ 请勿编辑此部分。 docs.microsoft.com ➟ GitHub 问题链接必须具有此部分。

ID: 0ac43307-75d4-cb6f-e561-0dd9008f28c6
Version Independent ID: 8fc47e76-5ee1-6aeb-73e2-c64c44e3e4ed
Content: Microsoft 标识平台范围、权限和同意
Content Source: articles/active-directory/develop/v2-permissions-and-consent.md
Service: active-directory
Sub-service: develop
GitHub Login: @rwike77
Microsoft Alias: ryanwi

@srvbpigh commented on Sat Mar 14 2020
Hello, @lzw-all-in
Thank you for your feedback.
We are actively reviewing your comments and will get back to you soon.
Kind regards,
Microsoft DOCS International Team

@CeciAc  Thanks for paying attention to my question here.
My major problem is how to apply for ADL permission for my own app on Azure. Or who can grant the ADL permission to my app ?
Thanks teams.
Here is the permission I need.

@CeciAc @lzw-all-in You will need to contact your Azure AD tenants global admin to get this approved. The API permission you have selected required Admin consent and hence you will need to contact your internal IT to get any of the Azure AD global admins to approve your request. after that you will be able to get your application working without a problem. As per our work and discussion internally on IM , we will now proceed with closure of this issue. Should you have any further queries , please feel free to let us know.
Thank you.

@shashishailaj Thanks you so much ! You are very friendly and helpful : )

@lzw-all-in You are welcome . :)


chore: running make test has a side effect of creating new folders and files

When running make test, the following folders are created:
example-docs/copy-protected_images/
example-docs/layout-parser-paper-fast_images/
Desired behavior is not create or to clean up any created files and folders by make test

Duplicate of https://github.com/Unstructured-IO/unstructured-inference/issues/177

Duplicate of https://github.com/Unstructured-IO/unstructured-inference/issues/177

👍
normally would close a dupe but let's leave it open to make sure upstream fix resolves the issue.


fancy_v_count doesn't work when GAMBLR.results is loaded

> library(GAMBLR.results)
> fancy_v_count(this_sample_id = "DOHH-2")
This function only works with matched samples for now
trying to find output from: battenberg
looking for flatfile:
Error in if (!file.exists(battenberg_file)) { :
  argument is of length zero

This is because assign_cn_to_ssm is used internally, which calls get_sample_wildcards to try to find the battenberg file path. However, get_sample_wildcards doesn’t work with unmatched samples.
fancy_v_count works fine if using the bundled data (GAMBLR.data).

Other functions that use assign_cn_to_ssm and might get the same problem:
comp_report
fancy_v_count
fancy_v_chrcount
fancy_circos_plot
copy_number_vaf_plot
fancy_v_sizedis
fancy_snv_chrdistplot
fancy_ideogram

All the mentioned plotting function will have their internal call of assign_cn_to_ssm replaced with appropriate GAMBLR functions for retrieving the required plot data. I will self-assign this issue and provide the required update when ready.


vulkaninfo: optical flow queue flag is written with typo

optical flow flag is written QUEUE_OPTICAL_FLOW_BIT_N (wihtout a 'V' at the end)
	queueProperties[5]:
	-------------------
		minImageTransferGranularity = (1,1,1)
		queueCount                  = 1
		queueFlags                  = QUEUE_TRANSFER | QUEUE_SPARSE_BINDING | QUEUE_OPTICAL_FLOW_BIT_N
		timestampValidBits          = 64
		present support             = false

as seen in https://github.com/KhronosGroup/Vulkan-Tools/blob/1a66f59f23ac2e45af4e38ee49c711df022a8f22/vulkaninfo/generated/vulkaninfo.hpp#L973

Ah, yes that definitely is wrong.
Feel free to review the PR that fixes it, and thanks for reporting it!


Course catalog

I did some course catalog parsing; it isn't terribly reliable and will almost definitely break next school year, but I figure it's infeasible to create a consistent parsing system across years. Seems to work ok for 2022-2023 after some manual combing.

Next step is to actually put the data on the frontend


SQL: 13 INTERNAL: Received RST_STREAM with code 0

Environment details
which product (packages/*): @google-cloud/sql": "^0.1.0"
OS: MacOS Ventura 13.4.1
Node.js version: v20.4
npm version: 9.7.2
Steps to reproduce
I get the following error message:
13 INTERNAL: Received RST_STREAM with code 0
Using the following code:
import { SqlDatabasesServiceClient } from "@google-cloud/sql";

const sqlClient = new SqlDatabasesServiceClient();
const [databases] = await sqlClient.list({
  instance: INSTANCE,
  project: PROJECT_ID,
});

Hi @folkg, thanks for reporting. This API is new and  a bit of a strange case, you need to pass in fallback: rest (like so) when initializing the client to get it to work properly. In the meantime we'll work on a default more permanent fix.
Thanks!


Terminal crashes on launch in Windows.UI.Xaml.dll

Environment
OS:
Platform ServicePack Version      VersionString
-------- ----------- -------      -------------
 Win32NT             10.0.18362.0 Microsoft Windows NT 10.0.18362.0

Terminal:
> get-appxpackage Microsoft.WindowsTerminal

Name              : Microsoft.WindowsTerminal
Publisher         : CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US
Architecture      : X64
ResourceId        :
Version           : 0.6.2951.0
PackageFullName   : Microsoft.WindowsTerminal_0.6.2951.0_x64__8wekyb3d8bbwe
InstallLocation   : C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_0.6.2951.0_x64__8wekyb3d8bbwe
IsFramework       : False
PackageFamilyName : Microsoft.WindowsTerminal_8wekyb3d8bbwe
PublisherId       : 8wekyb3d8bbwe
IsResourcePackage : False
IsBundle          : False
IsDevelopmentMode : False
NonRemovable      : False
Dependencies      : {Microsoft.VCLibs.140.00_14.0.27323.0_x64__8wekyb3d8bbwe,
                    Microsoft.VCLibs.140.00.UWPDesktop_14.0.27629.0_x64__8wekyb3d8bbwe}
IsPartiallyStaged : False
SignatureKind     : Store
Status            : Ok

I suspect I'm seeing the crash mentioned in PR #3027. While I can't recall the specific build number for Terminal that I first installed on this particular machine, it was one of the 0.5 releases. all versions I've tried from that first one to the version listed above by get-appxpackage have failed this same way (though I can't faithfully recall if it's been the same Event Viewer entry as reported below). I have used Feedback hub twice, I believe with different versions.
Steps to reproduce
Click Windows Terminal (Preview) in Start menu
Expected behavior
Application launches
Actual behavior
Empty Window frame is drawn, then closes. This is in Event Viewer under Windows Logs > Application
Faulting application name: WindowsTerminal.exe, version: 1.0.1910.22001, time stamp: 0x5daf7ab2
Faulting module name: Windows.UI.Xaml.dll, version: 10.0.18362.356, time stamp: 0x0825b5b0
Exception code: 0xc000027b
Fault offset: 0x0000000000712cf0
Faulting process id: 0x2b68
Faulting application start time: 0x01d58f6c56566369
Faulting application path: C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_0.6.2951.0_x64__8wekyb3d8bbwe\WindowsTerminal.exe
Faulting module path: C:\Windows\System32\Windows.UI.Xaml.dll
Report Id: 86621d0f-a0dd-4c17-9530-85a0b42ed873
Faulting package full name: Microsoft.WindowsTerminal_0.6.2951.0_x64__8wekyb3d8bbwe
Faulting package-relative application ID: App

Thanks for the report! The crash from #3027 was in TerminalControl.dll, not Windows.UI.Xaml.dll.
Would you mind sharing feedback, and your profiles.json (from %LOCALAPPDATA%\packages\Microsoft.WindowsTerminal_8wekyb3d8bbwe\LocalState\profiles.json)?
/feedback

I am facing the same issue....
please check https://aka.ms/AA6fsnn

ere's the two I filed in Feedback Hub
https://aka.ms/AA6g4mf
https://aka.ms/AA6f8wz
GitHub won't let me attach a json file, so here it is zipped. It should be the default one.
profiles.zip
I've tried using the App Settings > Repair/Reset and also deleting the whole %LOCALAPPDATA%\Packages\Microsoft.WindowsTerminal_* directory, with no effect.

This looks like a /dupe of #2277 (even though this doesn't involve resizing.)


feat: convert curried type functions into uncurried type functions

Adds a conversion TypeFun.ofCurried : CurriedTypeFun.{u, v} n -> TypeFun.{u, v} n,
where CurriedTypeFun.{u,v} n is a (def-eq) abbreviation for Type u -> ... -> Type u -> Type v, i.e., the type of curried functions taking n arguments of type Type u to return an element of Type v, and
TypeFun.{u, v} n is an abbreviation for TypeVec.{u} n -> Type v.
It is generally more idiomatic to define type functions with multiple arguments in the curried style, but in the QPF development we use the uncurried equivalent TypeVec n -> Type _. Thus, having a canonical conversion from curried functions to uncurried gives us a canonical way to ask if a curried type function, say Sum is a QPF (namely, "is there an instance of MvQPF (TypeFun.ofCurried Sum)").

This is code ported from  https://github.com/alexkeizer/QpfTypes, where this conversion is crucial in presenting a high-level interface, in terms of idiomatic, curried, type functions to users, while being built on QPFs (and thus, uncurried typefunctions) underneath.
There is also a conversion in the other direction, from uncurried to curried, which I've decided to PR later, to keep this PR smaller and hopefully easier to review.

[x] depends on: #10818 (all changes to Fin2.lean are part of 10818, not the current PR)

The dependency PR got merged, I merged master into this branch and fixed the linter errors, so this should now be ready for review!

Is this not a special case of the HasUncurry machinery?
It's close, but not exactly.

A big difference is that HasUncurry will uncurry it into Type x ... x Type -> Type, rather than TypeVec n -> Type, and I do specifically need the latter to interface with QPF instances. Some solutions for this:

Make specialized HasUncurry instances for Type -> ... -> Type -> Type, to uncurry this into TypeVec n -> Type. A downside is that TypeVec forces all arguments into the same universe, so if we have Type u -> Type v -> Type _, then this will become Type u x TypeVec.{v} 1 -> Type. Not exactly elegant.
Introduce a new HasTypeVecUncurry class which has those specialized instances: that is not really better than what we have in this PR
Introduce a coercions from Type x ... x Type to TypeVec, and then compose HasUncurry with this coercion. I guess this could work? To make it prettier, I could introduce a new syntax that is specific to type functions, which bundles the uncurrying with the coercion.

Another big difference is that HasUncurry delegates the recursion to typeclass synthesis machinery, and thus only works for concrete arities, while the solution in the current PR works for arbitrary/dynamic arities.
@YaelDillies for this reason I still prefer the implementation in this PR over HasUncurry.
However, I'll concede my current use-case does not need the arbitary arities (yet), so if you feel the extra maintenance burden is not worth it, I'm also happy to implement option (3) above.

That sounds like reasonable motivation. Could you please summarize your comment in the module doc of your new file/docstrings of your new defs AND cross-reference your new definitions from the docstrings/module doc for HasUncurry and friends?


fix: structured formatting

in structured formatting we do not need additional lines which makes it harder to parse the output. Also there was improper output for subdomain field, which brakes parsers.

Thanks, could you please also release new version to be available on community repo?


Add Aragon One's recommended development flow

It'd help new developers getting started to have a recommended workflow for their own apps.
At Aragon One, we typically develop apps following this flow:

Define requirements for the app:

Contract: storage, getters, and events
Frontend: mock ups

Build contract
Build frontend

Build self-contained UI components
Unify components into an implementation of the design that uses mock data (usually just a JSON file)
Connect UI to real chain data
Allowing users to "write" data by creating transactions

This sounds like a good outline for a app tutorial.


Updated data product to use Chr.Avro for anything Avro related

Codecov Report

Merging #68 (8208425) into main (addbc5a) will increase coverage by 7.74%.
The diff coverage is 43.57%.

:exclamation: Current head 8208425 differs from pull request most recent head cd7ed1e. Consider uploading reports for the commit cd7ed1e to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             main      #68      +/-   ##
==========================================
+ Coverage   30.90%   38.65%   +7.74%
==========================================
  Files          27       30       +3
  Lines         220      282      +62
==========================================
+ Hits           68      109      +41
- Misses        152      173      +21

Impacted Files
Coverage Δ

...tions/Relational/RelationalDataStoreOptionsBase.cs
100.00% <ø> (ø)

...DataProduct.Configuration/Options/SchemaOptions.cs
100.00% <ø> (ø)

...tryOptions/Providers/KafkaSchemaRegistryOptions.cs
0.00% <0.00%> (ø)

...tryOptions/Providers/LocalSchemaRegistryOptions.cs
100.00% <ø> (ø)

...DataMesh.DataProduct/Apis/ApisStartupExtensions.cs
0.00% <0.00%> (ø)

...taProduct/Apis/GraphQL/GraphQLStartupExtensions.cs
0.00% <ø> (ø)

...esh.DataProduct/Apis/Rest/RestStartupExtensions.cs
0.00% <ø> (ø)

...ataProduct/DataStore/DataStoreStartupExtensions.cs
0.00% <0.00%> (ø)

...Relational/RelationalDataStoreStartupExtensions.cs
0.00% <0.00%> (ø)

...lational/Repositories/EntityFrameworkRepository.cs
0.00% <0.00%> (ø)

... and 25 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Issue of max_seq_length in MLM pretraining data preprocessing

Hi,
I find that in the functions segment_pair_nsp_process and doc_sentences_process in examples/transformers/language-modeling/dataset_processing.py, the sequence length of the processed data is actually max_seq_length - tokenizer.num_special_tokens_to_add(pair=False) since variable max_seq_length is replaced by this value and have been passed to the tokenizer.prepare_for_model function.
Such as user set max_seq_length=128, and the processed data will have a sequence length of 125.
I'm not sure is it the standard way of pretraining data preprocessing?

Yes, this is consistent with my understanding and observation on the tokenized input data. Sorry that I have not state the issue clearly, what I mean is actually that with [CLS] token and 2 [SEP] tokens already added into the tokenized text as the input that will be passed to the model, its length is 125 instead of 128 which is set for max_seq_length.

I am not sure I understand your question. Please clarify it or add an example.

I use below command to pretrain MLM, with max_seq_length set to 128 and pad_to_max_length set to True, but I find that the shape of input_ids which is the tokenized input for the model is 32125 instead of 32128.
python run_mlm.py --config_name google/mobilebert-uncased --tokenizer_name google/mobilebert-uncased --datasets_name_config wikipedia:20200501.en bookcorpusopen  --do_train --data_process_type segment_pair_nsp --max_seq_length 128 --per_device_train_batch_size 32 --per_device_eval_batch_size 32 --gradient_accumulation_steps 32 --save_total_limit 3 --num_train_epochs 5 --pad_to_max_length --dataset_cache_dir preprocessed_datasets_wikipedia_bookcorpusopen_128 --output_dir test --overwrite_output_dir --preprocessing_num_workers 48


Add conn check for GATT notify handlers for LE audio implementations

Most of the implementations used the conn pointer in one way or another, but the pointer may be NULL, which was not accounted for.

What exactly is the use case for giving NULL to the callback? What else besides returning BT_GATT_ITER_CONTINUE might a callback want to do?

What exactly is the use case for giving NULL to the callback? What else besides returning BT_GATT_ITER_CONTINUE might a callback want to do?

From the documentation:
*  @param conn Connection object. May be NULL, indicating that the peer is being unpaired

Not sure what the callback would do besides returning BT_GATT_ITER_STOP or BT_GATT_ITER_CONTINUE, but I suppose that there may be (or have been) a use case for it?
It is also a bit curious as what "unpaired" means here; does it mean a disconnect, or that a bond has been removed?


[REQUEST] DiscoFlix Integration

For which application would like to see this Guide ?
DiscoFlix - An easy to setup Discord request bot and web ui for handling Radarr/Sonarr requests. Includes a REST API for scalability and open development.
Describe what you would like to see in the Tutorial/Guide
I noticed that you did not include any Discord-interfacing applications to facilitate user requests. If at any point you are open to it, I would be happy to create some easy to follow documentation that you could include wherever you'd see fit. I would make sure to follow the same style you have implemented in your other docs.
This documentation could include an install guide (for docker based installation & building from source), a guide to generating a Discord Bot API Token, API endpoints, etc. Open to your suggestions.
I can create a PR and edit the docs myself, or I can send you documentation that you could integrate, or you could create your own based on what currently exists in the DiscoFlix Github. I'm open to whatever works for you.
Why it would be a good fit
I may be biased as it's creator, but I think DiscoFlix is a top contender when it comes to Radarr & Sonarr requests via Discord. It's easy to setup with Docker or even build from source. It has a clean user interface. It has been and will continue to be consistently maintained/improved.
I am monitoring for all relevant updates to the Radarr / Sonarr API as well as Discord's Bot API. If something breaks, it will be immediately addressed. Other similar utilities might not be able to say the same.
If you'd require DiscoFlix to meet a specific criteria that it currently does not, please let me know and I will do my best to accommodate that. If you simply do not want to include this information in your docs, I'd understand. Either way, I appreciate you putting this together!

I try to cover in the guide stuff I use myself, being it's much easier to maintain and give support about apps you actually use.
One of the reasons why I don't have a Lidarr guide, and I had several people who wanted to do it and maintain it.
Only one person actually showed me a setup he wanted to use and that was also the things I heard about it.
Or something I'm using notifiarr to sync my custom formats from the guide and more from the guide.
And I use it also for notifications, but I don't use any of the other options it supports like discord request for the Starr apps like yours.
The app has its own wiki that covers everything, so there was no need for me to maintain the guide here and I dropper it.
Looking fast at your readme it seems you already have most of it covered if not all
Only thing what I'm missing is a docker compose example.
If you have a proper readme/wiki there is no need for a 3rd party guide


Stars on landing page rating demo are repeated

See:

@henvic I'm not seeing what you mean...

@jonnilundy see the repeated "rating" part of the bottom left. There should be five stars, not 25.

I thought this was intentional. @vitorfernandes?

go to amazon, type wine and scroll down to see the same pattern. it's a common pattern @henvic.


Add support for requesting permissions

@DRSchlaubi Thank you for your PR! Could you please provide a brief explanation of the prompt and provide a test URL?

@DRSchlaubi Thank you for your PR! Could you please give a brief explanation of the prompt and provide a test URL?

you can use https://webcamtests.com