Dark text on light background is hard to read on non-HiDPI displays

Is there an existing issue for this?

[X] I have searched the existing issues.

Is this issue related to iced?

[X] My hardware is compatible and my graphics drivers are up-to-date.

What happened?
I've been checking out iced because 0.12 came out today. But immediately the text stood out as very wrongly rendered. Initially I thought the hinting just isn't very good, but comparing Paint (on the top) to Iced (at the bottom), the problem is much rather an issue with the "apparent font weight":

This immediately stood out to me as an issue with sRGB linear vs. gamma corrected, so I pulled out RenderDoc where at first glance the textures seem to use the correct color space. Extracting the glyph cache by itself, we actually get a proper rendering of the glyphs, confirming that up until this point everything is done correctly:

This strongly implied that the shader is causing the problems, and it probably needs to do a sRGB linear to gamma corrected conversion, and lo and behold, the text is now properly rendered:

My quick fix amounted to the following in glyphon:
fn linear_to_srgb(x: f32) -> f32 {
    if x <= 0.00031308 {
        return 12.92 * x;
    } else {
        return 1.055 * pow(x,(1.0 / 2.4) ) - 0.055;
    }
}

return vec4<f32>(in_frag.color.rgb, in_frag.color.a * linear_to_srgb(textureSampleLevel(mask_atlas_texture, atlas_sampler, in_frag.uv, 0.0).x));

Almost certainly too hacky (although the same hack already exists), but it should give a good starting point for investigation where the color space mess up happens and what the correct fix may be.
I believe the main problem here is that what iced and glyphon are doing is technically the correct color space handling, but no other application interprets the alpha channel of glyphs this way, so while technically being more correct, it ends up looking very washed out and inconsistent with other applications.
The tiny-skia backend does NOT have this problem, solidifying that it's at least a bug in one of the two backends.

Interestingly both Paint and tiny-skia are still clearer, but that may just be the hacky sRGB conversion function.
What is the expected behavior?
See above
Version
crates.io release
Operating System
Windows
Do you have any log output?
No response

I believe the main problem here is that what iced and glyphon are doing is technically the correct color space handling, but no other application interprets the alpha channel of glyphs this way, so while technically being more correct, it ends up looking very washed out and inconsistent with other applications.

Close, but not quite! This is not just a color space issue, but also an alpha blending issue.
Most GUI toolkits operate (e.g. perform alpha blending) on sRGB colors while assuming they are linear. See Web color is still broken.
iced tries to do the correct thing by default. If you want "broken colors", you can enable the web-colors feature.
web-colors disabled

web-colors enabled

Like I believe even without "web colors" it should probably behave this way for blending the fonts.

I'm happy to consider our options here, as long as we know what we are doing.
Gamma correcting an alpha channel seems quite strange. It does improve the final alpha blending result, but it feels like we are just smashing things. How is brighter text on a dark background affected? Is there any previous work on this?
In order to produce the web-colors result only for text, we would not be able to rely on hardware color conversions. We would have to declare every texture as linear RGB and manually do all of the sRGB conversions in the shaders, except for the text one. Or maybe declare a dedicated render pass for every text layer... equally undesirable.

We could also consider enabling the web-colors feature by default, since it's in the end what will be expected from most users.
It is quite unfortunate that we have to adopt a broken rendering technique just because it's become the norm, though.

Is there an option of doing something like stem darkening? Freetype experimented with it (as explained here in an article from 8 years ago).

@TheDecryptor Yes! I remember that paper! I think it would certainly be an interesting feature for glyphon when using ColorMode::Accurate.
cc @grovesNL

Yeah that could be useful for sure, I'd like to understand the state-of-the-art for this better before we decide on stem darkening though. If I remember correctly some approaches for this problem don't work well with display scaling (especially macOS versions in the last decade or so) but don't have a reference handy right now.

IANA text rendering expert, but I remember someone in Gnome saying RGBA blending can't do the right thing with subpixel AA on both dark and light backgrounds, and per-channel opacity is needed.
Freetype's Harmony renderer has an interesting first-principles approach to AA, where subpixel rendering is seen as a correction for hardware's misaligned color planes, instead of some kind of "triple resolution" hack. It could be implemented directly at some performance cost by rendering color channels separately and swizzling at the end. That handles vertical and triangular subpixel arrangements just as easily as horizontal, although I don't think it's quite right for RGBW which could have color-dependent offsets.

It is quite unfortunate that we have to adopt a broken rendering technique just because it's become the norm, though.

Perhaps text and graphics should be treated separately? The font designer was probably looking dark-on-light output rendered with that broken technique when tuning the small sizes, so in some sense it is ultimately correct. The text renderer could attempt to adapt the appearance of incorrectly-blended dark-on-light to whatever the actual output target is.
And then color emojis are a whole 'nother kettle of fish.

Perhaps text and graphics should be treated separately? The font designer was probably looking dark-on-light output rendered with that broken technique when tuning the small sizes, so in some sense it is ultimately correct. The text renderer could attempt to adapt the appearance of incorrectly-blended dark-on-light to whatever the actual output target is.

Yes. As I said in my previous comment, we could do it but it seems it would cost us performance and composability when it comes to our rendering primitives. But I'm not a graphics wizard, so maybe there is a way!


QAT v. 0.1

This PR introduces the QIR adaptor tool (QAT) to the main repository. This tool allows the user to take a generic QIR and adapt to a specific profile. Through these transformations, the use is able to adapt a generic QIR to a constrained QIR that is more suitable for a specific architecture.

There are still a couple of comments open that should be addressed in a follow up PR. As per offline discussions, merging this with the expectation that there will be a follow up PR addressing these comments.
@troelsfr @kuzminrobin @DmitryVasilevsky


Should fbjs be a peer dependency?

I'm very leery of peer dependencies because of the complexity they introduce when it comes time to upgrade apps. Why isn't fbjs just a regular dependency? Looks like it's only being used for warning and invariant. It's not clear to me why those need to be a peer dep.

While we're on the subject, we shouldn't even be using fbjs.
I think the right course of action may be to have normal dependencies on "mirror" implementations of warning and invariant.

Awesome, that looks much better :+1: Thank you!

Thanks for the suggestion :) If you end up using Seamstress let me know. :beers:


support for touch devices

ref. to https://github.com/Codiad/Codiad/issues/772 and https://github.com/Codiad/Codiad/pull/780

Please test on several devices and add results to this PR.
Currently verified:

Chrome for iOS on an iPad

someone else could ttest that? Due I have no device anymore, I am unable to process with that. Otherwhise I would like to close it.

I can confirm that opening files by tapping now works, tested on Chrome 36/Android 4.4!
What else works

opening and closing folders in the files navigation
switching between open windows
opening the right side bar

What does not work

i cannot hide the location bar by scrolling down. that's a shame, because the screen space would have been really useful.
clicking the "x" to close an open file in a tab
closing the right sidebar: when I click on this icon in openend state, nothing happens
and, perhaps, most fundamentally: editing text. the keyboard gets very confused, probably by the JS handing cursors - for example, a space becomes a backspace on my device (I use SwiftKey, maybe there is a relationship)

So in conclusion: the PR succeeds in its purpose (enabling files to be opened on mobile devices), and I'd be happy to see it merged. This makes Codiad suitable as a mobile file viewer at least.
In the future, there are some more enhancements that could make Codiad a really great option for "emergency" code access from a mobile


Adding helm charts to deploy Istio and its components

This PR adds helm charts which could be used to deploy Istio. Variables defined in values.yaml control which components get deployed and their corresponding values.

@sbezverk: Adding do-not-merge/release-note-label-needed because the release note process has not been followed.

One of the following labels is required "release-note", "release-note-action-required", or "release-note-none".
Please see: https://github.com/kubernetes/community/blob/master/contributors/devel/pull-requests.md#write-release-notes-if-needed.
Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/assign @andraxylia  @rshriram  for comments

Cc @ayj for admission controller problems
On Fri, Dec 8, 2017, 9:01 AM Serguei Bezverkhi<EMAIL_ADDRESS>wrote:

/assign @andraxylia https://github.com/andraxylia @rshriram
https://github.com/rshriram for comments
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/2030#issuecomment-350284211, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AJGIxsSavxOdvQgCH3EdAtWgjVx9d7p4ks5s-U80gaJpZM4Q7NXS
.

Istio is designed to be deployed in an "all or nothing" approach - meaning you receive all of it, or receive none of it, at least on a Kubernetes runtime.

Allowing for incremental adoption of Istio is important (see https://groups.google.com/forum/#!topic/istio-steering/GXI5Wl8iJjk).

@ayj The modularity in deployment is the exact goal I am after. Each component can easily be turned off. Basically you can deploy just generic Istio configuration pieces. If you see any things in the proposed approach which prevented high level modularity, please point them out I will address them right away.

Question to everybody. There are lots of values hard coded into config maps, do you see value to expose these constants and make them variables and configurable in values.yaml file?

Thanks a lot for this effort!
Absolutely yes, wrt config parameterization as long as defaults are set.
Also cc @gyliu513 who has tried to do the helm thing for istio. You might want to take a look at the pending PR in istio.github.io repo. Other comments inline

The configmap called MeshConfig is actually close to the feature flags that you want to have. Empty values for mixer, zipkin, pilot, ingress, auth, stats should lead to immediate turn-off of the subsystem (although we don't always test it). I don't think we care about specific non-empty value (e.g. mixer:9090) so no need to expose that, but presence should be roughly the feature flag.

Question here is what happens when we move to CRD?

We haven't yet. These are two distinct problems: deploying a component and using a component. You may want to deploy and not use a component. You may want not to deploy and use an existing component.

@rshriram as long as you do not create CRD in the code, which is recommended approach, then not much will changes, configmap template will be replaced by the one instantiating CRD.

xref: https://github.com/istio/istio/pull/1895
We should try to converge these efforts. cc @ayj @kyessenov thoughts?

Why adding the chart here? I think that hosting the chart in https://github.com/kubernetes/charts/tree/master/incubator/ is a good place, also we already have a  istio chart here https://github.com/kubernetes/charts/tree/master/incubator/istio
/cc @lachie83

@gyliu513: GitHub didn't allow me to request PR reviews from the following users: lachie83.
Note that only istio members can review this PR, and authors cannot review their own PRs.

In response to this:

Why adding the chart here? I think that hosting the chart in https://github.com/kubernetes/charts/tree/master/incubator/ is a good place, also we already have a  istio chart here https://github.com/kubernetes/charts/tree/master/incubator/istio
/cc @lachie83

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@gyliu513 hmm, thanks for letting me know, I asked on the mailer before doing it, if there were already charts and thew answer was no.. Are those charts are 100% functional?

@gyliu513 Another thing, hosting in helm's incubator poses certain issues, example updating charts, something like bumping up the version to match to current istio version would require changes in the repo you do not have control. When charts are bundled with repo, imho it is more convenient.

On Fri, Dec 8, 2017 at 5:37 PM Serguei Bezverkhi<EMAIL_ADDRESS>wrote:

@gyliu513 https://github.com/gyliu513 Another thing, hosting in helm's
incubator poses certain issues, example updating charts, something like
bumping up the version to match to current istio version would require
changes in the repo you do not have control. When charts are bundled with
repo, imho it is more convenient.

I think we should remove the chart we have there. It’s much easier for us
to maintain this, turn on or off defaults. Once things stabilize, we can
move to main helm.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/2030#issuecomment-350389688, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AH0qd_TLn01LGYtK7EuYAheX4YTSwgh1ks5s-bowgaJpZM4Q7NXS
.

@rshriram With helm you can deploy or just render templates, rendering is done by using  "helm template Istio"
Here is the link to see results of running just rendering part:
https://gist.github.com/sbezverk/aea4337e8ab7d07792e7b1d657302102
(Actual deployment will be tested when I get back home)
Current templates allowing:

Deploy whole Istio with a single command. (no need for a user to bother deploying individual components if he/she does not want to)

Deploy individual components by disabling others in values.yaml or by using command line parameter, Istio base configuration also can also be disabled either by modifying values.yaml or from the command line:

helm template Istio --set Istio.deploy_base_config=false

All Add-ons  will be added in the follow up PR if you decide to go with this version of templates.

I think we should remove the chart we have there. It’s much easier for us
to maintain this, turn on or off defaults. Once things stabilize, we can
move to main helm.

@rshriram What do you mean of main helm? You mean that we can incubator the helm chart in istio/istio repo, then once it become stable, we can move it to kubernetes/charts, right?

@sbezverk yes, it can work as expected for now, you can have a try according to the readme. However, there are some bugs for now, please check https://github.com/kubernetes/charts/issues?utf8=✓&q=is%3Aissue+is%3Aopen+istio for detail.

I think that regardless of whether we for now maintain the helm charts in the istio/istio or in the kubernetes/charts repo, we should start with the current incubator implementation from @lachie83, instead of the ones in this PR. They seem to be further ahead, including addons and are much more configurable (see https://github.com/kubernetes/charts/blob/master/incubator/istio/values.yaml).
Also, if we're going integrate this with install/updateVersion.sh, we should remove the istio install templates (https://github.com/istio/istio/tree/master/install/kubernetes/templates) and just generate the manual installation files (istio.yaml, etc.) using the helm templates. Having to update and keep synchronized 2 different sets of templates for the same resources is not good.

@frankbu Adding "addons" and additional config variables is a matter of 1 day at most, I see the main advantage of this approach is a modularity. You can install Istio in a piece by piece manner or in one shot. I agree that having two template sets is not efficient and generating the only one istio.yaml based on helm charts makes sense. There is one issue, you cannot template render helm's Chart.yaml(version) and values.yaml with helm itself, so updateVersion script still needed to update these two files to by in sync with Istio's version in the repo.
If you manage to get previously developed charts to the level you want, great I will close this PR. Otherwise I will continue to working on this PR until it is 100% functional and answers community requirements.

what @frankbu means is that we already have a bunch of non-helm template files in install/ directory.. We concatenate them, and generate the files. If we are going to use Helm to generate these install files, we should get rid of the non-helm template files.
Once we settle on the path, we can incrementally improve this in subsequent PRs. Lets not make this PR bigger than necessary, but if you are willing to take ownership of this entire thing and file a sequence of issues, then I would be happy to merge the first cut and incrementally improve this to the point that we get full functionality of the helm chart that @lachie83 has, as well as the ability to do incremental installation.

From that point of view, for the moment, add support for updating both old templates and your helm chart templates. Once the helm chart templates are fully functional, we can get rid of our custom made templates completely.

@sbezverk Yes, I agree that install/updateVersion.sh is still needed. I just meant the script should be changed to use helm instead of expanding and merging it's own set of templates. If you are confident that you can quickly include the addons and the configuration values that @lachie83 has in his charts, then I'm fine with using this implementation instead. I'm just concerned about a functional regression, since right now it is missing some of the configuration that we know is needed (e.g., https://github.com/kubernetes/charts/blob/master/incubator/istio/templates/configmap/initializer-config.yaml#L10).
@rshriram: if we don't change updateVersion.sh right away, we need to make sure that everybody is aware of the need to change both sets of templates. Here's one change that's coming: https://github.com/istio/istio/pull/2037. The corresponding change in the helm templates would probably be in values.yaml, assuming the statsd to prom exporter image is parameterized in the template (which I think it should be).

To be clear, updateVersion.sh change must be part of this PR. We will only advertise istio helm charts after its fully compatible with the one outside our org. That gives us time to fix the charts and iterate.

I am also in favor of having charts in Istio repo so that we can release charts with every Istio point release on a monthly basis.   This also gives us an opportunity easily to work through incremental option of Istio in the istio chart ourselves.

On Mon, Dec 11, 2017 at 6:48 PM Andra Cismaru<EMAIL_ADDRESS>wrote:

@andraxylia requested changes on this pull request.
let's not have a new set of templates in Istio, but rather take the
existing files under templates and transform them to use the heml syntax.

That’s the intent. Use helm templates to generate kube templates. Rather
than nuking everything in one go, we are doing it step by step. It’s just a
transient state

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/2030#pullrequestreview-82679878, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AH0qd9uNZhGqpelZ4DCTOpPZ3zw7WPxKks5s_b9FgaJpZM4Q7NXS
.

@sbezverk this will need to integrate this work: https://github.com/istio/istio/pull/2026

@sdake hm, do not think so, that PR just adds "imagePullPolicy: IfNotPresent". The issue I see with images is different.

I am Ok to do it in steps, as long as in the end we will have a single version of a template.

Adding helm chart for istio.

/release-note

@gyliu513: the /release-note and /release-note-action-required commands have been deprecated.
Please edit the release-note block in the PR body text to include the release note. If the release note requires additional action include the string action required in the release note. For example:
```release-note
Some release note with action required.
```

In response to this:

/release-note

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Seems both @rshriram and @andraxylia have already approved this PR, so I think that we can merge this now. Thanks @sbezverk and @sdake for the great work!

another thing about helm is helm repo, guys can i reach istio ver0.4  with in some helm repo ?


Warlock SIM - Chest reforge crit > hit imported as mastery > hit

Using v2.11.2 WowSimsExporter:

Character in-game stats:

Chest reforge crit > hit:

Generated import data:
{"talents":"000000000000000000-3312222300310212211-3320200000000000000","glyphs":{"minor":[{"spellID":58094},{"spellID":56238},{"spellID":58070}],"prime":[{"spellID":56218},{"spellID":56228},{"spellID":63303}],"major":[{"spellID":63312},{"spellID":56240},{"spellID":63320}]},"class":"warlock","unit":"player","race":"Orc","name":"Aenima","gear":{"items":[{"id":65093,"gems":[68780,52217],"reforging":115,"enchant":4207},{"id":62416,"gems":[],"reforging":144},{"id":63463,"gems":[52205],"enchant":4200},{"id":59457,"gems":[],"reforging":144,"enchant":4115},{"id":67133,"gems":[52205,52217],"reforging":165,"enchant":4063},{"id":63484,"gems":[],"reforging":145,"enchant":4089},{"id":62364,"gems":[52217],"reforging":165,"enchant":4061},{"id":54503,"gems":[52217,52207],"reforging":138},{"id":60250,"gems":[52208,52207],"reforging":154,"enchant":4113},{"id":65069,"gems":[52207],"reforging":165,"enchant":4092},{"id":56380,"gems":[],"reforging":145},{"id":56276,"gems":[],"reforging":167},{"id":62465,"gems":[]},{"id":62047,"gems":[],"reforging":165},{"id":61329,"gems":[],"reforging":144,"enchant":4083},{"id":62234,"gems":[],"enchant":4091},{"id":56296,"gems":[],"reforging":167}]},"professions":[{"name":"Engineering","level":525},{"name":"Tailoring","level":525}],"level":85,"spec":"demonology","realm":"Gehennas"}
Change in stats after import to sim:

Chest reforge in the sim showing mastery > hit

Changing this reforge to crit > hit corrects the stats discrepancy:

Issue created on wrong area, remade on the correct location: https://github.com/wowsims/exporter/issues/16


Narrowing does not work with instanceof operator and type parameters

class A<T> {
	private x: number;
	public m(p: T): T {
		return p;
	}
}
class B extends A<string> {
	private y: number;
}
var a: A<number> | B;
if (a instanceof B) {
	a.m(''); // `a` is `B`
}
if (a instanceof A) {
	a.m(1); // `a` is `A<number> | B`, should be `A<number>`
}

Why would you want to narrow here? B extends from A, so even if a had the type B at runtime, a instanceof B would turn out to be true.

When I remove the type parameter, type narrowing succeeds in that case.
class A {
    private x: number;
    public m(p: number): number {
        return p;
    }
}
class B extends A {
    private y: number;
}
var a: A | B;
if (a instanceof A) {
    a.m(1); // `a` is `A`, not `A | B`
}

Therefore, this narrowing failure caused by type parameters. Why does narrowing fail just when I use type parameters? It seems a bug.

I think that's just because we perform subtype reduction within that branch, so you lose B. I'm not sure if we should be doing that. Thoughts @ahejlsberg and @weswigham?

Though, in some sense, I don't see why it's wrong to do that. Is there anything you couldn't do if we didn't perform subtype reduction?

This is my actual case.
// monad.ts
export abstract class Monad<T> {
  private MONAD: T;
  constructor(protected thunk?: () => Monad<T>) {
  }
  public extract(): T
  public extract<U>(transform: (val: any) => U): T | U
  public extract<U>(transform?: (val: any) => U): T | U {
    return this.memory_
      ? this.memory_.extract(transform)
      : this.evaluate().extract(transform);
  }
  protected memory_: this;
  protected evaluate(): this {
    return this.memory_ = this.memory_ || <this>this.thunk();
  }
  public constrain<S extends Monad<T>>(type?: S): Monad<T> {
    return this;
  }
}

// maybe.ts
import {Monad} from './monad';

export class Maybe<T> extends Monad<T> {
  private MAYBE: T | void;
  constructor(protected thunk?: () => Maybe<T>) {
    super(thunk);
  }
  public bind(f: (val: T) => Maybe<T>): Maybe<T>
  public bind<U>(f: (val: T) => Maybe<U>): Maybe<U>
  public bind<U>(f: (val: T) => Maybe<U>): Maybe<U> {
    return new Maybe<U>(() => {
      const m: Just<T> | Nothing | Maybe<T> = this.evaluate();
      if (m instanceof Just) {
        return f(m.extract());
      }
      if (m instanceof Nothing) {
        return m;
      }
      if (m instanceof Maybe) {
        return (<Maybe<T>>m).bind<U>(f);
      }
      throw new TypeError(`ArchStream: Maybe: Invalid monad value.\n\t${m}`);
    });
  }
  public extract(): T
  public extract<U>(transform: () => U): T | U
  public extract<U>(transform?: () => U): T | U {
    return super.extract(transform);
  }
  public constrain<S extends Just<any>>(type?: S): void
  public constrain<S extends Nothing>(type?: S): void
  public constrain<S extends Maybe<T>>(type?: S): Maybe<T>
  public constrain<S>(type?: S): Maybe<T> {
    return this;
  }
}

export class Just<T> extends Maybe<T> {
  private TYPE: Just<T>;
  constructor(private val_: T) {
    super();
  }
  public bind(f: (val: T) => Just<T>): Just<T>
  public bind(f: (val: T) => Nothing): Nothing
  public bind<U>(f: (val: T) => Just<U>): Just<U>
  public bind(f: (val: T) => Maybe<T>): Maybe<T>
  public bind<U>(f: (val: T) => Maybe<U>): Maybe<U>
  public bind<U>(f: (val: T) => Maybe<U>): any {
    return new Maybe(() => this).bind(f);
  }
  public extract<U>(transform?: () => U): T {
    return this.val_;
  }
  public constrain<S extends Just<T> | Maybe<T>>(type?: S): Just<T> {
    return this;
  }
}

export class Nothing extends Maybe<any> {
  private TYPE: Nothing;
  public bind(f: (val: any) => Maybe<any>): Nothing {
    return this;
  }
  public extract(): any
  public extract<U>(transform: () => U): U
  public extract<U>(transform?: () => U): U {
    if (!transform) throw void 0;
    return transform();
  }
  public constrain<S extends Nothing | Maybe<any>>(type?: S): Nothing {
    return this;
  }
}

expected:
      if (m instanceof Maybe) {
        return m.bind<U>(f); // `m` is `Maybe<T>`
      }

actual:
      if (m instanceof Maybe) {
        return (<Maybe<T>>m).bind<U>(f); // `m` is `Nothing | Maybe<T>`
      }

It looks you do want subtype reduction and you aren't getting it. The reason is extremely subtle (and it took a non-trivial amount of time to exploit it). Here's a smaller repro:
class A<T> {
    prop: T;
}

class B<T> extends A<T> {
}

class C extends A<any> {
    cProp: any;
}

var x: A<string> | B<string> | C;
if (x instanceof A) {
    x;
}

First we'll narrow your types against A<any>, and since A<any> is a subtype of A<string>, B<string>, and C, they'll all stick around.
Then we'll try to perform subtype reduction. Since B<string> is a subtype of A<string>, we'll get rid of B<string>. So far so good.
What about C? Is C a subtype of A<string>? Surprisingly, it's not!
The issue is that you derived from A<any> which means that C has the property prop: any. Meanwhile, A<string>'s prop has the type string. While any is assignable to string, it's not a subtype. That means that C can't be reduced to a A<string> during subtype reduction.
We'll also try to check if A<string> is a subtype of C, which it's not since A has a cProp.
I'm going to close this issue so that we can tackle the real problem that you ran into.

@ahejlsberg @RyanCavanaugh @weswigham any thoughts?
Personally, I think this might just not be the right way to go about this anyway. Maybe<T> isn't usually an open-ended type. You shouldn't need to account for other extensions, so I would turn Maybe into a MaybeBase, and then define Maybe as
type Maybe<T> = Just<T> | Nothing;

At least,
class A<T> {
    prop: T;
}

class B<T> extends A<T> {
}

class C extends A<any> {
}

var x: A<string> | B<string> | C;
if (x instanceof A) {
    x; // x is C, not A
}

This type resolving is very odd.

What's a simplified example that results in actual bad behavior inside the if ? I'm seeing a lot of structurally-identical types here so it's hard to distinguish where this would actually matter.

@RyanCavanaugh at least, if that is the design, type of m should be Just<T> | Nothing | Maybe<T>. Nothing | Maybe<T> has no reason.
Additionaly, current matching order is left to right and non-generics to generics, but should not have order of non-generics to generics if TypeScript cannot select the most closest type. TypeScript should provide the way that select the most closest type for alternate method of pattern matching.
class A<T> {
    prop: T;
}

class B<T> extends A<T> {
}

class C extends A<any> {
}

var ab: A<string> | B<string>;
if (ab instanceof A) {
    ab; // A<string>
}
var ba: B<string> | A<string>;
if (ba instanceof A) {
    ba; // B<string>
}
var ca: C | A<string>;
if (ca instanceof A) {
    ca; // C
}
var ac: A<string> | C;
if (ac instanceof A) {
    ac; // C, should be A<string> at least
}

@DanielRosenwasser
type Maybe<T> = Just<T> | Nothing;

It is right in functional language, but TypeScript is not. Sometimes we must check the instance type instead of pattern matching. In your solution, we must check the instance type by m instanceof Just || m instanceof Nothing || m instanceof Maybe(need m instanceof Maybe because this monad has Call-by-need evaluation strategy). In my solution, instance type checks is only m instanceof Maybe. This difference for monad need in object oriented language.type Maybe<T> = Just<T> | Nothing does not exist in the runtime of object oriented language, and also m instanceof Just || m instanceof Nothing is not correct because this monad has Call-by-need evaluation strategy. Monad types cannot determine either Just or Nothing in thunk.

A<T> | C seems a combination of narrowed generics and narrowed non-generics.
class A<T> {
	a: T;
}
class B<T> extends A<T> {
	b: T;
}
class C extends B<string> {
	c: string;
}
class D extends C {
	d: string;
}
var x: A<number> | B<number> | C | D;
if (x instanceof A) {
    x; // `a` is `A<number> | C`, should be `A<number>`
}

If it is right, TypeScript should reduce these types. Then x should be A<number> because C has no reason. In any case, current type narrowing is incorrect. x should narrow to A<number> or A<number> | B<number> | C | D, but B<number>, C and D is useless in that if statement.
I think, instanceof operator must select a most closest type because finding and showing of all derived types is useless and too complex when there are too many derived types, and TypeScript should reduce those unnecessary costs for optimization. Additionally, in general, sets of types must narrow by operations, but instanceof operator widen types. Roles and effects of typings for abstraction is reducing of calculation size based on sets size.

I split this issue: #6607
https://github.com/Microsoft/TypeScript/issues/6589#issuecomment-174401344 thanks @DanielRosenwasser , I understand. Then I want to use this issue for Type narrowing strategy for most closest type selection.

Some short examples to clarify what you mean would be helpful

Narrowing to the closest runtime type by instanceof operator:
class A<T> {
    prop: T;
}
class B<T> extends A<T> {
}
class C extends B<any> {
}

var x: A<string> | B<any> | C;
if (x instanceof A) {
    x; // closest type is A, now B
}
if (x instanceof B) {
    x; // closest type is B, now B
}
if (x instanceof C) {
    x; // closest type is C, now B
}
if (x instanceof Object) {
    x; // closest type is A, now B
}
if (x instanceof Array) {
    x; // no closest type, must be contextual type `A<string> | B<any> | C`
}

Motivation
Sometimes we must check the instance type instead of pattern matching. TypeScript should provide the way that select the most closest type for alternate method of pattern matching.
// maybe monad
  public bind<U>(f: (val: T) => Maybe<U>): Maybe<U> {
    return new Maybe<U>(() => {
      const m: Just<T> | Nothing | Maybe<T> = this.evaluate();
      if (m instanceof Just) {
        return f(m.extract());
      }
      if (m instanceof Nothing) {
        return m;
      }
      if (m instanceof Maybe) {
        return (<Maybe<T>>m).bind(f); // should be `m.bind(f)`
      }
      throw new TypeError(`ArchStream: Maybe: Invalid monad value.\n\t${m}`);
    });
  }

Searching and showing of all derived types is useless and too complex when there are many derived types, and TypeScript should reduce those unnecessary costs for optimization.
class A<T> {
    prop: T;
}
class B<T> extends A<T> {
}
class C extends B<any> {
}

var x: A<string> | B<any> | C;
if (x instanceof A) {
    x; // this scope narrowed by A, B and C are useless, but x is B
}

In general, sets of types must narrow by operations, but instanceof operator widen types. Roles and effects of typings for abstraction is reducing of calculation size based on sets size.
class A<T> {
    a: T;
}

class B<T> extends A<T> {
	b: T;
}

class C extends A<any> {
    c: any;
}

var x: A<string> | B<string> | C;
if (x instanceof A) {
    x; // x should be A of the closest type.
    // if you want B or C, it should narrow by those types.
}

instanceof operater compare constructors in js engine, but TypeScript compiler compare structural types. It is mismatch behavior.

@falsandtru the two examples are actually diffrent. the example using A, B and C is about subtype reduction. your example using Maybe, Nothing and Just, is about control flow.
function f<T>() {
    const m: Just<T> | Nothing | Maybe<T> = this.evaluate();
    if (m instanceof Just) {
       m // Just<T>
    }
    if (m instanceof Nothing) {
        m // Nothing
    }
    else if (m instanceof Maybe) {
        m // Maybe<any>
    }
}

notice the else. the problem is that the return currently does not help narrow the type.  https://github.com/Microsoft/TypeScript/issues/2388 should make this work as intended.

In my impl, Just and Nothing are subtypes of Maybe for call-by-need evaluation strategy. See https://github.com/Microsoft/TypeScript/issues/6589#issuecomment-174382681.

this essentially the same issue as https://github.com/Microsoft/TypeScript/issues/7271


Fix initial ReadTheDocs build

Completely forgot a couple of essential files RTD needs lol

Voila - the PR integration should be working now. You can always check that doc chhanges render correctly by clicking on the details button from the PR checks screen

Currently don't have one.

Really? It's not belong the chromatic bot and above the pre-commit, that you can see?

Nevermind. We now have a scrollbar because there are so many checks...my bad!


[IOTDB-2445] Fix overlapped data should be consumed first bug

Coverage increased (+0.02%) to 67.859% when pulling b4c0468540594c934321a6f2deabcaddc1a7b561 on OverlappedPageBug into e45636ec27c1fda9fe10c39f9417038f3c40f3d4 on master.


HHH-12454 - Offer flag to consider id generator with local scope (legacy non JPA behavior)

https://hibernate.atlassian.net/browse/HHH-12454
@sebersole any idea for a better name than ID_GENERATOR_NAMES_GLOBAL_SCOPE = "hibernate.id.generator.global_scope_name";?
Backporting to 5.2 do we have to keep the default value to true?

@sebersole tried to follow your suggestions, let me know what do u think.
Thanks

@dreab8 I see 2 PRs linked to this Jira - this one, and https://github.com/hibernate/hibernate-orm/pull/2232.  I assume one is out-of-date?  Anything else you are waiting on?

@sebersole can you give a look at this last commit, it's related with JpaCompliance immutability.
The other PR is for the 5.2 branch

Sure.. but first, do you think I'm being overly paranoid about the mutability?

@sebersole no I don't think you are :)

Applied upstream

Found the property name global_id_generators really confusing today; as to me it implied that the generators themselves would be shared globally, not only their definitions / names.
https://forum.hibernate.org/viewtopic.php?p=2338260 and https://vladmihalcea.com/hibernate-hidden-gem-the-pooled-lo-optimizer/ finally helped me to grasp the concept I guess (hope) :D

Hey @icyerasor ! Does this issue's flag fix problems with higher Hibernate versions with specifying @SequenceGenerator for extending a base entity with an id as mentioned in the comments in the top-answer here: https://stackoverflow.com/questions/8589928/mappedsuperclass-change-sequencegenerator-in-subclass

Which comment are you referring to? The one that it doesn't work anymore with hibernate 5.4 (here)?
It does still work for me with hibernate 5.6.10. Seems it was only broken for a few minor versions, see info (here and the links 1, 2).
Warning is logged as expected: HHH000069: Duplicate generator name.
Haven't tried 6+ yet though.


Shouldn't use sudo for npm

http://stackoverflow.com/questions/16724259/npm-command-sudo-or-not

You're right. Thank you.


removed false info that breaks gathers

removed text that indicated in postCallsCallIdGather.md that the terminatingDigits parameter can be empty and emphasized that terminatingDigits cannot be set to "". This actually breaks the gather
For the Committer

[x] I made sure that the site looks great and functions perfectly live
[x] I ran splell check
[x] I'm ready for these changes to be made live

For the Reviewer
General Review

[ ] All changes/updates requested were intentionally changed or added (no extraneous file or partial updates)
[ ] The live rendered page behaves and looks like intended.
[ ] All links work as intended. (click every link to make sure that it takes the user to the expected place).
[ ] The markdown was rendered to HTML as intended. (Tables look right, code samples are gated properly).
[ ] Any style or CSS changes are tested on multiple different pages to ensure nothing unexpected broke or changed.
[ ] I proof-read the live site (gitbook serve) and there are no oblivious splelling or grammer misteaks.

Gitbook Specific

[ ] Any new documentation pages have been added to the SUMMARY.md file so they are rendered to HTML.
[ ] Any links to other pages within this documentation use relative links ([read more about ...](guides/voice/voicemail.md)).
[ ] Gitbook can install and build the documentation (gitbook install & gitbook build)

Code Specific

[ ] All code has been tested against the live API.
[ ] All code can be run without throwing errors or new warnings.
[ ] All code is formatted correctly and consistently (spaces, tabs).
[ ] All code is legible and idiomatic to the language.
[ ] All code uses sensical method, function, and variable names.
[ ] All code samples use the appropriate syntax highlighting.
[ ] Any examples re-use same phone numbers, uuids, variable names, etc... throughout (IE don't have something like {from: '+19191231234', to: '+19191231234'} unless it's meant to show the same phone number calling/texting itself ).

TL;DR

[ ] I did these things
[ ] I'm ready for these changes to be made live

Flagging this again for requested changes.

Preview Changes at:
http://bw-apdocs-samantha-little-fixes.s3-website-us-east-1.amazonaws.com/

Preview Changes at:
http://bw-apdocs-samantha-little-fixes.s3-website-us-east-1.amazonaws.com/


add policy filter to sessions

add policy filter to sessions;
fix #14 ;

Will a release be cut for this soon?


Extend logs collector to collect ecs-init logs

Summary
Extended logs collector to collect ecs-init logs
Testing Done
Manually ran the script on an ECS Instance and verified that collect/system/ecs-init directory contains ECS Init log files

ecs-init isn't present on all the operating systems that the log collector supports.  Can we make sure that collection of ecs-init logs is either skipped or fails gracefully for those operating systems?

@samuelkarp If ECS Init logs cannot be found at /var/log/ecs/, you'll end up with an empty directory under collect/collect/system/ecs-init/. The script doesn't exit with any errors. In fact, that's how it deals with most of the log directories IIUC. It seems to be a best effort thing.


FSharp.Core 4.0.0 FileNotFound exception

Hi,
I downloaded the CoffeeSharp-0.5.zip from the CoffeeSharp website but when I try to run Coffee.exe I get a "FSharp.Core 4.0.0 FileNotFound" exception.
I am running on Win8 64bit and have VS2013 with F# installed.
Do you have any idea why I am getting this error? Is there something wrong with the package?

I'm cleaning up old open issues. Sorry for not replying sooner.
If this is still an issue, please reopen this.


chore(badges): remove donate badge

Because this is a PayPal project and I am sorta already paid to work on it 😅

Codecov Report

Merging #153 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #153   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          10     10
  Lines         138    138
  Branches       35     35
=====================================
  Hits          138    138

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9b77296...732b80a. Read the comment docs.


[PT Run] [VSCodeWorkspaces]  Show previous opened VSCode workspaces #14931

Summary of the Pull Request
What is this about:

Showing previous opened workspaces that are configured in a file with extension ".code-workspace"

What is included in the PR:

Support to search and open previous opened workspaces ".code-workspace"
Rename Workspace to Project Folder

How does someone test / validate:

Open PT Run
Type '{'
Search for projects/remote ssh machines/workspace (including .code-workspace)
Enter to open the "result"

Quality Checklist

[x] Linked issue: #14931
[x] Communication: I've discussed this with core contributors in the issue.
[ ] Tests: Added/updated and all pass
[ ] Installer: Added/updated and all pass
[ ] Localization: All end user facing strings can be localized
[ ] Docs: Added/ updated
[ ] Binaries: Any new files are added to WXS / YML

[x] No new binaries
[ ] YML for signing for new binaries
[ ] WXS for installer for new binaries

Contributor License Agreement (CLA)
A CLA must be signed. If not, go over here and sign the CLA.

What is the difference between folder and workspace in vs code? Do they have different features?
Should we use different icons for workspace and folders?

What is the difference between folder and workspace in vs code? Do they have different features?
Should we use different icons for workspace and folders? (Btw: What is the icon source?)

So, a workspace is "A Visual Studio Code "workspace" is the collection of one or more folders that are opened in a VS Code window (instance). In most cases, you will have a single folder opened as the workspace but, depending on your development workflow, you can include more than one folder, using an advanced configuration called Multi-root workspaces." https://code.visualstudio.com/docs/editor/workspaces
I think we should keep the same icon. For me a workspace is a project folder too.

Where do you got the icons from?

Where do you got the icons from?

I don't remember where i got them

Agree to you that we can keep the icons we have:

Folder for folder and workspace file.
Computer for machine connections.


Fix #8308 - Chunked file uploads generate multiple files in the backend

This PR fixes the issue described in #8308.

I can confirm the issue (files > 2MB splitting into smaller parts upon upload) but this fix causes a new JS error for me ( Uncaught TypeError: Cannot set property 'ccm_token' of undefined ) when dragging and dropping a simple jpg file to the file manager interface. This happens for a basic drag-and-drop action; clicking 'upload files' and then drag-and-dropping; or creating an image block instance and then accessing the image upload interface as a modal. I am now unable to upload any files. See the attached image for details.

Happy to help with further bug tracking if required.
concrete5 Version
Core Version - 8.5.2
Version Installed - 8.5.2
Database Version -<PHONE_NUMBER>2210
Database Information
Version: 5.7.28-31-57-log
SQL Mode: ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
concrete5 Packages
Cookies Notice (1.3.4), ExchangeCore reCAPTCHA (1.1.1), Styled Maps (1.2.0), Table Block (0.9.5), Vivid Carousel (1.0.1)
concrete5 Overrides
[snip]
concrete5 Cache Settings
Block Cache - On
Overrides Cache - On
Full Page Caching - On - If blocks on the particular page allow it.
Full Page Cache Lifetime - Every 6 hours (default setting).
Server Software
Apache
Server API
apache2handler
PHP Version
7.2.28
PHP Extensions
apache2handler, bcmath, bz2, calendar, Core, ctype, curl, date, dom, exif, fileinfo, filter, ftp, gd, gettext, hash, iconv, igbinary, imagick, imap, intl, json, ldap, libxml, mbstring, mcrypt, mysqli, mysqlnd, openssl, pcre, PDO, pdo_dblib, pdo_mysql, pdo_sqlite, Phar, posix, redis, Reflection, session, shmop, SimpleXML, snmp, soap, sockets, SPL, sqlite3, standard, sysvmsg, sysvsem, sysvshm, tokenizer, varnish, wddx, xml, xmlreader, xmlwriter, xsl, Zend OPcache, zip, zlib
PHP Settings
max_execution_time - 30
log_errors_max_len - 1024
max_file_uploads - 20
max_input_nesting_level - 64
max_input_time - 60
max_input_vars - 1000
memory_limit - 768M
post_max_size - 8M
upload_max_filesize - 2M
ldap.max_links - Unlimited
mysqli.max_links - Unlimited
mysqli.max_persistent - Unlimited
pcre.backtrack_limit - 1000000
pcre.recursion_limit - 100000
session.cache_limiter - no value
session.gc_maxlifetime - 86400
soap.wsdl_cache_limit - 5
opcache.max_accelerated_files - 4000
opcache.max_file_size - 0
opcache.max_wasted_percentage - 5

@losttheplot See #8492


"Last page" button in Znode list goes to empty page

Clicking the button on the far right on the first image above goes to an empty page, with no possibility to get back to the Znode page with navigating away from it. It seems to reset to the original page once a new Znode update comes in.

Remote Znode list is being removed.


Implement complementary_channel method in the Kraus class

What is the expected behavior?
Given a set of Kraus operators for a channel, the method should return the set of Kraus operators for the complementary channel.
Usefulness
This channel is needed to compute the quantum mutual information and the coherent information of a given channel, for example in https://arxiv.org/abs/1905.01286. The Kraus class already has methods for the adjoint channel, the conjugate channel and the transpose channel. Adding the complementary channel and the adjoint complementary channel 'completes' the picture.

I will give it a shot and report back during the week

I wrote a quick and dirty implementation of the algorithm at this link in this file (look at the complementary_channel property in the class Channel). Will need to be improved a lot, at least for readability and maintainability, before adding it to Qiskit. @tyrolize mabe we could work together on this?

sure, let me know if I can help you with something.
otherwise it looks like you already have some good progress, I'll start working on other issues


Reset current directory on Core

Full-framework MSBuild was resetting the current directory at the end of build requests and at node idle, which helps avoid holding handles to project directories that the user may wish to delete while a worker node is still alive in the background. That wasn't done on .NET Core because at the time of the original port, there wasn't an Environment.SystemDirectory to use.
That property is now available, but returns the empty string on Linux, which fails Directory.SetCurrentDirectory(). So I'm avoiding it by using the current MSBuild tools directory, which should exist as long as the MSBuild process keeps running.
Fixes #2977.

macOS: Possible issue connecting to GitHub or cloning the repository. This could be caused by a lack of network connectivity or a lack of disk space. Please email netciadmin.
@dotnet-bot test OSX10.13 Build for CoreCLR please

The feature flag is sadly still used, in a confusing place for bad reasons.
https://github.com/Microsoft/msbuild/blob/0dcae6d2841e71d9ca1031c099c6883e64583afd/src/Shared/UnitTests/FileUtilities_Tests.cs#L558-L746
Those tests still need to be disabled, but I couldn't come up with a better name for the flag that describes how they're broken. Totally open to suggestions!

Hmm, perhaps the tests should just not use Environment.SystemDirectory?  it looks like they could use any directory

@dotnet-bot test this please
(don't trust stale results from February, but want this in)

@dotnet-bot test Ubuntu16.04 Build for CoreCLR please
An odd hang:
MSBuild Failure: From calculating based on the memorystream, about to read n = -21. length = 135784, rawPosition = 135805, readLength = 21, stringLength = 108, currPos = 87.
Waiting for debugger to attach to process: 75712


Support artifact classifier when downloading driver files from Maven

Is your feature request related to a problem? Please describe.
ClickHouse JDBC driver supports multiple protocols(http, tcp, and grpc), hence it comes with multiple classifiers for user to choose. As of now, DBeaver does not support user to specify classifier when downloading driver files from Maven.

Describe the solution you'd like
It would be great if we can add one more dropdown list next to Version column shown above to allow user to select classifier(e.g. all, cli, grpc, and http for ClickHouse, and perhaps empty for others)
Describe alternatives you've considered
N/A.
Additional context
The discussion started with a bug reported at ClickHouse/clickhouse-java#1257.

Hello @zhicwu,
Thanks for filing a feature request.
Note: specifying the classifier via code in the plugin.xml is possible, but it can't be changed via UI. In fact, after opening the edit dialog and saving changes, the classifier is lost.


How to get the list of resolution qualities?

Hello Brothers,
I am using Shaka player 4.0.2 to play DRM content. My App is Vue 2 + Nuxt 2 on Microsoft Edge.
I wonder if Shaka player has any built-in function/method to get the list of resolution qualities of the current playing content (I mean Shaka can auto detect the list of resolution quality: for example Auto adaptive, 480p, 720p, 1080p,...).
We only have 1 stream link for this content which look like:
https://example.cdn.net/live_dzones/max.smil/manifest.mpd.
I have searched in Internet as well as your docs but I found nothing.
Thank you.

Hi! It's here getVariantTracks https://shaka-player-demo.appspot.com/docs/api/shaka.Player.html#getVariantTracks It should be able to give you the resolution qualities.

@WINOFFRG thanks for your rep. I see it return list of shaka.extern.Track
I am confused that this case "track" mean text track/ audio track? Am I wrong?

As the method name getVariantTracks you will get list of all combinations of A/V. So if a stream has 3 Video Tracks and 2 Audio Tracks this will return you 6 combinations. To get the language and resolution or meta for that specific track just look at the defintion of shaka.extern.Track.
TL;DR It return the combination of both.


Session token fixes

Right now, after a user successfully signs up and clicks on their auth link in their email, they see an error pop-up saying we could not get their user information. When they return to the app, some users are getting in fine, and others it seems are not.
On Monday, the bridge team pushed code that made the session refresh after the user uploads consent.  While this should ideally not have been an issue for our app, it exposed a few bugs in our code that starting causing reproducible user facing errors on sign up.
Here are the bugs and their solutions:

Session token from signIn call isn’t being saved. This leads to the first 401 when attempting to sign consent.  The bug was because createAuthStateFromStoredCredentials was always overwriting the session token to be a blank sting "", when in most cases, we wanted it to keep its value, like after a successful sign in.
Server is switching the session token when you sign consent. This the second 401 when the app calls get activities for the first time.  This was what caused the app to get in one more bad state that pushed it over the edge.
App calls get activities twice, seemingly simultaneously.  This is because when you sign up, we request 3 weeks of activities (14 day request and then a 7 day request) in advance to the app can work offline for the duration of the study.  To solve this, I switched over to a max duration of 30 days, which is something Dwayne informed me they added to Bridge recently.  This makes it so this scenario never happens anymore, and is also a fix in BP repo, not here. https://github.com/Sage-Bionetworks/BloodPressureApp-Android/pull/505

I merged this in to do a build, but it still needs reviewed.


Adding an article on Smashing Magazine

Lyza Danger Gardner wrote a case study for Smashing Magazine on Service Workers.

Thanks @MarcoHengstenberg


Not able to rest the serverless worked locally with concurrency  (example provided on Runpod documentation fails)

Bug
Running concurrent server locally exits. with the error message:
WARNING:  You must pass the application as an import string to enable 'reload' or 'workers'.
After this message, the process exits instead of being a live server for testing.
To Reproduce
Steps to reproduce the behavior:

Follow the steps here: https://docs.runpod.io/serverless/workers/development/concurrency
Run "python runpod_example/main.py --rp_serve_api  --rp_api_concurrency 4"
See error

Expected behavior
Expected a lock server with concurrency.
Desktop (please complete the following information):

OS: Linux 4c03a197c102 6.8.0-45-generic #45~22.04.1-Ubuntu

@shreyassaxena What Python version are you using?


adding struct to Thread.Item

If someone in the DirectMessage sent Text or share Profile then it could not be read - fixed

Perfect , Thank you , Can you write some test files and make new PR ?

I will try. I just started learning go


inside  creates too much whitespace

This demonstrates turndown – an HTML to Markdown converter in JavaScript.

results in
[This demonstrates turndown – an HTML to Markdown converter in JavaScript.](https://github.com/domchristie/turndown)

while

This demonstrates turndown – an HTML to Markdown converter in JavaScript.

results in
[

This demonstrates turndown – an HTML to Markdown converter in JavaScript.

](https://github.com/domchristie/turndown)

In HTML5, paragraph tags are valid content for anchors tags and while browsers render the HTML snippets above the same Turndown seems to add too much whitespace around the paragraph element when it is a child of an anchor element. They should result in the same markdown without the extra new lines, etc. Thank you.

Yes, conversions of anchors with block-level elements to markdown presents a problem, since it is often not clear how it should be presented. For example, the following is valid but has no markdown equivalent:

Hello world

This whole block is linked

We could link the
 and not the paragraph, but other situations might not be so straightforward, so it becomes extremely difficult to cater for every case. (There's even a debate as to whether we should be using block links.)
The best option for now is to add a custom rule as suggested in https://github.com/domchristie/turndown/issues/278#issuecomment-583549989
Closing #278


Properly cleanup processes and queues for MPRS

Fixes issue about MPRS.finalize when dataloader2.shutdown() is called
Changes

DataLoader2 should always clean up datapipe_iter at shutdown
Guard MPRS to finalize once
Fix the problem of ConnectionError when DataLoader early exits

This is caused by queue is joined when main/worker/dispatching process exits. No more request/response can be passed across processes.

Consumer process shouldn't join the req_queue at exit to make sure producer process can still access the remaining request. And, consumer will close req_queue after clean up to prevent any further request sent to queue.
Produce process shouldn't join the res_queue at exit to make sure consumer process can still access response. And, producer will close res_queue after clean up to prevent any further response sent to queue.

Main (Consumer) <-> Worker (Producer)
Worker (Consumer) -> Dispatching (Producer)

Add tests to validate

TBH, I hope we can cherrypick this PR to release branch.


Add option to precompile assets when building the JAR

The logic is copied from the 3.x branch. It is turned off by default to prevent issues with non-rails apps. To run:
bundle exec torquebox jar --precompile-assets

Hmm, the two jobs that failed don't appear to be related to the code changes: https://travis-ci.org/torquebox/torquebox/builds/146284803

You're right that the failures aren't related. It looks like Rack 2 was released but won't work on JRuby 1.7.x and some of our Travis tests still run under JRuby 1.7.x. I'll investigate a fix. Thanks!


Fixing scenario id, adding support for case manipulation on target

@isra-fel The swagger PR this is based on is https://github.com/Azure/azure-rest-api-specs/pull/6885

The top two checks (ci Build and ci Test MacOS) have been running for 24 hours. That doesn't seem right. Can someone take a look at this?

To be clearer, the two tasks (build and test macos) looks like they finished yesterday but their status is stuck.

Yeah, they actually have passed.
https://dev.azure.com/azure-sdk/public/_build/results?buildId=87998&view=results


Error:  Cannot find module '../package.json' from 'node_modules/react-native-cashfree-pg-sdk/lib/commonjs/index.js'

Bug foung inside node_modules\react-native-cashfree-pg-sdk\lib\commonjs\index.js. You are trying to import package.json file:

Here it is clearly checking the package.json file in lib where as the package.js is provided in react-native-cashfree-pg-sdk folder. Due to this we are facing the issue.
Lib Folder Contents:

This clearly doesnot contain the package.json file
react-native-cashfree-pg-sdk Contents:

This folder contains the package.json file.
Solution to solve the problem: Changing the path for acquiring package.json in index.js will solve this issue (or) copying the package.json from react-native-cashfree-pg-sdk and pasting it into lib folder.

I think there is nobody in their team to maintain this library  🤔

I fixed that issue by removing .tsx and adding the .js to my file extension. for example,
App.tsx -> App.js


Shell completion (Bash)

I'm looking into adding Bash completion locally.
Per the https://github.com/benkehoe/aws-sso-util#quickstart the "main" command to produce shell completion code is _AWS_SSO_UTIL_COMPLETE=source_bash aws-sso-util, which I don't know why produces empty output for me — is this expected/known or did I manage to tangle my local setup somehow?
aws-sso-util is installed via Homebrew and OS is Ubuntu Jammy if it might be important:
> brew info aws-sso-util
==> aws-sso-util: stable 4.31.0 (bottled), HEAD
Smooth out the rough edges of AWS SSO (temporarily, until AWS makes it better)
https://github.com/benkehoe/aws-sso-util
/home/linuxbrew/.linuxbrew/Cellar/aws-sso-util/4.31.0 (3,103 files, 99.5MB) *
  Poured from bottle using the formulae.brew.sh API on 2023-01-27 at 17:00:56
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/aws-sso-util.rb
License: Apache-2.0
==> Dependencies
Required<EMAIL_ADDRESS>✔, six ✔
==> Options
--HEAD
        Install HEAD version
==> Analytics
install: 41 (30 days), 21 (90 days), 5,038 (365 days)
install-on-request: 41 (30 days), 20 (90 days), 4,988 (365 days)
build-error: 0 (30 days)

> lsb_release -d
Description:    Ubuntu 22.04.2 LTS

> _AWS_SSO_UTIL_COMPLETE=source_bash aws-sso-util | wc -l
0

Appreciate any help.

I'm afraid I can't be of much help directly; shell completion is a feature of click, the CLI library. The docs for completion are here: https://click.palletsprojects.com/en/8.1.x/shell-completion/ let me know if you do or don't make progress!

@benkehoe Does this mean that _AWS_SSO_UTIL_COMPLETE=source_bash aws-sso-util does produce non-empty output to you?

Aha: I missed updating the documentation. In click 8 (which aws-sso-util requires as of v4.31), the value changed from source_bash to bash_source (and similar for other shells) (no idea why). It was working for me using the click 8 docs, but the links in the README here and the example are using the click 7 version.

@benkehoe Worked like a charm! Thanks a lot into looking into this!


Expose useful metrics to Prometheus

This PR is an extension to #247 to implement #191.
In #247, simple metrics were exposed from Fleetspeak, and the PR's main goal was to make the metrics scrapable by Prometheus; this PR's goal is to take these metrics, expand them and make them more complete and beneficial to our users.
Some of the changes include:

Incorporated most of the conclusions discussed with @mbushkov and @bgalehouse in #247 that were not already resolved there. I have also changed and/or expanded most of the metrics that were merged to be more elaborated and versatile, such that they can be further filtered and analysed when a Grafana dashboard will be created for Fleetspeak (similar to the ones provided with GRR).
The commits' names (and comments) within this PR elaborate which metrics were expanded or changed, and how exactly. Most changes can be summarized as; Counters were switched to CounterVecs with useful labels, and HistogramVecs were added where applicable.
Note that right now, all histograms are using the default buckets provided by Prometheus (those are DefBuckets which are defined under Constants). They work quite well, but in order to create truly meaningful buckets that will be the most useful for our users, we probably need to run end-to-end tests and full working deployments of Fleetspeak and aggregate as much metrics data as possible, so that we can analyze and research it. Such research will be conducted as part of a separate PR.

ptal

Added a label service to the metric MessageProcessed so that we'll be able to graph message processing latency per service.


Consider moving client-only and frame-only primitives out of primitives

primitives is meant to contain shared components.

Pinging @seunlanlege @gilescope

[ ] #9214

Will take a crack at this as v similar to yesterday's.

Duplicate of: https://github.com/paritytech/substrate/issues/5485


Use 'isSearchingForReports' to control green circular loading

Details

Make sure the Green Loading Indicator works same everywhere we would expect.
Fixed Issues

$ https://github.com/Expensify/App/issues/37144
PROPOSAL: https://github.com/Expensify/App/issues/37144#issuecomment-1961608043
Tests

Launch New Expensify app.
Follow the Steps to access the 'Search Field' below (Conclude the test for each page)
Type something on the Search Field.
Verify that a Green Circular Loading indicator appears when we search for a contact and disappears when the search is complete.

Steps to access the 'Search Field' of each Page:

Request money (Manual):

Click Global FAB
Press Request money
Select Manual
Fill the amount and press Next
Click on the Search Field

Request money (Scan):

Click Global FAB
Press Request money
Select Scan
Upload a receipt
Click on the Search Field

Request money (Distance):

Click Global FAB
Press Request money
Select Distance
Select 2 different waypoints and press Next
Click on the Search Field

Send money:

Click Global FAB
Press Send money
Fill the amount and press Next
Click on the Search Field

Room Invite Page:

Click Global FAB
Press Start chat
Select Room
Fill the name, workspace and create a room
Go to the room
Open room details
Press Members
Press Invite
Click on the Search Field

Share Log List Page:

Go to Profile -> Troubleshoot
Enable Client side logging if disabled
Click View debug console
Click Share log
Click on the Search Field

Assign task:

Click Global FAB
Press Assign task
Fill the Title and press Next
Press Assignee
Click on the Search Field

Assign task - share somewhere:

Click Global FAB
Press Assign task
Fill the Title and press Next
Press Share somewhere
Click on the Search Field

Invite to workspace:

Go to Profile -> Workspaces
Create a new workspace or join one where you can add members.
Go to Members
Press Invite member
Click on the Search Field

[x] Verify that no errors appear in the JS console

Offline tests

N/A
QA Steps

Same as Tests
PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native
https://github.com/Expensify/App/assets/95647348/7736b8ca-abea-4a50-8708-215a54c49243

iOS: mWeb Safari

https://github.com/Expensify/App/assets/95647348/20e3c969-3d1f-45b3-9bcf-839a8f538d31

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/95647348/d82c4b10-9d91-4169-a00f-0bca6a7f9764

MacOS: Desktop

https://github.com/Expensify/App/assets/95647348/ab01b0da-15bd-49c5-8a04-258730e34b96

PR Checklist:

[x] Search page
[x] New chat
[ ] New room chat - RoomInvitePage
[x] Request money - manual
[x] Request money - scan
[x] Request money - distance
[x] Send money
[ ] Assign task
[x] Assign task - share somewhere
[ ] Invite to workspace
[ ] Invite to workspace chat
[ ] Share logs

@mollfpr our PR is ready for review.
Im struggling with my Android build, can you assist me with test/videos here?
Thank you very much.

On the workspace invite member page, when clearing the input does not show all the users.

Good catch, thank you!
Fixed! When implementing useDebouncedState I was missing updating one useEffect dependency. Now it's fine.

@brunovjk Could you fix the conflict, I'll finish the recording after that. Thanks!

Done @mollfpr :D

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[ ] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[ ] I verified that all the inputs inside a form are aligned with each other.
[ ] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

@mollfpr, @lakchote is "Out of office until 4:00 AM, Sat, Apr 27", do we wait for him to return or do we ask another internal eng for help? I believe we can wait, we are not in a hurry here, right? Thanks.

LGTM. @brunovjk could you please resolve conflicts? Thank you!

Re-testing

@lakchote what do you think about use useOnyx instead of withOnyx here? In all pages

@brunovjk I think the hook one is preferable and the property we need seems simple enough.

@brunovjk I think the hook one is preferable and the property we need seems simple enough.

I concur with what has been said.
It'd be great if we can use useOnyx instead for performance reasons.
It's usage will likely be enforced in the future (see here).
@brunovjk can you do it?

Awesome! I will do it, re test, and you guys know

I'm seeing a performance test error on the ChatFinder page (unexpected extra render) – any ideas what might be causing it, given the changes seem similar to other working pages?
I retested and updated the PR videos/screenshots. Everything looks fine to me, except the perf test.
@mollfpr, @lakchote

[ChatFinderPage] should interact when text input changes [render]: 28.7 ms → 33.1 ms (+4.4 ms, +15.3%) 🔴 | 6 → 7 (+1, +16.7%) 🔴

The mean count is increased by 1, but I tried the perf test locally with baseline as the latest main output is also 7. So I didn't get the error on the reassure test this PR.

I left ChatFinderPage almost untouchable. In fact, we wouldn't need to change it now, since "Green Loading" was working before there. I just standardized the "boolean" conditional and later removed withOnyx. @mollfpr @lakchote  What do you think?

LGTM.

Reverted ChatFinderPage, only made a boolean standardization, since the "Green Loading" it was working before. Later we can come back to remove withOnyx to useOnyx and fix the perf test. How does that sound? The rest LGTM.
@mollfpr @lakchote

@brunovjk Looks good to me too!

LGTM to me too.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to production by https://github.com/marcaaron in version: 1.4.71-6 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


update babel and jest packages

Coverage remained the same at 87.845% when pulling be0852ba2e8fab553ea178d16c21c458868c25d5 on update-babel-and-jest into 178c196c22db5ab9ecad1f40815a1cd47d871165 on develop.


[CS] Fixes #2480 Add RemovePHPStormTodoCommentFixer

All checks have passed :tada: @TomasVotruba it is ready for review.

Thank you :+1:


fs.find() returns a list instead of a dictionary

What happened:
I am running into errors with ds.dataset() from pyarrow which expects a dictionary instead of a list from fs.find() from adlfs.
What you expected to happen:
The code seems to show that a dictionary should be returned if detail=True, but I wasn't able to get this to work.
Minimal Complete Verifiable Example:
dev is my Azure Blob container and testing is a dataset written with pq.write_to_dataset() with two partition columns being used.
fs = fsspec.filesystem(protocol="abfs", account_name=name, account_key=key)

print(type(fs.find("dev/testing", detail=True)))
<class 'list'>

print(type(fs.find("dev/testing")))
<class 'list'>

fs = adlfs.AzureBlobFileSystem(account_name=name, account_key=key)
print(type(fs.find("dev/testing", detail=True)))
<class 'list'>

Anything else we need to know?:
Environment:
I installed this on a fresh PC using conda forge. Python was not installed previously, so I do not think there should be any conflicts with versions.

Dask version: 2.23 typically but I do not have it installed in this example
Python version: 3.8.6
Operating System: Ubuntu 20.04
Install method (conda, pip, source): conda env file

name: pyarrow
channels:
  - conda-forge
  - defaults
dependencies:
  - python=3.8
  - pandas
  - numpy
  - pyarrow
  - adlfs
  - azure-storage-blob

Full conda list output:
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
abseil-cpp                20200225.2           he1b5a44_2    conda-forge
adal                      1.2.5              pyh9f0ad1d_0    conda-forge
adlfs                     0.5.7              pyhd8ed1ab_0    conda-forge
aiohttp                   3.7.3            py38h25fe258_0    conda-forge
arrow-cpp                 2.0.0           py38ha9d6ab6_6_cpu    conda-forge
async-timeout             3.0.1                   py_1000    conda-forge
attrs                     20.3.0             pyhd3deb0d_0    conda-forge
aws-c-common              0.4.59               h36c2ea0_1    conda-forge
aws-c-event-stream        0.1.6                had2084c_6    conda-forge
aws-checksums             0.1.10               h4e93380_0    conda-forge
aws-sdk-cpp               1.8.70               h57dc084_1    conda-forge
azure-core                1.9.0              pyhd3deb0d_0    conda-forge
azure-datalake-store      0.0.51             pyh9f0ad1d_0    conda-forge
azure-identity            1.4.1              pyh9f0ad1d_0    conda-forge
azure-storage-blob        12.6.0             pyhd3deb0d_0    conda-forge
blinker                   1.4                        py_1    conda-forge
brotli                    1.0.9                he1b5a44_3    conda-forge
brotlipy                  0.7.0           py38h8df0ef7_1001    conda-forge
bzip2                     1.0.8                h516909a_3    conda-forge
c-ares                    1.17.1               h36c2ea0_0    conda-forge
ca-certificates           2020.11.8            ha878542_0    conda-forge
certifi                   2020.11.8        py38h578d9bd_0    conda-forge
cffi                      1.14.3           py38h1bdcb99_1    conda-forge
chardet                   3.0.4           py38h924ce5b_1008    conda-forge
cryptography              3.2.1            py38h7699a38_0    conda-forge
dbus                      1.13.6               hfdff14a_1    conda-forge
expat                     2.2.9                he1b5a44_2    conda-forge
fsspec                    0.8.4                      py_0    conda-forge
gettext                   <IP_ADDRESS>          hf34092f_1004    conda-forge
gflags                    2.2.2             he1b5a44_1004    conda-forge
glib                      2.66.2               h58526e2_0    conda-forge
glog                      0.4.0                h49b9bf7_3    conda-forge
grpc-cpp                  1.33.2               h1870a98_1    conda-forge
idna                      2.10               pyh9f0ad1d_0    conda-forge
isodate                   0.6.0                      py_1    conda-forge
jeepney                   0.6.0              pyhd8ed1ab_0    conda-forge
keyring                   21.5.0           py38h578d9bd_0    conda-forge
krb5                      1.17.2               h926e7f8_0    conda-forge
ld_impl_linux-64          2.35.1               hed1e6ac_0    conda-forge
libblas                   3.9.0                3_openblas    conda-forge
libcblas                  3.9.0                3_openblas    conda-forge
libcurl                   7.71.1               hcdd3856_8    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.10               hcdb4288_3    conda-forge
libffi                    3.2.1             he1b5a44_1007    conda-forge
libgcc-ng                 9.3.0               h5dbcf3e_17    conda-forge
libgfortran-ng            9.3.0               he4bcb1c_17    conda-forge
libgfortran5              9.3.0               he4bcb1c_17    conda-forge
libglib                   2.66.2               hbe7bbb4_0    conda-forge
libgomp                   9.3.0               h5dbcf3e_17    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0                3_openblas    conda-forge
libnghttp2                1.41.0               h8cfc5f6_2    conda-forge
libopenblas               0.3.12          pthreads_h4812303_1    conda-forge
libprotobuf               <IP_ADDRESS>             h8b12597_0    conda-forge
libssh2                   1.9.0                hab1572f_5    conda-forge
libstdcxx-ng              9.3.0               h2ae2ef3_17    conda-forge
libthrift                 0.13.0               h5aa387f_6    conda-forge
libutf8proc               2.5.0                h516909a_2    conda-forge
lz4-c                     1.9.2                he1b5a44_3    conda-forge
msal                      1.6.0              pyhd3deb0d_0    conda-forge
msal_extensions           0.2.2              pyh9f0ad1d_0    conda-forge
msrest                    0.6.19             pyh9f0ad1d_0    conda-forge
msrestazure               0.6.2                      py_0    conda-forge
multidict                 4.7.5            py38h1e0a361_2    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
numpy                     1.19.4           py38hf0fd68c_1    conda-forge
oauthlib                  3.0.1                      py_0    conda-forge
openssl                   1.1.1h               h516909a_0    conda-forge
orc                       1.6.5                hd3605a7_0    conda-forge
pandas                    1.1.4            py38h0ef3d22_0    conda-forge
parquet-cpp               1.5.1                         2    conda-forge
pcre                      8.44                 he1b5a44_0    conda-forge
pip                       20.2.4                     py_0    conda-forge
portalocker               1.7.0            py38h32f6830_1    conda-forge
pyarrow                   2.0.0           py38h5703e5b_6_cpu    conda-forge
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pyjwt                     1.7.1                      py_0    conda-forge
pyopenssl                 19.1.0                     py_1    conda-forge
pysocks                   1.7.1            py38h924ce5b_2    conda-forge
python                    3.8.6           h852b56e_0_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python_abi                3.8                      1_cp38    conda-forge
pytz                      2020.4             pyhd8ed1ab_0    conda-forge
re2                       2020.11.01           h58526e2_0    conda-forge
readline                  8.0                  he28a2e2_2    conda-forge
requests                  2.25.0             pyhd3deb0d_0    conda-forge
requests-oauthlib         1.3.0              pyh9f0ad1d_0    conda-forge
secretstorage             3.2.0            py38h578d9bd_0    conda-forge
setuptools                49.6.0           py38h924ce5b_2    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
snappy                    1.1.8                he1b5a44_3    conda-forge
sqlite                    3.33.0               h4cf870e_1    conda-forge
tk                        8.6.10               hed695b0_1    conda-forge
typing-extensions         <IP_ADDRESS>                       0    conda-forge
typing_extensions         <IP_ADDRESS>                    py_0    conda-forge
urllib3                   1.25.11                    py_0    conda-forge
wheel                     0.35.1             pyh9f0ad1d_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yarl                      1.6.3            py38h25fe258_0    conda-forge
zlib                      1.2.11            h516909a_1010    conda-forge
zstd                      1.4.5                h6597ccf_2    conda-forge

Ah, I spotted the error:
https://github.com/dask/adlfs/blob/54d5826627a25dc8541e35754bf4e9512543a027/adlfs/spec.py#L788-L793
The **kwargs passed to find are not passed through to maybe_sync(self._find, ..)
So the _find properly handles the detail=True case, but the public find method ignores it this way.

Nice! I added a pull request for this as well since it really is a very quick and easy fix.
    def find(self, path, maxdepth=None, withdirs=False, **kwargs):
        return maybe_sync(self._find, self, path, maxdepth, withdirs, **kwargs)

This fixes my issue with the latest versions of adlfs and azure-storage-blob v12 since I am able to read a dataset now (I was already able to use write_dataset() before).
dataset = ds.dataset(source="dev/testing9", filesystem=fs, partitioning="hive")
table = dataset.to_table()
print(table.num_rows)
23566

Do you think I should close https://issues.apache.org/jira/browse/ARROW-10517? I have a remaining issue with my production environment with an older version of adlfs and azure-blob-storage v2 which might also be a quick fix, but the existing issue is quite long and confusing.

@ldacey -- I did not see your PR, but added the fix with #136, which is now in master.  Please let me know if this does not fix your issue.

Thanks, yes your changes resolved this issue.


CLS features all nan when passing in 4 channel images.

I am using scDINO on my image sets. I have four channel fluorescence images. To make my images compatible with the pre-trained model, I made a 5th channel with all 0's.
This is an example of how I make my blank channel(s).
Where image is the loaded image via the tifffile python package
if image.shape[-1] < 5:
        channels_to_add = 5 - image.shape[-1]
        for channel in range(channels_to_add):
            # add a new channel of all zeros
            new_channels = np.zeros((image.shape[0], image.shape[1], 1))
            image_merge = np.concatenate((image,new_channels), axis=-1)
    print(image_merge.shape)

The issue I am having in scDINO seems to arise in the normalize_numpy_0_to_1 & normalize_tensor_per_channel functions in the pyscripts/utils.py file.
When I have a channel of a uniform distribution of pixel values I end up with a normalized array value in the uniformly distributed channel of NaN due to the zerodivision that occurs on lines 78 & 108.
I have a proposed change that would help scDINO use multichannel images that contain uniform distributions.
If the maintainers of scDINO agree with the proposed change I can happily open a PR from this issue!
The proposed change is below.
def normalize_numpy_0_to_1(x):
    print("x",x.shape)
    x_min = x.min(axis=(0,1), keepdims=True)
    x_max = x.max(axis=(0,1), keepdims=True)
    diff_min_max = x_max - x_min
    if check_nan(diff_min_max):
        print("diff_min_max is nan")
    if check_nan(x-x_min):
        print("x-x_min is nan:")
    if check_nan(x):
        print("x contains nan before normalization")
    if check_zero(diff_min_max):
        print("diff_min_max has zero")
        print("x_max",x_max)
        print("x_min",x_min)
        print("diff_min_max",diff_min_max)
        print("x",x.shape)
        # replace x_max 0 values with 1
        for i in range(len(x_max[0][0])):
            if x_max[0][0][i] == 0:
                x_max[0][0][i] = 1
    x = (x - x_min)/(x_max-x_min)
    if check_nan(x):
        print("x contains nan after normalization")
    return x

Excellent work on scDINO and implementations! I would love to talk more in the future!

Hi Mike, your solution sounds great to me. I accepted your PR. Thanks for your interest in scDINO and your contribution!😃


Prefab Missing Issue

Hello Sir,
please help me to provide a solution for bellow issue.
QRCodeSample scene gives an error when try to upgrade the project in 2020.3.13 unity version. i will also installed Mixed Reality OpenXR Plugin and MixedReality ToolKit using mixed reality feature toolkit and Microsoft.MixedReality.QR but still the prefab scripts are missing.
issue is:- Axis and QRCodeManage prefab is missing

this is a duplicate of https://github.com/microsoft/MixedReality-QRCode-Sample/issues/4

New update is made resolved this issue


[SPARK-32306][SQL][DOCS] Clarify the result of percentile_approx()

What changes were proposed in this pull request?
More precise description of the result of the percentile_approx() function and its synonym approx_percentile(). The proposed sentence clarifies that  the function returns one of elements (or array of elements) from the input column.
Why are the changes needed?
To improve Spark docs and avoid misunderstanding of the function behavior.
Does this PR introduce any user-facing change?
No
How was this patch tested?
./dev/scalastyle

Test build #128974 has started for PR 29835 at commit e09e423.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/33596/
Test PASSed.

Test build #128980 has started for PR 29835 at commit 1daac42.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/33602/
Test PASSed.

Test build #128974 has finished for PR 29835 at commit e09e423.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128974/
Test PASSed.

Test build #128980 has finished for PR 29835 at commit 1daac42.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128980/
Test PASSed.

Thanks! Merging to master.

@MaxGekk Can you backport to 2.4/3.0?

Here are PRs for:

2.4 https://github.com/apache/spark/pull/29847
3.0 https://github.com/apache/spark/pull/29845