Deadlock in ThreadFreezer::ThreadFreezer()

I was diagnosing a startup deadlock in Atomic Heart and discovered the code being used to freeze all threads during hook install deadlocks 9 times out of 10.
 They are slow as hell to capture and unsafe to walk from within a running process. In truth, that API is meant to be used by a separate debugger process on a suspended process.
NtQuerySystemInformation (SystemProcessInformation, ...) should be used instead.
If you would like, I can provide you with a working implementation written in C.

The only way I was able to get DLSSTweaks working in Atomic Heart (Microsoft Store version, if it matters) was by disabling the SuspendThread (...), Get|SetThreadContext (...) and ResumeThread (...) calls.
I effectively gimped your thread freezing system, but deadlock goes away and hooks are successful despite multiple threads running.
I believe if you throw out the dependency on Toolhelp32 and snapshot using NtQuerySystemInformation instead, you can walk over these threads and safely suspend/resume.

Perhaps an INI option to forgo freezing and thawing threads during hook installation could serve as a temporary solution?

when launching atomic heart with your latest release AND with your .12-beta3, the game is freezing almost immediately.
am i missing something? playing with microsoft store version

@snakeeater4526 are you using SK or any other kind of DLL wrappers? Looks like there is some issue with deadlocking when used with SK here unfortunately, but don't really have a fix for that yet.
(gutting the ThreadFreezer code seems to stop the deadlock, but then doesn't look like any hooks here are actually active... but I didn't try loading with SK plugin system yet, maybe that would help make the hooks here take precedence, not sure)

@snakeeater4526 are you using SK or any other kind of DLL wrappers? Looks like there is some issue with deadlocking when used with SK here unfortunately, but don't really have a fix for that yet.
(gutting the ThreadFreezer code seems to stop the deadlock, but then doesn't look like any hooks here are actually active... keeping the gutted ThreadFreezer with SK removed does seem to let it work. I didn't try loading via SK's plugin system yet though, maybe that would help make the hooks here take effect, not sure)
Here's a build with the ThreadFreezer stuff removed anyway, maybe it can help even if you aren't using SK: dlsstweaks-0.123.12beta3a.zip

This fixed Hogwarts Legacy's freeze at launch for me. Thanks emoose 👍
And I'm using SK too.

@SnowMaw could you check if your dlsstweaks.log file mentions this line inside it?

DLSS functions found & parameter hooks applied!

If it does then it should all be working fine, otherwise it might be having the issue I mentioned with hooks not being active.
btw how are you loading it in with SK? did you just rename the tweaks DLL or did you set it as a plugin?

@SnowMaw could you check if your dlsstweaks.log file mentions this line inside it?

DLSS functions found & parameter hooks applied!

If it does then it should all be working fine, otherwise it might be having the issue I mentioned with hooks not being active.
btw how are you loading it in with SK? did you just rename the tweaks DLL or did you set it as a plugin?

Yeah I got that line:
[2023-02-25 19:25:31.007] [info] DLSS functions found & parameter hooks applied!
I'm using it as dxgi.dll and SK (global injection)

@snakeeater4526 are you using SK or any other kind of DLL wrappers? Looks like there is some issue with deadlocking when used with SK here unfortunately, but don't really have a fix for that yet.
(gutting the ThreadFreezer code seems to stop the deadlock, but then doesn't look like any hooks here are actually active... but using the gutted ThreadFreezer version with SK removed does seem to let it work. I didn't try loading via SK's plugin system yet though, maybe that would help make the hooks here take effect, not sure)
Here's a build with the ThreadFreezer stuff removed anyway, maybe it can help even if you aren't using SK: dlsstweaks-0.123.12beta3a.zip

It's working with this special build you provided, i'm using no special software at all or injector that i'm aware. btw what is " SK " ?

I've gone ahead and put in an alternative way of freezing threads (https://github.com/cursey/safetyhook/pull/21). If you run into any issues with SafetyHook please let me know by creating an issue on my repo. I found out about this issue through a friend. I'm very motivated to make SafetyHook as reliable and easy to use as possible.
Thanks.

@snakeeater4526 are you using SK or any other kind of DLL wrappers? Looks like there is some issue with deadlocking when used with SK here unfortunately, but don't really have a fix for that yet.
(gutting the ThreadFreezer code seems to stop the deadlock, but then doesn't look like any hooks here are actually active... but using the gutted ThreadFreezer version with SK removed does seem to let it work. I didn't try loading via SK's plugin system yet though, maybe that would help make the hooks here take effect, not sure)
Here's a build with the ThreadFreezer stuff removed anyway, maybe it can help even if you aren't using SK: dlsstweaks-0.123.12beta3a.zip

It's working with this special build you provided, i'm using no special software at all or injector that i'm aware. btw what is " SK " ?

SK is
https://special-k.info/
I recommend to join his discord in case you need assistance > https://discord.gg/specialk

I tryed dlsstweaks-0.123.12beta3a with hogwarts legacy. Here are my results:

It does not get stuck at the starting warning messages. 👍

The DLSS presets are selected correctly (using DLSS 3.1.1). 👍

The DLSS quality levels seem to not be working, they default to one of the predefined levels no matter what I choose in the "dlsstweaks.ini". 👎

I haven't been able to get the DLSS quality levels to work with any of the dlss tweaks builds so far, and at this point I don't know if it is a problem with my system, the game, or your software.

Are you able to override the quality levels for any of the non default ones? So something that is NOT 0.33333334, 0.5, 0.58, 0.66666667, 0.777778? I agree I should probably create a thread only for hogwarts legacy related problems.

I just tried quality at 0.72 and that worked fine (and, again, it showed 4k @ 72% resolution in the game render settings).


Explorer icon disappears

VSCode Version: 1.40.0
OS Version: Windows 10

In the new version, every time I click a different icon the icon for Explorer disappears:
Right after opening VSCode:

And when I click Search:

/needsMoreInformation can you please a screen recording that illustrates the problem.

Here you go:

I have the same issue
Any solution yet? This is very provocating.

@moseleyi I 100% agree that this issue would be annoying, unfortunately, I just cannot reproduce it yet, so I am not sure the cause.
@sandy081 have you seen this, or have ideas at what could be unique here?

I had the same issue and i fixed it by deleting my personal settings of vsc into my home.

I have not seen this issue too.
Can you please reproduce it on a fresh user data directory? You can use command line option code --user-data-dir <path_to_dir>  and launch code.

Version 1.41.1 still happening

If I click CTRL + SHIFT + N to open fresh window of VSCode then the icon is completely gone. Then I select OPEN FOLDER and choose different folder - the icon disappears

@moseleyi I am still not able to reproduce the issue, did you try the steps provided by @sandy081


[Database] move query builder to base database component from the database-pdo sub-component

The database component should hold the query builder, as it can be re-used in future sub-components that use other libraries or ORMs. The Query builder must then be injected into the library as a dependency, and it can thus be re-used in the library just as before.

Resolved.


add SOP link

Add SOP link to either 'Research' tab or maybe 'Dicty Stock Center' tab:
https://northwestern.app.box.com/s/p0f8m70whgiuib2u0wt8gtn497ncmq8i

Added to the navbar JSON. I moved it to the DSC dropdown since it is linked from the DSC homepage.


Support limit push down in to_html

This patch inlines to_html docs from pandas, and supports limit push down when max_rows is specified, so we don't need to pull in all the data.

LGTM. Let me cherry-pick workaround separately right way at https://github.com/databricks/koalas/pull/209


ENH: heatmap: support row_colors and feature metadata annotation on plots

Current Behavior
Currently supports label annotation of sample metadata information.
Proposed Behavior
Please allow the same for features. And please support annotation with row_colors and col_colors instead of or in addition to label annotation. E.g., see the following plot from the seaborn clustermap docs.

E.g., color by phylum affiliation. Allow level selection from semicolon-delimited feature IDs? This would be useful for taxa but also, e.g., KEGG annotation.

also moving this one back to feature-table since vizard's heatmap is more general-purpose


Clear values

Hello!
I found the weird row in source code at update method:
rawValue = rawValue || inputElement.value
In my case, I use React component and outside Store, that controlling my state of form (inputs with react-text-mask).
I have method .clear, that set all fields to ''. But it doesn't work, obviously.
So, how I can clear values from outside state controller?

Can you show us an example of what your trying to do..

Simplified version:
import * as React from 'react';
import MaskedInput from 'react-text-mask';

class TestComponent extends React.Component<any, any> {
  constructor(props) {
    super(props);

    this.state = {
      value: '',
    };
  }

  private handleClear = () => {
    this.setState({
      value: '',
    });
  }

  private handleChange = (ev) => {
    this.setState({
      value: ev.target.value,
    });
  }

  public render() {
    return (

        <MaskedInput
          mask={[/\d/, /\d/]}
          onChange={this.handleChange}
        />

        <button
          type="text"
          onClick={this.handleClear}
        >
          Clear
        </button>

    );
  }
}

export { TestComponent }

I found the weird row in source code at update method:
rawValue = rawValue || inputElement.value

what makes you say this line is weird? ..and is your problem fixed if you comment out that line?
I don't really know React well enough to offer much help..
@browniefed are you able to help with this?

rawValue = rawValue || inputElement.value Says that if rawValue = "" then default to the existing value on the inputElement.
Which means if you change the value by passing in a new prop then it will break.

So you can never programmatically clear the value in the fields. This is indeed a bug.
Potential fix is just to check if rawValue === undefined rather than just rawValue ||

is this specific to React?, or should I be able to re-create this with any component (or tests in the core, prefereably)

rawValue = rawValue || inputElement.value is just JS. If it exists elsewhere then not just a React issue.

Passing in a prop like <TextMask value={this.props.value} /> is called a controlled component.
That means whatever value gets passed in should be the value of the input field.

yes.. i get what your saying.. I will look into it, and I think your right to check for undefined instead

We actually operate in a pseudo controlled fashion. We use defaultValue due to an IE10 caret position issue. But we do an update in the componentDidUpdate call to update the new value.

Is this code in core yet? I don't see it?
Even testing for undefined this might break the uncontrolled nature of ReactTextMask.
This is something I fought against in the beginning to only ship a controlled component.

Is this code in core yet? I don't see it?

which code?

The rawValue = rawValue || inputElement.value code

yes, here

Ah, yes so this will effect EVERY component.

This is the issue with the function being impure and holding on to an input state instead of just passing it in every time.

It's my fault, I didn't post a link to the source code line with a detailed comment. Sorry!
Yes, it is non-react problem and you are right, guys.

Either I'm not really understanding the issue completely (missing something), or the React component is the only one that suffers from this bug.
I've tried with the Angular1 & 2, Ember and Vue components and they all behave as I expect.
In each one I created a button that programatically sets the model value to ''.  And in all cases the model and the input element value were both updated to display an empty value (or placeholder in the case of input element).
I think it is perhaps because of the way the react component is implemented that it is failing on this... The problem for me is that I know too little about how react works.
I've trie to create a button that clears the value but failed to do it in React.. (all other frameworks I can works this out.. but React..., I must be missing something really obvious...).
I'd appreciate it if someone could create a branch and update the react example app to demonstrate the probem..

I've managed to re-create it in the React example app.  So I can now see the issue.
And replacing rawValue = rawValue || inputElement.value with
rawValue = typeof rawValue !== 'undefined' ? rawValue : inputElement.value

seems to fix it.

Even testing for undefined this might break the uncontrolled nature of ReactTextMask.

This is something I fought against in the beginning to only ship a controlled component.

Tell me more about this..

Controlled vs Uncontrolled.
Controlled value of the input ONLY changes when outside updates the property passed in. For example <input value={this.state.value} onChange={(e) => this.setState({value: e.target.value})} />
Here you can we we are updating state, re-rendering and then input will be updated.
An Uncontrolled input would be something like
<input defaultValue={this.state.value} />
We only set the initial value but if an outside force (for example you change this.state.value to be an empty string). It will not effect the input value. Thus not controlled by outside forces.

I've managed to re-create it in the React example app. So I can now see the issue.
@lozjackson oh, ok. I will stop updating branch in my fork,

And so, (correct me if i'm wrong here) the problem is that the controlled input is relying on the onChange method to update the value of the input element.
It does update the value of the model (this.state.value is updated to empty string when I clear the value).. just the input element is not udpateing because the onchange call is not setting the value

The actual way to reproduce this is if someone else (not an input change) effects the values. Like in the example above
As you can see value is held outside. When something else changes it (the clear call).
The this.props.value inside of the React component is now "" but inputElement.value is whatever value was in there previously.
import * as React from 'react';
import MaskedInput from 'react-text-mask';

class TestComponent extends React.Component<any, any> {
  constructor(props) {
    super(props);

    this.state = {
      value: '',
    };
  }

  private handleClear = () => {
    this.setState({
      value: '',
    });
  }

  private handleChange = (ev) => {
    this.setState({
      value: ev.target.value,
    });
  }

  public render() {
    return (

        <MaskedInput
          value={this.state.value}
          mask={[/\d/, /\d/]}
          onChange={this.handleChange}
        />

        <button
          type="text"
          onClick={this.handleClear}
        >
          Clear
        </button>

    );
  }
}

export { TestComponent }

just the input element is not udpateing because the onchange call is not setting the value

Because rawValue = rawValue || inputElement.value every time is inputElement.value.
rawValue === '';
Boolean(rawValue) === false
So, after update method (inside core of text-mask) invocation value is not updating "form the top"

onChange updating the state and value inside react-text-mask component is right (''). But after the invocation of maskedInput.update(value) this value doesn't go ahead.

The thing that confuses me about this is that rawValue and inputElement.value should be the same thing (and should have the same value).
But in React it seems to have the previous value before the key was pressed.

In React we can manage state outside of the component. And say: "hey input! Your new value is ..."
onChange is a callback, that takes value from change event and set it to "state manager".
We have 'blabla' text inside input. External state.value is 'blabla'. Because after change event callback onChange updated the value.
But also we can just say to "state manager": "change the value to X". (this.setState({ value: 'X' }))
rawValue = rawValue || inputElement.value -> rawValue === 'X', but inputElement.value is 'blabla'. Value inside input dom element is not updated yet.
After this expression rawValue === X, because Boolean('X') === true.

I've added a check for undefined.. can you try this in your app and let us know if that works for you.

Even testing for undefined this might break the uncontrolled nature of ReactTextMask

@browniefed Are you suggesting that checking for undefined is going to break other things?

@lozjackson That works for me!
Thank you!

@lozjackson give it a shot, I think it'll be fine. Just think we should test it on the examples page.

@zetoke how does it work for you?? Its not working for me, I also took your example and tried, but failed. What might be wrong with this?
https://codesandbox.io/s/KrLOVBPvx

@mohammedzamakhan you forgot value={this.state.value} for MaskedInput
It should be
<MaskedInput
  ref={maskInput => (this.maskInput = maskInput)}
  mask={[/\d/, /\d/]}
  onChange={this.handleChange}
  value={this.state.value}
/>

not
<MaskedInput
  ref={maskInput => (this.maskInput = maskInput)}
  mask={[/\d/, /\d/]}
  onChange={this.handleChange}
/>

@zetoke I'm coming across an issue with controlled/uncontrolled and was wondering what, if anything, was done to resolve the issue. It seems that the component using defaultValue vs value adds to the issue.


Add support for collection of primitives in DefaultTempDataSerializer

The TempDataSerializer based on Newtonsoft.Json supports some collection types. We should add parity features to the System.Text.Json based serializer.

Also make sure IdentityUI works. See https://github.com/aspnet/AspNetCore/issues/10139.


Documentation Request: Provide suggested best practices for parallelization

It appears based on my initial reading thus-far, that while fMRIprep will run against a complete BIDS dataset, it does not parallelize across subjects. It does take advantage of multiple CPUs some semi-controllable, but undocumented ways (see #1736) based on which tool I'm sure.
I'd like to request some documentation be added regarding best practices for parallelization, useful information might include

a tool to launch fmriprep against clusters https://github.com/pipitone/qbatch comes to mind here
a way to run locally in parallel (qbatch supports this)
some estimates on what tools inside fmriprep can take advantage of local parallelism for job packing purposes
some estimates on what the memory requirements are for "common" datasets (1x1x1 iso T1W + bold) again to assist in job packing
a warning be added if users run fMRIprep against a bids dataset with no subject selected, noting this probably isn't the best way to handle this

Some of this I can help with, but I'm not a fMRI guy, I'm supported a new deployment of fMRIprep and I want to make sure the users get the most out of the computing available to them

There is a lot of great information and benchmarking here: https://neurostars.org/t/how-much-ram-cpus-is-reasonable-to-run-pipelines-like-fmriprep/1086/5
One subject per job is common but will randomly fail with multiple jobs for 1.4 <= version < 1.5.0 due to #1667.

There was a suggestion in the aforementioned thread to add this to documentation, but I'm not sure if it was ultimately added.
There is a caveat, of course, that some of this may not necessarily be applicable to current versions.

Pinging @surchs here :) - Any chance you could contribute your benchmarks to the documentation?

Thanks for your invitation! @oesteban
I also encountered the same problem, because of the huge amount of data, I have to transfer the data processing from the local to our cluster. Although our cluster has many nodes, due to the kernel version of the cluster is too low (centos 6) and administrator permissions, I can only install singularity (version=2.4.6, our administrator said that the use of docker requires administrator privileges) to use fmriprep. Since the node calls need to be queued, my solution is to separate the tasks into 30 shares, and then each task commits to reduce the thread's call (but not too little, which will slow down the processing, depending on each node's threads). The 1200+ subjects were processed in about 3 weeks (AROMA was used and fs-reconall was not used). I think my solution is not a clever way, and I hope that if fmriprep can add GPU-accelerated features later, it will be a big improvement.


The package is empty

The package on npmjs.com is empty. The build log confirms it
https://github.com/simenandre/firestore-graphql-scalars/actions/runs/4805748867/jobs/8552476094#step:8:11

Haha! That is fun!
Let me have a look

Fixed in https://github.com/simenandre/firestore-graphql-scalars/releases/tag/v3.0.1

It seems the issues have been resolved:
<EMAIL_ADDRESS>


links to broader and narrower concepts can't be solved

After selecting a concept its relations and links to narrower and broader ceoncepts can't be solved via the X-button on the right side. So right now there is no possibility to remove a relation.

so with "can't be solved" you mean "can't be removed"?


Replace gutil.colors.stripColor with stripAnsi

Fixes #458

Good quick fix; though the issue should be raised with gulp-util to update their version of chalk. i have created an issue / PR here to help them: https://github.com/gulpjs/gulp-util/issues/120

Thanks but this was caused by a mistaken bad release in the chalk library which has since been pulled from npm.
https://github.com/chalk/chalk/issues/106


fzf fails to download and install

Category

[ ] fzf binary
[ ] fzf-tmux script
[ ] Key bindings
[ ] Completion
[ ] Vim
[ ] Neovim
[x] Etc.

OS

[x] Linux
[ ] Mac OS X
[ ] Windows
[ ] Etc.

Shell

[x] bash
[ ] zsh
[ ] fish

When I try to install from git according to readme, I get an error during install script:
Downloading bin/fzf ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (77) error setting certificate verify locations:
  CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
  - Failed to download fzf-0.13.2-linux_amd64 !!!

Looks like something's wrong with your installation of curl. The below command is used to download the binary, which for some reason fails on your system.
curl -fL https://github.com/junegunn/fzf-bin/releases/download/0.13.2/fzf-0.13.2-linux_amd64.tgz | tar -xz

I'm sure you'll be able to find the solution after some Googling.
Anyway, can you try this command instead? Does it work?
wget -O - https://github.com/junegunn/fzf-bin/releases/download/0.13.2/fzf-0.13.2-linux_amd64.tgz | tar -xz

Yes, wget works.

For what it's worth, copying certificates from one directory to another (as described in this SO question) fixed curl for me.

Thanks!

No problem, thanks.


Junk code at the end of dist/flatpickr.css?

@chmln dist/flatpickr.css recently got a lot larger... possibly due to the extra sourceMap code at the bottom. Is this intentional?

Its some postcss junk.
Frankly, I'm re-evaluating the need to have postcss in the first place.
Will likely phase it out once I conduct multi-browser tests

Ah, ok.

Gone with the removal of postcss


Add video duration limit option when recording on default camera.

Any update on this???

Any update on this???


3.1.8 throws "Id of WAL operation can not be duplicated"

OrientDB Version: 3.1.8
Actual behavior
Opening any db causes
java.lang.IllegalStateException: Id of WAL operation can not be duplicated
	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.fetchNextOperationId(OAbstractPaginatedStorage.java:6502)
	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreFromBeginning(OAbstractPaginatedStorage.java:6221)
	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.restoreFromWAL(OAbstractPaginatedStorage.java:6161)
	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.recoverIfNeeded(OAbstractPaginatedStorage.java:5320)
	at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:401)
	at com.orientechnologies.orient.core.db.OrientDBEmbedded.getAndOpenStorage(OrientDBEmbedded.java:500)
	at com.orientechnologies.orient.core.db.OrientDBEmbedded.openNoAuthorization(OrientDBEmbedded.java:455)
	at com.orientechnologies.orient.core.db.OrientDBEmbedded.openNoAuthorization(OrientDBEmbedded.java:79)
	at com.orientechnologies.orient.server.OSystemDatabase.openSystemDatabase(OSystemDatabase.java:93)
	at com.orientechnologies.orient.server.OSystemDatabase.checkServerId(OSystemDatabase.java:169)
	at com.orientechnologies.orient.server.OSystemDatabase.init(OSystemDatabase.java:157)
	at com.orientechnologies.orient.server.OSystemDatabase.<init>(OSystemDatabase.java:50)
	at com.orientechnologies.orient.server.OServer.initSystemDatabase(OServer.java:1275)
	at com.orientechnologies.orient.server.OServer.activate(OServer.java:445)
	at com.orientechnologies.orient.server.OServerMain$1.run(OServerMain.java:49)

@scrabb 3.1.x release is pretty old. Have you considered upgrading to the latest version?

@laa any update on this?

Hi guys, according to stack trace you try to recover crashed database. We do not support recovering crashed databases between versions. Please recover using the same version and then migrate to the new version.

@laa - I'm also facing the same issue. Can you please guide - how to migrate database to a new version which is recommended way?
My current OrientDB version is 3.0.27 which needs to be upgraded to 3.1.9
Copying the  from the databases folder of 3.0.27 to 3.1.9 databases folder, throws this exception.

We do not support recovering crashed databases between versions.

Not even between same minor versions (between 3.1.x)? That makes no sense in a production environment. See #9598.

FYI: This duplicate WAL Id issue is still present in 3.2.2. To reproduce the issue, sample database can be downloaded from #9542.


Use ip4s for CIDR and IP validating

https://github.com/Comcast/ip4s
This is something we could use both for our CIDR rules logic, as well as for any ip validations we do, which is awesome. Some thoughts on where:

A record data should be IPv4 type
AAAA record data should be IPv6 type
Batch PTR input should be IPAddress
Look at ReverseZoneHelpers file for misc cidr things (classless reverse zone delegations, and ACL rules use these)

Scope of this card includes:

Remove dependency on orchard CIDR library, use ip4s, make all tests pass.
Replace validations with ip4s where needed
Check the portal, there may be no changes there

Hundo-P - we should normalize on IP4S, we are already using it in most places and should do it in other places as well.  Rather low hanging fruit, could touch a fair bit of code, shouldn't be too complex


Stripe::Customer.create does not include a default_source

customer = Stripe::Customer.create({
  email<EMAIL_ADDRESS>  card: stripe_helper.generate_card_token
})

running the basic test example in the README, this customer does not have any default_source, also the sources.data field is empty.
But when I look at the code, https://github.com/rebelidealist/stripe-ruby-mock/blob/38b4a31e93844b991675b7298d564ea03edc253f/spec/shared_stripe_examples/customer_examples.rb#L17 I see the sources should be returned.

Sorry, I see the difference. The card field now is changed to source in the create. Just have to update the README. closing this.

@zhaohanweng thanks for this! Fwiw the README still says card :( In the first code block under https://github.com/rebelidealist/stripe-ruby-mock#description


Require all Model Outputs to be Logits

Some models currently allow for the direct application of Softmax or Sigmoid on their outputs (originally aimed at convenience; now basically legacy), e.g.: CNN10:
https://github.com/autrainer/autrainer/blob/d1ca7efbb2a42961ebf4313ef8546e8b1856678a/autrainer/models/cnn_10.py#L11-L19
However all target transforms expect model outputs to be the actual logits leading to a bug where e.g. for multi-label classification we would apply the Sigmoid function twice:

once in the model when passing sigmoid_output = True
once in the target transform when obtaining the predictions: https://github.com/autrainer/autrainer/blob/d1ca7efbb2a42961ebf4313ef8546e8b1856678a/autrainer/datasets/utils/target_transforms.py#L190-L202

This essentially leads to the issue that all model predictions are wrong, also causing the published autrainer/edansa-2019-cnn10-32k-t to be affected.

The question is, do we introduce a new method, i.e. use predict_batch or predict_probabilities to get the sigmoided/softmaxed outputs, and then postprocess_output to do what we now do in predict_batch?

During inference we have the following pipeline (M = Model, T = target transform): M.outputs -> T.predict_batch -> T.decode (map) -> (decoded).
During output tracking we currently call  M.outputs -> T.predict_batch -> T.decode (map) -> (save dataframe).
We could extend both pipelines to include e.g.  M.outputs -> T.probabilities_batch -> T.predict_batch -> ... with T.probabilities_batch applying the following operations:

(single-target) regression: identity OP; still scaled (no update of T.predict_batch required)
multi-class classification: Softmax (no update of T.predict_batch required; we only argmax the outputs)
multi-label classification: Sigmoid (remove Sigmoid from T.predict_batch)

Then in all cases we can save the outputs as well as the result of T.probabilities_batch (+ destructure the probabilities to separate columns in the inference results.csv).


Fix _nextFrameReadyEvent not initialised before display callback

Reported in discord.

Gonna merge because this is guaranteed to not go wrong now.


[CROSSDATA-202] Upgrade Spark to 1.6.1

Upgrade C* connector to 1.6.0-M2
Upgrade elasticsearch-spark to 2.3.1

LGTM

+1


What "main" distribution WSL 2 uses?

I am aware of wsl --system (CBL-Mariner) but as far as my understanding goes this doesn't log me into distribution that is behind regular user distros imported into WSL.
Is there any information about the "main" distribution used with WSL 2 kernel? I am just curious and would like to explore it by myself (like whether there are any differences in init process).
In the image below that i have from one presentation i can see it is Debian (probably more like Debian based). I found some information about CBL-Delridge which is Microsoft distro based on Debian so maybe it is CBL-Delridge?

Thx for any comments :)

Is there any information about the "main" distribution used with WSL 2 kernel?

There is no "main" distribution like thing. WSL2 will use the Linux distribution which is installed by the user.

Is there any information about the "main" distribution used with WSL 2 kernel?

There is no "main" distribution like thing. WSL2 will use the Linux distribution which is installed by the user. In that image, Debian is provided an example. It could be any Linux distribution or could be user's own distribution.

I believe the user distro is used in similar fashion like container image. So there is another distro behind.

There is only two distributions.

User distro run by user.
System distro CBL-Mariner.

There is only two distributions.

User distro run by user.
System distro CBL-Mariner.

Yes you are right my bad. I rewatched the presentation from where i have the image and there is nothing about debian based internal distro. In my remarks i have note that there is internal distro that "starts" user distros using linux namespace feature and i was sure it comes from the presentation where it was mentioned. Don't know how did i figured this out.
Thanks.

There is only two distributions.

User distro run by user.
System distro CBL-Mariner.

I found where i got the idea of "main" distro.
It is from this accepted answer https://superuser.com/questions/1719393/how-does-wsl-wsl2-wslg-work-without-systemd

WSL2:
It's important to understand that, when you are running a WSL2 distribution, that distribution is not running in a virtual machine. WSL2 itself is running a virtual machine with its own distribution (probably Mariner-based, if I had to take a guess).
Inside that hidden virtual machine is where your distribution is run in its own set of namespaces. If you run more than one distribution at the same time in WSL2, they are all running in that same VM, each with its own:

So i quess that part of answer on superuser is not correct?

So i guess that part of answer on superuser is not correct?

I do not know what the author @NotTheDr01ds meant to state. The answer is not wrong but the first paragraph is somewhat ambiguous.
Also, what do you mean by "main" distribution? The terminology may not be right.

So i guess that part of answer on superuser is not correct?

I do not know what the author @NotTheDr01ds meant to state. The answer is not wrong but the first paragraph is somewhat ambiguous.
Also, what do you mean by "main" distribution? The terminology may not be right.

My view of state is that WSL kernel is used together with Microsoft based distribution (something like CBL-M or CBL-D) that is probably not open sourced at all and this "main" distribution contains stuffs (something similar to docker) that are responsible for "starting" user distro in new nemespace when wsl -d <name> is typed in terminal.
Basically i thought that the linux namespace and cgroups features are used from some user space daemon that is part of the Microsoft "main" distribution and this user space daemon communicates with windows services.
Thanks.

OK, understood. The "main" distribution you are talking about is called system distribution here. Just some terminology.
Yes, you are right. Just small correction, the user space daemon, the /init binary, is not part of CBL-Mariner distribution. The /init binary is either in C:\Windows\System32\lxss directory or in WSL from Windows Store installation directory.

...that is probably not open sourced at all

The CBL-Mariner distribution is open-source, MIT licensed, see https://github.com/microsoft/CBL-Mariner. But the /init binary is closed-source.

OK, understood. The "main" distribution you are talking about is called system distribution here. Just some terminology.

Thanks for helping me to clarify this. Anyway i have one more thing. I am aware of the wsl --system which logs me into so called system distribution with wslg user.
You think that exactly this distribution instance is what i call "main" distribution? Because i think it was introduced for the purpose of wslg so there have to be another distribution that existed before wslg. Or we call all of these "internal" distributions as system?

You think that exactly this distribution instance is what i call "main" distribution?

Yes. You can look into the init binary if you want. In the recent WSL releases, the system distribution is mounted first to do some initialization like creating root file system directories, configure network, create swap, configure time etc. wslg is part of another job for that system distribution.
BTW, I did not develop WSL and I just found the process while playing with it.

You think that exactly this distribution instance is what i call "main" distribution?

Yes. You can look into the init binary if you want.

Great, will definitely look at the binary. Thanks.

@jhribal  and @Biswa96 Sorry for not spotting this sooner - I just happened to come across it while searching for "mount" issues for some ideas.
I do believe that there are actually three "distributions" involved:

There is a distribution running in the WSL2 VM itself that is shared amongst all running WSL2 distributions.  We can access some portion of it via wsl --debug-shell.  Note that:

This is the same root namespace that used to be activated via the now-deprecated debugShell=true in .wslconfig
It is different than the WSLg system distribution we see with wsl --system -d <distro> in that it is the parent namespace to those system distributions.
Unlike the System distributions, it must be launched as a Windows administrator
From inside this root namespace shell, we can see all processes running in all WSL2 distributions (including the WSLg system distributions).  For instance, try starting two different WSL2 distributions.  In one, run sleep 20000 and in the other, sleep 10000.  From the debug-shell, run ps axf and you'll see both sleep commands running in their respective distributions.
According to cat /etc/os-release, this distribution is also CBL-Mariner.
It is not, to my knowledge, possible to override this root distribution.

For each running distribution, there is a running WSLg system distribution:

This distribution can be accessed with wsl --system -d <distro>.
The WSL system distribution shares some namespaces with the "user" distribution (per the README).  For instance, if you run the same sleep 10000/sleep 20000 test as above, you'll be able to see only one of those processes in the system distribution process tree.
This is where Weston/Xwayland/Pulse runs, again for each user distribution.
Per the contributing guide, you can override this distribution with your own using the systemDistro setting in .wslconfig.  AFAIK, this would not change the root VM distribution (see below).

Of course, as long as we are enumerating the distributions, the third and final category is the user distribution, which is what we as end-users use directly.  There can be multiple distributions installed and running at one time.

While I have never built my own system distribution, all indications are that this would not change the root VM distribution.  For starters, the System contributing page defines the system distribution as:

essentially a Linux container packaged and distributed as a vhd

In other words, it is similar to a user distribution in that it is running in its own namespace container.
The root VM distribution, by its nature, cannot be a container.  It must be a virtual machine which runs the actual Linux kernel for each of the distributions (a.k.a. containers) to run inside.
In addition, you can do a simple test of touching a new file in, for example /usr/lib in both the Debug Shell and a system distribution.  You'll find that neither of these distributions sees the changes in /usr/lib from the other distro.
I'd welcome more discussion on this, since I'm just mucking around here myself and don't have any specialized knowledge outside what I'm reading and able to experiment with.  However, it would probably be better for this to have been on the Discussions.
That, or you can (as you probably know) typically find me monitoring for WSL questions on various Stack Exchange sites:

Ask Ubuntu: for Ubuntu specific questions
Super User: for general WSL questions
Stack Overflow: for programming questions around WSL
Unix & Linux:  If you really feel your question falls more into Linux territory than Windows (this one might have been interesting there).

We can access some portion of it via wsl --debug-shell

I have look into the init binary. I think that the debug shell is from CBL-Mariner distribution. It is mounted and forked before running WSLg and user distribution. Our observation may differ. Let us wait when WSL becomes open source 🙃

@jhribal and @Biswa96 Sorry for not spotting this sooner - I just happened to come across it while searching for "mount" issues for some ideas.
.
.
.

Thank you very much for this comprehensive answer. It looks like that i finally found what i called the main distribution. I didn't payed much attention to wslg so i didn't noticed the remark that every user distro has its own system distribution and i accepted that wsl --system will log me into the main distribution.
The so called main or root distribution (basically the first one used directly with kernel) is the distribution accessible from terminal using wsl --debug-shell. Terminal must be run as administrator and wsl have to be already started.


Print statistics and attributes of both inputted and new lists

It could either be on or off by default, not sure yet. Things to include:

[x] list length
[x] entropy per word
[x] shortest word length
[x] longest word length
[x] minimum edit distance
[x] minimum unique prefix
[x] is over brute force line for English lowercase
[x] if free of prefix codes

Maybe using a table-printing crate like prettytable-rs or comfy-table.

I've now calculated all but one of the desired statistics in a function in main.rs. Rather than use a table crate, I'm just printing it as nicely as I can by hand. Example:
Attributes of new list
----------------------
List length            : 5133
Entropy per word       : 12.3256
Length of shortest word: 3 (aim)
Length of longest word : 9 (zoologist)
Free of prefix words   : true
Above brute force line : true
Shortest edit distance : 2

Done! Think I got the "Unique character prefix" statistic correct.
Could still explore using a crate to make a nicer table.


Update template notification settings and add a way to install plugins

Suggestions:

Change :immediately to :delayed

template "/etc/default/go-server" do
  source   "go-server-default.erb"
  mode     "0644"
  owner    "root"
  group    "root"
  notifies :restart, "service[go-server]", :immediately
end

Add something along the lines of this:

Where plugins is a hash map:
"github-pr-status": "https://github.com/gocd-contrib/gocd-build-status-notifier/releases/download/1.1/github-pr-status-1.1.jar"
node['gocd']['server']['plugins'].each do |pluginName, pluginURI|
  remote_file "#{node['gocd']['server']['work_dir']}/plugins/external/#{pluginName}.jar" do
    source pluginURI
    owner 'go'
    group 'go'
    mode 0770
    retries 5
    notifies :restart, 'service[go-server]', :delayed
  end
end

Change :immediately to :delayed

Seems like a good idea.

... plugins

How about an LWRP?
gocd_plugin 'github-pr-status' do
  url '"https://github.com/gocd-contrib/gocd-build-status-notifier/releases/download/1.1/github-pr-status-1.1.jar"'
end

That would be perfect, I'm just too lazy myself :)

I have added small gocd_plugin resource


ng-options group by duplication (1.4.8)

 <select ng-model="ctrl.model" ng-options="reason.description group by reason.type for reason in ctrl.possibleReasons"></select>

With a collection of:
this.possibleReasons = [{ type : 1, description: 'Test1'}, { type: 2, description: 'Test2'}];

produces a dom element that has 4 opt-groups output instead of the expected two. This creates a duplication of the groups within the select element.

Duplicate of #13487.
It has already been fixed in master. The fix will land with v1.4.9 and v1.5.0-rc.1.


Add image with NeptuneUI

Adds an image that includes NeptuneUI. In order to keep a qt free version of the Pelux reference this is done as a new image.

(assuming you are speeking of the .bbappend) Yes I agree.


[Feature request]: Ability to specify filter sets per resource type

Describe the proposed solution
Ability to specify which resource types applies to which filters.
Say we have both TaxonImage and ProductImage implement the ImageInterface from the plugin both of these would be considered processable resources.
However, you typically has a set of filters that applies only to product images, and another set of filters that applies to taxons. Currently, taxon variants would be generated for all ProductImages, and the other way around, all product variants would be generated for all TaxonImages.
In most cases it might not be a huge problem, since there are often significantly less taxon images than product images, and fewer taxon specific variants than product specific variants, but it is easy to imagine scenarios where this might not be true.
Describe alternatives you've considered
No response
Additional context
No response

From my point of view there are two different ways the configuration format can be expanded to accomodate this information.
Either
a) Resources are specified per variant
b) Variants are specified per resource
A
setono_sylius_image:
    filter_sets:
        app_one:
            resources:
                - 'sylius.product_image'
                - 'sylius.taxon_image'
                - 'app.energy_label_image'
        app_two:
            resources:
                - 'sylius.product_image'
        app_three: ~

We could assume that if resources are not specified, we default to all eligible resources (resources that implement the ImageInterface)?
Otherwise the resources node acts as a 'whitelist'
B
setono_sylius_image:
    resources:
        sylius.product_image: # are dots even allowed in keys?
            - 'app_one'
            - 'app_two'
        sylius.taxon_image:
            - 'app_one'
        app.energy_label_image:
            - 'app_one'
        app.some_other_image: ~

The resources section could serve two purposes

A place to specify which resources the plugin should handle (replacing default behaviour of ProcessableResourceProvider?) - this is not necessarily relevant to this issue, but if this config format is chosen, it could be confusing (in this case perhaps rename resources key)
A place to specify variants per resource type

Similarly to A - ~ would specify "all variants" and otherwise it acts as a whitelist for variants to generate
Pros / cons
I'm personally in favor of A. Neither format can avoid some repetition.
Storing info in Variant
Regardless of whether approach A or B is chosen, I guess this info should be added on Variant.
/** $resources array<array-key, string> */
public array $resources;

And then when a decision needs to be made perhaps a service with access to %sylius.resources% that can infer the information from a Variant and an ImageInterface. Something like:
<?

class VariantEligiblityChecker
{
    private array $resources;

    public function __construct(array $resources /* sylius.resource */)
    {
        $this->resources = $resources;
    }

    public function isEligible(ImageInterface $image, Variant $variant): bool
    {
        ....
    }
}

We go with the proposed solution B acknowledging that it has some unresolved issues around how to version these things "per resource" or force updates of just a specific kind of resources.
Choosing B also allows for future addition of settings per resource


Import contains rule

This adds a ContainsRule class that stores information for constraints that use the contains keyword https://github.com/HL7/fhir-shorthand/wiki/2.4-Constraints. It also adds logic to the importer that will use the ContainsRule class to store information when the contains keyword is used on a Profile or an Extension, and it adds tests of this functionality.
This also update .gitignore so that any directory starting with out is ignored. I was running the cli a few times and this became an issue for me.

I went with item and also used the existing functionality for resolving an Alias when setting a value to set the item on the ContainsRule.


Password with special characters

The login over this Api on the UniFi controller will fail, when the used password contains a ". Not exactly sure why it fails, but UniFi allows such passwords.

Thanks for reporting! That should be easy enough to reproduce and fix. I'll look into it.

The dot doesn't seem to be the issue. Are there more special characters in the password? I'll try some of those.

Also that seems to work well. I.e. I created a temporary account with the password Jemsf%_&Sk#fek.34324 and it authenticated without any problems. Can you provide more information on what exactly goes wrong and against what kind of UniFi controller you are authenticating? Is it self hosted, cloud hosted, cloud key, which version, etc?

Jemsf%_&Sk#fek.34324

does not contain any quotation mark (" or '). I had trouble with a password that contains quotation marks. I think it has to do with the escaping before sending the login details per Json to the controller.
I have a self hosted controller, but the login worked in the browser just not with the API. I also checked my code with the debugger, the password was correct before sending it to the controller.

Ah I read your message as the dot being the guilty one, not the quotes. Let me try it with the quotes as well.

Found the issue. Releasing a fix for it now.

Fixed in version <IP_ADDRESS>. Thanks for reporting it.


Unify permissions and permission checks throughout dojo

Is your feature request related to a problem? Please describe
Concerns have already been brought up in https://github.com/DefectDojo/django-DefectDojo/issues/1048, where @valentijnscholten  summarizes the problems of the existing permission and role handling.
Describe the solution you'd like
As I'm currently developing an entirely new report builder component for dojo, I thought of starting to follow a different approach on how to handle permissions.
There exists a small and well-matured framework for object-level permission checks rules, which makes declaring permissions for objects of a model very straightforward and expressive. It integrates well with django through an authentication backend, which allows checking permissions via the regular User.has_perm() method and some decorators and view mixins for convenience.
Permissions will have to be declared centrally and only once per model using predicates, like so:
import rules

@rules.predicate
def is_authorized_for_product(user, product):
    return product.authorized_users.all().exists(user)

rules.add_perm("dojo.add_product", rules.is_staff)
rules.add_perm("dojo.change_product", rules.is_staff | is_authorized_for_product)
rules.add_perm("dojo.delete_product", rules.is_staff | is_authorized_for_product)
rules.add_perm("dojo.filter_product", rules.is_authenticated)
rules.add_perm("dojo.view_product", rules.is_staff | is_authorized_for_product)

It'll be crystal clear how will be allowed to perform a specific action.
Django has become smarter over the last years as well, hence I'm developing my new component with class-based views where appropriate. I implemented a view mixin which automatically requires a user to have permission to do the action the view it was mixed in performs and raise a 404 to hide the object otherwise. Views for filtering and deletion could then look as simple as:
from django.views.generic.edit import DeleteView
from django_filters.views import FilterView

from dojo.permissions import DojoUserPermissionMixin
from dojo.filters import ProductFilter
from dojo.models import Product

class ProductFilterView(DojoUserPermissionMixin, FilterView):
    filterset_class = ProductFilter

class ProductDeleteView(DojoUserPermissionMixin, DeleteView):
    model = Product

Everything else is taken care of automatically. No more manual object retrieval, explicit permission checking, context construction and boiler-plate form declarations. While this isn't strictly related to reworking the permission checking, it shows how nice it plays with class-based views.
A @require_permission("dojo.delete_product") decorator for legacy function-based views is provided as well.
Even integration with the rest framework can be done easily.
I think this is a great approach to go away from opaque user roles to a well-understandable role model with, and that's the most important, centrally managed permissions. My overall goal is to make the code base of dojo clearer and less repetitive, both in views and templates, to which this is a first step.
How about using rules for the new report builder component as a prototype and then rolling it out to the other models step by step? This is a non-invasive change that won't affect components it hasn't been applied to in any way, so it sounds reasonable to me.
Best regards
Robert

Ah, the second half of this is an addition to django.db.models.Manager, which I subclassed to allow providing a queryset with restrictions for the current user applied. In the case of products, it would look like:
class Product(models.Model):
    objects = DojoModelManager(
        filter_user_qs=lambda qs, user:
            qs if user.is_staff else qs.filter(authorized_users__in=[user])
    )

    # ... normal model declaration

A filterset class like ProductFilter would then be changed to inherit from DojoFilterSet and the base queryset is restricted automatically. The same goes for the API viewsets, which can have their queryset restricted as well by just adding a custom mixin.

If you want more detailed examples and code on how I plan to implement this, let me know. Probably @aaronweaver is the one who decides about this after all.

@efficiosoft I think this makes a lot of sense. My only suggestion at the moment is that we should do this after migrating to python3. We did a PoC on that and it went fairly well and as the deadline looms for EOL on 2.

@aaronweaver Glad to hear you see it this way, too.
I'm currently working on this branch and everything I did so far is compatible with:

Python 2.7/3.5+
Django 1.11/2.2+
django-filter 1.1/2.0+

as is with TODOs with suggestions at places where code could be simplified or changed to be more performant after Python 2 support was dropped. The changes of existing code are very minimal (<100 lines, most of which is just spacing), as said I'll only implement this for my new component first and then it can be rolled out everywhere.
Do you have an approximate date for the switch to Python 3? Which one, 3.7?

I like this initiative and am happy to help testing.

@valentijnscholten You're very welcome. I'll let you know when my branch, especially the generic pieces for permission handling, are in a testable state.

@efficiosoft Sounds good, then feel free to add a PR when ready. I believe we will target early late June. The GSOC folks are working on unittests so that python3 will have the necessary testing that has been missing.

Just want to give short feedback. I rebased the new code onto latest dev with Python 3 this morning, which was really easy and works as expected. The docs aren't ready yet and I need to do some code cleanup, but then I'll file a PR with my new report builder, which will be integrating new object level permissions using the rules framework and a set of template snippets and classes to simplify enforcing the permissions.

@efficiosoft Glad to hear Python3 went smoothly. Looking forward to the PR!

Hi @efficiosoft wanted to check in. Did you need any help with this? This is one we're all very excited about and what to do what we can, should you need anything.

Hi @devGregA and @valentijnscholten
A design review would be great indeed. The code can be found in this branch. Let me describe how the whole system works.

I separated base classes and helper functions that are used for all models into dojo.models_base.

Each component (like product, finding etc.) can have its own models.py file in the corresponding subdirectory, like dojo.reportng.models. All the models are then imported from the main dojo.models so that django can register them as normal. When fully migrated one day, this split will help us keep model functionality separate without producing circular import problems.

In a component's models.py, models are created as normal, using the bases and helpers from dojo.models_base. Additionally, this is where the rules framework is used for declaring object-level permissions for each model in a unified way, right after declaring the model itself. See dojo.reportng.models.

There is a new module named dojo.mixins, which provides some concise but powerful mixins to be used for creating class-based views per model which then take care of checking permissions automatically (and some extras). Integration with rest framework is also present there. See how that works in dojo.reportng.views.

A huge part of Dojo are the filtersets. In dojo.filters, there is the new DojoFilterSetNew (can be renamed later easily). It bundles pagination support, multi-level ordering, selecting individual objects for bulk actions and a little more, all of which works out of the box with the new template snippets (see below). Working implementations of such filtersets are in dojo.reportng.filters, split up per component like the models.

Finally, to bring this down to the template level, there are a number of new template snippets in dojo/templates/dojo/snippets. All of them follow a common pattern regarding their usage. A README.rst describing the usage can be found in this directory as well.

The class-based views by default automatically derive template names from the model name and type of view. This will unify all the different template naming schemes that have evolved over the years. Check the reportng_*.html templates to see it in action.

Just a few new template tags (dojo.templatetags.dojo_util_tags) have been registered to be available in all templates implicitly.

The code is fully documented with docstrings and comments, so it should be relatively straightforward to use. New files have been formatted using the black code formatter, which will now be used for the django project as well and produces beautifully unified code. I've chosen Python 3.6 as the target platform, but could be changed to 3.5 as well if required.
There also is user documentation for ReportNG, should you want to try it out to see how all the parts play together.
It happened that this new base parts are bundled with the new report builder I wrote. While this makes the diff huge, it provides a real world application using the new code. Code-wise, the base parts are completely decoupled from ReportNG, so they could be broken out easily.
A requirement was that old and new-style components can coexist. The changes don't interfere with existing code and everything will run as normal.
After all, the code complexity and expressiveness can be reduced dramatically with this new approach. While everything works great so far, the design is by no means fixed. Any suggestion is highly welcome!
Best regards
Robert

Sounds great. Haven't looked at the code yet, but how would an admin assign permissions for a product to a user?

Sounds great. Haven't looked at the code yet, but how would an admin assign permissions for a product to a user?

The same way as before. This changes mainly provide a way to declare how permissions are checked. Have a look at the Product model which now uses the new DojoModelManager.

The view that actually changes the authorized_users however, will be an UpdateView and thus be guarded by the dojo.change_product permission for the product in question.

@valentijnscholten I edited point 3 in yesterdays comment to make it more clear.

Well, we now also have inheritance of related fields when filtering a user's base queryset, so that we can express something like this, without having to repeat the conditions for product:

Only show engagements whose corresponding products would be shown as well.

If I remember correctly, @aaronweaver was interested in this as well, so you're of course also invited to review.

Or would it help if I create a PR? Actually, I don't want to have the changes merged as they are right now, so I thought an issue would be better suited for the time being.

I think a PR is easier to see the diff etc. Are there any unit tests to tests the new permission handling?

I think a PR is easier to see the diff etc. Are there any unit tests to tests the new permission handling?

No tests until we decide for an API design. This is a fundamental part that'll change how views can be written in the future and I want to get that finalized before investing time in unittests.
That branch contains the reportng directory, which isn't related to the permission handling at all and would make the diff quite large.
The two new files we have to talk about are models_base.py and mixins.py and the diff in filters.py. Everything else is just an application of the new features.

PR is created. Let's continue the discussion over there.


Fix invalid config example

Commit Message: docs: fix invalid config example
Additional Description:
Risk Level: Low
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Deprecated:]
[Optional API Considerations:]

hi @exoego thanks for this
we have a system to check standalone yaml files which should catch this (there is also one for snippets in docs but not sure why it but doesnt always catch issues)
would you be willing to move this snippet to a full config and replace the code-block with a literalinclude ?
if not i can land this and follow up myself, but as it stands i have no immediate way of testing the change

@phlax I will try that!

this is failing the example configs test here https://dev.azure.com/cncf/envoy/_build/results?buildId=88372&view=logs&j=8c169225-0ae8-53bd-947f-07cb81846cb5&t=6b0ace90-28b2-51d9-2951-b1fd35e67dec&l=22272
its kinda why i wanted to move it to this validation, as it picks up more issues
in this case tho, im not sure if its the tests or the config that is at issue - checking the fail log in test artefacts it says
unknown file: Failure
C++ exception with description "Function call: customStatNamespaces
    The mock function has no default action set, and its return type has no default value set." thrown in the test body.

ill investigate a bit further, but it would also be helpful if one of us could try starting envoy with the config in this PR to see if it works outside tests

@exoego see my last comment above
the easiest way for this to work is have the literalinclude point to the existing wasm yaml example file (and not having the yaml file point to the binary as here)
if that example file does not cover what is required, then you can just use lib/...wasm for the file
where it is tested in ci the path will always be lib/...wasm

@exoego it would be good to land this update/fix - let me know if there is anything i can do help with that