[ENH] New Machine Learning Features

Brief Description
I've been thinking of a few ML additions for pyjanitor. Below are 3 ideas that I think would be helpful to add to the API:

A method to drop feature based on a variance threshold (think sklearn VarianceThreshold)
A method to cap outliers that fall below or above a certain quantile (if below .05 replace with .05 or specified value, if above 0.95 replace with 0.95 value or specified value)
Simple standardize and scale methods (think sklearn StandardScaler and MinMaxScaler)

I realize that (1) and (3) above have sklearn implementations, but I am thinking it might be nice to have a simplified version of these that we can reach for when wanting to do some quick analysis/data prep without having to load sklearn and fit the object. This might be nice for pyjanitor's chaining approach. Notes in the docstrings can point users to sklearn for more robust solutions (and if desiring to retain information to apply to other datas).
Example API
# Drop features based on variance threshold
>>> df = pd.DataFrame({
...     'A': [1, 2, 3, 4, 5],
...     'B': [2, 2, 2, 2, 2],
...     'C': [1, 1, 1, 1, 2]
... })
>>> df
   A  B  C
0  1  2  1
1  2  2  1
2  3  2  1
3  4  2  1
4  5  2  2
>>> df.var()
A    2.5
B    0.0
C    0.2
dtype: float64
>>> threshold = 0.0  # default
>>> keep_features = df.var() > threshold
>>> df.loc[:, keep_features]
   A  C
0  1  1
1  2  1
2  3  1
3  4  1
4  5  2
>>> threshold = 0.5  # custom
>>> keep_features = df.var() > threshold
>>> df.loc[:, keep_features]
   A
0  1
1  2
2  3
3  4
4  5

# Cap outliers based on certain quantile
>>> df = pd.DataFrame({
...     'A': [2, 4, 8, 16, 32],
...     'B': [-999, 2, 4, 6, 999]
... })
>>> df
    A    B
0   2 -999
1   4    2
2   8    4
3  16    6
4  32  999
>>> df.quantile(.05)
A      2.4
B   -798.8
Name: 0.05, dtype: float64
>>> df.mask(df < df.quantile(.05), df.quantile(.05), axis=1)
      A      B
0   2.4 -798.8
1   4.0    2.0
2   8.0    4.0
3  16.0    6.0
4  32.0  999.0
>>> df.quantile(.95)
A     28.8
B    800.4
>>> df.mask(df > df.quantile(.95), df.quantile(.95), axis=1)
      A      B
0   2.0 -999.0
1   4.0    2.0
2   8.0    4.0
3  16.0    6.0
4  28.8  800.4
>>> df.mask(df > df.quantile(.95), df.quantile(.95), axis=1).mask(df < df.quantile(.05), df.quantile(.05), axis=1)
      A      B
0   2.4 -798.8
1   4.0    2.0
2   8.0    4.0
3  16.0    6.0
4  28.8  800.4
>>> df.mask(df > df.quantile(.95), 42, axis=1).mask(df < df.quantile(.05), -1, axis=1)
    A   B
0  -1  -1
1   4   2
2   8   4
3  16   6
4  42  42

# Standardize
>>> df = pd.DataFrame({
...     'A': [2, 4, 8, 16, 32],
...     'B': [1, 2, 3, 4, 5],
...     'C': [-15, -10, 0, 10, 15],
... })
>>> df
    A  B   C
0   2  1 -15
1   4  2 -10
2   8  3   0
3  16  4  10
4  32  5  15
>>> (df - df.mean())/df.std(ddof=0)
          A         B         C
0 -0.953206 -1.414214 -1.315587
1 -0.769897 -0.707107 -0.877058
2 -0.403280  0.000000  0.000000
3  0.329956  0.707107  0.877058
4  1.796427  1.414214  1.315587

# Just as a sniff test (can have ddof as an argument into the function)
# https://drorata.github.io/posts/2017/Sep/10/why-do-we-need-to-divide-by-n-1/index.html#Summary
>>> from sklearn.preprocessing import StandardScaler
>>> standardizer = StandardScaler()
>>> standardizer.fit_transform(df)
array([[-0.95320625, -1.41421356, -1.31558703],
       [-0.76989735, -0.70710678, -0.87705802],
       [-0.40327957,  0.        ,  0.        ],
       [ 0.32995601,  0.70710678,  0.87705802],
       [ 1.79642716,  1.41421356,  1.31558703]])

# Scale Method
>>> df
    A  B   C
0   2  1 -15
1   4  2 -10
2   8  3   0
3  16  4  10
4  32  5  15

>>> (df - df.min()) / (df.max() - df.min())
          A     B         C
0  0.000000  0.00  0.000000
1  0.066667  0.25  0.166667
2  0.200000  0.50  0.500000
3  0.466667  0.75  0.833333
4  1.000000  1.00  1.000000

>>> (df - df.min()) / (df.max() - df.min()) * (1 - -1) + -1
          A    B         C
0 -1.000000 -1.0 -1.000000
1 -0.866667 -0.5 -0.666667
2 -0.600000  0.0  0.000000
3 -0.066667  0.5  0.666667
4  1.000000  1.0  1.000000

# Just as a sniff test
>>> from sklearn.preprocessing import MinMaxScaler
>>> scaler = MinMaxScaler()
>>> scaler.fit_transform(df)
array([[0.        , 0.        , 0.        ],
       [0.06666667, 0.25      , 0.16666667],
       [0.2       , 0.5       , 0.5       ],
       [0.46666667, 0.75      , 0.83333333],
       [1.        , 1.        , 1.        ]])
>>> scaler = MinMaxScaler(feature_range=(-1,1))
>>> scaler.fit_transform(df)
array([[-1.        , -1.        , -1.        ],
       [-0.86666667, -0.5       , -0.66666667],
       [-0.6       ,  0.        ,  0.        ],
       [-0.06666667,  0.5       ,  0.66666667],
       [ 1.        ,  1.        ,  1.        ]])

@ericmjl curious to hear your thoughts before starting to develop any/all of these ideas. Thanks!

FYI, we have a min_max_scale already https://pyjanitor-devs.github.io/pyjanitor/reference/janitor.functions/janitor.min_max_scale.html#janitor-min-max-scale

These are great functions to have as a convenience, @loganthomas! I'm in favour of having these in the library. The docstrings, however, should come with a strong warning: in proper machine learning practice, any transformations should still be part of an sklearn pipeline so that we don't have info leakage from the training set into the test set. (How that happens goes deep, we can definitely talk about this in more detail.)
Super glad you brought this one up, Logan!


Make abi3audit python 3.8 compatible

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

LGTM overall! I just cut 2023.01.27 for abi3info, so bumping to that here should resolve the compatibility problem on 3.8.

Used newer version of abi3info and also used future annotations instead of Dict/Set/etc.


installing openh264 in firefox on linux arm64

Hello.
I'm trying to install the OpenH264 Cisco vide codec on my Firefox, Kali Linux ARM64.
It cannot be installed via Firefox, though. Could you please give me a piece of advice?
Thanks,
Vincenzo

Does it work if you use Flatpak Firefox? Flatpak's ffmpeg is compiled with noopenh264 which can make use of downloaded openh264.

Hello Darkspirit.  What should I do once Flatpak Firefox is installed?

I believe this issue can be closed now. We started shipping OpenH264 binaries for Linux ARM64 in early 2023.
https://bugzilla.mozilla.org/show_bug.cgi?id=1795014


plugin images too large

Your 3 plugin images, when held in memory, consume 26mb of memory. Please shrink the images down to a sane size.

Resolved in 8eb07afaff44d5b24560a6f650e608c0b5463670


Style Birthday sign-up form field

When date of birth is included in the signups form, it styled differently from others.

fixed


Proxy checker

Hi cool idea
Would be nice if it has proxy checker too because bot all proxy works

That would be cool, It would be great if you could open a PR with implementation.


how to set spacebar

i want to make the second button spacebar

https://github.com/kuroni/bongocat-osu/wiki/Settings


fix database name contains uppercase

Fix the ingestion failure when the database name contains uppercase, and the package library name does not exist
#7208
Checklist

[ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[x] Links to related issues (if applicable)
[x] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
[ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

This seems dangerous, as it will change existing URNs for folks. This means broken links, duplicate entities, and worse.

This change is not needed, as the reported issue is already fixed for mysql through two tier sql datasources in latest versions.


[BugFix] AWAE attempting to get size of possibly unallocated array

This is ready to merge.
Feature or improvement description
PR #1464 introduced a new Lidar module into InflowWind.  There are some arrays that don't get allocated if it is not used, but the size of these arrays is checked inside the AWAE module.  Taking the size of an unallocated array will give an ambiguous result, which can lead to segmentation faults.
Some if(allocated()) tests have been added to correct the issue.
Related issue, if one exists
#1521
#1464

Closes #1521


udf + s3a write

tried but this is 10x slower
2 reasons

returning a list in an udf results in ton of useless serializatio
s3a is slow

this results in significantly increased memory usage

not improved by .config('spark.hadoop.fs.s3a.bucket.all.committer.magic.enabled', 'true')
maybe using an rdd and doing the partitioning different could do the trick


Failing CI

CI is currently failing.
https://travis-ci.org/local-npm/local-npm
Detailed Description

CI is failing due to API changes.

working on it https://github.com/local-npm/local-npm/projects/1


fix: Performance issue in FluentIcon

🐛 Bug Report
Multiple FluentIcons in a FluentDataGrid lead to very slow repaint (250ms and higher).
The repro contains two pages that are identical, except one page has a FluentButton with a FluentIcon in each data grid row (performance issue) and the other page has a FluentButton with text in each data grid row (ok).
The delete buttons in the repro toggle the visibility of a loading indicator so the performance issue is visible directly.
💻 Repro or Code Sample
Click me
🤔 Expected Behavior
When clicking the delete button, a repaint is triggered which should not take more than a few ms.
😯 Current Behavior
When clicking the delete button, a repaint is triggered that can take more than 250ms.
🔦 Context
I added a delete button to each data grid row (with a delete icon).
🌍 Your Environment
Windows 11
Visual Studio 2022
Microsoft Edge
.NET 7.0
ASP.NET Core Components WASM 7.0.3 (see project file)
FluentUI Blazor 2.1.1 (see project file)

Further investigation has revealed that the delay lies (very) deep in the wasm bowels (apparently in function 221):

If anyone has knowledge that can help here that would be much appreciated. Until then this will stay like it is, I guess.


setup error: FATA[0000] could not open key file: open configuration/hs256.json: no such file or directory  env=dev

on clean setup....
seems i need a config.
x-MacBook-Pro:papernet apple$ mkdir data
x-MacBook-Pro:papernet apple$ go run cmd/cli/*.go index create --index=data/papernet.index --mapping=bleve/mapping.json
Index created at data/papernet.index
x-MacBook-Pro:papernet apple$ go run cmd/web/main.go
../../gin-gonic/gin/binding/default_validator.go:7:2: cannot find package "gopkg.in/go-playground/validator.v8" in any of:
	/usr/local/opt/go/libexec/src/gopkg.in/go-playground/validator.v8 (from $GOROOT)
	/Users/apple/workspace/go/src/gopkg.in/go-playground/validator.v8 (from $GOPATH)
x-MacBook-Pro:papernet apple$ go get gopkg.in/go-playground/validator.v8
x-MacBook-Pro:papernet apple$ go run cmd/web/main.go
FATA[0000] could not open key file: open configuration/hs256.json: no such file or directory  env=dev
exit status 1
x-MacBook-Pro:papernet apple$ go run cmd/cli/*.go index create --index=data/papernet.index --mapping=bleve/mapping.json
Index data/papernet.index already exists, no need to create it
x-MacBook-Pro:papernet apple$ go run cmd/web/main.go
FATA[0000] could not open key file: open configuration/hs256.json: no such file or directory  env=dev
exit status 1

so i figured maybe the CLI allows me to create my auth key. But no luck..
x-MacBook-Pro:papernet apple$ go run cmd/cli/*.go auth
FATA[0000] could not open key file: open configuration/hs256.json: no such file or directory  env=dev
exit status 1

is there a CLi command to create the required hs256.json credrentials ?
Or some readme instructions ?

Indeed, it is not specified anywhere. I'll add that to the README :+1:
I used https://mkjwk.org/ to generate it with the following:

Also, I'm currently migrating everything to go-kit, this is why it's all a bit messy.

ok. I grabbed the keypair resultset ( the middle pane).
slapped it in the hs256.json file.
Now getting further 👍
x-MacBook-Pro:papernet apple$ go run cmd/web/main.go
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:	export GIN_MODE=release
 - using code:	gin.SetMode(gin.ReleaseMode)

[GIN-debug] GET    /ping                     --> github.com/bobinette/papernet/gin.New.func3 (4 handlers)
[GIN-debug] GET    /auth/v2/me               --> github.com/bobinette/papernet/gin.wrapH.func1 (4 handlers)
[GIN-debug] POST   /auth/v2/bookmarks        --> github.com/bobinette/papernet/gin.wrapH.func1 (4 handlers)
[GIN-debug] GET    /auth/v2/teams            --> github.com/bobinette/papernet/gin.wrapH.func1 (4 handlers)
[GIN-debug] POST   /auth/v2/teams            --> github.com/bobinette/papernet/gin.wrapH.func1 (4 handlers)
[GIN-debug] DELETE /auth/v2/teams/:id        --> github.com/bobinette/papernet/gin.wrapH.func1 (4 handlers)
[GIN-debug] POST   /auth/v2/teams/:id/invite --> github.com/bobinette/papernet/gin.wrapH.func1 (4 handlers)
[GIN-debug] POST   /auth/v2/teams/:id/kick   --> github.com/bobinette/papernet/gin.wrapH.func1 (4 handlers)
[GIN-debug] POST   /auth/v2/teams/:id/share  --> github.com/bobinette/papernet/gin.wrapH.func1 (4 handlers)
FATA[0000] could not instantiate google service          env=dev
exit status 1

Hmm oops, it seems I forgot to log the error here: https://github.com/bobinette/papernet/blob/master/cmd/web/main.go#L109
If you add it, the message will definitely be clearer
I think you are missing the google configuration to use google login. You need credentials to be able to start the server. I'll add that to the README as well, and if/when I have the time, make a no-auth version of it.
If you are using the front as well, I think you might be able to skip the google part already by:

removing the lines in web/cmd/main.go dealing with google auth
creating a user via the cli
changing the initial state of the auth reducer in the front project, here: https://github.com/bobinette/papernet-front/blob/master/src/auth/reducer.js#L5 to pretend the user is loaded and already has the id you want

I don't have much time to work on it these days, but I'll fix those as soon as I can, I'll let you know here :slightly_smiling_face:

ok thanks
On Wed, May 17, 2017 at 4:27 PM Xavier<EMAIL_ADDRESS>wrote:

Hmm oops, it seems I forgot to log the error here:
https://github.com/bobinette/papernet/blob/master/cmd/web/main.go#L109
If you add it, the message will definitely be clearer
I think you are missing the google configuration to use google login. You
need credentials to be able to start the server. I'll add that to the
README as well, and if/when I have the time, make a no-auth version of it.
If you are using the front as well, I think you might be able to skip the
google part already by:

removing the lines in web/cmd/main.go dealing with google auth
creating a user via the cli
changing the initial state of the auth reducer in the front project,
here:
https://github.com/bobinette/papernet-front/blob/master/src/auth/reducer.js#L5
to pretend the user is loaded and already has the id you want

I don't have much time to work on it these days, but I'll fix those as
soon as I can, I'll let you know here 🙂
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bobinette/papernet/issues/18#issuecomment-302107619,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALcac0eTi5qhRgkbJjSkeChlhnNDeauAks5r6wPXgaJpZM4Nd0Wd
.

There is now a new login system based on email and password that you can use locally instead of the Google oauth system. It does not come with email confirmation and password reset but as it is intended to be used locally only (at least for now), I do not think those features are needed 🙂
To get rid of the google oauth provider, edit the configuration:
# other stuff

[oauth.google]
enabled=false

# other stuff

This should do the trick. Regarding the cli, it should work as long as you do not try to use the oauth sub-command. Let me know if you run into more problems, I'll fix them as soon as I can 🙂

ah nice. will make it easier to get going.
As far as email confirmation and 2 factor auth its pretty easy to add that these days actually.
i use twillo (sms), but also yubikey hardware.
email libs for golang are pretty good. https://github.com/emersion/go-imap is pretty solid.

I'm quite happy with Google auth in "production" so far, but if/when email confirmation is needed, I'll have a look at those :wink:

ok. maybe laters then ..


#include <FastNoise/FastNoise.h> fails?

I did also try including the directory path and was in the process of adding an external include when I realized that the page I was referring to was linking a .dll and.. this isn't a .dll integration.
I am so sorry for reaching out again - I'm using the project exactly as cloned with the command:
git clone --recurse-submodules https://github.com/caseymcc/UE4_FastNoise2.git
Except that I opened up the .UProject with 5.1 as the target and generated all of the files in doing so before closing the Unreal Editor and opening up the .sln in VS 2022
I'm presently just trying to include FastNoise into the FastNoise2ExampleGameModeABase.h and it doesn't show up in the autofill options when I type in #include <Fa
I did check (as before) every closed Issue prior to opening this one and I looked for a way to reach out privately in order to keep this out of the Issues tab, but I figured that it's at least now at a location where somebody else might see it who needs to see it in the future?
But yeah, neither of these worked:
#include "E:/Software/Game Development/Ue4FastNoise/UE4_FastNoise2/Source/Deps/FastNoise2"
#include <FastNoise/FastNoise.h>

And Source/Deps isn't inside of my solution explorer which is why I figured it's maybe an includes issue - so I went to the official documentation for FastNoise2 but there's just no information related to this use case beyond "Use this repo instead".

In the project's Intermediate directory there should be a CMakeTarget/FastNoise/build directory with some file buildinfo_{buildtype}.output. What is in that file? This is the file that is generated from the CMake information on fastnoise2 that is read in by the UnrealBuild tools so that everything can be added to Unreal Target/ModuleRules for the build. It might also be beneficial since you have changed the UE4CMake version to delete the Intermediate/CMakeTarget/FastNoise directory and let it rebuild it.
I have to admit I have not tested with VS in a while have been doing all my Unreal development as of late with VS code. When I get a chance I will test it with VS.

Went to my computer as soon as I saw this, admittedly I've been in bed and am probably going to sleep soon unfortunately but to answer your question the build folder is empty. There's nothing inside Ue4FastNoise\UE4_FastNoise2\Intermediate\CMakeTarget\FastNoise\build

Looks like when fixing the linux builds I left the stdc++ link in for all builds, fixed it. Pull that latest and you should be able to compile. Also add in some test code to the FatNoise2ExampleGameModeBase.cpp.

Looks like when fixing the linux builds I left the stdc++ link in for all builds, fixed it. Pull that latest and you should be able to compile. Also add in some test code to the FatNoise2ExampleGameModeBase.cpp.

Awesome! Thank you so much, I apologize for taking a bit to reply. I confirmed that the project now can build and run UE5 5.1 on my system and the code in the GameMode class seems fine - I confirmed that the appropriate functions show up when I type os2Noise->
From here it's really just me going through the FastNoise2 information that I can find and replacing my much easier to write FastNoise Portable code. Hopefully I can find better information than I found in their docs and issues but that's a FastNoise2 issue, not yours.
Which brings me to what took so long! I couldn't find any information and ChatGPT was of no help. I've provided an example for others to be able to learn from that can be easily added by creating an Actor class, adding the following code, making the class a BP instance and shoving it into the world.
.h
	// Sets default values for this actor's properties
	AFastNoiseExampleActor();
	UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "FastNoise Example")
	int WorldSeed = 214879;

.CPP
void AFastNoiseExampleActor::BeginPlay()
{
	Super::BeginPlay();
    // Create Simplex Fractal FBm with 5 octaves
    auto fnSimplex = FastNoise::New<FastNoise::Simplex>();
    auto fnFractal = FastNoise::New<FastNoise::FractalFBm>();
    fnFractal->SetSource(fnSimplex);
    fnFractal->SetOctaveCount(5);

    // Generate random numbers and print them to the screen
    for (int i = 0; i < 20; i++)
    {
        FVector location = FVector(i * 100.0f, 0.0f, 0.0f);
        float noiseSet[1];
        float frequency = 0.02f + i * 0.001f; // calculate frequency based on iteration index
        fnFractal->GenUniformGrid3D(noiseSet, location.X, location.Y, location.Z, 1, 1, 1, frequency, WorldSeed);
        float noiseValue = noiseSet[0];
        GEngine->AddOnScreenDebugMessage(-1, 5.0f, FColor::White, FString::Printf(TEXT("Noise value at location (%f, %f, %f): %f"), location.X, location.Y, location.Z, noiseValue));
    }
}

It's not a lot to give back compared to how much you've helped me, but any showing of gratitude and enabling others to take advantage of the work that you've done is what I can do :)

If you have any issues or want to help out feel free to drop in on the discord server for FastNoise (https://discord.gg/SHVaVfV). The original author is generally there and quite responsive.


"monitor.isOver()" returning false

In short, I've a homebrew tree-view React component with nodes as drag sources and overlays as drop targets. For some reason, hover event isn't firing and monitor.isOver() always returns false, as checked through canDrop. Also, canDrop is firing for every rendered tree node. Is this normal behaviour?
A more detailed description with code is posted at StackOverflow. I can copy it here if necessary.

i encounter sample problem


Update to FAPI 0.91.0+1.20.1

This PR
This PR updates QFAPI to match FAPI 0.91.0+1.20.1.
Associated

While making this update, an issue in QSL was discovered: https://github.com/QuiltMC/quilt-standard-libraries/issues/357

I can squash some of the commits if need be.

I can squash some of the commits if need be.

we don't squash commits on QFAPI, don't worry about it

What is the status on this PR? does it need to sync QSL?

Edit: Oh i see. no QSL sync needed. How is the rest of the PR?


@deck.gl/json: Document new syntactic sugar

For #3640
Documentation for new syntactic sugar

Background

Change List

Coverage remained the same at 83.065% when pulling 062861c56f48e4216ad53180463a2165e1244849 on ajd/update-json-docs into 7cdf28457a3a296bd7977d8c55c6adbde6ade1c6 on master.


"Decorated" progress report should be disabled when stdout is not a TTY

Problem description
Kind of a follow-up from the example in https://github.com/prefix-dev/pixi/issues/1004 : when two commands are run in the same pixi environment, one piped into another, the output is often mangled as both processes are trying to output "interactive" messages that erase the command line in-place. The output then becomes a random combination of different message lines.
pixi run --frozen --manifest-path pixi.toml echo | pixi run --frozen --manifest-path pixi.toml cat
⠁ updating packages in 'default'
⠁ updating packages in 'default'
⠁ updating packages in 'default'

Solution: detect whether stdout/stdin is a tty with https://doc.rust-lang.org/stable/std/io/trait.IsTerminal.html or https://crates.io/crates/is-terminal and disable "interactive" output accordingly

Yeah thats a bit annoying, especially the errors being overwritten.
We always output interactive messages to stderr. In your example, stderr would still be interactive right? Since you only pipe stdout.

I think pixi info --json is an example where that might not be the case.
I implemented a --no-progress flag in https://github.com/prefix-dev/pixi/pull/1105 which removes all decorated progress bars. I hope that can help with this problem?


Stretch-to-fill container

Currently if the end date isn't far enough future (for given scale) the timeline is displayed shorter than parent. We could stretch to fill space (would also have to listen to debounced window resize event).

Sorted


Add 5 new links

Grep replacement:

ugrep - a faster, more user-friendly and more featureful replacement for grep (The name of command: ug)

Cat replacement (new):

bat - A rewrite of cat with paging, syntax highlighting, and line numbers
rich-cli - Syntax-highlights files from the command line
glow - A command-line markdown viewer/renderer

Video processing (new):

ffmpeg - An incredibly powerful tool for video processing and conversion

I also fixed the link for fd (it previously linked to the_silver_searcher).

Thank you! Looks very nice tools. I will merge it.


⚠ Processors has degraded performance

In 13d36ea, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 124 ms

Resolved: Processors performance has improved in 4d9df04.


[SPARK-25837] [Core] Fix potential slowdown in AppStatusListener when cleaning up stages

What changes were proposed in this pull request?

Update AppStatusListener cleanupStages method to remove tasks for those stages in a single pass instead of 1 for each stage.
This fixes an issue where the cleanupStages method would get backed up, causing a backup in the executor in ElementTrackingStore, resulting in stages and jobs not getting cleaned up properly.

Tasks seem most susceptible to this as there are a lot of them, however a similar issue could arise in other locations the KVStore view method is used. A broader fix might involve updates to KVStoreView and InMemoryView as it appears this interface and implementation can lead to multiple and inefficient traversals of the stored data.
How was this patch tested?
Using existing tests in AppStatusListenerSuite
This is my original work and I license the work to the project under the project’s open source license.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Seems reasonable. Ping @vanzin

ok to test

btw I'm well aware that InMemoryView is kinda slow on the read side; improvements there would be welcome. The main issue is that the write path should be as fast as possible, since that can actually cause data loss if you have a burst of new events.

Test build #98329 has started for PR 22883 at commit 178f7c3.

I would be happy to start thinking about that.
Can you provide more details on what you mean by data loss on write in the InMemoryStore?
I now see that the types being inserted have an annotation around which fields they are allowed to be indexed by, but I don't have a good idea of how you would allow multiple indexes in a collection short of running something like SQLite in memory.

Test build #98329 has finished for PR 22883 at commit 178f7c3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98329/
Test PASSed.

Test build #98331 has finished for PR 22883 at commit 82fef86.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98331/
Test PASSed.

Merging to master / 2.4.

(Merged to 2.3 also.)


Bug automaticly "", cant edit all tags tab manually

Everytime i load the dataset images with text files, it always gave me this \  how do i edit the "all tags" manually?

this sign \

Thanks, fixed in 1.6.2.


TypeError: 'numpy.float64' object cannot be interpreted as an index

binaries, stars = generate_initial_population_grid(
        0.5 | units.MSun, 1.5 | units.MSun, 3, #mass range
        0.9, 0.9, 1,                           #mass ratios range
        10 | units.RSun, 100 | units.RSun, 120,    #semi major axis range
        0.0 , 1.0, 120                          #eccentricity range
    )

... Traceback (most recent call last):
File "", line 5, in
File "binary_population_synthesis.py", line 110, in generate_initial_population_grid
grid = multidimensional_meshgrid(primary_masses, mass_ratios, separations, eccentricities)
File "binary_population_synthesis.py", line 42, in multidimensional_meshgrid
array = array.reshape(shape)
TypeError: 'numpy.float64'

code from binary_population_synthesis.py
error about numpy appears when running generate_initial_population_grid,
Anyone who knows how to fix it? Thx
FYI. Install amuse on Mac os X 10.11 following [http://www.amusecode.org/doc/install/install-prerequisites-osx.html]

Fixed.


[Plugin-x] FacebookAgent api function should treat non-text type differently

Otherwise, it will lead to crash on native platforms
Originally reported : http://discuss.cocos2d-x.org/t/my-dilemma-with-loading-online-image-from-facebook/19026/3

Hi,
Is it easy to fix manually?


Error unable to access page  https://alanjuden.com/

Hello Alan Juden i am trying to learn about AlanJuden.MvcReportViewer.NetCore but the link https://alanjuden.com/2016/11/10/mvc-net-core-report-viewer/ failed to view. Can you give specific instructions on how to use AlanJuden.MvcReportViewer.NetCore?

@phinq17192016, thanks for reaching out. I've got my site working again...sorry about that!

Thanks @alanjuden I see the site is back up and running


fix(operate): use post request for decision definitions and decision instances endpoints

closes #262
DecisionInstance type should also include processInstanceKey, which is also addressed in this PR
Description of the change
[Describe your changes here]
Type of change

[ ] Bug fix
[ ] New feature
[ ] Breaking change
[ ] Documentation update

Checklist

[ ] I have read the CONTRIBUTING doc
[ ] I have opened this pull request against the alpha branch
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)

Further comments
[If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...]

Thanks for this. Can you open this against the alpha branch please. All code goes PR -> alpha -> main at the moment.

@jwulf done 👍


Missing Code of Conduct

Missing Code of Conduct
There has been a lot of traffic on this repository, and the Author feels the need to establish some community guidelines in place before opening up this repo to discussions.

Please read #11 before reaching out to the Author for any purposes.


ORC-1635: Try downloading orc-format from dlcdn.apache.org before archive.apache.org

What changes were proposed in this pull request?
Try downloading orc-format from dlcdn.apache.org before archive.apache.org
This replaces https://github.com/apache/orc/pull/1820 which required dlcdn to have the current version.
Why are the changes needed?
https://archive.apache.org/ discourages heavy use, and its rate limits can cause CI systems building Apache ORC to be banned.
How was this patch tested?
It builds from a clean repo
Was this patch authored or co-authored using generative AI tooling?
no

Merged to main for Apache ORC 2.1.0. Thank you, @progval .

+1, LGTM (Pending CIs) for Apache ORC 2.1.0.
Merged to main for Apache ORC 2.1.0. Thank you, @progval .

Thanks, but why not merging to all active branches? (at least 1.9 which is advertised as the current release and 2.0)? In a context of this being a fix mainly for CI workloads, makes sense to apply it on the current release version.

v1.9 does not download orc-format so it's not applicable to that branch. But I agree it would be nice to have it on v2.0

I fully understand why you (@douardda and @progval) are thinking in that way. However, this is categorized as Improvement instead of Bug, isn't it? Technically, dlcdn.apache.org is a caching layer without any guarantee of file existence.

From my perspective, main branch is the most active branch (for developing and receiving pull requests) which contributing the download traffic to ASF site. It's good to have because it will be in sync always with orc-format repository and its latest version. However, for the release branches, we don't think that's true.
In general, as one of the open source community,

Apache ORC community follows Semantic Versioning officially.
For all release branches, we don't backport the Improvement with one exception which tools module. There was an official decision about tools module exception in the dev mailing list.
FYI, the feature freeze of Apache ORC v2.0 happens already when we cut a branch, branch-2.0. After that, we are focusing QA activity(testing, bug fixing and polishing) instead of core changes.

In short, this improvement contribution is simply late to be a part of 2.0. It will be 2.1.

I open the JIRA ticket as Bug instead of Improvement because I see this as a fixing a regression: v1.x didn't download the file so it builds fine from anywhere, v2.0 downloads the file from archive.apache.org so it can't be built on CIs

I believe that we already agreed that your original claim was wrong here (https://github.com/apache/orc/pull/1820#issuecomment-1964748734)

I opened the JIRA ticket as Bug instead of Improvement because I see this as a fixing a regression: v1.x didn't download the file so it builds fine from anywhere, v2.0 downloads the file from archive.apache.org so it can't be built on CIs

I'm wondering if you are still claiming that "it can't be built on CIs". What are you unable to build on CIs?

I'm wondering if you are still claiming that "it can't be built on CIs".

I can now build the main branch, but not the v2.0 branch.

What are you unable to build on CIs?

I can't build the C++ code in the v2.0 branch of this repository on my CI because I get this error:
13:54:56    -- Downloading...
13:54:56       dst='/var/lib/jenkins/workspace/DGRPH/gitlab-builds/target/debug/build/orcxx-8d1ca2e7d12cd415/orc/orc-format_ep-prefix/src/orc-format-1.0.0.tar.gz'
13:54:56       timeout='none'
13:54:56       inactivity timeout='none'
13:54:56    -- Using src='https://archive.apache.org/dist/orc/orc-format-1.0.0/orc-format-1.0.0.tar.gz'
13:54:56
13:54:56    --- stderr
13:54:56    CMake Error at orc-format_ep-stamp/download-orc-format_ep.cmake:170 (message):
13:54:56      Each download failed!
13:54:56
13:54:56        error: downloading 'https://archive.apache.org/dist/orc/orc-format-1.0.0/orc-format-1.0.0.tar.gz' failed
13:54:56              status_code: 28
13:54:56              status_string: "Timeout was reached"
13:54:56              log:
13:54:56              --- LOG BEGIN ---
13:54:56                Trying <IP_ADDRESS>:443...
13:54:56        Trying [2a01:4f9:1a:a084::2]:443...
13:54:56
13:54:56      Immediate connect fail for 2a01:4f9:1a:a084::2: Cannot assign requested
13:54:56      address
13:54:56
13:54:56      connect to <IP_ADDRESS> port 443 failed: Connection timed out
13:54:56
13:54:56      Failed to connect to archive.apache.org port 443 after 129277 ms: Couldn't
13:54:56      connect to server
13:54:56
13:54:56      Closing connection 0

after being blocked by archive.apache.org

To @progval , it's your network environment network issue which doesn't follow the ASF policy correctly.

after being blocked by archive.apache.org

As I mentioned before, when Apache ORC community release Apache ORC Format 1.0.1, your claim will be broken again because your network environment will block you to access Apache ORC Format 1.0.0. I'd like to recommend to fix your network permanently to allow the archive link.

For the record, on Apache Arrow we are getting similar download failures on public GitHub CI.

Ack. Thank you for reporting @pitrou .

FYI: We must use https://www.apache.org/dyn/closer.lua/orc/orc-format-${ORC_FORMAT_VERSION}/orc-format-${ORC_FORMAT_VERSION}.tar.gz?action=download instead. The dlcdn.apache.org URL doesn't work when orc-format-2.0.0 is released and orc-format-1.0.0 is removed from https://dist.apache.org/repos/dist/release/orc/ . If we use the closer.lua URL, we don't need to list dlcdn.apache.org URL (for the latest release) and archive.apache.org URL (for old releases). closer.lua redirects to dlcdn.apache.org or something for the latest release and archive.apache.org for old releases automatically.
See also:

https://infra.apache.org/release-download-pages.html#closer
https://infra.apache.org/release-download-pages.html#current-and-older-releases

Sure, I'll do the follow-up in Apache ORC side to use https://www.apache.org/dyn/closer.lua, @kou . Thanks!


Mixins are not merged correctly for built-in options in this.$options

Version
3.0.11
Reproduction link
SFC Playground
Steps to reproduce
Define a component with options API:
{
  mixins: [
    { computed: { bar: () => 2 } }
  ],
  computed: {
    foo: () => 1
  }
}

What is expected?
this.$options.computed should contain both foo and bar as keys:
this.$options.computed // { foo: () => 1, bar: () => 2 }

What is actually happening?
this.$options.computed // { foo: () => 1 }

It seems to affect all built-in options: https://github.com/vuejs/vue-next/blob/870f2a7ba35245fd8c008d2ff666ea130a7e4704/packages/runtime-core/src/componentOptions.ts#L973
This also makes Devtools unable to show data/computed/... defined in mixins as it relies on this.$options.

It seems like Vue 2 does add them to $options https://jsfiddle.net/posva/p2gz45j8/

Yeah I've checked it's how devtools currently manage to categorize properties into data, computed, etc.

@posva  Is this issue the same as the one described by https://github.com/vuejs/vue-next/issues/2119?

@HcySunYang Yeah I believe both issues rely on options being merged correctly.

@HcySunYang I think it's a bit different.
#2119 was initially about being able to retrieve beforeRouteEnter() and other custom options added by the router when the user adds them through mixins. Because they are custom, we would also need a custom merge strategy.
So, IMO solving this issue doesn't require solving the other one and this one is more about consistency with Vue 2 while the other one is about a missing feature given the new behavior and also about devtools. So we should prioritize this one over the other one

Currently devtools for Vue 3 only handles computed/inject keys on the component's base definition, so even if we expose $options like in Vue 2 it still doesn't work.
It's trivial for devtools to account for mixins though, we just need to do a merge on the devtools side for computed/provide/inject here /cc @Akryum
I don't think app code should rely on core merge behavior for $options, and the trade-off isn't worth it considering the extra amount of code needed. The behavior consistency is covered in the migration build, but in proper Vue 3 users should only use $options for custom options, not core options.

Will take a look tomorrow

Done here for the compatibility build:
https://github.com/vuejs/vue-next/blob/6f8fe4eac9fe8f4aad71a829d20897dc676c155c/packages/runtime-core/src/componentPublicInstance.ts#L235

It's missing support for extends though

Yeah currently resolveMergedOptions intentionally does not account for built-in options like computed because it's somewhat wasteful (one of the perf bottlenecks in v2).
What I'm suggesting is for devtools to also check mixins and extends when resolving computed/provide/inject state for an instance.


Can't plot AKMicrophone using AKNodeOutputPlot

I am trying to plot the wave generated by AKMicrophone but I think this is not working...
Error:
2017-03-08 12:44:00.823693 Microphone_Analyser_AK[32413:8926966] [central] 54:   ERROR:    >avae> AVAudioNode.mm:565: CreateRecordingTap: required condition is false: _recordingTap == nil
2017-03-08 12:44:01.161 Microphone_Analyser_AK[32413:8926966] *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'required condition is false: _recordingTap == nil'
*** First throw call stack:
(
0   CoreFoundation                      0x0000000109536d4b __exceptionPreprocess + 171
1   libobjc.A.dylib                     0x0000000108f9821e objc_exception_throw + 48
2   CoreFoundation                      0x000000010953ae42 +[NSException raise:format:arguments:] + 98
3   AVFAudio                            0x000000010e71dfd0 _Z19AVAE_RaiseExceptionP8NSStringz + 176
4   AVFAudio                            0x000000010e76acf8 _ZN19AVAudioNodeImplBase18CreateRecordingTapEmjP13AVAudioFormatU13block_pointerFvP16AVAudioPCMBufferP11AVAudioTimeE + 144
5   AVFAudio                            0x000000010e768ded -[AVAudioNode installTapOnBus:bufferSize:format:block:] + 209
6   AudioKit                            0x00000001084fdafc TTSf4g_n_n_n___TFC8AudioKit16AKNodeOutputPlotcfTCS_6AKNode5frameVSC6CGRect10bufferSizeSi_S0 + 700
7   AudioKit                            0x00000001084fd364 TFC8AudioKit16AKNodeOutputPlotCfTCS_6AKNode5frameVSC6CGRect10bufferSizeSi_S0 + 116
8   Microphone_Analyser_AK              0x0000000108221268 TFC22Microphone_Analyser_AK14ViewController9setupPlotfT_T + 376
9   Microphone_Analyser_AK              0x00000001082216cf TFC22Microphone_Analyser_AK14ViewController7inspectfT_T + 319
10  Microphone_Analyser_AK              0x0000000108221702 TToFC22Microphone_Analyser_AK14ViewController7inspectfT_T + 34
11  Foundation                          0x0000000108af18d4 __NSFireTimer + 83
12  CoreFoundation                      0x00000001094c8db4 CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION + 20
13  CoreFoundation                      0x00000001094c8a43 __CFRunLoopDoTimer + 1075
14  CoreFoundation                      0x00000001094c85ca __CFRunLoopDoTimers + 250
15  CoreFoundation                      0x00000001094c02f1 __CFRunLoopRun + 2065
16  CoreFoundation                      0x00000001094bf884 CFRunLoopRunSpecific + 420
17  GraphicsServices                    0x000000010fb46a6f GSEventRunModal + 161
18  UIKit                               0x000000010af3cc68 UIApplicationMain + 159
19  Microphone_Analyser_AK              0x0000000108223a4f main + 111
20  libdyld.dylib                       0x000000010e14668d start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
(lldb)
And this is the default code (caught from playground) :
@IBOutlet var audioInputPlot: AKNodeOutputPlot!
func setupPlot() {
let plot = AKNodeOutputPlot(soundAnalyser.mic, frame: audioInputPlot.bounds)
plot.plotType = .rolling
plot.shouldFill = true
plot.shouldMirror = true
plot.color = UIColor.blue
audioInputPlot.addSubview(plot)
}
I am simply creating an UIView and setting it's class as AKNodeOutputPlot. Is this wrong?

That's kind of weird how you're adding a a subview of the kind of plot that you're telling it it already is.  Try instead to just do:
func setupPlot() {
audioInputPlot.node = mic
audioInputPlot.plotType = .rolling
audioInputPlot.shouldFill = true
audioInputPlot.shouldMirror = true
audioInputPlot.color = UIColor.blue
}

I run into the same error. I can see my plot on the screen but as soon as I try to start recording, I get:
AVAudioNode.mm:565: CreateRecordingTap: required condition is false: _recordingTap == nil

Here is my code, it is super simple but can't get it to work... @aure I would really appreciate if you can help me.
    private func setup() {
        // Clean tempFiles !
        AKAudioFile.cleanTempDirectory()

        // Session settings
        AKSettings.bufferLength = .medium
        do {
            try AKSettings.setSession(category: .playAndRecord, with: .allowBluetoothA2DP)
        } catch {
            AKLog("Could not set session category.")
        }
        startPlot()

        recorder = try? AKNodeRecorder(node: mic)
        recorder.durationToRecord = GlobalConstants.maxClipDuration

        AudioKit.start()
    }

    private func startPlot() {
        plot = AKNodeOutputPlot(mic, frame: bounds)
        plot.backgroundColor = UIColor.white
        plot.color = UIColor.customBlue
        plot.gain = 5
        plot.plotType = EZPlotType.rolling
        plot.shouldFill = true
        plot.shouldMirror = true
        addSubview(plot)
    }

    func startRecording() {
        do {
            try recorder.record()
        } catch { print("Can't record!") }
    }

@victorBaro I don't see where mic is defined.

Thank you @aure for your quick reply. This is my declaration of mic (most of the code is copied from your amazing playgrounds examples :) ).
let mic = AKMicrophone()

I figured it was that, but I kinda need to see it all in context to try to figure out whats going wrong. The playgrounds work, I checked that much.

@aure any updates on this? Still not working.

@victorBaro pretty much looking to replace the plotting system with a metal powered alternative.

@victorBaro Meanwhile: https://github.com/AudioKit/AudioKit/pull/1042 might help?

Hi, is there any update on this issue? I'd like to update to Xcode 9.3, but I can't get AKMicrophone to work with AKNodeOuputPlot either. This one problem is holding me back from updating my version of Xcode.


Sync players and mobs over network

Instantiate and sync both the players and the enemies over network. It should still be possible to play single player as well

Do not merge before #3

We need to create a sorting order for environments, because the spawn platform renders on top of the player. I'll create a task

Made a comment in the task, will add fixes


Add index types for compatibility with graphql-tools

I have checked the issue https://github.com/prisma/graphqlgen/issues/124 Right now graphql-tools is complaining because index signatures are missing. To make it type-safe, you probably need to generate enum types for keys and, instead of any, to use union types. This PR makes it compatible with graphql-tools.

Sorry for double-review. GitHub had an outage during the time of review.


make rewrite more robust

I'm still using apply concat, apply ap ..., apply H often as substitute for rewrite H. Rewrite is very nice when it works, but it still fails often. Then it gives a unhelpful error message and there's not always a good way to give extra information. Currently, I (and Jakob as well) find that it is faster to give some more explicit information with concat/ap/apply. But maybe the rewrite tactic can be improved to work for all our cases.
For example, in the definition of issue #469 I don't know how to use rewrite to make progress. In the following example, I give quite some information (more than I usually want to give), but it still fails.
open eq is_equiv funext

variables {A : Type} {B : A → Type} {C : Πa, B a → Type}

definition homotopy2 [reducible] (f g : Πa b, C a b) : Type :=
Πa b, f a b = g a b
notation f `∼2`:50 g := homotopy2 f g

definition eq_of_homotopy2 {f g : Πa b, C a b} (H : f ∼2 g) : f = g :=
eq_of_homotopy (λa, eq_of_homotopy (H a))

definition apD100 {f g : Πa b, C a b} (p : f = g) : f ∼2 g :=
λa b, apD10 (apD10 p a) b

definition foo (f g : Πa b, C a b) (H : f ∼2 g) (a : A)
  : apD100 (eq_of_homotopy2 H) a = H a :=
begin
  esimp {apD100, eq_of_homotopy2},
  rewrite {apD10 (eq_of_homotopy (λ(a : A), eq_of_homotopy (H a)))}(retr apD10 (λ(a : A), eq_of_homotopy (H a))),
end

In a different situation I tried to rewrite using theorems like the following
  definition transport_eq_l (p : a1 = a2) (q : a1 = a3)
    : transport (λx, x = a3) p q = p⁻¹ ⬝ q :=

However, this fails because there is a binder in the conclusion. Would it be an idea to match (subterms of) the goal with transport ?M_1 ?M_2 ?M_3, i.e. to replace a binder with a metavariable (just as the matcher replaced "?M_1 applied to ?M_2" with a fresh metavariable in issue #433).

I'm confused. The goal is:
(λ (b : B a), apD10 (apD10 (eq_of_homotopy2 H) a) b) = H a

We can eta reduce it to
apD10 (apD10 (eq_of_homotopy2 H) a) = H a

The type of retr apD10 (λ(a : A), eq_of_homotopy (H a)) is
apD10 (apD10⁻¹ (λ (a : A), eq_of_homotopy (H a))) =  λ (a : A), eq_of_homotopy (H a)

So, its left-hand-side does not match any subterm in the goal.

Then it gives a unhelpful error message and there's not always a good way to give extra information.

It is not easy to give a good error message. The tactic is trying to match the left-hand-side of the given equation with all subterms in the goal (and/or hypotheses). There are many failures.
It would be a tsunami of information to report all failures.
What about the following compromise?

We display which term the rewriting is using for pattern-matching.
We report which sub-terms in the goal matched the pattern.
For the ones that matched, we report the unification failure error message: given type vs expected type.

In a different situation I tried to rewrite using theorems like the following

  definition transport_eq_l (p : a1 = a2) (q : a1 = a3)
    : transport (λx, x = a3) p q = p⁻¹ ⬝ q :=

Could you post the complete example?
Thanks

I'm confused.

I forgot to also unfold eq_of_homotopy (maybe we should make that reducible). When also unfolding eq_of_homotopy (and eq_of_homotopy2, which I did do), the goal becomes (eta-equivalent to)
apD10 (apD10 (apD10 ⁻¹ (λ (a : A), apD10 ⁻¹ (H a))) a) = H a

so we can (I think) match the left-hand-side of the type of the rewrite term with the _ in apD10 (_ a) = H a. However, also unfolding eq_of_homotopy doesn't help (and note from #469 that using
apply concat, apply (ap (λx, @apD10 _ (λx : B a, _) _ _ (x a))), apply (retr apD10),
does work)

It is not easy to give a good error message.

I understand. My comment was not meant as "this should be changed ASAP" but as a comment that unfortunately the current error messages don't help very well with finding the problem when a rewrite tactic doesn't work. I think your suggestion is very good, and can the resulting information can be very helpful. Maybe it will sometimes print too much information, but that's usually better than too little.

Could you post the complete example?

A simple example would be
import types.eq

open eq
  variables {A : Type} {a1 a2 a3 : A}
  definition my_transport_eq_l (p : a1 = a2) (q : a1 = a3)
    : transport (λx, x = a3) p q = p⁻¹ ⬝ q :=
  begin
    rewrite transport_eq_l,
  end

A more involved example would be replacing line 44 in types.fiber with rewrite transport_eq_Fl

I added the unfold eq_of_homotopy. The rewrite doesn't succeed because the pattern matcher does perform matching modulo eta-reduction. We can add a tactic for eta reduction, or even do it at esimp. If we do eta by hand, the rewrite works if eq_o_homotopy is marked reducible.
Here is the working version:
local attribute eq_of_homotopy [reducible]

definition foo (f g : Πa b, C a b) (H : f ∼2 g) (a : A)
  : apD100 (eq_of_homotopy2 H) a = H a :=
begin
  esimp {apD100, eq_of_homotopy2, eq_of_homotopy},
  -- goal is
  -- (λ (b : B a), apD10 (apD10 (apD10 ⁻¹ (λ (a : A), apD10 ⁻¹ (H a))) a) b) = H a
  -- Applying eta by hand
  change (apD10 (apD10 (apD10 ⁻¹ (λ (a : A), apD10 ⁻¹ (H a))) a) = H a),
  -- It will only match because eq_of_homotopy is reducible
  rewrite (retr apD10 (λ(a : A), eq_of_homotopy (H a)))
end

The rewrite doesn't succeed because the pattern matcher does not perform matching modulo eta-reduction.

That's good to know. I think this won't be a problem if there is some tactic to do eta reduction (using change by hand is quite tedious)

We can add a tactic for eta reduction

That would be nice. Tactics typically become much more readable after eta-reducing, so an eta command is useful. I think it would also be nice to automatically do eta-reduction with esimp, especially since esimp itself sometimes adds extra eta-reducible expressions (like in this example).

I added support for eta-reduction in the esimp tactic.
It is not enough for the problem above because the type of (retr apD10 (λ(a : A), eq_of_homotopy (H a))) also contains non-eta reduced terms (in the implicit arguments).
So, I also added support for matching modulo eta.
Now, we can do it using
local attribute eq_of_homotopy [reducible]

definition foo (f g : Πa b, C a b) (H : f ∼2 g) (a : A)
  : apD100 (eq_of_homotopy2 H) a = H a :=
begin
  esimp {apD100, eq_of_homotopy2, eq_of_homotopy},
  rewrite (retr apD10 (λ(a : A), eq_of_homotopy (H a))),
  apply (retr apD10)
end

I'll create another issue for improving the rewrite tactic error message.

issue was fixed by:
https://github.com/leanprover/lean/commit/a9fa0fead9175165dea48a96b88bf10d05e51135
https://github.com/leanprover/lean/commit/7de3d5771df1e80591289f19c782836d052f5c55


Jitpack not found

Hi. Let me check it.

The version is there. Also, the newer version is available:

And check your settings.gradle.kts if its correct:
dependencyResolutionManagement {
    repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
    repositories {
	    mavenCentral()
	    maven { url 'https://jitpack.io' }
    }
}

It's okay, i've just pushed some commits, and GitHub actions failed with this error again and again, i solved this by forking and releasing repo


Lists render incorrectly

In particular, the number is not visible. See: https://undp-accelerator-labs.github.io/national_innovation_ecosystems_toolkit/How to upskill civil servants for innovation/MOOC public and social innovation concepts and tools - Ecuador/Factsheet

Fixed.


Docker - intial login

I renamed all the config.env.example -> config.env and ran docker-compose up
When I get to the login screen - I attempt to use
<EMAIL_ADDRESS>DEFAULT_USER_PASSWORD=password
It doesn't login - I see these errors in console.
Creating new user doesn't work. Can't get in.
y":"Value is sensitive and has been redacted. To see the value set config key "log.leak_sensitive_values = true" or environment variable "LOG_LEAK_SENSITIVE_VALUES=true".","remote":"<IP_ADDRESS>:57456","scheme":"http"},"http_response":{"status":200,"text_status":"OK","took":2222173},"level":"info","msg":"completed handling request","time":"2021-08-10T01:07:43Z"}
oathkeeper_1      | [cors] 2021/08/10 01:07:43 Handler: Actual request
oathkeeper_1      | [cors] 2021/08/10 01:07:43   Actual request no headers added: missing origin
oathkeeper_1      | {"http_request":{"headers":{"accept":"image/avif,image/webp,image/apng,image/svg+xml,image/,/;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","cookie":"Value is sensitive and has been redacted. To see the value set config key "log.leak_sensitive_values = true" or environment variable "LOG_LEAK_SENSITIVE_VALUES=true".","referer":"http://<IP_ADDRESS>:4455/.factly/kavach/web/auth/login?flow=cc2485e1-c392-4bf0-87a8-8e8ed3a5ae20","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36"},"host":"<IP_ADDRESS>:4455","method":"GET","path":"/.factly/kavach/web/favicon.ico","query":null,"remote":"<IP_ADDRESS>:57456","scheme":"http"},"level":"info","msg":"started handling request","time":"2021-08-10T01:07:43Z"}
oathkeeper_1      | {"audience":"application","error":{"debug":"","message":"Requested url does not match any rules","reason":"","status":"Not Found","status_code":404},"granted":false,"http_host":"<IP_ADDRESS>:4455","http_method":"GET","http_url":"http://<IP_ADDRESS>:4455/.factly/kavach/web/favicon.ico","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36","level":"warning","msg":"Access request denied","service_name":"ORY Oathkeeper","service_version":"v0.38.14-beta.1","time":"2021-08-10T01:07:43Z"}
oathkeeper_1      | {"code":404,"debug":"","details":{},"error":"Requested url does not match any rules","level":"error","msg":"An error occurred while handling a request","reason":"","request-id":"","status":404,"time":"2021-08-10T01:07:43Z","writer":"JSON"}
oathkeeper_1      | {"http_request":{"headers":{"accept":"image/avif,image/webp,image/apng,image/svg+xml,image/,/;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","cookie":"Value is sensitive and has been redacted. To see the value set config key "log.leak_sensitive_values = true" or environment variable "LOG_LEAK_SENSITIVE_VALUES=true".","referer":"http://<IP_ADDRESS>:4455/.factly/kavach/web/auth/login?flow=cc2485e1-c392-4bf0-87a8-8e8ed3a5ae20","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36"},"host":"<IP_ADDRESS>:4455","method":"GET","path":"/.factly/kavach/web/favicon.ico","query":null,"remote":"<IP_ADDRESS>:57456","scheme":"http"},"http_response":{"status":404,"text_status":"Not Found","took":1199864},"level":"info","msg":"completed handling request","time":"2021-08-10T01:07:43Z"}
oathkeeper_1      | [cors] 2021/08/10 01:07:43 Handler: Actual request
oathkeeper_1      | [cors] 2021/08/10 01:07:43   Actual request no headers added: missing origin
oathkeeper_1      | {"http_request":{"headers":{"accept":"/","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","if-modified-since":"Sun, 07 Mar 2021 18:16:38 GMT","if-none-match":"W/"1ec-1780de7cb70"","referer":"http://<IP_ADDRESS>:4455/.factly/kavach/web/auth/login?flow=cc2485e1-c392-4bf0-87a8-8e8ed3a5ae20","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36"},"host":"<IP_ADDRESS>:4455","method":"GET","path":"/.factly/kavach/web/manifest.json","query":null,"remote":"<IP_ADDRESS>:57588","scheme":"http"},"level":"info","msg":"started handling request","time":"2021-08-10T01:07:43Z"}
oathkeeper_1      | {"audience":"application","granted":true,"http_host":"<IP_ADDRESS>:4455","http_method":"GET","http_url":"http://kavach-web:3000/.factly/kavach/web/manifest.json","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36","level":"warning","msg":"Access request granted","service_name":"ORY Oathkeeper","service_version":"v0.38.14-beta.1","subject":"guest","time":"2021-08-10T01:07:43Z"}
oathkeeper_1      | {"http_request":{"headers":{"accept":"/","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","if-modified-since":"Sun, 07 Mar 2021 18:16:38 GMT","if-none-match":"W/"1ec-1780de7cb70"","referer":"http://<IP_ADDRESS>:4455/.factly/kavach/web/auth/login?flow=cc2485e1-c392-4bf0-87a8-8e8ed3a5ae20","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36"},"host":"<IP_ADDRESS>:4455","method":"GET","path":"/.factly/kavach/web/manifest.json","query":null,"remote":"<IP_ADDRESS>:57588","scheme":"http"},"http_response":{"status":304,"text_status":"Not Modified","took":2544453},"level":"info","msg":"completed handling request","time":"2021-08-10T01:07:43Z"}
oathkeeper_1      | {"http_request":{"headers":{"accept":"image/avif,image/webp,image/apng,image/svg+xml,image/,/;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","cookie":"Value is sensitive and has been redacted. To see the value set config key "log.leak_sensitive_values = true" or environment variable "LOG_LEAK_SENSITIVE_VALUES=true".","referer":"http://<IP_ADDRESS>:4455/.factly/kavach/web/auth/login?flow=cc2485e1-c392-4bf0-87a8-8e8ed3a5ae20","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36"},"host":"<IP_ADDRESS>:4455","method":"GET","path":"/.factly/kavach/web/logo192.png","query":null,"remote":"<IP_ADDRESS>:57456","scheme":"http"},"level":"info","msg":"started handling request","time":"2021-08-10T01:07:43Z"}
oathkeeper_1      | [cors] 2021/08/10 01:07:43 Handler: Actual request
oathkeeper_1      | [cors] 2021/08/10 01:07:43   Actual request no headers added: missing origin
oathkeeper_1      | {"audience":"application","error":{"debug":"","message":"Requested url does not match any rules","reason":"","status":"Not Found","status_code":404},"granted":false,"http_host":"<IP_ADDRESS>:4455","http_method":"GET","http_url":"http://<IP_ADDRESS>:4455/.factly/kavach/web/logo192.png","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36","level":"warning","msg":"Access request denied","service_name":"ORY Oathkeeper","service_version":"v0.38.14-beta.1","time":"2021-08-10T01:07:43Z"}
oathkeeper_1      | {"code":404,"debug":"","details":{},"error":"Requested url does not match any rules","level":"error","msg":"An error occurred while handling a request","reason":"","request-id":"","status":404,"time":"2021-08-10T01:07:43Z","writer":"JSON"}
oathkeeper_1      | {"http_request":{"headers":{"accept":"image/avif,image/webp,image/apng,image/svg+xml,image/,/;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","cookie":"Value is sensitive and has been redacted. To see the value set config key "log.leak_sensitive_values = true" or environment variable "LOG_LEAK_SENSITIVE_VALUES=true".","referer":"http://<IP_ADDRESS>:4455/.factly/kavach/web/auth/login?flow=cc2485e1-c392-4bf0-87a8-8e8ed3a5ae20","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36"},"host":"<IP_ADDRESS>:4455","method":"GET","path":"/.factly/kavach/web/logo192.png","query":null,"remote":"<IP_ADDRESS>:57456","scheme":"http"},"http_response":{"status":404,"text_status":"Not Found","took":338748},"level":"info","msg":"completed handling request","time":"2021-08-10T01:07:43Z"}

@johndpope User might be not be created due to weak password.
Password must contain

At least 8 characters
A mixture of both numbers, uppercase and lowercase letters
Inclusion of at least one special character

Update credentials as below and restart  dega-migrate container by docker-compose restart dega-migrate
<EMAIL_ADDRESS>DEFAULT_USER_PASSWORD=2ssad32sadADSd@!@4

thank you.


added links to the committed changes so we can reference them via links.

added links to each committed change so I can reference them.

Happy to merge, if you move Id to the model and keep the view simpler. The generation of Change model is at https://github.com/jakubgarfield/Bonobo-Git-Server/blob/master/Bonobo.Git.Server/RepositoryBrowser.cs#L317

updated code to work as you said.


Error with import of "inference" in nmt.py

I can't get past the import of "inference.py".  Here is my error message:
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 163, in _run_module_as_main
mod_name, _Error)
File "/usr/lib/python2.7/runpy.py", line 102, in _get_module_details
loader = get_loader(mod_name)
File "/usr/lib/python2.7/pkgutil.py", line 464, in get_loader
return find_loader(fullname)
File "/usr/lib/python2.7/pkgutil.py", line 474, in find_loader
for importer in iter_importers(fullname):
File "/usr/lib/python2.7/pkgutil.py", line 430, in iter_importers
import(pkg)
File "nmt.py", line 28, in
from . import inference
ValueError: Attempted relative import in non-package

I get the same issue:
Traceback (most recent call last):   File "C:/Users/nave01314/IdeaProjects/nmt/nmt/nmt.py", line 28, in <module>     from . import inference ImportError: cannot import name 'inference'

I solved this problem by executing the command in the "gnmt-master" directory,  I guess you executing the command in the "gnmt-master/nmt" directory.

I was also in the nmt sub-directory.  This is my entire command-line:
(env) dludwig1@AspireMachineLearning:/media/DataHD2/nmt/nmt$ python -m nmt.nmt \

--src=vi --tgt=en \
--vocab_prefix=/media/DataHD2/nmt/nmt_data/vocab  \
--train_prefix=/media/DataHD2/nmt/nmt_data/train \
--dev_prefix=/media/DataHD2/nmt/nmt_data/tst2012  \
--test_prefix=/media/DataHD2/nmt/nmt_data/tst2013 \
--out_dir=/media/DataHD2/nmt/nmt_model \
--num_train_steps=12000 \
--steps_per_stats=100 \
--num_layers=2 \
--num_units=128 \
--dropout=0.2 \
--metrics=bleu

Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 163, in _run_module_as_main
mod_name, _Error)
File "/usr/lib/python2.7/runpy.py", line 102, in _get_module_details
loader = get_loader(mod_name)
File "/usr/lib/python2.7/pkgutil.py", line 464, in get_loader
return find_loader(fullname)
File "/usr/lib/python2.7/pkgutil.py", line 474, in find_loader
for importer in iter_importers(fullname):
File "/usr/lib/python2.7/pkgutil.py", line 430, in iter_importers
import(pkg)
File "nmt.py", line 28, in
from . import inference
ValueError: Attempted relative import in non-package
Am I using the wrong version of Python?

Fansiawant - you are absolutely right!  I moved up one directory and it worked fine.
Thank you, Dana

I'm still having issues with this. I don't have a directory to move up into - when I cloned into Intellij, I ended up with the path ~\IdeaProjects\nmt... .
Screenshot of issue, project structure, and run configuration: http://imgur.com/RVNDhf0

nave01314: Your error message was a bit different than mine.  I had other errors before too, and one thing that helped was when I did a completely fresh download of the scripts just before I ran the scripts.  It seems that the scripts are actively evolving, along with the data, and you need a complete snapshot of the demo to prevent some bugs.

@danaludwig Recloning and rerunning did not fix the issue.

Sounds worth asking others; I don't know how to un-close this thread.  Perhaps you could post your bug on a new thread?

I've done so.

nave01314: I think you just need to move up one directory to the first "nmt", then your path maybe like this:
~\IdeaProjects\nmt
You can execute the command in that path like this:
~\IdeaProjects\nmt$ python -m nmt.nmt ....blabla....

I try by changing directry,  but it can't resolve my same error:
python nmt.py \
> --src=vi --tgt=en \
> --vocab_prefix=tmp/nmt_data/vocab  \
> --train_prefix=tmp/nmt_data/train \
> --dev_prefix=tmp/nmt_data/tst2012  \
> --test_prefix=tmp/nmt_data/tst2013 \
> --out_dir=/tmp/nmt_model \
> --num_train_steps=12000 \
> --steps_per_stats=100 \
> --num_layers=2 \
> --num_units=128 \
> --dropout=0.2 \
> --metrics=bleu

Again, it gives me error as:
File "nmt.py", line 28, in
from . import inference
ValueError: Attempted relative import in non-package

Look at here https://github.com/tensorflow/nmt/issues/313
I've already got to know why it runs with python -m nmt.nmt? The first nmt is the directory and second nmt that actually is nmt.py. So there is a little perplexity to most programmers I think.


Yo code gives Object.fromEntries is not a function

Does this issue occur when all extensions are disabled?: Yes/No

VS Code Version: 1.59.0
OS Version: Windows 10 Pro, 21H1, 19043.1110

Steps to Reproduce:

Updated VSCode, yo and generator-code to the latest versions
I then do a 'yo code' and fill out the questions but I am then met with an error code

$ yo code
(node:31260) ExperimentalWarning: queueMicrotask() is experimental.
(node:31260) ExperimentalWarning: The fs.promises API is experimental
 _-----_     ╭──────────────────────────╮
|       |    │   Welcome to the Visual  │
|--(o)--|    │   Studio Code Extension  │

---------´   │        generator!        │     ( _´U_ )    ╰──────────────────────────╯
/A\   /
|  ~  |
'..'_
´    |° ´ Y
? What type of extension do you want to create? New Extension (TypeScript)
? What's the name of your extension? HelloWorld
? What's the identifier of your extension? helloworld
? What's the description of your extension?
? Initialize a git repository? Yes
? Bundle the source code with webpack? No
? Which package manager to use? npm
Writing in C:\Users<username>\temp\helloworld...
Error code
Object.fromEntries is not a function
@UnitName MINGW64 ~/temp
$

What is your version of Node.js (node -v)?
Looks like you are using "old" version of Node.js as Object.fromEntries should be supported in v12+

Just updated node.js to 16.6.1 and that seems to have fixed it.
Thanks.


What is the extrinsic of imu and gt (Leica) in hall data?

First, thank you for providing the dataset!
I am trying to compare GT and LiO-SAM pose results using hall data.
I would like to compare the pose results with respect to the IMU coordinate system.
So, can you give me the extrinsic between GT (Leica) and handsfree IMU?
For example, translational and rotational parameters.
Thanks.

Asking for the same question. Would like to know the extrinsic matrix so I can align the trajectories and calculate the absolute pose error. Do you manage to find out the answer?

Asking for the same question.


Support panic and trap for jvm target

Purpose
Support panic and trap statement in bir and jvm codegen.
Approach
This section covers the error type and value, panic and trap constructs. Here error handling will be described in two sub sections with BIR and JVM target.
BIR Design
Error type and literal constructor
%8 = new_error %9 %10;

Panic
Panic will be a terminator in BIR.
%4 = new_error %5 %6;
panic %4;

Trap
We will use error table to mark the trap expression in BIR. New basic block will be created for each trap expression.
	Error Table
		BB	|errorOp
		bb1	|%6
		bb3	|%11
		bb4	|%10

JVM Code Gen

Ballerina construct
JVM Representation

error
Runtime Exception

panic
Throw

trap
Try catch

Samples
Panic example
ex.
type AccountNotFoundError error<string, string>;
function getAccountBalance(int accountID) returns int {

        string errorReason = "Account Not Found";
        AccountNotFoundError accountNotFoundError =  error(errorReason,"3");
        if (accountID == 0){
            panic accountNotFoundError;
        }
        return accountID;

}

public function main() returns int {
    var result = getAccountBalance(0);
	return result;
}

this will result in
Exception in thread "main" Account Not Found 3
at hello.getAccountBalance(Unknown Source)
at hello.main(Unknown Source)
at hello.main(Unknown Source)
Trap sample
public function main() returns error? {
   var x =   trap foo(0);
   if ( x is error) {
       return x;
   }
}
public function foo(int x) returns error|int  {
     if(x == 0){
        panic error("reason", {"message":"detail"});
    } else {
        return x;
    }
}

this will result in
reason {"message":"detail"}
Remarks
We use java stacktrace directly when printing. This required to be formatted to comply with bal stacktrace.
Check List

[x] Read the Contributing Guide
[ ] Required Balo version update
[ ] Updated Change Log
[x] Added necessary tests

[x] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

@warunalakshitha , conflicts, lets fix them

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@7328126). Click here to learn what that means.
The diff coverage is 73.25%.

@@            Coverage Diff            @@
##             master   #14759   +/-   ##
=========================================
  Coverage          ?   64.35%
  Complexity        ?      549
=========================================
  Files             ?     2286
  Lines             ?   100520
  Branches          ?    13317
=========================================
  Hits              ?    64685
  Misses            ?    30771
  Partials          ?     5064

Impacted Files
Coverage Δ
Complexity Δ

...org/wso2/ballerinalang/compiler/bir/BIRGenEnv.java
100% <ø> (ø)
0 <0> (?)

.../org/ballerinalang/packerina/cmd/BuildCommand.java
20.28% <ø> (ø)
0 <0> (?)

...java/org/ballerinalang/packerina/BuilderUtils.java
69.35% <0%> (ø)
0 <0> (?)

...rg/wso2/ballerinalang/compiler/bir/BIREmitter.java
0% <0%> (ø)
0 <0> (?)

...2/ballerinalang/compiler/bir/model/BIRVisitor.java
3.84% <0%> (ø)
0 <0> (?)

...allerinalang/compiler/bir/model/BIRTerminator.java
95% <100%> (ø)
0 <0> (?)

...ain/java/org/ballerinalang/test/util/BRunUtil.java
72.52% <100%> (ø)
67 <0> (?)

...tiveimpl/jvm/methodvisitor/VisitTryCatchBlock.java
100% <100%> (ø)
0 <0> (?)

...erinalang/compiler/bir/writer/BIRBinaryWriter.java
88.4% <100%> (ø)
0 <0> (?)

...lang/compiler/bir/writer/BIRInstructionWriter.java
87.41% <100%> (ø)
0 <0> (?)

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7328126...4de4081. Read the comment docs.


feat: add memoize package

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 85.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
15

Relevant Lines:
16

💛  - Coveralls


Use action dispatch

https://github.com/jetruby/apollo_upload_server-ruby/pull/51 modified the application to use rack from ActionDispach, but the effect of this change caused the existing application to stop working.
Specifically, the same implementation as in this comment(https://github.com/jetruby/apollo_upload_server-ruby/issues/10#issuecomment-406928478) is not able to use to_io due to the type change, resulting in a NoMethodError.
I would like to revert from rack to ActionDispach, and also upgrade to version <IP_ADDRESS> to be compatible with Rails 7.
If this is ok, please use this pull request.

Thank you!

Published to rubygems


Better icon placement in the Tree View

Please Select if your Request is Either something new or an Enhancement

[x] Enhancement of an existing Feature.
[ ] Request of a new feature.

Please select the area your request applys to. (Multiple selections are Possible)

[ ] Onboard. Initial vault setup and import/export pods
[ ] Create. Note creation, lookup, snippets and templates
[ ] Retrieve. Backlinks, references, graph view
[ ] Structure. Refactoring, multi-vault and schemas
[ ] Publish. Sharing your repo with the world
[x] Misc (Choose this if your not sure)

Is your feature request related to a problem? Please describe
As I understand, the schema-added icons in the Dendron tree view perform a different role as compared to VSCode file type icons in its explorer view. Since every file has a type (even file with unknown types are given a placeholder icon), each entry in the explorer view will have an icon and folder icons can additionally indicate that something can be expanded:

In the Dendron tree, an icon just indicates a relation to an existing schema and both leaf and non-leaf nodes can have the same icons. Since not all nodes have icons, the tree view gets confusing at times when it's not possible to tell visually the structure of the node.
Consider this structure:

one

four
three

blah

foo

five

two

The above renders as follows:

In the image, the node two seems to belong to the node five which is not the case.
I think indenting of some nodes with icons and not indenting others leads to this confusion as node labels are not longer reliably aligned.
If all nodes are given an icon (even an empty square one), like in the explorer view, then the problem goes away, but the tree becomes too visually cluttered since most of the icons are not needed.
Describe the solution you'd like
As an alternative, I suggest moving the schema-added icon to the right of the node name which makes the hierarchy cleanly visible and schema icons scannable at a glance as they are right-aligned:

Additional context
As an added benefit, it would be harder to accidentally click the + button that creates an actual note from a stub. I'm not sure about others, but I often do that when working with the tree.

oh wow, I really like this idea! Thanks @aleksey-rowan

Somewhat connected to this issue:
It would be nice if there was some highlight depending on the note type.
For instance something like this for tasks and their status in the tree view:

Just an example (not colour blindness compatible).
I can create a new issue if this is unconnected.

Perhaps instead of the plus symbol, the folder without a note could be sligtly darker like in github?


invalid character @ in pkgid (rust 1.58.1)

Steps to reproduce the bug with the above code
git clone "https://gitlab.com/arnaudgolfouse/ielr.git"
cd ielr/
git reset --hard 834cfe85f0e5ddb78acf32c4f2d62c8ff6d110c4
cargo semver-checks check-release

Actual Behaviour
The output of the above commands is:
Error: Failed when running cargo-doc on <...>/ielr/target/semver-checks/local-ielr-0_1_0/Cargo.toml:
error: invalid character `@` in pkgid<EMAIL_ADDRESS>characters must be Unicode XID characters (numbers, `-`, `_`, or most letters)

Expected Behaviour
The project is checked without this error
Generated System Information
Software version
cargo-semver-checks 0.20.0 (e28327b)
Operating system
Linux 5.19.0-41-generic
Command-line
/home/arnaud/.cargo/bin/cargo-semver-checks semver-checks --bugreport

cargo version
> cargo -V
cargo 1.58.0 (f01b232bc 2022-01-19)

Compile time information

Profile: release
Target triple: x86_64-unknown-linux-gnu
Family: unix
OS: linux
Architecture: x86_64
Pointer width: 64
Endian: little
CPU features: fxsr,sse,sse2
Host: x86_64-unknown-linux-gnu

Build Configuration
In .cargo/config.toml
[target.'cfg(feature = "cargo-clippy")']
# Catch debugging remnants.
# They cannot be in lib.rs, because of a clippy bug: see <https://github.com/rust-lang/rust-clippy/issues/6610>.
rustflags = [
    "-Wclippy::print_stdout",
    "-Wclippy::print_stderr",
    "-Wclippy::dbg_macro",
]

In rust-toolchain.toml:
[toolchain]
channel = "1.58"
components = ["rustfmt", "clippy"]
profile = "minimal"

Additional Context
No response

Unfortunately, this is due to the Rust version which is older than what cargo-semver-checks supports. It isn't practical from a maintenance point of view in this project to support too wide of a range of Rust versions, since:

the rustdoc JSON format keeps changing
older Rust releases have bugs in the generated rustdoc that are only fixed in newer Rust versions
resources are limited, and every minute spent on older Rust versions is a minute that could have been spent on the ever-growing list of new features requested by the community

As a result, cargo-semver-checks v0.20 supports Rust 1.65+, and the future v0.21 supports Rust 1.67+. Technically, this isn't an MSRV in the classical sense: cargo-semver-checks will compile fine on older Rust too — but the rustdoc JSON format it reads must be generated via a sufficiently-new Rust.
I have confirmed that on a newer Rust, semver-checking this crate succeeds without issue:
$ cargo semver-checks check-release
    Updating index
     Parsing ielr v0.1.0 (current)
     Parsing ielr v0.1.0 (baseline)
    Checking ielr v0.1.0 -> v0.1.0 (no change)
   Completed [   0.045s] 43 checks; 43 passed, 0 unnecessary

I'm sorry if this is not the answer you were hoping for 😔 Is running the check on newer Rust an option in your use case?

I understand, and checking with a newer Rust is perfectly acceptable to me.
Maybe the rust version could be detected, and the error message would tell you that you need at least Rust 1.xx ? I will try to look into that.
Thank you for your answer !


Produce valid XHTML by closing tags

Close the image and stylesheet link tags to:

produce valid XHTML for more extensive compatibility
provide better (IMHO) readability

It was XHTML before, but then at some point (don't rememebr why) we made it more HTML5-ish :D which allows <link href="..."> tags.
And I prefer to stick with this style. I would accept a PR that adds ability to switch default templates (probably with presets) though.


column names within sparklyr

How can I access column names in the mutate series of functions within sparklyr to do calculations that involve the column name?  Specifically how can I deparse the . to a string?

@alexanderchernyakovgithub could you provide an example of what you're trying to do?

sure: imagine a dataframe that has numbers in the column names, i want to use mutate_at to modify certain columns and use the number contained in their name in the calculation. this is just a contrived example obviously but i think illustrates the point.


Add test coverage to assignments

This adds test coverage for AssignmentVariant structure.
Estimated coverage 78%.
Note

This depends on this fix to be applied on AssignmentsVariant::zero_balanced(). For now the test logic is reversed here.

// Check sum verification of commitments
        assert_eq!(
            amount::Confidential::verify_commit_sum(positve, negative),
            false
        );

verify_commit_sum() should return true once zero_balanced() is fixed. Wrong values are not tested here as it has already been covered in test cases for Ammount structures.

This huge structure PEDERSAN_VARIANT is this big because of the confidential amounts. Each variant type contains 4 types of assignments, Revealed, Confidential, ConfidentialSeal, ConfidentialAmount. As each confidential amount is 712 bytes as encoded data, a pedersan type assignment variant with all 4 types of assignments becomes quite large.

Also let me know if any more test cases to be added.

If you re-base this onto master it will not require the fix — it's already in the latest commit there.

Hmm. It seems I am already rebased on top of master. And the "commit_sum_verify()" is still returning false. Probably I am going somewhere off in the testing logic. Will try to debug this tomorrow.

Right... Here was my fix: https://github.com/LNP-BP/rust-lnpbp/commit/61ce085f955346f189864e9315757c5295630983
Strange. Ok, let's investigate this tomorrow

Ok, I read the code and see why it's not working. The idea of zero_balanced is that it balances inputs and outputs, with outputs split into ours (i.e. with known seal data) and theirs (only confidential seals). You mix all together: inputs, outputs, ours, theirs (literaly using the same data for both inputs and outputs), and then split them back into inputs and outputs just by filtering with >3. This can't work... Pls let me know if I need to provide more explanations on how zero_balanced works

Updated the zero_balance() test to correct logic. Commit sum verification passing now.

Added auto-generated testing. It now tests 20 times with randomly generated input vector of random sizes.

Thank you. While this can be a separate PR, we clearly need to cover the followinng cases:

When inputs and outputs are not equivalent in both (a) amounts and (b) count (i.e. sum(inputs) = sum(outputs), but count(inputs) != count (outputs)
Edge cases when inputs/outputs contains amounts = u64::MAX, 0
Cases where either ours or theirs is empty
Failed tests where sum(inputs) != sum (outputs), including edge situations when they differ for 1 or u64::MAX

Added all the suggested test conditions. Also modified the testing function so that any other conditions can be easily added.

PS If you can add all the requested changes as a new commitment(s) on top of the previous ones, for easier review, that will be very good

Added requested updates. Changes are divided into two commits. 1st one refactors the test helpers and 2nd one modifies the assignment tests.
Note:

The random amounts loop is run for 5 times to keep the total testing time within 60 sec.

The overflow conditions has to be modified slightly.

Travis is complaining on overflows
error: this arithmetic operation will overflow
    --> src/rgb/contract/assignments.rs:1181:40
     |
1181 |         assert!(compute_zero_balance(&[u64::MAX + 1], &[0], 1));
     |                                        ^^^^^^^^^^^^ attempt to compute `u64::MAX + 1_u64` which would overflow
     |
     = note: `#[deny(arithmetic_overflow)]` on by default
error: this arithmetic operation will overflow
    --> src/rgb/contract/assignments.rs:1183:15
     |
1183 |             &[u64::MAX + u64::MAX],
     |               ^^^^^^^^^^^^^^^^^^^ attempt to compute `u64::MAX + u64::MAX` which would overflow
error: this arithmetic operation will overflow
    --> src/rgb/contract/assignments.rs:1188:16
     |
1188 |             &[(u32::MAX * u32::MAX - 1) as u64],
     |                ^^^^^^^^^^^^^^^^^^^ attempt to compute `u32::MAX * u32::MAX` which would overflow
error: this arithmetic operation will overflow
    --> src/rgb/contract/assignments.rs:1193:15
     |
1193 |             &[(u32::MAX * u32::MAX) as u64],
     |               ^^^^^^^^^^^^^^^^^^^^^ attempt to compute `u32::MAX * u32::MAX` which would overflow
error: this arithmetic operation will overflow
    --> src/rgb/contract/assignments.rs:1198:16
     |
1198 |             &[(u32::MAX * u32::MAX + 1) as u64],
     |                ^^^^^^^^^^^^^^^^^^^ attempt to compute `u32::MAX * u32::MAX` which would overflow
error: aborting due to 5 previous errors
error: could not compile `lnpbp`.

pushed a fix for Travis.

Pushed the last requested change. ./contrib/test.sh is passing fully in my local.

Not sure how to fix this Travis error
thread 'rgb::contract::assignments::test::test_zero_balance' panicked at 'attempt to add with overflow', src/rgb/contract/assignments.rs:1182:40

Added  #[allow(arithmetic_overflow)], but that doesn't seem to be working.

I think it is required to use https://doc.rust-lang.org/std/num/struct.Wrapping.html

As the error is not occurring in my local, can't verify it, but this should work. Let's see if Travis complains.

We did nearly everything wrong with this overflow tests :) Pls see my fix commitment: https://github.com/LNP-BP/rust-lnpbp/commit/0f8674d39f4eb38e4178ba6f3c33579774183beb
There is no point in overflow testing outside of Pedersen commitment: we test just compiler, not zero_balance summation. Then, with Pedersen commitments we operate u256, not u64, so they will not overflow there :)
Also there were two slight issues causing tests to fail non-deterministically, which I fixed.


MWE.xsl 47-54 For the MWE metric, how to calculate the topics is not implemented.

To calculate the Maximal Weighted Entropy (MWE) the "topic" is needed. Currently this process is not implemented.
To calculate the topics: a method contains words excluding programming language keywords, stop words and punctuation. So, to calculate the MWE metric each "variable's name" is a topic.
A High Cohesive Class has two characteristics:

High Occupancy (Oi): the main topic is present in all the methods.
High Distribution (Di): all the methods are related to the same topic.

For more information on how to calculate the MWE metric, check the PDF [lyu09] in the papers.

@yegor256/z please, pay attention to this issue

@uurl we have a puzzle for that, right?

@yegor256 the puzzle #144 talked about it, but currently it doesn't specify how calculate the topics

@uurl this is a problem related to the puzzle. No need to create a new ticket for it. While you (or someone else) will be solving the puzzle, the problem will appear and it will lead to new puzzles.

@yegor256 Ok, thanks


🐞 [Bug]: Gateway contract isn't deleted after deleting CasperLabs instance

Is there an existing issue for this?

[X] I have searched the existing issues

which package/s did you face the problem with?
Dashboard
What happened?
After deleting a Casper Labs instance from its solution page, I found a contract of gateway type wasn't deleted.
https://github.com/threefoldtech/tf-images/issues/226#issuecomment-2081513550
Steps To Reproduce
No response
which network/s did you face the problem on?
QA
version
2.4.0-rc1
Twin ID/s
52
Node ID/s
3
Farm ID/s
1
Contract ID/s
27305
Relevant screenshots/screen records

Relevant log output
{
  "version": 0,
  "twin_id": 52,
  "contract_id": 27305,
  "metadata": "",
  "description": "",
  "expiration": 0,
  "signature_requirement": {
    "requests": [
      {
        "twin_id": 52,
        "required": false,
        "weight": 1
      }
    ],
    "weight_required": 1,
    "signatures": [
      {
        "twin_id": 52,
        "signature": "80bb14dd44dbb0e0b3733fb440360c36ce2d7eade34d4dad1fda0c2f45249a6494650f4ea108e7b0de459ed425f566329e15babb0d66a027c89550b330713780",
        "signature_type": "sr25519"
      }
    ],
    "signature_style": ""
  },
  "workloads": [
    {
      "version": 0,
      "name": "cl52clv2dtc",
      "type": "gateway-name-proxy",
      "data": {
        "name": "cl52clv2dtc",
        "tls_passthrough": false,
        "backends": [
          "http://<IP_ADDRESS>:80"
        ],
        "network": "nwkcdiv"
      },
      "metadata": "",
      "description": "",
      "result": {
        "created":<PHONE_NUMBER>,
        "state": "ok",
        "message": "",
        "data": {
          "fqdn": "cl52clv2dtc.gent02.qa.grid.tf"
        }
      }
    }
  ]
}

Issue Update
Investigation and Solution:
Indeed, the issue persists. It appears that the project name is not being included in the solutionHasGateway function, causing it to return true if the solution has a gateway, and false if not. To resolve this, we need to ensure that the project name is properly integrated into the solutionHasGateway function's logic.

Issue update
Work done
Fix deleting the gateway contract in the Casperlabs solution by adding the 'Casperlabs' project name in the solutionHasGateway function.
Changes

added the Casperlabs project name in the solutionHasGateway function.

Screenshots/Video
Screencast from 05-07-2024 02:12:16 PM.webm

Verified, Devnet 1b19f4f,
I deployed a few solution instance that use a gateway then deleted them, and wasn't able to reproduce the case again.


Update reference files for Windows

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
63

Relevant Lines:
63

💛  - Coveralls


DSND-2453: Fixes or some bugs found during bulk integration tests

Describe the changes

IntegrationDataGenerator now looks for generated Perl FH documents by various methods while building test data sets
Added extra logging for errors when transforming deltas
Added extra test data

Related Jira tickets
Jira Ticket
Developer check list
General

[ ] Is the PR as small as it can be?
[ ] Has the Jira ticket been updated with any relevant comments?
[ ] Is the README up to date?
[ ] Has the POM been updated for the latest versions of dependencies?
[ ] Has the code been double-checked against main?
[ ] Does the code adhere standards in this repository, including SonarQube checks?

Testing

[ ] Do the code changes have unit and integration tests with code coverage > 80%?
[ ] Has the code been tested locally and/or passed the API Karate tests on Tilt?
[ ] Have mandatory manual test cases been run?
[ ] Are extra Karate tests required?
[ ] Are all the test data resources up to date?

Release preparation
Where possible, add links to the respective Jira tickets when an item is checked

[ ] Have these changes been included in a release ticket?
[ ] Are changes required to the docker-chs-development deployment or test data?
[ ] Are any changes required to the environment added to chs-configs?

Notes to the tester
Add any testing hints or instructions here.

CI: Security warnings found!


Consume lots of API limit if batch status is Not Processed

When the batch status is not completed then it will try to hit API and API limit exceed. It calls in every one second because in bulk.js they set up hardcoded value for the interval.
I am using this API and it exceeds my limit 100K in 4-5 Hours.

This should be fixed in v3, see: https://github.com/jsforce/jsforce/discussions/1529


API similar to pytorch for loading data

I've been using keras for some projects where I've needed each batch to contain multiple samples of each class, e.g. each batch should contain twenty classes and each class should have four samples. I can't really use the standard ImageDataGenerators flow_from_directory for this since it returns either the images in order or the images in random order.
However, pytorch/torchvision has a really nice data loading api where you have the following abstractions.

A dataset that includes all images/labels for your data. The dataset doesn't need to store its data in memory, examples of datasets are ImageFolder, MNIST, and so forth.
A sampler that defines how to draw samples from the dataset, a sampler could visit all items in the dataset in order, or it could draw items randomly or in some user-defined way.
A sequence of transforms that define how to transform the dataset.
A dataloader that draws batches from a dataset by the order defined by the sampler and yields minibatches of the dataset. The dataloader is responsible for multi-threading, pinning memory and so forth.

So for my use case I just need to write my own sampler, and then I can use the rest of the API transparently instead of having to write my own multi-threaded ImageDataGenerator-workalike.
Having written my own dataloaders, and having used tensorflows queues I must say that I'm quite impressed with the simplicity of pytorch in this case, and being that Keras is a really nice high-level framework for deep learning I think a similar API would be of use here too.
So my code would be something like:
class ClassSampler(Sampler):
    def __init__(self, dataset, num_samples):
        super(ClassSampler, self).__init__()
        self.dataset = dataset
        self.num_samples = num_samples
        self.class_imgs = defaultdict(list)
        self.current_class_imgs = defaultdict(list)
        for img_idx, (_, class_) in enumerate(self.dataset.imgs):
            self.class_imgs[class_].append(img_idx)

    def __iter__(self):
        classes = list(self.class_imgs.keys()):
        random.shuffle(classes)
        for c in classes:
            for _ in range(self.num_samples):
                if not self.current_class_imgs[c]:
                    self.current_class_imgs[c] = self.class_imgs[c][:]
                    random.shuffle(self.current_class_imgs[c])
                yield self.current_class_imgs[c].pop()

    def __len__(self): return len(self.dataset.imgs)

trainset = ImageFolder(path)
sampler = ClassSampler(trainset, 4)
train_transform = transforms.Compose([transforms.RandomCrop((160, 160)), transforms.RandomHorizontalFlip(), transforms.Lambda(preprocess_input)])
trainloader = DataLoader(trainset, batch_size=128, num_workers=8, drop_last=True, sampler=sampler)

model.fit_generator(trainloader, ...)

Does that make sense? There is a lot more to read on this issue in the pytorch documentation, for example this tutorial.

@dnaq have you tried this API on a custom dataset? I am looking for something like Pytorch's Dataloader.

@Adityakunwar, You may want to try Sequence