Memory Leak with PollEvent

Hey I seem to be running into a memory leak issue with PollEvent. Here is a simple example:
for running {
	for event = sdl.PollEvent(); event != nil; event = sdl.PollEvent() {
		switch event.(type) {
		case *sdl.QuitEvent:
			running = false
		}
	}

	window.GLSwap()
}

if you set GLSetSwapInterval(0) this problem is more apparent. You can see memory keeps increasing in task manager or activity monitor (I haven't tested on linux). If you have pprof write heap profile to a file you can also see it allocating memory:
File: pollEvents.exe
Type: inuse_space
Time: Apr 2, 2019 at 6:56pm (CDT)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 3.50MB, 100% of 3.50MB total
      flat  flat%   sum%        cum   cum%
    3.50MB   100%   100%     3.50MB   100%  github.com/veandco/go-sdl2/sdl.PollEvent.func1
         0     0%   100%     3.50MB   100%  github.com/veandco/go-sdl2/sdl.PollEvent
         0     0%   100%     3.50MB   100%  main.main
         0     0%   100%     3.50MB   100%  runtime.main

And if we list that function in pprof:
ROUTINE ======================== github.com/veandco/go-sdl2/sdl.PollEvent.func1 in<EMAIL_ADDRESS>    3.50MB     3.50MB (flat, cum)   100% of Total
         .          .    866:}
         .          .    867:
         .          .    868:// PollEvent polls for currently pending events.
         .          .    869:// (https://wiki.libsdl.org/SDL_PollEvent)
         .          .    870:func PollEvent() Event {
    3.50MB     3.50MB    871:   ret := C.SDL_PollEvent(&cevent)
         .          .    872:   if ret == 0 {
         .          .    873:           return nil
         .          .    874:   }
         .          .    875:   return goEvent((*CEvent)(unsafe.Pointer(&cevent)))
         .          .    876:}

I'm not sure what is causing this, I tried to move ret out of the function like what was done with cevent but that didn't seem to do anything. I tried to use go tool compile -m on the event.go file in go-sdl to see if anything escapes to heap but that is tricky due to the use of C.
System Info:
SDL 2.0.9
Go 1.12
Windows 10 Version 1809 (gcc 8.3.0 built by MSYS2)
AND
OS 10.14.3 (same Go version, same SDL version) (clang 10.0.1)
Not going to lie, I'm using goland so I'm not sure what my build command is.
I also haven't tested this in C at all, I would be shocked if there is a leak in SDL itself though (but could definitely be a thing)
Here is my full example program:
package main

import (
	"fmt"
	"github.com/veandco/go-sdl2/sdl"
	"log"
	"os"
	"runtime/pprof"
)

func main() {
	var window *sdl.Window
	var context sdl.GLContext
	var event sdl.Event
	var running = true
	var err error

	if err := sdl.Init(sdl.INIT_EVERYTHING); err != nil {
		panic(err)
	}

	window, err = sdl.CreateWindow("test", sdl.WINDOWPOS_CENTERED, sdl.WINDOWPOS_CENTERED, 1280, 720, sdl.WINDOW_OPENGL)
	if err != nil {
		panic(err)
	}

	context, err = window.GLCreateContext()
	if err != nil {
		panic(err)
	}
	if context == nil {
		fmt.Println(sdl.GetError())
	}

	// Having an unlimited frame rate helps illustrate the problem.
	err = sdl.GLSetSwapInterval(0)
	if err != nil {
		panic(err)
	}
	defer window.Destroy()
	defer sdl.GLDeleteContext(context)
	defer sdl.Quit()

	for running {
		for event = sdl.PollEvent(); event != nil; event = sdl.PollEvent() {
			switch event.(type) {
			case *sdl.QuitEvent:
				running = false
			}
		}

		window.GLSwap()
	}

	f, err := os.Create("test.prof")
	if err != nil {
		log.Fatal(err)
	}
	pprof.WriteHeapProfile(f)
	f.Close()
}

Hi @krazzei, I couldn't reproduce the issue with the code on my Ubuntu 19.04 beta. I'm currently abroad and that's the only OS I have right now but I'll try again on my PC and/or Macs once I get back in two days :smiley:

I tried again on a fresh Fedora 30 Beta distro and it does seem to leak, even when I call SDL2's functions directly like this:
package main

// #cgo windows LDFLAGS: -lSDL2
// #cgo linux freebsd darwin pkg-config: sdl2
// #include <SDL.h>
import "C"

func main() {
        C.SDL_Init(C.SDL_INIT_EVERYTHING)

        window := C.SDL_CreateWindow(C.CString("test"), C.SDL_WINDOWPOS_CENTERED, C.SDL_WINDOWPOS_CENTERED, 1280, 720, C.SDL_WINDOW_OPENGL)
        context := C.SDL_GL_CreateContext(window)
        C.SDL_GL_SetSwapInterval(0)

        for {
                var cevent C.SDL_Event
                C.SDL_PollEvent(&cevent)
                C.SDL_GL_SwapWindow(window, 0);
        }

        C.SDL_GL_DeleteContext(context)
        C.SDL_DestroyWindow(window)
        C.SDL_Quit()
}

and when I write the C version, there doesn't seem to be a problem:
#include <SDL2/SDL.h>

int main()
{
    SDL_Window *window;
    SDL_GLContext context;
    int running = 1;

    SDL_Init(SDL_INIT_EVERYTHING);

    window = SDL_CreateWindow("test", SDL_WINDOWPOS_CENTERED, SDL_WINDOWPOS_CENTERED, 1280, 720, SDL_WINDOW_OPENGL);

    context = SDL_GL_CreateContext(window);

    SDL_GL_SetSwapInterval(0);

    while (running) {
        SDL_Event event;

        while (SDL_PollEvent(&event)) {
            switch (event.type) {
                case SDL_QUIT:
                    running = 0;
                    break;
            }
        }

        SDL_GL_SwapWindow(window);
    }

    SDL_GL_DeleteContext(context);
    SDL_DestroyWindow(window);
    SDL_Quit();
}

I'm not sure what is causing this :thinking:. I might need to ask around.

That's super odd. I wonder if Go doesn't know what to do with the C.int returned from PollEvent so it always stores it on the heap? Good to know that the C version doesn't have this problem, I assume you are referencing the same SDL libraries between Go and C?
I also wonder why it shows on Fedora but not Ubuntu. Maybe SDL was built differently?

It seems the leak happens if I call SDL_GL_SwapWindow. If I uncomment it, it doesn't happen:
package main

// #cgo windows LDFLAGS: -lSDL2
// #cgo linux freebsd darwin pkg-config: sdl2
// #include <SDL.h>
import "C"
import "runtime"

func main() {
	runtime.LockOSThread() // need to make sure everything runs on main thread for SDL_PollEvent

        C.SDL_Init(C.SDL_INIT_EVERYTHING)

        window := C.SDL_CreateWindow(C.CString("test"), C.SDL_WINDOWPOS_CENTERED, C.SDL_WINDOWPOS_CENTERED, 1280, 720, C.SDL_WINDOW_OPENGL)
        context := C.SDL_GL_CreateContext(window)
        C.SDL_GL_SetSwapInterval(0)

        for {
                var cevent C.SDL_Event
                C.SDL_PollEvent(&cevent)
                //C.SDL_GL_SwapWindow(window, 0); // uncomment this causes memory leak
        }

        C.SDL_GL_DeleteContext(context)
        C.SDL_DestroyWindow(window)
        C.SDL_Quit()
}

I'm guessing it has something to do with C and Go contexts but I'm not entirely sure.
This one is on macOS (PS. Sorry I kept changing OS, I seem to always be at different places with different hardware available when I'm checking on this issue 😅)

I found out that re-unlocking OS thread and setting GOMAXPROCS to 1 prevents the memory leak but I don't know why...
package main

import (
	"fmt"
	"log"
	"os"
	"runtime"
	"runtime/pprof"

	"github.com/veandco/go-sdl2/sdl"
)

func main() {
	var window *sdl.Window
	var context sdl.GLContext
	var running = true
	var err error

	runtime.UnlockOSThread()
	runtime.GOMAXPROCS(1)

	if err := sdl.Init(sdl.INIT_EVERYTHING); err != nil {
		panic(err)
	}

	window, err = sdl.CreateWindow("test", sdl.WINDOWPOS_CENTERED, sdl.WINDOWPOS_CENTERED, 1280, 720, sdl.WINDOW_OPENGL)
	if err != nil {
		panic(err)
	}

	context, err = window.GLCreateContext()
	if err != nil {
		panic(err)
	}
	if context == nil {
		fmt.Println(sdl.GetError())
	}

	// Having an unlimited frame rate helps illustrate the problem.
	err = sdl.GLSetSwapInterval(0)
	if err != nil {
		panic(err)
	}
	defer window.Destroy()
	defer sdl.GLDeleteContext(context)
	defer sdl.Quit()

	for running {
		for event := sdl.PollEvent(); event != nil; event = sdl.PollEvent() {
			switch event.(type) {
			case *sdl.QuitEvent:
				running = false
			}
		}

		window.GLSwap()
	}

	f, err := os.Create("test.prof")
	if err != nil {
		log.Fatal(err)
	}
	pprof.WriteHeapProfile(f)
	f.Close()
}

Did a quick test and although I do see allocations (as expected) I don't see a leak.
Try adding a call to runtime.GC() after the swap and check if you still see it.
I'm running v0.3.1-0.20190402152124-61ef6c98c6b1
(I also removed the heap profile)

I'm nowhere near qualified to give a definitive answer, but i'll share my point of view.
The allocations you are seeing are normal, it's the pointers to cevent passed to cgo.
Since we're in a hot loop calling swap, go is too busy to run gc, and defers it until it is absolutely necessary.
It is not a leak, it's just a case of lazy gc.
This is further corroborated by the fact that manual GC triggering keeps a stable memory footprint, and also from the fact that even without calling it manually, it eventually stabilizes.
Go will let the heap grow, as long as it doesn't get out of control (which by default is 2x the previous size after gc, iirc).

I'm not familiar with how Go works internally but you are probably right @flga. I have discovered that if I run the program without the tweaks, the memory starts lower around 8MB but climbs up until around 12MB and then it stops increasing. If I do the tweaks, it immediately starts at 12MB but won't increase anymore. I don't know why that is happening but it seems to match what you said :smiley:
Does that happen to you too @krazzei?

Sorry, just had time to look at this. Yeah Adding runtime.GC() stops the memory form getting out of control, but that feels like a band aid and I wouldn't want to run the gc every frame.
And adding runtime.UnlockOSThread() and runtime.GOMAXPROCS(1) seems to have removed the allocations from the memory profile but in task manager (or activity monitor) the memory keeps increasing, not sure what that is.
In regards to the cevent pointer being the problem, I thought that was fixed in another ticket (I can't seem to find it right now). The solution was to move that cevent variable from the function and make it a package global variable.

@krazzei you do not need to call GC manually, it was only to prove a point, go will do it for you.
Leaks are not collected, if after GC the retained size is the same, there's no leak.
Try to run your program for a long period of time (5 mins for example) and see if the growth is unbounded or if it stabilizes eventually.
A good way to stress it is to move your mouse.
It will probably be ok :)


MCAD Quota Management Image issue on OC 4.13

@asm582 has the his test image quay.io/asmalvan/quota-mgmt-0712 working on OC 4.11 but he noticed issues with OC 4.13.  I tried it myself on OC 4.13 and seeing that appwrappers aren't getting scheduled.
oc logs -f mcad-controller-68c575f55d-6db7h -n kube-system |grep ScheduleNext
E0713 21:13:19.865360       1 queuejob_controller_ex.go:1210] [ScheduleNext] Failed to updated status in etcd for app wrapper 'default/defaultaw-schd-spec-with-timeout-11', status = {Pending:0 Running:0 Succeeded:0 Failed:0 MinAvailable:0 CanRun:false IsDispatched:false State:Pending Message: SystemPriority:9 QueueJobState:HeadOfLine ControllerFirstTimestamp:2023-07-13 21:11:20.054349544 +0000 UTC m=+61.083581462 ControllerFirstDispatchTimestamp:0001-01-01 00:00:00 +0000 UTC FilterIgnore:true Sender:before ScheduleNext - setHOL Local:false Conditions:[{Type:Init Status:True LastUpdateMicroTime:2023-07-13 21:11:20.05458499 +0000 UTC m=+61.083816907 LastTransitionMicroTime:2023-07-13 21:11:20.05458508 +0000 UTC m=+61.083816977 Reason: Message:} {Type:Queueing Status:True LastUpdateMicroTime:2023-07-13 21:11:20.054809526 +0000 UTC m=+61.084041423 LastTransitionMicroTime:2023-07-13 21:11:20.054809606 +0000 UTC m=+61.084041493 Reason:AwaitingHeadOfLine Message:} {Type:HeadOfLine Status:True LastUpdateMicroTime:2023-07-13 21:11:25.642716112 +0000 UTC m=+66.671948019 LastTransitionMicroTime:2023-07-13 21:11:25.642716212 +0000 UTC m=+66.671948119 Reason:FrontOfQueue. Message:}] PendingPodConditions:[] TotalCPU:0 TotalMemory:0 TotalGPU:0 RequeueingTimeInSeconds:0 NumberOfRequeueings:0}, err=appwrappers.mcad.ibm.com "defaultaw-schd-spec-with-timeout-11" not found

However, on this cluster, prior to installing mcad I did have the codeflare 0.0.6 and ODH 1.7.0 installed.
Will try again on a fresh cluster.

Installation steps:

Fresh OC 4.13 cluster
git clone https://github.com/project-codeflare/multi-cluster-app-dispatcher.git
cd multi-cluster-app-dispatcher/deployment/mcad-controller
helm install the following

helm upgrade --install mcad-controller .  --namespace kube-system --wait --set image.repository=quay.io/asmalvan/quota-mgmt-0712  --set image.tag=latest  --set configMap.name=mcad-controller-configmap --set configMap.podCreationTimeout='"120000"'  --set coscheduler.rbac.apiGroup=scheduling.sigs.k8s.io --set coscheduler.rbac.resource=podgroups --set loglevel=10 --set resources.limits.cpu=1500m --set resources.requests.cpu=1160m --set configMap.quotaEnabled='"true"' --set configMap.preemptionEnabled='"true"'

start a few mcad appwrappers

../../test/perf-test/perf.sh

Watch to see that they get scheduled:

oc logs -f mcad-controller-68c575f55d-6db7h -n kube-system |grep ScheduleNext

While trying the install on a fresh OC 4.13 system, I'm getting a timeout on the helm install:
helm ls
NAME           	NAMESPACE  	REVISION	UPDATED                                	STATUS	CHART                	APP VERSION
mcad-controller	kube-system	1       	2023-07-13 15:54:11.184788043 -0700 PDT	failed	mcad-controller-0.1.0

and
oc logs -f mcad-controller-c6bc4cd4b-dffsx
E0713 23:06:13.528330       1 reflector.go:138] go/pkg/mod/k8s.io/client-go@v0.20.0/tools/cache/reflector.go:167: Failed to watch *v1.QuotaSubtree: failed to list *v1.QuotaSubtree: quotasubtrees.ibm.com is forbidden: User "system:serviceaccount:kube-system:mcad-controller" cannot list resource "quotasubtrees" in API group "ibm.com" at the cluster scope

@jbusche This PR might help: https://github.com/project-codeflare/multi-cluster-app-dispatcher/pull/475

OK, I did the following:

Clone MCAD

git clone https://github.com/project-codeflare/multi-cluster-app-dispatcher.git
cd multi-cluster-app-dispatcher/deployment/mcad-controller

Fix the clusterrole

vi templates/deployment.yaml

and search for "xqueuejobs" and add ibm.com and quotasubtrees just above it like this:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  name: system:controller:xqueuejob-controller
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
rules:
- apiGroups:
  - mcad.ibm.com
  - ibm.com
  resources:
  - quotasubtrees
  - xqueuejobs
  - queuejobs
  - schedulingspecs
  - appwrappers
  - appwrappers/finalizers
  - appwrappers/status

Install MCAD with the latest built image:

helm upgrade --install mcad-controller .  --namespace kube-system --wait --set image.repository=quay.io/asmalvan/quota-mgmt-0712  --set image.tag=latest  --set configMap.name=mcad-controller-configmap --set configMap.podCreationTimeout='"120000"'  --set coscheduler.rbac.apiGroup=scheduling.sigs.k8s.io --set coscheduler.rbac.resource=podgroups --set loglevel=10 --set resources.limits.cpu=1500m --set resources.requests.cpu=1160m --set configMap.quotaEnabled='"true"' --set configMap.preemptionEnabled='"true"'

Start a few appwrappers

../../test/perf-test/perf.sh

Monitor the mcad controller log:

oc logs -f mcad-controller-c6bc4cd4b-cmnqp |grep ScheduleNext
I0714 00:22:14.513152       1 queuejob_controller_ex.go:1459] [ScheduleNext] [Agent Mode] backing off app wrapper 'default/defaultaw-schd-spec-with-timeout-1' after waiting for 424.792µs activeQ=false Unsched=true &qj=0xc00294ef00 Version=71472 Status={Pending:0 Running:0 Succeeded:0 Failed:0 MinAvailable:0 CanRun:false IsDispatched:false State:Pending Message: SystemPriority:9 QueueJobState:HeadOfLine ControllerFirstTimestamp:2023-07-14 00:21:34.428277 +0000 UTC ControllerFirstDispatchTimestamp:0001-01-01 00:00:00 +0000 UTC FilterIgnore:true Sender:before ScheduleNext - setHOL Local:false Conditions:[{Type:Init Status:True LastUpdateMicroTime:2023-07-14 00:21:34.428444 +0000 UTC LastTransitionMicroTime:2023-07-14 00:21:34.428444 +0000 UTC Reason: Message:} {Type:Queueing Status:True LastUpdateMicroTime:2023-07-14 00:21:34.428665 +0000 UTC LastTransitionMicroTime:2023-07-14 00:21:34.428665 +0000 UTC Reason:AwaitingHeadOfLine Message:} {Type:HeadOfLine Status:True LastUpdateMicroTime:2023-07-14 00:21:34.439785 +0000 UTC LastTransitionMicroTime:2023-07-14 00:21:34.439785 +0000 UTC Reason:FrontOfQueue. Message:} {Type:Backoff Status:True LastUpdateMicroTime:2023-07-14 00:21:34.454989 +0000 UTC LastTransitionMicroTime:2023-07-14 00:21:34.454989 +0000 UTC Reason:AppWrapperNotRunnable. consumer NAMESPACE_default_AWNAME_defaultaw-schd-spec-with-timeout-1 already allocated on forest MCAD-CONTROLLER-FOREST Message:Insufficient quota to dispatch AppWrapper.}] PendingPodConditions:[] TotalCPU:0 TotalMemory:0 TotalGPU:0 RequeueingTimeInSeconds:0 NumberOfRequeueings:0}

@jbusche Thanks, looks like there is no quota tree: https://github.com/project-codeflare/multi-cluster-app-dispatcher/blob/main/test/e2e-kuttl/install-quota-subtree.yaml, can you add this quota tree, delete old AW and submit a new AW?

If possible please share the status of AWs and corresponding status of pods

Hey, that worked!

On my OC 4.13.4 cluster:

oc version
Client Version: 4.13.4
Kustomize Version: v4.5.7
Server Version: 4.13.4
Kubernetes Version: v1.26.5+7d22122

I've got the mcad running with image

    Image:         quay.io/asmalvan/quota-mgmt-0712:latest

I have the clusterrole so it has the ibm.com and qutoasubtrees added:

oc describe clusterrole system:controller:xqueuejob-controller |grep quotasubtrees
  quotasubtrees.ibm.com                []                 []              [create delete deletecollection get list patch update watch]
  quotasubtrees.mcad.ibm.com           []                 []              [create delete deletecollection get list patch update watch]

I applied the multi-cluster-app-dispatcher/test/e2e-kuttl/install-quota-subtree.yaml script

oc apply -f multi-cluster-app-dispatcher/test/e2e-kuttl/install-quota-subtree.yaml

I kicked off a few Appwrappers

test/perf-test/perf.sh

and it results in two completed pods in default namespace:
oc get appwrappers,jobs,pods -n default
NAME                                                         AGE
appwrapper.mcad.ibm.com/defaultaw-schd-spec-with-timeout-1   5m52s
appwrapper.mcad.ibm.com/defaultaw-schd-spec-with-timeout-2   5m51s

NAME                                           COMPLETIONS   DURATION   AGE
job.batch/defaultaw-schd-spec-with-timeout-1   1/1           16s        5m51s
job.batch/defaultaw-schd-spec-with-timeout-2   1/1           17s        5m49s

NAME                                           READY   STATUS      RESTARTS   AGE
pod/defaultaw-schd-spec-with-timeout-1-7gptc   0/1     Completed   0          5m51s
pod/defaultaw-schd-spec-with-timeout-2-pxzt9   0/1     Completed   0          5m49s

Describing one of the appwrappers, the status looks good:
oc describe appwrapper defaultaw-schd-spec-with-timeout-1 -n default

Status:
  Succeeded:  1
  Conditions:
    Last Transition Micro Time:      2023-07-14T17:48:37.798780Z
    Last Update Micro Time:          2023-07-14T17:48:37.798780Z
    Status:                          True
    Type:                            Init
    Last Transition Micro Time:      2023-07-14T17:48:37.798971Z
    Last Update Micro Time:          2023-07-14T17:48:37.798971Z
    Reason:                          AwaitingHeadOfLine
    Status:                          True
    Type:                            Queueing
    Last Transition Micro Time:      2023-07-14T17:48:37.809700Z
    Last Update Micro Time:          2023-07-14T17:48:37.809700Z
    Reason:                          FrontOfQueue.
    Status:                          True
    Type:                            HeadOfLine
    Last Transition Micro Time:      2023-07-14T17:48:38.205067Z
    Last Update Micro Time:          2023-07-14T17:48:38.205067Z
    Reason:                          AppWrapperRunnable
    Status:                          True
    Type:                            Dispatched
    Last Transition Micro Time:      2023-07-14T17:48:42.995906Z
    Last Update Micro Time:          2023-07-14T17:48:42.995906Z
    Reason:                          PodsRunning
    Status:                          True
    Type:                            Running
    Last Transition Micro Time:      2023-07-14T17:48:58.052701Z
    Last Update Micro Time:          2023-07-14T17:48:58.052701Z
    Reason:                          PodsCompleted
    Status:                          True
    Type:                            Completed
  Controllerfirstdispatchtimestamp:  2023-07-14T17:48:48.012632Z
  Controllerfirsttimestamp:          2023-07-14T17:48:37.798579Z
  Filterignore:                      true
  Number Of Requeueings:             0
  Queuejobstate:                     Completed
  Requeueing Time In Seconds:        0
  Sender:                            before [manageQueueJob] setCompleted
  State:                             Completed
  Systempriority:                    9
  Totalcpu:                          10
  Totalmemory:                       10485760
Events:                              <none>

This behaves the same on OC 4.12.22 as well.
    Status:                          True
    Type:                            HeadOfLine
    Last Transition Micro Time:      2023-07-18T20:11:29.045201Z
    Last Update Micro Time:          2023-07-18T20:11:29.045201Z
    Message:                         Insufficient quota to dispatch AppWrapper.
    Reason:                          AppWrapperNotRunnable. Missing required quota designation: quota_service, quota_context.
    Status:                          True
    Type:                            Backoff
    Last Transition Micro Time:      2023-07-18T20:11:31.483367Z
    Last Update Micro Time:          2023-07-18T20:11:31.483367Z
    Reason:                          AppWrapperRunnable
    Status:                          True
    Type:                            Dispatched
    Last Transition Micro Time:      2023-07-18T20:11:41.650059Z
    Last Update Micro Time:          2023-07-18T20:11:41.650059Z
    Reason:                          PodsRunning
    Status:                          True
    Type:                            Running
    Last Transition Micro Time:      2023-07-18T20:11:57.616541Z
    Last Update Micro Time:          2023-07-18T20:11:57.616541Z
    Reason:                          PodsCompleted
    Status:                          True
    Type:                            Completed
  Controllerfirstdispatchtimestamp:  2023-07-18T20:11:47.481787Z
  Controllerfirsttimestamp:          2023-07-18T20:11:29.019473Z
  Filterignore:                      true
  Number Of Requeueings:             0
  Queuejobstate:                     Completed
  Requeueing Time In Seconds:        0
  Sender:                            before [manageQueueJob] setCompleted
  State:                             Completed
  Systempriority:                    9
  Totalcpu:                          10
  Totalmemory:                       10485760
Events:                              <none>

These are in now - closing issue...


feat: As a user, I want to have a landing page

What does this PR do?
Creates the official Verax landing page
Related ticket
Fixes #363
Type of change

[ ] Chore
[ ] Bug fix
[X] New feature
[ ] Documentation update

Check list

[ ] Unit tests for any smart contract change
[ ] Contracts and functions are documented

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (a0d64e5) 93.52% compared to head (892fa6a) 93.52%.

:exclamation: Current head 892fa6a differs from pull request most recent head 84a4959. Consider uploading reports for the commit 84a4959 to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##              dev     #408   +/-   ##
=======================================
  Coverage   93.52%   93.52%
=======================================
  Files          18       18
  Lines         309      309
  Branches       68       68
=======================================
  Hits          289      289
  Misses         10       10
  Partials       10       10

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Cleanup of layout to make  less contextually specific

changes include:

switches layout to use flex (eliminates fixed positioning since that
could be problematic when  is put in other contexts (like
origin-web-console))
moves app-specific styling to app.less since landing-page.less should
be contextually agnostic (the app will need to control “advanced”
layout features like inner scrolling)
the panel now slides down in front of the overlay

known issues:

the layout of the content on the landing page is a bit janky, but
@cshinn is working on that in parallel to this effort
the snap-up icon isn’t fully visible on clicking snap-down, but I
think the entire mechanism needs rethinking now that @spadgett changed
the layout the cards/tiles

The design of the landing page is for the search component and the side bar to be fixed. @serenamarie125 @beanh66

Shouldn't there be a left margin on the page? The text runs right up against the window edge.

The edit and create project panels no longer show the panel titles or field labels.

The edit and create project panels are fixed but the sidebar isn't. If the user scrolls, the panel does not move with the target it points to.

The design of the landing page is for the search component and the side bar to be fixed. @serenamarie125 @beanh66

@jeff-phillips-18, that position fixing needs to be done at the app level, which I can add back to this demo app, but it will need to be addressed in each context (app) that utilizes  since position: fixed is relative to the viewport, and we won't know what's at the top, right, left, and bottom of the viewport.  For example, when  gets put in to origin-web-console, the offset properties are going to vary from what's in this demo app since the navbar height is going to differ (and is even customized by customers in some instances).

Shouldn't there be a left margin on the page? The text runs right up against the window edge.

We'll fix that when we style the content.  @cshinn is working on the apps section in parallel to this effort.

The edit and create project panels no longer show the panel titles or field labels.

Sorry.  I meant to document that as a known issue.  That'll get fixed when I style the sidebar stuff (which is blocked by these layout change needs).

The design of the landing page is for the search component and the side bar to be fixed.

What happens if the content of the sidebar is too long to fit on the screen?

What happens if the content of the sidebar is too long to fit on the screen?

To be clear, I'm not saying it shouldn't be fixed, but there are some problems we'll have to solve like different banner heights in different contexts (as @rhamilto points out) and content that overflows.

The edit and create project panels are fixed but the sidebar isn't. If the user scrolls, the panel does not move with the target it points to.

Missed that one since I wasn't focused on the contents.  Lemme fix.

The edit and create project panels are fixed but the sidebar isn't. If the user scrolls, the panel does not move with the target it points to.

Missed that one since I wasn't focused on the contents. Lemme fix.

Done.

Ok.  I believe I've fixed all the noted issues.  PTAnotherL.


Spelling correction

Category

[x] Content fix

What's in this Pull Request?
Simple spelling correction. Updated stucture to structure :-)

thx @stephrosem, appreciated.


Build Works, But Cannot Run "ng test"

Bug Report or Feature Request (mark with an x)
- [X ] bug report -> please search issues before submitting
- [ ] feature request

Versions.
@angular/cli: 1.1.2
node: 8.1.0
os: darwin x64
@angular/animations: 4.2.3
@angular/common: 4.2.3
@angular/compiler: 4.2.3
@angular/core: 4.2.3
@angular/forms: 4.2.3
@angular/http: 4.2.3
@angular/platform-browser: 4.2.3
@angular/platform-browser-dynamic: 4.2.3
@angular/router: 4.2.3
@angular/cli: 1.1.2
@angular/compiler-cli: 4.2.3
@angular/language-service: 4.2.3

Repro steps.
Run ng build or ng serve and the app works great. Run ng test and tests do not run...errors are below.
I can provide a sample repo if needed. Just need to strip some identifying info out of some files.
I've tried ng newing a project on this exact machine and had no issues running ng test. Unfortunately, this project was set aside for a while. When I came back to it today, tests would not run. It would be time-consuming to port everything to the latest version (long term stability is always appreciated).
The log given by the failure.
 10% building modules 1/1 modules 0 active29 08 2017 16:40:22.015:WARN [karma]: No captured browser, open http://localhost:9876/
29 08 2017 16:40:22.024:INFO [karma]: Karma v1.7.0 server started at http://<IP_ADDRESS>:9876/
29 08 2017 16:40:22.025:INFO [launcher]: Launching browser Chrome with unlimited concurrency
29 08 2017 16:40:22.030:INFO [launcher]: Starting browser Chrome                                                                                ERROR in<EMAIL_ADDRESS>(41,33): Cannot find name 'Node'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/FromEventObservable.d.ts (11,39): Cannot find name 'EventTarget'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/FromEventObservable.d.ts (11,103): Cannot find name 'NodeList'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/FromEventObservable.d.ts (11,114): Cannot find name 'HTMLCollection'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/AjaxObservable.d.ts (16,23): Cannot find name 'XMLHttpRequest'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/AjaxObservable.d.ts (77,59): Cannot find name 'Event'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/AjaxObservable.d.ts (82,13): Cannot find name 'Event'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/AjaxObservable.d.ts (97,20): Cannot find name 'Event'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/AjaxObservable.d.ts (98,10): Cannot find name 'XMLHttpRequest'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/AjaxObservable.d.ts (108,32): Cannot find name 'Event'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/AjaxObservable.d.ts (108,44): Cannot find name 'XMLHttpRequest'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/AjaxObservable.d.ts (119,10): Cannot find name 'XMLHttpRequest'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/AjaxObservable.d.ts (124,39): Cannot find name 'XMLHttpRequest'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/AjaxObservable.d.ts (132,22): Cannot find name 'XMLHttpRequest'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/WebSocketSubject.d.ts (10,29): Cannot find name 'MessageEvent'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/WebSocketSubject.d.ts (11,33): Cannot find name 'Event'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/WebSocketSubject.d.ts (12,34): Cannot find name 'CloseEvent'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/WebSocketSubject.d.ts (15,63): Cannot find name 'WebSocket'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/WebSocketSubject.d.ts (27,13): Cannot find name 'WebSocket'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/WebSocketSubject.d.ts (28,32): Cannot find name 'Event'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/WebSocketSubject.d.ts (29,33): Cannot find name 'CloseEvent'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/WebSocketSubject.d.ts (32,63): Cannot find name 'WebSocket'.

ERROR in /Users/user/ng2-sandbox/node_modules/rxjs/observable/dom/WebSocketSubject.d.ts (36,23): Cannot find name 'MessageEvent'.

ERROR in<EMAIL_ADDRESS>(32,13): Cannot find name 'Blob'.

ERROR in<EMAIL_ADDRESS>(75,30): Cannot find name 'FormData'.

ERROR in<EMAIL_ADDRESS>(75,55): Cannot find name 'Blob'.

ERROR in<EMAIL_ADDRESS>(35,43): Cannot find name 'Blob'.
29 08 2017 16:41:22.034:WARN [launcher]: Chrome have not captured in 60000 ms, killing.
29 08 2017 16:41:22.148:INFO [launcher]: Trying to start Chrome again (1/2).
29 08 2017 16:42:22.151:WARN [launcher]: Chrome have not captured in 60000 ms, killing.

Desired functionality.
I just want my unit tests to run. That's it.

+1, cannot run ng test !

Heya, errors of that sort usually mean you need to update src/tsconfig.spec.json to include the necessariy libs. Have a look at https://github.com/angular/angular-cli/wiki/stories-1.0-update#one-tsconfig-per-app for details about this.

Hey @filipesilva, that solved my problem. Thanks for your help...I would have never figured that out on my own!


[Android Sys Utils] add new utility functions

This also updates the start method in such a way to split between launching an app using an action and using a component name (i.e. start_activity and start_action).
Signed-off-by: Michele Di Giorgio<EMAIL_ADDRESS>

Cool merged!


[SP-2701] Backport of PDI-15258 - Null pointer Exception on JSON input (6.1 Suite)

Head build failed with return code 1

Head build failed with return code 1

Build completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands:

ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -Dtestreports.dir=bin/reports/integration-test -f engine/build.xml jacoco-integration checkstyle publish-local
ant -Dtestreports.dir=bin/reports/unit-test -f plugins/kettle-json-plugin/build.xml clean-all resolve jacoco && ant -Dtestreports.dir=bin/reports/integration-test -f plugins/kettle-json-plugin/build.xml jacoco-integration checkstyle publish-local

Cleanup Commands:
rm -r ~/.ivy2/local || echo no publish local to remove
Changed files:
engine/test-src/org/pentaho/di/base/LoadSaveBase.java
engine/test-src/org/pentaho/di/job/entry/loadSave/LoadSaveTester.java
engine/test-src/org/pentaho/di/job/entry/loadSave/TransStepLoadSaveTester.java
engine/test-src/org/pentaho/di/trans/RowStepCollector.java
engine/test-src/org/pentaho/di/trans/TransTestFactory.java
engine/test-src/org/pentaho/di/trans/steps/loadsave/LoadSaveTester.java
engine/test-src/org/pentaho/di/trans/steps/loadsave/initializer/InitializerInterface.java
engine/test-src/org/pentaho/di/trans/steps/loadsave/initializer/JobEntryInitializer.java
engine/test-src/org/pentaho/di/trans/steps/loadsave/initializer/StepMetaInitializer.java
plugins/kettle-json-plugin/src/org/pentaho/di/trans/steps/jsoninput/JsonInput.java
plugins/kettle-json-plugin/src/org/pentaho/di/trans/steps/jsoninput/JsonInputData.java
plugins/kettle-json-plugin/src/org/pentaho/di/trans/steps/jsoninput/JsonInputMeta.java
plugins/kettle-json-plugin/src/org/pentaho/di/trans/steps/jsoninput/JsonReader.java
plugins/kettle-json-plugin/src/org/pentaho/di/trans/steps/jsoninput/NJSONArray.java
plugins/kettle-json-plugin/test-src/org/pentaho/di/trans/steps/jsoninput/JsonInputIntTest.java
plugins/kettle-json-plugin/test-src/org/pentaho/di/trans/steps/jsoninput/JsonInputMetaLoadSaveTest.java
Unit test coverage change:
org.pentaho.di.core.auth.core.impl.DefaultAuthenticationConsumerFactory:

Branch Change : -7.14%
Complexity Change : -7.69%

org.pentaho.di.job.entries.sql.JobEntrySQL:

Instruction Change :  + 1.64%
Line Change :  + 1.97%

org.pentaho.di.repository.kdr.KettleDatabaseRepositoryCreationHelper:

Branch Change :  + 1.64%
Complexity Change :  + .69%
Instruction Change :  + 2.43%
Line Change :  + 2.95%

org.pentaho.di.trans.step.StepMeta:

Instruction Change :  + .19%
Line Change :  + .27%

org.pentaho.di.trans.steps.accessinput.AccessInputField:

Complexity Change :  + 2.13%
Instruction Change :  + 2.12%
Line Change :  + 2.47%
Method Change :  + 2.78%

org.pentaho.di.trans.steps.accessinput.AccessInputMeta:

Branch Change :  + 2.03%
Complexity Change :  + 1.21%
Instruction Change :  + 2.86%
Line Change :  + 2.42%
Method Change :  + 1.16%

org.pentaho.di.trans.steps.addsequence.AddSequenceMeta:

Complexity Change :  + 2.00%
Instruction Change :  + .64%
Line Change :  + 1.22%
Method Change :  + 2.63%

org.pentaho.di.trans.steps.analyticquery.AnalyticQueryMeta:

Complexity Change :  + 1.96%
Instruction Change :  + .66%
Line Change :  + 1.44%
Method Change :  + 3.33%

org.pentaho.di.trans.steps.autodoc.AutoDocMeta:

Complexity Change :  + 1.89%
Instruction Change :  + .60%
Line Change :  + 1.05%
Method Change :  + 2.50%

org.pentaho.di.trans.steps.changefileencoding.ChangeFileEncodingMeta:

Complexity Change :  + 3.12%
Instruction Change :  + 1.05%
Line Change :  + 1.71%
Method Change :  + 3.70%

org.pentaho.di.trans.steps.clonerow.CloneRowMeta:

Complexity Change :  + 2.44%
Instruction Change :  + .77%
Line Change :  + 1.54%
Method Change :  + 3.70%

org.pentaho.di.trans.steps.cubeinput.CubeInputMeta:

Complexity Change :  + 3.70%
Instruction Change :  + 1.60%
Line Change :  + 2.30%
Method Change :  + 4.76%

org.pentaho.di.trans.steps.cubeoutput.CubeOutputMeta:

Complexity Change :  + 4.55%
Instruction Change :  + 2.11%
Line Change :  + 3.17%
Method Change :  + 5.26%

org.pentaho.di.trans.steps.delay.DelayMeta:

Complexity Change :  + 3.23%
Instruction Change :  + 1.29%
Line Change :  + 2.44%
Method Change :  + 5.56%

org.pentaho.di.trans.steps.detectlastrow.DetectLastRowMeta:

Complexity Change :  + 5.26%
Instruction Change :  + 2.64%
Line Change :  + 3.92%
Method Change :  + 6.25%

org.pentaho.di.trans.steps.edi2xml.Edi2XmlMeta:

Complexity Change :  + 4.35%
Instruction Change :  + 1.55%
Line Change :  + 3.03%
Method Change :  + 5.88%

org.pentaho.di.trans.steps.excelinput.ExcelInputMeta:

Branch Change :  + .60%
Complexity Change :  + .51%
Instruction Change :  + 2.61%
Line Change :  + 2.41%
Method Change :  + .88%

org.pentaho.di.trans.steps.excelwriter.ExcelWriterStepField:

Complexity Change :  + 3.45%
Instruction Change :  + 3.65%
Line Change :  + 4.00%
Method Change :  + 3.45%

org.pentaho.di.trans.steps.excelwriter.ExcelWriterStepMeta:

Branch Change :  + 2.56%
Complexity Change :  + 1.57%
Instruction Change :  + 1.30%
Line Change :  + 1.32%
Method Change :  + 1.14%

org.pentaho.di.trans.steps.execprocess.ExecProcessMeta:

Complexity Change :  + 2.94%
Instruction Change :  + 1.15%
Line Change :  + 1.77%
Method Change :  + 3.85%

org.pentaho.di.trans.steps.fieldschangesequence.FieldsChangeSequenceMeta:

Complexity Change :  + 2.70%
Instruction Change :  + 3.32%
Line Change :  + 4.20%
Method Change :  + 4.35%

org.pentaho.di.trans.steps.fieldsplitter.FieldSplitterMeta:

Complexity Change :  + 1.52%
Instruction Change :  + 8.84%
Line Change :  + 7.09%
Method Change :  + 2.04%

org.pentaho.di.trans.steps.fileexists.FileExistsMeta:

Complexity Change :  + 3.33%
Instruction Change :  + 1.36%
Line Change :  + 2.13%
Method Change :  + 4.17%

org.pentaho.di.trans.steps.filelocked.FileLockedMeta:

Complexity Change :  + 4.17%
Instruction Change :  + 1.78%
Line Change :  + 2.70%
Method Change :  + 5.00%

org.pentaho.di.trans.steps.filestoresult.FilesToResultMeta:

Complexity Change :  + 5.56%
Instruction Change :  + 3.36%
Line Change :  + 5.13%
Method Change :  + 5.88%

org.pentaho.di.trans.steps.flattener.FlattenerMeta:

Complexity Change :  + 3.70%
Instruction Change :  + 1.30%
Line Change :  + 2.63%
Method Change :  + 5.56%

org.pentaho.di.trans.steps.formula.FormulaMeta:

Branch Change :  + 8.82%
Complexity Change :  + 6.06%
Instruction Change :  + 9.04%
Line Change :  + 8.33%
Method Change :  + 6.25%

org.pentaho.di.trans.steps.formula.FormulaMetaFunction:

Complexity Change :  + 4.35%
Instruction Change :  + 1.76%
Line Change :  + 2.82%
Method Change :  + 4.76%

org.pentaho.di.trans.steps.getfilenames.GetFileNamesMeta:

Complexity Change :  + 1.22%
Instruction Change :  + 3.87%
Line Change :  + 3.23%
Method Change :  + 1.75%

org.pentaho.di.trans.steps.getrepositorynames.GetRepositoryNamesMeta:

Complexity Change :  + 2.63%
Instruction Change :  + 6.11%
Line Change :  + 4.85%
Method Change :  + 3.70%

org.pentaho.di.trans.steps.getslavesequence.GetSlaveSequenceMeta:

Complexity Change :  + 4.55%
Instruction Change :  + 1.74%
Line Change :  + 3.08%
Method Change :  + 4.76%

org.pentaho.di.trans.steps.getsubfolders.GetSubFoldersMeta:

Complexity Change :  + 1.96%
Instruction Change :  + 3.14%
Line Change :  + 3.19%
Method Change :  + 3.03%

org.pentaho.di.trans.steps.jobexecutor.JobExecutorMeta:

Complexity Change :  + .57%
Instruction Change :  + .20%
Line Change :  + .41%
Method Change :  + .96%

org.pentaho.di.trans.steps.mergejoin.MergeJoinMeta:

Complexity Change :  + 2.44%
Instruction Change :  + 4.91%
Line Change :  + 6.03%
Method Change :  + 4.00%

org.pentaho.di.trans.steps.mysqlbulkloader.MySQLBulkLoaderMeta:

Complexity Change :  + 1.06%
Instruction Change :  + 2.19%
Line Change :  + 2.24%
Method Change :  + 1.85%

org.pentaho.di.trans.steps.normaliser.NormaliserMeta:

Branch Change :  + 5.26%
Complexity Change :  + 4.76%
Instruction Change :  + 5.87%
Line Change :  + 5.76%
Method Change :  + 4.35%

org.pentaho.di.trans.steps.numberrange.NumberRangeMeta:

Complexity Change :  + 3.23%
Instruction Change :  + .95%
Line Change :  + 1.69%
Method Change :  + 4.17%

org.pentaho.di.trans.steps.olapinput.OlapInputMeta:

Complexity Change :  + 2.86%
Instruction Change :  + 1.25%
Line Change :  + 1.80%
Method Change :  + 3.57%

org.pentaho.di.trans.steps.randomccnumber.RandomCCNumberGeneratorMeta:

Complexity Change :  + 2.50%
Instruction Change :  + 5.43%
Line Change :  + 5.34%
Method Change :  + 3.57%

org.pentaho.di.trans.steps.randomvalue.RandomValueMeta:

Complexity Change :  + 2.17%
Instruction Change :  + 4.41%
Line Change :  + 5.66%
Method Change :  + 5.00%

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta:

Complexity Change :  + .96%
Instruction Change :  + 5.96%
Line Change :  + 5.16%
Method Change :  + 2.22%

org.pentaho.di.trans.steps.reservoirsampling.ReservoirSamplingMeta:

Complexity Change :  + 4.35%
Instruction Change :  + 2.69%
Line Change :  + 3.92%
Method Change :  + 6.25%

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta:

Complexity Change :  + 2.86%
Instruction Change :  + 5.19%
Line Change :  + 5.94%
Method Change :  + 5.26%

org.pentaho.di.trans.steps.ssh.SSHMeta:

Complexity Change :  + 1.52%
Instruction Change :  + .55%
Line Change :  + .85%
Method Change :  + 2.00%

org.pentaho.di.trans.steps.switchcase.SwitchCaseMeta:

Branch Change :  + 3.85%
Complexity Change :  + 3.17%
Instruction Change :  + 3.17%
Line Change :  + 2.87%
Method Change :  + 2.70%

org.pentaho.di.trans.steps.switchcase.SwitchCaseTarget:

Complexity Change :  + 50.00%
Instruction Change :  + 50.00%
Line Change :  + 33.33%
Method Change :  + 50.00%

org.pentaho.di.trans.steps.symmetriccrypto.symmetriccryptotrans.SymmetricCryptoTransMeta:

Complexity Change :  + 1.85%
Instruction Change :  + .84%
Line Change :  + 1.38%
Method Change :  + 2.56%

org.pentaho.di.trans.steps.textfileinput.TextFileInputMeta:

Branch Change :  + .53%

org.pentaho.di.trans.steps.textfileoutput.TextFileField:

Complexity Change :  + 3.23%
Instruction Change :  + 2.84%
Line Change :  + 3.28%
Method Change :  + 3.23%

org.pentaho.di.trans.steps.textfileoutput.TextFileOutputMeta:

Branch Change :  + 1.69%
Complexity Change :  + 1.34%
Instruction Change :  + 1.15%
Line Change :  + 1.20%
Method Change :  + 1.11%

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta:

Complexity Change :  + 2.70%
Instruction Change :  + 5.08%
Line Change :  + 5.31%
Method Change :  + 3.70%

org.pentaho.di.trans.steps.uniquerowsbyhashset.UniqueRowsByHashSetMeta:

Complexity Change :  + 3.45%
Instruction Change :  + 5.16%
Line Change :  + 5.88%
Method Change :  + 4.35%

org.pentaho.di.trans.steps.univariatestats.UnivariateStatsMeta:

Branch Change :  + 7.14%
Complexity Change :  + 5.26%
Instruction Change :  + 6.77%
Line Change :  + 6.32%
Method Change :  + 5.88%

org.pentaho.di.trans.steps.jsoninput.JsonInput:

Branch Change :  + 2.04%
Complexity Change :  + 2.45%
Instruction Change :  + 3.60%
Line Change :  + 5.07%
Method Change :  + 4.35%

org.pentaho.di.trans.steps.jsoninput.JsonInputField:

Branch Change :  + 16.67%
Complexity Change :  + 25.00%
Instruction Change :  + 70.76%
Line Change :  + 55.36%
Method Change :  + 30.77%

org.pentaho.di.trans.steps.jsoninput.JsonInputMeta:

Branch Change :  + 37.93%
Complexity Change :  + 33.63%
Instruction Change :  + 65.61%
Line Change :  + 55.65%
Method Change :  + 33.33%

org.pentaho.di.trans.steps.jsoninput.JsonInputMeta.InputFiles:

Branch Change :  + 100.00%
Complexity Change :  + 50.00%
Instruction Change :  + 40.00%
Line Change :  + 27.78%
Method Change :  + 33.33%

Integration test coverage change:
org.pentaho.di.trans.steps.http.HTTPMeta:

Complexity Change :  + 1.47%
Instruction Change :  + 3.97%
Line Change :  + 3.90%
Method Change :  + 1.96%

org.pentaho.di.trans.steps.httppost.HTTPPOSTMeta:

Complexity Change :  + 1.33%
Instruction Change :  + 4.08%
Line Change :  + 4.28%
Method Change :  + 1.72%

org.pentaho.di.trans.steps.rest.RestMeta:

Complexity Change :  + .98%
Instruction Change :  + 4.01%
Line Change :  + 3.79%
Method Change :  + 1.52%

License header violations:

Copyright year is not the current year on engine/test-src/org/pentaho/di/job/entry/loadSave/TransStepLoadSaveTester.java. Found 2015. Was expecting 2016

Checkstyle violations:
See: https://github.com/pentaho/pentaho-coding-standards
File /home/buildguy/project/engine/test-src/org/pentaho/di/base/LoadSaveBase.java

Message: File does not end with a newline. at Line: 0

File /home/buildguy/project/engine/test-src/org/pentaho/di/job/entry/loadSave/LoadSaveTester.java

Message: File does not end with a newline. at Line: 0

File /home/buildguy/project/engine/test-src/org/pentaho/di/trans/steps/loadsave/LoadSaveTester.java

Message: File does not end with a newline. at Line: 0
Message: Using a static member import should be avoided - org.junit.Assert.assertNotSame. at Line: 25

File /home/buildguy/project/engine/test-src/org/pentaho/di/trans/steps/loadsave/initializer/InitializerInterface.java

Message: File does not end with a newline. at Line: 0

File /home/buildguy/project/engine/test-src/org/pentaho/di/trans/steps/loadsave/initializer/JobEntryInitializer.java

Message: File does not end with a newline. at Line: 0

File /home/buildguy/project/engine/test-src/org/pentaho/di/trans/steps/loadsave/initializer/StepMetaInitializer.java

Message: File does not end with a newline. at Line: 0

File /home/buildguy/project/plugins/kettle-json-plugin/test-src/org/pentaho/di/trans/steps/jsoninput/JsonInputIntTest.java

Message: Using a static member import should be avoided - org.junit.Assert.assertNotNull. at Line: 25
Message: Using a static member import should be avoided - org.junit.Assert.assertTrue. at Line: 26

Build completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands:

ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -Dtestreports.dir=bin/reports/integration-test -f engine/build.xml jacoco-integration checkstyle publish-local
ant -Dtestreports.dir=bin/reports/unit-test -f plugins/kettle-json-plugin/build.xml clean-all resolve jacoco && ant -Dtestreports.dir=bin/reports/integration-test -f plugins/kettle-json-plugin/build.xml jacoco-integration checkstyle publish-local

Cleanup Commands:
rm -r ~/.ivy2/local || echo no publish local to remove
Changed files:
engine/test-src/org/pentaho/di/base/LoadSaveBase.java
engine/test-src/org/pentaho/di/job/entry/loadSave/LoadSaveTester.java
engine/test-src/org/pentaho/di/job/entry/loadSave/TransStepLoadSaveTester.java
engine/test-src/org/pentaho/di/trans/RowStepCollector.java
engine/test-src/org/pentaho/di/trans/TransTestFactory.java
engine/test-src/org/pentaho/di/trans/steps/loadsave/LoadSaveTester.java
engine/test-src/org/pentaho/di/trans/steps/loadsave/initializer/InitializerInterface.java
engine/test-src/org/pentaho/di/trans/steps/loadsave/initializer/JobEntryInitializer.java
engine/test-src/org/pentaho/di/trans/steps/loadsave/initializer/StepMetaInitializer.java
plugins/kettle-json-plugin/src/org/pentaho/di/trans/steps/jsoninput/JsonInput.java
plugins/kettle-json-plugin/src/org/pentaho/di/trans/steps/jsoninput/JsonInputData.java
plugins/kettle-json-plugin/src/org/pentaho/di/trans/steps/jsoninput/JsonInputMeta.java
plugins/kettle-json-plugin/src/org/pentaho/di/trans/steps/jsoninput/JsonReader.java
plugins/kettle-json-plugin/test-src/org/pentaho/di/trans/steps/jsoninput/JsonInputIntTest.java
plugins/kettle-json-plugin/test-src/org/pentaho/di/trans/steps/jsoninput/JsonInputMetaLoadSaveTest.java
Unit test coverage change:
org.pentaho.di.core.auth.core.impl.DefaultAuthenticationConsumerFactory:

Branch Change :  + 7.14%
Complexity Change :  + 7.69%

org.pentaho.di.core.lifecycle.KettleLifecycleSupport:

Branch Change : -8.33%
Complexity Change : -9.09%

org.pentaho.di.trans.step.StepMeta:

Instruction Change :  + .19%
Line Change :  + .27%

org.pentaho.di.trans.steps.accessinput.AccessInputField:

Complexity Change :  + 2.13%
Instruction Change :  + 2.12%
Line Change :  + 2.47%
Method Change :  + 2.78%

org.pentaho.di.trans.steps.accessinput.AccessInputMeta:

Branch Change :  + 2.03%
Complexity Change :  + 1.21%
Instruction Change :  + 2.86%
Line Change :  + 2.42%
Method Change :  + 1.16%

org.pentaho.di.trans.steps.addsequence.AddSequenceMeta:

Complexity Change :  + 2.00%
Instruction Change :  + .64%
Line Change :  + 1.22%
Method Change :  + 2.63%

org.pentaho.di.trans.steps.analyticquery.AnalyticQueryMeta:

Complexity Change :  + 1.96%
Instruction Change :  + .66%
Line Change :  + 1.44%
Method Change :  + 3.33%

org.pentaho.di.trans.steps.autodoc.AutoDocMeta:

Complexity Change :  + 1.89%
Instruction Change :  + .60%
Line Change :  + 1.05%
Method Change :  + 2.50%

org.pentaho.di.trans.steps.changefileencoding.ChangeFileEncodingMeta:

Complexity Change :  + 3.12%
Instruction Change :  + 1.05%
Line Change :  + 1.71%
Method Change :  + 3.70%

org.pentaho.di.trans.steps.clonerow.CloneRowMeta:

Complexity Change :  + 2.44%
Instruction Change :  + .77%
Line Change :  + 1.54%
Method Change :  + 3.70%

org.pentaho.di.trans.steps.cubeinput.CubeInputMeta:

Complexity Change :  + 3.70%
Instruction Change :  + 1.60%
Line Change :  + 2.30%
Method Change :  + 4.76%

org.pentaho.di.trans.steps.cubeoutput.CubeOutputMeta:

Complexity Change :  + 4.55%
Instruction Change :  + 2.11%
Line Change :  + 3.17%
Method Change :  + 5.26%

org.pentaho.di.trans.steps.delay.DelayMeta:

Complexity Change :  + 3.23%
Instruction Change :  + 1.29%
Line Change :  + 2.44%
Method Change :  + 5.56%

org.pentaho.di.trans.steps.detectlastrow.DetectLastRowMeta:

Complexity Change :  + 5.26%
Instruction Change :  + 2.64%
Line Change :  + 3.92%
Method Change :  + 6.25%

org.pentaho.di.trans.steps.edi2xml.Edi2XmlMeta:

Complexity Change :  + 4.35%
Instruction Change :  + 1.55%
Line Change :  + 3.03%
Method Change :  + 5.88%

org.pentaho.di.trans.steps.excelinput.ExcelInputMeta:

Branch Change :  + .60%
Complexity Change :  + .51%
Instruction Change :  + 2.61%
Line Change :  + 2.41%
Method Change :  + .88%

org.pentaho.di.trans.steps.excelwriter.ExcelWriterStepField:

Complexity Change :  + 3.45%
Instruction Change :  + 3.65%
Line Change :  + 4.00%
Method Change :  + 3.45%

org.pentaho.di.trans.steps.excelwriter.ExcelWriterStepMeta:

Branch Change :  + 2.56%
Complexity Change :  + 1.57%
Instruction Change :  + 1.26%
Line Change :  + 1.32%
Method Change :  + 1.14%

org.pentaho.di.trans.steps.execprocess.ExecProcessMeta:

Complexity Change :  + 2.94%
Instruction Change :  + 1.15%
Line Change :  + 1.77%
Method Change :  + 3.85%

org.pentaho.di.trans.steps.fieldschangesequence.FieldsChangeSequenceMeta:

Complexity Change :  + 2.70%
Instruction Change :  + 3.32%
Line Change :  + 4.20%
Method Change :  + 4.35%

org.pentaho.di.trans.steps.fieldsplitter.FieldSplitterMeta:

Complexity Change :  + 1.52%
Instruction Change :  + 8.84%
Line Change :  + 7.09%
Method Change :  + 2.04%

org.pentaho.di.trans.steps.fileexists.FileExistsMeta:

Complexity Change :  + 3.33%
Instruction Change :  + 1.36%
Line Change :  + 2.13%
Method Change :  + 4.17%

org.pentaho.di.trans.steps.filelocked.FileLockedMeta:

Complexity Change :  + 4.17%
Instruction Change :  + 1.78%
Line Change :  + 2.70%
Method Change :  + 5.00%

org.pentaho.di.trans.steps.filestoresult.FilesToResultMeta:

Complexity Change :  + 5.56%
Instruction Change :  + 3.36%
Line Change :  + 5.13%
Method Change :  + 5.88%

org.pentaho.di.trans.steps.flattener.FlattenerMeta:

Complexity Change :  + 3.70%
Instruction Change :  + 1.30%
Line Change :  + 2.63%
Method Change :  + 5.56%

org.pentaho.di.trans.steps.formula.FormulaMeta:

Branch Change :  + 8.82%
Complexity Change :  + 6.06%
Instruction Change :  + 9.04%
Line Change :  + 8.33%
Method Change :  + 6.25%

org.pentaho.di.trans.steps.formula.FormulaMetaFunction:

Complexity Change :  + 4.35%
Instruction Change :  + 1.76%
Line Change :  + 2.82%
Method Change :  + 4.76%

org.pentaho.di.trans.steps.getfilenames.GetFileNamesMeta:

Complexity Change :  + 1.22%
Instruction Change :  + 3.87%
Line Change :  + 3.23%
Method Change :  + 1.75%

org.pentaho.di.trans.steps.getrepositorynames.GetRepositoryNamesMeta:

Complexity Change :  + 2.63%
Instruction Change :  + 6.11%
Line Change :  + 4.85%
Method Change :  + 3.70%

org.pentaho.di.trans.steps.getslavesequence.GetSlaveSequenceMeta:

Complexity Change :  + 4.55%
Instruction Change :  + 1.74%
Line Change :  + 3.08%
Method Change :  + 4.76%

org.pentaho.di.trans.steps.getsubfolders.GetSubFoldersMeta:

Complexity Change :  + 1.96%
Instruction Change :  + 3.14%
Line Change :  + 3.19%
Method Change :  + 3.03%

org.pentaho.di.trans.steps.jobexecutor.JobExecutorMeta:

Complexity Change :  + .57%
Instruction Change :  + .20%
Line Change :  + .41%
Method Change :  + .96%

org.pentaho.di.trans.steps.mergejoin.MergeJoinMeta:

Complexity Change :  + 2.44%
Instruction Change :  + 4.91%
Line Change :  + 6.03%
Method Change :  + 4.00%

org.pentaho.di.trans.steps.mysqlbulkloader.MySQLBulkLoaderMeta:

Complexity Change :  + 1.06%
Instruction Change :  + 2.19%
Line Change :  + 2.24%
Method Change :  + 1.85%

org.pentaho.di.trans.steps.normaliser.NormaliserMeta:

Branch Change :  + 5.26%
Complexity Change :  + 4.76%
Instruction Change :  + 5.87%
Line Change :  + 5.76%
Method Change :  + 4.35%

org.pentaho.di.trans.steps.numberrange.NumberRangeMeta:

Complexity Change :  + 3.23%
Instruction Change :  + .95%
Line Change :  + 1.69%
Method Change :  + 4.17%

org.pentaho.di.trans.steps.olapinput.OlapInputMeta:

Complexity Change :  + 2.86%
Instruction Change :  + 1.25%
Line Change :  + 1.80%
Method Change :  + 3.57%

org.pentaho.di.trans.steps.randomccnumber.RandomCCNumberGeneratorMeta:

Complexity Change :  + 2.50%
Instruction Change :  + 5.43%
Line Change :  + 5.34%
Method Change :  + 3.57%

org.pentaho.di.trans.steps.randomvalue.RandomValueMeta:

Complexity Change :  + 2.17%
Instruction Change :  + 4.41%
Line Change :  + 5.66%
Method Change :  + 5.00%

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta:

Complexity Change :  + .96%
Instruction Change :  + 5.89%
Line Change :  + 5.16%
Method Change :  + 2.22%

org.pentaho.di.trans.steps.reservoirsampling.ReservoirSamplingMeta:

Complexity Change :  + 4.35%
Instruction Change :  + 2.69%
Line Change :  + 3.92%
Method Change :  + 6.25%

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta:

Complexity Change :  + 2.86%
Instruction Change :  + 5.19%
Line Change :  + 5.94%
Method Change :  + 5.26%

org.pentaho.di.trans.steps.ssh.SSHMeta:

Complexity Change :  + 1.52%
Instruction Change :  + .55%
Line Change :  + .85%
Method Change :  + 2.00%

org.pentaho.di.trans.steps.switchcase.SwitchCaseMeta:

Branch Change :  + 3.85%
Complexity Change :  + 3.17%
Instruction Change :  + 3.17%
Line Change :  + 2.87%
Method Change :  + 2.70%

org.pentaho.di.trans.steps.switchcase.SwitchCaseTarget:

Complexity Change :  + 50.00%
Instruction Change :  + 50.00%
Line Change :  + 33.33%
Method Change :  + 50.00%

org.pentaho.di.trans.steps.symmetriccrypto.symmetriccryptotrans.SymmetricCryptoTransMeta:

Complexity Change :  + 1.85%
Instruction Change :  + .84%
Line Change :  + 1.38%
Method Change :  + 2.56%

org.pentaho.di.trans.steps.textfileoutput.TextFileField:

Complexity Change :  + 3.23%
Instruction Change :  + 2.84%
Line Change :  + 3.28%
Method Change :  + 3.23%

org.pentaho.di.trans.steps.textfileoutput.TextFileOutputMeta:

Branch Change :  + 1.69%
Complexity Change :  + 1.34%
Instruction Change :  + 1.15%
Line Change :  + 1.20%
Method Change :  + 1.11%

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta:

Complexity Change :  + 2.70%
Instruction Change :  + 5.25%
Line Change :  + 5.31%
Method Change :  + 3.70%

org.pentaho.di.trans.steps.uniquerowsbyhashset.UniqueRowsByHashSetMeta:

Complexity Change :  + 3.45%
Instruction Change :  + 5.16%
Line Change :  + 5.88%
Method Change :  + 4.35%

org.pentaho.di.trans.steps.univariatestats.UnivariateStatsMeta:

Branch Change :  + 7.14%
Complexity Change :  + 5.26%
Instruction Change :  + 6.77%
Line Change :  + 6.32%
Method Change :  + 5.88%

org.pentaho.di.trans.steps.jsoninput.JsonInput:

Branch Change :  + 2.04%
Complexity Change :  + 2.45%
Instruction Change :  + 3.60%
Line Change :  + 5.07%
Method Change :  + 4.35%

org.pentaho.di.trans.steps.jsoninput.JsonInputField:

Branch Change :  + 16.67%
Complexity Change :  + 25.00%
Instruction Change :  + 70.76%
Line Change :  + 55.36%
Method Change :  + 30.77%

org.pentaho.di.trans.steps.jsoninput.JsonInputMeta:

Branch Change :  + 37.93%
Complexity Change :  + 33.63%
Instruction Change :  + 65.61%
Line Change :  + 55.65%
Method Change :  + 33.33%

org.pentaho.di.trans.steps.jsoninput.JsonInputMeta.InputFiles:

Branch Change :  + 100.00%
Complexity Change :  + 50.00%
Instruction Change :  + 40.00%
Line Change :  + 27.78%
Method Change :  + 33.33%

Integration test coverage change:
org.pentaho.di.trans.step.BaseStep:

Branch Change :  + .44%
Complexity Change :  + .18%
Instruction Change :  + .20%
Line Change :  + .16%

org.pentaho.di.trans.steps.http.HTTPMeta:

Complexity Change :  + 1.47%
Instruction Change :  + 3.97%
Line Change :  + 3.90%
Method Change :  + 1.96%

org.pentaho.di.trans.steps.httppost.HTTPPOSTMeta:

Complexity Change :  + 1.33%
Instruction Change :  + 4.08%
Line Change :  + 4.28%
Method Change :  + 1.72%

org.pentaho.di.trans.steps.rest.RestMeta:

Complexity Change :  + .98%
Instruction Change :  + 4.01%
Line Change :  + 3.79%
Method Change :  + 1.52%

License header violations:

Copyright year is not the current year on engine/test-src/org/pentaho/di/job/entry/loadSave/TransStepLoadSaveTester.java. Found 2015. Was expecting 2016

Checkstyle violations:
See: https://github.com/pentaho/pentaho-coding-standards
File /home/buildguy/project/engine/test-src/org/pentaho/di/trans/steps/loadsave/LoadSaveTester.java

Message: Using a static member import should be avoided - org.junit.Assert.assertNotSame. at Line: 25

File /home/buildguy/project/plugins/kettle-json-plugin/test-src/org/pentaho/di/trans/steps/jsoninput/JsonInputIntTest.java

Message: Using a static member import should be avoided - org.junit.Assert.assertNotNull. at Line: 25
Message: Using a static member import should be avoided - org.junit.Assert.assertTrue. at Line: 26

Build completed
:x: This pull request has errors. They will need to be addressed before it can be accepted. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands:

ant -Dtestreports.dir=bin/reports/unit-test -f engine/build.xml clean-all resolve jacoco && ant -Dtestreports.dir=bin/reports/integration-test -f engine/build.xml jacoco-integration checkstyle publish-local
ant -Dtestreports.dir=bin/reports/unit-test -f plugins/kettle-json-plugin/build.xml clean-all resolve jacoco && ant -Dtestreports.dir=bin/reports/integration-test -f plugins/kettle-json-plugin/build.xml jacoco-integration checkstyle publish-local

Cleanup Commands:
rm -r ~/.ivy2/local || echo no publish local to remove
Changed files:
engine/test-src/org/pentaho/di/base/LoadSaveBase.java
engine/test-src/org/pentaho/di/job/entry/loadSave/LoadSaveTester.java
engine/test-src/org/pentaho/di/job/entry/loadSave/TransStepLoadSaveTester.java
engine/test-src/org/pentaho/di/trans/RowStepCollector.java
engine/test-src/org/pentaho/di/trans/TransTestFactory.java
engine/test-src/org/pentaho/di/trans/steps/loadsave/LoadSaveTester.java
engine/test-src/org/pentaho/di/trans/steps/loadsave/initializer/InitializerInterface.java
engine/test-src/org/pentaho/di/trans/steps/loadsave/initializer/JobEntryInitializer.java
engine/test-src/org/pentaho/di/trans/steps/loadsave/initializer/StepMetaInitializer.java
plugins/kettle-json-plugin/src/org/pentaho/di/trans/steps/jsoninput/JsonInput.java
plugins/kettle-json-plugin/src/org/pentaho/di/trans/steps/jsoninput/JsonInputData.java
plugins/kettle-json-plugin/src/org/pentaho/di/trans/steps/jsoninput/JsonInputMeta.java
plugins/kettle-json-plugin/src/org/pentaho/di/trans/steps/jsoninput/JsonReader.java
plugins/kettle-json-plugin/test-src/org/pentaho/di/trans/steps/jsoninput/JsonInputIntTest.java
plugins/kettle-json-plugin/test-src/org/pentaho/di/trans/steps/jsoninput/JsonInputMetaLoadSaveTest.java
Unit test coverage change:
org.pentaho.di.core.auth.core.impl.DefaultAuthenticationConsumerFactory:

Branch Change : -7.14%
Complexity Change : -7.69%

org.pentaho.di.core.lifecycle.KettleLifecycleSupport:

Branch Change : -8.33%
Complexity Change : -9.09%

org.pentaho.di.trans.step.StepMeta:

Instruction Change :  + .19%
Line Change :  + .27%

org.pentaho.di.trans.steps.accessinput.AccessInputField:

Complexity Change :  + 2.13%
Instruction Change :  + 2.12%
Line Change :  + 2.47%
Method Change :  + 2.78%

org.pentaho.di.trans.steps.accessinput.AccessInputMeta:

Branch Change :  + 2.03%
Complexity Change :  + 1.21%
Instruction Change :  + 2.86%
Line Change :  + 2.42%
Method Change :  + 1.16%

org.pentaho.di.trans.steps.addsequence.AddSequenceMeta:

Complexity Change :  + 2.00%
Instruction Change :  + .64%
Line Change :  + 1.22%
Method Change :  + 2.63%

org.pentaho.di.trans.steps.analyticquery.AnalyticQueryMeta:

Complexity Change :  + 1.96%
Instruction Change :  + .66%
Line Change :  + 1.44%
Method Change :  + 3.33%

org.pentaho.di.trans.steps.autodoc.AutoDocMeta:

Complexity Change :  + 1.89%
Instruction Change :  + .60%
Line Change :  + 1.05%
Method Change :  + 2.50%

org.pentaho.di.trans.steps.changefileencoding.ChangeFileEncodingMeta:

Complexity Change :  + 3.12%
Instruction Change :  + 1.05%
Line Change :  + 1.71%
Method Change :  + 3.70%

org.pentaho.di.trans.steps.clonerow.CloneRowMeta:

Complexity Change :  + 2.44%
Instruction Change :  + .77%
Line Change :  + 1.54%
Method Change :  + 3.70%

org.pentaho.di.trans.steps.cubeinput.CubeInputMeta:

Complexity Change :  + 3.70%
Instruction Change :  + 1.60%
Line Change :  + 2.30%
Method Change :  + 4.76%

org.pentaho.di.trans.steps.cubeoutput.CubeOutputMeta:

Complexity Change :  + 4.55%
Instruction Change :  + 2.11%
Line Change :  + 3.17%
Method Change :  + 5.26%

org.pentaho.di.trans.steps.delay.DelayMeta:

Complexity Change :  + 3.23%
Instruction Change :  + 1.29%
Line Change :  + 2.44%
Method Change :  + 5.56%

org.pentaho.di.trans.steps.detectlastrow.DetectLastRowMeta:

Complexity Change :  + 5.26%
Instruction Change :  + 2.64%
Line Change :  + 3.92%
Method Change :  + 6.25%

org.pentaho.di.trans.steps.edi2xml.Edi2XmlMeta:

Complexity Change :  + 4.35%
Instruction Change :  + 1.55%
Line Change :  + 3.03%
Method Change :  + 5.88%

org.pentaho.di.trans.steps.excelinput.ExcelInputMeta:

Branch Change :  + .60%
Complexity Change :  + .51%
Instruction Change :  + 2.61%
Line Change :  + 2.41%
Method Change :  + .88%

org.pentaho.di.trans.steps.excelwriter.ExcelWriterStepField:

Complexity Change :  + 3.45%
Instruction Change :  + 3.65%
Line Change :  + 4.00%
Method Change :  + 3.45%

org.pentaho.di.trans.steps.excelwriter.ExcelWriterStepMeta:

Branch Change :  + 2.56%
Complexity Change :  + 1.57%
Instruction Change :  + 1.26%
Line Change :  + 1.32%
Method Change :  + 1.14%

org.pentaho.di.trans.steps.execprocess.ExecProcessMeta:

Complexity Change :  + 2.94%
Instruction Change :  + 1.15%
Line Change :  + 1.77%
Method Change :  + 3.85%

org.pentaho.di.trans.steps.fieldschangesequence.FieldsChangeSequenceMeta:

Complexity Change :  + 2.70%
Instruction Change :  + 3.32%
Line Change :  + 4.20%
Method Change :  + 4.35%

org.pentaho.di.trans.steps.fieldsplitter.FieldSplitterMeta:

Complexity Change :  + 1.52%
Instruction Change :  + 8.84%
Line Change :  + 7.09%
Method Change :  + 2.04%

org.pentaho.di.trans.steps.fileexists.FileExistsMeta:

Complexity Change :  + 3.33%
Instruction Change :  + 1.36%
Line Change :  + 2.13%
Method Change :  + 4.17%

org.pentaho.di.trans.steps.filelocked.FileLockedMeta:

Complexity Change :  + 4.17%
Instruction Change :  + 1.78%
Line Change :  + 2.70%
Method Change :  + 5.00%

org.pentaho.di.trans.steps.filestoresult.FilesToResultMeta:

Complexity Change :  + 5.56%
Instruction Change :  + 3.36%
Line Change :  + 5.13%
Method Change :  + 5.88%

org.pentaho.di.trans.steps.flattener.FlattenerMeta:

Complexity Change :  + 3.70%
Instruction Change :  + 1.30%
Line Change :  + 2.63%
Method Change :  + 5.56%

org.pentaho.di.trans.steps.formula.FormulaMeta:

Branch Change :  + 8.82%
Complexity Change :  + 6.06%
Instruction Change :  + 9.04%
Line Change :  + 8.33%
Method Change :  + 6.25%

org.pentaho.di.trans.steps.formula.FormulaMetaFunction:

Complexity Change :  + 4.35%
Instruction Change :  + 1.76%
Line Change :  + 2.82%
Method Change :  + 4.76%

org.pentaho.di.trans.steps.getfilenames.GetFileNamesMeta:

Complexity Change :  + 1.22%
Instruction Change :  + 3.87%
Line Change :  + 3.23%
Method Change :  + 1.75%

org.pentaho.di.trans.steps.getrepositorynames.GetRepositoryNamesMeta:

Complexity Change :  + 2.63%
Instruction Change :  + 6.11%
Line Change :  + 4.85%
Method Change :  + 3.70%

org.pentaho.di.trans.steps.getslavesequence.GetSlaveSequenceMeta:

Complexity Change :  + 4.55%
Instruction Change :  + 1.74%
Line Change :  + 3.08%
Method Change :  + 4.76%

org.pentaho.di.trans.steps.getsubfolders.GetSubFoldersMeta:

Complexity Change :  + 1.96%
Instruction Change :  + 3.14%
Line Change :  + 3.19%
Method Change :  + 3.03%

org.pentaho.di.trans.steps.jobexecutor.JobExecutorMeta:

Complexity Change :  + .57%
Instruction Change :  + .20%
Line Change :  + .41%
Method Change :  + .96%

org.pentaho.di.trans.steps.mergejoin.MergeJoinMeta:

Complexity Change :  + 2.44%
Instruction Change :  + 4.91%
Line Change :  + 6.03%
Method Change :  + 4.00%

org.pentaho.di.trans.steps.mysqlbulkloader.MySQLBulkLoaderMeta:

Complexity Change :  + 1.06%
Instruction Change :  + 2.19%
Line Change :  + 2.24%
Method Change :  + 1.85%

org.pentaho.di.trans.steps.normaliser.NormaliserMeta:

Branch Change :  + 5.26%
Complexity Change :  + 4.76%
Instruction Change :  + 5.87%
Line Change :  + 5.76%
Method Change :  + 4.35%

org.pentaho.di.trans.steps.numberrange.NumberRangeMeta:

Complexity Change :  + 3.23%
Instruction Change :  + .95%
Line Change :  + 1.69%
Method Change :  + 4.17%

org.pentaho.di.trans.steps.olapinput.OlapInputMeta:

Complexity Change :  + 2.86%
Instruction Change :  + 1.25%
Line Change :  + 1.80%
Method Change :  + 3.57%

org.pentaho.di.trans.steps.randomccnumber.RandomCCNumberGeneratorMeta:

Complexity Change :  + 2.50%
Instruction Change :  + 5.43%
Line Change :  + 5.34%
Method Change :  + 3.57%

org.pentaho.di.trans.steps.randomvalue.RandomValueMeta:

Complexity Change :  + 2.17%
Instruction Change :  + 4.41%
Line Change :  + 5.66%
Method Change :  + 5.00%

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta:

Complexity Change :  + .96%
Instruction Change :  + 6.03%
Line Change :  + 5.16%
Method Change :  + 2.22%

org.pentaho.di.trans.steps.reservoirsampling.ReservoirSamplingMeta:

Complexity Change :  + 4.35%
Instruction Change :  + 2.69%
Line Change :  + 3.92%
Method Change :  + 6.25%

org.pentaho.di.trans.steps.selectvalues.SelectValuesMeta:

Branch Change :  + 1.55%
Complexity Change :  + 1.47%
Instruction Change :  + 2.10%
Line Change :  + 2.56%
Method Change :  + 2.56%

org.pentaho.di.trans.steps.selectvalues.SelectValuesMeta.SelectField:

Complexity Change :  + 10.00%
Instruction Change :  + 9.76%
Line Change :  + 6.25%
Method Change :  + 10.00%

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta:

Complexity Change :  + 2.86%
Instruction Change :  + 5.19%
Line Change :  + 5.94%
Method Change :  + 5.26%

org.pentaho.di.trans.steps.ssh.SSHMeta:

Complexity Change :  + 1.52%
Instruction Change :  + .55%
Line Change :  + .85%
Method Change :  + 2.00%

org.pentaho.di.trans.steps.switchcase.SwitchCaseMeta:

Branch Change :  + 3.85%
Complexity Change :  + 3.17%
Instruction Change :  + 3.17%
Line Change :  + 2.87%
Method Change :  + 2.70%

org.pentaho.di.trans.steps.switchcase.SwitchCaseTarget:

Complexity Change :  + 50.00%
Instruction Change :  + 50.00%
Line Change :  + 33.33%
Method Change :  + 50.00%

org.pentaho.di.trans.steps.symmetriccrypto.symmetriccryptotrans.SymmetricCryptoTransMeta:

Complexity Change :  + 1.85%
Instruction Change :  + .84%
Line Change :  + 1.38%
Method Change :  + 2.56%

org.pentaho.di.trans.steps.textfileinput.TextFileInputMeta:

Branch Change :  + .53%

org.pentaho.di.trans.steps.textfileoutput.TextFileField:

Complexity Change :  + 3.23%
Instruction Change :  + 2.84%
Line Change :  + 3.28%
Method Change :  + 3.23%

org.pentaho.di.trans.steps.textfileoutput.TextFileOutputMeta:

Branch Change :  + 1.69%
Complexity Change :  + 1.34%
Instruction Change :  + 1.15%
Line Change :  + 1.20%
Method Change :  + 1.11%

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta:

Complexity Change :  + 2.70%
Instruction Change :  + 4.90%
Line Change :  + 5.31%
Method Change :  + 3.70%

org.pentaho.di.trans.steps.uniquerowsbyhashset.UniqueRowsByHashSetMeta:

Complexity Change :  + 3.45%
Instruction Change :  + 5.16%
Line Change :  + 5.88%
Method Change :  + 4.35%

org.pentaho.di.trans.steps.univariatestats.UnivariateStatsMeta:

Branch Change :  + 7.14%
Complexity Change :  + 5.26%
Instruction Change :  + 6.77%
Line Change :  + 6.32%
Method Change :  + 5.88%

org.pentaho.di.trans.steps.jsoninput.JsonInput:

Branch Change :  + 2.04%
Complexity Change :  + 2.45%
Instruction Change :  + 3.60%
Line Change :  + 5.07%
Method Change :  + 4.35%

org.pentaho.di.trans.steps.jsoninput.JsonInputField:

Branch Change :  + 16.67%
Complexity Change :  + 25.00%
Instruction Change :  + 70.76%
Line Change :  + 55.36%
Method Change :  + 30.77%

org.pentaho.di.trans.steps.jsoninput.JsonInputMeta:

Branch Change :  + 37.93%
Complexity Change :  + 33.63%
Instruction Change :  + 65.61%
Line Change :  + 55.65%
Method Change :  + 33.33%

org.pentaho.di.trans.steps.jsoninput.JsonInputMeta.InputFiles:

Branch Change :  + 100.00%
Complexity Change :  + 50.00%
Instruction Change :  + 40.00%
Line Change :  + 27.78%
Method Change :  + 33.33%

Integration test coverage change:
org.pentaho.di.trans.step.BaseStep:

Branch Change :  + .30%
Complexity Change :  + .18%
Instruction Change :  + .05%
Line Change :  + .08%

org.pentaho.di.trans.steps.http.HTTPMeta:

Complexity Change :  + 1.47%
Instruction Change :  + 3.97%
Line Change :  + 3.90%
Method Change :  + 1.96%

org.pentaho.di.trans.steps.httppost.HTTPPOSTMeta:

Complexity Change :  + 1.33%
Instruction Change :  + 4.08%
Line Change :  + 4.28%
Method Change :  + 1.72%

org.pentaho.di.trans.steps.rest.RestMeta:

Complexity Change :  + .98%
Instruction Change :  + 4.01%
Line Change :  + 3.79%
Method Change :  + 1.52%

Checkstyle violations:
See: https://github.com/pentaho/pentaho-coding-standards
File /home/buildguy/project/engine/test-src/org/pentaho/di/trans/steps/loadsave/LoadSaveTester.java

Message: Using a static member import should be avoided - org.junit.Assert.assertNotSame. at Line: 25

File /home/buildguy/project/plugins/kettle-json-plugin/test-src/org/pentaho/di/trans/steps/jsoninput/JsonInputIntTest.java

Message: Using a static member import should be avoided - org.junit.Assert.assertNotNull. at Line: 25
Message: Using a static member import should be avoided - org.junit.Assert.assertTrue. at Line: 26

Tested the changes locally.
Checkstyle violations contain only legal static imports.
Merging.


ISPN-7526 Write skew check throws even if the previous value was not read

https://issues.jboss.org/browse/ISPN-7526

@rvansa I took a look at the other PR. I guess I can close this one?

@pruivo Okay, closing.


Allow underscores as unused variable identifiers

Closes #319

[x] I have updated CHANGELOG.md
[x] I have added tests to demonstrate the contribution is correctly implemented: this usually includes both positive and negative tests, showing the happy path(s) and featuring intentionally broken cases
[x] I have run all the tests locally and no test failure was reported
[x] I did not do unrelated and/or undiscussed refactorings

As of now, it only works for foreach and catch, right?

As of now, it only works for foreach and catch, right?

Are there any other cases where we need it?

Are there any other cases where we need it?
I don't think so, at least now. When we'll have #298 in the future, identifiers in let declarations would them though.

When we'll have https://github.com/tact-lang/tact/issues/298 in the future, identifiers in let declarations would need them though.

Actually, let-statements should have underscores even now. This can be used as a clear indication that some functions gets called not for its return value but for its side-effects:
let _ : Foo = effectfulFun(x, y, z);

When we have a bit of type inference, the snippet above will become
let _ = effectfulFun(x, y, z);

In this case a static analyzer won't complain about unused values, whereas it should issue a warning for
effectfulFun(x, y, z);

if effectfulFun is a non-void function.

Added support for let-statements and also resolved all merge conflicts

resolved merge conflicts

@anton-trunov resolved merge conflicts yet again after merge of #198

@anton-trunov resolved everything


Variational Expectation of Poisson

Hi,
I was wondering where the analytical form of the variational Expectation
\int q(f)logp(y|f) df for the Poisson Likelihood comes from (when using a log link).
In the documentation it is only mentioned that one can approximate the intractable likelihood,
however I'm not familiar enough with the topic to know what exactly is used.
    let g(.) be the inverse-link function, then this likelihood represents

    p(y_i | f_i) = Poisson(y_i | g(f_i) * binsize)

    Note:binsize
    For use in a Log Gaussian Cox process (doubly stochastic model) where the
    rate function of an inhomogeneous Poisson process is given by a GP.  The
    intractable likelihood can be approximated by gridding the space (into bins
    of size 'binsize') and using this Poisson likelihood.

Could you maybe point me to that approximation method?

@FabricioArendTorres this is a question more suited to stackoverflow, please repost it there under the gpflow tag (https://stackoverflow.com/tags/gpflow) and I'll be happy to answer there.

For future reference, answer is at:
https://stackoverflow.com/questions/56423927/variational-expectation-of-poisson


Check if task is parked before parking thread.

So, I'm worried about a race condition in future's like Join when one of the sub-futures calls thread::park() internally. Basically, what if we have the following sequence of events:

Someone calls spawn(future).wait_future() where future is a Join of two sub-futures.
Sub-future 1 parks the task and schedules it to be unparked.
Sub-future 1 returns NotReady.
An event happens and the task is unparked. This will call thread::unpark on the current thread.
Future 2 communicates with another thread and, in the process, some internal method calls Thread::park. This isn't really the "correct" way to communicate in an async environment but I can see some library doing this if it knows that it's thread will be unparked really quickly.
The Join returns NotReady.
Spawn's wait_future method calls thread::park and waits indefinitely because step 4 cleared the park flag.

A fix for this is to set a task_is_parked flag, set/unset it from within task::park()/Task::unpark() and check this before calling thread::park(). This is still racy but the race is safe: If task_is_parked is unset after we check it but before we call thread::park(), we know that no user-specified code could have called thread::park() and swallowed our unpark event in the interim. This also protects us from user code that returns NotReady without parking.
Is this actually a problem? I haven't tested it (writing concurrent tests like this is not fun) but I can't see how the current code could prevent this.

Thanks for the report! This is something thought that I believe is handled by thread::park and such. A call to thread::current doesn't clear any flags or actually block the current thread, and thread::park itself isn't called until a call to poll is finished entirely.
In that sense if a call to Thread::unpark comes it'll just prevent the later thread::park from blocking. That is, in step 4 above the flag I believe isn't cleared.

The problem is that the internal thread::park eats that flag.

We tell some scheduler to unpark us when ready.
Our scheduler unparks us.
Some user-specified code (on our thread) calls while !some_atomic_condition() { thread::park() }.
Eventually some other user-specified code (not on our thread) sets the condition and unparks our thread.
We call thread::park() and wait forever because that park loop above ate our flag.

Hm are you basically saying that if futures arbitrarily call thread::park that'll wreak havoc? I believe that will, but it also seems counter to "a poll implementation should return quickly"

My point is that a small, harmless delay (a user spinning up a thread at the beginning of the program) could turn into a deadlock.
On September 9, 2016 3:31:47 AM EDT, Alex Crichton<EMAIL_ADDRESS>wrote:

Hm are you basically saying that if futures arbitrarily call
thread::park that'll wreak havoc? I believe that will, but it also
seems counter to "a poll implementation should return quickly"
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
https://github.com/alexcrichton/futures-rs/issues/132#issuecomment-245840791

--
Steven Allen

Hm I'm not sure I quite follow that part, could you clarify with a code example of what you're thinking? I do agree that thread::park during a call to poll would cause problems, but I'm not sure how surrounding code outside a poll would cause problems.

I'm talking about a call to thread::park during a call to poll. A better, concrete example is the following:
lazy_static!(static ref LOGGER: MyLogger = MyLogger::new());
// ...
my_future.and_then(|x| {
    my_logger.log(x);
})
// ...
my_future2.and_then(|x| {
    my_logger.log(x);
})

If both of these try to log from two different threads at the very beginning of the program (before the logger has been initialized), once of them will synchronize using thread::park() (internally, lazy_static uses sync::Once which calls thread::park). This will only happen early on in the execution of the program so from an efficiency standpoint, it's not an issue. However, a developer will definitely not expect their program to deadlock here and will have a vary hard time figuring out why it's deadlocking.

Oh dear, that does indeed sound bad! Ok if we're only worked about park during a poll, though, we may be able to just implement wait differently with our own version of park and unpark. That way we can guarantee that all calls to park are our own.

Oh @carllerche pointed out that we just need an AtomicUsize here actually, I'll just do that!

To be fair, this is also what @Stebalien suggested in his original report :)

Gah sorry I'm bad at reading recently :(

I'm coming back to read this issue from that linked PR (https://github.com/alexcrichton/futures-rs/pull/566). @alexcrichton could you clarify these parts for me:

Sub-future 1 parks the task and schedules it to be unparked.

This also protects us from user code that returns NotReady without parking.

Do these sentences refer to an old way that the futures library used to work, which no longer applies? The way the futures library currently works, the only park function available to user code is std::thread::park, which most futures should never call, yes?

Nah I think everything is roughtly the same there's just been a few renamings. It's rare, yes, to call std::thread::park but also perhaps not unheard of

I wouldn't even say it is "rare". It is completely legitimate to use locks w/ small critical sections in a future. For example, futures::sync::mpsc does this. Which also means that it is perfectly fine to call std::thread::park (and potentially even "common").


Provide a prebuilt PDF book in the releases

It would be very desirable that you provide a prebuilt PDF version of the book. When you released 0.3.x, you did. But in the latest 0.4.0, you didn't. I belong to the "low on dependencies" trend, so getting the PDF without having to check if I have all the software for building it, would be really nice.

yes this would be more convenient since i just had to change path to my xsl stylesheets location because the CopyWebsite.lua did not detect it correctly


Add AES-CTR and AES-CBC support

See RFC9459 for the detail.
added

AESCTR and AESCBC (refer to here)
AESCTRKey and AESCBCKey
TestAESCTRKey and TestAESCBCKey

test case are almost the same as TestAESGCMKey
but test_aescbc_padding_with_aes_cbc_a128cbc() is added for padding value and length check

I want to add ESDH-ES + AES-KW tests in another PR.


Async gRPC service under envoy. A lot of concurrent requests

I have async gRPC service written on Python
My envoy.yaml:
    static_resources:
      listeners:
      - reuse_port: true
        address:
          socket_address:
            address: <IP_ADDRESS>
            port_value: 50051
        filter_chains:
        - filters:
          - name: envoy.http_connection_manager
            config:
              access_log:
              - name: envoy.file_access_log
                config:
                  path: "/dev/stdout"
              codec_type: AUTO
              stat_prefix: ingress_https
              route_config:
                name: local_route
                virtual_hosts:
                - name: https
                  domains:
                  - "*"
                  routes:
                  - match:
                      prefix: "/"
                    route:
                      cluster: my-service-grpc
              http_filters:
              - name: envoy.health_check
                config:
                  pass_through_mode: false
                  headers:
                  - name: ":path"
                    exact_match: "/healthz"
                  - name: "x-envoy-livenessprobe"
                    exact_match: "healthz"
              - name: envoy.router
                config: {}
      clusters:
      - name: my-service-grpc
        connect_timeout: 0.5s
        type: STRICT_DNS
        dns_lookup_family: V4_ONLY
        lb_policy: ROUND_ROBIN
        http2_protocol_options:
          max_concurrent_streams: 10
        load_assignment:
          cluster_name: my-service-grpc
          endpoints:
          - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: my-service-headless.utils.svc.cluster.local
                    port_value: 9000
        health_checks:
          timeout: 1s
          interval: 10s
          unhealthy_threshold: 2
          healthy_threshold: 2
          grpc_health_check: {}
        health_checks:
          timeout: 1s
          interval: 5s
          unhealthy_threshold: 2
          healthy_threshold: 2
          grpc_health_check: {}
        circuit_breakers:
          thresholds:
            - priority: default
              max_pending_requests: 10000
              max_requests: 10000
              max_connections: 10000
            - priority: high
              max_pending_requests: 10000
              max_requests: 10000
              max_connections: 10000
    admin:
      access_log_path: "/dev/stdout"
      address:
        socket_address:
          address: <IP_ADDRESS>
          port_value: 8090

As I understand properly, envoy will send request to host if it sees any available open socket(because it's async gRPC it doens't use pending requests queue, because async socket could keep number connection, not only one as single-threaded sync gRPC server), so I tried to make 200rps and I got very high latency for N requests -> overflow for N+X requests -> timeout for N + X + Y requests
My question is how to prevent a lot of concurrent requests(concurrent tasks in my python event loop) and put all requests which are over restriction to some queue?

Also, I have to say that I tried to use example configuration of adaptive concurrency and I got exception that concurrency limit is reached

Guys, can anybody help me please? :(


Come up with Feedback session facilitation structure and discuss ways of giving good feedback

Context
Given that our team has grown in size, it is a good time to review our speedback practice, share helpful tips, and decide on cadence.
Goal: Create a template to be used by team/s
Why are speedback sessions useful?

encourage new members of the team by delivering affirming feedback
they can be an opportunity to share your personal goals with your teammates. If your teammates have visibility into your growth goals, they can better support you by noticing actions you do on the team that are in alignment with your goals.

For example, if you are looking to become a better presenter, you can share this during speedback. Then if you volunteer to present/facilitate a meeting, the team member you shared this with can take notes and provide you with feedback after, in the next speedback session, in a summit feedback request, etc.

allows us to enable and empower our teammates

Questions
Do we want/need a rotation for speedback?
References

Feedback Playbook
Speedback 101
How to Give Feedback

Tasks

[x] Send meeting invitation for May 9

[x] Add agenda
[ ]  add link/s to useful artifacts

[x] create google sheet with entire team giving every member of the team feedback (for now)

[ ] Ask team: do we split into 2 groups or 1 group?

[ ]  create google doc with feedback/speedback best practices
[ ] add to team resources column on trello
[ ] Prepare speedback 101 learning session a few days before first speedback session

[ ] send out meeting invitation

Started a doc: Speedback: A How To Guide for Facilitators and Participants

Notified team on slack with more information, including the spreadsheet that contains the pairs for each round.


Python jobs fail with large DNS request - Could not resolve hostname

Requirement - what kind of business use case are you trying to solve?

When using Jaeger with Elasticsearch (ES)
When the ES endpoint is a DNS entry containing all the data nodes
When the number of data nodes > 'large' (i.e. ~20ish)
-- And therefore have a DNS request with a response > 512bytes

DNS requests will fail on the components:

Rollover job
Init jobs
Cleanup jobs

This is because the DNS request is truncated. DNS uses UDP, yet when it reaches > 512 bytes it gets a state 'TC' which means it will/want to retry on TCP.
Problem - what in Jaeger blocks you from solving the requirement?
It seems these python components cannot handle a DNS truncate / retry
Proposal - what do you suggest to solve the problem or improve the existing situation?
Implement edns0
Workaround
Currently we have altered our CoreDNS setup to use 'prefer_udp' - this is "fine"
IMO a nicer solution would be to use edns0. Also I'm aware of the possibility to not use these jobs, or at least for the cleaner/rollover - via lifecycle policies in ES.
I still wanted to make this issue for others and perhaps an improvement in Jaeger :)

Not knowing much about edns0, would it be a relatively straightforward change to adopt it?
Are you aware of any shortcomings using it?

I'm not that known with python but the implementation would simply be adding an extra argument; https://www.dnspython.org/docs/1.14.0/dns.resolver.Resolver-class.html#use_edns
edns is defined in https://tools.ietf.org/html/rfc6891
The shortcoming might be that some odd and old resolver cannot handle it. edns should be backwards compatible though. I've checked the code in for example Thanos and it's also using edns for resolving.
Reason I've made an issue is that, however I'm aware of edns, I'm not an expert on DNS (who is :P )


Add Chinese wikipedia collection in sentence level

TODO:

add paragraph and doc level indexer
solve the Chinese segmentation problem
a. combine different analyzer
b. processs before indexing

@Peilin-Yang Your comment fixed.

Hi @Victor0118 I'm not sure why you need a new class WikipediaZHSentenceCollection - why isn't the existing Json collection sufficient?
Also, Anserini has moved forward quite quickly over the last few days... our upgrade to Lucene 7 changed a lot. It might be easier for you to start over with the new HEAD and reapply your changes?

@lintool There are some minor reasons for the new collection:

The output of wikiextracter is a binary file (and I haven't found the word to generate a text file). So I need FileStream to read it.
I index the Wikipedia in sentence level and split a document using 。. JsonCollection can only do indexing in document level. In the future, we can add the function of indexing Wikipedia-paragraph to this collection (just change a splitter).

You are right. I need to redo the PR.


 The trace_cards function is rewritten so that you get rid of the goto. And I've also replaced some bitshifts with easier to read multiplications and divisions. It is an optimization that doesn't make sense anymore because these days it is as fast or even faster to multiply or divide instead of bit shifting.

Merged, thanks!


Fix for usernames with capital letters

Good catch LGTM

Coverage remained the same at 94.313% when pulling 7fc366e4ca45cbbdf2c1ab4a476af40ba88a9ae8 on btoueg:patch-1 into 972c670879b492b7e794ef5431841c0af7a50d9d on kencochrane:master.


[Sema] Diagnose ambiguous for RValue to LValue - pick overload choice that is settable

With multiple overload choices, TreatRValueAsLValue used to just pick the first solution. We get better diagnostics if we pick a solution where the chosen VarDecl is settable, if there is one.
I.e. if you conform to two protocols with same-named property, one of which is {get} and the other is {get set} and both possibilities end up with the same fix in the same location, it isn't much help to use the "it's a get-only property" choice.
Resolves #46265

@swift-ci Please smoke test.

Thanks for all the work you have done recently, I will try to take a look at other PRs asap!

I think it would be better to reflect this in the score at the point where the fix is recorded instead of having to retroactively rank the fixes, this would mean that we can prune solutions more effectively.

Makes sense! I'll look at that.


newMove foresight

At the moment, newMove operates by simply moving to the nearest exit to the next room on the route. This will generally be less efficient motion-wise because the creep will not necessarily take fastest path to it's overall destination.
At the cost of cpu, it might be useful to allow the pathfinder to target a little farther ahead in the path.
CAUTION: if we give creeps the option to move to rooms NOT on the designated route, they may get stuck on exits due to the vision inconsistencies.

Green = preferred route
Red = route taken

This is also a side effect of making highways cost half as much as regular rooms in findRoute. backRoadPenalty decreased to 1.5 accordingly.

how about making it scan 2 rooms instead of 1? (total of 5, NSEW and current)

Read the caution statement. Specifically wrt novice walls and other structures that can inhibit motion but require room vision

we could have a function in getPath that can extrapolate novice walls to the center of neighboring rooms, based off of which exit tiles in the creep's room have walls on them

we can tell if a room has novice walls based on whether it is next to a novice room

Rooms neighboring creep’s room now get novice walls written in, so we should be able to open up pathfinding to rooms not on the route

The exception here would be that opposing players could still mess stuff up by making their own walls, but that adds so many edge cases it would be easier to treat that as a military problem.
That being said, when we do get around to assigning players threat levels, players who like to make lots of walls in their rooms should be a higher threat level

portal bug is caused by this, so I'll leave this issue open until we can confirm that the portal issue is solved


Allow CONFIG_BT_NIMBLE_MAX_CONNECTIONS to be defined by project

CONFIG_BT_NIMBLE_MAX_CONNECTIONS is defined solely in the nimconfig.h header of the library. This requires a change to the library file to increase the number of connections. To make it more portable, I would like to define this macro in the bluild_flags of my PlatformIO project, like so:
build_flags =
    -D CONFIG_NIMBLE_CPP_ENABLE_RETURN_CODE_TEXT
    -D CONFIG_BT_NIMBLE_MAX_CONNECTIONS=9

This currently generates a compiler warning, because it's already defined:
.pio/libdeps/lolind32pro-lolintft24/NimBLE-Arduino/src/nimconfig.h:105:0: warning: "CONFIG_BT_NIMBLE_MAX_CONNECTIONS"
redefined #define CONFIG_BT_NIMBLE_MAX_CONNECTIONS 3

It would be nice if the #define in the nimconfig.h header would be enclosed by #ifndef checks:
#ifndef CONFIG_BT_NIMBLE_MAX_CONNECTIONS
#define CONFIG_BT_NIMBLE_MAX_CONNECTIONS 3
#endif

This way, the macro can be set in the project without the need to touch any of the library files. This is especially handy when using the platformio library version of NimBLE-Arduino. The change will be transparent for users that don't use build flags.

There might be some confusion around the usage of MYNEWT_VAL_BLE_MAX_CONNECTIONS which is overriding CONFIG_BT_NIMBLE_MAX_CONNECTIONS in a few locations

@fvanroie thank you for the suggestion, this is a good idea.
@renatobo MYNEWT_VAL_BLE_MAX_CONNECTIONS is defined as the CONFIG_BT_NIMBLE_MAX_CONNECTIONS value. Unless you've seen an instance where it is not I don't see any issue. Could you elaborate?

I could not get more clients to connect using MYNEWT_VAL_BLE_MAX_CONNECTIONS.
The only workaround I have found is to specify a CONFIG_BT_NIMBLE_TASK_STACK_SIZE to circumvent the Arduino portion in the nimconfig.h file. This also requires all 30-something macros to be defined in my project, and not just CONFIG_BT_NIMBLE_MAX_CONNECTIONS. Using #ifndef it would be very easy to specify from outside the library.
For my project I only want to increase the number of clients, but maybe other settings could benefit from being configurable using preprocessor flags too...

Yes, nimconfig.h was designed such that Arduino users could set the config settings and still have compatibility with the NimBLE stack in IDF which has the defines already in sdkconfig. It would make sense to support platformio in a similar fashion.
MYNEWT_VAL_BLE_MAX_CONNECTIONS doesn't work for the C++ wrapper because the code does not check that value, only CONFIG_BT_NIMBLE_MAX_CONNECTIONS.
Ideally all the settings could be configured in this way, I'm not sure how valuable that would be though. It would start making the nimconfig.h a bit messy.

Maybe it's best to start with the few ones that make sense, based on the likelihood they need to be changed from the defaults.
While max connections is an obvious one, I'm not even sure what most the other ones do... so that one would suffice for me.
Some macros are commented by default, so they can already be set using flags when needed without touching the file.

@renatobo MYNEWT_VAL_BLE_MAX_CONNECTIONS is defined as the CONFIG_BT_NIMBLE_MAX_CONNECTIONS value. Unless you've seen an instance where it is not I don't see any issue. Could you elaborate?

Your comment 'MYNEWT_VAL_BLE_MAX_CONNECTIONS doesn't work for the C++ wrapper' clarified thanks!
I suggest allowing external defines for the following values as they can help reducing flash usage

CONFIG_BT_NIMBLE_ROLE_CENTRAL (Client)
CONFIG_BT_NIMBLE_ROLE_PERIPHERAL (Server)

while

CONFIG_BT_NIMBLE_ROLE_BROADCASTER (Advertising)
CONFIG_BT_NIMBLE_ROLE_OBSERVER (Scan)
are not required as they are later defined if the previous two (Client, Server role) are defined.

@renatobo Thank you for that suggestion as well. It makes sense to put conditionals around those as well.

Just pushed the changes. Please let me know if I missed any.

Thanks, this handles my use case for setting CONFIG_BT_NIMBLE_MAX_CONNECTIONS .
For disabling a role that is set be default, I think the logic needs to be reversed or turned into an 0/1 value instead.
For example these roles are always defined, whether or not you define them in your project:
/** @brief Comment out if not using NimBLE Client functions \n
 *  Reduces flash size by approx. 7kB.
 */
#ifndef CONFIG_BT_NIMBLE_ROLE_CENTRAL
#define CONFIG_BT_NIMBLE_ROLE_CENTRAL
#endif

If the goal is to allow it to be disabled in the project, I would suggest something like:
#ifndef CONFIG_BT_NIMBLE_ROLE_CENTRAL
#define CONFIG_BT_NIMBLE_ROLE_CENTRAL 1
#endif

Then checking for CONFIG_BT_NIMBLE_ROLE_CENTRAL ==1 to actually enable it.

Haha yes, you're right, wasn't really thinking when I made that change. Thanks!

I ended up just renaming the macros for the command line by appending _DISABLED. If I had values set instead I would have to make many changes to the core files.

Indeed, that should work too and keeps the rest of the flow in line.

Closing this as resolved.