[Flaky] Integration tests - Using cohorts for sharing unused resources

What type of PR is this?
/kind flake

What this PR does / why we need it:
Make sure previous workload has finished so cq has enough resources for another workload.
To avoid flaky test.
Which issue(s) this PR fixes:

Fixes #3633
Special notes for your reviewer:
Does this PR introduce a user-facing change?

NONE

/ok-to-test

Comparison of successful and failed test run:
Success
STEP: Finishing second workload for prod ClusterQueue @ 12/05/24 12:33:30.03
  2024-12-05T12:33:30.039455+01:00      LEVEL(-2)       workload-reconciler     core/workload_controller.go:681 Workload update event   {"workload": {"name":"p-wl-2","namespace":"core-cxq98"}, "queue": "prod-queue", "status": "finished", "prevStatus": "admitted", "prevClusterQueue": "prod-cq"}
  2024-12-05T12:33:30.039603+01:00      LEVEL(-2)       workload-reconciler     queue/manager.go:489    Attempting to move workloads    {"workload": {"name":"p-wl-2","namespace":"core-cxq98"}, "queue": "prod-queue", "status": "finished", "cohort": "all", "root": "all"}
  2024-12-05T12:33:30.039752+01:00      LEVEL(-2)       core/workload_controller.go:143 Reconciling Workload    {"controller": "workload", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "Workload", "Workload": {"name":"p-wl-2","namespace":"core-cxq98"}, "namespace": "core-cxq98", "name": "p-wl-2", "reconcileID": "553dd96a-3023-481d-9a3d-da60c21fd094", "workload": {"name":"p-wl-2","namespace":"core-cxq98"}}
  2024-12-05T12:33:30.110434+01:00      LEVEL(-3)       scheduler       queue/manager.go:531    Obtained ClusterQueue heads     {"attemptCount": 13, "count": 2}
  2024-12-05T12:33:30.110767+01:00      LEVEL(-2)       scheduler       scheduler/scheduler.go:543      Workload assumed in the cache   {"attemptCount": 13, "workload": {"name":"p-wl-3","namespace":"core-cxq98"}, "clusterQueue": {"name":"prod-cq"}}
  2024-12-05T12:33:30.110836+01:00      LEVEL(-3)       scheduler       scheduler/logging.go:43 Workload evaluated for admission        {"attemptCount": 13, "workload": {"name":"p-wl-3","namespace":"core-cxq98"}, "clusterQueue": {"name":"prod-cq"}, "status": "assumed", "reason": ""}
  2024-12-05T12:33:30.110891+01:00      LEVEL(-3)       scheduler       scheduler/logging.go:43 Workload evaluated for admission        {"attemptCount": 13, "workload": {"name":"d-wl-1","namespace":"core-cxq98"}, "clusterQueue": {"name":"dev-cq"}, "status": "skipped", "reason": "Workload no longer fits after processing another workload"}
  2024-12-05T12:33:30.110982+01:00      LEVEL(-2)       scheduler       scheduler/scheduler.go:636      Workload re-queued      {"attemptCount": 13, "workload": {"name":"d-wl-1","namespace":"core-cxq98"}, "clusterQueue": {"name":"dev-cq"}, "queue": {"name":"dev-queue","namespace":"core-cxq98"}, "requeueReason": "FailedAfterNomination", "added": true, "status": "skipped"}

Fail
  STEP: Finishing second workload for prod ClusterQueue @ 12/05/24 12:33:46.336
  2024-12-05T12:33:46.342581+01:00      LEVEL(-3)       scheduler       queue/manager.go:531    Obtained ClusterQueue heads     {"attemptCount": 12, "count": 2}
  2024-12-05T12:33:46.342903+01:00      LEVEL(-2)       scheduler       scheduler/scheduler.go:543      Workload assumed in the cache   {"attemptCount": 12, "workload": {"name":"d-wl-1","namespace":"core-c628d"}, "clusterQueue": {"name":"dev-cq"}}
  2024-12-05T12:33:46.342956+01:00      LEVEL(-2)       scheduler       scheduler/scheduler.go:228      Workload requires preemption, but there are no candidate workloads allowed for preemption   {"attemptCount": 12, "workload": {"name":"p-wl-3","namespace":"core-c628d"}, "clusterQueue": {"name":"prod-cq"}, "preemption": {"reclaimWithinCohort":"Never","borrowWithinCohort":{"policy":"Never"},"withinClusterQueue":"Never"}}
  2024-12-05T12:33:46.34301+01:00       LEVEL(-3)       scheduler       scheduler/logging.go:43 Workload evaluated for admission        {"attemptCount": 12, "workload": {"name":"d-wl-1","namespace":"core-c628d"}, "clusterQueue": {"name":"dev-cq"}, "status": "assumed", "reason": ""}
  2024-12-05T12:33:46.343057+01:00      LEVEL(-3)       scheduler       scheduler/logging.go:43 Workload evaluated for admission        {"attemptCount": 12, "workload": {"name":"p-wl-3","namespace":"core-c628d"}, "clusterQueue": {"name":"prod-cq"}, "status": "", "reason": "couldn't assign flavors to pod set main: insufficient unused quota for cpu in flavor on-demand, 1 more needed"}
  2024-12-05T12:33:46.343146+01:00      LEVEL(-2)       scheduler       scheduler/scheduler.go:636      Workload re-queued      {"attemptCount": 12, "workload": {"name":"p-wl-3","namespace":"core-c628d"}, "clusterQueue": {"name":"prod-cq"}, "queue": {"name":"prod-queue","namespace":"core-c628d"}, "requeueReason": "", "added": true, "status": ""}
  2024-12-05T12:33:46.344572+01:00      LEVEL(-2)       workload-reconciler     core/workload_controller.go:681 Workload update event   {"workload": {"name":"p-wl-2","namespace":"core-c628d"}, "queue": "prod-queue", "status": "finished", "prevStatus": "admitted", "prevClusterQueue": "prod-cq"}
  2024-12-05T12:33:46.344729+01:00      LEVEL(-2)       workload-reconciler     queue/manager.go:489    Attempting to move workloads    {"workload": {"name":"p-wl-2","namespace":"core-c628d"}, "queue": "prod-queue", "status": "finished", "cohort": "all", "root": "all"}
  2024-12-05T12:33:46.344934+01:00      LEVEL(-2)       core/workload_controller.go:143 Reconciling Workload    {"controller": "workload", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "Workload", "Workload": {"name":"p-wl-2","namespace":"core-c628d"}, "namespace": "core-c628d", "name": "p-wl-2", "reconcileID": "ae2e1b24-84a8-481c-88f4-de04deb9db09", "workload": {"name":"p-wl-2","namespace":"core-c628d"}}

What we can see here (this is all without the change) is that:
Successful run starts with p-wl-2 going from admitted to finish.
Then workloads are evaluated for admission and p-wl-3 is assumed.
In failed run first workloads are evaluated fro admission and d-wl-1 gets assumed...
So the order is reversed, and d-wl-1 is assumed bc there is enough resources for it (1 CPU released already from p-wl-1).

Comparison of successful and failed test run:
Success
STEP: Finishing second workload for prod ClusterQueue @ 12/05/24 12:33:30.03
  2024-12-05T12:33:30.039455+01:00      LEVEL(-2)       workload-reconciler     core/workload_controller.go:681 Workload update event   {"workload": {"name":"p-wl-2","namespace":"core-cxq98"}, "queue": "prod-queue", "status": "finished", "prevStatus": "admitted", "prevClusterQueue": "prod-cq"}
  2024-12-05T12:33:30.039603+01:00      LEVEL(-2)       workload-reconciler     queue/manager.go:489    Attempting to move workloads    {"workload": {"name":"p-wl-2","namespace":"core-cxq98"}, "queue": "prod-queue", "status": "finished", "cohort": "all", "root": "all"}
  2024-12-05T12:33:30.039752+01:00      LEVEL(-2)       core/workload_controller.go:143 Reconciling Workload    {"controller": "workload", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "Workload", "Workload": {"name":"p-wl-2","namespace":"core-cxq98"}, "namespace": "core-cxq98", "name": "p-wl-2", "reconcileID": "553dd96a-3023-481d-9a3d-da60c21fd094", "workload": {"name":"p-wl-2","namespace":"core-cxq98"}}
  2024-12-05T12:33:30.110434+01:00      LEVEL(-3)       scheduler       queue/manager.go:531    Obtained ClusterQueue heads     {"attemptCount": 13, "count": 2}
  2024-12-05T12:33:30.110767+01:00      LEVEL(-2)       scheduler       scheduler/scheduler.go:543      Workload assumed in the cache   {"attemptCount": 13, "workload": {"name":"p-wl-3","namespace":"core-cxq98"}, "clusterQueue": {"name":"prod-cq"}}
  2024-12-05T12:33:30.110836+01:00      LEVEL(-3)       scheduler       scheduler/logging.go:43 Workload evaluated for admission        {"attemptCount": 13, "workload": {"name":"p-wl-3","namespace":"core-cxq98"}, "clusterQueue": {"name":"prod-cq"}, "status": "assumed", "reason": ""}
  2024-12-05T12:33:30.110891+01:00      LEVEL(-3)       scheduler       scheduler/logging.go:43 Workload evaluated for admission        {"attemptCount": 13, "workload": {"name":"d-wl-1","namespace":"core-cxq98"}, "clusterQueue": {"name":"dev-cq"}, "status": "skipped", "reason": "Workload no longer fits after processing another workload"}
  2024-12-05T12:33:30.110982+01:00      LEVEL(-2)       scheduler       scheduler/scheduler.go:636      Workload re-queued      {"attemptCount": 13, "workload": {"name":"d-wl-1","namespace":"core-cxq98"}, "clusterQueue": {"name":"dev-cq"}, "queue": {"name":"dev-queue","namespace":"core-cxq98"}, "requeueReason": "FailedAfterNomination", "added": true, "status": "skipped"}

Fail
  STEP: Finishing second workload for prod ClusterQueue @ 12/05/24 12:33:46.336
  2024-12-05T12:33:46.342581+01:00      LEVEL(-3)       scheduler       queue/manager.go:531    Obtained ClusterQueue heads     {"attemptCount": 12, "count": 2}
  2024-12-05T12:33:46.342903+01:00      LEVEL(-2)       scheduler       scheduler/scheduler.go:543      Workload assumed in the cache   {"attemptCount": 12, "workload": {"name":"d-wl-1","namespace":"core-c628d"}, "clusterQueue": {"name":"dev-cq"}}
  2024-12-05T12:33:46.342956+01:00      LEVEL(-2)       scheduler       scheduler/scheduler.go:228      Workload requires preemption, but there are no candidate workloads allowed for preemption   {"attemptCount": 12, "workload": {"name":"p-wl-3","namespace":"core-c628d"}, "clusterQueue": {"name":"prod-cq"}, "preemption": {"reclaimWithinCohort":"Never","borrowWithinCohort":{"policy":"Never"},"withinClusterQueue":"Never"}}
  2024-12-05T12:33:46.34301+01:00       LEVEL(-3)       scheduler       scheduler/logging.go:43 Workload evaluated for admission        {"attemptCount": 12, "workload": {"name":"d-wl-1","namespace":"core-c628d"}, "clusterQueue": {"name":"dev-cq"}, "status": "assumed", "reason": ""}
  2024-12-05T12:33:46.343057+01:00      LEVEL(-3)       scheduler       scheduler/logging.go:43 Workload evaluated for admission        {"attemptCount": 12, "workload": {"name":"p-wl-3","namespace":"core-c628d"}, "clusterQueue": {"name":"prod-cq"}, "status": "", "reason": "couldn't assign flavors to pod set main: insufficient unused quota for cpu in flavor on-demand, 1 more needed"}
  2024-12-05T12:33:46.343146+01:00      LEVEL(-2)       scheduler       scheduler/scheduler.go:636      Workload re-queued      {"attemptCount": 12, "workload": {"name":"p-wl-3","namespace":"core-c628d"}, "clusterQueue": {"name":"prod-cq"}, "queue": {"name":"prod-queue","namespace":"core-c628d"}, "requeueReason": "", "added": true, "status": ""}
  2024-12-05T12:33:46.344572+01:00      LEVEL(-2)       workload-reconciler     core/workload_controller.go:681 Workload update event   {"workload": {"name":"p-wl-2","namespace":"core-c628d"}, "queue": "prod-queue", "status": "finished", "prevStatus": "admitted", "prevClusterQueue": "prod-cq"}
  2024-12-05T12:33:46.344729+01:00      LEVEL(-2)       workload-reconciler     queue/manager.go:489    Attempting to move workloads    {"workload": {"name":"p-wl-2","namespace":"core-c628d"}, "queue": "prod-queue", "status": "finished", "cohort": "all", "root": "all"}
  2024-12-05T12:33:46.344934+01:00      LEVEL(-2)       core/workload_controller.go:143 Reconciling Workload    {"controller": "workload", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "Workload", "Workload": {"name":"p-wl-2","namespace":"core-c628d"}, "namespace": "core-c628d", "name": "p-wl-2", "reconcileID": "ae2e1b24-84a8-481c-88f4-de04deb9db09", "workload": {"name":"p-wl-2","namespace":"core-c628d"}}

What we can see here (this is all without the change) is that: Successful run starts with p-wl-2 going from admitted to finish. Then workloads are evaluated for admission and p-wl-3 is assumed.
In failed run first workloads are evaluated fro admission and d-wl-1 gets assumed... So the order is reversed, and d-wl-1 is assumed bc there is enough resources for it (1 CPU released already from p-wl-1).

I would like to discuss the name of the test:
Should start workloads that are under min quota before borrowing
I don't see the special setup with borrowing or lending quotas here, maybe that could help here, but now it's just how it works.
wdyt @mbobrovskyi ?

cc @gabesaba

What we can see here (this is all without the change) is that: Successful run starts with p-wl-2 going from admitted to finish. Then workloads are evaluated for admission and p-wl-3 is assumed.
In failed run first workloads are evaluated fro admission and d-wl-1 gets assumed... So the order is reversed, and d-wl-1 is assumed bc there is enough resources for it (1 CPU released already from p-wl-1).

I would like to discuss the name of the test: Should start workloads that are under min quota before borrowing
I don't see the special setup with borrowing or lending quotas here, maybe that could help here, but now it's just how it works. wdyt @mbobrovskyi ?

Yeah, that what I thought. dWL1 admitted before pWL3. for some reason.

/cherry-pick release-0.9
To make the release branch more stable

/cherry-pick release-0.9


go/lint: update gitleaks to v8.2.0

Codecov Report

Merging #237 (12478ca) into master (09cb5d0) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #237   +/-   ##
=======================================
  Coverage   78.37%   78.37%
=======================================
  Files           1        1
  Lines          37       37
=======================================
  Hits           29       29
  Misses          4        4
  Partials        4        4

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 09cb5d0...12478ca. Read the comment docs.


How to generate plots for driving times?

I saw in the readme that you made some plots / graphs
like: https://github.com/jsschl/ev_mobility_model/blob/main/img/placeofstayGER.png
This is not included in the code on github. How did you generate these? Could you upload the plotting file?
I am doing a master thesis on vehicle to grid charging of EV's simulations for 2030. I would like to use the graphs and make an export of the data and use this in my national scale renewable electricity system model.

Hi Mayk, thanks for your interest. I'm currently in holiday, I'll look into
the issue end of April/beginning of May.
Best Jonas
On Mon, 11 Apr 2022, 05:29 Mayk Thewessen, @.***> wrote:

I saw in the readme that you made some plots / graphs
like:
https://github.com/jsschl/ev_mobility_model/blob/main/img/placeofstayGER.png
This is not included in the code on github. How did you generate these?
Could you upload the plotting file?
I am doing a master thesis on vehicle to grid charging of EV's simulations
for 2030. I would like to use the graphs and make an export of the data and
use this in my national scale renewable electricity system model.
—
Reply to this email directly, view it on GitHub
https://github.com/jsschl/ev_mobility_model/issues/1, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHEQWQNIVAF4H3UXCISBFELVEQLI5ANCNFSM5TDA35EA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Golang Scalability Problem With Interface Type Transformation

I’m working on multi-core systems with golang. I find that there exist a scalability problem when interface type transformation involved.
The test program consists of several go routines accessing its own private data (i.e. one golang map), which means each go routine would never exchange any data with others. The number of go routines is the same as the number of cores used. Ideally, this should be fully scalable, that is, with more cores added, the throughput of each go routine would not change.
This program tests two cases of read and write data in golang maps. Each case is tested with and without interface type transformation.
I have made experiments on one machine with 4 NUMA nodes. Each NUMA node comprises 8 physical cores and 24MB shared L3 cache. Each physical core has 32KB L1 cache and 256KB L2 cache. My experiments only include 100 keys for each map to avoid the shared L3 cache problem. I report the operations executed of each core per second. One operation could be reading or writing data in the private golang map. I test my program with 1 and 32 cores. Here are the results:
           | 1 core | 32 cores

------------ | ------------- | -------------
interface-read	 | 555285 | 105001
interface-write | 526274 | 99069
no-interface-read | 983947 | 1004945
no-interface-write | 977462 | 1006021
It is found that the pre-core throughput with interface transformation is much slower under 32 cores (i.e. less than 20% of 1 core test). However, the pre-core throughput without interface transformation has the similar performance under both 1 core and 32 cores tests.
The source code of my test program is as follows:
package main

import (
	"flag"
	"fmt"
	"math/rand"
	"runtime"
	"sync"
	"time"
)

var nsecs = flag.Int("nsecs", 5, "number of seconds to run")
var ncores = flag.Int("ncores", 1, "number of cores to scale")
var nkeys = flag.Int("nkeys", 100, "number of keys per worker")
var read = flag.Bool("read", true, "true for read; otherwise, write")
var iface = flag.Bool("i", true, "use interface or not")

type Value interface{}

type Record struct {
	intVal int
}

func (r *Record) UpdateValue(val Value) {
	r.intVal = val.(int)
}

func (r *Record) GetValue() Value {
	return r.intVal
}

func (r *Record) UpdateInt(intVal int) {
	r.intVal = intVal
}

func (r *Record) GetInt() int {
	return r.intVal
}

func main() {
	flag.Parse()

	// set max cores used
	runtime.GOMAXPROCS(*ncores)
	workers := *ncores

	var wg sync.WaitGroup
	for i := 0; i < workers; i++ {
		wg.Add(1)
		go func(n int) {
			store := make(map[int]*Record)
			for i := 0; i < *nkeys; i++ {
				store[i] = &Record{
					intVal: i,
				}
			}
			var txn int
			var val Value = nil
			var intVal int
			rnd := rand.New(rand.NewSource(1))
			end_time := time.Now().Add(time.Duration(*nsecs) * time.Second)
			for {
				tm := time.Now()
				if !end_time.After(tm) {
					break
				}
				for j := 0; j < 10; j++ {
					k := rnd.Intn(*nkeys)
					r := store[k]
					if *iface {
						if *read {
							val = r.GetValue()
						} else {
							r.UpdateValue(0)
						}
					} else {
						if *read {
							intVal = r.GetInt()
						} else {
							r.UpdateInt(0)
						}
					}
				}
				txn++
			}
			if val != nil {
				intVal = val.(int)
			}
			intVal++
			fmt.Printf("Worker %d issues %v transactions\n", n, txn)
			wg.Done()
		}(i)
	}
	wg.Wait()
}

Could you rewrite this as a standard go benchmark ?

There is an allocation involved in the assignment
val = r.GetValue()
(building an interface{} from a non-pointer).  I suspect it is the garbage collector that isn't scaling.
@aclements @RLH

What about the statement
r.UpdateValue(0)

I think there is no allocation here
@randall77

No, there is.  Converting int(0) to an interface{} allocates.
It doesn't have to.  There are optimizations that could get rid of this allocation (and the allocation on the read).  But the current Go still allocates.

Here is my go benchmark
package main

import (
	"math/rand"
	"testing"
)

type Value interface{}

type Record struct {
	intVal int
}

func (r *Record) UpdateValue(val Value) {
	r.intVal = val.(int)
}

func (r *Record) GetValue() Value {
	return r.intVal
}

func (r *Record) UpdateInt(intVal int) {
	r.intVal = intVal
}

func (r *Record) GetInt() int {
	return r.intVal
}

func BenchmarkIfaceRead(b *testing.B) {
	nkeys := 100
	b.RunParallel(func(pb *testing.PB) {
		store := make(map[int]*Record)
		rnd := rand.New(rand.NewSource(1))
		for i := 0; i < nkeys; i++ {
			store[i] = &Record{
				intVal: i,
			}
		}
		var val Value = nil
		var intVal int
		for pb.Next() {
			k := rnd.Intn(nkeys)
			r := store[k]
			val = r.GetValue()
		}
		if val != nil {
			intVal = val.(int)
		}
		intVal++
	})
}

func BenchmarkIfaceWrite(b *testing.B) {
	nkeys := 100
	b.RunParallel(func(pb *testing.PB) {
		store := make(map[int]*Record)
		rnd := rand.New(rand.NewSource(1))
		for i := 0; i < nkeys; i++ {
			store[i] = &Record{
				intVal: i,
			}
		}
		for pb.Next() {
			k := rnd.Intn(nkeys)
			r := store[k]
			r.UpdateValue(0)
		}
	})
}

func BenchmarkNoIfaceRead(b *testing.B) {
	nkeys := 100
	b.RunParallel(func(pb *testing.PB) {
		store := make(map[int]*Record)
		rnd := rand.New(rand.NewSource(1))
		for i := 0; i < nkeys; i++ {
			store[i] = &Record{
				intVal: i,
			}
		}
		var val Value = nil
		var intVal int
		for pb.Next() {
			k := rnd.Intn(nkeys)
			r := store[k]
			intVal = r.GetInt()
		}
		if val != nil {
			intVal = val.(int)
		}
		intVal++
	})
}

func BenchmarkNoIfaceWrite(b *testing.B) {
	nkeys := 100
	b.RunParallel(func(pb *testing.PB) {
		store := make(map[int]*Record)
		rnd := rand.New(rand.NewSource(1))
		for i := 0; i < nkeys; i++ {
			store[i] = &Record{
				intVal: i,
			}
		}
		for pb.Next() {
			k := rnd.Intn(nkeys)
			r := store[k]
			r.UpdateInt(0)
		}
	})
}

I try the command "go test -bench=. -cpu=1,32". The result is:
PASS
BenchmarkIfaceRead      	10000000	       177 ns/op
BenchmarkIfaceRead-32   	50000000	        31.6 ns/op
BenchmarkIfaceWrite     	        10000000	       189 ns/op
BenchmarkIfaceWrite-32  	100000000	        30.0 ns/op
BenchmarkNoIfaceRead    	20000000	        96.7 ns/op
BenchmarkNoIfaceRead-32 	500000000	         3.23 ns/op
BenchmarkNoIfaceWrite   	20000000	        97.3 ns/op
BenchmarkNoIfaceWrite-32	500000000	         3.13 ns/op
ok  	test/scale-test	16.772s

@randall77
If I change the implementation of
func (r *Record) UpdateValue(val Value) {
    r.intVal = val.(int)
}

into
func (r *Record) UpdateValue(val Value) {
        intVal := val.(int)
        r.intVal = intVal
}

It scales well. When I tried the command "go test -bench=. -cpu=1,32" again, I see
BenchmarkIfaceRead      	10000000	       178 ns/op
BenchmarkIfaceRead-32   	50000000	        30.1 ns/op
BenchmarkIfaceWrite     	        10000000	       128 ns/op
BenchmarkIfaceWrite-32  	300000000	         4.12 ns/op
BenchmarkNoIfaceRead    	20000000	        96.9 ns/op
BenchmarkNoIfaceRead-32 	500000000	         3.04 ns/op
BenchmarkNoIfaceWrite   	20000000	        97.2 ns/op
BenchmarkNoIfaceWrite-32	500000000	         3.12 ns/op
which shows that UpdateValue (i.e. BenchmarkIfaceWrite) scales well.

@totemtang The difference between these two versions of UpdateValue is that the second version doesn't allocate, while the first does one allocation. (you can use the '-benchmem' flag to see allocations and use '-memprofile ' to create a profile of your allocations. see here for more information. you can also use '-gcflags -S' to see the assembly generated by the compiler)

@totemtang, have you tried a differential profile to see where the extra time is going? Also, do you know, based on the CPU time versus the wall-clock time, whether the time is going to computation (e.g., spinning) or idle (blocking)?

It looks like in your latter case the compiler allocates the interface on the stack.  In the first case it doesn't.  I'm not really sure why that is the case.  Both instances get inlined.
I'll open a second bug for that problem.  Let's keep this bug about scaling.

@aclements
I follow the comment from @nuss-justin and see that
func (r *Record) UpdateValue(val Value) {
        intVal := val.(int)
        r.intVal = intVal
}

does not have any memory allocation while
func (r *Record) UpdateValue(val Value) {
    r.intVal = val.(int)
}

has one allocation per operation. Therefore, I think the extra time is spent on memory allocation.
I do not understand why there is an extra memory allocation since both versions do the same task. I think it is related to how golang itself compiles the two statements.

@totemtang, sorry, that's not what I meant by a differential profile. That indicates why one version is slower than another, but not why the version that doesn't scale slows down with more CPUs. I meant to CPU profile the one that doesn't scale at 1 CPU and 32 CPUs and compare those profiles.

Therefore, I think the extra time is spent on memory allocation.

That may be true, but "memory allocation" is a complex operation with many secondary effects (e.g., the actual allocation could be perfectly scalable, but the increased memory pressure makes other parts of the runtime not scale). That's why I was hoping you could perform a differential profile to compare where the time is going at a small number of cores with where the time is going with a large number of cores. If we're spending a much larger fraction of the time in some code at a large number of cores, we'll know that's the culprit and we may be able to fix it.

I do not understand why there is an extra memory allocation since both versions do the same task. I think it is related to how golang itself compiles the two statements.

Yes; this is just a flaw in how the compiler sees these two programs. To you and me they're obviously equivalent, but right now the compiler can't see that. @randall77 opened #13807 to investigate this specific issue.

@aclements Thanks for your reply.
I have profiled the time consumption of BenchmarkIfaceWrite under 1 Core and 32 Cores.

The result of 1 Core (i.e. the first figure) show that memory allocation, which I guess is the "runtime.newObject", occupies about 31% of the total time, while in 32 Core benchmark (i.e. the second figure) memory allocation occupies about 16.7%  of the total time. The reason for this decrease is that 25.9% time is spent on "runtime.morestack", which does not appear in 1 Core benchmark.
I also profiled the time consumption of BenchmarkNoIfaceWrite under 32 Cores, which does not involve memory allocation. I see "runtime.newObject" and "runtime.morestack" do not appear also.

I hope this could help you improve Golang. In fact, the advice from the above comments help me a lot with my project.

I looked into this and believe it has to do with the way values are escaped during assignments where the source is a dottype expression. Consider the following source:
https://gist.github.com/tshprecher/208b160a6dd9732d5510
Running go tool compile -m on the file returns the following:

set_interface.go:14: can inline setOneLine
set_interface.go:18: can inline setTwoLines
set_interface.go:59: inlining call to setOneLine
set_interface.go:60: inlining call to setTwoLines
set_interface.go:14: leaking param: v
set_interface.go:14: setOneLine p does not escape
set_interface.go:18: setTwoLines p does not escape
set_interface.go:18: setTwoLines v does not escape
set_interface.go:23: leaking param: v
set_interface.go:23: setOneLineNoInline p does not escape
set_interface.go:34: setTwoLinesNoInline p does not escape
set_interface.go:34: setTwoLinesNoInline v does not escape
set_interface.go:51: err escapes to heap
set_interface.go:53: f escapes to heap
set_interface.go:59: 1 escapes to heap
set_interface.go:61: 3 escapes to heap
set_interface.go:51: main ... argument does not escape
set_interface.go:59: main &i does not escape
set_interface.go:60: main &i does not escape
set_interface.go:60: main 2 does not escape
set_interface.go:61: main &i does not escape
set_interface.go:62: main &i does not escape
set_interface.go:62: main 4 does not escape

Notice on lines 59 and 61 the constant values 1 and 3 escape to the heap for the calls that set on one line. It turns out inlining is not a factor. The parameter 'v' leaks.
I created the following patch with test: https://go-review.googlesource.com/#/c/19489/
Running go tool compile -m with the patch:

set_interface.go:14: can inline setOneLine
set_interface.go:18: can inline setTwoLines
set_interface.go:59: inlining call to setOneLine
set_interface.go:60: inlining call to setTwoLines
set_interface.go:14: setOneLine p does not escape
set_interface.go:14: setOneLine v does not escape
set_interface.go:18: setTwoLines p does not escape
set_interface.go:18: setTwoLines v does not escape
set_interface.go:23: setOneLineNoInline p does not escape
set_interface.go:23: setOneLineNoInline v does not escape
set_interface.go:34: setTwoLinesNoInline p does not escape
set_interface.go:34: setTwoLinesNoInline v does not escape
set_interface.go:51: err escapes to heap
set_interface.go:53: f escapes to heap
set_interface.go:51: main ... argument does not escape
set_interface.go:59: main &i does not escape
set_interface.go:59: main 1 does not escape
set_interface.go:60: main &i does not escape
set_interface.go:60: main 2 does not escape
set_interface.go:61: main &i does not escape
set_interface.go:61: main 3 does not escape
set_interface.go:62: main &i does not escape
set_interface.go:62: main 4 does not escape

The parameter 'v' no longer leaks and the literal values 1 and 3 do not escape to the heap.


[BEAM-9918] Make TryCrossLanguage match non Try API

Try* versions of functions in the beam package should match the APIs of their non-Try counterparts, other than returning an error.
That TryCrossLanguage didn't is an oversight. Given the experimental state, this breaking change seems relatively safe, and most usage should be directed through beam.CrossLanguage anyway.
Main functional change is to pre-set coders on the outbound types incase they are needed later by cross language expansion. Real output pcollections are generated against the returned MultiEdge.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @youngoli

Agreed. I think the consistency benefits outweigh the current potential for user inconvenience.


We just crossed 12000 GH Stars

Congratulations to the awesome core team and contributors!! :smile: :sparkles: :tada:
P. S. Please feel free to close this whenever you want.

Congrats!!


Fix price decimals

What is the purpose of the change

ClickUp Task
ClickUp Task URL
Brief Changelog

Testing and Verifying

Documentation and Release Note

Is fixed in pool details page, but not pools overview

@sunnya97 strange, that is only happening to me in previews

@sunnya97 strange, that is only happening to me in previews

wdym?  Its not happening on local machine?

@sunnya97 strange, that is only happening to me in previews

wdym? Its not happening on local machine?

No I can't repro locally

What?  Why would that be??  Does it have to do with some query whitelists?  Or maybe something launch darkly related?

Preview:

Local dev:

wtf


P6 | Part 2, Task 4: Assignment report: add paragraph regarding ESSENCE standard

From the assignment instructions: Assess your way of working (p. 58 in the Essence standard v1.2) by evaluating the checklist on p. 60:
In what state are you in? Why? What are obstacles to reach the next state? How have you improved during the course, and where is more improvement possible?

Merged to report branch


add py37 to test env list

Coverage remained the same at 89.565% when pulling d3055708b51ff41cad0397d68762aa5a6256b6e6 on py37 into 2f0d6ac99d247b68e0269a417c277ad73f86ba7d on master.


'main': Fix highlighting of «<->»-style numeric globs.

This highlights print 0<-> as a redirection.

... instead of as a glob pattern for "the character 0 followed by a number".

35270f7c97b9 ?

Would incorrectly match foo<bar-baz>quux, but (z) saves us by splitting the redirects into separate words. Needs a rebase post arg-quoting merge.

Would incorrectly match fooquux, but (z) saves us by splitting the redirects into separate words.

So, no changes needed, right?

Needs a rebase post arg-quoting merge.

Yeah, and a squash.  Will do.

So, no changes needed, right?
Yep. Just the rebase.

The rebase is blocked by #468.


Add menu drawer depending on screen size

Resolves #68.

The event listener is a bit overkill for responsive design, but it's fine for now. Next time, use media queries. You can write them directly in css or use MUI as per the link: https://mui.com/material-ui/react-use-media-query/.


Evilginx to google gmail and iCloud

I have seen that you have removed gmail and iCloud when creating evilginx2 and I would like to know if it is possible to create my own phishlets for these sites without any problems

the tool was made as a "proof of concept" only.  You can create phishlets to any  web site you want.
Have a look at https://breakdev.org/ and https://github.com/kgretzky/evilginx2/wiki/Phishlet-File-Format-(2.3.0) you will find all necessary info.

I spent the last few hours trouble shooting however the phishing page I created for google request a captcha when the original site doesn’t, google is also blocking my account and I’m not able to obtain sessions.

Now it’s just redirecting to the Original site

Check this example
https://youtu.be/579OVkzaCTE

https://github.com/kgretzky/evilginx2/blob/33a9e49399773443ad7208ec85ab4f40debf34b1/phishlets/okta.yaml

@pucka20@jelledesnelle thanks I already figured google out I’m just stuck on iCloud now it’s very difficult

Good to know! Do you know how to run evilginx on localhost in kali Linux?

@pucka20@jelledesnelle thanks I already figured google out I’m just stuck on iCloud now it’s very difficult

Hi, I'm still struggling with with gmail. Can you save me, bro and share it with me? Would appreciate it..

@mac4533 , I can't get gmail phishlet to work. Kindly help

This is an "issues tracker" and this is not an "issue". Closing.

I want to know, at the moment, the work of phishlets for google exists????

Check this example https://youtu.be/579OVkzaCTE

The video is private. Can you please provide the access? @pucka20


Query not working when using OR with document hierarchies

Hello !
When doing a linq query with OR, then all the documents of the table are returned.
var query = session.Query<MegaUser>()
                       .Where(_ => _.DisplayName == "Yann" || _.DisplayName == "Robin");

When outputing the sql, I can see that the OR operator is used on the type field, and this is why all the rows are returned
select d.data, d.id, d.mt_doc_type, d.mt_version from public.mt_doc_user as d where (d.data ->> 'DisplayName' = :arg0 or d.data ->> 'DisplayName' = :arg1 or (d.mt_doc_type = 'mega_user'))
See full demo here https://gist.github.com/kYann/9fe0fd4e860049552439984789e35e8f
Thanks !

Duplicate #932


Use dirname(@FILE) instead of Pkg.dir

so that the package can be installed elsewhere
Add testing against 0.5 to Travis
this runs the most recent release candidate now, release once final tags are done

Current coverage is 100% (diff: 100%)

Merging #9 into master will not change coverage

@@           master    #9   diff @@
===================================
  Files           2     2
  Lines          96    96
  Methods         0     0
  Messages        0     0
  Branches        0     0
===================================
  Hits           96    96
  Misses          0     0
  Partials        0     0

Powered by Codecov. Last update a82f3ba...4697d6e


High CPU-Usage - even when the Game is not running

When i start the Sidekick Beta (and in the non-beta version), the App is constantly using 10-20% of my cpu, even when POE2 is not even running.
I have not changed any settings, the app has default settings.
I am running a AMD Ryzen 7500F. What is the app doing there?
I hope you are not running a loop for checking if the shortcut is pressed, are you?

I hope they are not crypto mining, are they?

Yup, the performance since I updated to 2<IP_ADDRESS>2 has been terrible. Constantly having to close and reopen it.
I have seen it idle at 20-30% cpu usage too.

I hope they are not crypto mining, are they?

From what I have seen is, they are respawning threads, I don‘t know why, but it does not look like they are mining… but I could be wrong 🤪

I swapped from beta to release version yesterday because I noticed that sidekick beta+msedgewebview2.exe are causing high CPU load (around 10-15% total CPU load on a 8 core/16 thread machine).
Now this "bug" is in the newest release version as well.

Freezing and have to use task manager to close it every hour.

It's a memory leak somewhere in the code, and from a recent change. We'll look into it.


I want to provide a simple reference implementation of the plugin usage

Description
I want to provide a simple reference implementation of the plugin usage. I tried the plugin and documented the usage steps and the sample code. Instructions in ReadMe. Source code is here. I thought it can be helpful for starters to learn using the plugin.
How to reproduce
Not Applicable
What do you expect
I thought my repo can help those who want to quickly try the plugin. So, contributing.
What happened instead
Not Applicable
Software:

docker-maven-plugin version: 0.4.2
maven version: 3.3

Full backtrace

Thanks for this example Sid. If you could provide example of what changes would be required in Dockerfile to automatically build the image in docker hub, it would be a great help.

@sid-saddala thanks. Feel free to open a PR to improve the docs. I'll close this in the meantime.


TECH : 0.2.1 allow logger object to be customized

Purpose of this PR

Allow logger objects to have other members and not only interface functions: ease integrations with custom loggers
Remove logger object from logs

Checks

[ ] Documentation is very clear, even for a newcomer
[ ] All relevant usecases of are tested
[ ] These changes won't cause memory / CPU problems even if business grows by a factor of 100
[ ] It's ok if these changes go to production and then are rolled-back

Codecov Report

Merging #5 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master     #5   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           4      4
  Lines         126    126
=====================================
  Hits          126    126

Impacted Files
Coverage Δ

lib/schema/common.schema.js
100% <ø> (ø)
:arrow_up:

lib/createWorkers.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f156f43...c10b0dc. Read the comment docs.

You are right @jeanyvesp, the worker configuration / options are documented only in the code, not in the README. I will update the README to be more explicit.


create vagrant ads-on-centos box with openvz in it

Document for running ADS from existing vagrant box : https://github.com/vlead/setup-ovpl-centos/blob/develop/user-docs/run-ads-using-vagrant-box.org#run-ads-on-existing-vagrant-box
Document for creating vagrant box which comes with ADS : https://github.com/vlead/vlead-templates/blob/develop/vagrant-boxes/setup-ads-on-centos.org
Vagrant box published : http://files.vlabs.ac.in/downloads/vagrant-boxes/ads-on-centos.box


Dunner as Go module

This builds Dunner as GoModule and removes the use of dep.
Pros:

No external dependency to install dep
Project does not have to be in GOPATH. This becomes useful in containers.

Cons:

GoModules has to be set explicitly since its not yet default, in travis as: GO111MODULE=on

@apoorvam  Can you look into why the Travis Script failed?

Its failing for an issue which is fixed in different PR. Build will pass if this PR can be merged first: https://github.com/leopardslab/dunner/pull/170/files @agentmilindu

Wdyt about using go module vs dep? @agentmilindu

@apoorvam I strongly agree with moving from Dep to Mod

Codecov Report

Merging #177 into develop will increase coverage by 5.09%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           develop    #177      +/-   ##
==========================================
+ Coverage     73.1%   78.2%   +5.09%
==========================================
  Files            9       9
  Lines          751     890     +139
==========================================
+ Hits           549     696     +147
+ Misses         129     118      -11
- Partials        73      76       +3

Impacted Files
Coverage Δ

pkg/config/config.go
85.31% <0%> (+7.87%)
:arrow_up:

pkg/dunner/dunner.go
82.81% <0%> (+7.98%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf6f188...b4d0b6a. Read the comment docs.


wip

Oops, this wasn't ready for merging just yet! 😀

Oh brother - then I definitely misunderstood your last instructions about which way the rebase of this branch was to be applied.

I could just revert (or you could), if that helps.

No worries, will remove it from main tomorrow 😄

Just removed this commit, will read the code in another commit once it's ready :)


Content in API docs of branch main disappeared

https://docs.python-arango.com/en/main/specs.html
Only headers are present, no content.
The dev branch looks fine.

This is now resolved. Sorry for the delay and inconvenience.


Refactor IOStreamer and QSPI

This refactors the entire IOStreamer infrastructure and how it's used with QSPI.
Here's a rough sketch of what the structure looks like:

This PR is correct with respect to all of the pending bugs that the existing IOStreamer and QSPI controller has, that is:

DDR sampling latency is correct
Supports sample delay functionality (with the applet auto-calculating some reasonable defaults)
QSPI controller now always samples and drives on the correct edge (or relative to the correct edge)
There is no more risk of losing a sample when the skid buffer both receives and sends a sample on the same clock cycle
A combinatorial i_stream.ready will not block

In addition what's new in the PR:

sample_delay_half_clocks is runtime-configurable, with max_sample_delay_half_clocks allows IOStreamer to configure the right amount of resources
We have discussed how we can avoid having a shift register stage, and a skid buffer stage for every sample delay step. I came to the concolusion that:

divisor must be known by IOStreamerTop in order to make optimizations like that, so I have integrated IOStretcher into IOStreamerTop.
divisor is a runtime setting, so we cannot minimize logic based on its value, so I have added a min_divisor configuration option. divisor must always be >= min_divisor. The min_divisor option can be used to optimize away skid buffer stages, and to optimize the shift register stages that delay samples from entering the skid buffer.
the size of the skid buffer can be made as small as ceil(max_latency / min_divisor), which can sometimes end up being as small as a single stage.
the shift registers that are used to delay the samples and meta from entering the skid buffer, can be implemented more  efficiently by using a counter. Unfortunately if the maximum latency is large, then the counter will only count up to min_divisor, and beyond that the rest of the latency will be implemented with shift registers. However if the maximum latency is small, The counter will suffice.
If we don't constrain divisor with a min_divisor configuration option, then I don't think that we can make any kind of optimization and we are forced to have max_latency shift register stages for i_en/meta and max_latency skid buffer stages as well.

when it comes to the 1-lane to 2-lane conversion (i.e. having two 1-lane transactions be merged into a single two-lane transaction) it's a bit complicated, and one of the following options must be chosen:

1-lane transactions could always come in pairs. So the first would always belong to lane 0 and the second would always belong to lane 1. I have chosen to not implement this solution, because we need to have i_en individually for each lane. And that means that i_en can be set high only for one lane, so while on 2-to-1-lane conversion we always get pairs, the returning samples will not be paired anymore. One way we could handle that would be to have i_en be an attribute outside of lanes, or to have i_en be or-ed together, and overridden to be the same for all lanes, before 2-to-1 conversion, but this is undesireable.
Another option would be to always convert a 1-lane transaction to a 2-lane transaction where the second lane is an empty lane. This is undesirable, because QSPIController now has to be aware if we're in the DDR+divisor=0 case, in which case samples will return on lane [1], and in all other cases samples would return on lane [0]. I seems very hackish from the perspective of the QSPIController. Samples should always return on the same lane independent of speed configuration. We can't just return ddr+divisor=0 lane[1] samples on lane[0], because that's not a generic solution, sometimes we care about samples taken during the rising edge, and sometimes for other protocols we may actually want to sample on both edges. So this option is undesirable.
What I have implemented, is that IOClockers's IO2LaneTo1Lane converter adds extra information to meta, that identifies which lane each transaction originally belonged to. Then then entire IOStreamerTop structure processes the transaction with the expanded meta field, and when samples come back to IOClockerDeframer, they have the corresponding fields tag and last.

tag will tell the IO1LaneTo2Lane upconverter, to which lane each sample belongs to
last will tell the IO1LaneTo2Lane upconverter, if it needs to wait for another sample before assembling the 2-lane transaction and sending it to QSPIDeframer.
Note that last is generated in IO2LaneTo1Lane by looking at lane 1's i_en. If lane[1].i_en is high, than lane[0].meta.last is low, and lane[1].meta.last is high, otherwise lane[0].meta.last is high.
This schema results in:

QSPIController simply sees lane[0] as clock falling edge, and lane[1] as clock rising edge. It will simply set i_en high for the edge(s) for which it's interested in receiving a sample, and the sample will come back on the same lane number(s) as on which i_en was set high. This makes things generic for future protocols.

TLDR: I think I was wrong about the folloing statement and I want to rework sample_delay-related resource optimizations, to make it possible to use smaller skid-buffer and less delayer shift register stages at slow clock speed.

If we don't constrain divisor with a min_divisor configuration option, then I don't think that we can make any kind of optimization and we are forced to have max_latency shift register stages for i_en/meta and max_latency skid buffer stages as well.

I re-thought the statement above and I think I was wrong,  I think an optimization that is a lot closer to what Cathrine originally described on discord is possible, without needing non-zero min_divisor. I think there's a way to constrain sample_delay_half_clocks differently: It could be a signal of size divisor_width, and if there is a rule that sample_delay_half_clocks <= divisor_width * 2 at all times, then we can get away with 3 skid buffer stages (that is: maximum Buffer latency + 1), and we could use only a counter for the SampleDelayer (or even re-use the existing counter in IOStretcher) (plus some shift registers for the buffer latency).
This can then optionally be further extended if we add a max_additional_sample_delay_half_clocks setting. The additional clocks would be expensive (just as expensive as the current implementation in this PR), because they would need max_additional_sample_delay_half_clocks // 2 + max_additional_sample_delay_half_clocks % 2 shift register stages, and ceil((max_additional_sample_delay_half_clocks // 2 + max_additional_sample_delay_half_clocks % 2) / min_divisor) additional skid buffer elements.
Now assuming min_divisor is 0 (which it would be for most hardware that has reconfigurable divisor), I'm not sure what is better, for example:

hardcoded max_sample_delay_half_clocks = 4, with ICE40 ddr buffers, would result in 5 shift registers, and skid buffer depth of 4.
hardcoded max_additional_sample_delay_half_clocks = 2, with ICE40 ddr buffers, would result in a (re-)using a 16-bit counter and logic used to look at the counter, 4 shift registers, skid buffer depth of 4

so similar usage as my original implementation, and would have the same amount of supported sample delay at divisor==0, however it would have the advantage of supporting much larger sample delays at larger divisors.
i.e. is the added complexity worth the additional large sample delay support that only works at large divisors?

I think I'm tending towards it's worth it.
Imagine we're driving not an ideal transmission line, but a high impedance high capacitance bus, where the response also comes back on a high impedance and hight capacitance line. Then we're forced to drive it at a slow clocks speed, where divisor a tad larger than the propagation time one way measured in f_sync_cycles. Then if we sample on the following opposite edge, then we haven't seen the return signal yet, so we'd want a sample delay of at least as much time as divisor*2 half-clock-cycles, and with the revised idea, we basically get that for free for arbitrarily slow buses, without additional resources. The current implementation in this PR would make it very expensive, depending on how slow a bus we would want to support.
Next time I'm looking at this I think I'll try giving this idea an implementation.

Just to be clear, I still plan to re-do the infrastructure myself at some point. As long as you're fine with duplicated work, that's going to be OK.

That wasn't clear, any way, thanks for clarifying. I'll set this to draft, feel free to reuse any of my code if you find it useful.

The reason I'm saying that is because I'd like the implementation to be suitable to upstream to amaranth-stdio. Yours is very complex and has a lot of idiosyncrasies, which would be fine for a Glasgow applet but isn't going to cut it for a generic and reusable module that I expect everyone to use. I've scrolled through it and I don't feel that it is suitable to be moved to amaranth-stdio; I do appreciate that you've put an enormous amount of work into it and I would like to discuss the mechanism with you to make sure that whatever eventually ends up in amaranth-stdio is of high quality.


plutono-image-renderer

I see in the log complaints about needing this plugin but I can't find any information about the plugin and how to get it or install it.
Running plutono-cli plugins doesn't show the option to install plugins. Just list/remove,

3.5.0 works with plutono

can be closed

3.5.0 works with plutono


Record a longer, more in-depth demo video

Whatever pain point this is supposed to solve, I personally hope to solve it with a good README (please correct me if I'm wrong)

🤷 People learn in different ways, seems good to have both modalities.


CachedDbAccess insert functions should first write to rocksDB then to cache

Hi : )
I'm new to this project and slowly reading up the code and see if there are some contributions I can make.
Here in general it is a good practice to first add the data to the permanent store before storing it in the cache. In case somewhere the insertion throws an error and this error is caught somewhere, then you'd have inconsistency between the cache and the store.

I'll defer for specifics to @michaelsutton but generally (from the performance engineering standpoint), you do want the data in the cache to be available while the insert (and serialization) take place.  This is a heavily threaded environment, so the cache might get multiple hits while the database is doing its thing. Also, a database error is unacceptable, as such a db error should cause a complete daemon teardown.  (recovery from a halt is a separate subject).
Once again, this is not something I work on but my brief peek and understanding of what this does is as I described.  I might be wrong tho. :)
What you are saying is true for a typical "server" that uses a typical database to provide a service and can encounter errors, perhaps recover from them, and typically would need to maintain a db-cache consistency. In the case of the p2p node, the data integrity should be maintained by panicking or halting.  At the same time, in an ad-hoc p2p environment, data relay or its use via in-memory cache has no ill side effects.

Also, please take a look at the tests as the code in your PR doesn't build.  I would recommend running /check in the root of the repository, followed by /test before the PR.

@aspect I see, what you said totally make sense to me. Maybe I should not be too eager to submit PRs without a holistic understanding. I'll take a look at the error tonight

Feel free to jump into Discord on #development and ask any questions.


feat(api): warn tc-lid/gantry collision in simulation

If a given protocol would target the labware loaded into a thermocycler module, while that module is known to be closed, simulation will fail with a helpful message.
Closes #4044
review requests
Attempt to upload a protocol in which you:

[ ] close the TC lid and then pipette into it (should raise)
[ ] close the TC lid and then pipette from it (should raise)
[ ]  pipette into it with lid open (should simulate)

Also tested locally in simulation


[doc][import.rst]Typo nas_blog => ontap_nas

Typo fix from nas_blog to ontap_nas
This is related to
https://github.com/NetApp/trident/issues/498

I forgot CCLA. so I close this ticket.


formatting and clarity for pow (x,y,[z])

Line 2: edited for more descriptive title
Line 4: formatted per Python documentation (brackets '[]' around 'z')

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

Line 2: edited for more descriptive title
Line 4: formatted per Python documentation (brackets '[]' around 'z')


Add Envelop dapp

Revision checks

[x] Used smart contracts were audited.
[x] Your Safe App include a manifest.json file at the root with the required data. https://appv1.envelop.is/manifest.json
[x] The app can be loaded as a custom Safe App in the Apps section of https://app.safe.global.
[x] The app auto-connects to the Safe.
[x] You are able to trigger and execute one transaction with a Safe.
[x] RPC requests are optimized (not triggering many requests in a very short time period).

Name/Description
Envelop Dapp
Cross-chain multi-tool protocol providing true value to your NFT
Type

New addition

Compatible Networks
- Mainnet
- Goerli
- Binance
- Polygon

Audit document
Link to smart contracts audit:
https://github.com/dao-envelop/envelop-protocol-v1/blob/master/audit/20230420_iber_envelop_audit_rev2.pdf
Code for review
Link to git repository where the app is published.
https://gitlab.com/niftsy/web3app_v1
IPFS hash/App URL
https://appv1.envelop.is/
Team information
- Project logo: https://envelop.is/assets/img/envelop.svg
- Project landing page: https://envelop.is/
- Team or discord contact: https://t.me/envelop_en
- Twitter link: https://twitter.com/Envelop_project
- Github link: https://github.com/dao-envelop

sourcify.dev
https://sourcify.dev/#/lookup/0x44C1bCE3f284e9a672fa4ba6f5b02E8651da88a0
https://sourcify.dev/#/lookup/0x53c55bB901812551aa36cbf022B5df35B24C9f59
https://sourcify.dev/#/lookup/0x3Ad9D520A9bF85f86C829B291870d77AF5B30697
https://sourcify.dev/#/lookup/0x27772e884e9d238F5478c07D44C515bcae585b1d
https://sourcify.dev/#/lookup/0x15A623559E29C0Fc39b9561c3fD304b677E66087

Hey @maxsiz thank you for the submission!
The dApp looks good!
Could you grant me read permissions to the frontend repo to have a quick look to the implementation?
Thanks!

Hi @maxsiz
It seems that the project is not visible for me:

Hmmm
Check please,  I have granted access
MC
пн, 3 июл. 2023 г. в 18:49, Dani Somoza @.***>:

Hi @maxsiz https://github.com/maxsiz
It seems that the project is not visible for me:
[image: gitlab-repo-404]
https://user-images.githubusercontent.com/26763673/250547182-97b9c906-757f-432a-82e1-ba06fcf77e17.jpg
—
Reply to this email directly, view it on GitHub
https://github.com/5afe/safe-apps-list/issues/259#issuecomment-1618066186,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEDPSB2TMFQZD4ZH5QTSI7DXOKWUNANCNFSM6AAAAAAY4GIG5E
.
You are receiving this because you were mentioned.Message ID:
@.***>

Could you update old @gnosis.pm packages to our new @safe-global packages?
see: https://gitlab.com/envelop/web3app_v1/-/blob/master/package.json#L6
old packages:
@gnosis.pm/safe-apps-provider
@gnosis.pm/safe-apps-react-sdk

new packages:
@safe-global/safe-apps-provider
@safe-global/safe-apps-react-sdk

https://gitlab.com/envelop/web3app_v1/-/commit/ccf545fd800f6e775f18c6d715d35867f8221320
Done
MC
пн, 3 июл. 2023 г. в 19:12, Dani Somoza @.***>:

Could you update old @gnosis.pm packages to our new @safe-global packages?
see: https://gitlab.com/envelop/web3app_v1/-/blob/master/package.json#L6
old packages:
@gnosis.pm/safe-apps-provider
@gnosis.pm/safe-apps-react-sdk
new packages:
@safe-global/safe-apps-provider
@safe-global/safe-apps-react-sdk
—
Reply to this email directly, view it on GitHub
https://github.com/5afe/safe-apps-list/issues/259#issuecomment-1618125078,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEDPSB4ZZYSWN7AJD57RNDTXOKZJNANCNFSM6AAAAAAY4GIG5E
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hi! QA from safe apps team here 😄  Some comments from me:
The mint feature does not look to be working as expected, always prompting to connect a wallet.
https://github.com/5afe/safe-apps-list/assets/92332710/054fce17-b0cc-4444-b56e-8fd6932f87ac
The create tickets feature does not seem to work inside of an iframe given the console error shown:

Farming and launchpad features on the top menu always seem to redirect to a new tab which will make it not compatible with a safe. Is this expected?
Cheers!

Hi,  thanks for update.
Actually for starters, we would like to make  available for Gnosis users only  main our dapps features of the protocol.

wrap https://appv1.envelop.is/wrap
dashboard https://appv1.envelop.is/list

Other features(in fact separate applications) not adopted for Gnosis right now

I understand but what worries me is that this will be quite tricky for our users. Even if those two are compatible, the rest that are not will be easily accesible from the app too.

Just let us make new build to solve this issue. I will ping you when done

Okay
Could you please check  with this URI
https://app.envelop.is

Much nicer now, thank you! Note that minting does still not work:
https://github.com/5afe/safe-apps-list/assets/92332710/48021047-0671-4bae-893a-a5b2cf8cc0e7

https://bscscan.com/tx/0xb43436d343d8efc4b80deaedab4257ec225aa52150a96a9a3f6f76d96f58171f

You are right, my bad. All good with the app now, we will let the team know for a final 👀  and we will post a confirmation message here once the app is listed.
Cheers!

Hello @maxsiz!
The app has been published

Could you please add our dapp for Arbitrum also?

Should be available now


Feat: add footer

Is your feature request related to a problem? Please describe.
Add a Footer section that includes -

copyrights - "SQL Online Editor - Copyright ©2023"
the creator's GitHub profile - https://github.com/janvi01
"Found an issue? Report it here"

Take reference from other open-source projects like https://reactplay.io/

Hello @janvi01, please assign me this issue.

assigned @Ferdinand-Ogama feel free to drop in any doubts.

Ok.

@janvi01 I can seem to find the hex code green version of the Navbar to also use in the footer. I have tried inspecting elements, but I still can't find.
Do you have a way of doing it?
Apologies, I am new to Web and React, but I will deliver :)

Of course with your guidance.

I found it. It's teal

Hello, I get fatal errors whenever I try to commit. Not sure where the problem is? Can't find solutions online. Will Keep trying.

@Ferdinand-Ogama What errors are you getting?

I resolved it. Sent a PR.

Hey there! Can you assign me.

Please Can you assign this to me

Hey @tutrinh16 @08Uzair this issue is already assigned. Please take a look at other issues in the issue section.

@janvi01 created a PR for this issue 3 days ago. Care to look at it?

Sure @Ferdinand-Ogama I'll look when I can, quite busy with some other task.


Make the state of master tests and deploys more obvious

Per the discussion from #3712, we would like to investigate making the current state of master tests and deploys more obvious to OperatorSDK developers. Currently there is a tiny badge on the latest commit line at the top of the base page that status of the most recent master build. When a master job fails an email is sent out to repo admins, but the jobs are flakey and rerun when another PR merges so they are often not investigated.
It would be nice if the state of these jobs was more obvious. Some possibilities include:

Making some sort of CI board with the state of running/finished jobs
Make something send Slack notifications to OperatorSDK devs when stuff blows up
sending emails to more people than just repo admins

Per the discussion in the 11/2 triage meeting, it was decided that the current badge status indicator is good enough, so there isn't anything actionable on this at this time.


[DRAFT] RFC: Gradual struct initialization

Rendered
This is a draft version of an RFC for you to review, before a formal proposal is made for consideration.
r? @pnkfelix @nikomatsakis -- I have left some TODOs for you to fill in... :)

note to self: squash the commits before publishing!

I think the RFC should explain what is new behavior, what is making behavior consistent with a similar case and what is really new. For example, nothing in "Definitive initialization and usage" is new AFAIK, this is already how we treat local variables (when they are not fields).
Basically, most of this RFC makes it so that fields of a struct behave even more like local variables behave already. it should be clear which part goes beyond that, if any. Conversely, the normative text of the RFC should apply to struct fields and locals alike, to make it clear that behavior is consistent after this RFC gets accepted.
The part that (seems to me) goes beyond this is the part about self-referential structs. Is that really as simple as the RFC makes it sound? Will this require dedicated support in NLL or so, or does it just fall out of everything else?

Should we allow gradual initialization for types implementing Drop?
While this might make the language more consistent and uniform, [...]

Actually I'd argue it does the opposite: We already cannot move out of fields of a Drop struct, and hence it is just consistent that we also cannot partially initialize "into" such a struct. Basically, structs which implement Drop are always fully nor not-at-all initialized. I think that makes a lot of sense.
Since this concerns code that is automatically inserted by the compiler (drop glue), which can already sometimes be hard to reason about, I'd say let us be careful here and  not make this any more complicated than it already is.

I think the RFC should explain what is old behavior, what is making behavior consistent with a similar case and what is really new. For example, nothing in "Definitive initialization and usage" is new AFAIK, this is already how we treat local variables (when they are not fields).

Will do.

Basically, most of this RFC makes it so that fields of a struct behave even more like local variables behave already. That's great, I think we should do that! It should be clear which part goes beyond that, if any. Conversely, the normative text of the RFC should apply to struct fields and locals alike, to make it clear that behavior is consistent after this RFC gets accepted.

I'll try to make this clearer. :) Should be fairly easy as the self referential struct bits are somewhat separate textually.

The part that (seems to me) goes beyond this is the part about self-referential structs. Is that really as simple as the RFC makes it sound? Will this require dedicated support in NLL or so, or does it just fall out of everything else?

I could be wrong but it seems to me that it is really mostly a syntactic restriction that is being lifted and that NLL/polonius already supports this, but I could be wrong! This seems like an excellent bit for @pnkfelix and @nikomatsakis to elaborate upon and fill in in the RFC. =P

Should we allow gradual initialization for types implementing Drop?
While this might make the language more consistent and uniform, [...]

Actually I'd argue it does the opposite: We already cannot move out of fields of a Drop struct, and hence it is just consistent that we also cannot partially initialize "into" such a struct. Basically, structs which implement Drop are always fully nor not-at-all initialized. I think that makes a lot of sense.

Right so the idea is that restrictions around Drop are somewhat arbitrary restrictions; it would be more consistent to not have such restrictions and then everything would be net more consistent. However, if we don't allow you to move out of fields of a Drop struct then we should probably not permit gradual init for Drop structs either; they are inexorably linked.
I would be glad to integrate any textual changes you propose that improves on nuance here.
I'm not sure how to improve it myself at the moment tho.

Right so the idea is that restrictions around Drop are somewhat arbitrary restrictions; it would be more consistent to not have such restrictions and then everything would be net more consistent. However, if we don't allow you to move out of fields of a Drop struct then we should probably not permit gradual init for Drop structs either; they are inexorably linked.
I would be glad to integrate any textual changes you propose that improves on nuance here.
I'm not sure how to improve it myself at the moment tho.

I'd just add a sentence when Drop comes up that not permitting partial initialization on impl Drop types is consistent with not allowing moving out of them: The compiler ensures they are either fully initialized or not at all, to avoid any ambiguity about whether the destructor runs or not.

I just stumbled upon this beautiful RFC. It feels like a natural extension of existing features, shall we resuscitate it?