observed performance regression

I was evaluating switching to the new google maintained tcmalloc project. As part of the evaluation, I did rerun the benchmark that Sanjay initially did at:
https://gperftools.github.io/gperftools/tcmalloc.html
I do not know what, but it seems like the 2024 gperftools tcmalloc is much less performant than what was there back then...
this is mostly flagrant when you compare the 8 threads 64 bytes allocation Mops.
https://github.com/google/tcmalloc/issues/260#issuecomment-2312896329
It went from 8 Mops to 2 Mops...
I do not know what but it seems like something did seriously break in between...

I have included jemalloc into the benchmark...
something suspicious is that all allocators appears to suffer from a lock contention except glibc... what could be the common denominator?
I have no clue on how the thread_local keyword is implemented but this could be this guy...

I went as far as recompiling tag perftools-1.0 and the same crappy performance is found with this version so the problem is external to the project code...
I fail to see what it could be...
that would be really nice if someone could test:
LD_PRELOAD=libtcmalloc_minimal.so time ./ptmalloc_test1 1 1 1000000 64
LD_PRELOAD=libtcmalloc_minimal.so time ./ptmalloc_test1 8 8 1000000 64

to validate that he sees the same performance deterioration as soon as several threads are used.

yes I have compiled src/tests/ptmalloc/t-test1.c.
not with -O2 but with -O3.. beside that this is what I have done

my investigation has made me find a minor glitch in the project code... I'll prepare a pull request to share this finding soon...

I have done this:
diff --git a/src/thread_cache.cc b/src/thread_cache.cc
index 3ac1019..c88b7de 100644
--- a/src/thread_cache.cc
+++ b/src/thread_cache.cc
@@ -80,7 +80,8 @@ ThreadCache::ThreadCache() {
   ASSERT(Static::pageheap_lock()->IsHeld());

   size_ = 0;
-
+  central_fetch_cnt_ = 0;
+  central_release_cnt_ = 0;
   max_size_ = 0;
   IncreaseCacheLimitLocked();
   if (max_size_ == 0) {
@@ -114,6 +115,13 @@ ThreadCache::~ThreadCache() {
       ReleaseToCentralCache(&list_[cl], cl, list_[cl].length());
     }
   }
+  FILE *fp = fopen("tc_stats.txt", "a");
+
+  if (fp) {
+      fprintf(fp, "central fetch:%lu release:%lu\n",
+              central_fetch_cnt_, central_release_cnt_);
+      fclose(fp);
+  }
 }

 // Remove some objects of class "cl" from central cache and add to thread heap.
@@ -128,6 +136,7 @@ void* ThreadCache::FetchFromCentralCache(uint32_t cl, int32_t byte_size,
   void *start, *end;
   int fetch_count = Static::central_cache()[cl].RemoveRange(
       &start, &end, num_to_move);
+  ++central_fetch_cnt_;

   if (fetch_count == 0) {
     ASSERT(start == NULL);
@@ -166,6 +175,7 @@ void ThreadCache::ListTooLong(FreeList* list, uint32_t cl) {

   const int batch_size = Static::sizemap()->num_objects_to_move(cl);
   ReleaseToCentralCache(list, cl, batch_size);
+  ++central_release_cnt_;

   // If the list is too long, we need to transfer some number of
   // objects to the central cache.  Ideally, we would transfer
diff --git a/src/thread_cache.h b/src/thread_cache.h
index 7454688..23bd1cf 100644
--- a/src/thread_cache.h
+++ b/src/thread_cache.h
@@ -305,6 +305,8 @@ class ThreadCache {
   // All ThreadCache objects are kept in a linked list (for stats collection)
   ThreadCache* next_;
   ThreadCache* prev_;
+  size_t       central_fetch_cnt_;
+  size_t       central_release_cnt_;

   // Ensure that this class is cacheline-aligned. This is critical for
   // performance, as false sharing would negate many of the benefits

for:
LD_PRELOAD=/home/lano1106/dev/packages/gperftools/src/gperftools/.libs/libtcmalloc_minimal.so TCMALLOC_RELEASE_RATE=0 TCMALLOC_TRANSFER_NUM_OBJ=256 time ./ptmalloc_test1 1 1 10000000 64
Using posix threads.
total=1 threads=1 i_max=10000000 size=64 bins=125000
Created thread 7841c66006c0.
Done.
0.62user 0.01system 0:00.94elapsed 67%CPU (0avgtext+0avgdata 13576maxresident)k
0inputs+32outputs (0major+2065minor)pagefaults 0swaps

I got:
central fetch:1085 release:461
central fetch:2 release:0
central fetch:2 release:0
central fetch:2 release:0

for
LD_PRELOAD=/home/lano1106/dev/packages/gperftools/src/gperftools/.libs/libtcmalloc_minimal.so TCMALLOC_RELEASE_RATE=0 TCMALLOC_TRANSFER_NUM_OBJ=256 time ./ptmalloc_test1 8 8 10000000 64
Using posix threads.
total=8 threads=8 i_max=10000000 size=64 bins=15625
Created thread 7a553fa006c0.
Created thread 7a553f0006c0.
Created thread 7a553e6006c0.
Created thread 7a553dc006c0.
Created thread 7a553d2006c0.
Created thread 7a553c8006c0.
Created thread 7a553be006c0.
Created thread 7a553b4006c0.
 I was expecting 1!
The amount of central fetch is higher than I would have expected for a reasonably bounded allocation usage pattern
TBH, I do not fully understand what t-test1.c is doing... It does the specified amount of iterations but at each iteration, it seems to choose a random action... I honestly cannot affirm that the test program is correct. If it is leaking, that could explain why the central heap is that much solicited. OTOH, glibc appears to manage much better the situation by adopting a more performant strategy to cope with what is thrown at it...

Hi. I'll give more elaborate comment later (likely tomorrow), but here are some quick thoughts.

you're likely seeing not a regression in gperftools or ("abseil") tcmalloc but glibc getting better (and in this benchmark too)

I am thankful for your insightful comments and general interest in gperftools. We need more people, even if it is just for extra code review (but we have some "writing code" projects too). But do keep in mind than way back when original tcmalloc was written, glibc and some common mallocs were not so great for common C++-ful workloads. Nearly any benchmark was good enough to enable good progress. But today, ~all common mallocs got their basics right, so now benchmark and/or workloads specifics matter. This benchmark, do keep in mind, is not necessary great representation of whatever workloads you're optimizing.

you make me think about something... I must not be calculating mops the same way than Sanjay did.
all I am doing is iterations/time
by looking at graph on
https://gperftools.github.io/gperftools/tcmalloc.html
I now realize that he must be doing num_threads*iteration/time
maybe not regression... but certainly valid interrogations...
I fail to see why there is so much access to the central heap...
The number is thread caches is suspicious too...
if it is only a matter of configuration... the library is incredibly opaque about suboptimal operation... I have been few days struggling to make sense of what I am seeing and no one has been able to provide a satisfying explanation yet...
thanks for the good words and I am looking forward the more elaborated comment!

2. Nearly any benchmark was good enough to enable good progress. But today, ~all common mallocs got their basics right, so now benchmark and/or workloads specifics matter. This benchmark, do keep in mind, is not necessary great representation of whatever workloads you're optimizing.

I get what you say here... the simple benchmark may not be representative of my actual workload...
yet it makes tcmalloc exhibit a behavior that I want to avoid in my real application. my app is perhaps having a 15MB memory footprint running on a 128GB RAM dedicated system... I absolutely don't care at all to accumulate memory in the thread cache. The only thing that I want is the fastest allocation possible... I have multiple threads having memory allocated by openssl... t-test1.c is representative in the sense that it does not appear to allocate an unreasonable amount of memory yet the library release very often memory to the central heap.
I am perhaps few hours away from finally fully understand how tcmalloc take this decision and how to configure it to behave as I want (or maybe shorter if I get some help)...
the question that I have, if it is only a matter of configuration is: is tcmalloc default settings adequate for a general purpose usage? Unless you tell me that there is a lot of embedded systems users in tcmalloc userbase, then maybe they could be a bit more liberal like what glibc choose to have to exhibit a better performance in t-test1.c that I believe to be reasonably representative to an average heap usage...

oh here is small gem that I have discovered... I am using GCC 14.1.2 and I am using the switch -std=c++26 when I compile tcmalloc... I do not know what in C++26 is doing that but with this settings I have seen some results from malloc_bench be 10% faster than if compiled with the default standard of -std=c++17...
10% improvement... this is not something to disregard...

in t-test1.c malloc_test() function. I identify 3 sections

init bins. 50% of them will be allocated. (Most the central heap fetch are going to occur there)
main test random deallocation/allocation are going to be performed 'iterations' times
teardown. all the memory will be released (this is where the majority of the central heap release will occur, IMHO)

I think that it would be interesting to know how many times the central heap is accessed in step 2. possibly never or very little (hopefully)... I'll try to modify my debug hooks to answer this question...
beside that, it might be possible to improve the central heap access logic to minimize access to it...
I am throwing few random ideas that come my mind:
Make the fetch ramp-up batchsize customizable... It could be interesting to give the option to the users to initialize FreeList::max_length_ with Static::sizemap()->num_objects_to_move(cl)

I have enhanced my ThreadCache log trace to include ThreadCache::max_size_
the result obtained with
./ptmalloc_test1 1 1 10000000 64
central fetch:1085 release:461 max_size_:196608
central fetch:2 release:0 max_size_:65536
central fetch:2 release:0 max_size_:65536
central fetch:2 release:0 max_size_:65536

where does this ridiculously low value of 196608 come from?
https://gperftools.github.io/gperftools/tcmalloc.html
mention that TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES default to 32MB (or 16MB if you refer to the value in the text)

So another quick thing. I am not sure what is going on on your end.

That 10000000 "imax" parameter is way way too short. It runs 15 ms (yes, milliseconds) on my box. I need to run 100x more to get plausible figures.

profile of that run shows ~all malloc ticks in fast-path. No central free list business. In fact, for "size" parameter of 64 we'll get only few size-classes in use and so we should be able to adapt reasonably quickly to to only 125k "bins" that this test uses.

Ah btw gperftools is faster than glibc on this workloads (because, again, we're pretty good on this fast-path thingy)
So I am not sure what you're chasing after.

you are performing the test on a much faster machine than me. My dev machine is a old sandybridge system.

I got results in the 15ms range with 1,000,000 iterations... with 10,000,000 iterations it takes a couple of seconds (4 ish)
Cut in half the 125k "bins" number. Roughly only have of them are allocated:

		if(RANDOM(&ld, 2) == 0)
			bin_alloc(&p.m[b], RANDOM(&ld, p.size) + 1,
					  lran2(&ld));

do you have env vars configured that would explain why you get different result?
Sure, if you never access the central heap as I do, you will get a much faster run...
I do not understand how it is possible that you get away from ever accessing the central heap...
max_size_ is initialized at kStealAmount (64kb) and is incremented by kStealAmount each time salvage() is called....
I am going to create a profiler run output like you did to show you how different my run looks like...
BTW, I have solved the bogus ThreadCache thing... I think that temporary ThreadCache objects are created because I am calling functions that allocated memory from the ThreadCache destructor... I have still improved my trace by adding few more fields... The bogus cache objects are reusing the same addresses:
central fetch:2163 release:0 max_size_:917504 unclaimed_cache_space_:32505856 thread_heap_count_:3 this:0x653987e22040
central fetch:2    release:0 max_size_:65536  unclaimed_cache_space_:33357824 thread_heap_count_:3 this:0x653987e23080
central fetch:2    release:0 max_size_:65536  unclaimed_cache_space_:33357824 thread_heap_count_:3 this:0x653987e22040
central fetch:2    release:0 max_size_:65536  unclaimed_cache_space_:33357824 thread_heap_count_:3 this:0x653987e23080

btw, I have succeeded to have a run with zero releases... I have achieved this by changing the
kMaxDynamicFreeListLength value from 8192 to 32768...
at this point, I am just playing the various knobs to see what they do...

also, since we are running on different HW, it does not matter that your run is much faster than mine... what you want to check is how the time for a 8 threads run compare with a 1 thread run... What I have observed is 8x slowdown caused by central heap accesses... If you do not have the same contention, then the expected result should be 1thread test time ~==~ 8threads test time

I do not understand how it is possible that you get away from ever accessing the central heap... max_size_ is initialized at kStealAmount (64kb) and is incremented by kStealAmount each time salvage() is called....

It is not that they don't happen. It is just peanuts compared to the total amount of work done.
I don't set any environment variables (except LD_PRELOAD and CPUPROFILE).
Yes, it shouldn't matter if you're doing it on older machine or not. I guess I try this on rpi 4 I have to double check.

I have reproduced your profiler profile on my side:

IMHO, it is not conclusive...
yes it is peanuts in the single thread setup since there is no contention. The real test is comparing the exec time on the same HW and same parameters. If the multiple threads test exhibits an exec time 8x slower it is sign of a contention... The only contention that I am aware of is the central heap lock one... If it is peanuts, I would expect the 8 threads setup perform almost as fast as the single thread one...

Absolutely no signs of lock contention. Here is RPI 4:
pi@raspberrypi:~/src/ptmalloc-bench $ perf stat \time sh -c 'LD_PRELOAD=/home/pi/gpt/lib/libtcmalloc_minimal.so ./t-test1 1 1 100000000 64'
Using posix threads.
total=1 threads=1 i_max=100000000 size=64 bins=125000
Created thread 7f918ff160.
num_bin_allocs: 50047547
Done.
14.54user 0.00system 0:14.55elapsed 99%CPU (0avgtext+0avgdata 11744maxresident)k
0inputs+0outputs (0major+2259minor)pagefaults 0swaps

 Performance counter stats for 'time sh -c LD_PRELOAD=/home/pi/gpt/lib/libtcmalloc_minimal.so ./t-test1 1 1 100000000 64':

          14554.16 msec task-clock                       #    1.000 CPUs utilized
                85      context-switches                 #    5.840 /sec
                 0      cpu-migrations                   #    0.000 /sec
              2312      page-faults                      #  158.855 /sec
<PHONE_NUMBER>0      cycles                           #    1.800 GHz
<PHONE_NUMBER>8      instructions                     #    0.43  insn per cycle
   <not supported>      branches
         142574508      branch-misses

      14.558668832 seconds time elapsed

      14.545619000 seconds user
       0.009988000 seconds sys

pi@raspberrypi:~/src/ptmalloc-bench $
pi@raspberrypi:~/src/ptmalloc-bench $ perf stat \time sh -c 'LD_PRELOAD=/home/pi/gpt/lib/libtcmalloc_minimal.so ./t-test1 4 4 100000000 64'
Using posix threads.
total=4 threads=4 i_max=100000000 size=64 bins=31250
Created thread 7f8c2df160.
Created thread 7f8bacf160.
Created thread 7f8b2bf160.
Created thread 7f8aaaf160.
num_bin_allocs: 50024031
num_bin_allocs: 50040030
num_bin_allocs: 50005881
num_bin_allocs: 50021897
Done.
95.46user 0.07system 0:24.09elapsed 396%CPU (0avgtext+0avgdata 11920maxresident)k
0inputs+0outputs (0major+2302minor)pagefaults 0swaps

 Performance counter stats for 'time sh -c LD_PRELOAD=/home/pi/gpt/lib/libtcmalloc_minimal.so ./t-test1 4 4 100000000 64':

          95548.26 msec task-clock                       #    3.965 CPUs utilized
               451      context-switches                 #    4.720 /sec
                 2      cpu-migrations                   #    0.021 /sec
              2356      page-faults                      #   24.658 /sec
<PHONE_NUMBER>10      cycles                           #    1.800 GHz
<PHONE_NUMBER>4      instructions                     #    0.26  insn per cycle
   <not supported>      branches
         283899065      branch-misses

      24.098579251 seconds time elapsed

      95.468168000 seconds user
       0.077941000 seconds sys

Bumping thread count proportionally scales up amount of work. And yes we have each core getting about 2x slower, but we have total number of instructions scaled 4x doing 4x more work. So whatever slowdown is due to microarchitectural reasons (thermal budget, shared caches things like that).
And 100% cpu utilization too. Lock contention would lower it due to sleeping, or we'd see spinlock busy looping (we do a bit) on profiles. None of that is there.
Okay, I think this can be closed as 'not a bug'.
Ah, final thing. Profile you show above has catastrophically small number of profiling ticks. That screams noise everywhere. And likely explains oddities you report like random unrelated flags showing big impact that is "impossible".

ok my bad... I must have confused user time with elapsed time (aka clock time)

here are the updated graphes with the corrected measurement methodology

Thanks. I am slightly surprised to see "abseil" tcmalloc to loose to us here.
Could be "prefetch" thingy though. I.e. they do prefetch in allocation path which has some evidence (although I wasn't 100% sure I buy it) that it benefits real world app performance despite doing slightly more work within malloc. And certainly degrades benchmark numbers a bit.

yes... it is very surprising... their documentation is very well done... the features/benefits list is very enticing...
but OTOH, I had few interrogation points:

bazel build support only
Must be statically built. ODR issue if app is already linking with Abseil https://github.com/google/tcmalloc/issues/27#issuecomment-2304702069
Incompatibility with glibc because of unorthodox RSEQ usage https://github.com/google/tcmalloc/issues/144

with all those friction points, I wanted to be absolutely sure that transitioning to this variant was the best possible move. This is the driver for the benchmark effort.
because of a stupid error from me, I spent a lot of time, (and some of yours) trying to resolve an imaginary problem. OTOH, I have learned a lot from it... I can now claim to have a pretty good understanding about how tcmalloc works...
that was an incredible learning activity...

my opinion about "abseil" tcmalloc... (I am not aware of the prefetch thing), it is that they have traded off some allocation/deallocation performance to offer a per-cpu cache...

Not entirely correct. Yes, for them per-cpu caches matter or at least they think so. Because of fibers use-case (think of it like, thread per request; so that would be a ton of threads to "waste" memory on per-thread caches). But per-cpu caches actually enable some performance, despite some (fairly small) rseq overhead. This is because pushing and poping elements to the per-size-class lists doesn't involve pointer chasing. And it seems to matter quite a bit. Also enables them do to much improved central free list design (also saving tons of pointer chasing). This plus some pretty good attention to performance makes abseil tcmalloc pretty good performance wise.
And there are certainly a ton of workloads, non-Google workloads, that would benefit from per-cpu caches today (e.g. Mongo).
I am not sure what causes the gaps on graphs above, but they don't represent reality in practice for C++ workloads. Mentioning C++ workloads because one of changes there, they turned malloc into memalign to conform to nonsensical minimal alignment requirements formally part of C standard (BTW C++ might actually do same). So operator new is full performance. Malloc is slightly slower (but I think still faster then our memalign).
Ah and another thing. If you have some appetite for central free list performance chasing there is actually one semi-prominent benchmark that we lose today and it involves CFL. Here: https://github.com/gperftools/gperftools/issues/1448

when you mention pointer chasing... this is a tcmalloc aspect that did not appear in my code revision. I suppose that this pointer chasing business occur during deallocation to know to which size class a pointer belongs to... Hence the C++14 sized delete operator is a welcome addition as it totally shortcut the pointer chasing thing.
I think that recent C++ standard revisions also mandate some minimal alignment requirements...
I am not sure why you say that t-test1 benchmark does not represent C++ workload... What is a representative C++ workload? How would you qualify it?
The only difference that I possibly can imagine between t-test1 workload and a real app workload is that the real app might have more variety in its allocation sizes... to me it is pretty convincing beside that...
t-test1 measure how fast an allocator can service a sequence of random allocation/deallocation of a certain size across N threads... this seems to me like a valid way to measure and compare allocators performance...
Do you have data that shows abseil tcmalloc on its best side? I have not been able to find any...
Otherwise, what would be the ultimate way to see what the allocator is capable to do with a real workload? Profile the app using different allocators and see where malloc is positioned in the top profiled functions list?
beside that, thank you for the link but no... thank you... my craving for benchmark numbers is satisfied... sure I am sure that there is exist benchmarks that press all on the wrong buttons to make certain allocators looks bad simply because of their design choices... the type of situation that does not show up in real scenarios... I am trying to put back the top on the pot concerning this specific topic...
I am going to stay at my current conclusion that for a series of random allocation/deallocation, abseil tcmalloc perform less well than vanilla tcmalloc until new data convince me otherwise...

I've yet to check that link you posted above about RSEQ compatibility issues, but that numbers you posted above don't look right. I speculate (but plausibly) that your abseil tcmalloc doesn't run in per-cpu mode.
I now see you statement above above "compromising" performance for per-cpu mode in potentially new light. So let me clarify. It is arguably part of big "weakness" of sorts of Abseil thing in general, but tcmalloc abseil especially. They only care above their use-case and not interested in any compromises. Open-source in this project is to enable sharing with other Alphabet bets and encouraging external contributions (like from Intel or Arm, stuff like that). They're not interested in other contributions as long they don't help their somewhat narrow use-case.
So in this regard, per-cpu mode is the only mode they optimize for. There have been numerous internal discussions while I was part of that team about even dropping "legacy" per-thread stuff. I can tell you with certainty that I have seen abseil tcmalloc to perform much better than stats above and I know both Chris and Dmitry won't regress performance.
So as long as you're not doing per-cpu mode in practice, you're not seeing its true might. And if (again, saying this without really checking) lack of per-cpu mode happens because of some glibc/tcmalloc RSEQ incompatibility, yes, that is a problem. But don't expect them to bother much, sadly.
Now for the pointer chasing. Classic case for pointer chasing is traversing linked lists. When you have a code that continuously pops elements off free list that is classic pointer chasing. Yes it can be argued that as part of malloc/operator new you're reading memory that you're about to return and which your caller is likely to use anyways, but it adds work. Even more work and slowness it adds to "pop a bunch of per-thread list" operation that we do as part of central free list operations. And pushing elements to free list as part of free, we also have "extra" memory store, touching and potentially bringing into a caches a memory that is being freed.
For the difference in workloads. I don't have super-great understanding but real workloads have:
a) non-uniform distribution of object sizes. Sizes in ball-park of 32 bytes or so are most common and there is typical "peak"-ful distribution.
b) non-uniform distribution of object lifetimes. With quite heavy head towards very short lifetimes.
c) dependency in object lifetimes. Stuff allocated together is likely to be freed together. Also threads often run in some sort of phases, where for some time it may allocate a bunch of stuff (say, to populate hash table or something) and then does a burst of frees.
None of that is simulated by ptmalloc bench (or any other bench I've seen yet). And, yes, C++ workloads at Google (but I suspect not much beyond) are by far sized delete, which may affect some design choices.

the alignment aspect is possibly a non-issue in t-test1 benchmark considering that all the sizes are perfect power of 2 values...

Alignment aspect is the (very tiny) issue because abseil tcmalloc malloc() does more work than operator new.

it is a super interesting discussion...
I am aware about the per-cpu cache mode can possibly be disabled because of glibc... I can assure you that the bench-malloc numbers are with per-cpu caches enabled.
tbh, I took many measurements when benchmarking t-test1 only to realize that I forgot GLIBC_TUNABLES=glibc.pthread.rseq=0
and you know what? that difference was barely noticeable... It did look very much like the graphs coming alone my experimental commit https://github.com/gperftools/gperftools/pull/1554
to me, it seems like everyone who has been involved with the abseil tcmalloc has the strong belief that it must be better than the legacy project.
I have not seen any evidence of this so far... Of course, I can imagine scenarios where abseil tcmalloc shines and leaves every other allocators far behind...
ie: having hundred of threads on a 32 CPU system...
I am sure that the per-cpu cache possibly enable allocations that can optimize the CPU cache usage... and possibly despite a longer allocation time, it delivers an overall better app performance...
but sw design is always a matter of tradeoffs... if abseil tcmalloc is doing more complex stuff, it will need to give up some raw speed over the simpler design...
concerning the idea that they are unwilling to compromise outside of their very narrow use-case... I understood that point by myself... The bazel only support is another sign of this... This is coming straight from Titus Winter legacy of building the whole world at each build...
I definitely invite you to check the RSEQ ticket... RSEQ creator Mathieu Desnoyers did chime in and was not very happy about some aspect to the point where he committed something in the kernel to express his disapproval... I am sure that it will end up with a happy compromise or a win-win solution but it is very fascinating what is happening around tcmalloc... It feels like a soap opera... It is very entertaining to watch very talented people trying to break the limits to create better software...

I definitely invite you to check the RSEQ ticket... RSEQ creator Mathieu Desnoyers did chime in and was not very happy about some aspect to the point where he committed something in the kernel to express his disapproval... I am sure that it will end up with a happy compromise or a win-win solution but it is very fascinating what is happening around tcmalloc... It feels like a soap opera... It is very entertaining to watch very talented people trying to break the limits to create better software...

Note that I have not pushed anything upstream that would prevent users from misusing the rseq ABI, but I indeed have a patch ready. I want to find alternative solutions in collaboration with the tcmalloc people first before starting to enforce proper ABI use from the kernel.
Note that I've just posted a patch series improving cache-locality of RSEQ mm_cid. You may want to try running your benchmarks with it:
<EMAIL_ADDRESS>

Mathieu, if this can help to see how the patch impact the result measured in my benchmark, I can give it a shot...
yes I know that you did not commit anything that would impede anyone from misusing RSEQ... If I understood it correctly it was logging warnings when configured for debugging...
please, explain how mm_cid can impact tcmalloc performance?
I am not sure that I did see anything named mm_cid in what is shared between the kernel and RSEQ users...
and keep in mind too that in every benchmark setups, there are always much less threads than CPUs (my testing setup is not a NUMA machine) so I am not expecting a lot of tasks migrations or excessive context switches that could impact performance due to suboptimal cache optimization...

I can assure you that the bench-malloc numbers are with per-cpu caches enabled.

I encourage you to double check. At least on my system (up-to-date kernel and Debian Sid) running latest abseil tcmalloc's tcmalloc_bencmark thingy, I see that cpu_cache_active is false. Don't really want to bother to check why. It used to work.
Broadly, I don't think we have much disagreement. We broadly agree that a number of choices made as part of abseil makes it very hard if at all possible to use their malloc.
It is just when you say "per-cpu mode having to do more work", you activate my "someone is wrong on internet" mode :) I encourage you check disassembly of operator new and operator delete in both gperftools and tcmalloc and you'll see your statement isn't true (yes, despite some tiny extra overhead of storing into "active rseq" field using upstream rseq compared to what it could have been).
Also don't underestimate performance hit when per-thread mode is having when a number of threads want to grow their per-thread cache and fight each other. It really gets nasty very fast. And per-cpu mode fixes it nicely despite all the troubles. And despite that, I am not really 100% on-board yet with this approach. Even more so for their "virtual cpu id" stuff. It it just your "per-cpu mode slow" statement that provokes me some

Mathieu, if this can help to see how the patch impact the result measured in my benchmark, I can give it a shot...
yes I know that you did not commit anything that would impede anyone from misusing RSEQ... If I understood it correctly it was logging warnings when configured for debugging...

Well actually the approach I intend to take is to kill any process that corrupt RSEQ fields documented as read-only by user-space. But as I said, I want to discuss a way forward with the tcmalloc developers before introducing this kind of validation.

please, explain how mm_cid can impact tcmalloc performance?
I am not sure that I did see anything named mm_cid in what is shared between the kernel and RSEQ users...

Hrm, indeed in the tcmalloc master branch I see that struct kernel_rseq has padding where the node_id and mm_cid fields are located, and then has a union with vcpu stuff that's really not something that is present in the upstream ABI (and will conflict with future upstream kernel changes). So I guess you need a Google-internal version of the kernel to have those custom fields.
I'd recommend wiring up tcmalloc with the upstream mm_cid field instead to replace all this custom vcpu stuff. See include/uapi/linux/rseq.h for details.

and keep in mind too that in every benchmark setups, there are always much less threads than CPUs (my testing setup is not a NUMA machine) so I am not expecting a lot of tasks migrations or excessive context switches that could impact performance due to suboptimal cache optimization...

When using mm_cid without my recent cache locality improvements, even with few threads running intermittently across a few cores, you would end up having cache line bouncing due to allocation of the same concurrency ID across all used cores. My recent patch fixes that. This cache line bouncing is not present when the workload runs threads across all used cores at least every 100ms, which may explain why this slowdown does not show up in typical stress-tests, but might show up in real-life workload.

I can assure you that the bench-malloc numbers are with per-cpu caches enabled.

I encourage you to double check. At least on my system (up-to-date kernel and Debian Sid) running latest abseil tcmalloc's tcmalloc_bencmark thingy, I see that cpu_cache_active is false. Don't really want to bother to check why. It used to work.
Broadly, I don't think we have much disagreement. We broadly agree that a number of choices made as part of abseil makes it very hard if at all possible to use their malloc.
It is just when you say "per-cpu mode having to do more work", you activate my "someone is wrong on internet" mode :) I encourage you check disassembly of operator new and operator delete in both gperftools and tcmalloc and you'll see your statement isn't true (yes, despite some tiny extra overhead of storing into "active rseq" field using upstream rseq compared to what it could have been).
Also don't underestimate performance hit when per-thread mode is having when a number of threads want to grow their per-thread cache and fight each other. It really gets nasty very fast. And per-cpu mode fixes it nicely despite all the troubles. And despite that, I am not really 100% on-board yet with this approach. Even more so for their "virtual cpu id" stuff. It it just your "per-cpu mode slow" statement that provokes me some

I suppose the reason why the per-cpu cache used to be enabled and it is not anymore has something to do with a glibc upgrade. RSEQ registration by glibc has been introduced in v2.35... if tcmalloc fails to register rseq because glibc did it first, per-cpu cache is disabled. The simple fix for that is to prepend the benchmark program with GLIBC_TUNABLES=glibc.pthread.rseq=0
I even did add a print statement in bench_malloc to confirm that I was using the per-cpu cache... I am 100% confident on this...
https://github.com/google/tcmalloc/issues/260#issuecomment-2309031769
I even went as far as comparing the result with/without the per-cpu cache... the difference is noticeable but marginal... not a big deal at all (in the benchmark context).
concerning your "someone is wrong on internet" mode, ok I might be wrong... It is my best educated guess to explain the observation...
but the fact that abseil tcmalloc is slower than legacy tcmalloc remains... If this is unexpected, some investigation should be performed to find out why... this is not something that I am going to do... I am going to leave that task to the maintainers if they are interested in discovering and potentially fix a problem in that area...

Mathieu, if this can help to see how the patch impact the result measured in my benchmark, I can give it a shot...
yes I know that you did not commit anything that would impede anyone from misusing RSEQ... If I understood it correctly it was logging warnings when configured for debugging...

Well actually the approach I intend to take is to kill any process that corrupt RSEQ fields documented as read-only by user-space. But as I said, I want to discuss a way forward with the tcmalloc developers before introducing this kind of validation.

please, explain how mm_cid can impact tcmalloc performance?
I am not sure that I did see anything named mm_cid in what is shared between the kernel and RSEQ users...

Hrm, indeed in the tcmalloc master branch I see that struct kernel_rseq has padding where the node_id and mm_cid fields are located, and then has a union with vcpu stuff that's really not something that is present in the upstream ABI (and will conflict with future upstream kernel changes). So I guess you need a Google-internal version of the kernel to have those custom fields.
I'd recommend wiring up tcmalloc with the upstream mm_cid field instead to replace all this custom vcpu stuff. See include/uapi/linux/rseq.h for details.

and keep in mind too that in every benchmark setups, there are always much less threads than CPUs (my testing setup is not a NUMA machine) so I am not expecting a lot of tasks migrations or excessive context switches that could impact performance due to suboptimal cache optimization...

When using mm_cid without my recent cache locality improvements, even with few threads running intermittently across a few cores, you would end up having cache line bouncing due to allocation of the same concurrency ID across all used cores. My recent patch fixes that. This cache line bouncing is not present when the workload runs threads across all used cores at least every 100ms, which may explain why this slowdown does not show up in typical stress-tests, but might show up in real-life workload.

I going to summarize my takeaway from this reply.
that a RSEQ user app use the mm_cid field or not, prior to the fix, there was a cache line bouncing issue at the kernel level and that would impact every RSEQ enabled apps.
Is this correct?

Hrm, indeed in the tcmalloc master branch I see that struct kernel_rseq has padding where the node_id and mm_cid fields are located, and then has a union with vcpu stuff that's really not something that is present in the upstream ABI (and will conflict with future upstream kernel changes). So I guess you need a Google-internal version of the kernel to have those custom fields.
I'd recommend wiring up tcmalloc with the upstream mm_cid field instead to replace all this custom vcpu stuff. See include/uapi/linux/rseq.h for details.

you are correct. the vcpu stuff must be from an internal custom kernel. the tcmalloc github version disable the usage of that field.
See:
https://github.com/google/tcmalloc/blob/91f53c7725c8427083bb303fcca6d36691d54fe3/tcmalloc/internal/percpu.cc#L97

[...]

I going to summarize my takeaway from this reply.
that a RSEQ user app use the mm_cid field or not, prior to the fix, there was a cache line bouncing issue at the kernel level and that would impact every RSEQ enabled apps.
Is this correct?

No, it only affects cache locality of userspace data structures indexed using the rseq mm_cid field.

Okay, so to close this off. Indeed turning glibc rseq transforms abseil tcmalloc performance.
And its asm code looks reasonable and tight (I don't recognise that "btr" stuff which seems new and frame pointer stuff looks new too). It is (as expected) tighter than ours. And yet it does lose to gperftools a bit. On the other hand performance seems to depend a lot on microarchitectural details and compilers. (gcc beating clang, again; as seems to be the most common occurrence). And the difference isn't huge. Plus, as I noted they do prefetches which are strictly overhead on micro-benchmarks, but somewhat helpful on real-world performance.
Anyways, thanks for bringing this up. And summary is still: we lack trustful memory allocator benchmarks that show real-world strengths and weaknesses.

alk, it has been a pleasure to exchange with you. Yes I think that it is time to close this off and move one... but hopefully, someone will see this exchange and pick up the challenge to find a way to improve abseil tcmalloc performance...
as a departure comment, I just wanted to let you know that this morning, I have just realized that my LD_PRELOAD=/usr/lib/libtcmalloc_minimal.so on my main app fails to accomplish it purpose... the loader is encountering issue and falls back on using glibc allocator...
I have noticed this bug by looking in the /proc process virtual address mapping and seeing that tcmalloc lib was missing... The performance benefit from one allocator over another one is so subtle that this bug went unnoticed for weeks....


Added sanity checks and integrated them in CMake. Updated sample name…

…s and sample mapping.
Signed-off-by: Stefan Andritoiu<EMAIL_ADDRESS>

Merged, thank you!


Is their a way to customize PCFDEV to run on machine with 4GB ram.

Hi,
Minimum requirement of RAM for PCFDEV is 3072Mb and most of the dev machines will not have so much of free space of RAM.
Is their some way to reduce the number of features of PCF Dev Cloud Foundary so that It can run on a Laptop with 4 GB of RAM?
I would like to customize it with reduced features like no redis service, light mysql servce reduced default memory allocation for VM and reduced size of droplets and number of programs it runs(55) at the beginning.
It wont be called cloud foundry in that case because of reduced fetaure but atleast developer can run along with other enterprise applications and build a prototype to showcase to the client.
Regards
Amit Shukla

Hi Amit,
The next release of PCF Dev will allow you to disable Redis and RabbitMQ, which reduces the base system memory usage down to around 2.2GB. This doesn't include user apps, so we don't plan to reduce the minimum memory requirement of 3GB.
What other CF components would you leave out to reduce memory usage? The two largest non-essential components for running containers (Cloud Controller and UAA) are necessary to use the cf CLI. MySQL is necessary to provide a DB for Cloud Controller and UAA. We've already tuned the UAA JVM to be as small as possible.
Thanks,
Stephen

Thanks for the info Stephen.


Update onnx support to 1.7.0

Forward port of https://github.com/apache/incubator-mxnet/pull/19017
cc @josephevans

Superseded by https://github.com/apache/incubator-mxnet/pull/19573. Thank you @josephevans


[ENHANCEMENT] Custom CSI Provisioner for VM Storageclass

Is your enhancement related to a problem? Please describe.
I would like to use Harvester to provision VMs with a custom storageclass that doesn't use the Longhorn provisioner.
Describe the solution you'd like
Since Harvester itself is is a Kubernetes cluster. I wish to install my own CSI provisioner (particularly https://github.com/minio/directpv but ideally any CSI provisioner should be supported), and use it for my VMs and guest clusters.

Install custom CSI provisioner
Create custom storage class that uses this provisioner
Create VMs that use storage provisioned by custom storageclass

Describe alternatives you've considered
The docs mention using a custom storageclass but I believe this only enables customizing replica and migrate-ability of the storage.
Additional context
Our company is a blockchain node operator, and using Longhord introduces to much overhead on IOPS for our workloads. We would like to use directpv specifically as it provides near native performance on NVMe drives at the cost of fewer features (eg. no snapshotting).
Please let me know what you think the roadmap to this enhancement would be. I really believe enabling custom CSI provisioners can open up new use cases for Harvester. I would also be open to working on a contribution if you can guide me on what the steps to implement this would be.

@leovigna would this help? https://github.com/harvester/harvester/issues/2405
we are planning to allow external storage for non system partitions.
This would mean you can have additional volumes attached to your VM, which can be from a 3rd party csi.

I think so yeah. So like the OS for the VM would be on Longhorn but I can add additional partitions from any CSI.
It seems the PR is almost ready too. Exciting!


Hive ethereum/rpc suite hangs on http/CanonicalChain test

Describe the bug
Currently, every test except for http/CanonicalChain completes in the ethereum/rpc test suite. In the following run from #1348, every other test succeeds - you can see this because the http/CanonicalChain test has the ID test=6, and every other test (besides test=1, which completes only after the suite is done) has a log message with either pass=true or pass=false.
The timed out ethereum/rpc actions run:
https://github.com/paradigmxyz/reth/actions/runs/4272628614/jobs/7438032380
Local logs for this test show that the latest block is being polled, but the chain does not advance:

Simulator RPC logs

>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":{"hash":"0x23234c585795e91d7a867b20116da9a5986b8f2ba08c04390fe0a74b6181aed0","parentHash":"0x0000000000000000000000000000000000000000000000000000000000000000","sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","author":"0x0000000000000000000000000000000000000000","miner":"0x0000000000000000000000000000000000000000","stateRoot":"0x2ac377ac2d4e6fc9d38239f818814469077479601e18dadf570babdd4129cdb6","transactionsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","receiptsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","withdrawalsRoot":null,"number":"0x0","gasUsed":"0x0","gasLimit":"0x2fefd8","extraData":"0x0000000000000000000000000000000000000000000000000000000000000000658bdf435d810c91414ec09147daa6db624063790000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","timestamp":"0x1234","difficulty":"0x20000","mixHash":"0x0000000000000000000000000000000000000000000000000000000000000000","nonce":"0x0000000000000000","size":"0x26e","totalDifficulty":"0x20000","uncles":[],"transactions":[],"size":"0x273","withdrawals":null},"id":25}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
<<  {"jsonrpc":"2.0","result":"0x0","id":27}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
>>  {"jsonrpc":"2.0","id":26,"method":"eth_getBlockByNumber","params":["latest",true]}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":28,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":{"hash":"0x23234c585795e91d7a867b20116da9a5986b8f2ba08c04390fe0a74b6181aed0","parentHash":"0x0000000000000000000000000000000000000000000000000000000000000000","sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","author":"0x0000000000000000000000000000000000000000","miner":"0x0000000000000000000000000000000000000000","stateRoot":"0x2ac377ac2d4e6fc9d38239f818814469077479601e18dadf570babdd4129cdb6","transactionsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","receiptsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","withdrawalsRoot":null,"number":"0x0","gasUsed":"0x0","gasLimit":"0x2fefd8","extraData":"0x0000000000000000000000000000000000000000000000000000000000000000658bdf435d810c91414ec09147daa6db624063790000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","timestamp":"0x1234","difficulty":"0x20000","mixHash":"0x0000000000000000000000000000000000000000000000000000000000000000","nonce":"0x0000000000000000","size":"0x26e","totalDifficulty":"0x20000","uncles":[],"transactions":[],"size":"0x273","withdrawals":null},"id":26}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
<<  {"jsonrpc":"2.0","result":"0x0","id":28}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
>>  {"jsonrpc":"2.0","id":27,"method":"eth_getBlockByNumber","params":["latest",true]}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
<<  {"jsonrpc":"2.0","result":{"hash":"0x23234c585795e91d7a867b20116da9a5986b8f2ba08c04390fe0a74b6181aed0","parentHash":"0x0000000000000000000000000000000000000000000000000000000000000000","sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","author":"0x0000000000000000000000000000000000000000","miner":"0x0000000000000000000000000000000000000000","stateRoot":"0x2ac377ac2d4e6fc9d38239f818814469077479601e18dadf570babdd4129cdb6","transactionsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","receiptsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421","withdrawalsRoot":null,"number":"0x0","gasUsed":"0x0","gasLimit":"0x2fefd8","extraData":"0x0000000000000000000000000000000000000000000000000000000000000000658bdf435d810c91414ec09147daa6db624063790000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","timestamp":"0x1234","difficulty":"0x20000","mixHash":"0x0000000000000000000000000000000000000000000000000000000000000000","nonce":"0x0000000000000000","size":"0x26e","totalDifficulty":"0x20000","uncles":[],"transactions":[],"size":"0x273","withdrawals":null},"id":27}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
>>  {"jsonrpc":"2.0","id":29,"method":"eth_blockNumber"}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}
<<  {"jsonrpc":"2.0","result":"0x0","id":29}

The test is implemented here:
https://github.com/ethereum/hive/blob/master/simulators/ethereum/rpc/ethclient.go#L290
This is because the entire test suite depends on reth being able to mine blocks with clique:
https://github.com/ethereum/hive/blob/e808ab73c768c2003f5a6138fa528e01ef8e38d3/simulators/ethereum/rpc/main.go#L30-L34
So we need to figure out a way to regularly produce new blocks, or exclude this test from the CI run until we can do so properly.
Steps to reproduce
Run the hive tests using the instructions in #851, running the ethereum/rpc simulator.
Node logs
No response
Platform(s)
No response
What version/commit are you on?
No response
Code of Conduct

[X] I agree to follow the Code of Conduct

This test only checks whether or not uncles, receipts, and transactions can be fetched over RPC, and are consistent. We can implement a --mine flag, or something similar, which produces blocks on an interval. We can do this without implementing clique block production machinery.
This does not need to be a valid proof of work or clique chain, and we can use the HIVE_CLIQUE_PERIOD environment variable as the argument to the flag.

Another note - this also effects at least these tests in the rpc simulator/suite:

http/EstimateGas
http/ContractDeployment
http/ContractDeploymentOutOfGas
http/TransactionCount
http/Receipt
http/BalanceAndNonceAt
http/ABITransact
http/TransactionInBlock

Closed by https://github.com/paradigmxyz/reth/pull/1817 hopefully.

@Rjected I debugged this and it's because the provided genesis state:
"config": {
    "clique": {
      "period": 1
    },
    "chainId": 7,
    "homesteadBlock": 0,
    "eip150Block": 0,
    "eip155Block": 0,
    "eip158Block": 0
  }

does not activate Paris and we end up here:
https://github.com/paradigmxyz/reth/blob/236a10e73b3887744198e2573b73c7fa5b945a03/crates/blockchain-tree/src/blockchain_tree.rs#L378-L385
2023-06-20T15:03:47.310044Z  INFO try_insert_validated_block{block=(1, 0x412c4f63d5c760589b6de4c604ae7dd250ef18cc772ac024ad7bbc6700a07647)}: blockchain_tree: return=Err(InsertBlockError { error: Execution(Validation(BlockPreMerge { hash: 0x412c4f63d5c760589b6de4c604ae7dd250ef18cc772ac024ad7bbc6700a07647 })), hash: 0x412c4f63d5c760589b6de4c604ae7dd250ef18cc772ac024ad7bbc6700a07647, number: 1, parent_hash: 0xf24584aea2f75656cf2064042fc1660108f7438a8bc585553c32611be6155f10, num_txs: 0, .. })
2023-06-20T15:03:47.310078Z DEBUG consensus::engine: Failed to insert downloaded block err=InsertBlockError { error: Execution(Validation(BlockPreMerge { hash: 0x412c4f63d5c760589b6de4c604ae7dd250ef18cc772ac024ad7bbc6700a07647 })), hash: 0x412c4f63d5c760589b6de4c604ae7dd250ef18cc772ac024ad7bbc6700a07647, number: 1, parent_hash: 0xf24584aea2f75656cf2064042fc1660108f7438a8bc585553c32611be6155f10, num_txs: 0, .. }

not sure how to fix properly, all solutions that come to mind rn are a bit hacky, like manually adding the paris hardfork for clique...

marking as closed because this used legacy


Find the Running Median

Problem Link
https://www.hackerrank.com/challenges/find-the-running-median/problem
Coding platform
HackerRank
Programming language

[ ] C
[x] C++
[ ] Java
[ ] Python

Additional context (OPTIONAL)
Add any other context or screenshots about the feature request here.

Please review this problem and assign me this in C++

Assign this to me in python

I wanna contribute in java

I will do it in C

@ItsmeSauravSonu Are you working?

Yes
On Tue, Feb 23, 2021, 9:24 PM Nivedita Singh<EMAIL_ADDRESS>wrote:

@ItsmeSauravSonu https://github.com/ItsmeSauravSonu Are you working?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Nivedita967/Data-Structure-and-Algorithms/issues/311#issuecomment-784302739,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANZ5XWBCWCP3NKL364QYH73TAPFS7ANCNFSM4X2WS2MQ
.


Add option to hide a component when printed

By adding govuk-!-display-none-print class. Particularly relevant for navigation components.

Fixed by #127


feat: add support for wildcard members in workspace

Greate work! Could you add unit tests for your changes in lib/cargo.js?
https://coveralls.io/builds/64235354/source?filename=lib%2Fcargo.js

Great work! Could you add unit tests for your changes in lib/cargo.js?
https://coveralls.io/builds/64235354/source?filename=lib%2Fcargo.js

Hi @kaicoh, I have updated the test cases for wildcard members in workspace.
However, I encountered an issue with the absence of an else path in the existing code. To address this, I made slight modifications to ensure comprehensive test coverage without altering the intended functionality.
Please review the updated code and let me know if there are any further adjustments needed.


Inconsistency in underlay v6 support

VNET-to-VNET doc (Overview section) says that underlay v6 should be supported. While BMv2 pipeline doesn't expect to receive v6 packets (for instance VIP table). So, there is an issue in doc about underlay v6 support or there is an issue in BMv2 pipeline.
Does DASH expect that underlay v6 should be supported also, for example VIP v6, IPv6 routing, etc. ?
VIP table declaration: https://github.com/Azure/DASH/blob/main/dash-pipeline/bmv2/dash_pipeline.p4#L38
Vnet Overview doc: https://github.com/Azure/DASH/blob/main/documentation/vnet2vnet-service/vnet-to-vnet-service.md#overview
Regards,
Volodymyr

Eventually both will be supported, it is a question of effort.


Add modif. filter serialization to MultiCubeQuery

Added filter serialization to MultiCubeQuery.

LGTM
It would be good to have a unit test case to catch this.


Fix node version

Description

Related issues/PRs

Todos

[ ] Tests
[ ] Documentation
[ ] Release note

Release Note

None required

Deploy preview for calico ready!
Built with commit 42746b4394cfcb43afa6ab0e645d132169f03024
https://deploy-preview-3050--calico.netlify.com


Fix Multiple issues with _AsyncSingleShotTimer

This PR serves to fix the issues we had with 1.10.0, but currently will only highlight the issue.
I removed the check for timer.is_running() which shouldn't fail the test, yet it does.  We put a band-aid on 1.11.0, but we should still fix this.
Check the reset() function in start_transition_timer.  The ValueError() is never raised! What on earth?

I also ran into this problem addressed in https://github.com/basnijholt/adaptive-lighting/pull/529#issuecomment-1500875200


[$250] Skeleton of money request preview has no animation

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.62-0
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: @dukenv0307
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1712745451877529
Action Performed:

Open a DM
Create some money requests (4 - 5)
Open IOU report and send a message
Go back to the DM and send a message?
Logout and login again
After the DM is loaded go offline
Click on report preview to open IOU report

Expected Result:
The IOU preview skeleton should have animation like other skeletons view in the App
Actual Result:
The IOU preview skeleton is displayed without animation
Workaround:
unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence
https://github.com/Expensify/App/assets/38435837/34285702-ce88-4968-8507-3d1800b0f3aa
https://github.com/Expensify/App/assets/38435837/e54d1989-d2b4-405c-83e6-54dabe22c897

View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01f1edc0cbcada32a1
Upwork Job ID:<PHONE_NUMBER>229447680
Last Price Increase: 2024-04-15

Proposal
Please re-state the problem that we are trying to solve in this issue.
The IOU preview skeleton is displayed without animation
What is the root cause of that problem?
The problem is backgroundColor and foregroundColor of MoneyRequestSkeletonView.tsx are the same color. So this skeleton has no animation.
https://github.com/Expensify/App/blob/8ae5491b7decddaa4102850aaf89e19e2e5adf34/src/components/MoneyRequestSkeletonView.tsx#L16-L17
https://github.com/Expensify/App/blob/8ae5491b7decddaa4102850aaf89e19e2e5adf34/src/styles/theme/themes/light.ts#L12-L13
https://github.com/Expensify/App/blob/8ae5491b7decddaa4102850aaf89e19e2e5adf34/src/styles/theme/themes/dark.ts#L12-L13
What changes do you think we should make in order to solve the problem?
We should change backgroundColor or foregroundColor to another color. This can be confirmed by design team.
https://github.com/Expensify/App/blob/8ae5491b7decddaa4102850aaf89e19e2e5adf34/src/components/MoneyRequestSkeletonView.tsx#L16-L17
Or we can use theme.skeletonLHNIn and theme.skeletonLHNOut as we use for other skeletons of the App.
https://github.com/Expensify/App/blob/8ae5491b7decddaa4102850aaf89e19e2e5adf34/src/components/ReportActionsSkeletonView/SkeletonViewLines.tsx#L21-L22
What alternative solutions did you explore? (Optional)
NA

Thanks @dukenv0307! I assigned you as C+.

Proposal
Please re-state the problem that we are trying to solve in this issue.
Animation doesn't show up for the money request skeleton preview.
What is the root cause of that problem?
We use the same color for the background and the foreground of the skeleton view as used here
https://github.com/Expensify/App/blob/2d19c9529b9211fff6dc173a638f2e83b8659197/src/components/MoneyRequestSkeletonView.tsx#L16-L17
and defined here
https://github.com/Expensify/App/blob/2d19c9529b9211fff6dc173a638f2e83b8659197/src/styles/theme/themes/light.ts#L12-L13
What changes do you think we should make in order to solve the problem?

We should use the following prop values as used at other places such as here.
We should also update the same for CurrentUserPersonalDetailsSkeletonView.
https://github.com/Expensify/App/blob/02059dd0004eae2f0af60cd7b5a2af61c1978092/src/components/CurrentUserPersonalDetailsSkeletonView/index.tsx#L40-L41
backgroundColor={theme.skeletonLHNIn}
            foregroundColor={theme.skeletonLHNOut}

What alternative solutions did you explore? (Optional)
Reminder: Please use plain English, be brief and avoid jargon. Feel free to use images, charts or pseudo-code if necessary. Do not post large multi-line diffs or write walls of text. Do not create PRs unless you have been hired for this job.

Sounds good, let's see what the design team says for which color to use.

Or we can use theme.skeletonLHNIn and theme.skeletonLHNOut as we use for other skeletons of the App.

I think we should do this (mentioned at the bottom of the proposal) to stay consistent with other skeleton loaders in the app. Is there any reason why we wouldn't?

I think we should do this (mentioned at the bottom of https://github.com/Expensify/App/issues/40060#issuecomment-2048829783) to stay consistent with other skeleton loaders in the app.

I agree with this.

Yes, sorry for the delay there. All paid out.


[Bug]: Collection modify metadata raising value is not a valid dict (type=type_error.dict) error

Collection modify metadata raising value is not a valid dict (type=type_error.dict) error
Hello, I am trying to modify the metadata of an already existing collection in python. The old metadata is like
chroma_client = chromadb.Client(settings=chroma_settings)
chroma_collection = client.get_collection('langchain', embedding_function=emb)
metadatas = chroma_collection.get()['metadatas']
# metadatas = [
#  {'summary_id': 1, 'lesson_id': 1},
#  {'summary_id': 2, 'lesson_id': 3}
# ]

and now I create a new list of dicts like this
update_meta = [
  {'summary_id': 1, 'lesson_id': 1, 'updated_at': '2020-01-01 00:00:00'},
  {'summary_id': 2, 'lesson_id': 3, 'updated_at': '2020-01-01 00:00:00'}
]

Now I want to replace the old metadata with the new and run the following command
chroma_collection.modify(metadata=update_meta)
chroma_client.persist()
chroma_client = None

And until here no error is given.
Once I read the ChromaDb again
collection = client.get_collection('langchain', embedding_function=emb)

this gives the error
pydantic.error_wrappers.ValidationError: 1 validation error for Collection
metadata
  value is not a valid dict (type=type_error.dict)

Am I doing something wrong or is this an issue? I have not tried the update function since I do not want to specify any ids since all of them should be updated.
Versions
Chroma v0.3.25, Python v3.9.6, MacOS v12.6.1
Relevant log output
No response

I believe you're mixing up metadatas (a dict for each document in your collection), with the metadata attribute of  Collection
collection.modify() sets the collection's metadata, not the metadata for each document.
The error you see comes from setting metadata to an array instead of a dict (though, that should probably have thrown an error earlier..)
To modify the metadata for each document you will have to get the ids and use collection.update()
chroma_client = chromadb.Client(settings=chroma_settings)
chroma_collection = client.get_collection('langchain', embedding_function=emb)
ids = chroma_collection.get(include=[])['ids']

update_meta = [
  {'summary_id': 1, 'lesson_id': 1, 'updated_at': '2020-01-01 00:00:00'},
  {'summary_id': 2, 'lesson_id': 3, 'updated_at': '2020-01-01 00:00:00'}
]

chroma_collection.update(ids, metadatas=update_meta)

thank you @Russell-Pollari !

Thank you for the detailed answer! Closing the issue


debug logging should not be on by default

When using spanner-jdbc there are many lines output like the following:

Jun 12, 2017 9:14:41 AM io.grpc.internal.ManagedChannelImpl
INFO: [ManagedChannelImpl@3246fb96] Created with target spanner.googleapis.com:443

Could they be suppressed, or controlled by a flag?

Log levels are configured using a standard Java Logging properties file and specifying the properties file as a VM argument when starting the application. I have attached an example properties file that suppresses the INFO messages from the Google CloudSpanner API.

handlers					= java.util.logging.ConsoleHandler
nl.topicus.level			= INFO
io.grpc.internal.level		= WARNING
java.util.logging.SimpleFormatter.format=%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS %4$s %2$s %5$s%6$s%n

I have added some logic that changes the default logging level of the VM if no logging configuration has been provided.


[Rendering] MS Teams Adaptive Card not rendering full width

Target Platforms
Other
SDK Version
1.4
Application Name
Microsoft Teams
Problem Description
Adaptive cards do not appear to be rendering full width in Microsoft Teams despite Full Width being declared in the json msteams property.
When the width properly is not declared as full, the card renders at half width. When the width property is declared with the value "full", the card renders at roughly 2/3rd's width. See screenshots for example.
This seems to be consistent across the MS Teams desktop application on Windows 11 and in the MS Teams web application in both Edge and Chrome browsers. This also seems to be a consistent issue regardless of the version declared in the payload json.
In this specific case the card JSON is being POST'ed to a Teams inbound webhook (tested using Thunder Client and also a notification payload from New Relic), I've not been able to test if the same behavior is observed when sending cards from a Logic App.
Screenshots

This image shows the rendering width of a card without the width full being declared.

This image shows the rendering width of the card with width full being declared.
Card JSON
{
  "type": "message",
  "attachments": [
    {
      "contentType": "application/vnd.microsoft.card.adaptive",
      "contentUrl": null,
      "content": {
        "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
        "msteams": {
          "width": "full"
        },
        "$data": {},
        "type": "AdaptiveCard",
        "version": "1.4",
        "body": [
          {
            "type": "TextBlock",
            "text": "For Samples and Templates, see [https://adaptivecards.io/samples](https://adaptivecards.io/samples)"
          },
          {
            "type": "Image",
            "url": "https://adaptivecards.io/content/cats/1.png"
          }
        ]
      }
    }
  ]
}

Sample Code Language
No response
Sample Code
No response

I'm seeing this behavior as well. Previously full-width AdaptiveCards are now rendering at a fixed width.
Using dev tools to inspect one of my AdaptiveCards in Teams, I see the following:
.ts-adaptive-card.adaptive-card-responsive.adaptive-card-fullwidth {
    width: 60.6rem;
    max-width: 100%;
}

When I remove (or uncheck) the width attribute in that style the card shows full-width again. Not sure if that style comes from Teams or some AdaptiveCard tooling but that is causing the issue.
This was working as expected until 24-48 hours ago.

Card is not rendering full sized from yesterday after 6 pm EST. All previously sent card with same script that appeared full size started to appear short now. I thought it would start working morning but its not working. I just found this thread. Mine Is MacOS and I reinstalled MS Teams but that didn’t resolve the problem.

This issue has been copied over to Microsoft Teams repo here: https://github.com/MicrosoftDocs/msteams-docs/issues/7638

Hello,  issue occured again this morning in my company Ms Teams version <IP_ADDRESS>573 updated on 12/10/2023…. Adaptative card are not displaying full width! How do you manage regression Test at Microsoft?

Hi @Wajeed-msft I am still not able to have a full adaptive card I tried to use the schema mentioned in https://learn.microsoft.com/en-us/microsoftteams/platform/task-modules-and-cards/cards/cards-format?tabs=adaptive-md%2Cdesktop%2Cconnector-html#construct-full-width-cards.
I still see that there is white space.


It's gonna be soooo good if you can read data from a link instead of hard drive

So my suggestion is to use the link to download .osu file like this https://osu.ppy.sh/osu/3640199, and read data from it on RAM instead of the Hard drive, that's gonna save us alot of pc resources, please do that.

Providing adequate flexibility when it comes to making requests is a rabbit hole that I don't think I'm willing to go down in this crate so I don't like adding download support.

and read data from it on RAM instead of the Hard drive

You don't need the .osu file itself, you just need its content. You can download the bytes yourself and then just pass a &[u8] to Beatmap::parse instead of storing them as a file.

So weird to say but I'm using your rosu-pp as like a local website, so my java discord bot can get fetch data from it, but i think you can do it bro, i mean the data is just a bunch of bytes.

@CaoTrongThang i think you can do it bro, i mean the data is just a bunch of bytes.

@CaoTrongThang i think you can do it bro, i mean the data is just a bunch of bytes.

That wasn't what I meant, he capable to do it more than I, both could give the same result but read data from a buffer is better than saving it to a file and then read it again, i mean it's an extra step

Glad you found a way to use the lib even though downloading files yourself is a little more cumbersome.
In case you're using the rosu-pp-js or rosu-pp-py bindings, I believe they currently do not support passing data as bytes and instead indeed require a file path which is unfortunate; I will be adding a way to pass them as bytes to make this at least a little bit more convenient (https://github.com/MaxOhn/rosu-pp-js/issues/2) :+1:
Downloading files in of itself is not hard but covering all or at least a decent amount of the ways to do so not only requires a lot of work but also shifts the focus of this crate away from what I intend it to be, a pure difficulty and performance calculation tool.
This doesn't even account the fact that providing a way to easily download files on the fly would encourage a more careless interaction with the osu!api - that is, unless I put even more work into the whole system to handle file caching.
So as it currently stands, download support won't be added.


Crash when writing to IInteger node that defines range larger than int32 in Attribute Controller

Describe the bug
When trying to write to an IInteger node that defines a range larger than int32 (via the Attribute Controller window), the GUI will crash as soon as you double click the field where the value is to be filled in. I found this in an Integer node that defines the following range:
        <Integer Name="x">
            <pValue>Reg_x</pValue>
            <Min>0</Min>
            <Max>4294967295</Max>
        </Integer>

The crash message is:
WARNING: Traceback (most recent call last):
  File "XXX/lib/python3.8/site-packages/harvesters_gui/_private/frontend/pyqt5/feature_tree.py", line 339, in createEditor
    w.setRange(feature.min, feature.max)
OverflowError: argument 2 overflowed: value must be in the range -2147483648 to<PHONE_NUMBER>
Aborted (core dumped)

To Reproduce
Steps to reproduce the behavior:

Start harvesters GUI, connect to a camera with an IInteger node with range > int32
Open the Attribute Controller window
Go to the troublesome IInteger node and double click the value, such that you can write your own value
Crash on the double click (never get a chance to input a value)

Expected behavior
No crash, expecting compatibility with 64-bit int
Desktop:

OS: Ubuntu 20
Version: 1.3.8

Additional information
The problem is that the QSpinBox object used in the GUI is limited to int32: https://doc.qt.io/qt-5/qspinbox.html#maximum-prop but per the GenICam standard (I'm looking at v2.1.1) an integer node its range is 64-bits.

Hi, I happened to run into this crash also and have been using this commit as a fix.
Have been meaning to get a PR created for this, but just hadn't done it yet.  I'm curious to know if this fixes the issues you're seeing.  Thanks!

Good news, progress!  Thanks for testing.
Yeah, the changes in this commit are probably still a bit rough and need a bit more rigor.  I'll try to look at the additional issues found with your testing... appreciate you flagging these.


Language server issues in outline pane

Problem Description
Several "symptoms" involving the outline pane; I'm unsure how best to document, so looking for direction:

The outline often collapses. e.g. I expand all, then when working in the code it collapses.
The server busy spinner (next to the dropdown in the pane) randomly starts spinning. Sometimes this is accompanied by the editor and console hanging

What steps reproduce the problem?

Open a project
Work in some code

What is the expected output? What do you see instead?
Outline retains state (what is collapsed or not remains collapsed or not)
UI does not hang with spinning server
Paste Traceback/Error Below (if applicable)

<many more of these messages>
WARNING:spyder.plugins.completion.languageserver.client:The send queue is full! Retrying...
WARNING:spyder.plugins.completion.languageserver.client:The send queue is full! Retrying...
WARNING:spyder.plugins.completion.languageserver.client:The send queue is full! Retrying...
WARNING:spyder.plugins.completion.languageserver.client:The send queue is full! Retrying...
WARNING:spyder.plugins.completion.languageserver.client:The send queue is full! Retrying...
WARNING:spyder.plugins.completion.languageserver.client:The send queue is full! Retrying...
WARNING:spyder.plugins.completion.languageserver.client:The send queue is full! Retrying...

<a few of these messages>
QProcess: Destroyed while process ("c:\\users\\bbate\\envs\\keras-gpu-6\\scripts\\pythonw.exe") is still running.

 You said:

The outline often collapses. e.g. I expand all, then when working in the code it collapses.

Could you post a simple, reproducible example of that? Otherwise is hard for us to know how to tackle this problem.

The server busy spinner (next to the dropdown in the pane) randomly starts spinning. Sometimes this is accompanied by the editor and console hanging

I'll try to come up with something for our next version (4.2.1), to be released next week. Otherwise, I'll take a look at it for sure for 4.2.2.

Minimum repro code:
# -*- coding: utf-8 -*-
"""
Created on Fri Dec 11 12:18:17 2020

@author: bbate
"""
# %% block 1
#
def foo(x):
    result = x**2
    return result
#
# %% block 2
#
def bar(y):
    result = y / 2
    return result
#
# %% block 3
#
def fubar(z, x, y):
    result = z + x * 10 - y
    return result
#
# %% main code
#
x = 10
y = 20
z = 30
print(fubar(z, foo(x), bar(y)))

Attached video shows the outline collapse on editing the code

Expand the outline
put cursor on line 29 (assignment to z)
hit return

outline collapses
No errors in the internal console for this issue
The other issue (hanging, internal server errors) I have not found a simple repro yet.
IMG_1735.zip

Hey @eafpres, about the problems you reported:

The outline often collapses. e.g. I expand all, then when working in the code it collapses.

I can't reproduce this error in our latest 4.x branch. So please wait until you can install Spyder 4.2.2 and let us know if the error is still there.

The server busy spinner (next to the dropdown in the pane) randomly starts spinning. Sometimes this is accompanied by the editor and console hanging

I just merged PR #13864, which should help with that. Again, please test it once 4.2.2 is out.

please test it once 4.2.2 is out.

Okay, will do. Thank you.

@eafpres 4.2.2 is out, FYI

eafpres 4.2.2 is out, FYI

As far as I can tell, the collapsing issue is solved (actually at least from 4.2.1).
The spinning issue is there, but I need to attempt some kind of reproducible example in 4.2.2
If anything new / else appears, I'll open a new issue.
More to come, and as always, thanks.

So when I updated to 4.2.2 a code I'm working on (3200++ lines in the editor)  started crashing constantly. This was not happening in 4.2.1. I found that if I commented out a bunch of code analysis errors like so:
# pylint: disable=invalid-name, too-many-lines, line-too-long, wrong-import-position, wrong-import-order, ungrouped-imports, no-name-in-module, no-member, unidiomatic-typecheck, import-outside-toplevel, protected-access, self-assigning-variable, too-many-branches

The problem in 4.2.2 goes away.
It does "appear" the spinning thing in the (long) outline is "better" in 4.2.2 than 4.2.1.  However, I think there are issue when there are a lot of outline elements or the code is large or something along those lines.
Right now I can't use 4.2.2 due to another issue I submitted-error dialog popping up constantly in the editor. In 4.2.1, the outline gets confused often as to exactly where in the code things are, so I have to not use it when the spinning thing is there, or it will go to some place other than where I want to go.  I'm hoping that's more reliable in 4.2.2, and look forward to testing that more.

Sorry you're running into these issues! Strange; disabling Pylint errors shouldn't have any effect, since Spyder doesn't use Pylint for real-time linting, only on-demand. The fact that you're seeing one, unless there's something else going on here, could possibly be indicative of Python Language Server using the Pylint backend for linting even when its never used, since pylint is really slow compared to pyflakes/pycodestyle (which Spyder uses), slowing things down considerably on longer/more complex files.
Could you describe how you updated Spyder from 4.2.1 to 4.2.2? With Anaconda (default channel or conda forge?), with the installer, or another method? Based on what you've told us, it seems likely the change was caused by an upgrade to the PyLS or PyLS-Spyder version, as opposed to Spyder itself—if you're able, can you downgrade to 4.2.1 without modifying any other dependencies (or the minimum possible) and check if the problem is still present? Finally, is this problem the same as you've reported on #14835 , and if not, does it occur to the same copy of Spyder (i.e. so your troubleshooting output there would be valid here)? Thanks!

To be clear, I don't really know what is going on, but I observed this change.  The popup error thing goes away with exactly the same code files (same project) just exiting Spyder, running pip install spyder==4.2.1 in Powershell, and restarting Spyder.
So I think I have done what you asked--only changed the Spyder version and the error went away.  The issue I raised as not being above to completely verify this issue in 4.2.2 is indeed reported in 14835; I wanted that separate as I think it is (not sure).
I don't know what is happening when the outline panel is spinning, other than it does not know about code changes and may go to the wrong line in the code if I click on the outline.
I will attempt to repro this behavior regarding commenting out pylint errors when I have a bit of time.  I realize my code has a lot of moving pieces and this could be subtle.

So when I updated to 4.2.2 a code I'm working on (3200++ lines in the editor) started crashing constantly

What do you mean exactly by this? What are the crashes you're observing?

So when I updated to 4.2.2 a code I'm working on (3200++ lines in the editor) started crashing constantly

What do you mean exactly by this? What are the crashes you're observing?

The entire application suddenly terminates.  I can use the recovery file depending on the circumstances. Because of the way it crashes, I have no internal console etc. I can try to formulate a repro, when I get more time.

@eafpres Okay, I assume you don't get any relevant output in the Powershell console window you ran it from? Can you try running in debug mode, with spyder --debug-info verbose?

I assume you don't get any relevant output in the Powershell console window you ran it from? Can you try running in debug mode, with spyder --debug-info verbose?

Correct. No Powershell output. I’ll get back to repro this once I’m past a Wednesday project milestone.

Today I tested with my large (ish) code (3000+ lines) in 4.2.1.
Changing the pylint comment seems to have no effect; my observation may have been coincidental. I also tested re-running the code analysis without the items commented out, and this had no effect on performance.
Running in debug mode produced no messages when the outline panel was spinning (or any messages)
I confirm that if I make changes, the spinner starts
It "seems" (hard to validate) that if the change is flagged as an error, the spinning lasts longer
During spinning, if the # of code lines was changed, jumping to a location using the outline goes to the wrong place; i.e. the number of lines or however the outline syncs to the code becomes un-matched

Closing because this error hasn't been reported since.


Prepare bn layer tc

This PR add a python function that generates bn_layer forward/backward
pass.
Return value's order & contents are subject to change for the time
being.
Self evaluation:

Build test: [X]Passed [ ]Failed [ ]Skipped
Run test: [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Jihoon Lee<EMAIL_ADDRESS>How to evaluate:
Run python function

:memo: TAOS-CI Version: 1.4.20191203. Thank you for submitting PR #251. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://nnsuite.mooo.com/.

We generate a report if there are dangerous coding constructs in your code. Please read http://nnsuite.mooo.com/nntrainer/ci/repo-workers/pr-checker/251-202007021957210.27779698371887-f27521d156e20a085148d9eaaab9ddeb221b0cff/report/pylint_result.txt.


Push p4runtime-shell package to PyPI in CI

We push every time the main branch is updated and every time a release tag is created.
For #99

Codecov Report

Merging #100 (7ffde41) into main (aa7e7e3) will decrease coverage by 0.07%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #100      +/-   ##
==========================================
- Coverage   70.32%   70.24%   -0.08%
==========================================
  Files           8        8
  Lines        2463     2470       +7
==========================================
+ Hits         1732     1735       +3
- Misses        731      735       +4

Impacted Files
Coverage Δ

p4runtime_sh/p4runtime.py
60.83% <0.00%> (-0.32%)
:arrow_down:

p4runtime_sh/shell.py
70.10% <0.00%> (-0.05%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


In the future, consider using modern managed pointers.

This code uses all "naked" pointers. The potential problem with this is that you forget to delete them, or you forget to initialize them. Or you shared objects ownership between multiple objects.
// so now you do
 router = new MidiRouter(0);

// you could instead do:
std::shared_ptr<MidiRouter> router;
router = std::make_shared<MidiRouter>();
// or use std::unique_ptr<T>

I wouldn't change all your code right now, but it's something to think about in the fugure.
Of course many Rack APIs will give you a pointer to something, but you don't own it, so you would not want to use one of these kinds of pointers, in effect messing up memory ownership>

This is a great idea. I studied the smart pointers quite a bit before getting started but didn't have the confidence to put them to use. I'll keep this one open because it's worth doing at some point.

I usually use std::shared_ptr for everything. Some ppl who really care about the last drop of efficiency use std::unique_ptr. unique_ptr is what you are supposed to use if there is only one client that needs to point to an object. But it's slightly more difficult to use (for me, anyway).

Thanks for the encouragement!


Update celo-blockchain to v1.1.1, attestation service to v1.1.0, add backup_node for chaindata backups

Description
Update celo-blockchain to v1.1.1: also upgrades underlying golang.  Removes genesis and bootnode cruft, as this is now handled automatically.
Update attestation service to v1.1.0
Add a backup_node instance type for chaindata backups, which does a nightly tarball backup, and rsync backups every 4 hours.  Previously this was running on the txnode used by the attestation service, which would make attestation unavailable during backups.  This provides higher integrity, fresher backups without any downtime to attestation service.  Backups are stored in GCS in a versioned bucket.
Other changes
Add scripts for handling generating proof of possessions, cUSD exchanges, etc.
Tested
On both baklava and mainnet.
Related issues
n/a
Backwards compatibility
n/a

Updated blockchain to v1.2.3 and fixed some bugs in the syslog and backup/restore code.


Demo code

Hello,
Great addon,
Could you publish the demo code ?
Thank you.

Hi,
I have pushed the demo site code to this repo under the demo folder.
Thanks

Hello,
Great addon,
Could you publish the demo code ?
Thank you.

Hi,
I have pushed the demo site code to this repo under the demo folder.
Thanks

Thank you very much. It was pretty helpful for me to integrate the modal.


How to publish plain text lyrics?

I would like to publish plain text lyrics, because I sometimes don't want to go through an entire album to sync lyrics, so it could better that at least plain text lyrics were available for other people to later easier sync them.
I can download plain text lyrics, but I can't publish it (or I don't see where).

you can do it directly through api if you know how. however, the lrcget client does not allow submitting plain lyrics as of writing

Is this by design? Maybe you could add a button for it

yes you could do it if you can create a PR

Resolved in v0.3.0, thank you again for your work!


Build fails: ERROR: infinite symlink expansion detected

I'm getting this error on FreeBSD, it fails in jemalloc even though I answered "n" when it asked if to build it:
workspace: /usr/ports/science/py-tensorflow/work-py27/tensorflow-1.10.0-rc1
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
ERROR: infinite symlink expansion detected
[start of symlink chain]
/usr/ports/science/py-tensorflow/work-py27/tensorflow-1.10.0-rc1/bazel_ot/2cc1156aa366d993caf19cae8ee5ba0d/external/org_tensorflow
/usr/ports/science/py-tensorflow/work-py27/tensorflow-1.10.0-rc1
[end of symlink chain]
ERROR: /usr/ports/science/py-tensorflow/work-py27/tensorflow-1.10.0-rc1/tensorflow/tools/pip_package/BUILD:123:1: error loading package '@jemalloc//': Encountered error while reading extension file 'third_party/common.bzl': no such package '@org_tensorflow//third_party': Could not access /usr/ports/science/py-tensorflow/work-py27/tensorflow-1.10.0-rc1/bazel_ot/2cc1156aa366d993caf19cae8ee5ba0d/external/org_tensorflow: Infinite symlink expansion and referenced by '//tensorflow/tools/pip_package:licenses'
ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package' failed; build aborted: error loading package '@jemalloc//': Encountered error while reading extension file 'third_party/common.bzl': no such package '@org_tensorflow//third_party': Could not access /usr/ports/science/py-tensorflow/work-py27/tensorflow-1.10.0-rc1/bazel_ot/2cc1156aa366d993caf19cae8ee5ba0d/external/org_tensorflow: Infinite symlink expansion
INFO: Elapsed time: 5.260s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (75 packages loaded)

Looks similar to https://github.com/tensorflow/tensorflow/issues/5732 but there are no links mentioned there.

Nagging Assignee @poxvoculi: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

The FreeBSD port has been deleted because it fails to build, in part due to this reason.
This needs to be resolved for the port to be reinstated.

Nagging Assignee @poxvoculi: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

I will mark this as community support, as we do not have freebsd systems to experiment on. So no official support for FreeBSD exist.
In the similar issue you saw, it looks like the problem was an existing symlink in the system.
Is it possible you are seeing a similar issue?

On second look, to your issue looks different, but I am guessing this is a bad interaction between bazel and the OS.

Hi,
Cannot really help but I've seen this behavior when $HOME is set to a directory inside source directory. You might want to try overriding HOME env on bazel build call to /tmp/something and see if it helps...
Bazel.................................................

I am facing issue like this on ubuntu 18.04(LTS)
when i am installing tensorflow from source https://www.tensorflow.org/install/source
on running bellow line i get error
./configure
Extracting Bazel installation...
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
You have bazel 0.23.0 installed.
Please downgrade your bazel installation to version 0.21.0 or lower to build TensorFlow! To downgrade: download the installer for the old version (from https://github.com/bazelbuild/bazel/releases) then run the installer.

I've encountered same problem when I set custom bazel output directory with manually setting $TEST_TMPDIR.
I still not understand fully, but I found a work-around at least for my case: if you want to set your own bazel output directory, set it outside of tensorflow root directory.
This issue seems to occur when setting bazel output directory inside(i.e. subdirectory) or tensorflow root directory. Symlink ${bazel_output_dir}/external/org_tensorflow is supposed to point tensorflow root directory, but if it is inside of tensorflow root itself, it seems to occur some kind of infinite symlink expansion.
In my case, I encountered this issue as building T/F using docker. If you are in the same case, just make docker volume and mount it to /root/.cache/bazel which is default bazel output directory. All artifacts can be stored in docker volume in this way.

I've encountered same problem

Problems in bazel aren't fixed for months or years.

Hi @yurivict !
1.x versions are not supported anymore. Feel free to test build from source instruction from 2.9 . Thank you!


Upgrade to Argo Rollouts v1.7.2

Update to latest release of Argo Rollouts: https://github.com/argoproj/argo-rollouts/releases/tag/v1.7.2
Before merging this PR, ensure you check the Argo Rollouts change logs and release notes:

ensure there are no changes to the Argo Rollouts install YAML that we need to respond to with changes in the operator
ensure there are no backwards incompatible API/behaviour changes in the change logs

There is a small change in the ClusterRole of v1.7.2. Should we include it in the operator? https://github.com/argoproj/argo-rollouts/compare/v1.7.1...v1.7.2
Everything else looks good.

There is a small change in the ClusterRole of v1.7.2. Should we include it in the operator? https://github.com/argoproj/argo-rollouts/compare/v1.7.1...v1.7.2

The permission already exists https://github.com/argoproj-labs/argo-rollouts-manager/blob/06589e6aaac19504d53db37ed84c71324add2267/controllers/resources.go#L891

Thanks @chetan-rns!


Make getUnicodeCandidates public

Calling extractEmojis only returns a list of strings. In order to provide more information about emojis found, access to the UnicodeCandidates would be useful. Changing the getUnicodeCandidates method from protected to public makes this possible and should have little impact.

Coverage increased (+0.3%) to 93.498% when pulling 412780f8a76633a9ada6d8dcec3129b07caf91aa on viktornordling:patch-1 into 8cf5fbe0d7c1020b926791f2b342a263ed07bb0f on vdurmont:master.

Build failures seem unrelated to the change.

Hello @viktornordling !
First, let me apologize for the lack of answer on my side. I'm dusting off the library and upgrading it so I'm going through the PRs right now.
I am hesitant to expose the getUnicodeCandidates method as it is something "internal" that was not meant to be publicly accessible. I'm always very conservative when it comes to adding more public APIs because then we need to maintain them.
It would be very helpful if you could explain a little bit more the kind of usage you see for this function to help me understand why this would be a good thing.
Thanks in advance!

Thanks @vdurmont. No worries about the delay, I understand. :) So we use this library to find emojis in documentation and we fail the build in case any are found, but it's quite annoying when this happens because the message doesn't actually say which emoji was used. This is why I want to make this method public. Hope that explains my reasoning.
Best,
Viktor


Attempting to create new account fails with error message 'Insufficient Delegation'.

I've tried creating an account several times.  My account has 203 liquid steem at the time of account creation attempt.  I can't understand what the issue is.
current_delegation >= target_delegation: Inssufficient Delegation 61759.461351 VESTS required, 61759.461350 VESTS provided. account_creation_fee=0.200 STEEM o.fee=0.200 STEEM o.delegation=59700.812639 VESTS

61759.461351 VESTS required
61759.461350 VESTS provided
This happens occasionally and is a bug in dsteem, and happens because the amount of VESTS required slowly increases over time. If you wait too long on the final screen, this can happen.
That explains why, but we'll have to come up with a fix for it.

Thanks for the reply.
When I attempted to create the account, I was in a power down.  Today I stopped the power down and was able to successfully create a new account.


I AM DEADDDD!!!!!!

Alignoskii cheythoskii sireiiii

Well done ma bwoy

I'm the GLAAADDDD
On Tue, May 12, 2020 at 12:53 PM Sreeram Venkitesh<EMAIL_ADDRESS>wrote:

Well done ma bwoy
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/fillerInk/ieee-covid-poster-generator/pull/8#issuecomment-627162753,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKYL3N2T3AOGUJMF5V5DMU3RRD2OZANCNFSM4M6DGG6Q
.