Bug Parameter ' p_node ' is null. when working in bigger project

Godot version:
3.1 89f8b84
OS/device including version:
Windows 10
Issue description:
When my project become bigger then I noticed that editor sometimes print this error:
ERROR: is_a_parent_of: Parameter ' p_node ' is null.
   At: scene/main/node.cpp:1404

every time when I click at some key at keyboard, but only until I click by mouse at editor.
It happens rarely, maybe 1-3 times per day.
Usually it happens in Script Editor, probably when it lost focus.

This bug seems to not appear in empty project but in my project(which I can't attach here but maybe someone could reproduce this) it happens when I trying save with key combination CTRL + SHIFT + S and exit this window, then every single pressed key should show this error
https://streamable.com/xjd8g

Yeah, it still sometimes happens(less than few moths ago, however), thank for report

@reduz
Version: 3.1-rc1_x11.64
Same issue when i start godot from the terminal and terminated with this command: ./Godot_v3.1-rc1_x11.64 after then, CTRL+C

I am getting it too:
 scene/main/node.cpp:1432 - Parameter ' p_node ' is null.

+1
scene/main/node.cpp:1418 - Parameter ' p_node ' is null
Godot version: v3.2.beta1.official
OS: Mojave 10.14.6

I can confirm, that there is a reliable way to reproduce the same issue in the latest 3.2 build, and it does not matter what size is your project. You can make a new one, save the scene (because you need to run it), attach an empty script (only extends Node statement), and you can reproduce it.
The catalyst is the search panel. If it is open before your scene is run, then any key press after that scene is stopped is going to produce an error. Simply holding any keyboard button triggers the message multiple times. As soon as you click on editor window, the issue is gone.

So, after a bit of investigating, I can tell, what's causing the problem, however I'm not sure of a solution.
Issue comes from editor\code_editor.cpp method FindReplaceBar::_unhandled_input. Code reaches condition
if (k->is_pressed() && (text_edit->has_focus() || vbc_lineedit->is_a_parent_of(get_focus_owner()))) {

It first checks, whether the event was triggered on button press, and not release. Then it tests if a text input field for the search panel has focus, directly or indirectly. For indirect approach get_focus_owner() method is used. Problem here is that after a scene was started, for whatever reason, there is no focus owner and get_focus_owner() returns NULL, which then causes the error in OP.
Now, there is no problem with focus from simply alt-tabbing or otherwise changing an active window. It seems to be directly related to the fact, that we run a scene. Additionally, it turns out, there is no need to stop your running scene, to have this issue. Just return focus to the Editor window after you start a scene using window header, alt-tab, or system dock, and try to press any button on your keyboard.
One idea I have is that the last focus owner before a scene is run is this little button:

This button and other buttons in that panel change as soon as a scene is started. At a glance, it seems to be a simple icon change. But there may be something that I don't see that happens at the same moment and can affect focus owner.
We can handle NULL value of get_focus_owner() explicitly in FindReplaceBar, just to get rid of the error. I'm just not sure, though, if there is an underlying problem, that is going to be covered by it.

This bug still exists, version 3.2.1.

It also happens to me quite often and only restarting the project gets rid of it. In version 3.2.1.

I have the same problem:
 scene/main/node.cpp:1430 - Parameter "p_node" is null.

I can trigger it by following the steps above. At one point I was able to trigger it continuously by simply holding control while in the script editor. I haven't been able to reproduce that yet though.

I have the same problem,
Happens when I close the game and press control, if I click the editor and press control again it doesn't appear anymore.


JVM Crash on Amazon Linux 6.1

Please provide a brief summary of the bug
Hey Team,
We have been having an issue with JVM startup on our Control-M system which uses OpenJDK 11.0.21+9. JVM crashes immediately after startup.
The issue happens in the native code, particularly on libstdc++.so.6+0xac96c (which points to /usr/lib64/libstdc++.so.6.0.29). The JVM system is Control-M system running on version <IP_ADDRESS>. Please see below the hs_error file
`
A fatal error has been detected by the Java Runtime Environment:

SIGSEGV (0xb) at pc=0x00007f78678ac96c, pid=125270, tid=125270

JRE version: OpenJDK Runtime Environment Temurin-11.0.21+9 (11.0.21+9) (build 11.0.21+9)
Java VM: OpenJDK 64-Bit Server VM Temurin-11.0.21+9 (11.0.21+9, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
Problematic frame:
C  [libstdc++.so.6+0xac96c]  __gxx_personality_v0+0x22c

Core dump will be written. Default location: Core dumps may be processed with "/bin/false" (or dumping to /control/ctmem/ctm_em/log/bimsrv/core.125270)

If you would like to submit a bug report, please visit:
https://github.com/adoptium/adoptium-support/issues
The crash happened outside the Java Virtual Machine in native code.
See problematic frame for where to report the bug.

---------------  S U M M A R Y ------------
Command Line: -Xmx600m -XX:+CrashOnOutOfMemoryError abort exit vfprintf -XX:OnOutOfMemoryError=/control/ctmem/ctm_em/bin/heapMemErr BIM 600 0 5 300 125270 -Djava.security.egd=file:/dev/./urandom
Host: AMD EPYC 7R13 Processor, 4 cores, 30G, Amazon Linux release 2023 (Amazon Linux)
Time: Mon Feb 12 00:24:19 2024 UTC elapsed time: 0.487046 seconds (0d 0h 0m 0s)
---------------  T H R E A D  ---------------
Current thread (0x00000000025ec000):  JavaThread "main" [_thread_in_native, id=125270, stack(0x00007ffef7b20000,0x00007ffef7c20000)]
Stack: [0x00007ffef7b20000,0x00007ffef7c20000],  sp=0x00007ffef7c1dbd0,  free space=1014k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libstdc++.so.6+0xac96c]  __gxx_personality_v0+0x22c
C  0x00007ffef7c1e020
siginfo: si_signo: 11 (SIGSEGV), si_code: 128 (SI_KERNEL), si_addr: 0x0000000000000000
Register to memory mapping:
RAX=0x0000000000000006 is an unknown value
RBX=0x0000000001ec0868 points into unknown readable memory: 0x474e5543432b2b00 | 00 2b 2b 43 43 55 4e 47
RCX=0x0000000000000001 is an unknown value
RDX=0x0000000001ec0888 points into unknown readable memory: 0x00007f78768a4350 | 50 43 8a 76 78 7f 00 00
RSP=0x00007ffef7c1dbd0 is pointing into the stack for thread: 0x00000000025ec000
RBP=0x00007ffef7c1dda0 is pointing into the stack for thread: 0x00000000025ec000
RSI=0x00007f787668a6a3: _ZTS8CTDError+0x0000000000000000 in /control/ctmem/ctm_em/lib/libbmc-corekit.so.4.0 at 0x00007f7876600000
RDI=0x00007f787668a6a3: _ZTS8CTDError+0x0000000000000000 in /control/ctmem/ctm_em/lib/libbmc-corekit.so.4.0 at 0x00007f7876600000
R8 =0x00007f78766a31f3: <offset 0x00000000000a31f3> in /control/ctmem/ctm_em/lib/libbmc-corekit.so.4.0 at 0x00007f7876600000
R9 =0x00007f78766a31f8: <offset 0x00000000000a31f8> in /control/ctmem/ctm_em/lib/libbmc-corekit.so.4.0 at 0x00007f7876600000
R10=0x0000000000001c00 is an unknown value
R11=0x0000000000000038 is an unknown value
R12=0x00007f78768a4370: _ZTI8CTDError+0x0000000000000000 in /control/ctmem/ctm_em/lib/libbmc-corekit.so.4.0 at 0x00007f7876600000
R13=0x0000000000000002 is an unknown value
R14=0xfffffffffffffffd is an unknown value
R15=0x00007f7876685061: _ZN24Osl_UnicodeConfiguration13ReadLocaleIniEv+0x0000000000001c4b in /control/ctmem/ctm_em/lib/libbmc-corekit.so.4.0 at 0x00007f7876600000
Registers:
RAX=0x0000000000000006, RBX=0x0000000001ec0868, RCX=0x0000000000000001, RDX=0x0000000001ec0888
RSP=0x00007ffef7c1dbd0, RBP=0x00007ffef7c1dda0, RSI=0x00007f787668a6a3, RDI=0x00007f787668a6a3
R8 =0x00007f78766a31f3, R9 =0x00007f78766a31f8, R10=0x0000000000001c00, R11=0x0000000000000038
R12=0x00007f78768a4370, R13=0x0000000000000002, R14=0xfffffffffffffffd, R15=0x00007f7876685061
RIP=0x00007f78678ac96c, EFLAGS=0x0000000000010297, CSGSFS=0x002b000000000033, ERR=0x0000000000000000
TRAPNO=0x000000000000000d
Top of Stack: (sp=0x00007ffef7c1dbd0)
0x00007ffef7c1dbd0:<PHONE_NUMBER>ec0868<PHONE_NUMBER>683738
0x00007ffef7c1dbe0:   00007f78766a2dd4<PHONE_NUMBER>000000
0x00007ffef7c1dbf0:   00007f7876685061 00007ffef7c1dc40
0x00007ffef7c1dc00:   00007ffe00000000 00007ffef7c1dda0
Instructions: (pc=0x00007f78678ac96c)
0x00007f78678ac86c:   4c 8d 74 24 60 4c 8d 6c 24 68 66 2e 0f 1f 84 00
0x00007f78678ac87c:   00 00 00 00 0f b6 ac 24 99 00 00 00 31 f6 89 ef
0x00007f78678ac88c:   e8 5f fd ff ff 4c 89 e2 4c 89 f9 89 ef 48 89 c6
0x00007f78678ac89c:   e8 cf fa ff ff 0f b6 ac 24 99 00 00 00 31 f6 49
0x00007f78678ac8ac:   89 c4 89 ef e8 3b fd ff ff 4c 89 e2 4c 89 f1 89
0x00007f78678ac8bc:   ef 48 89 c6 e8 ab fa ff ff 0f b6 ac 24 99 00 00
0x00007f78678ac8cc:   00 31 f6 49 89 c4 89 ef e8 17 fd ff ff 4c 89 e9
0x00007f78678ac8dc:   4c 89 e2 89 ef 48 89 c6 e8 87 fa ff ff 31 ff 31
0x00007f78678ac8ec:   c9 49 89 c4 41 0f b6 14 24 49 83 c4 01 48 89 d0
0x00007f78678ac8fc:   83 e0 7f 48 d3 e0 83 c1 07 48 09 c7 84 d2 78 e4
0x00007f78678ac90c:   48 8b 44 24 58 48 03 44 24 70 48 39 d8 0f 86 b1
0x00007f78678ac91c:   00 00 00 f6 44 24 0c 01 0f 84 c6 00 00 00 45 31
0x00007f78678ac92c:   ff 45 31 c0 45 31 ed 48 83 7c 24 18 01 66 49 0f
0x00007f78678ac93c:   6e c0 b8 06 00 00 00 66 48 0f 3a 22 44 24 10 01
0x00007f78678ac94c:   0f 87 83 fe ff ff 48 8b 1c 24 48 8b 44 24 50 44
0x00007f78678ac95c:   89 6b dc 48 89 43 f8 b8 06 00 00 00 4c 89 7b f0
0x00007f78678ac96c:   0f 29 43 e0 e9 60 fe ff ff 45 85 ed 0f 88 a1 00
0x00007f78678ac97c:   00 00 66 90 48 8b 14 24 31 f6 48 89 ef e8 22 0d
0x00007f78678ac98c:   ff ff 49 63 d5 be 01 00 00 00 48 89 ef e8 12 0d
0x00007f78678ac99c:   ff ff 4c 89 fe 48 89 ef e8 87 46 ff ff b8 07 00
0x00007f78678ac9ac:   00 00 e9 22 fe ff ff 4c 8b 7c 24 68 48 8b 6c 24
0x00007f78678ac9bc:   30 4d 85 ff 75 3e b8 08 00 00 00 e9 09 fe ff ff
0x00007f78678ac9cc:   0f 1f 40 00 48 03 44 24 60 48 39 d8 77 d9 4c 39
0x00007f78678ac9dc:   a4 24 90 00 00 00 0f 87 98 fe ff ff e9 32 ff ff
0x00007f78678ac9ec:   ff 0f 1f 00 f6 44 24 0c 08 74 6f e8 a4 46 ff ff
0x00007f78678ac9fc:   0f 1f 40 00 4c 03 7c 24 78 48 85 ff 75 6e 4d 85
0x00007f78678aca0c:   ff 74 b3 44 8b 6c 24 0c 41 83 e5 01 75 a8 e9 61
0x00007f78678aca1c:   ff ff ff 48 8b 74 24 10 48 8d 54 24 70 48 89 ef
0x00007f78678aca2c:   e8 2f fc ff ff 0f b6 bc 24 98 00 00 00 48 89 ee
0x00007f78678aca3c:   e8 af fb ff ff 0f b6 bc 24 98 00 00 00 48 89 ee
0x00007f78678aca4c:   48 89 84 24 80 00 00 00 e8 97 fb ff ff 48 8b 1c
0x00007f78678aca5c:   24 48 89 43 f0 e9 1a ff ff ff 48 83 7c 24 18 01
Stack slot to memory mapping:
stack at sp + 0 slots: 0x0000000001ec0868 points into unknown readable memory: 0x474e5543432b2b00 | 00 2b 2b 43 43 55 4e 47
stack at sp + 1 slots: 0x0000000101683738 is an unknown value
stack at sp + 2 slots: 0x00007f78766a2dd4: <offset 0x00000000000a2dd4> in /control/ctmem/ctm_em/lib/libbmc-corekit.so.4.0 at 0x00007f7876600000
stack at sp + 3 slots: 0x0 is NULL
stack at sp + 4 slots: 0x00007f7876685061: _ZN24Osl_UnicodeConfiguration13ReadLocaleIniEv+0x0000000000001c4b in /control/ctmem/ctm_em/lib/libbmc-corekit.so.4.0 at 0x00007f7876600000
stack at sp + 5 slots: 0x00007ffef7c1dc40 is pointing into the stack for thread: 0x00000000025ec000
stack at sp + 6 slots: 0x00007ffe00000000 is an unknown value
stack at sp + 7 slots: 0x00007ffef7c1dda0 is pointing into the stack for thread: 0x00000000025ec000
---------------  P R O C E S S  ---------------
Threads class SMR info:
_java_thread_list=0x0000000003e2db58, length=11, elements={
0x00000000025ec000, 0x0000000003332800, 0x000000000333c000, 0x00000000033b1000,
0x00000000032b5800, 0x00000000032b4000, 0x00000000032b2000, 0x00000000032c0800,
0x0000000003473800, 0x00000000032f1800, 0x0000000003f01800
}
Java Threads: ( => current thread )
=>0x00000000025ec000 JavaThread "main" [_thread_in_native, id=125270, stack(0x00007ffef7b20000,0x00007ffef7c20000)]
0x0000000003332800 JavaThread "Reference Handler" daemon [_thread_blocked, id=125277, stack(0x00007f7873700000,0x00007f7873800000)]
0x000000000333c000 JavaThread "Finalizer" daemon [_thread_blocked, id=125278, stack(0x00007f7873300000,0x00007f7873400000)]
0x00000000033b1000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=125279, stack(0x00007f7872f00000,0x00007f7873000000)]
0x00000000032b5800 JavaThread "Service Thread" daemon [_thread_blocked, id=125280, stack(0x00007f7872b00000,0x00007f7872c00000)]
0x00000000032b4000 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=125281, stack(0x00007f7872700000,0x00007f7872800000)]
0x00000000032b2000 JavaThread "C1 CompilerThread0" daemon [_thread_blocked, id=125282, stack(0x00007f7872300000,0x00007f7872400000)]
0x00000000032c0800 JavaThread "Sweeper thread" daemon [_thread_blocked, id=125283, stack(0x00007f7871f00000,0x00007f7872000000)]
0x0000000003473800 JavaThread "Common-Cleaner" daemon [_thread_blocked, id=125285, stack(0x00007f7870700000,0x00007f7870800000)]
0x00000000032f1800 JavaThread "Tomcat JDBC Pool Cleaner[258754732:1707697459553]" daemon [_thread_blocked, id=125286, stack(0x00007f7870300000,0x00007f7870400000)]
0x0000000003f01800 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=125287, stack(0x00007f786fd00000,0x00007f786fe00000)]
Other Threads:
0x00000000032f7800 VMThread "VM Thread" [stack: 0x00007f7874100000,0x00007f7874200000] [id=125276]
0x0000000003455000 WatcherThread [stack: 0x00007f7871500000,0x00007f7871600000] [id=125284]
0x0000000002619800 GCTaskThread "GC Thread#0" [stack: 0x00007f7878500000,0x00007f7878600000] [id=125271]
0x0000000002a96800 ConcurrentGCThread "G1 Main Marker" [stack: 0x00007f7877500000,0x00007f7877600000] [id=125272]
0x0000000002a95000 ConcurrentGCThread "G1 Conc#0" [stack: 0x00007f7877100000,0x00007f7877200000] [id=125273]
0x00000000030be800 ConcurrentGCThread "G1 Refine#0" [stack: 0x00007f7876d00000,0x00007f7876e00000] [id=125274]
0x00000000030bd000 ConcurrentGCThread "G1 Young RemSet Sampling" [stack: 0x00007f7876100000,0x00007f7876200000] [id=125275]
Threads with active compile tasks:
VM state:not at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: None
---------------  S Y S T E M  ---------------
OS:Amazon Linux release 2023 (Amazon Linux)
uname:Linux 6.1.61-85.141.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Nov  8 00:39:18 UTC 2023 x86_64
OS uptime: 0 days 6:01 hours
libc:glibc 2.34 NPTL 2.34
rlimit (soft/hard): STACK 8192k/10240k , CORE 0k/0k , NPROC infinity/infinity , NOFILE 65535/65535 , AS infinity/infinity , CPU infinity/infinity , DATA infinity/infinity , FSIZE infinity/infinity , MEMLOCK infinity/infinity
load average:0.39 0.25 0.26
/proc/meminfo:
MemTotal:       32132332 kB
MemFree:        18735904 kB
MemAvailable:   24301312 kB
Buffers:            2168 kB
Cached:          5737588 kB
SwapCached:            0 kB
Active:          4619756 kB
Inactive:        8198368 kB
Active(anon):       1028 kB
Inactive(anon):  7086680 kB
Active(file):    4618728 kB
Inactive(file):  1111688 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Zswap:                 0 kB
Zswapped:              0 kB
Dirty:               528 kB
Writeback:             0 kB
AnonPages:       7055540 kB
Mapped:           496212 kB
Shmem:              9340 kB
KReclaimable:     294924 kB
Slab:             373368 kB
SReclaimable:     294924 kB
SUnreclaim:        78444 kB
KernelStack:       18472 kB
PageTables:        29244 kB
SecPageTables:         0 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    16066164 kB
Committed_AS:   11380812 kB
VmallocTotal:<PHONE_NUMBER>7 kB
VmallocUsed:       26780 kB
VmallocChunk:          0 kB
Percpu:             2048 kB
HardwareCorrupted:     0 kB
AnonHugePages:      8192 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:       98736 kB
DirectMap2M:     5455872 kB
DirectMap1G:    28311552 kB
/sys/kernel/mm/transparent_hugepage/enabled:
always [madvise] never
/sys/kernel/mm/transparent_hugepage/defrag (defrag/compaction efforts parameter):
always defer defer+madvise [madvise] never
Process Memory:
Virtual Size: 2389044K (peak: 2389048K)
Resident Set Size: 131140K (peak: 131140K) (anon: 60840K, file: 70268K, shmem: 32K)
Swapped out: 0K
C-Heap outstanding allocations: 0K, retained: 0K
glibc malloc tunables: (default)
/proc/sys/kernel/threads-max (system-wide limit on the number of threads):
250870
/proc/sys/vm/max_map_count (maximum number of memory map areas a process may have):
65530
/proc/sys/kernel/pid_max (system-wide limit on number of process identifiers):
4194304
container (cgroup) information:
container_type: cgroupv2
cpu_cpuset_cpus: not supported
cpu_memory_nodes: not supported
active_processor_count: 4
cpu_quota: no quota
cpu_period: 100000
cpu_shares: no shares
memory_limit_in_bytes: unlimited
memory_and_swap_limit_in_bytes: unlimited
memory_soft_limit_in_bytes: unlimited
memory_usage_in_bytes: 11921336 k
memory_max_usage_in_bytes: not supported
memory_swap_current_in_bytes: unlimited
memory_swap_max_limit_in_bytes: unlimited
maximum number of tasks: unlimited
current number of tasks: 947
KVM virtualization detected
Steal ticks since vm start: 0
Steal ticks percentage since vm start:  0.000
CPU:total 4 (initial active 4) (4 cores per cpu, 2 threads per core) family 25 model 1 stepping 1 microcode 0xa0011d1, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, mmxext, 3dnowpref, lzcnt, sse4a, ht, tsc, tscinvbit, tscinv, bmi1, bmi2, adx, sha, fma
CPU Model and flags from /proc/cpuinfo:
model name      : AMD EPYC 7R13 Processor
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch topoext invpcid_single pti ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr rdpru wbnoinvd arat npt nrip_save vaes vpclmulqdq rdpid
Online cpus:
0-3
Offline cpus:
BIOS frequency limitation:

Frequency switch latency (ns):

Available cpu frequencies:
container (cgroup) information:
container_type: cgroupv2
cpu_cpuset_cpus: not supported
cpu_memory_nodes: not supported
active_processor_count: 4
cpu_quota: no quota
cpu_period: 100000
cpu_shares: no shares
memory_limit_in_bytes: unlimited
memory_and_swap_limit_in_bytes: unlimited
memory_soft_limit_in_bytes: unlimited
memory_usage_in_bytes: 11921336 k
memory_max_usage_in_bytes: not supported
memory_swap_current_in_bytes: unlimited
memory_swap_max_limit_in_bytes: unlimited
maximum number of tasks: unlimited
current number of tasks: 947
KVM virtualization detected
Steal ticks since vm start: 0
Steal ticks percentage since vm start:  0.000
CPU:total 4 (initial active 4) (4 cores per cpu, 2 threads per core) family 25 model 1 stepping 1 microcode 0xa0011d1, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, mmxext, 3dnowpref, lzcnt, sse4a, ht, tsc, tscinvbit, tscinv, bmi1, bmi2, adx, sha, fma
CPU Model and flags from /proc/cpuinfo:
model name      : AMD EPYC 7R13 Processor
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch topoext invpcid_single pti ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr rdpru wbnoinvd arat npt nrip_save vaes vpclmulqdq rdpid
Online cpus:
0-3
Offline cpus:
BIOS frequency limitation:

Frequency switch latency (ns):

Available cpu frequencies:

Current governor:

Core performance/turbo boost:

Memory: 4k page, physical 32132332k(18735904k free), swap 0k(0k free)
vm_info: OpenJDK 64-Bit Server VM (11.0.21+9) for linux-amd64 JRE (11.0.21+9), built on Oct 17 2023 21:44:42 by "" with gcc 7.5.0
END.
`
OS Version:
Linux aws-ctmem-01 6.1.61-85.141.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Nov  8 00:39:18 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Did you test with the latest update version?
Yes. This was tested on multiple JDK versions, include: 11.0.16, 11.0.21+9, and 11.0.21+13

Please provide steps to reproduce where possible
No response
Expected Results
JVM startup completes and JVM is up and running
Actual Results
JVM startup failure
What Java Version are you using?
openjdk 11.0.21 2023-10-17 OpenJDK Runtime Environment Temurin-11.0.21+9 (build 11.0.21+9) OpenJDK 64-Bit Server VM Temurin-11.0.21+9 (build 11.0.21+9, mixed mode)
What is your operating system and platform?
Amazon Linux : Linux aws-ctmem-01 6.1.61-85.141.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Nov  8 00:39:18 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
How did you install Java?
Downloading the JDK/JRE from https://adoptium.net/en-GB/download/ . untar the file and copy to the required path. Finally, executing update-alternatives to update the system alternatives.
Did it work before?
NA. This has never been tested before on the same OS using the same JDK version

Did you test with other Java versions?
No response
Relevant log output
No response

@aalwadi-wcq This is a crash in native code that happens to propagate back through the JVM.  Are you able to post this bug to the author of the CTM software?

Hi Karianna,
Thank you for the quick response.
I did raise a support ticket for this with the vendor, but I haven't had a response yet. I will update the ticket as soon as I get a response


allow running benchmarks with slurm constraints rather than explicit node lists

For now we use --nodelist with srun to define which nodes hpcbench should request with srun to run on. This is not compatible with a production system, because we will often find the specific nodes already allocated. Much more effective and useful for most benchmarks is a mode in which we define tags with constraints that should be fulfilled.
Specific example:
A simple osu sbatch script would look like this:
#!/bin/bash -l
#SBATCH -t 08:00:00
#SBATCH -p prod

module purge
module load nix/osu-mpi-bench

echo "SLURM_NTASKS: $SLURM_NTASKS"
echo "SLURM_NTASKS_PER_NODE: $SLURM_NTASKS_PER_NODE"
echo "SLURM_NODELIST: $SLURM_NODELIST"

srun --account=proj16 --mpi=pmi2 osu_allgatherv -m 32:1024 -x 200 -i 100

calling this sbatch script on 6 UC1 nodes and 6 UC2 nodes can be done as follows:
sbatch --account=proj16 --partition=prod --exclusive \
  -C "uc1*6|uc2*6" -N 12 --ntasks-per-node=36 ./osu_coll_mva.sbatch

Note the -C or --constraint argument.

I am starting the implementation of the new tag constraint type.
The campaign.yaml documentation has to be updated according to:

new slurm executor / new ben-sh options
sbatch Jinja templates
constraint tag

@ohm314 I suggest you take care of the slurm executor, I take the rest. Would that be OK?

Sounds good. Updating documentation and cleaning up code was anyway my next
step.
On Fri, 13 Apr 2018 at 08:08, Tristan Carel<EMAIL_ADDRESS>wrote:

I am starting the implementation of the new tag constraint type.
The campaign.yaml documentation has to be updated according to:

new slurm executor / new ben-sh options
sbatch Jinja templates
constraint tag

@ohm314 https://github.com/ohm314 I suggest you take care of the slurm
executor, I take the rest. Would that be OK?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tristan0x/hpcbench/issues/137#issuecomment-381033730,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAnIvMFPImlRYBpZPJm7R8BmTe6XEkZ1ks5toEDcgaJpZM4TFPQD
.

constraints can now be defined inside the network section


[FIX] Fix norm issues under utils folder

Norm issue PR could be considered merging first

The regression test failed is randomly happen not related to this PR

@itislu After fix all norm issue I will fine-tune the header file structure

Let us leave the header structure in another pr

Okay. Would you like to fix the other 3 comments?

No, bc i want to change all header files


Dotnet build fails because in bin folder WorkerExtensions.csproj

Description
In debug folder of a project WorkerExtensions.csproj is placed. Its broken since 1.20
Please release new version without this.
Steps to reproduce
During dotnet restore - in debug folder of a project WorkerExtensions.csproj is placed. Its bug.
During dotnet build --no-restore - it tries to restore in bin folder everything including WorkerExtensions.csproj (which references .Net 6 etc) and fails when private azure nuget fields are used.
Builds for Azure Pipelines, and Builds for Docker use separate dotnet restore and dotnet build. So both are broken now.

@drweb86 Can you share which version of Microsoft.Azure.Functions.Worker.Sdk package you are using?

@drweb86 - WorkerExtensions.csproj is placed into a sub-folder of IntermediateOutputPath, this is expected. If you are seeing this csproj in your final output location, that means you have your intermediate output path pointing to the same location as output, which is not a good idea with msbuild.

With 1.14 it works. But with latest versions - it breaks compilation. If to use private nuget packages repository and doing builds in 2 stages
dotnet restore - works
dotnet build --no-restore - it fails because it detects in obj directory WorkerExtensions.csproj and .Net 6 and tries to restore them and fails to do so because of authentication to private repositories which are not part of the WorkerExtensions.csproj.

The latest version places the WorkerExtensions.csproj in the IntermediateOutputPath, which is obj by default. If you are seeing this file appear in bin, then you have reassigned that MSBuild property. It is not recommended to have intermediate and final output go to the same directory.


[GHA] Merge LBT workflow into ci-test

Merged LBT workflow into ci-test for better signal
tested with /canary for both land_blocking_compat and land_blocking suites
Motivation
(Write your motivation for proposed changes here.)
Have you read the Contributing Guidelines on pull requests?
(Write your answer here.)
Test Plan
(Share your test plan here. If you changed code, please provide us with clear instructions for verifying that your changes work.)
Related PRs
(If this PR adds or changes functionality, please take some time to update the docs at https://github.com/diem/diem/tree/master/developers.diem.com, and link to your PR here.)
If targeting a release branch, please fill the below out as well

Justification and breaking nature (who does it affect? validators, full nodes, tooling, operators, AOS, etc.)
Comprehensive test results that demonstrate the fix working and not breaking existing workflows.
Why we must have it for V1 launch.
What workarounds and alternative we have if we do not push the PR.

In order to land this we'll probably need to have the land-blocking workflow be empty, and always pass. Then we can change bors to stop looking at it at which point we can remove the workflow

Just for double check, if we leave lbt workflow to be empty which means it will not be triggered? or we still keep the head of workflow but leave jobs empty?

Just for double check, if we leave lbt workflow to be empty which means it will not be triggered? or we still keep the head of workflow but leave jobs empty?

The latter.  The LBT workflow needs to be a no-op and must keep this line https://github.com/diem/diem/blob/6b4f8a8c2da26c34597ce93a1f92e9b1803c16fa/.github/workflows/land-blocking.yml#L12  Bors listens for two repo checks' signal (one just mentioned and another in ci-test) to vet a PR and merge it.
Here's a overview of the steps.

land this PR
update Bors config to listen for ci-test
deprecate the former standalone LBT

/canary

:lock: Merge Conflict

/canary

:broken_heart: Test Failed - ci-test-success

/canary

:broken_heart: Test Failed - Build images and run cluster test

/canary

:broken_heart: Test Failed - Build images and run cluster test

/canary

:broken_heart: Test Failed - ci-test-success

Looks like this is super close to being finished, just need to fix the lint:
Error:  [eof-newline] [content .github/workflows/land-blocking.yml]: missing newline at EOF

Looks like this is super close to being finished, just need to fix the lint:
Error:  [eof-newline] [content .github/workflows/land-blocking.yml]: missing newline at EOF

yea, functionally works but I am trying do some code refactory, I posted plan in #cluster-test, and I think most of time I used  is on the script syntax and I admit I do not very good at it😢😢😢😢😢

Looks like this is super close to being finished, just need to fix the lint:
Error:  [eof-newline] [content .github/workflows/land-blocking.yml]: missing newline at EOF

yea, functionally works but I am trying do some code refactory, I posted plan in #cluster-test, and I think most of time I used is on the script syntax and I admit I do not very good at it😢😢😢😢😢, I opened another draft PR to aviod keep annoying u guys🤣

I struggle quit a bit trying to mess with these yml files too so no worries :)
And there's no need to spin up another PR if it causes more work for you and you're only interested in not having notifications set to the people subscribed to this PR. Please do whatever is easiest for your workflow!

I think there are a couple small things that could be improved but don't need to happen now.

thx for reviewing @rexhoffman , I will address last two comments before merge. Happy monday~

/canary

:broken_heart: Test Failed - ci-test-success

/land

/cherry-pick release-1.1

@bmwill :exclamation: cherry-pick failed, possibly due to conflicts. You can perform the cherry-pick yourself by running the following commands:
git fetch https://github.com/diem/diem.git release-1.1 a73f984cade27708738349c95343212b0a1bf80d
git checkout release-1.1
git cherry-pick ba1aec8c6425e4b701b2f22ff1c490119dee5aaa..a73f984cade27708738349c95343212b0a1bf80d


chore: Make gradle publisher an action

Closes https://github.com/slsa-framework/slsa-github-generator/issues/2464 and https://github.com/slsa-framework/slsa-github-generator/issues/2467

Ready for review.


Null checks added

Added Null checks in ef-coap.c to avoid segmentation faults

Hi, can you say which was the command that generated the segmentation faults?

Hi, can you say which was the command that generated the segmentation faults?

Hi,
It was not a particular command. Segmentation fault occured while doing fuzz tests. So some random frame generated a SIGSEV error at the NULL checks that was added .

Don't you have a log or something that shows all the generated commands at run-time? Because then you can extract the command that generated the random frame.
I am not saying it is not an issue but it would be nice to understand what is the command to see if other frames types are having the same issue.

Hi Horatiu ,
Sorry for the delay, took some time to analyze the actual problem that causes the Segmentation fault.
So , the random frame that got generated was :

eth dmac 1 smac 2 ipv4 sip <IP_ADDRESS> dip <IP_ADDRESS> dip <IP_ADDRESS> udp sport 1000 dport 5683 coap ver 1 type 0 code 0.03 msgid                              0x5233 token<PHONE_NUMBER>ABCD01 coap-opt num 11 val ascii c coap-opt num . val hex 8c
The command is the coap-opt which calls the function option_fill_details() which creates try to find the fields "num" and "val"
In the above case , after the "num" filed  "." is given which is invalid.
So Segmentation fault when it tries to read the value from the object fnum->val in : https://github.com/microchip-ung/easyframes/blob/0d1c8c257e29c8e5594082d6f38c9271c12bcfbf/src/ef-coap.c#L189

eth dmac 1 smac 2 ipv4 sip <IP_ADDRESS> dip <IP_ADDRESS> dip <IP_ADDRESS> udp sport 1000 dport 5683 coap ver 1 type 0 code 0.03 msgid                              0x5233 token<PHONE_NUMBER>ABCD01 coap-opt num 11 val ascii c coap-opt num 0
This is also similar to the first case , just that error occurs in : https://github.com/microchip-ung/easyframes/blob/0d1c8c257e29c8e5594082d6f38c9271c12bcfbf/src/ef-coap.c#L186
This is because the frame is basically without "val" field , so when it tries to read fval->val Segmentation fault occurs.

Thanks for the detail explanation and for finding the commands to reproduce the issue.
I have looked a little bit through the code and maybe easier would be to do like the following:
diff --git a/src/ef-coap.c b/src/ef-coap.c
index a43d87d..50dd5f0 100644
--- a/src/ef-coap.c
+++ b/src/ef-coap.c
@@ -178,7 +178,7 @@ static int options_fill_defaults(struct frame *f, int stack_idx) {
     field_t *fnum = find_field(hdr, "num");
     field_t *fval = find_field(hdr, "val");
-    if (!fnum) {
+    if (!fnum->val || !fval->val) {
         return 0;
     }

Thank you  for the suggestion,
One concern is that the check is added inside !fnum check,
if(!fnum){    if (!fnum->val || !fval->val) { the problem is that , the check is useful only when fnum is not NULL      return 0;     } }
But the segmentation occurs when fnum->val or fval->val is NULL, I have also further tested it out
So It could be changed to the following:
if(!fnum){      return 0;     } if(!fnum->val ||!fval->val){     return 0; }
We could have two separate checks.

I didn't mean to put the check inside
if (!fnum) {
    return 0;
}

but to replace that code with
if (!fnum->val || !fval->val) {
    return 0;
}

Because fnum can't be NULL(if I read correctly the code) so there is no point to have that check anymore.

Yes okay Horatiu, Understood now.
I have updated the branch now with the said suggestion.
The PR is also automatically updated.
Thank you

Please squash those 2 commits into one single commit and add a single comment. Then I think it is ready to be merged.

Yes updated into one commit.
Thank you

This looks good to me. Thanks for doing all this!


Settings page creation and linking with home page

Created the settings page with 3 buttons, and linked it with the home page

Looks good to me! Styling obviously needs to be done, but this completes this task


Updated architecture document based on Phil's diagram

Phil's diagram: https://app.diagrams.net/#G17TO-2VH-59Pm8FMenrYBpN8xAnI-PfGI#{"pageId"%3A"4bL5o4j64XjnUdqXuRuk"}

As discussed in Manchester technical roadmap meeting.

https://docs.google.com/document/d/1K6BX94iHXgxpTDCNwVgkKoNodaZBoufqUu2VcAZmdLU/edit was started but not filled in

Architecture requirements for "Cratey" is in #16

Blocked by #16.

@stain to chase with Tim.


--conda-frontend value not passed on to cluster jobs

Snakemake version

v6.6.1
Describe the bug

Running a snakemake workflow on a slurm cluster --cluster 'sbatch <some stuff for submitting>' and --conda-frontend conda my submitted jobs fail. Related to #1009 ?
Logs

Error: mamba package manager is not available. The mamba package manager (https://github.com/mamba-org/mamba) is an extremely fast and robust conda replacement. It is the recommended way of using Snakemake's conda integration. It can be installed with `conda install -n base -c conda-forge mamba.If you still prefer to use conda, you can enforce that by setting `--conda-frontend conda`.

Minimal example

Additional context

No profiles are installed, conda is available on the nodes because I can run regular jobs using conda on nodes just fine.
The --conda-frontend  argument simply isn't working.

It is doing it again for me. This is hugely problematic.

I was also having this issue with v6.9.1, but using LSF. I was able to get things working again by purging all modules I had loaded and then running snakemake again, but don't know how generalizable this is.

anyone has a solution? I faced same error.

Anyone find a workaround for this?


Fixes #2219 - adding a subcommand to enable deleting parts of a repo in the local cache

Initial thoughts:
Going with repo-cache, use case would be something like:
> huggingface-cli repo-cache delete --all repo_id     # same functionality as delete-cache but quick to delete just the one repo
> huggingface-cli repo-cache delete --include (glob) repo_id.  # use same glob format as download --include flag; mark files to __include__ for delete action
> huggingface-cli repo-cache delete --exclude [glob] repo_id. # same as --include, but opposite
> huggingface-cli repo-cache delete --file-list (list of files to remove) repo_id

Also, a starting point is that the code references a frozen class DeletesCacheStrategy. I will do more digging, but there may be a framework that needs a little polish to make this very straightforward. Dangerous words in this business, I tell ya.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hi @sealad886, thanks for the proposal and sorry for the long delay! So I've looked back to your snippet (and suggestions in https://github.com/huggingface/huggingface_hub/issues/2219#issuecomment-2050055659) and I think we could modify it a bit. What do you it of:
> huggingface-cli cache delete-repo repo_id
> huggingface-cli cache delete-repo repo_id --repo-type=dataset
> huggingface-cli cache delete-repo repo_id --include (glob)
> huggingface-cli cache delete-repo repo_id --exclude (glob)
> huggingface-cli cache delete-repo repo_id --files=file1,file2,file3,file4

?
Let's not focus too much on what's existing in huggingface-cli delete-cache BTW. I would prefer to have a unified experience to delete things from the cache but that's not really possible while maintaining an intuitive API. At least we can start by having a subcommands in a generic huggingface-cli cache part.

@Wauplin those look like excellent starts. I'll probably also includes a --dirs (or if there's another naming convention used elsewhere in the repo, I'll rename it) to be explicit about deleting whole directories.
I had been looking at some of the git -i stuff only recently, and it looks a lot like a utility that I've been developing for Ollama CLI (also centering on cache management). (Ollamaultil)[https://github.com/sealad886/ollama_util] is my take on a interface-based cache management utility (different focus: I use it to move cache files between on-disk and external storage--and back--since I'm on a laptop); it's not really adapted for components to be called at the command line, but it could be. Just wanted to throw that out there.
In terms of design, I think I'm going to start by actually enhancing the DeleteCacheStrategy class as the helper tool it is, and then working up from there. I've also noticed that the directories can sometimes fail to be deleted if (laughs with this one) '.DS_Store' files exist in the directory that's being deleted, and the delete process overall fails mid-way through. I'll fix this too.
Also...verbosity.
So, to summarize my clearly scatterbrained thoughts:

CLI command delete-repo and associated options as noted above.
Fix .DS_Store or other hidden files preventing the final deletion of a directory when the whole repo is being deleted.
Add verbosity, for now only to give some concept of progress when deleting large amounts of cached data.
Possibility of some kind of utility in the future?

I have time this week to actually dig into this more, so I'll get crackin'.

Thanks for writing up your thoughts @sealad886! This looks like a good plan to me. Please let me know if yo u need any assistance :)


Cannot install additional data in Windows

Dear @ialbert,
I managed to install bio in my Windows machine via the command prompt and use some functions (e.g. fasta). But since I could not run 'bio --download', I cannot use all the functionalities of bio.
I printed the error I got below, which is related to a denied permission to one of the temporary files created during the download. I got the same error using the prompt as administrator or not, and I already try to change the access rights of my entire ./Temp folder, but nothing worked so far.
I tried to look online in some possible ideas on the cause of the problem, but it may be that next command line is trying to execute another task before the folder is fully created.
Any ideas? Downloading the 'biodata.tar.gz' directly into my ..bio folder would help?
Thanks!

can you please list the error message,
by default bio attempts to create a cache directory in your home directory where it will store the cache data:
~/.bio
It sound like there is no write access into that directory/location.
can you print your $HOME variable as well.
echo $HOME

Sorry @ialbert, here you are the parts I missed. Thanks again
Error:
C:\Users\renato>bio --download
# C:\Users\renato\.bio\biodata.tar.gz: 100%|███████████████████████████████████████▉| 165M/165M [00:18<00:00, 6.43MB/s]Traceback (most recent call last):
  File "c:\users\renato\appdata\local\programs\python\python38-32\lib\runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\users\renato\appdata\local\programs\python\python38-32\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\renato\AppData\Local\Programs\Python\Python38-32\Scripts\bio.exe\__main__.py", line 7, in <module>
  File "c:\users\renato\appdata\local\programs\python\python38-32\lib\site-packages\biorun\__main__.py", line 14, in run
    router()
  File "c:\users\renato\appdata\local\programs\python\python38-32\lib\site-packages\biorun\main.py", line 107, in wrapper
    func(*args, **kwargs)
  File "c:\users\renato\appdata\local\programs\python\python38-32\lib\site-packages\biorun\main.py", line 140, in router
    utils.download_prebuilt()
  File "c:\users\renato\appdata\local\programs\python\python38-32\lib\site-packages\biorun\utils.py", line 289, in download_prebuilt
    download(url=url, fname=path)
  File "c:\users\renato\appdata\local\programs\python\python38-32\lib\site-packages\biorun\utils.py", line 268, in download
    shutil.copyfile(fp.name, path)
  File "c:\users\renato\appdata\local\programs\python\python38-32\lib\shutil.py", line 259, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\renato\\AppData\\Local\\Temp\\tmplwl3x15g'

Not 100% sure if I got what it would be my $HOME variable. Is it the path to my user? If it is, the path is 'C:\Users\renato'.

Ok, I found the source of the problem, it looks like Windows treats temporary files a little differently than Unix.
I have pushed out a change, do a
pip install bio --upgrade

and you should be getting version 1.0.6 or higher, this version does download correctly for me on Windows

Thank you very much for your swift response @ialbert !
I already upgrade to the newer version and the download of the acessory data worked like a gem!
I will continue testing all package functionalities and I will get back to you if I find some issue related to this. But I will close the issue for now.
Thanks again!


NameError: global name 'ParsingError' is not defined

Traceback (most recent call last):
File "/usr/local/bin/beaver", line 31, in
worker_run(args)
File "/usr/local/lib/python2.7/site-packages/beaver/dispatcher/worker.py", line 17, in run
beaver_config = BeaverConfig(args, logger=logger)
File "/usr/local/lib/python2.7/site-packages/beaver/config.py", line 171, in init
self._parse(args)
File "/usr/local/lib/python2.7/site-packages/beaver/config.py", line 461, in _parse
config_parser=self._config_parser
File "/usr/local/lib/python2.7/site-packages/conf_d/init.py", line 28, in init
self.parse()
File "/usr/local/lib/python2.7/site-packages/conf_d/init.py", line 74, in parse
configs = self._parse_section(path=self._path, defaults=self._main_defaults, parser=self._main_parser, only_section=self._name)
File "/usr/local/lib/python2.7/site-packages/conf_d/init.py", line 110, in _parse_section
if len(config_parser.read(path)) != 1:
File "/usr/local/lib/python2.7/ConfigParser.py", line 305, in read
self._read(fp, filename)
File "/usr/local/lib/python2.7/site-packages/beaver/glob_safe_config_parser.py", line 70, in _read
e = ParsingError(fpname)
NameError: global name 'ParsingError' is not defined

Can you try installing 6d021289898b2287b0f113593de7d4995a38133f and seeing if it fixes your issue? Also, you have a parse error in your config.

Closing due to inactivity. Please reopen if this is still a problem.


Convert EloquentCollection to base if not solely filled with models

Some methods, such as map() or concat() may introduce non-model items
into an EloquentCollection. Now, this is checked not only for map() but
also for other methods.
Other methods can be known ahead of time to return a base Collection, in
those case the check is omitted and the call is simply passed to base.
This achieves a few things:

Prevent bugs by having more accurate return type hints
Ensure correct behaviour for code that has special treatment for EloquentCollection
Aid static analysis tools

Can you send this to the master branch?


--dry-run option

I want to be able to see what the tool is going to do (including the policy documents) without actually calling the AWS APIs.

This is actually a bit tricky to add. I could do it with a whole bunch of if dry_run: else:  statements, which is a bit ugly. I thought about wrapping the client returned by my make_client() function, but then I still need to mock return values from the client when it's running in dry run mode so that existing code doesn't break.

I'm going to use dry_run to set the various client objects to None so that it's impossible to accidentally trigger an API call while running in --dry-run mode, then I'll implement this feature with the ugly if statements.

Example output:
% s3-credentials create my-bucket --dry-run --read-only --duration 20m
Would create bucket: 'my-bucket'
Would ensure role: 's3-credentials.AmazonS3FullAccess'
Would assume role using following policy for 1200 seconds:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:GetObjectLegalHold",
                "s3:GetObjectRetention",
                "s3:GetObjectTagging"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket/*"
            ]
        }
    ]
}

% s3-credentials create my-bucket --dry-run
Would create bucket: 'my-bucket'
Would create user: 's3.read-write.my-bucket' with permissions boundary: 'arn:aws:iam::aws:policy/AmazonS3FullAccess'
Would attach policy called 's3.read-write.my-bucket' to user 's3.read-write.my-bucket', details:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:GetObjectLegalHold",
                "s3:GetObjectRetention",
                "s3:GetObjectTagging"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket/*"
            ]
        }
    ]
}
Would call create access key for user 's3.read-write.my-bucket'


Rename bash completion script as heroku

The install command accepts a hash in addition to a string. As a hash, the keys are the files to install, the values are the names to use when installed.
The cleaner implementation here would be to update the actual script filenames within the autocomplete package itself. (There's some jarring inconsistency here where brew/bash is a script, while brew/zsh is a directory that contains the script.)
Ideally, I'd probably expect the brew/bash file to be renamed as brew/bash/heroku and that would eliminate the need for this hash syntax. However, simply changing the Formula is probably easier.
(Also, this updates the formula to use homebrew's path helpers consistently.
Closes #16

Thanks for this patch! I've incorporated it into the main repository (which publishes to this repository upon release) in heroku/cli#1764 which is available in Homebrew as of Heroku CLI v7.51.0. You should be able to pick up this change by running brew update && brew upgrade. Apologies for the delay. 🙇


Check for execution property before inspecting task

When Project Manager for Java is installed alongside Quarkus Tooks, running Quarkus: Debug current Quarkus project task basically does nothing. This is because Project Manager for Java exposes a Task that has no execution property, so that the call to  src/utils/tasksUtils.ts#isQuarkusDevTask() throws an exception.
Fixes #324

Thanks so much for your help!


Create an s3 importer plugin and use that as the default

This may be a one-to-two-liner function; we may also want some dummy processor functions for this plugin, too.

The importer's job is to bring data into the system from outside, so in my view this issue is all about creating an "upload importer"--that is, an importer that allows you to upload a file from disk to S3, then creates the appropriate Dataset object from it.
Is that what you were thinking of too, @annehaley?

Originally, this idea had been that we would require the user to already have a file on s3 to reference with the source_uri field, but this was before we added a content field to the Dataset model. With the addition of the content field and making the source_uri field nullable, A user can already choose to either directly upload the file or point to it. Therefore, we do not need a default importer anymore, and this issue can be closed.

Do we have the infrastructure ready to support uploading files from the frontend?

Yes, the girder cookicutter project came ready with those endpoints. However, we decided not to add a "Create Dataset" endpoint at this point, choosing instead to create db objects on the backend and fetch them on the frontend.

Ah, I see. Yeah that's a good approach--we can stuff data in programmatically (via the populate script) and be on our way developing the research features. At some point we'll want that "create dataset" button so I'll file an issue, but it's low priority at the moment.


Improve the way how log part between timestamps is retrieved using awk

Take into consideration logs where each log line doesn't start with timestamp
https://github.com/EmbedITCZ/jbehave-support/issues/8

I think we should recreate the original jira issue in github issue tracker and avoid using JBUS-xy

minor: your commit use HCG user, I'm not sure if it intentional

Do we think there might be a solution to this issue by leveraging the existing timestampFormat without introducing the additional timestampPosixRegex? (not saying there is, just putting it up to try to avoid adding new config parameter if we can somehow avoid it :slightly_smiling_face: )

I think we should recreate the original jira issue in github issue tracker and avoid using JBUS-xy

Done: https://github.com/EmbedITCZ/jbehave-support/issues/8

minor: your commit use HCG user, I'm not sure if it intentional

I changed it. Thanks

Do we think there might be a solution to this issue by leveraging the existing timestampFormat without introducing the additional timestampPosixRegex? (not saying there is, just putting it up to try to avoid adding new config parameter if we can somehow avoid it 🙂 )

I was thinking about it, but I couldn't figure out how to do it. If anybody knows, let me know and I'll incorporate it.

Do we think there might be a solution to this issue by leveraging the existing timestampFormat without introducing the additional timestampPosixRegex? (not saying there is, just putting it up to try to avoid adding new config parameter if we can somehow avoid it slightly_smiling_face )

I was thinking about it, but I couldn't figure out how to do it. If anybody knows, let me know and I'll incorporate it.

I think I found a solution without introducing the regex, give me a couple of days to verify it - I will post an update once I have something working (or find out it cant be done)

@thradec @paveljandejsek
I pushed alternative solution which doesn't break backward compatibility.
I also addressed all comments.
Have a look again and let me know if there is something to add or merge.
Thanks


try to use console

hello
I try to create a console but

I get missing componens errors

the console shows up but hack the tab key for some reason(I use it in my game)

I can enter a command, but pressing enter dont do anything
avr. 18, 2015 9:47:38 PM de.lessvoid.nifty.elements.Element getNiftyControl
WARNING: missing element/control with id [6] for requested control class             [de.lessvoid.nifty.controls.Console]
avr. 18, 2015 9:47:38 PM de.lessvoid.nifty.elements.Element getNiftyControl
WARNING: missing element/control with id [7] for requested control class [de.lessvoid.nifty.controls.Console]
avr. 18, 2015 9:47:38 PM de.lessvoid.nifty.elements.Element getNiftyControl
WARNING: missing element/control with id [8] for requested control class [de.lessvoid.nifty.controls.Console]
avr. 18, 2015 9:47:38 PM de.lessvoid.nifty.elements.Element getNiftyControl
WARNING: missing element/control with id [9] for requested control class [de.lessvoid.nifty.controls.Console]
avr. 18, 2015 9:47:38 PM de.lessvoid.nifty.elements.Element getNiftyControl
WARNING: missing element/control with id [10] for requested control class [de.lessvoid.nifty.controls.Console]
avr. 18, 2015 9:47:38 PM de.lessvoid.nifty.elements.Element getNiftyControl
WARNING: missing element/control with id [11] for requested control class
...
avr. 18, 2015 9:47:38 PM de.lessvoid.nifty.elements.Element getNiftyControl
WARNING: missing element/control with id [console] for requested control class     [de.lessvoid.nifty.controls.Console]
[/java]
  <controlDefinition name="consoleComponent" controller="mygame.gui.CommandButtonController" >
      <effect>
          <onShow name="move" mode="fromOffset" offsetY="100" length="300" inherit="true"/>
          <onHide name="move" mode="toOffset" offsetY="100" length="200" inherit="true"/>
      </effect>
      <control id="console" name="nifty-console" lines="10" width="70%" valign="top" backgroundColor="#6874" visible="true"/>
  </controlDefinition>

the event is never reached if I press enter
@NiftyEventSubscriber(id="console")
    public void onConsoleCommand(final String id, final ConsoleExecuteCommandEvent command)
    {
        HideCommand hideCommand = new HideCommand();
        hideCommand.setController(this);
....
    }

what do I do wrong ?
thanks

sh... I cant paste xml tags

??

It seems your GUI element isn't bound correctly to your controller; can you post additional, relevant XML code?

    <!--   And it is it handling the right event type?
Check that, and if you have problems, go ahead and post the Java code for your controller.

nop, the id of the console is console
"consoleComponent" is the name of the component
<!--

What do you mean? Can you be more specific?

well I use TAB key binding and it seems the console too, becosas it is displayed, all keys go to the console input,
if I press that then my game gets back his inputs, so how do I change the tab binding for the tilde ~ ?
or any other key

To change the key mapping you can provide a class that implements NiftyInputMapping as the inputMapping attribute of an control. The NiftyInputMapping translates actually key events into NiftyInputEvent which are processed by the control later.
If you don't provide your own then DefaultInputMapping is used which maps the TAB key in combination with shift to toggle the input focus from one control to the other.
You could disable that with your own NiftyInputMapping but I'm not sure if that really helps you because using ~ to toggle the console and using TAB to change the focus to the next control (which is happening in your case, I think?) is not the same.
Maybe it would be better to set the inputMapping at the <screen ... level and let it toggle your console on and off? The DefaultInputMapping uses the KEY_F1 to generate the NiftyStandardInputEvent.ConsoleToggle event. I think one of the controls examples used this to toggle the console on or off with the KEY_F1.

ok ,that makes sens,  I'll check it out, thanks

I cant make it work,
I tried with buttons, TAB always pass from one button to the other
first, if I want to setup an inputMapping to my screen, I need the screen controller to implement KeyInputHandler otherwise I get some error, it does not make any sens, but whatever....
    <screen id="Hud" controller="mygame.appstates.HudAppState" inputMapping="mygame.gui.MyNiftyInputMapping">

then it just sont work at all, I return a null object, so nifty should not bother wit inputs anymore but it does
for some reason it creates NiftyInputEvent from thin air

public NiftyInputEvent convert(KeyboardInputEvent inputEvent)
{
return null; <<<<<<<< to make sure nifty dont process events
}
TAB keeps focusing buttons after buttons
<controlDefinition name="keyBindingComponent">
<panel childLayout="horizontal" backgroundColor="#ffffff33" padding="5px" height="30">
    <control width="20%" id="#bindingButton" name="button" label="$bindingText">
        <interact onClick="$action"/>
    </control>
</panel>

    <layer id="settingsLayer" childLayout="center" backgroundColor="#0000">
        <panel width="400" height="400" childLayout="vertical">
            <control name="keyBindingComponent" bindingDesc="forward" bindingText="???" action="DefineKeyBinding(forward)"/>
            <control name="keyBindingComponent" bindingDesc="backward" bindingText="???" action="DefineKeyBinding(backward)"/>
            <control name="keyBindingComponent" bindingDesc="left" bindingText="???" action="DefineKeyBinding(left)"/>
            <control name="keyBindingComponent" bindingDesc="right" bindingText="???" action="DefineKeyBinding(right)"/>
        </panel>
    </layer>

ok, I made it work assigning it to the buttons instead of the screen


Error in inDL library/jsonlite/libs/x64/jsonlite.dll maximal number of DLLs reached

Commonly observed when calling library(DT) after number of errors invoked by third party library in prior code.
Error in inDL(x, as.logical(local), as.logical(now), ...) :
  unable to load shared object '//R-3.2.1/library/jsonlite/libs/x64/jsonlite.dll':
  `maximal number of DLLs reached...
Error during wrapup: cannot open the connection

Still alive under R.3.3.1 and DT_0.1 and  caret_6.0-70


remove web extension kind

Hi, this is developer from VS Code. As we are rolling out web extensions we are trying to fine tune and simplify the concepts around web extensions. As a result, we are no longer supporting web as an extension kind to identify as a web extension. Instead we infer it from other properties - See https://github.com/microsoft/vscode-docs/blob/vnext/api/extension-guides/web-extensions.md#web-extension-enablement for more details. With latest vsce, it will validate extension kind for appropriate values which are ui, workspace. Since this extension has web extension kind, this PR fixes it properly.

@lostintangent Forgot to inform you the impact of this change - In a web codespace with server, this extension runs on the server.


Add cameraMatrix parameter to findEssentialMat and recoverPose

Current implementations of findEssentialMat and recoverPose use one focal length and principal point parameters to internally calculate normalized points. However, usually there are two focal lengths: fx and fy, and most of the time fx differs from fy. So points normalization in these two functions is not as accurate as it might be. That's why I introduced cameraMatrix parameter to those two functions, because it contains both focal points, and principal point coordinates.
I've left the old function notations as an overloaded ones to maintain backwards compatibility.

thank you for the contribution and sorry for a huge delay! I finally came back to it and realized that you've added a separate function, not modified the existing one, so it can be merged as-is. :+1:


High CPU usage for a1.2.6

I don't know if it's just alpha 1.2.6, or my computer, but when running Alpha 1.2.6 through RetroMCP for testing, compared to a session of a1.2.6 (unmodified) running via BetaCraft, not only is RetroMCP's session have a more unstable CPU usage, but is also using about 40-50% more CPU usage than the BetaCraft session (30-low 50% CPU for BetaCraft, 60-high 90% for RetroMCP using the same settings), making for testing with 2 sessions under RetroMCP unusable due to my laptop lagging a ton.

If you have an Intel GPU, you can try using 3D acceleration by adding -XX:HeapDumpPath=MojangTricksIntelDriversForPerformance_javaw.exe_minecraft.exe.heapdump to your JVM arguments.
Otherwise, I can't really think of anything else that would massively affect performance.

Is this still an issue?

i havent tried it yet, been busy as of late so i havent been reading my
emails too much

Closing issue as inactive.
Please reopen this issue if the issue persists and is reproducible.


I only get token no files

Can't do that much, need error if you can get it

wym u mean that fake error code

if you mean that when i got it i still didn't get anything i followed the tutorial in the dc server

No, I need to get the error that is sent by the code, I can't fix it if I dont know hwere the error is

and it may be, because he closes it

still doesnt work

send error or enable debug mode and send txt

tried it again does work now

xD


Cupscale causing my PC to crash

whenever I try to upscale an image, my PC crashes
there are no Errors, it just crashes for no reason
I'm using my RTX2070S to upscale and the BSRGAN model
the attached file is the pic I want to upscale but it never works

Then you have faulty GPU drivers or hardware.

cupscale causes GPU to work pretty heavily, im going to guess your pc is shutting off from overheating


Permissions and session tokens have different lifetime

Describe the bug
After some time of inactivity, instead of the list of projects, you see the message
Oops!
It looks like you do not have permission to view this environment. Please contact a member with administrator privileges.

To Reproduce
Steps to reproduce the behavior:

Go to any page with the admin privileges
Wait until the permissions token expires
See the error

Expected behavior
The user can see the list of projects or you need to re-login
Screenshots

How are you running Flagsmith?

[ ] Self Hosted with Docker
[ ] Self Hosted with Kubernetes
[x] SaaS at flagsmith.com
[ ] Some other way

Additional context
The browser's page reload won't fix the problem but after you navigate to a different route and come back everything is fine

Thanks for the report - we will take a look!

@Myroslav-mint can you confirm what you mean by 'permissions' and 'session tokens'? Where are you obtaining these tokens from? If you could also provide example curl requests / screenshots to illustrate the issue, that would be a great help. Thanks!

I didn't dig into the codebase, so maybe the problem is in a different place, but all steps to reproduce and a screenshot with the result are on the first message.

@Myroslav-mint I'm sorry, but we are unable to reproduce with the information you have provided. It is not clear what you mean by 'permissions token' or 'any page with the admin privileges'. If you can provide further information about exactly how you are reproducing this issue as per my previous message then we can continue to investigate this. I'm sorry, but without these clarifications we cannot investigate any further. Hope you understand. Thanks!

Ok, let's try one more time. I have full admin privileges and then I go to any project's page and I can use it as normal. After some amount of inactive time I see this screen, but I still logged in and I still have all privileges as before. Why I see this screen? My suggestion was that something wrong with auth tokens, but you say it's not, so please, try to figure out what is going on.

'some amount of inactive time' - can you be any more specific here? Is this a few minutes? A few hours? A few days?
Could you also confirm if you experience this issue using an incognito browser window?

@Myroslav-mint if it's a short enough amount of time, a video of your screen to exhibit the issue would be very helpful too.

Not minutes and neither days. IDK the exact amount of time, but it happens during a workday

I don't think that incognito mode will affect the behavior because several teammates reported the same issue

@Myroslav-mint I'm really sorry but, since we are not able to reproduce it ourselves and no other customers have reported any similar issues, for us to be able to investigate this effectively, we will need your cooperation here.
Please could you try using an incognito browser to access flagsmith for a day. Also, since you mentioned it happens to other teammates, perhaps it is a corporate network issue? Could you try using a mobile network for example instead, or perhaps confirm if this happens when you are not working from an office environment / a VPN?
Also, please could you confirm how you are able to resolve the issue when this happens? Does it kick you back to the login screen?

I'll try it in the incognito mode.
No, this isn't a corporate network issue because we are a full-remote team w/o the corporate network.
After it happens, I can resolve the issue only by clicking on the logo because it keeps me in a 404 redirection loop on other pages.

Are you able to provide a debug console log with any requests that are returning something other than 2XX?

here is an example of an error

This error is totally irrelevant. Please let us know if there are any non 2XX responses in the network tab of the developer console.

No, that shouldn't be related - but it does indicate to me that something is perhaps happening to your network connection around the time that those requests start failing which could be related.
Are there any other failing requests?

No, I don't think so. The tab was idle

Hi @Myroslav-mint, I have tried to reproduce this by logging in as your user and leaving a browser tab running all day and I have not experienced the issue you have raised.
I would recommend trying an incognito window, a different browser and/or a different network.
Since we're unable to reproduce this issue and this is the only report we've had, I am going to close this issue but, since you have a paid account on our SaaS platform, please feel free to reach out via our support chat widget if you'd like further assistance.


Add support for Bezier curves/surfaces syntax

Hello, looks like cstype is not supported either.
Originally posted by @phillipsconsulting in https://github.com/sczerwinski/wavefront-obj-intellij-plugin/issues/470#issuecomment-1523910009

Source: http://paulbourke.net/dataformats/obj/
The following syntax statements are listed in order of use.

cstype rat type

    Free-form geometry statement.

    Specifies the type of curve or surface and indicates a rational or
    non-rational form.

    rat is an optional argument.

    rat specifies a rational form for the curve or surface type. If rat
    is not included, the curve or surface is non-rational

    type specifies the curve or surface type. Allowed types are:

	bmatrix		basis matrix
	bezier		Bezier
	bspline		B-spline
	cardinal        Cardinal
	taylor		Taylor

    There is no default. A value must be supplied.

deg degu degv

    Free-form geometry statement.

    Sets the polynomial degree for curves and surfaces.

    degu is the degree in the u direction. It is required for both
    curves and surfaces.

    degv is the degree in the v direction. It is required only for
    surfaces. For Bezier, B-spline, Taylor, and basis matrix, there is
    no default; a value must be supplied. For Cardinal, the degree is
    always 3. If some other value is given for Cardinal, it will be
    ignored.

bmat u matrix

bmat v matrix

    Free-form geometry statement.

    Sets the basis matrices used for basis matrix curves and surfaces.
    The u and v values must be specified in separate bmat statements.

    NOTE: The deg statement must be given before the bmat statements
    and the size of the matrix must be appropriate for the degree.

    u specifies that the basis matrix is applied in the u direction.
    v specifies that the basis matrix is applied in the v direction.

    matrix lists the contents of the basis matrix with column subscript
    j varying the fastest. If n is the degree in the given u or v
    direction, the matrix (i,j) should be of size (n + 1) x (n + 1).

    There is no default. A value must be supplied.

    NOTE: The arrangement of the matrix is different from that commonly
    found in other references. For more information, see the examples
    at the end of this section and also the section, "Mathematics for
    free-form curves and surfaces."

step stepu stepv

    Free-form geometry statement.

    Sets the step size for curves and surfaces that use a basis
    matrix.

    stepu is the step size in the u direction. It is required for both
    curves and surfaces that use a basis matrix.

    stepv is the step size in the v direction. It is required only for
    surfaces that use a basis matrix. There is no default. A value must
    be supplied.

    When a curve or surface is being evaluated and a transition from
    one segment or patch to the next occurs, the set of control points
    used is incremented by the step size. The appropriate step size
    depends on the representation type, which is expressed through the
    basis matrix, and on the degree.

    That is, suppose we are given a curve with k control points:
			{v , ... v }
			  1       k

    If the curve is of degree n, then n + 1 control points are needed
    for each polynomial segment. If the step size is given as s, then
    the ith polynomial segment, where i = 0 is the first segment, will
    use the control points:
			{v    ,...,v      }
			  is+1      is+n+1

    For example, for Bezier curves, s = n .

    For surfaces, the above description applies independently to each
    parametric direction.

    When you create a file which uses the basis matrix type, be sure to
    specify a step size appropriate for the current curve or surface
    representation.

curv u0 u1 v1 v2 . . .

    Element statement for free-form geometry.

    Specifies a curve, its parameter range, and its control vertices.
    Although curves cannot be shaded or rendered, they are used by
    other Advanced Visualizer programs.

    u0 is the starting parameter value for the curve. This is a
    floating point number.

    u1 is the ending parameter value for the curve. This is a floating
    point number.

    v is the vertex reference number for a control point. You can
    specify multiple control points. A minimum of two control points
    are required for a curve.

    For a non-rational curve, the control points must be 3D. For a
    rational curve, the control points are 3D or 4D. The fourth
    coordinate (weight) defaults to 1.0 if omitted.

curv2  vp1  vp2   vp3. . .

    Free-form geometry statement.

    Specifies a 2D curve on a surface and its control points. A 2D
    curve is used as an outer or inner trimming curve, as a special
    curve, or for connectivity.

    vp is the parameter vertex reference number for the control point.
    You can specify multiple control points. A minimum of two control
    points is required for a 2D curve.

    The control points are parameter vertices because the curve must
    lie in the parameter space of some surface. For a non-rational
    curve, the control vertices can be 2D. For a rational curve, the
    control vertices can be 2D or 3D. The third coordinate (weight)
    defaults to 1.0 if omitted.

surf  s0  s1  t0  t1  v1/vt1/vn1   v2/vt2/vn2 . . .

    Element statement for free-form geometry.

    Specifies a surface, its parameter range, and its control vertices.
    The surface is evaluated within the global parameter range from s0
    to s1 in the u direction and t0 to t1 in the v direction.

    s0 is the starting parameter value for the surface in the u
    direction.

    s1 is the ending parameter value for the surface in the u
    direction.

    t0 is the starting parameter value for the surface in the v
    direction.

    t1 is the ending parameter value for the surface in the v
    direction.

    v is the reference number for a control vertex in the surface.

    vt is an optional argument.

    vt is the reference number for a texture vertex in the surface.  It
    must always follow the first slash.

    vn is an optional argument.

    vn is the reference number for a vertex normal in the surface.  It
    must always follow the second slash.

    For a non-rational surface, the control vertices are 3D.  For a
    rational surface the control vertices can be 3D or 4D.  The fourth
    coordinate (weight) defaults to 1.0 if ommitted.

    NOTE: For more information on the ordering of control points for
    survaces, refer to the section on surfaces and control points in
    "mathematics of free-form curves/surfaces" at the end of this
    appendix.

You can specify additional information for free-form curve and surface
elements using a series of statements called body statements. The
series is concluded by an end statement.

Body statements are valid only when they appear between the free-form
element statement (curv, curv2, surf) and the end statement. If they
are anywhere else in the .obj file, they do not have any effect.

You can use body statements to specify the following values:

o       parameter
o       knot vector
o       trimming loop
o       hole
o       special curve
o       special point

You cannot use any other statements between the free-form curve or
surface statement and the end statement. Using any other of type of
statement may cause unpredictable results.

This portion of a sample file shows the knot vector values for a
rational B-spline surface with a trimming loop. Notice the end
statement to conclude the body statements.

    cstype rat bspline
    deg 2 2
    surf -1.0 2.5 -2.0 2.0 -9 -8 -7 -6 -5 -4 -3 -2 -1
    parm u -1.00 -1.00 -1.00 2.50 2.50 2.50
    parm v -2.00 -2.00 -2.00 -2.00 -2.00 -2.00
    trim 0.0 2.0 1
    end

vp u v w

    Free-form geometry statement.

    Specifies a point in the parameter space of a curve or surface.

    The usage determines how many coordinates are required. Special
    points for curves require a 1D control point (u only) in the
    parameter space of the curve. Special points for surfaces require a
    2D point (u and v) in the parameter space of the surface. Control
    points for non-rational trimming curves require u and v
    coordinates. Control points for rational trimming curves require u,
    v, and w (weight) coordinates.

    u is the point in the parameter space of a curve or the first
    coordinate in the parameter space of a surface.

    v is the second coordinate in the parameter space of a surface.

    w is the weight required for rational trimming curves. If you do
    not specify a value for w, it defaults to 1.0.

    NOTE: For additional information on parameter vertices, see the
    curv2 and sp statements


Implements a command line interface.

The CLI, which shares a common design with the to-be-implemented CLIs of the other Ion implementations, serves as the foundation of the to-be-implemented cross-implementation test harness, and provides extension points (for filtering, custom traversals, performance testing, symbol table generation, etc.) to enhance its utility as a general-purpose Ion processing tool.
The core of the CLI is the events module, which provides support for reading Ion streams as a stream of in-memory IonEvents, which are used to describe reader behavior and dictate writer behavior. These IonEventStreams may also be compared for equivalence under the Ion data model.
This PR is bigger than it looks because some files both moved and changed, which shows up as a delete/create in the diff. These files are:

test/ion_assert.cpp -> tools/events/ion_event_equivalence.cpp
test/ion_assert.h -> tools/events/ion_event_equivalence_impl.h
test/ion_event_stream.cpp -> tools/events/ion_event_stream.cpp
test/ion_event_stream.h -> tools/events/inc/ion_event_stream.h

Additionally, floating_point_util.h is copied (with attribution) from google test to avoid a google test dependency in non-test code.
The following files contain fixes to minor problems that were uncovered during implementation of the CLI, but aren't necessarily directly related to the CLI. I fixed them as I went, so they got intertwined with the other commits. They can be reviewed in a separate logical unit than the rest.

ionc/inc/ion_debug.h
ionc/inc/ion_int.h
ionc/inc/ion_symbol_table.h
ionc/inc/ion_writer.h
ionc/ion_binary.c
ionc/ion_binary.h
ionc/ion_helpers.c
ionc/ion_helpers.h
ionc/ion_int.c
ionc/ion_stream.c
ionc/ion_stream_impl.h
ionc/ion_writer.c
ionc/ion_writer_binary.c
ionc/ion_writer_impl.h
ionc/ion_writer_text.c

The following files contain nothing more than basic renames and small refactorings, and can be skimmed:

test/ion_test_util.h/cpp
test/test_ion_binary.cpp
test/test_ion_decimal.cpp
test/test_ion_symbol.cpp
test/test_ion_text.cpp
test/test_ion_values.cpp
test/gather_vectors.h/cpp
test/test_vectors.cpp

Of the remaining files, I'd suggest reviewing the following groups of files in order:

tools/events/ion_event_util.h, tools/events/ion_event_util.cpp
tools/events/inc/ion_event_stream.h, tools/events/ion_event_stream_impl.h, tools/events/ion_event_stream.cpp
tools/events/inc/ion_event_equivalence.h, tools/events/ion_event_equivalence_impl.h, tools/events/ion_event_equivalence.cpp
tools/cli/main.cpp, tools/cli/cli.h, tools/cli/cli.cpp
test/test_ion_cli.cpp

have built this locally and experimented a bit with the executable at cmake-build-debug/toos/cli
Given:
-f1.ion with contents { a:"this is a struct"}
-f2.ion with contentx { a:"this is a struct", b: "this is another field" }
Executing the command ion compare f1.ion f2.ion yields 2 identical structs to the output.  This doesn't seem intended.
I have only just begun reviewing this...  More comments to come...

Also,
ion compare f1.ion yields no output.  It might be nice to explain that at least 2 files are required for the compare operation.

@dlurton Here's the output I get when I run your example:
{
  result_type:"NOT_EQUAL",
  message:"Did not find matching field for (text=b, local_sid=-1, location=(NULL, 0))",
  lhs:{
    location:"f2.ion",
    event:{
      event_type:SCALAR,
      ion_type:STRING,
      field_name:{
        text:"b"
      },
      value_text:"\"this is another field\"",
      value_binary:[
        224,
        1,
        0,
        234,
        142,
        149,
        116,
        104,
        105,
        115,
        32,
        105,
        115,
        32,
        97,
        110,
        111,
        116,
        104,
        101,
        114,
        32,
        102,
        105,
        101,
        108,
        100
      ],
      depth:1
    },
    event_index:2
  },
  rhs:{
    location:"f1.ion",
    event:{
      event_type:CONTAINER_END,
      ion_type:STRUCT,
      depth:0
    },
    event_index:2
  }
}
{
  result_type:"NOT_EQUAL",
  message:"Did not find matching field for (text=b, local_sid=-1, location=(NULL, 0))",
  lhs:{
    location:"f2.ion",
    event:{
      event_type:SCALAR,
      ion_type:STRING,
      field_name:{
        text:"b"
      },
      value_text:"\"this is another field\"",
      value_binary:[
        224,
        1,
        0,
        234,
        142,
        149,
        116,
        104,
        105,
        115,
        32,
        105,
        115,
        32,
        97,
        110,
        111,
        116,
        104,
        101,
        114,
        32,
        102,
        105,
        101,
        108,
        100
      ],
      depth:1
    },
    event_index:2
  },
  rhs:{
    location:"f1.ion",
    event:{
      event_type:CONTAINER_END,
      ion_type:STRUCT,
      depth:0
    },
    event_index:2
  }
}

That looks correct to me. Which revision did you build and run?
Also, it is expected that ion compare f1.ion yields no output, because f1.ion is equivalent to itself. Two files are not required for the compare operation.

I checked out this PR before building and the result is the same for me.  Why repeat the same struct twice when there is only one difference between f1.ion and f2.ion?

@dlurton Because reflexivity is not assumed. f1 is compared to f2, then f2 is compared to f1.

@tgregg I checked out this repo, pulled down this PR in a local branch, checked out said local branch, then built, then tested, in that order.
What is an example of Ion values for ionValue1 and ionValue2 where ionValue1 is equivalent to ionValue2 while ionValue2 is not equivalent to ionValue1?

@dlurton Then did you just misinterpret the results, or did you actually see incorrect results?
Under the Ion data model, it can't happen. But the CLI is necessarily coupled to an implementation, and it's not hard to imagine an implementation which contains bugs that prevent it from correctly handling reflexive equality. That's why the CLI should check for it.

@tgregg If this was the intended design then I did not see incorrect results, however I question the utility of always outputting both comparisons.  Is the entity that is intended to be executing this CLI another program or a human working at the command line who wants to know the differences between sets of Ion data?  Reflexivity shouldn't be a concern of the human who (hopefully) can assume that the implementation is without bugs.  It's a good idea for the CLI to check for reflexivity, but from the perspective of the human it would be very helpful if the CLI tool quietly performed this check and only displayed the second result (and an error message) when irreflexivity was detected.
On the other-hand, if the only intended use for this CLI is to be executed by a another program, I could see scenarios where it would be useful to have both results.
Perhaps the happy medium is to only show the first result (except in case of irreflexivity) by default and include an option to always display both results?

@dlurton It's intended to be executed by both non-test (humans) and test (other programs such as the centralized test harness) users. I agree that non-test users should be able to assume that the code is bug-free, and should therefore be able to assume that reflexivity holds. It does complicate the code a little bit, because the tool will always need to know which comparisons are reflexive, but I don't think it will be too bad. I'll think about whether there should be an option to display both, or whether displaying one and erroring if the reflexive result differs will work for both test and non-test users.
See #102

I went through the instructions located here to get valgrind on my macbook and then ran it against the all_tests executable.  There are a number of accesses of uninitialized memory and memory leaks.  Is this normal for your all_tests and if not is this a valid way to get valgrind on MacOS?

@dlurton Yes, there are known leaks in some of the all_tests unit tests. They do need to be fixed, but using all_tests as the entry point for memory profiling ion-c won't give an accurate reflection of the library as vended to users.


usdchecker doesn't recognize .mtlx file references

When usdchecker runs on a StandardShaderBall commit, it fails to recognize the .mtlx files.
Maybe the USD we're using just needs to be updated to the latest, which includes UsdMaterialX by default?

The GH script installs the latest usd-core PyPI package, which is 23.11. However, it seems like usd-core has been built without MaterialX support enabled :(
https://github.com/PixarAnimationStudios/OpenUSD/blob/3bcae24e7d62b732b9cb2ac47cdcaab41bf9ca7f/azure-pypi-pipeline.yml#L90
Additionally, this PR would be needed to make usdchecker happy: https://github.com/PixarAnimationStudios/OpenUSD/pull/2966
I guess one workaround could be, since the actual Python script resides in this repo, to ignore/demote error messages of a certain type according to a RegEx: https://github.com/usd-wg/assets/blob/746b850badda03fdc1961dd82fda50b114c90538/.github/usdchecker.py#L122C5-L140

Argh :( That's unfortunate, thanks for figuring that out.
@jcowles what do you think about the suggestion by @pablode, to modify our usdchecker script to ignore missing references that refer to .mtlx documents? Or if we have a PR where that's the only failure (like #80), do we just merge it anyway? With the hope that usd-core will eventually end up with mtlx included?

Closing as the PR was merged.


hadoop ai needs to be deployed separately?

nope, hadoop-ai have only build process, it will be used as base image in yarn and hdfs deployment.

Thank you for your answer


Enhancement Request : Highlight Active Tab with better contrast colors

It is hard to differentiate which tab is active tab when working with multiple tabs .

Increase in height of tab and higer padding will be better
Something like below will be much better

Check the latest version.


ModuleNotFoundError: No module named 'face_alignment'

hi, when i click on submit button, it give me this:
croping pose_video
Traceback (most recent call last):
File "data_preprocess/crop_video.py", line 1, in
import face_alignment
ModuleNotFoundError: No module named 'face_alignment'
File not found: test_data\crop_resample_pose_source1.mp4
where can i get that :
ModuleNotFoundError: No module named 'face_alignment'?

hi, when i click on submit button, it give me this:
croping pose_video Traceback (most recent call last): File "data_preprocess/crop_video.py", line 1, in import face_alignment ModuleNotFoundError: No module named 'face_alignment' File not found: test_data\crop_resample_pose_source1.mp4
where can i get that : ModuleNotFoundError: No module named 'face_alignment'?

Please first install face-alignment by pip install face-alignment


refactor: define trait for portal data store

What was wrong?
There is no abstraction for the data store layer, so a database must be set up and torn down even in tests where the particular database is not relevant.
How was it fixed?

Define trait PortalContentStore to provide abstraction over the data store layer.
Implement PortalContentStore for PortalStorage.
Replace PortalStorage with PortalContentStore generic type parameter in OverlayProtocol and OverlayService.
Define MemoryPortalContentStore for use in tests.

To-Do

[ ] Add entry to the release notes (may forgo for trivial changes)
[ ] Clean up commit history

Here I removed the tests for should_store and distance_to_key. For should_store, that method was removed, so there is no test to have for it. For distance_to_key, it was not clear to me what we test there.
I preferred the combination of is_key_within_radius and get in the PortalContentStore trait over should_store. Those two methods are useful in isolation, and when combined, they are equivalent to should_store. get would be in the trait regardless.

@njgheorghita I appreciate the healthy skepticism! The intent here is to abstract the portal data store, which is effectively a key-value store. With this abstraction, we can use something like a HashMap in tests where we do not care about the particular database storing the data. This removes failure modes from certain tests and also improves the performance of tests where we do not need to set up and tear down database connections.
Based on the comments, a point of contention is the change in naming, i.e. why put instead of store? My main argument here is that this abstraction is a key-value store, and get and put are the canonical names there. As of now, put is a wrapper around the pre-existing store method of PortalStorage, but this wrapping was only done to minimize the diff. In a subsequent refactor, there would only be one method that contains this logic.
Another point of contention is the absence of should_store in the trait definition, and as a consequence, the need to call is_key_within_radius in combination with get to achieve the same thing. Why is is_key_within_radius useful in isolation? It seems useful to know that you are not storing something because it falls outside of the data store's radius versus because it already exists in the data store. This is also a question of API design, which is more subjective. My preference is for an API that provides the base functionality a caller needs to use the thing correctly. That might require more of the caller, for example to combine function calls, but it keeps the trait definition simple.

@jacobkaufmann Just playing devils advocate here: w/r to "It seems useful to know that you are not storing something because it falls outside of the data store's radius versus because it already exists in the data store." ... Ok, but why? What is a situation where this will be useful?

@njgheorghita I will reiterate my argument above that the new interface is actually a simplification. IMO this is the stronger part of the argument in favor of the decomposition.
Consider
if store.should_store(key) { .. }

and
if store.is_key_within_radius(key) && !store.contains_key(key) { .. }

It's very unclear what sort of logic is going on in the former. I understand that there is an argument that we should not care if that logic is opaque, and that is the benefit of the abstraction. I'm not very sympathetic to that argument here. The latter is only slightly more verbose, and much more straightforward to reason about. The latter is also easier to implement and test for some arbitrary data store.
I admit the utility of is_key_within_radius by itself is limited. One thing that I could imagine is that it would be helpful to assess whether our peers have an accurate view of our radius. If our peers often offer us data that is outside the radius, then it would seem that those peers are behaving incorrectly or do not have an accurate view of our radius.
The purpose of the PR is to define a solid abstraction for the data store layer. If we were to break things up, I would prefer to leave out the in-memory data store and focus on getting the abstraction right.

@jacobkaufmann Good point, calling code does need to know why we shouldn't store data. We should eventually be acting on the case where we're offered data outside our radius, and keeping these functions separate is a better solution than should_store returning a custom enum that specifies the reason or something like that.
I'm fine with either finalizing that change here (with contains_key added to the trait) or keeping should_store and then modifying the trait later when we modify the calling code to act on out-of-radius OFFERs.
Two more questions:

How can we test that these updates to the calling code don't break any OFFER/ACCEPT logic?

For distance_to_key, it was not clear to me what we test there.

Why not test PortalStorage::distance_to_key?

@jacobkaufmann Point taken, that api was somewhat obscure. I'm 👍 on  the api changes. Let's keep both the api changes and hashmap support changes in this pr.

@njgheorghita @mrferris so I was talking with @carver offline last week, and the middle ground that we reached was to keep the shape of the storage API the same, but to rename should_store to something more descriptive, and to remove the store_if_should pattern.
We agreed that the key-value store should be separate from the logic associated with radius management. Ultimately, this may involve a further abstraction for the key-value store within the content store, but for now we can use a single outer trait.
(I may have misremembered some of the details about the discussion referenced above.)
To address your questions @mrferris:

Can you clarify?
Are we just testing the conversion of the content key to a content ID? That is quite straightforward. The distance computation logic is already covered by the tests for the distance function.

@jacobkaufmann

No longer relevant, but I was wondering if there's any testing that would validate that the should_store logic being moved out of should_store and made more granular into overlay.rs was behaving properly (eg wouldn't accidentally accept content we don't want to accept). This isn't relevant anymore with the direction you've gone with renaming should_store to is_key_within_radius_and_unavailable.

Yea good point about the distance functions themselves already being tested

@mrferris any objections? The PR is still showing you as 👎🏻

@carver I think @mrferris was waiting on the re-request for review (just done) to acknowledge all comments addressed.

@jacobkaufmann Re-reviewing now, and yea @carver I thought that doing a "comment" review after a "request changes" would change from a 👎 to a neutral, but apparently not!

@mrferris @njgheorghita I added some logic to accommodate the accumulator in the MemoryContentStore along with some tests for the MemoryContentStore. If there are any final comments (aside from resolving merge conflicts), please let me know. Otherwise I will merge by EOD.


[backport] Fix PickleDataset crash when using multiprocessing

Backport of #7625

[automatic post] Jenkins, test this please.

Jenkins CI test (for commit fed20c1b0109f5a6f22a8c958fc912b9e9ba12b3, target branch v6) failed with status FAILURE.

CI unrelated


Explore the use of canvas for some drawing instead of svg

D3 v4 now has better support for rendering to canvas.
We should try to move some of our svg elements to canvas (e.g. filled areas) to see if we get a performance boost.
cc @batpad @kepta @geohacker

For inspiration:   https://bocoup.com/blog/smoothly-animate-thousands-of-points-with-html5-canvas-and-d3

Maybe it is possible to go straight to regl.
Guy did so:

UPDATE: That blog post has now been written. Check out how to animate 100,000 points in regl for details.
https://peterbeshai.com/beautifully-animate-points-with-webgl-and-regl.html

It look realistically. Everything on Earth supports WebGL now.

It looks like project matured to truly stable form year ago (graph)[https://github.com/regl-project/regl/graphs/contributors].
It may look like it died-out. But no, - it's maturity of Web-based library.
Main developer reviews PRs and bugreports. And many issues/PRs closed due to functionality already present. Or discussions goes on. PRs get merged.
He responds to messages: regl-project/regl#424, guy there says that it uses it in serious production and it is good for him.
But looks like main developer not eager to write new code. Maybe he busy, or life.
He reviews, merges, writes responses.
But there is also a lot of superb other choices:
(https://gist.github.com/dmnsgn/76878ba6903cf15789b712464875cfdc)

HTML5Test.com results

There is a lot of superb WebGL choices:
(https://gist.github.com/dmnsgn/76878ba6903cf15789b712464875cfdc)

d3-jetpack has a thing to help with sharing coordinate systems between svg and canvas:
https://bl.ocks.org/1wheel/f9b9909f10ed0f01780c5338ad38bd50


CI/CD fix related to OpenGL installation

Fix based on https://github.com/pyansys/pydpf-core/pull/196/files.

Fix copied in #68


[Docs] add distributed dataset mapping usage

What does this PR do?
This PR adds a usage guide on disitributed mapping a function on datasets

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.