Test fails in SIMULATION mode

I followed the guide to build openEnclave (https://github.com/Microsoft/openenclave/blob/v0.5.x/docs/GettingStartedDocs/Contributors/SimulatorGettingStarted.md) on Ubuntu 16.04 LTS 64-bits running in a VM and I got tons of ctest failures like this ( I just run the first that fail for readability):
 ken@openenclavesim ⮀ ~/openenclave/build/tests/safecrt ⮀ ⭠ master ⮀ OE_SIMULATION=1 ctest -V
UpdateCTestConfiguration  from :/home/ken/openenclave/build/tests/safecrt/DartConfiguration.tcl
UpdateCTestConfiguration  from :/home/ken/openenclave/build/tests/safecrt/DartConfiguration.tcl
Test project /home/ken/openenclave/build/tests/safecrt
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 1
    Start 1: tests/safecrt

1: Test command: /home/ken/openenclave/build/tests/safecrt/host/safecrt_host "/home/ken/openenclave/build/tests/safecrt/enc/safecrt_enc"
1: Test timeout computed to be: 10000000
1/1 Test #1: tests/safecrt ....................***Exception: Illegal  0.28 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.28 sec

The following tests FAILED:
	  1 - tests/safecrt (ILLEGAL)
Errors while running CTest

any clue?

@akiannillo
You could do a couple of this to gain more info.

ctest -V -R safecrt
This should just run the safecrt test in verbose mode.

Can you try running it under the debugger (oegdb)?
oegdb (or oe-gdb) should exist under build
$ export OE_SIMULATION=1
$ output/bin/oegdb --args tests/safecrt/host/safecrt_host tests/safecrt/enc/safecrt_enc
And press 'r' when the debugger prompt comes up

Hopefully the above should give us information to figure out what is going on.

Yes, no problem.
FOR POINT 1
ken@openenclavesim ⮀ ~/openenclave/build ⮀ ⭠ master ⮀ export OE_SIMULATION=1
ken@openenclavesim ⮀ ~/openenclave/build ⮀ ⭠ master ⮀ ctest -V -R safecrt
UpdateCTestConfiguration  from :/home/ken/openenclave/build/DartConfiguration.tcl
Parse Config file:/home/ken/openenclave/build/DartConfiguration.tcl
UpdateCTestConfiguration  from :/home/ken/openenclave/build/DartConfiguration.tcl
Parse Config file:/home/ken/openenclave/build/DartConfiguration.tcl
Test project /home/ken/openenclave/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 2
    Start 2: tests/safecrt

2: Test command: /home/ken/openenclave/build/tests/safecrt/host/safecrt_host "/home/ken/openenclave/build/tests/safecrt/enc/safecrt_enc"
2: Test timeout computed to be: 1500
1/1 Test #2: tests/safecrt ....................***Exception: Illegal  0.31 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.32 sec

The following tests FAILED:
	  2 - tests/safecrt (ILLEGAL)
Errors while running CTest

FOR POINT 2
ken@openenclavesim ⮀ ~/openenclave/build ⮀ ⭠ master ⮀ output/bin/oegdb --args tests/safecrt/host/safecrt_host tests/safecrt/enc/safecrt_enc
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Source directories searched: /home/ken/openenclave/build/output/lib/openenclave/debugger/gdb-sgx-plugin:$cdir:$cwd
No symbol table is loaded.  Use the "file" command.
No symbol table is loaded.  Use the "file" command.
No symbol table is loaded.  Use the "file" command.
Setting environment variable "LD_PRELOAD" to null value.
Reading symbols from tests/safecrt/host/safecrt_host...done.
oegdb: Found global enclave list.
oegdb: 0 enclaves in global enclave list.
(gdb) c
The program is not being run.
(gdb) run
Starting program: /home/ken/openenclave/build/tests/safecrt/host/safecrt_host tests/safecrt/enc/safecrt_enc
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
oegdb: Enclave /home/ken/openenclave/build/tests/safecrt/enc/safecrt_enc loaded in simulation mode

Program received signal SIGILL, Illegal instruction.
0x00007ffff50ae533 in oe_cleanup_xstates () at /home/ken/openenclave/enclave/core/sgx/exception.c:417
417	    __builtin_ia32_xrstor64(xsave_area, restore_mask);
(gdb) c
Continuing.

Program received signal SIGILL, Illegal instruction.
0x00007ffff50ae533 in oe_cleanup_xstates () at /home/ken/openenclave/enclave/core/sgx/exception.c:417
417	    __builtin_ia32_xrstor64(xsave_area, restore_mask);
(gdb) c
Continuing.

Program terminated with signal SIGILL, Illegal instruction.
The program no longer exists.
(gdb) c
The program is not being run.
(gdb)

@akiannillo Thanks for the info. Interesting. Going by the oegdb output, xrstor64 instruction is raising an ILLEGAL instruction.
Can you post the output of the lscpu command in your VM? I suspect that the instruction is somehow not available in your VM.

@akiannillo @anakrish - I can take a look at this later today. I added a workaround for xrstor64 faulting with optimized builds as the reserved guard area was incorrectly accessed but in hardware mode (#144 ).

As @anakrish mentioned, this could be due to the instruction not supported on your platform.

This is the output of lscpu:
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    1
Core(s) per socket:    2
Socket(s):             4
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            15
Model:                 6
Model name:            Common KVM processor
Stepping:              1
CPU MHz:               2194.916
BogoMIPS:              4389.83
Hypervisor vendor:     KVM
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              4096K
L3 cache:              16384K
NUMA node0 CPU(s):     0-7
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology pni cx16 x2apic hypervisor lahf_lm

@akiannillo - I don't see xsave in the flags. This is needed for xrstor64 to work.

For comparison, here is lspu on my VM:
anakrish@accVM:/mnt/build/master/tests/echo$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  1
Core(s) per socket:  2
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               158
Model name:          Intel(R) Xeon(R) E-2176G CPU @ 3.70GHz
Stepping:            10
CPU MHz:             3696.003
BogoMIPS:            7392.00
Virtualization:      VT-x
Hypervisor vendor:   Microsoft
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            12288K
NUMA node0 CPU(s):   0,1
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti tpr_shadow vnmi ept vpid fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xsaves

@akiannillo
My VM has xsaveopt xsavec xsaves features.
Is it possible for you to check whether the host supports xsave by running lscpu on your host?
Is it possible to enable xsave CPU feature in your KVM virtual machine?
I am not sure about the exact syntax, but the command here (https://stackoverflow.com/questions/34801420/creating-a-vm-with-multiple-vnics-using-qemu) seems to enable xsave feature for a KVM based VM.

@anakrish - xsave, which is also seen in your list earlier should be enough for XRSTOR64 to work (hence I did not mention the others).
xsavec/xsaveopt are the compacted and optimized versions of xsave.
xsaves indicates support for XSAVE that can be executed only in supervisor mode.
These options can be enabled in the Linux kernel/VM but not required for OE simulation runs (or hardware runs either).

I changed the emulated cpu with this:
ken@openenclavesim ⮀ ~/openenclave/build ⮀ ⭠ master ⮀ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    1
Core(s) per socket:    2
Socket(s):             4
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 79
Model name:            Intel(R) Xeon(R) CPU E5-4650 v4 @ 2.20GHz
Stepping:              1
CPU MHz:               2194.916
BogoMIPS:              4389.83
Hypervisor vendor:     KVM
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              4096K
L3 cache:              16384K
NUMA node0 CPU(s):     0-7
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt arat

and now the test passes

This is great.

@akiannillo Thanks for reporting back.

I am closing this issue. Follow up items created in #1770


CFP: Seamless migration from KVStore to CRD identity allocation mode

Problem Context and Description
We are currently considering migrating from KVStore-backed identities (--identity-allocation-mode=kvstore) to CRD-backed identities (--identity-allocation-mode=crd) so as to get rid of dedicated etcd clusters for Cilium. The main goals are reducing costs, reducing operational complexity and simplifying credential management.
In order to perform this migration, the official docs suggest using the cilium-dbg preflight migrate-identity CLI tool to perform a "one-shot" identity migration from KVStore to CRDs and then re-deploy Cilium with --identity-allocation-mode=crd. The main problems with this approach are:

All the identities created in the KVStore after the migrate-identity tool finished running and before all Cilium agents are re-deployed with the new --identity-allocation-mode=crd flag, will be missing from CRDs
There is no easy way to revert back to --identity-allocation-mode=kvstore if something goes wrong after the migration or no easy way to test the CRD mode performance without actually switching production environments to it.
Too many manual ops (we have many dozens of etcd clusters to migrate from so we'll need to invest in some additional automation to run this migration tooling)

Solution Proposal
Solution Constraints
The main constraints we considered when choosing a new solution were:

Possibility of slow and gradual automated rollout across Kubernetes clusters
Possibility of quick rollback
Identities should end up being consistent without manual intervention
Migration should be transparent to applications
It should be possible to monitor the state of identities in etcd vs CRD and it should be possible to get alerted about inconsistencies

Solution Description
We propose to add a new "Double Write" Identity Allocation Mode (--identity-allocation-mode=double-write) which will have Cilium allocating identities as KVStore values and as CRDs at the same time. This mode will also have a flag which would allow controlling whether reads happen from KVStore or from CRDs.
The high-level migration plan will look as follows:

Starting state: Cilium is running in KVStore mode

Switch Cilium to “Double Write” mode with all reads happening from the KVStore still. This is almost the same as the pure KVStore mode with the only difference being that all identities are duplicated as CRDs but are not used.

Switch Cilium to “Double Write” mode with all reads happening from CRDs. This is equivalent to Cilium running in pure CRD mode but identities will still be updated in etcd for possibility of fast rollback

Switch Cilium to CRD mode. etcd will no longer be used and will be ready for decommission

This will allow us to perform a gradual and seamless migration while gaining confidence in the CRD mode with possibility of fast rollback at steps 2 and 3. This won't require any additional tooling or ops since we will simply follow our usual Cilium deployment process.
In order to monitor the migration, we can emit more metrics and logs from the Cilium Operator (see the PoC PR linked below)
Proof of Concept Code
I wrote the necessary code to implement this new "double-write" mode and successfully performed a seamless identity migration in one of our test Kubernetes clusters with Cilium 1.13 with 7000+ Identities.
The main proposal is this PR for Cilium 1.16: https://github.com/cilium/cilium/pull/30690.
For full reference, I also have the 1.13 PoC here: https://github.com/DataDog/cilium/commit/3b5410cb2b8375fe31847e207973f99ab178d270 (The main idea is the same but the implementation is obviously a bit different because of various changes introduced to the codebase in between these versions)

cc @marseel

So I was thinking about how safe each step of this migration is and 1->2, 3->4 migrations steps seem to be safe, but 2->3 has a risk of introducing conflicts - I think it's important that both 2 and 3 steps have the same order of writing identities to backends - for example, first CRD and then kvstore - essentially we don't want to have two identities with the same id but different labels in two  different backends. Also, as kvstore and CRD identity GC works differently, depending on "read" mode we might end up with choosing identity id that is still being used in other backend.
Another thing to consider is that upgrade 2->3 might take a while and is not instantaneous.
I would suggest that probably it makes sense for mode 2&3 to actually read identities from both backends at the same time so the risk of introducing new conflicting identity between backends will be minimal.

I would suggest that probably it makes sense for mode 2&3 to actually read identities from both backends at the same time so the risk of introducing new conflicting identity between backends will be minimal.

When we read from both backends at the same time, should we fail or attempt to remediate if we detect an inconsistency in the labels for the same identity id? Not sure how to properly remediate this to be honest, since a given Agent might not necessarily be able to tell whether it's the CRDs or etcd which should be considered as the source of truth in the cluster.
Just a quick illustration for this case during the upgrade from stage 2 to stage 3:

At this point, Agent 1 has been updated to consider CRDs as the source of truth for all identities, whereas Agent 2 and 3 are still using etcd and they can't really know what's the actual expected source of truth here, so remediation looks complicated without having a way to signal to all Agents which is the actual source of truth (maybe the Operator can do this somehow?)
Otherwise, I wonder if the Agents should just fail when an inconsistency is detected on a read and we can have the Operator do async remediation in a dedicated controller ?

This is an interesting use-case. I do have a couple of questions:

What made you choose etcd as the KVStore in the first place?
What scale are you running at?

We chose etcd years ago when it was basically the only viable option to run Cilium at our scale (which is in the 10000s of nodes / 100000s of pods across many clusters)

That makes sense. Does CRD identity allocation now work at that scale? After reading the docs it seems like etcd is still the recommended way. We've found the same behavior in our testing.

CRD identity allocation is usually not the biggest contributing factor to k8s control plane overall load so I wouldn't worry too much about it. The biggest contributing factor is the propagation of CiliumEndpoints (churn is much higher as compared to identities). Right now, CiliumEndpointSlices (they are still beta though) aim at lowering this overhead for k8s control plane. I hope to perform some tests of CiliumEndpointSlices with CRDs in upcoming months for the scale of 5k nodes, but our current understanding is that they should work at that scale - of course assuming proper k8s control plane configuration with k8s upstream recommendations.

Not stale

I think that we can close this issue now that https://github.com/cilium/cilium/pull/31920 has been merged, thanks! :rocket:


stdenv: get rid of libkrb5 in the darwin closure

Motivation for this change
There's a curlFull for cases where these extra features are required,
curl is part of the darwin stdenv so most options should not be enabled
there.
/cc @NixOS/darwin-maintainers
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of the stdenv and some basic packages
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Are there any updates on this pull request, please?

4 years old, closing.


Update Composer dependencies (2024-06-06-00-24)

Visual regression test passed!
View the visual regression test report


Disable NFC scan from phone home screen, make it possible to scan tag only from dashboard or force claim screens

QA passed


Cannot find the requested resource (get f5ipams.fic.f5.com)

Setup Details
FIC Version : 0.1.3
CIS Version : 2.4.1
FIC Build: f5networks/f5-ipam-controller:latest
CIS Build: f5networks/k8s-bigip-ctlr:latest
BIGIP Version: Big IP 15.1
AS3 Version: 3.28
Orchestration: K8S
Orchestration Version: 1.20.8
Additional Setup details: <Platform/CNI Plugins/ cluster nodes/ etc>
1 standalone BIGIP
1 K8s cluster
Flannel
Description
I deployed the F5 IPAM Controller as per the instructions on https://clouddocs.f5.com/containers/latest/userguide/ipam/ but I am getting the following error on the logs of the controller.
Any configuration I do, using the ipamLabel, the configuration is never pushed on the F5 BIGIP.
***************************************************************************************
2021/07/05 12:08:17 [DEBUG] [STORE]  37  <IP_ADDRESS> 1 Production
2021/07/05 12:08:17 [DEBUG] [STORE]  38  <IP_ADDRESS> 1 Production
2021/07/05 12:08:17 [DEBUG] [STORE]  39  <IP_ADDRESS> 1 Production
2021/07/05 12:08:17 [DEBUG] [STORE]  40  <IP_ADDRESS> 1 Production
2021/07/05 12:08:17 [INFO] [CORE] Controller started
2021/07/05 12:08:17 [INFO] Starting IPAMClient Informer
I0705 12:08:17.519011       1 shared_informer.go:240] Waiting for caches to sync for F5 IPAMClient Controller
E0705 12:08:17.536543       1 reflector.go:138] github.com/F5Networks/f5-ipam-controller/pkg/ipammachinery/informers.go:35: Failed to watch *v1.F5IPAM: failed to list *v1.F5IPAM: the server could not find the requested resource (get f5ipams.fic.f5.com)
E0705 12:08:18.826250       1 reflector.go:138] github.com/F5Networks/f5-ipam-controller/pkg/ipammachinery/informers.go:35: Failed to watch *v1.F5IPAM: failed to list *v1.F5IPAM: the server could not find the requested resource (get f5ipams.fic.f5.com)
E0705 12:08:21.941636       1 reflector.go:138] github.com/F5Networks/f5-ipam-controller/pkg/ipammachinery/informers.go:35: Failed to watch *v1.F5IPAM: failed to list *v1.F5IPAM: the server could not find the requested resource (get f5ipams.fic.f5.com)
E0705 12:08:27.279356       1 reflector.go:138] github.com/F5Networks/f5-ipam-controller/pkg/ipammachinery/informers.go:35: Failed to watch *v1.F5IPAM: failed to list *v1.F5IPAM: the server could not find the requested resource (get f5ipams.fic.f5.com)

****************************************************************************************************************************

I am getting a similar log from the CIS Controller when i enable the F5 IPAM ("--ipam=true")
*******************************************************************************************
2021/07/05 12:05:35 [DEBUG] [CORE] NodePoller (0xc0007faa20) notifying listener: {l:0xc0005acc60 s:0xc0005accc0}
2021/07/05 12:05:35 [DEBUG] [CORE] NodePoller (0xc0007faa20) listener callback - num items: 4 err: <nil>
2021/07/05 12:06:02 [DEBUG] [2021-07-05 12:06:02,237 __main__ DEBUG] config handler woken for reset
2021/07/05 12:06:02 [DEBUG] [2021-07-05 12:06:02,238 __main__ DEBUG] loaded configuration file successfully
2021/07/05 12:06:02 [DEBUG] [2021-07-05 12:06:02,238 __main__ DEBUG] NET Config: {}
2021/07/05 12:06:02 [DEBUG] [2021-07-05 12:06:02,238 __main__ DEBUG] loaded configuration file successfully
2021/07/05 12:06:02 [DEBUG] [2021-07-05 12:06:02,238 __main__ DEBUG] updating tasks finished, took 0.0006988048553
466797 seconds
E0705 12:06:02.546954       1 reflector.go:138] github.com/F5Networks/f5-ipam-controller/pkg/ipammachinery/informe
rs.go:35: Failed to watch *v1.F5IPAM: failed to list *v1.F5IPAM: the server could not find the requested resource
(get f5ipams.fic.f5.com)
2021/07/05 12:06:05 [DEBUG] [CORE] NodePoller (0xc0007faa20) ready to poll, last wait: 30s

2021/07/05 12:06:05 [DEBUG] [CORE] NodePoller (0xc0007faa20) notifying listener: {l:0xc0005acc60 s:0xc0005accc0}
2021/07/05 12:06:05 [DEBUG] [CORE] NodePoller (0xc0007faa20) listener callback - num items: 4 err: <nil>

*****************************************************************************************************************************

Steps To Reproduce

https://clouddocs.f5.com/containers/latest/userguide/ipam/

---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: ipam-ctlr-clusterrole-binding
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ipam-ctlr-clusterrole
subjects:
  - apiGroup: ""
    kind: ServiceAccount
    name: ipam-ctlr
    namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: ipam-ctlr-clusterrole
rules:
  - apiGroups: ["fic.f5.com"]
    resources: ["f5ipams","f5ipams/status"]
    verbs: ["get", "list", "watch", "update", "patch"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ipam-ctlr
  namespace: kube-system

---

CRDs
https://raw.githubusercontent.com/F5Networks/f5-ipam-controller/main/docs/_static/schemas/ipam_schema.yaml

Expected Result
Actual Result
Diagnostic Information
<Configuration files, error messages, logs>
Note: Sanitize the data. For example, be mindful of IPs, ports, application names and URLs
Note: The following F5 article outlines the information required when opening an issue.
https://support.f5.com/csp/article/K60974137

Observations (if any)

@skenderidis - Please share ipam deployment YAML used in this setup.

ipam-controller.txt
I have attached it.
You can also find it below
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
name: f5-ipam-controller
name: f5-ipam-controller
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: f5-ipam-controller
template:
metadata:
labels:
app: f5-ipam-controller
spec:
containers:
- args:
- --orchestration=kubernetes
- --ip-range='{"Test":"<IP_ADDRESS>-<IP_ADDRESS>","Production":"<IP_ADDRESS>-<IP_ADDRESS>"}'
- --log-level=DEBUG
command:
- /app/bin/f5-ipam-controller
image:  f5networks/f5-ipam-controller:latest
imagePullPolicy: IfNotPresent
name: f5-ipam-controller
serviceAccount: ipam-ctlr
serviceAccountName: ipam-ctlr

@skenderidis - From CIS side, you need to update the clusterRole with rules to support f5.ipam. Please add L17,L18 and L19 from https://github.com/F5Networks/k8s-bigip-ctlr/blob/master/docs/config_examples/crd/Install/clusterrole.yml#L17 to CIS ClusterRole and restart CIS deployment.
This should fix the issue.

Resolved the issue. closing the same.


Register should work from a ServiceWorker with an uncontrolled client

Step 3 of registration:

If the global object specified by the incumbent settings object is a WorkerGlobalScope instance, and the serviceWorkerRegistration’s active worker is not currently controlling any clients, reject promise with an InvalidAccessError and abort these steps.

This should probably only reject if it has no clients. Uncontrolled clients should be fine, right?

Yes, uncontrolled clients should be okay -- as long as they're window clients, at least. That covers the case where a service worker is registered, and then immediately used to register syncs. The API shouldn't require waiting for the second page load.

The conditions here should be the same as https://github.com/slightlyoff/BackgroundSync/issues/95 - I think any window client on the origin is fine.

Any window client on the origin works for me


Add language switcher on Settings page

What is the motivation for adding/enhancing this feature?

What are the acceptance criteria?

[ ] ...

Can you complete this feature request by yourself?

[ ] YES
[ ] NO

Additional information

Reference: https://github.com/hotwax/bopis/issues/141
UI: https://github.com/hotwax/bopis/issues/169

Closing as completed #175


Jupyter notebook kernels showing Python interpreters that aren't actually persisted as kernels.

 Jupyter does not show it.
Of course for a regular Interactive Window session, this selection COULD work since it's just a 3.8.8 conda environment. That conda environment activates and ipython can run successfully within it. To me this seems like all that should be necessary for the Interactive Window.

This is by design, since even a jupyter user can run jupyter from within a default non-kernel python environment and use it like a kernel. Its referred to as simply "Python3".


Stable Branch/Version?

Hey, i am unable to build with latest changes/refactoring. "Melih" branch is working great but your latest changes really enhanced your webrtc implementation
One major issue is that Result.csproj cannot be loaded.

HI there,
Sorry for the break. I have added the Result project to Externals now and this should sort out your build problem.
I am having a major refactoring especially on Middleware and Servers, so code breakings are possible.

When the MediaSoup support is completed, I will tag the code and make a new NuGet releases.

Thank you very much! cant wait for the final release


Enable custom test-suite commands

At the moment the test-suite command is fixed to be nox -r, this should be customizable. Probably like this:
[tool.recite]
run_tests = ["tox", "run", "-e", "mytests"]

in the pyproject.toml.

The default test-suite command should also run slow tests to make sure all tests are run before a release.


Expected doesn't work with inheritance and unique_ptr

A fairly serious bug:
#include <memory>
#include <system_error>
#include <iostream>
using namespace tl;

struct parent {};
struct child : public parent {};

auto doit() -> expected<std::unique_ptr<child>, std::error_code>
{
    return make_unexpected(std::error_code{});
}

int main ()
{
    // Works
    expected<std::unique_ptr<child>, std::error_code> res1 = doit();
    if (res1) std::cout << "All OK\n"; else std::cout << "An error!" << std::endl;

    // Upcast removes the error!
    expected<std::unique_ptr<parent>, std::error_code> res2 = doit();
    if (res2) std::cout << "All OK\n"; else std::cout << "An error!" << std::endl;
    return 0;
}

Output:
An error!
All OK

Expected output:
An error!
An error!

Thanks, there was an issue in the converting constructors.

Thanks. I'm finding a lot of bugs though, in a small test app :(

Yeah, there's definitely some coverage analysis to be done, it's just a case of my finding the time to do it!

@TartanLlama Completely understand. Just wondering if I should dare taking it into production or not ;) But seems like everything I need is working now, so thanks for all the help!


Dummy PR

Signed-off-by: ArchanaKakani<EMAIL_ADDRESS>

This PR was created only to get the static analyzer logs created during the CI build.


Outdated setuptools causing package installation issues

Installing python-memcached==1.58 inside app engine started breaking for me today.
Running gcloud app deploy gives decent output up until the installation of python-memcached
The following output is what I get at that point

This fails on the docker build step RUN pip install -r requirements.txt
Upgrading setuptools to setuptools-36.5.0 fixed this problem. If I replicate the Docker build steps that run for app engine but add a line RUN pip install -U pip setuptools before running the pip install above, this is resolved completely.
This is the related issue in the python-memcached repo - https://github.com/linsomniac/python-memcached/issues/103
How would we tackle this? Is it an option to upgrade the setuptools version? Do we pin it to a version? Or do you think it's acceptable keeping a broad version unpinned upgrade?

Can I submit a PR for this? Or do you not agree with the approach?

Can you verify that this problem is fixed for you now?

Appears to be fixed, closing:
$ docker run -it --entrypoint=/bin/bash gcr.io/google-appengine/python
root@56d393372fca:/home/vmagent/app# virtualenv -p python3.6 ~/env
Running virtualenv with interpreter /opt/python3.6/bin/python3.6
Using base prefix '/opt/python3.6'
New python executable in /root/env/bin/python3.6
Also creating executable in /root/env/bin/python
Installing setuptools, pip, wheel...done.
root@56d393372fca:/home/vmagent/app# . ~/env/bin/activate
(env) root@56d393372fca:/home/vmagent/app# pip install python-memcached
Collecting python-memcached
  Downloading python-memcached-1.58.tar.gz
Collecting six>=1.4.0 (from python-memcached)
  Downloading six-1.11.0-py2.py3-none-any.whl
Building wheels for collected packages: python-memcached
  Running setup.py bdist_wheel for python-memcached ... done
  Stored in directory: /root/.cache/pip/wheels/16/75/11/86f65dfdcfcfbdaee336d5a3c125b5ac1c53973c4515c77ef5
Successfully built python-memcached
Installing collected packages: six, python-memcached
Successfully installed python-memcached-1.58 six-1.11.0
(env) root@56d393372fca:/home/vmagent/app#

Hi,
This is still causing an issue. Setuptools is now stuck at version 36, and for "dj-stripe@master" I need version 38. How can I override the version built into the image? Is there a way to run two "pip install", 1st one to upgrade setuptools, second one to install my requirements.txt?
Many thanks,
Ed

I will refresh setuptools with the next image update.  Meanwhile, you can try adding setuptool==38.5.0 to your requirements.txt.  Sometimes this works, depending on where in the install process the new setuptools is needed (i.e. it doesn't work if the new setuptools changes how requirements specs are parsed).

New image released.


snapshots/fake: initingCheckouter waits for Init

Current coverage is 71.19% (diff: 100%)

Merging #88 into master will decrease coverage by 0.28%

@@             master        #88   diff @@
==========================================
  Files            42         42
  Lines          2097       2111    +14
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           1499       1503     +4
- Misses          522        528     +6
- Partials         76         80     +4

Powered by Codecov. Last update b85d7b7...ab06c1e

lgtm


Remove limitation that requires @ mention for a bot to receive a channel message

I am aware of https://github.com/microsoft/BotFramework-Hubot/issues/28#issuecomment-438772116 but it is really a deal breaker for using Teams. I've been asked to move our department from Slack to Teams, but I cannot transition to a service that doesn't "work". I can't imagine why a bot would be limited in this way. It's almost like an "output only" thing that you've made ...but with the ability to sometimes interact with it. It's really confusing as to how this is a good idea for anyone.

It doesn't look like this was at all possible using the bot framework, however I was just notified that there's new functionality "in public preview" which allows bots to be given permission to receive all messages within a team. Maybe this is a step towards removing the mention requirement?
"Remove Bot Mention Requirement" - Uservoice Suggestion
Receive all channel messages with RSC


Documentation is down

Your documentation is down. http://react.semantic-ui.com/
Errors from the console :
vendor.efab3c6822920f9f6af1.js:1 Uncaught SyntaxError: Unexpected token <
app.d0cf20c18c32b5eeee50.js:1 Uncaught SyntaxError: Unexpected token <

Working fine for me

Ok, i've looked it up, and Zain (an ISP in Bahrain) is blocking your website because it thinks cdn.rawgit.com it is malicious. Thanks anyway...

Is there a way for me to download the documentation so i don't have to go through a vpn everytime ?

How about trying to build your own docs locally?
git clone https://github.com/Semantic-Org/Semantic-UI-React.git
npm run docs
// https://github.com/Semantic-Org/Semantic-UI-React/blob/master/package.json#L14

Perfect, thanks a lot.

Web site is down again. by down I mean that nothing shows at all.

Works fine to me. Can you check your's browser console?

adsbygoogle.js Failed to load resource: net::ERR_BLOCKED_BY_CLIENT
vendor.6f18c3b4638e2693a9f5.js Failed to load resource: net::ERR_BLOCKED_BY_CLIENT
app.c26c115a832541f037ef.js Failed to load resource: net::ERR_BLOCKED_BY_CLIENT

There is something with your Adblock or browser settings, make sure that https://cdn.rawgit.com is whitelisted there.

Heads up, this is just the Google ad script for the Advertisement component.  We do not use advertisements nor trackers on the docs, so you're safe to disabled ABP for react.semantic-ui.com.
If you leave it enabled, there is not much impact either.  The Advertisement component docs just won't load ads.

Yep, addblock was the issue. Thanks.
But for a normal user point of view, this should not happen, me as a user I should not experience this.
I consider this to be a problem.
Thanks again :)


Include timestamp TLV

@jwhui , this is a work-in-progress PR.
After merging this PR, it helps us to pass 9.3.2, but I don't think this is the final one.
I have some questions for the managing timestamp.
Now we have a mTimestamp in dataset that is used to store active/pending timestamp.
But there are 2 problems with current implementation.

Not timestamp TLV in response, if configuring via CLI. I would suppose that dataset of Leader would be set by the user via CLI or any other configuration method.
There are 2 copies of timestamp (mTlvs and mTimestamp), and they may be different, if receiving MGMT_ACTIVE_SET/MGMT_PENDING_SET from other nodes.

I would prefer that we keep one copy of timestamp in mTlvs, and remove mTimestamp, and make other corresponding changes.
But the problem here is that we need to explicitly to tell whether  it is active or pending while doing DatasetManager.mLocal/mNetwork.SetTimestamp.
I am not quite sure about your thoughts on mTimestamp while in your initial implementation.
Could you let me know your comments for this?

I think it's conceptually cleaner to keep everything in mTlvs and not special case the timestamp as is currently done.  Part of the reason why I did special case mTimestamp is because all of the comparisons are done using the timestamp.  I see three options every time you need to reference the timestamp:

Search for the timestamp every time.
Maintain a pointer to the timestamp.
Keep the timestamp in a well-known location (this is what's implemented today).

I'm fine with moving to either of the other alternatives if you find the logic to be simpler.

Current coverage is 67.20% (diff: 0.00%)

Merging #564 into master will increase coverage by 2.55%

@@             master       #564   diff @@
==========================================
  Files            94         84     -10
  Lines         13904      11561   -2343
  Methods        1900       1591    -309
  Messages          0          0
  Branches       1631       1381    -250
==========================================
- Hits           8990       7770   -1220
+ Misses         4405       3389   -1016
+ Partials        509        402    -107

Powered by Codecov. Last update ef2a1a0...7fce521

This looks like a good improvement overall.  See my comments above.


[WIP] new(drivers): collect reaper_pid from the kernel

What type of PR is this?
/kind feature
Any specific area of the project related to this PR?
/area driver-kmod
/area driver-bpf
/area driver-modern-bpf
Does this PR require a change in the driver versions?
/version driver-SCHEMA-version-minor
Not clear if we will increment it in this PR or not, let's see
What this PR does / why we need it:
This PR is the first step towards  https://github.com/falcosecurity/libs/issues/1139.
The reaper info is necessary to implement a correct reparenting logic in userspace.
If we were able to collect the reaper info during the /proc scan, instrumenting prctl could be enough since we should be able to store all the reapers of the system, but unfortunately during /proc scan we are not able to recover the is_child_subreaper info (there are no ways as far as I know...).
For this reason, we need to find an alternative way to extract this info from the kernel, the chosen one is to do that when the process terminates and we need to reparent its children, so during PROC_EXIT event. One possible alternative could be to directly instrument this kernel function https://elixir.bootlin.com/linux/latest/source/kernel/exit.c#L617 but considering the state of the art of our drivers probably the proposed solution seems the easier to put in place.
What the proposed solution does is simply emulate the behavior of find_new_reaper with some small difference. The behavior is quite simple:

if the current thread has no children we don't call the new logic (saving clock cycles) and we send a reaper_pid == 0
if we find a reaper we send its pid (kernel lingo). Please note that we should be always able to find a valid reaper apart from bpf cases (see next point)
if we are in BPF (both probes) and we are not able to loop over all threads, we cannot identify the correct reaper so we send a special value to userspace (-1) and the userspace will try to compute the correct reaper with its own custom logic and without complexity limits imposed by the verifier

Which issue(s) this PR fixes:
Part of  https://github.com/falcosecurity/libs/issues/1139
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
NONE

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

we have issues with 2.6 builds i will take a look :/

Summary:

x < 3.4.0 There is no concept of sub_reaper, prctl is not defined, so it's enough to send -1 to userspace as we do for bpf verifier limits. If there are no sub_reapers involved the userspace should be able to determine the right reaper for orphaned children so no need for kernel logic
3.4 <= x < 3.19 the find_new_reaper logic was slightly different https://elixir.bootlin.com/linux/v3.4/source/kernel/exit.c#L696 but for us, it should be enough to use the while_each_thread macro instead of for_each_thread one. Please note that the kernel introduced for_each_thread macro in version 3.14.0 but it is used in reaper logic only starting from version 3.19.0, so we do the same.
x>=3.19 we use the kernel logic with the for_each_thread macro

Great research @Andreagit97 !

Or actually saw you just pulled it out of WIP, maybe the better strategy is to
/hold
to signal folks that we would want to examine this more carefully. Is this ok with you? @falcosecurity/libs-maintainers plz take a look. If possible let's try to come to a conclusion early next week, it would be good to not block this work for too long.

@Andreagit97 I remember the child_reaper ... just glanced over it, but let me find some more time later this week to think and check. In any case would you be ok marking it as WIP and allow a few folks to examine it as it's quite a crucial, but also intrusive patch?
Need to probably poke some more possible edge cases plus ensure no regression etc and obviously check on perf implications.

/hold
Sure! no rush at all, take all the time you need to review it!
Yes, the child_reaper is the reaper for that namespace, but before reparenting children to the child_reaper we need to check for alive threads in the same thread group and for possible child_subreapers. Directly from the kernel:
/*
 * When we die, we re-parent all our children, and try to:
 * 1. give them to another thread in our thread group, if such a member exists
 * 2. give it to the first ancestor process which prctl'd itself as a
 *    child_subreaper for its children (like a service manager)
 * 3. give it to the init process (PID 1) in our pid namespace
 */

Just to let you all understand why we need it if we want a complete solution:
Let's imagine a very concrete case, if i run Falco on my machine without this patch and i check for the process hierarchy of a "firefox" thread (something completely random) I will see something like this:
⬇ {firefox:gdrv0} tid: 4708, pid: 4551, rptid: 2165, rppid: 2165
⬇ [gnome-shell] tid: 2165, pid: 2165, rptid: 1448, rppid: 1448
⬇ [systemd] tid: 1, pid: 1, rptid: 0, rppid: 0

while if I use the patch I will see
⬇ {firefox:gdrv0} tid: 4708, pid: 4551, rptid: 2165, rppid: 2165
⬇ [gnome-shell] tid: 2165, pid: 2165, rptid: 1448, rppid: 1448
⬇ [systemd]💀 tid: 1448, pid: 1448, rptid: 1, rppid: 1
⬇ [systemd] tid: 1, pid: 1, rptid: 0, rppid: 0

The reason is that ([systemd]💀 tid: 1448, pid: 1448, rptid: 1, rppid: 1) was marked as a reaper before Falco was up and running, and for this reason, we don't know if it is a reaper... so when we will try to build the process hierarchy we will skip it
Same for a k8s use case:
imagine an nginx pod up and running, then you start Falco. If you type something like:
kubectl exec nginx -- sh -c  'tail -f /proc/self/status'

with the patch, you should see something like:
⬇ [tail] tid: 44588, pid: 44588, rptid: 44582, rppid: 44582
⬇ [sh] tid: 44582, pid: 44582, rptid: 44199, rppid: 44199
⬇ [containerd-shim]💀 tid: 44199, pid: 44199, rptid: 1, rppid: 1
⬇ [systemd] tid: 1, pid: 1, rptid: 0, rppid: 0

while without it you should see
⬇ [tail] tid: 44588, pid: 44588, rptid: 44582, rppid: 44582
⬇ [sh] tid: 44582, pid: 44582, rptid: 44199, rppid: 44199
⬇ [systemd] tid: 1, pid: 1, rptid: 0, rppid: 0

The reason is exactly the same as before, containerd-shim was already present before we started the capture :/
TL:DR; we need to figure out what we want, if we want a correct hierarchy we need something similar to this patch and we need to pay the price in terms of perf, while if we accept something sub-optimal we can ignore this patch and we can just use the userspace logic

Thanks Andrea, I'll sit down and experiment as well!
re perf so I have been gathering new stats in prod now for over one week and scheduler process exit tracepoint really triggers so rarely comparatively (not surprising obviously) but instead of spending like 200-400 ns on avg  you do in the enter and exit syscall tracepoints you spend over 2K ns per invocation on average in the scheduler process exit tracepoint (I guess absolute numbers vary a lotvbased on env and servers but you get the ratios here).
I think having a highly correct process tree is super important and we should implement it and I doubt it would explode perf, but I am happy to test deploy something and check concrete perf data, for that though we need to take the risk and merge to master after reviewing it. Nowish is a good time to be more experimental as we could still course correct if it ends up being really terrible in terms of perf.

re perf so I have been gathering new stats in prod now for over one week and scheduler process exit tracepoint really triggers so rarely comparatively (not surprising obviously) but instead of spending like 200-400 ns on avg you do in the enter and exit syscall tracepoints you spend over 2K ns per invocation on average in the scheduler process exit tracepoint (I guess absolute numbers vary a lotvbased on env and servers but you get the ratios here).

Oh, that's super useful!  did you use bpftool stats to get these metrics? BTW with the current implementation, it's quite strange to see something like 2K the actual filler (without the patch) does almost nothing...

I think having a highly correct process tree is super important and we should implement it and I doubt it would explode perf, but I am happy to test deploy something and check concrete perf data, for that though we need to take the risk and merge to master after reviewing it. Nowish is a good time to be more experimental as we could still course correct if it ends up being really terrible in terms of perf.

let's say the good news is that if we face some perf issue we could patch it very easy, in the end the userspace will work like "if the kernel sends me a valid reaper i will trust it otherwise I will use my userspace logic to compute the reaper"

Started some experiments, let's chat in the issue https://github.com/falcosecurity/libs/issues/1139#issuecomment-1605292172 there are still problems with the process tree :thinking: ...

I removed the SCHEMA_VERSION Bump since it was already bumped

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, jasondellaluce
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

New changes are detected. LGTM label has been removed.

just rebased on master :)

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, incertum, jasondellaluce
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,incertum,jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 356b48f0083f9c4fc4b90701b8973b4c9cd2a572

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, incertum, jasondellaluce
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,incertum,jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/unhold

/milestone 5.1.0+driver


Getting TypeError when running in Jupyter Notebook

Had this error message when running example code
using Anaconda Jupyter notebook
python 3.6
tensorflow ==1.8.0
tensor-hub ==0.1.1
`import tensorflow as tf
import tensorflow_hub as hub
module_url = "https://tfhub.dev/google/universal-sentence-encoder/2"
embed = hub.Module(module_url)`

this snippet works in console but not in  notebook , any idea ?

Can't repro. Are you sure you're really running tensorflow>=1.8.0? The _FlagValues class existed in TensorFlow versions up to and including 1.4, but was removed in https://github.com/tensorflow/tensorflow/commit/2652704b576adc16b4d735f651cea1024e88b72e#diff-92ab93112cecf0bbfb502800268d86f7
FWIW, tensorflow_hub=0.1.1 no longer contains a run-time check for the version for tensorflow, like tensorflow 0.1.0 did, because the regex in the original check choked on tensorflow 1.10.0. But tensorflow>=1.8.0 is still required.

@arnoegw  Thanks for the reply. it's my bad. my tensorflow was 1.4.0 when I ran this snippet for the first time and got this error. Then I updated tensorflow and reloaded the package but it wasn't actually effective, it was still 1.4.0.
It works now on tensorflow 1.8.0.
Thanks. closing issue.


Improving our conformance support story

Background
We've been piggy backing off of golang's testing package along with some bash scripts as our conformance test driver. We've also built up some tooling in knative.dev/pkg/test to support this effort. We're at a point now where we have enough insight and experience with conformance testing that we can do a pass at improving the experience for both upstream authors and downstream implementors.
Requirements
Here's my first pass of the requirements we need:

Build upon golang testing given the supporting tools and services that already exist
Conformance Authors need the ability to mark certain features with their maturity/state (alpha, beta, stable)
Conformance Authors need the ability to mark functionality with different requirement levels
Conformance Authors need the ability to compose, consume & reuse common configuration (ie. environment settings)
Conformance Authors need the ability to provide defaults for various test options.
Downstream implementors need the ability to consume and invoke upstream conformance tests.
Downstream implementors need the ability to invoke a subset of tests based on requirements levels.
Downstream implementors need the ability to invoke a subset of tests based on the maturity of a feature.
Downstream implementors need the ability to consistently supply test options and override defaults.

Next Steps

Solicit feedback of other conformance authors & runners before I proceed.
Turn the pkg POC into a new pkg/test/v2 so that we can gradually onboard different conformance tests
🎉

Solutioning on the various requirements
I've been experimenting a POC but I'll provide more context wrt. to each requirement listed above.
knative.dev/pkg poc diff
knative.dev/serving poc diff
The order I talk about the requirements below differ from above since they build on each other. Above I just partitioned based on persona so it reads better.
1. Build upon golang testing given the supporting tools and services that already exist
We should use go test package as our driver for invoking conformance tests. This lets us continue using our current infrastructure of capturing test results and viewing them in test grid.
What we can do is build upon the testing package to offer higher level semantics for conformance authors. See below for more details.
6. Downstream implementors need the ability to consume and invoke upstream conformance tests.
When the project switched from dep to go modules we lost our ability to invoke vendored tests. Thus to support implementors we started to expose the ingress conformance as a library that can be invoked downstream in their own tests.
the tl;dr right now conformance authors expose a function for downstream implementors. Golang subtests (t.Run) offer a nice mechanism to compose the larger test suite of small focused feature tests.
package conformance

// in a standard go file
func Run(t *testing.T) {
    t.Run("some feature", TestSomeFeature)
    t.Run("some other feature", TestSomeOtherFeature)
}

func TestSomeFeature(t *testing.T) {...}
func TestSomeOtherFeature(t *testing.T) {...}

Downstream implementors would author test file as follows
// in a _test.go file
import "knative.dev/upstream/conformance"

func TestConformance(t *testing.T) {
  conformance.Run(t)
}

2. Conformance Authors need the ability to mark certain features with their maturity/state (alpha, beta, stable)
It should be easy to mark tests based on a feature's maturity/state.  If we build upon golang's testing package by offering our own 'test context' we can offer some first class helpers similar to subtests (t.Run) that then feel idiomatic to the language.
// our test framework - knative.dev/pkg/test/v2
package test

type T struct {
  *testing.T
}

func (t T) Alpha(name string, func (t *T)) {...}
func (t T) Beta(name string, func (t *T)) {...}
func (t T) Stable(name string, func (t *T)) {...}

Conformance authors would then utilize these methods as follows
// in a standard go file
import "knative.dev/pkg/test/v2"

func Run(t *test.T) {
    t.Alpha("some alpha feature", TestAlphaFeature)
    t.Beta("some beta feature", TestBetaFeature)
    t.Stable("some stable feature", TestStableFeature)
}

func TestAlphaFeature(t *test.T) {}
func TestBetaFeature(t *test.T) {}
func TestStableFeature(t *test.T) {}

3. Conformance Authors need the ability to mark functionality with different requirement levels
Similar to above we extend test.T to allow conformance authors to mark different assertions with specific requirement levels.
// our test framework - knative.dev/pkg/test/v2
package test

type T struct {
  *testing.T
}

// One option is to use a similar subtest approach
func (t T) Must(name string, func (t *T)) {...}
func (t T) Should(name string, func (t *T)) {...}
func (t T) May(name string, func (t *T)) {...}

// Another option is to support exposing when certain requirements are to be tested
func (t T) Must() bool {...}
func (t T) Should() bool {...}
func (t T) May() bool {...}

Conformance authors would then utilize these methods as follows
func TestSomeStableFeature(t test.T) {
   // option 1
   t.Must("have some runtime thing", func() {
       // make your assertions
   })

  // option 2
  t.Must() {
    // make your assertions
  }
}

4. Conformance Authors need the ability to compose, consume & reuse common configuration.
Various properties are shared amongst different conformance tests. Tests shouldn't have to redefine these properties. We want to have consistency in the way these properties are supplied, read and composed. The current pattern is to read command line args into global flags. Sadly we're not consistent as some functions read environment variables directly.
The conformance lib can provide a base config that tracks what feature states, requirements should be exercise against a specific environment
type BaseConfig struct {
    Levels RequirementLevels
	States FeatureStates

	environment.Settings // similar to pkg/test env - k8s cluster, docker repo etc.
}

// We have enough info to build the client make it easier to do with this helper
func (BaseConfig) KubeClient() {..}
// ... add other helpers ...

// We can tie properties in this config to a flag set
func (BaseConfig) AddFlags(fs &flag.FlagSet) {...}

This config can be embedded into a specific conformance test's configuration
type Config struct {
   test.BaseConfig
   IngressClass string
}

// We can tie properties in this instance to a flag set
func (c Config) AddFlags(fs &flag.FlagSet) {
    c.BaseConfig.AddFlags(fs) // add all our base flags
    fs.StringVar(...) // add our additional properties
}

Our test context can carry an instance of this config that's accessible in tests. Note: go doesn't support inheritance/generics so we'll need our 'test context' to contain an interface of a config (long story)
// in our lib
package test

type T struct {
    *testing.T
    Config
}

// in a conformance test
func TestStableFeature(t test.T) {
    endpoint := t.Config.IngressEndpoint
    ... use endpoint ...
}

5. Conformance Authors need the ability to provide defaults for various test options.
This is pretty self explanatory. For a concrete example serving runs conformance in a default namespace (serving-tests) but allows overrides.
9. Downstream implementors need the ability to supply test options and override defaults in a consistent way.
We want to support different options for supplying test options & overriding defaults.
Options defined in code
Conformance authors should consume their a config when their test is invoked
func Run(tt *testing.T, config) {
    t := test.NewContext(tt, config)

    t.Alpha("some alpha feature", TestAlphaFeature)
    t.Beta("some beta feature", TestBetaFeature)
    t.Stable("some stable feature", TestStableFeature)
}

This allows implementors to write their configs in go
func TestIngressConformance(t *testing.T) {
   config := DefaultConfig()
   config.IngressClass = "my-ingress-class"

   Run(t, config)
}

Options & overrides via flags
We'd like to support our existing scripts and allow invokers of our conformance to override some settings. This can be accomplished via flags on our test binary. Since we want to avoid globals in our library packages we make the global in the test runner and attach the flags to the default command line.
var config Config = DefaultConfig()

// To be verified this might need to be in a TestMain
func init() {
	// allow overrides via flags
	config.AddFlags(flag.CommandLine)
}

func TestIngressConformance(t *testing.T) {
   Run(t, config)
}

7. Downstream implementors need the ability to invoke a subset of tests based on requirements levels.
This helps them know which requirements they're meeting and which they are not. This can be accomplished by running tests with different requirements enabled.
For example here's how we could override the requirement levels via. test flags

Ensure we're passing all absolute requirements
go test -requirement.must -requirement.mustnot
Ensure we're passing recommended requirements
go test -requirements.recommended
Ensure we're passing optional requirements
go test -requirements.may
Ensure we're passing all requirements
go test -requirements.all

8. Downstream implementors need the ability to invoke a subset of tests based on the maturity of a feature.
We want to avoid breaking downstream CIs when a new conformance test is added for an alpha feature. This provides a gradual onboarding for implementors as a feature progresses from alpha to stable.
For example here's how we could override the enabled feature states via. test flags

Ensure we're passing all stable and beta features
go test -features.stable -features.beta
Ensure we're passing alpha features
go test -features.alpha
Ensure we're passing all tests
go test -features.all

/assign @dprotaso

Conformance Authors need the ability to mark certain features with their maturity/state (alpha, beta, stable)
Conformance Authors need the ability to mark functionality with different requirement levels

I was wondering if "tags" would be a solution to both of these. Looking at the current signature-based solution to 2 & 3, it seems like expressing Beta + Must would require nesting the two. What would you think of using build tags instead, like so:
// +build should,beta must,stable
// +build v1

This would indicate that these tests should run against v1 "should" tests for beta and would become a "must" for stable.
Drawback: tests for different levels of maturity end up in different files; we may end up with a large number of files.

6. Downstream implementors need the ability to consume and invoke upstream conformance tests.
When the project switched from dep to go modules we lost our ability to invoke vendored tests. Thus to support implementors we started to expose the ingress conformance as a library that can be invoked downstream in their own tests.
the tl;dr right now conformance authors expose a function for downstream implementors. Golang subtests (t.Run) offer a nice mechanism to compose the larger test suite of small focused feature tests.
package conformance

// in a standard go file
func Run(t *testing.T) {
    t.Run("some feature", TestSomeFeature)
    t.Run("some other feature", TestSomeOtherFeature)
}

func TestSomeFeature(t *testing.T) {...}
func TestSomeOtherFeature(t *testing.T) {...}

Downstream implementors would author test file as follows
// in a _test.go file
import "knative.dev/upstream/conformance"

func TestConformance(t *testing.T) {
  conformance.Run(t)
}

I wonder if RunConformance could be built using go generate to store a slice of all the functions and then iterate over the slice in RunConformance.

Conformance Authors need the ability to mark certain features with their maturity/state (alpha, beta, stable)
Conformance Authors need the ability to mark functionality with different requirement levels

I was wondering if "tags" would be a solution to both of these. Looking at the current signature-based solution to 2 & 3, it seems like expressing Beta + Must would require nesting the two. What would you think of using build tags instead, like so:
// +build should,beta must,stable
// +build v1

This would indicate that these tests should run against v1 "should" tests for beta and would become a "must" for stable.
Drawback: tests for different levels of maturity end up in different files; we may end up with a large number of files.

Additional drawback: build flags won't show up if we do a reflection-based RunConformance.

I thought about build tags but decided it's not a good fit for the following reasons

Because we need to package our tests in a library (thanks go.mod) we need an entry point that coalesces all the tests together. Thus we need to be able to reference all the test funcs at compile time no matter the flags. With build flags we'll need noop tests when those tags aren't enabled. You could write a tool that mimics what go test does by looking at method prefixes but that is an anti-goal.
We may want to ship a conformance binary and enabling/disabling certain assertions dynamically via flags seems nice
tags require manipulation via go test - I'd like the option to have this to be done via code.

I wonder if RunConformance could be built using go generate to store a slice of all the functions and then iterate over the slice in RunConformance.

Maintaining the list manually isn't ideal but not troublesome at this point. Annotations or decorators would be nice here.
This could be a future improvement.

This would indicate that these tests should run against v1 "should" tests for beta and would become a "must" for stable.

This is interesting I would think I'd prefer requirements levels to be stable across feature states. This would signal the final intent to implementors earlier on

high level comment on first reading: I like it!
small low level comment: I wonder if we can avoid changing the test signature and wrapping testing.T, 'cos that'll break tooling a bit and make the tests a little non-idiomatic-seeming, and instead do something similar to mux.Var, i.e. just take T as an argument to a package-level function. So conformance.Must(t, func() { .. }) instead of t.Must(func() {..}). Not the most important thing in the world though 🤷.

This would be really helpful for me. The current solution to turn on a feature inside of the e2e-tests.sh script means there's an odd dependency for the test to run properly (and we put the test in its own package/path to make this easier to run). https://github.com/knative/serving/blob/0633c628d3e0a3b25511217351564960bc777b42/test/e2e-tests.sh#L115
Currently I'm looking at testing Garbage Collection. The ability to toggle features by maturity would be great but also having a go library to modify configMap values in the test would also make it easier to test several varieties of the scenario.

What about a helper at the beginning of each function?
func TestMust_Clause123(t *testing.T) {
  conformance.Verifies(t, "Clause123", conformance.Beta("serving"), conformance.Must())
}

func conformance.Verifies(t *testing.T, ref string, opts ...Options) {
  // If should not be enabled:
  t.SkipNow()
}

Ref: https://golang.org/pkg/testing/#T.SkipNow

first of all, thanks for doing this. I believe this is very needed and I'll share my thoughts about it

I think your head is in the right place, this is the direction Knative should head towards and the requirements generally seem to capture the essence of what we need to give our users and downstream implementors.
Conformance tests in eventing are tied to two aspects, a test context (i.e. given a GitHubSource CRD is installed), and a event/action(i.e when an object is created, or when an event is sent) which both need to be provided by the downstream implementors. In this proposal I am expecting that Config can be used for the context, but I'm struggling figuring out how can implementors provide the events/actions.
In Eventing WG we are about to establish a testing taskforce that is ideally across WGs to explore a couple of testing approaches and mainstream that effort hopefully also across Knative, these are some of the ones mentioned so far:

How k8s does conformance https://github.com/cncf/k8s-conformance
Sonobuoy https://github.com/vmware-tanzu/sonobuoy
Using cucumber and features
https://github.com/n3wscott/rigging
Checkout the bdd here https://github.com/n3wscott/cloudevents-observer
https://github.com/knative-sandbox/sample-controller/pull/287
https://github.com/knative-sandbox/sample-controller/pull/175

It would be great if maybe we compare this approach and others mentioned above in the taskforce and settle on one to work on and drive it through across the project.

@julz @evankanderson  re: package level functions & being non-idiomatic
So we're currently non-idiomatic because we're moving test funcs into non-test files. This is necessary for the tests to be consumed downstream. If we were to have package level functions then we'd need to access the test config (ie. what requirement levels & feature states are to be exercised) as a global and that's something I'd prefer to avoid.
The other bit by having our own defined T is we can augment it with new features/helpers (ie. avoid test.Setup everywhere)

You could use the config as an extensions point to provide specific functionality
ie.
// eventing conformance

type Config struct {
   test.BaseConfig
   ...
   SourceType metav1.TypeMeta
   SourceTriggerFunc func() {}
}

func Run(tt *testing.T, c Config) {
    t := test.NewContext(tt, c Config)
    // assert config is valid

    // eventing conformance assertions
    ...

    // Send event
    c.SourceTriggerFunc()
}

Downstream you could have

var sourceConfig conformance.Config

func init() {
  sourceConfig.AddFlags(flag.CommandLine)
}

func TestSomeSource(t *testing.T) {
   some := sourceConfig
   some.SourceType = // some metav1.Type
   some.SourceTriggerFunc = func() {
       // send the event
  }

  conformance.Run(t, some)
}

func TestSomeOtherSource(t *testing.T) {
   other := sourceConfig
   other.SourceType = // some other metav1.Type
   other.SourceTriggerFunc = func() {
       // send the event some way
  }

  conformance.Run(t, other)
}

Thus I prefer tests don't reach for globals

I mentioned this on the call, but I'd also love it if this enabled downstream to run higher iteration counts.
In net-contour I added this: https://github.com/knative-sandbox/net-contour/blob/677e434bcac7521d6884c0f11d428af40090ba14/test/conformance/ingress_test.go#L34-L39
However, this groups things on testgrid as: TestIngressConformance/4/hosts/multiple, where the grouping I really want is: TestIngressConformance/hosts/multiple/4 (see also testgrid)

@dprotaso please take a look on initial draft of Requirements Gathering for Knative Enhanced Testing Task Force and add if there is anything missing (I tried incorporate your Requirements from this issiue - please add if anything is missing or comment): https://docs.google.com/document/d/1zuGSMLXGJlsjyZIpuNZXhMbbCtE2yFe8dxbQ03FZrS0/edit?usp=sharing
also linked from https://github.com/knative/eventing/issues/3777#issuecomment-670644926

Following up I'm doing another pass of the conformance lib. I've merged the test.T type and Config in to one. This should make it easier for conformance test authors.
See: https://github.com/knative-sandbox/reconciler-test/pull/17
Serving POC is here: https://github.com/knative/serving/pull/9742

Originally posted by @dprotaso in https://github.com/knative-sandbox/reconciler-test/pull/17#issuecomment-705748603
Playing with this in https://github.com/knative/serving/pull/9742 leads to a few questions
1. Should test names be decorated with requirement levels & feature states
ie. right now it's
=== RUN   TestConformance/Service/multi_container

it could be
=== RUN TestConformance/Service/ALPHA_multi_container

It might make sense to do this only for non-stable & non-must/should requirements
=== RUN TestConformance/Service/ALPHA_multi_container/MAY_do_something

2. Test Structure
If tests aren't being consumed downstream you might not want to have a single golang test as your entry point. This means all your tests will have the same prefix ie. TestConformance/
The simplest thing I can think of is authors create their own explicit copy method. Thus the package variable captures flags and copies are initialized with test.Init
Ideally if we had generics test.Init could return a copy that we wouldn't need to cast

Work on using the framework in networking is on-going here: https://github.com/knative/networking/pull/233

/reopen
/lifecycle frozen

FYI - this has evolved into: https://github.com/knative-sandbox/reconciler-test
Please check out the README and offer feedback there


sccm client actions menu not available within deployment status windows

from the deployments list, you cannot open the "in progress", "error" or "unknown" list of clients and perform client actions against them.
These are normally referred to as "right click" functions by operators, because most people have recast tools installed and this is how to call them up.
Additionally, once you have a list of these open, for example, the "in progress" ones, you cannot go back to the deployment status page and pull up the "unknown" ones without closing the node.
This is also the case for multiple failure codes within "error". Once you have one error code open, you have to "close all nodes" to get the next ones.
Being able to open ALL THE CLIENTS that have an error, regardless of type, and run the right-click function against them would be very helpful. Like, "restart sccm client" or "reboot" or "fix remote comms issues" or "update compliance status" or "run update evaluation". Most of us use tools like Client Center for Config Manager for this, but it will not action 50 clients at once. Would be cool if it did or if sccm did it directly.

Thanks @chadparris for taking the time to provide feedback on Configuration Manager! This is an interesting feature idea, and I want to make sure it gets directly into the hands of the engineering team. Fortunately, we've got dedicated channels for that very purpose. :) See the Product feedback section in our docs. I think it’s important that you submit it directly, because that allows you to receive notifications and more closely monitor the progress. I'm closing this issue as there's nothing actionable for docs at this time.


Fixed Function Pipeline

Is it possible to use the fixed-function pipeline with SDL_gpu? Any simple examples of this?

P.S. I am developing on a Mac :)

Platform Issue (MacOSX): When MacOSX 10.7 introduced support for OpenGL beyond 2.1, they also introduced the core/compatibility dichotomy. However, they did not introduce support for the compatibility profile itself. Instead, MacOSX gives you a choice: core profile for versions 3.2 or higher, or just version 2.1. There is no way to get access to features after 2.1 and still access the Fixed Function Pipeline.

https://www.opengl.org/wiki/Core_And_Compatibility_in_Contexts

There's no compatibility profile for GL 3.x in OS X, as your reference points out.  To use the fixed function pipeline, you must use GL 2.1.  SDL_gpu lets you easily request a GL 2 context that should do the right thing:
GPU_Target* screen = GPU_InitRenderer(GPU_RENDERER_OPENGL_2, 800, 600, GPU_DEFAULT_INIT_FLAGS);
You can also request a more specific OpenGL version like:
GPU_Target* screen = GPU_InitRendererByID(GPU_MakeRendererID("OpenGL 2.whatever", GPU_RENDERER_OPENGL_2, 2, 1), 800, 600, GPU_DEFAULT_INIT_FLAGS);
The GPU_RENDERER_OPENGL_2 tells SDL_gpu which code path to use.  The 2, 1 tells OpenGL which version to use when creating a new context.  If it fails, you get a NULL screen pointer.

I have tried the suggestions but couldn't get the fixed pipeline to draw anything in my project. Only glClear had any effect.
Closing the issue, but would be useful to see a small example to demonstrate legacy fixed function code.


Zip creation errors when path has spaces in it

Zip creation failed: Failed to run '/Library/Java/JavaVirtualMachines/default/Contents/Home/bin/jar cvf /Users/bfrenkel/Documents/InstantGames/Google Play Instant Sample/SphereBlastPCV1.zip -C /var/folders/js/zwj26yx53j1bw9ywks160ryr00772k/T/ base.apk'
stdout:
added manifest
adding: base.apk(in = 16531357) (out= 15605940)(deflated 5%)
stderr:
Play : no such file or directory
Instant : no such file or directory
Sample/SphereBlastPCV1.zip : no such file or directory
exit code: 1

Thanks for the bug report. I'll prepare a fix.

NP! FYI, I've confirmed that removing the spaces from directory resolved the issue.


Couldn't find nullification for [Recipe]

Hi there. I couldn't find the solution in the docs (maybe I am blind or something).
Result is looking like Recipe is always returned

But I want it to become so Recipe can be null too.
Code:
class RecipeResolver {
  // ...
  @Query(returns => [Recipe], { nullable: true })
  async recipes(@Args() { title, startIndex, endIndex }: GetRecipesArgs) {
    // sample implementation
    let recipes = this.recipesCollection;
    if (title) {
      recipes = recipes.filter(recipe => recipe.title === title);
    }
    return recipes.slice(startIndex, endIndex);
  }
}

Found it  { nullable: "itemsAndList" }
  // ...
  @Query(returns => [Recipe], { nullable: true })
  async recipes(@Args() { title, startIndex, endIndex }: GetRecipesArgs) {
    // sample implementation
    let recipes = this.recipesCollection;
    if (title) {
      recipes = recipes.filter(recipe => recipe.title === title);
    }
    return recipes.slice(startIndex, endIndex);
  }
}


Update to RobotPy 2<IP_ADDRESS>

This also pins the RobotPy core libraries to an exact version.

@auscompgeek I will approve this on tuesday after we reflash the rio and make sure its not all broken. If someone sees this running before I do, they can approve it

Oh, I figured you were the one that updated the robot. I noticed the imaging tool was open earlier yesterday.

I opened the tool and didnt actually get around to doing the update. I will do it on Tuesday.

Fair enough. But then who updated RobotPy on the robot then?


Change error handling to exceptions instead of trigger_error

It cannot be assumed that user wants to terminate script on error. Try/catch and exception throwing should give the user more flexibility as for what to do in case an error arises.

trigger_error is now only used for non-error notices/warnings...


Quizzes: add to eliademy

@Protohedgehog Jon, do you think adding the quiz set to eliademy would make sense, or too early? I do have a bit of free time on my hands at the moment, and would be happy to do this :)
Also, and apart from that, do you need a wingman 👾 for any of the open tasks re: content development? Or should I just have a go at any one of them... ?

@tosteiner I think migrating it over might not be a bad idea. We can always edit it later on in Eliademy.
If you wanted to have a bash at the OA literature search task with @johav that might be nice too. Or go through all of the currently drafted content and add notes on things that you would explicitly expect to see there, to make sure I don't miss things out!

Fab! am off to meet friends now, but will continue with this later 🌞

note to self: pick up on https://github.com/OpenScienceMOOC/Module-6-Open-Access-to-Research-Papers/issues/11#issuecomment-515670986
Current state of issue:

[x] Quiz transfer to eliademy: see here
[ ] Re-check quiz for inconsistencies and missing links (questions 6 and 8) - fix prior to release!

all set with #23 - close


[Phcode.dev Bot] Publishing Release v.0.0.1 to Extension Store

Thank you for contributing to phcode.dev extension store.
Please track extension/theme publish status by clicking this link.
You may close this issue at any time.

failed

Extension successfully published to store. You can close this issue at any time.


Account tiers don't display when deploying cloud algorithms

If we deploy a local algorithm with the CLI or a cloud algorithm from quantconnect.com that uses the FTX brokerage, the account tiers display.

However, if we deploy a cloud algorithm with the CLI, the account tiers don't display

We have the same issue with verification tiers when deploying with the Kraken brokerage.

Solved by https://github.com/QuantConnect/lean-cli/pull/91


Initialize wallet from existing seed

The wallet should be able to initialize from an existing seed instead of always generating a new seed during initialization.

This functionality has been implemented in #1433, but I'll hold off on closing it until there is proper API support.


NumberingPlanIndicator does not handle Reserved values

I got some strange OA sending me NPI=5. According to some source this is maritme, but in official GSM 3.40 spec it is not shown as that, nor in SMPP 3.4 specs.

As NumberingPlanIndicator is a mandatory value for submit_sm and deliver_sm in the JSMPP API, I was thinking that either NumberingPlanIndicator should not be an enum, or some funky hacking like following would take place:
`
public enum NumberingPlanIndicator {
UNKNOWN(SMPPConstant.NPI_UNKNOWN),
ISDN(SMPPConstant.NPI_ISDN),
DATA(SMPPConstant.NPI_DATA),
TELEX(SMPPConstant.NPI_TELEX),
LAND_MOBILE(SMPPConstant.NPI_LAND_MOBILE),
NATIONAL(SMPPConstant.NPI_NATIONAL),
PRIVATE(SMPPConstant.NPI_PRIVATE),
ERMES(SMPPConstant.NPI_ERMES),
INTERNET(SMPPConstant.NPI_INTERNET),
WAP(SMPPConstant.NPI_WAP),
RESERVED((byte)0xff);
private byte value;
private NumberingPlanIndicator(byte value) {
    this.value = value;
}
private NumberingPlanIndicator withReservedValue(byte value) {
    this.value = value;
    return this;
}

/**
 * Return the value of NPI.
 * @return the value of NPI.
 */
public byte value() {
    return value;
}

/**
 * Get the associated <tt>NumberingPlanIndicator</tt> by it's value.
 *
 * @param value is the value.
 * @return the associated enum const for specified value.
 * @throws IllegalArgumentException if there is no enum const associated
 *      with specified <tt>value</tt>.
 */
public static NumberingPlanIndicator valueOf(byte value)
        throws IllegalArgumentException {
    for (NumberingPlanIndicator val : values()) {
        if (val.value == value)
            return val;
    }

    return RESERVED.withReservedValue(value);
}

}
`
So I added RESERVED enum option here and withReservedValue instead of throwing an exception returning a RESERVED with the actual value. Note that this is very hacky as I needed a quick solution to pass the NPI=5 traffic.
I myself think that making NumberingPlanIndicator a class would be the better option.
I'd be happy to send a pull request if we could agree on a way forward. For now I am keeping this hack in place.

Strangely wireshark decodes NPI 5 as service centre specific:

In SS7 , SCCP layer, NPI=5 is indeed maritime mobile.
https://en.wikipedia.org/wiki/Telephone_numbering_plan#Numbering_plan_indicator

Yes saw that, but I couldn't find it anywhere in official specs. Anyhow the problem is that JSMPP is not able currently to pass other values as NPI.

That's true.
I guess I just wanted to point out, that problems could still appear, when dealing with reserved values and so many different SMSC implementations. After all, the provider SMSC is responsible to translate your SMPP submit_sm into an mT-ForwardSM  and deliver it across SS7. And the other way around , the mO-ForwardSM into an SMPP deliver_sm.
But that's not JSMPP's problem in any way... totally agree with you.

Making it a class would make it more flexable. Together with SMPP5 additions, I will look into this!

Ok, if you need help with pull requests, let me know.


Rails 7 support

👋 Thank you for this wonderful library that we have been gratefully using for years at Wrapbook! We are in the process of upgrading to Rails 7 and noticed you could use some help (possibly with a pr 😉) and so I wanted to start up the conversation! I'm sure there are changes here you don't we can take care of that together 🦾
Overall changes

Add Ruby 3 and 3.1 to Travis
Add Rails 7 to the test matrix
Switch rails.master to rails.main to reflect their new branch name
Default to rails 7

Some of the misc stuff is an artifact of using a current PR on the repo for inspiration to also add Propshaft support, but I'll clean that up here shortly.

Let's merge this?

@fphilipe bubbling this up

Thanks for you PR. This is now obsolete.


Review domain models

Description
Found a few minor issues in the external models to fix
Tasks

[x] Serialize ContentType as "contentType" instead of "content-type".
[x] Make an external version of NotiricationChannel.
[x] Non nullable Subject and Body in EmailTemplateExt

Regression test successful


set end date to current date

Sorry I'm not set up to test this right now but will this simple fix work for setting end_date? Does it break the request if the date is more recent than the latest data?

Hey @7yl4r, that's the right idea! Thanks for taking a look.
Actually I already setup get_ss_grds() to default to the full range of the given Seascape dataset (ie global monthly or global 8-day), but just started with 2 years since my previous attempt to fetching the full dataset through ERDDAP were not responsive. Or I just didn't wait long enough.
I'm also caching tifs in get_ss_grds() with the dir_tif parameter. Ideally, I'll update that function to cherry pick from tifs that still need ERDDAP fetching which aren't already there.

Fyi @7yl4r,
I implemented the "freshening" of data by only fetching from ERDDAP (and by individual date so server doesn't hang) in get_ss_grds(). The get_data.R script got simplified to using the default arguments of the full date extent available from the ERDDAP dataset.
Commits:

seascapeR

+ get_ss_dates_all(), get_ss_grds() to get missing rasters vs dir_tif · marinebon/seascapeR@8394c6c
∆ sum_ss_grds_to_ts() recalc if not all grds$date in ts_csv$date · marinebon/seascapeR@cf7a464

seascape_app

get all data, new layout w/ longer time series · marinebon/seascape_app@2461ced


Sui Doc Bug: a horizontal scrollbar on index page

I found a small frontend bug in you index page https://sui.io/ . It's a unnecessary horizontal scrollbar. I didn't find github repo for index page therefore I can't create pull request, so for fixing bug = removing horizontal scrollbar you need to add to css this code:
.main-wrapper {
  max-width: 100%;
}

.bg-top {
  max-width: 100%;
}

After it horizontal scrollbar will disappear. My system: win10 x64, browser: lastest chrome x64

Thank you, Sergey! I've added folks who can confirm and fix. Thanks again for the report!


Can't open 50k-bots file

Hello,
I can run the default bonesi, however I can't run it by using the 50k-bots file, I keep getting error

File 50k-bots could not be opened.

I'm using ubuntu 16.04 as a guest OS inside my VB 5.2.4, my host OS is Windows 10.
The command I'm using is the following:
sudo ./bonesi -i 50k-bots ip:port
Please help me resolve this issue. Thanks.

Change the name of the file to 50k-bots.txt


Locate Button doesn't locate drone

Hi,
I'm having problems with the locate button using the DJI simulator and the GSDemo android app, the drone is connected to my laptop using a USB cable, and my android device is connected to the Spark's wi-fi network.
Whenever I press the locate button the google maps view just takes me to Egypt or the Atlantic ocean. Has anyone ever experienced this issue?

Hi @vsibanyo , Spark doesn't support Waypoint Mission, so this demo doesn't work for Spark. So the locate button won't work for Spark.

Yes  on locate click it takes me to the sea without any airplane shown

Yes  on locate click it takes me to the sea without any airplane shown


Creates scheduled tasks that send out notifications to the mobile app users.

Jira cards

MOB-68

HOW DO I TEST OUT THIS PR?
Coming soon...

@Baalmart @123MwanjeMike @Mnoble-19 @BenjaminSsempala
Merging this PR. Shall monitor performance on staging.


Daylight savings time bug

[2023-11-05 01:34:00.000] [error] clock: 2023-11-05 01:34:00 is ambiguous.  It could be
2023-11-05 01:34:00 EDT == 2023-11-05 05:34:00 UTC or
2023-11-05 01:34:00 EST == 2023-11-05 06:34:00 UTC
Probably re: https://gcc.gnu.org/pipermail/libstdc++-cvs/2022q4/039034.html

Dups of https://github.com/Alexays/Waybar/issues/2615 :)


Inverse X Y to Lat Lon, data not as expected

Hello,
I am trying to convert a script I wrote in Perl years ago to Python, and while I believe I've followed the directions, the results do not match up with what I'd expect.
Code:
from pyproj import Proj
nc = Proj(init='epsg:3359')  # I've tried most of the available NC options (site below)
y = [2110926.0]
x = [0738252.0]
lon, lat = nc(x, y, inverse=True)
print lon, lat
[-77.19861281130545] [52.46179322191929]
This address should return as -78.7122, 35.7684 or thereabouts.
Am I missing some step?  I've also tried entering all of the parameters individually with the same results.
http://spatialreference.org/ref/?search=carolina
Thanks

Are the coordinates in feet or meters?  Pyproj will assume that you want meters so you might try this nc = Proj(init='epsg:3359', preserve_units=True).  BTW that didn't exactly produce your target coordinates.
Do you have a proj4 string, WKT, shapefile, or something else that defines the projection?

Based on the setup I had in the Perl script (below), I am assuming they are in ft.  These are data exported from a 911 CAD system, so I don't have access to a better definition than that.
my $proj = Geo::Proj4->new(
proj => "lcc",
units => "us-ft",
datum => "NAD83",
lat_0  => 33.75,
lon_0 => -79,
x_0 => 609601.22,
y_0 => 0,
lat_1 => 34.33,
lat_2 => 36.166667
)
Based on that config, the values match EPSG reference is 32019 (which I'm thinking is working in feet also based on the to_meters option.
http://spatialreference.org/ref/epsg/32019/proj4/
I will try the preserve units.

Hold on epsg 32019 uses the older NAD27 datum, and the above uses NAD83.  Changing those can cause things willy nilly can make your data be off by hundreds of feet.
>>> from pyproj import Proj
>>> nc = Proj("+proj=lcc +lat_1=34.33333333333334 +lat_2=36.16666666666666 +lat_0=33.75 +lon_0=-79 +x_0=609601.2192024384 +y_0=0 +datum=NAD83 +to_meter=0.3048006096012192 +no_defs ", preserve_units=True)
>>> nc(2110926, 738252, inverse=True)
(-78.62602489780987, 35.77787470941835)

That seems to work.  I used the one that you defined in EPSG:32019 and combined it with the above.  Or translate the one used in the code.  I would encourage you to use a standard EPSG number or state plane projection if at all possible.

Here's what I get with github master:
from pyproj import Proj
p = Proj(init='epsg:3359')
lon, lat = p(21109260., 7382520., inverse=True)
print lon, lat

68.1559778764 -40.6009157221

Thank you all for your assistance.  It appears that ESRI:102719 was what I was looking for.  It differs minimally in the +x_0 setting for that posted above, but the answers are certainly within the limits allowed by this map.


Vam 182 orwu valdt

added validateDate and xsdToFmDate service methods and tests

oh wait, should I split up the test specs too?

Nah, just the models. I/we like the models separated out one model-per-file for quick visual tracking. As long as the tests test the individual model formatting, the specs are fine in one file.


Where's "elcinema" guide ?!

Description
Where's "elcinema" guide ?!

elcinema.com has been disabled because the guide shows the wrong time. The guide will be back, If a solution is found.
#994
#1918

No need to disaple guide for wrong time , i can edit time in my iptv player add +5 hours and its ok , plz activate guide again 🙏🏻🌹


Removed Typo

Category

[x] Content fix
[ ] New article

What's in this Pull Request?
Changed FUsing to Using

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/spfx/web-parts/get-started/provision-sp-assets-from-package.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

thx Stefan, appreciated.


Release File::Next 1.16

ack 2.16 needs an update to File::Next.

File::Next 1.16 was released back in July.


✔ Alpha v0.1.9

[changelog]
;f New commands
;equalizer - Adjust equalizer levels!
;playtop - Plays the song straight away.
;queue move - Moves a song within the queue.
;queue remove - Removes a song from the queue.
;karaoke - Lists the lyrics for a song as it plays!
;page - Easily jump to a page in a paged message.
;remindme - Ask the bot to remind you of a message!
;welcome - Allows you to create welcome messages.
;goodbye - Allows you to create goodbye messages.
;autorole - Automatically assigns roles to a user when they join the server.
;filters - Allows you to filter commands based on user, role or channel!
;slowmode - Sets an arbitrary slowmode for a channel.
;tempslowmode - Sets a slowmode temporarily and then reverts it.
;mute - Mutes a user.
;unmute - Unmutes a user.
;tempmute - Mutes a user temporarily.
;tempban - Bans a user temporarily.
;purge user - Now takes multiple users!
;f User Stuff
Localised colours
UI/UX Improvements (See PR #189)
;f Technical Stuff
Database Restructure
Move a good chunk over to Kotlin
;d We're excited to finally be updating the bot with a whole bunch of new features! After a long wait, we can finally push out new functionality to Cascade 🎉
Due to the massive rewrite of everything, there is probably a tad bit of buggage. As usual, please report any problems in the <#548975880924561409> channel!
To find the usage of a command, please use ;usage <command> 😃
;t The long awaited Alpha update!
;c green

;preview

;preview

;preview

;preview

;preview

;publish


Actively use the cache for query feeds

This stems from a brief discussion with @Minoru on IRC.
Currently newsboat does one iteration over the entire database per query feed in order to test whether an article belongs in it. To achieve this it also requires to keep all the articles loaded in RAM at all times (See also #111). This is fine for small caches but quickly becomes noticeable in bigger ones both CPU and memory-wise.
What I proposed was to attempt to use the cache to a bigger degree in query feeds and as a goal convert as much of the search logic as possible to SQL to speed things up.

My response on IRC was: this is possible, but I foresee some hurdles.

Need to make sure SQLite supports the same regex syntax as we use in Newsboat (for =~ and !~).

I don't think mapping a filter language expression into an SQL statement would be trivial. E.g. an innocuous query:test:feedtitle = "hello" actually requires an SQL join, because titles are in rss_feed table and item IDs are in rss_item.
I'm fearful of joins because they make it harder to piece two queries together and not drive performance into the ground in the process. We might need to write/find an SQL query builder to relieve some of that pain >_<

Some attributes aren't directly available in the DB; e.g. age need to be calculated, articleindex is outright absent from the DB because it depends on current sorting criteria, tags aren't in the DB either.

That last point makes me think that the new system will be a hybrid between SQL and in-software searches: do what you can in SQL, refine later. Which won't be easy with any complex statement. Consider (title = "hello" and age > 5) or title = "goodbye": I think the only sane approach is to run separate SQL queries for title = "hello" and title = "goodbye", refine results of the first one, and merge the two results.
We also briefly touched on adding new operators, to which I responded: I'd rather not introduce operators that are semantically the same as the ones we already have; i.e. there shouldn't be separate operators for "regex search" and "SQL regex search". Rationale: even though I see how it can help write faster queries, I think the complexity outweights the benefits. We'll have to explain the difference, and users will still cut themselves on differences in behaviour. So I'd really, really like to avoid that.

Regarding @Minoru's first point:

Need to make sure SQLite supports the same regex syntax as we use in Newsboat (for =~ and !~).

SQLite supports a REGEXP operator which can be used with a callback into our own code.
This makes it possible to use the exact regex implementation we currently use from that callback.
Relevant SQLite documentation:

REGEXP: https://www.sqlite.org/lang_expr.html#the_like_glob_regexp_and_match_operators
Callback registration: https://www.sqlite.org/appfunc.html

Regarding @Minoru's first point:

Need to make sure SQLite supports the same regex syntax as we use in Newsboat (for =~ and !~).

SQLite supports a REGEXP operator which can be used with a callback into our own code.
This makes it possible to use the exact regex implementation we currently use from that callback.
Relevant SQLite documentation:

REGEXP: https://www.sqlite.org/lang_expr.html#the_like_glob_regexp_and_match_operators
Callback registration: https://www.sqlite.org/appfunc.html


Conflicted Events Warning

Make the Conflicted Events tab red when such events are created.

Notes while at work
`
use backend\models\AgcCal;
$testConflict=0;
if(yii::$app->controller->hasPermission('calendar/conflict')) {
if(yii::$app->controller->hasPermission('calendar/all')) {
$testConflict = (new AgcCal)::find()->where(['conflict' => 1])->count();
} else {
$testConflict = (new AgcCal)::find()->where(['conflict' => 1])
->andWhere(['in','agc_calendar.club_id',json_decode(Yii::$app->user->identity->clubs)])
->count();
}
}
echo "Conflicts found: ".$testConflict;
if ($testConflict > 0);
?>
<style="color:red">glyphicon glyphicon-warning-sign `

Updated!  looks annoying!

I was almost there. I had the code to change the tab color and was trying to figure out the conflict=1 call when I had to get to bed. I then woke up this morning realizing how to do it. You obviously did it and was close to what I was thinking. However, I would have never figured out "json_decode(Yii::$app->user->identity->clubs)" in the Where line. I think it looks good!

I left comment on the issue.
On Thu, Feb 4, 2021 at 5:48 PM Shark<EMAIL_ADDRESS>wrote:

Closed #142 https://github.com/sharkbit/BadgeApp/issues/142.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/sharkbit/BadgeApp/issues/142#event-4293423243, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AQAZBWDUISLMU4FE7CWZER3S5MP37ANCNFSM4XB2KSIA
.

Ummm, your fix is not working properly. It shows flashing Conflict tab when there are no conflicts...

if you want to work on it you can add the line below to help troubleshoot, in the php section around line 19.
echo "# of Conflicts found: ". $testConflict


remove _current NULL check in ARM's _is_thread_user()

See #4080 .  the call to k_object_grant_access() in setup_new_thread() makes this necessary. revise the code so we don't need the extra NULL check in this very hot code path.

@andrewboie is this still valid?  we do not have k_object_grant_access anymore :)

@nashif it is, the API is called k_object_access_grant()


Error when deleting an object that doesn't exist

In this case I tried to do DeletePerson with identifier: "a uuid that wasn't in the db"
{
  "errors": [
    {
      "message": "Cannot read property 'identifier' of null",
      "locations": [
        {
          "line": 2,
          "column": 3
        }
      ],
      "path": [
        "DeletePerson"
      ],
      "extensions": {
        "code": "INTERNAL_SERVER_ERROR",
        "exception": {
          "stacktrace": [
            "TypeError: Cannot read property 'identifier' of null",
            "    at /app/src/transformers/subscriptionFieldTransformer.js:59:20",
            "    at process._tickCallback (internal/process/next_tick.js:68:7)"
          ]
        }
      }
    }
  ],
  "data": {
    "DeletePerson": null
  }
}

Thanks for reporting @alastair, I've found the problem. This error is returned if the node doesn't exist in the database. However, while investigating, I've discovered that the delete mutation should even be running that particular code (only for create mutations).
I will submit a PR fixing this.


[WIP] Fix comment handling

Currently comments are cut off incorrectly. @gokmen do you have an idea how to approach this problem?

Hi @thomaschaaf I need to take a look but this is the issue https://github.com/gokmen/hcl-to-json/blob/master/src/tokenize.coffee#L20

Fixed with https://github.com/gokmen/hcl-to-json/commit/f63fd84f80a773fa33f3824f39273831c93013b4 version bumped to 0.0.8 on npm.