cilium, cleanup: reorder bpf fs cleanup and remove attached BPF progs
cilium, cleanup: reorder bpf fs cleanup and remove attached BPF progs

This fixes a bug where during a Cilium upgrade test full connectivity is lost.
The major reason why it is happening is basically the test for ...

  if strings.Contains(bpfFilter.Name, "bpf_netdev.o") [...]

... which is buggy due to multi-dev support since we now end up with something
like bpf_netdev_eno1.o instead of bpf_netdev.o . This means that tc BPF progs
never get removed from the devices, yet we're tearing down everything else and
once we remove all files from the BPF fs then in case of tail call maps, the
kernel will purge all BPF programs inside tail call maps due to circular dependency
avoidance. However, this means that those still-attached programs go into a
drop-all mode due to missed tail calls.

Fix this in four ways: i) Match on "bpf_netdev" prefix instead so we catch old
but also new names, ii) Reorder BPF fs cleanup with tc BPF prog removal, meaning,
we first need to remove all progs from the devices and only then clean up BPF fs,
iii) When calling with --bpf-state only, then also remove progs from devices, and
iv) Last but not least, add also other tc prog names. Step ii) to iv) is necessary
since there can still be a small race window where we missed tail calls right
before the device BPF prog removal.

'K8sUpdates Tests upgrade and downgrade from a Cilium stable image to master' test
case is then fixed.

Signed-off-by: Daniel Borkmann <3d0f3b9ddcacec30c4008c5e030e6c13a478cb4f@iogearbox.net>

cilium/cilium,tgraf/cilium,michi-covalent/cilium,michi-covalent/cilium,michi-covalent/cilium,cilium-team/cilium,michi-covalent/cilium,cilium/cilium,tklauser/cilium,tgraf/cilium,tgraf/cilium,cilium/cilium,michi-covalent/cilium,tklauser/cilium,tklauser/cilium,cilium/cilium,tklauser/cilium,cilium/cilium,tgraf/cilium,cilium-team/cilium,tklauser/cilium,tgraf/cilium,tgraf/cilium