test: Avoid installing Cilium for K8sBandwidth if tests are skipped
test: Avoid installing Cilium for K8sBandwidth if tests are skipped

The overall structure for test K8sBandwidth looks to have been extracted
from K8sServices. It works fine but is more complex than necessary and
leads to unintended behavior when tests are skipped. This commit
simplifies the structure to have a single conditional Context
(conditioned on net-next kernel) inside which the three It tests are
run.

Cilium was also installed with the bandwidth manager enabled *before*
the conditional Context. That installation would therefore happen
regardless of whether bandwidth tests should actually be skipped,
sometimes even leading to flakes on 4.9 kernels [1].

Removing this initial installation of Cilium implies that the test pods
are now deployed (once for all tests) before Cilium is installed. We
therefore need to wait for the test pods, with a new helper
waitForTestPods(), after each re-installation of Cilium.

1 - https://jenkins.cilium.io/job/Cilium-PR-Ginkgo-Tests-K8s/3740/testReport/junit/Suite-k8s-1/16/K8sBandwidthTest_Checks_Bandwidth_Rate_Limiting/
Signed-off-by: Paul Chaignon <a027184a55211cd23e3f3094f1fdc728df5e0500@cilium.io>

tklauser/cilium,cilium/cilium,cilium/cilium,tgraf/cilium,michi-covalent/cilium,michi-covalent/cilium,michi-covalent/cilium,cilium/cilium,tklauser/cilium,tklauser/cilium,cilium/cilium,tklauser/cilium,tgraf/cilium,tgraf/cilium,tgraf/cilium,michi-covalent/cilium,cilium/cilium,tklauser/cilium,michi-covalent/cilium,tgraf/cilium,tgraf/cilium