Disable coverage checks in jobFiltering (#1318)
Disable coverage checks in jobFiltering (#1318)

At first, I wanted to disable this just to make the test suite
pass. After removing the coverage check, I wanted to see the coverage
results to understand how skewed the test is, but… apparently there's
no way to do it, at least not from test-framework.

While investigating to see how test-framework runs quickcheck, I
happened upon this in the code of the testProperty:

    where
      toPropertyStatus (Success {})                              = PropertyOK
      ...
      #if MIN_VERSION_QuickCheck(2,8,0) && !MIN_VERSION_QuickCheck(2,12,0)
        toPropertyStatus (InsufficientCoverage _ _ _)              = PropertyInsufficientCoverage
      #endif

Which made me very confused. Why are coverage results disabled with
newer QuickCheck? Turns out that QC 2.12 changelog says: “cover no
longer causes a property failure if coverage is insufficient”. And
then follows with: “New property combinator checkCoverage, which
checks coverage requirements in a statistically sound way, and *does*
fail if they are not met.”

So it appears that currently coverage checks are unreliable (and
that's why we have `stableCover` defined ourselves), and furthermore
more changes in QC 2.12 make it much better and also add reporting for
coverage checks (see
https://github.com/nick8325/quickcheck/blob/master/changelog for more
details).

Which means that for now disabling such checks is not that bad, and at
least it allows the filtering to be partially tested.

Signed-off-by: Iustin Pop <acf83a5274f31a43af6b914614365999469c1a31@k1024.org>

ganeti/ganeti,ganeti/ganeti,ganeti/ganeti,mbakke/ganeti,mbakke/ganeti,mbakke/ganeti