Fix MV decimation issue with 0 rate costs.
Fix MV decimation issue with 0 rate costs.

Currently, decimating a vertex _always_ reduces rate (dr < 0).
When we had extra "fake" vertices in the border, they were set to
 have dr == 0, but since they were outside the frame, their effect
 on distortion was also zero.
This allowed some inconsistencies to go unnoticed in the comparison
 function used to drive the priority queue.

The actual bug is that with the previous code, grouped into classes,
 (dr == 0, dd > 0) < (dr == 0, dd == 0) and
 (dr == 0, dd == 0) < (dr != 0), but
 (dr == 0, dd > 0) > (dr != 0).
Thus the comparison failed to be transitive.

Fixing these inconsistencies shouldn't affect current results, but
 it does affect experiments with new costs that _can_ go to zero
 (e.g., jmspeex's exp_mvflags branch).
In that case, what could happen is that early in the process, there
 would be many level 6 vertices with dr == 0 && dd == 0 (because
 those vertices do not have a cost added for their children's
 flags, since they have no children), all of which are put at the
 front of the queue.
It was then easy for decimation of these vertices to create
 vertices with dr == 0 and dd > 0 in their neighbors, near the head
 of the queue, and because of the comparison inconsistency, for one
 of these to accidentally be promoted to the head.
At this point, the optimizer would see that the value at the head
 of the queue (presumably the best choice) was not improving R-D
 performance, and stop, after performing almost no decimation.
The result would be a frame with a very large rate that takes a
 very long time to encode.

Allowing positive rate changes (that is, allowing removing a MV to
 make rate go up) may require more work, as the slope comparisons
 assume dr is non-positive on both sides to avoid divisions.

ycho/daala,xiphmont/daala,ycho/daala,xiphmont/daala,xiph/daala,kodabb/daala,HeadhunterXamd/daala,jmvalin/daala,luctrudeau/daala,HeadhunterXamd/daala,HeadhunterXamd/daala,jmvalin/daala,HeadhunterXamd/daala,KyleSiefring/daala,xiph/daala,luctrudeau/daala,KyleSiefring/daala,xiph/daala,luctrudeau/daala,luctrudeau/daala,kodabb/daala,kodabb/daala,xiph/daala,jmvalin/daala,KyleSiefring/daala,KyleSiefring/daala,xiphmont/daala,xiphmont/daala,jmvalin/daala,xiphmont/daala,kodabb/daala,ycho/daala,jmvalin/daala,ycho/daala,HeadhunterXamd/daala,KyleSiefring/daala,ycho/daala,xiph/daala,kodabb/daala,luctrudeau/daala