txscript: Rework check signature encoding test.
txscript: Rework check signature encoding test.

This reworks the tests for CheckSignatureEncoding to test for the
explicit reason for the failure to ensure that each test is actually
testing the intended condition and be more consistent with the more
recent tests.

It also adds a few more tests for some corner cases where signatures
will ultimately fail validation due to ECDSA requirements, but are still
valid encodings from the standpoint of preventing malleability.

decred/dcrd,decred/dcrd