test/encode/avcenc: increase the times of trying after overflow
test/encode/avcenc: increase the times of trying after overflow

Also unmap the coded buffer if overflow

Signed-off-by: Xiang, Haihao <8d677183e5993577d4fa73a419b76987913292d6@intel.com>

CyanogenMod/android_hardware_intel_libva,halleyzhao/libva-fdo,android-ia/hardware_intel_libva,CyanogenMod/android_hardware_intel_libva,CyanogenMod/android_hardware_intel_libva,android-ia/hardware_intel_libva,halleyzhao/libva-fdo,android-ia/hardware_intel_libva,halleyzhao/libva-fdo