qemu: adjust memlock for multiple vfio/vdpa devices
qemu: adjust memlock for multiple vfio/vdpa devices

When multiple VFIO or VDPA devices are assigned to a guest, the guest
can fail to start because the guest fails to map enough memory. For
example, the case mentioned in
https://bugzilla.redhat.com/show_bug.cgi?id=2111317 results in this
failure:

    2021-08-05T09:51:47.692578Z qemu-kvm: failed to write, fd=31, errno=14 (Bad address)
    2021-08-05T09:51:47.692590Z qemu-kvm: vhost vdpa map fail!
    2021-08-05T09:51:47.692594Z qemu-kvm: vhost-vdpa: DMA mapping failed, unable to continue

The current memlock limit calculation does not work for scenarios where
there are multiple such devices assigned to a guest. The root causes are
a little bit different between VFIO and VDPA devices.

For VFIO devices, the issue only occurs when a vIOMMU is present. In
this scenario, each vfio device is assigned a separate AddressSpace
fully mapping guest RAM. When there is no vIOMMU, the devices are all
within the same AddressSpace so no additional memory limit is needed.

For VDPA devices, each device requires the full memory to be mapped
regardless of whether there is a vIOMMU or not.

In order to enable these scenarios, we need to multiply memlock limit
by the number of VDPA devices plus the number of VFIO devices for guests
with a vIOMMU. This has the potential for pushing the memlock limit
above the host physical memory and negating any protection that these
locked memory limits are providing, but there is no other short-term
solution.

In the future, there should be have a revised userspace iommu interface
(iommufd) that the VFIO and VDPA backends can make use of. This will be
able to share locked memory limits between both vfio and vdpa use cases
and address spaces and then we can disable these short term hacks. But
this is still in development upstream.

Resolves: https://bugzilla.redhat.com/2111317

Signed-off-by: Jonathon Jongsma <c7254805a17bd5c41e24c3a278fcbf3afcdfb0fa@redhat.com>
Reviewed-by: Laine Stump <c23361c43fbf79fed83e8b76173707b083d6caf5@redhat.com>

libvirt/libvirt,olafhering/libvirt,zippy2/libvirt,zippy2/libvirt,libvirt/libvirt,olafhering/libvirt,jfehlig/libvirt,libvirt/libvirt,olafhering/libvirt,libvirt/libvirt,zippy2/libvirt,olafhering/libvirt,zippy2/libvirt,jfehlig/libvirt,jfehlig/libvirt,jfehlig/libvirt