doc: Sort out the relationship between <vcpu>, <vcpupin>, and <emulatorpin>
doc: Sort out the relationship between <vcpu>, <vcpupin>, and <emulatorpin>

These 3 elements conflicts with each other in either the doc
or the underlying codes.

Current problems:

Problem 1:

The doc shouldn't simply say "These settings are superseded
by CPU tuning. " for element <vcpu>. As except the tuning, <vcpu>
allows to specify the current, maxmum vcpu number. Apart from that,
<vcpu> also allows to specify the placement as "auto", which binds
the domain process to the advisory nodeset from numad.

Problem 2:

Doc for <vcpu> says its "cpuset" specify the physical CPUs
that the vcpus can be pinned. But it's not the truth, as
actually it only pin domain process to the specified physical
CPUs. So either it's a document bug, or code bug.

Problem 3:

Doc for <vcpupin> says it supersed "cpuset" of <vcpu>, it's
not quite correct, as each <vcpupin> specify the pinning policy
only for one vcpu. How about the ones which doesn't have
<vcpupin> specified? it says the vcpu will be pinned to all
available physical CPUs, but what's the meaning of attribute
"cpuset" of <vcpu> then?

Problem 4:

Doc for <emulatorpin> says it pin the emulator threads (domain
process in other context, perhaps another follow up patch to
cleanup the inconsistency is needed) to the physical CPUs
specified its attribute "cpuset". Which conflicts with
<vcpu>'s "cpuset". And actually in the underlying codes,
it set the affinity for domain process twice if both
"cpuset" for <vcpu> and <emulatorpin> are specified,
and <emulatorpin>'s pinning will override <vcpu>'s.

Problem 5:

When "placement" of <vcpu> is "auto" (I.e. uses numad to
get the advisory nodeset to which the domain process is
pinned to), it will also be overridden by <emulatorpin>,

This patch is trying to sort out the conflicts or bugs by:

1) Don't say <vcpu> is superseded by <cputune>

2) Keep the semanteme for "cpuset" of <vcpu> (I.e. Still says it
   specify the physical CPUs the virtual CPUs). But modifying it
   to mention it also set the pinning policy for domain process,
   and the CPU placement of domain process specified by "cpuset"
   of <vcpu> will be ingored if <emulatorpin> specified, and
   similary, the CPU placement of vcpu thread will be ignored
   if it has <vcpupin> specified, for vcpu which doesn't have
   <vcpupin> specified, it inherits "cpuset" of <vcpu>.

3) Don't say <vcpu> is supersed by <vcpupin>. If neither <vcpupin>
   nor "cpuset" of <vcpu> is specified, the vcpu will be pinned
   to all available pCPUs.

4) If neither <emulatorpin> nor "cpuset" of <vcpu> is specified,
   the domain process (emulator threads in the context) will be
   pinned to all available pCPUs.

5) If "placement" of <vcpu> is "auto", <emulatorpin> is not allowed.

6) hotplugged vcpus will also inherit "cpuset" of <vcpu>

Codes changes according to above document changes:

1) Inherit def->cpumask for each vcpu which doesn't have <vcpupin>
   specified, during parsing.

2) ping the vcpu which doesn't have <vcpupin> specified to def->cpumask
   either by cgroup for sched_setaffinity(2), which is actually done
   by 1).

3) Error out if "placement" == "auto", and <emulatorpin> is specified.
   Otherwise, <emulatorpin> is honored, and "cpuset" of <cpuset> is
   ignored.

4) Setup cgroup for each hotplugged vcpu, and setup the pinning policy
   by either cgroup or sched_setaffinity(2).

5) Remove cgroup and <vcpupin> for each hot unplugged vcpu.

Patches are following (6 in total except this patch)

zhlcindy/libvirt-1.1.4-maintain,agx/libvirt,novel/fbsd-libvirt,novel/fbsd-libvirt,datto/libvirt,nertpinx/libvirt,rlaager/libvirt,crobinso/libvirt,cbosdo/libvirt,andreabolognani/libvirt,jardasgit/libvirt,shugaoye/libvirt,zippy2/libvirt,novel/fbsd-libvirt,VenkatDatta/libvirt,zhlcindy/libvirt-1.1.4-maintain,iam-TJ/libvirt,nertpinx/libvirt,jeckersb/libvirt,olafhering/libvirt,agx/libvirt,novel/fbsd-libvirt,iam-TJ/libvirt,cbosdo/libvirt,zippy2/libvirt,taget/libvirt,siboulet/libvirt-openvz,elmarco/libvirt,eskultety/libvirt,andreabolognani/libvirt,novel/fbsd-libvirt,olafhering/libvirt,jfehlig/libvirt,eskultety/libvirt,crobinso/libvirt,bjzhang/libvirt,datto/libvirt,jardasgit/libvirt,andreabolognani/libvirt,eskultety/libvirt,fabianfreyer/libvirt,jfehlig/libvirt,libvirt/libvirt,siboulet/libvirt-openvz,siboulet/libvirt-openvz,rlaager/libvirt,iam-TJ/libvirt,fabianfreyer/libvirt,zhlcindy/libvirt-1.1.4-maintain,bjzhang/libvirt,VenkatDatta/libvirt,nertpinx/libvirt,fabianfreyer/libvirt,libvirt/libvirt,bjzhang/libvirt,datto/libvirt,datto/libvirt,elmarco/libvirt,fabianfreyer/libvirt,cbosdo/libvirt,trainstack/libvirt,elmarco/libvirt,shugaoye/libvirt,emaste/libvirt,jardasgit/libvirt,agx/libvirt,agx/libvirt,VenkatDatta/libvirt,VenkatDatta/libvirt,jeckersb/libvirt,trainstack/libvirt,cbosdo/libvirt,jfehlig/libvirt,shugaoye/libvirt,taget/libvirt,novel/fbsd-libvirt,nertpinx/libvirt,taget/libvirt,zhlcindy/libvirt-1.1.4-maintain,fabianfreyer/libvirt,bjzhang/libvirt,trainstack/libvirt,emaste/libvirt,jardasgit/libvirt,zippy2/libvirt,datto/libvirt,zippy2/libvirt,libvirt/libvirt,novel/fbsd-libvirt,jeckersb/libvirt,jeckersb/libvirt,rlaager/libvirt,trainstack/libvirt,rlaager/libvirt,taget/libvirt,siboulet/libvirt-openvz,elmarco/libvirt,jeckersb/libvirt,jeckersb/libvirt,emaste/libvirt,novel/fbsd-libvirt,elmarco/libvirt,trainstack/libvirt,shugaoye/libvirt,cbosdo/libvirt,novel/fbsd-libvirt,libvirt/libvirt,nertpinx/libvirt,andreabolognani/libvirt,crobinso/libvirt,crobinso/libvirt,iam-TJ/libvirt,agx/libvirt,andreabolognani/libvirt,iam-TJ/libvirt,emaste/libvirt,eskultety/libvirt,VenkatDatta/libvirt,zhlcindy/libvirt-1.1.4-maintain,jardasgit/libvirt,jeckersb/libvirt,bjzhang/libvirt,olafhering/libvirt,trainstack/libvirt,rlaager/libvirt,emaste/libvirt,iam-TJ/libvirt,shugaoye/libvirt,emaste/libvirt,jfehlig/libvirt,emaste/libvirt,eskultety/libvirt,taget/libvirt,trainstack/libvirt,iam-TJ/libvirt,olafhering/libvirt,siboulet/libvirt-openvz