rbd: make rbd_obj_notify_ack() synchronous
rbd: make rbd_obj_notify_ack() synchronous

The only user of rbd_obj_notify_ack() is rbd_watch_cb(). It used
asynchronously with no tracking of when the notify ack completes, so
it may still be in progress when the osd_client is shut down.  This
results in a BUG() since the osd client assumes no requests are in
flight when it stops. Since all notifies are flushed before the
osd_client is stopped, waiting for the notify ack to complete before
returning from the watch callback ensures there are no notify acks in
flight during shutdown.

Rename rbd_obj_notify_ack() to rbd_obj_notify_ack_sync() to reflect
its new synchronous nature.

Signed-off-by: Josh Durgin <51652c783f70f5b272ebd02f1fd20d3af7a2449f@inktank.com>
Reviewed-by: Alex Elder <f429030cf5c0faf36fac3d102073b6e63a647baa@linaro.org>

KristFoundation/Programs,KristFoundation/Programs,KristFoundation/Programs,KristFoundation/Programs,KristFoundation/Programs,KristFoundation/Programs