rbd: fail commands after rados osd op timeout seconds
rbd: fail commands after rados osd op timeout seconds

If "rados osd op timeout" seconds have passed disable the trasnport
tpg and fail the command, so the initiator can try another path.

Note:
This catches ceph network issues, but also device/OSD ones. For the
latter we will cause failovers when they are not needed because the same
failure will be encountered on other paths, but it is better than
being stuck on a bad path in the ESX case plus in the windows/linux
case it is what they are already doing,

In a future release I hope to add something to librbd so we can
distinguish
network vs device/OSD issues.

For rbd, the "rados osd op timeout" must be longer than the timers to
catch down/out OSDs, so we can retry a different OSD before passing the
command back to the initiator.

[client]
rados osd op timeout = 30

[osd]
osd heartbeat grace = 15
osd heartbeat interval = 5

Signed-off-by: Mike Christie <39daeb1549a929782e38aa87c4e3da7f3844b09a@redhat.com>

open-iscsi/tcmu-runner,open-iscsi/tcmu-runner,agrover/tcmu-runner