reactor: fix issue with hrtimer completions being lost
reactor: fix issue with hrtimer completions being lost

Our current code has a problem with missing triggers for hrtimers. The
problem only seem to happen without the overprovisioned flag but doesn't
seem to happen with it, which is why it is so far unseen.

After my poller refactoring the problem was brought to the surface and
now it happens regardless of flags.

The reason for the failure is that the hrtimer always must be registered
with the polling_io context so it can be reaped. It is not enough to
make sure it is registered with the preempting_io context because that
context is only reaped tied to the task quota lifetime, and may not be
reaped at all. See for instance run_tasks: we reset the preempt_io
context through reset_preemption_monitor, which is what register the
iocb, but we only do that if the loop ended through preemption.

When we go to sleep, we register the hrtimer with the polling I/O. But
once we wake up, we don't register it back.

Before my patchset, wait_and_process_events was used for both sleeping
and for querying epoll-related completions, which guaranteed that it
would always be reaped.

Funny enough, though, when we are not overprovisioned, we don't set
poll-aio, which in turns don't register an aio_eventfd. Without an
io_eventd we may not ever register the epoll poller. The epoll poller
was conditionally registered before. This is the reason why it worked
before on overprovisioned (the real culprit was poll-aio), but didn't
always worked without it: it would only work if something else had
registered the epoll poller.

Signed-off-by: Glauber Costa <7ba9037b0dd006f9408bd45b94707becd5cd4e87@scylladb.com>
Message-Id: <20200213170246.3954-1-7ba9037b0dd006f9408bd45b94707becd5cd4e87@scylladb.com>

scylladb/seastar,avikivity/seastar,dreamsxin/seastar,cloudius-systems/seastar,scylladb/seastar,cloudius-systems/seastar,syuu1228/seastar,cloudius-systems/seastar,dreamsxin/seastar,syuu1228/seastar,avikivity/seastar,dreamsxin/seastar,scylladb/seastar,avikivity/seastar,syuu1228/seastar