io: do not dispatch I/O outside the pollers
io: do not dispatch I/O outside the pollers

Currently when submit_io is called we have two possibilities with
respect to I/O dispatching:

- we may batch until the next poll period
- we may dispatch now if certain conditions apply.

In this patch I am proposing we revisit these conditions, and simplify
this code path by always batching. I will argue that the rationale
behind each of those conditions no longer apply in modern Seastar and
should be discarded.

Two major changes happened between now and then: we now have a CPU
scheduler, a mature I/O Scheduler, and the default task quota has
been severely reduced.

1) _io_queue->queued_requests() > 0

This was introduced so that systems bottlenecked by I/O would have
better latencies. We lose our batching abilities but should be
dispatching I/O sooner and flushing the queue sooner. However, the task
quota at that time was 2.0ms, and without the CPU scheduler we had no
guarantee that the I/O continuations would run in a reasonable time. The
task quota now is 0.5ms and we have plans to reduce it. Not to mention,
we have very good control of CPU continuations through the CPU scheduler
and usually won't starve the I/O Scheduler related continuations,
meaning that the added latency to a request that is not batched is never
too high. Batching should win.

2) _pending_aio.size() >= std::min(max_aio / 4, _io_queue->_capacity / 2))

This condition specifies that if we have a certain amount of requests
pending, we are better off flushing now. The first condition is
"max_aio/4". The idea behind such a condition is, we'll block on a
semaphore if reach max_aio pending requests, so better avoid it. This,
however, predates the I/O Scheduler. Now that the I/O Scheduler sits
between the request submission and the _pending_aio array. Usually the
I/O Scheduler's queue has less than max_aio capacity and we'll never
reach max_aio and this test is moot.

For the test "io_queue->_capacity/2", the argument is also one of
latency. As we saw for item (1), the added latencies are now bounded we
shouldn't bother with that.

Below, some results obtained by running a 250usec task quota on a single CPU

Summary Results:
================

I am using io_tester to evaluate the behavior of the I/O subsystem when
faced with this change. I am testing 3 scenarios:

Scenario 1   : latency-bound reads against 3 heavy CPU-hogs.
Discussion   : no visible change, as we tend not to trigger those
               conditions

Scenario 2   : throughput-bound reads against 3 heavy CPU-hogs.
Discussion   : The I/O throughput decreases as the latencies increase by
               around a task quota as expected. But CPU throughput increases.

Scenario 3   : I/O-only, throughput-bound
Discussion   : Small throughput increase (2 %) as we increase batching opportunities.

Raw results:
============

Scenario 1:

Before:
Class  0 (latency_reads: 100 shares, 4096-byte RAND READ, 4 concurrent requests, NO think time)
  Throughput         :    25461 KB/s
  Lat average        :      420 usec
  Lat quantile=  0.5 :      416 usec
  Lat quantile= 0.95 :      420 usec
  Lat quantile= 0.99 :      618 usec
  Lat quantile=0.999 :     5130 usec
  Lat max            :     6182 usec

Class  1 (cpu_hog: 10 shares, 200 us CPU execution time, 10 concurrent requests, NO think time)
  Throughput         :  1601.89 continuations/s

Class  2 (cpu_hog2: 10 shares, 200 us CPU execution time, 10 concurrent requests, NO think time)
  Throughput         :  1601.66 continuations/s

Class  3 (cpu_hog3: 10 shares, 200 us CPU execution time, 10 concurrent requests, NO think time)
  Throughput         :  1601.84 continuations/s

After:
Class  0 (latency_reads: 100 shares, 4096-byte RAND READ, 4 concurrent requests, NO think time)
  Throughput         :  25460.3 KB/s
  Lat average        :      420 usec
  Lat quantile=  0.5 :      417 usec
  Lat quantile= 0.95 :      421 usec
  Lat quantile= 0.99 :      617 usec
  Lat quantile=0.999 :     2843 usec
  Lat max            :     6140 usec

Class  1 (cpu_hog: 10 shares, 200 us CPU execution time, 10 concurrent requests, NO think time)
  Throughput         :  1601.48 continuations/s

Class  2 (cpu_hog2: 10 shares, 200 us CPU execution time, 10 concurrent requests, NO think time)
  Throughput         :  1602.88 continuations/s

Class  3 (cpu_hog3: 10 shares, 200 us CPU execution time, 10 concurrent requests, NO think time)
  Throughput         :  1601.98 continuations/s

---------------------------------------------------------------------------------------------------

Scenario 2:

Before:
Class  0 (latency_reads: 100 shares, 4096-byte RAND READ, 40 concurrent requests, NO think time)
  Throughput         :   229700 KB/s
  Lat average        :      426 usec
  Lat quantile=  0.5 :      491 usec
  Lat quantile= 0.95 :      526 usec
  Lat quantile= 0.99 :      739 usec
  Lat quantile=0.999 :      817 usec
  Lat max            :     6381 usec

Class  1 (cpu_hog: 10 shares, 200 us CPU execution time, 10 concurrent requests, NO think time)
  Throughput         :  1281.26 continuations/s

Class  2 (cpu_hog2: 10 shares, 200 us CPU execution time, 10 concurrent requests, NO think time)
  Throughput         :  1281.13 continuations/s

Class  3 (cpu_hog3: 10 shares, 200 us CPU execution time, 10 concurrent requests, NO think time)
  Throughput         :  1280.88 continuations/s

After:
Class  0 (latency_reads: 100 shares, 4096-byte RAND READ, 40 concurrent requests, NO think time)
  Throughput         :   210237 KB/s
  Lat average        :      504 usec
  Lat quantile=  0.5 :      492 usec
  Lat quantile= 0.95 :      736 usec
  Lat quantile= 0.99 :      746 usec
  Lat quantile=0.999 :      758 usec
  Lat max            :     6466 usec

Class  1 (cpu_hog: 10 shares, 200 us CPU execution time, 10 concurrent requests, NO think time)
  Throughput         :  1334.57 continuations/s

Class  2 (cpu_hog2: 10 shares, 200 us CPU execution time, 10 concurrent requests, NO think time)
  Throughput         :  1335.07 continuations/s

Class  3 (cpu_hog3: 10 shares, 200 us CPU execution time, 10 concurrent requests, NO think time)
  Throughput         :  1332.37 continuations/s

---------------------------------------------------------------------------------------------------

Scenario 3:

Before:
Class  0 (latency_reads: 100 shares, 4096-byte RAND READ, 40 concurrent requests, NO think time)
  Throughput         :   385557 KB/s
  Lat average        :      359 usec
  Lat quantile=  0.5 :      353 usec
  Lat quantile= 0.95 :      435 usec
  Lat quantile= 0.99 :      495 usec
  Lat quantile=0.999 :      631 usec
  Lat max            :     1302 usec

After:
Class  0 (latency_reads: 100 shares, 4096-byte RAND READ, 40 concurrent requests, NO think time)
  Throughput         :   393572 KB/s
  Lat average        :      382 usec
  Lat quantile=  0.5 :      374 usec
  Lat quantile= 0.95 :      467 usec
  Lat quantile= 0.99 :      550 usec
  Lat quantile=0.999 :      769 usec
  Lat max            :     1289 usec

Signed-off-by: Glauber Costa <7ba9037b0dd006f9408bd45b94707becd5cd4e87@scylladb.com>

dreamsxin/seastar,scylladb/seastar,cloudius-systems/seastar,syuu1228/seastar,avikivity/seastar,dreamsxin/seastar,scylladb/seastar,cloudius-systems/seastar,avikivity/seastar,syuu1228/seastar,cloudius-systems/seastar,dreamsxin/seastar,scylladb/seastar,avikivity/seastar,syuu1228/seastar