ARROW-11687: [Rust][DataFusion] RepartitionExec Hanging
ARROW-11687: [Rust][DataFusion] RepartitionExec Hanging

@andygrove

I found an interesting defect where the final partition of the `RepartitionExec::execute` thread spawner was consistently not being spawned via `tokio::spawn`. This meant that `RepartitionStream::poll_next` was sitting waiting forever for data that never arrived. I am unable to reproduce via DataFusion tests.

It looks like a race condition where the `JoinHandle` was not being `await`ed and something strange going on with the internals of tokio like lazy evaluation?

This PR fixes the problem.

Closes #9523 from seddonm1/tokio-race-condition

Authored-by: Mike Seddon <08143e5b7f3ef9b2c05b03e8981fb36a6affd6c2@gmail.com>
Signed-off-by: Andrew Lamb <02e0a999c50b1f88df7a8f5a04e1b76b35ea6a88@nerdnetworks.org>

laurentgo/arrow,wesm/arrow,laurentgo/arrow,cpcloud/arrow,kou/arrow,wesm/arrow,icexelloss/arrow,apache/arrow,apache/arrow,apache/arrow,laurentgo/arrow,kou/arrow,icexelloss/arrow,laurentgo/arrow,kou/arrow,icexelloss/arrow,laurentgo/arrow,kou/arrow,kou/arrow,apache/arrow,wesm/arrow,laurentgo/arrow,apache/arrow,kou/arrow,apache/arrow,apache/arrow,cpcloud/arrow,cpcloud/arrow,wesm/arrow,wesm/arrow,apache/arrow,wesm/arrow,wesm/arrow,laurentgo/arrow,wesm/arrow,cpcloud/arrow,wesm/arrow,icexelloss/arrow,wesm/arrow,cpcloud/arrow,cpcloud/arrow,kou/arrow,icexelloss/arrow,cpcloud/arrow,wesm/arrow,wesm/arrow,laurentgo/arrow,laurentgo/arrow,kou/arrow,apache/arrow,kou/arrow,icexelloss/arrow,cpcloud/arrow,icexelloss/arrow,icexelloss/arrow,kou/arrow,laurentgo/arrow,laurentgo/arrow,wesm/arrow,kou/arrow,apache/arrow,icexelloss/arrow,apache/arrow,icexelloss/arrow,apache/arrow,icexelloss/arrow,cpcloud/arrow,kou/arrow,apache/arrow,icexelloss/arrow,cpcloud/arrow,cpcloud/arrow,kou/arrow,apache/arrow,icexelloss/arrow,cpcloud/arrow,cpcloud/arrow,laurentgo/arrow,laurentgo/arrow,icexelloss/arrow,kou/arrow,wesm/arrow,laurentgo/arrow,cpcloud/arrow,laurentgo/arrow