Fix application stress tests (#4228)
Fix application stress tests (#4228)

Fixes https://github.com/ray-project/ray/issues/4227
richardliaw/ray,ujvl/ray-ng,ray-project/ray,ujvl/ray-ng,stephanie-wang/ray,stephanie-wang/ray,stephanie-wang/ray,pcmoritz/ray-1,atumanov/ray,stephanie-wang/ray,robertnishihara/ray,stephanie-wang/ray,ray-project/ray,atumanov/ray,richardliaw/ray,pcmoritz/ray-1,pcmoritz/ray-1,atumanov/ray,ujvl/ray-ng,stephanie-wang/ray,ray-project/ray,richardliaw/ray,robertnishihara/ray,atumanov/ray,atumanov/ray,pcmoritz/ray-1,robertnishihara/ray,robertnishihara/ray,pcmoritz/ray-1,richardliaw/ray,ray-project/ray,atumanov/ray,pcmoritz/ray-1,stephanie-wang/ray,ray-project/ray,robertnishihara/ray,pcmoritz/ray-1,robertnishihara/ray,pcmoritz/ray-1,robertnishihara/ray,ray-project/ray,richardliaw/ray,ray-project/ray,atumanov/ray,richardliaw/ray,richardliaw/ray,atumanov/ray,stephanie-wang/ray,stephanie-wang/ray,robertnishihara/ray,richardliaw/ray,robertnishihara/ray,ujvl/ray-ng,ray-project/ray