[SPARK-28903][STREAMING][PYSPARK][TESTS] Fix AWS JDK version conflict that breaks Pyspark Kinesis tests
[SPARK-28903][STREAMING][PYSPARK][TESTS] Fix AWS JDK version conflict that breaks Pyspark Kinesis tests

The Pyspark Kinesis tests are failing, at least in master:

```
======================================================================
ERROR: test_kinesis_stream (pyspark.streaming.tests.test_kinesis.KinesisStreamTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jenkins/workspace/SparkPullRequestBuilder2/python/pyspark/streaming/tests/test_kinesis.py", line 44, in test_kinesis_stream
    kinesisTestUtils = self.ssc._jvm.org.apache.spark.streaming.kinesis.KinesisTestUtils(2)
  File "/home/jenkins/workspace/SparkPullRequestBuilder2/python/lib/py4j-0.10.8.1-src.zip/py4j/java_gateway.py", line 1554, in __call__
    answer, self._gateway_client, None, self._fqn)
  File "/home/jenkins/workspace/SparkPullRequestBuilder2/python/lib/py4j-0.10.8.1-src.zip/py4j/protocol.py", line 328, in get_return_value
    format(target_id, ".", name), value)
Py4JJavaError: An error occurred while calling None.org.apache.spark.streaming.kinesis.KinesisTestUtils.
: java.lang.NoSuchMethodError: com.amazonaws.regions.Region.getAvailableEndpoints()Ljava/util/Collection;
	at org.apache.spark.streaming.kinesis.KinesisTestUtils$.$anonfun$getRegionNameByEndpoint$1(KinesisTestUtils.scala:211)
	at org.apache.spark.streaming.kinesis.KinesisTestUtils$.$anonfun$getRegionNameByEndpoint$1$adapted(KinesisTestUtils.scala:211)
	at scala.collection.Iterator.find(Iterator.scala:993)
	at scala.collection.Iterator.find$(Iterator.scala:990)
	at scala.collection.AbstractIterator.find(Iterator.scala:1429)
	at scala.collection.IterableLike.find(IterableLike.scala:81)
	at scala.collection.IterableLike.find$(IterableLike.scala:80)
	at scala.collection.AbstractIterable.find(Iterable.scala:56)
	at org.apache.spark.streaming.kinesis.KinesisTestUtils$.getRegionNameByEndpoint(KinesisTestUtils.scala:211)
	at org.apache.spark.streaming.kinesis.KinesisTestUtils.<init>(KinesisTestUtils.scala:46)
...
```

The non-Python Kinesis tests are fine though. It turns out that this is because Pyspark tests use the output of the Spark assembly, and it pulls in `hadoop-cloud`, which in turn pulls in an old AWS Java SDK.

Per Steve Loughran (below), it seems like we can just resolve this by excluding the aws-java-sdk dependency. See the attached PR for some more detail about the debugging and other options.

See https://github.com/apache/spark/pull/25558#issuecomment-524042709

Closes #25559 from srowen/KinesisTest.

Authored-by: Sean Owen <3f9d48d7c3f24f7009479c5fa630bd1131ffc724@databricks.com>
Signed-off-by: Sean Owen <3f9d48d7c3f24f7009479c5fa630bd1131ffc724@databricks.com>

maropu/spark,JoshRosen/spark,rednaxelafx/apache-spark,wangyum/spark,shaneknapp/spark,hvanhovell/spark,wzhfy/spark,ueshin/apache-spark,HyukjinKwon/spark,spark-test/spark,zero323/spark,wangmiao1981/spark,spark-test/spark,mahak/spark,shuangshuangwang/spark,BryanCutler/spark,witgo/spark,caneGuy/spark,taroplus/spark,ConeyLiu/spark,rezasafi/spark,mahak/spark,nchammas/spark,nchammas/spark,BryanCutler/spark,bdrillard/spark,wangyum/spark,ConeyLiu/spark,skonto/spark,xuanyuanking/spark,dongjoon-hyun/spark,hvanhovell/spark,pgandhi999/spark,WeichenXu123/spark,WeichenXu123/spark,rezasafi/spark,ueshin/apache-spark,dongjoon-hyun/spark,darionyaphet/spark,vinodkc/spark,srowen/spark,caneGuy/spark,skonto/spark,jkbradley/spark,goldmedal/spark,holdenk/spark,zuotingbing/spark,ueshin/apache-spark,apache/spark,ConeyLiu/spark,zzcclp/spark,rednaxelafx/apache-spark,pgandhi999/spark,holdenk/spark,ptkool/spark,zero323/spark,ptkool/spark,WeichenXu123/spark,shuangshuangwang/spark,dbtsai/spark,rezasafi/spark,wangmiao1981/spark,xuanyuanking/spark,zzcclp/spark,chuckchen/spark,jiangxb1987/spark,jkbradley/spark,bdrillard/spark,dongjoon-hyun/spark,JoshRosen/spark,xuanyuanking/spark,HyukjinKwon/spark,maropu/spark,mahak/spark,gengliangwang/spark,wzhfy/spark,WeichenXu123/spark,wangyum/spark,wzhfy/spark,xuanyuanking/spark,dbtsai/spark,skonto/spark,gengliangwang/spark,zero323/spark,zzcclp/spark,apache/spark,rezasafi/spark,rednaxelafx/apache-spark,pgandhi999/spark,kevinyu98/spark,skonto/spark,apache/spark,kevinyu98/spark,jkbradley/spark,shaneknapp/spark,wangyum/spark,wangmiao1981/spark,darionyaphet/spark,shuangshuangwang/spark,rezasafi/spark,gengliangwang/spark,hvanhovell/spark,zuotingbing/spark,dbtsai/spark,taroplus/spark,witgo/spark,hvanhovell/spark,caneGuy/spark,darionyaphet/spark,spark-test/spark,dbtsai/spark,mahak/spark,dbtsai/spark,srowen/spark,darionyaphet/spark,ueshin/apache-spark,chuckchen/spark,mahak/spark,darionyaphet/spark,zuotingbing/spark,WeichenXu123/spark,HyukjinKwon/spark,bdrillard/spark,holdenk/spark,HyukjinKwon/spark,wzhfy/spark,caneGuy/spark,ueshin/apache-spark,caneGuy/spark,goldmedal/spark,HyukjinKwon/spark,nchammas/spark,rednaxelafx/apache-spark,ptkool/spark,milliman/spark,shuangshuangwang/spark,jiangxb1987/spark,wangmiao1981/spark,WeichenXu123/spark,pgandhi999/spark,goldmedal/spark,kevinyu98/spark,dongjoon-hyun/spark,jiangxb1987/spark,cloud-fan/spark,rednaxelafx/apache-spark,shuangshuangwang/spark,jiangxb1987/spark,pgandhi999/spark,mahak/spark,ConeyLiu/spark,hvanhovell/spark,spark-test/spark,witgo/spark,vinodkc/spark,skonto/spark,goldmedal/spark,milliman/spark,zuotingbing/spark,zero323/spark,wzhfy/spark,shaneknapp/spark,taroplus/spark,skonto/spark,taroplus/spark,witgo/spark,milliman/spark,holdenk/spark,dongjoon-hyun/spark,darionyaphet/spark,apache/spark,milliman/spark,wzhfy/spark,caneGuy/spark,vinodkc/spark,zero323/spark,srowen/spark,vinodkc/spark,taroplus/spark,wangyum/spark,wangmiao1981/spark,apache/spark,chuckchen/spark,kevinyu98/spark,rezasafi/spark,rezasafi/spark,goldmedal/spark,JoshRosen/spark,hvanhovell/spark,taroplus/spark,shaneknapp/spark,dongjoon-hyun/spark,HyukjinKwon/spark,ptkool/spark,witgo/spark,zero323/spark,shuangshuangwang/spark,shaneknapp/spark,zero323/spark,cloud-fan/spark,ueshin/apache-spark,cloud-fan/spark,rednaxelafx/apache-spark,bdrillard/spark,rednaxelafx/apache-spark,JoshRosen/spark,taroplus/spark,JoshRosen/spark,spark-test/spark,vinodkc/spark,chuckchen/spark,xuanyuanking/spark,JoshRosen/spark,ueshin/apache-spark,BryanCutler/spark,WeichenXu123/spark,cloud-fan/spark,ptkool/spark,kevinyu98/spark,wangmiao1981/spark,bdrillard/spark,holdenk/spark,zuotingbing/spark,cloud-fan/spark,goldmedal/spark,BryanCutler/spark,JoshRosen/spark,jkbradley/spark,jiangxb1987/spark,jiangxb1987/spark,shaneknapp/spark,cloud-fan/spark,bdrillard/spark,mahak/spark,hvanhovell/spark,spark-test/spark,BryanCutler/spark,HyukjinKwon/spark,matthewfranglen/spark,vinodkc/spark,BryanCutler/spark,dbtsai/spark,ptkool/spark,nchammas/spark,xuanyuanking/spark,zuotingbing/spark,skonto/spark,milliman/spark,srowen/spark,pgandhi999/spark,dongjoon-hyun/spark,witgo/spark,bdrillard/spark,wangmiao1981/spark,spark-test/spark,jiangxb1987/spark,gengliangwang/spark,holdenk/spark,ptkool/spark,ConeyLiu/spark,vinodkc/spark,ConeyLiu/spark,maropu/spark,chuckchen/spark,cloud-fan/spark,caneGuy/spark,holdenk/spark,jkbradley/spark,jkbradley/spark,witgo/spark,srowen/spark,shuangshuangwang/spark,srowen/spark,BryanCutler/spark,dbtsai/spark,wzhfy/spark,zzcclp/spark,wangyum/spark,milliman/spark,maropu/spark,ConeyLiu/spark,maropu/spark,chuckchen/spark,zzcclp/spark,nchammas/spark,wangyum/spark,chuckchen/spark,jkbradley/spark,pgandhi999/spark,goldmedal/spark,zzcclp/spark,xuanyuanking/spark,kevinyu98/spark,nchammas/spark,apache/spark,darionyaphet/spark,maropu/spark,milliman/spark,zuotingbing/spark,gengliangwang/spark,gengliangwang/spark,apache/spark,srowen/spark,maropu/spark,gengliangwang/spark,kevinyu98/spark,shaneknapp/spark,zzcclp/spark,nchammas/spark