[SPARK-26148][PYTHON][TESTS] Increases default parallelism in PySpark tests to speed up
[SPARK-26148][PYTHON][TESTS] Increases default parallelism in PySpark tests to speed up

## What changes were proposed in this pull request?

This PR proposes to increase parallelism in PySpark tests to speed up from 4 to 8.

It decreases the elapsed time from

https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99163/consoleFull
Tests passed in 1770 seconds

to

https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99186/testReport/
Tests passed in 1027 seconds

## How was this patch tested?

Jenkins tests

Closes #23111 from HyukjinKwon/parallelism.

Authored-by: hyukjinkwon <2680965ad5aae29275c8fb10150cb10f5554e78b@apache.org>
Signed-off-by: hyukjinkwon <2680965ad5aae29275c8fb10150cb10f5554e78b@apache.org>

dongjoon-hyun/spark,LantaoJin/spark,srowen/spark,jiangxb1987/spark,WeichenXu123/spark,yanboliang/spark,rednaxelafx/apache-spark,ptkool/spark,dongjoon-hyun/spark,caneGuy/spark,techaddict/spark,spark-test/spark,skonto/spark,dbtsai/spark,HyukjinKwon/spark,mdespriee/spark,milliman/spark,HyukjinKwon/spark,maropu/spark,cloud-fan/spark,witgo/spark,spark-test/spark,pgandhi999/spark,witgo/spark,ConeyLiu/spark,shuangshuangwang/spark,dongjoon-hyun/spark,actuaryzhang/spark,apache/spark,jiangxb1987/spark,skonto/spark,JoshRosen/spark,spark-test/spark,HyukjinKwon/spark,holdenk/spark,nchammas/spark,mahak/spark,nchammas/spark,chuckchen/spark,highfei2011/spark,chuckchen/spark,milliman/spark,ConeyLiu/spark,JoshRosen/spark,hhbyyh/spark,spark-test/spark,rezasafi/spark,facaiy/spark,srowen/spark,WindCanDie/spark,hhbyyh/spark,hhbyyh/spark,rezasafi/spark,ueshin/apache-spark,vinodkc/spark,WeichenXu123/spark,shaneknapp/spark,apache/spark,srowen/spark,zero323/spark,darionyaphet/spark,srowen/spark,Aegeaner/spark,kiszk/spark,shuangshuangwang/spark,mdespriee/spark,apache/spark,cloud-fan/spark,mahak/spark,srowen/spark,caneGuy/spark,nchammas/spark,techaddict/spark,facaiy/spark,goldmedal/spark,LantaoJin/spark,Aegeaner/spark,vinodkc/spark,LantaoJin/spark,xuanyuanking/spark,rezasafi/spark,WeichenXu123/spark,shuangshuangwang/spark,wzhfy/spark,BryanCutler/spark,holdenk/spark,zero323/spark,chuckchen/spark,gengliangwang/spark,WindCanDie/spark,ptkool/spark,caneGuy/spark,gengliangwang/spark,zzcclp/spark,skonto/spark,icexelloss/spark,yanboliang/spark,facaiy/spark,darionyaphet/spark,darionyaphet/spark,BryanCutler/spark,mahak/spark,hvanhovell/spark,goldmedal/spark,apache/spark,kevinyu98/spark,wzhfy/spark,actuaryzhang/spark,cloud-fan/spark,maropu/spark,dongjoon-hyun/spark,rednaxelafx/apache-spark,wzhfy/spark,dbtsai/spark,milliman/spark,JoshRosen/spark,matthewfranglen/spark,zero323/spark,mahak/spark,actuaryzhang/spark,rezasafi/spark,hvanhovell/spark,ptkool/spark,apache/spark,hvanhovell/spark,hhbyyh/spark,shaneknapp/spark,kiszk/spark,kevinyu98/spark,ueshin/apache-spark,yanboliang/spark,icexelloss/spark,wangmiao1981/spark,cloud-fan/spark,hvanhovell/spark,kevinyu98/spark,darionyaphet/spark,taroplus/spark,witgo/spark,jiangxb1987/spark,cloud-fan/spark,kevinyu98/spark,pgandhi999/spark,witgo/spark,zero323/spark,facaiy/spark,HyukjinKwon/spark,wangmiao1981/spark,pgandhi999/spark,jkbradley/spark,shuangshuangwang/spark,zero323/spark,srowen/spark,BryanCutler/spark,Aegeaner/spark,mdespriee/spark,zuotingbing/spark,wangyum/spark,wangmiao1981/spark,cloud-fan/spark,nchammas/spark,WeichenXu123/spark,spark-test/spark,vinodkc/spark,BryanCutler/spark,bdrillard/spark,ptkool/spark,spark-test/spark,shaneknapp/spark,Aegeaner/spark,aosagie/spark,ConeyLiu/spark,actuaryzhang/spark,kevinyu98/spark,wangyum/spark,apache/spark,JoshRosen/spark,guoxiaolongzte/spark,darionyaphet/spark,maropu/spark,ueshin/apache-spark,zzcclp/spark,bdrillard/spark,rednaxelafx/apache-spark,zuotingbing/spark,zuotingbing/spark,BryanCutler/spark,dbtsai/spark,jkbradley/spark,zzcclp/spark,vinodkc/spark,techaddict/spark,milliman/spark,witgo/spark,WindCanDie/spark,mdespriee/spark,jiangxb1987/spark,witgo/spark,vinodkc/spark,techaddict/spark,kevinyu98/spark,bdrillard/spark,skonto/spark,shaneknapp/spark,spark-test/spark,wangyum/spark,mdespriee/spark,guoxiaolongzte/spark,guoxiaolongzte/spark,techaddict/spark,facaiy/spark,Aegeaner/spark,kevinyu98/spark,ptkool/spark,hhbyyh/spark,bdrillard/spark,zero323/spark,zzcclp/spark,bdrillard/spark,WindCanDie/spark,maropu/spark,milliman/spark,pgandhi999/spark,taroplus/spark,wangmiao1981/spark,dbtsai/spark,jkbradley/spark,aosagie/spark,zzcclp/spark,skonto/spark,mdespriee/spark,actuaryzhang/spark,zero323/spark,dbtsai/spark,JoshRosen/spark,hhbyyh/spark,jiangxb1987/spark,yanboliang/spark,ptkool/spark,chuckchen/spark,maropu/spark,highfei2011/spark,gengliangwang/spark,wzhfy/spark,rednaxelafx/apache-spark,shuangshuangwang/spark,kiszk/spark,HyukjinKwon/spark,ConeyLiu/spark,jkbradley/spark,caneGuy/spark,icexelloss/spark,WindCanDie/spark,JoshRosen/spark,ueshin/apache-spark,bdrillard/spark,LantaoJin/spark,srowen/spark,xuanyuanking/spark,gengliangwang/spark,Aegeaner/spark,wangmiao1981/spark,chuckchen/spark,icexelloss/spark,wangmiao1981/spark,mdespriee/spark,shuangshuangwang/spark,shaneknapp/spark,JoshRosen/spark,goldmedal/spark,yanboliang/spark,guoxiaolongzte/spark,zuotingbing/spark,nchammas/spark,gengliangwang/spark,dbtsai/spark,techaddict/spark,shuangshuangwang/spark,mahak/spark,LantaoJin/spark,rezasafi/spark,WindCanDie/spark,nchammas/spark,taroplus/spark,yanboliang/spark,hvanhovell/spark,vinodkc/spark,caneGuy/spark,WindCanDie/spark,icexelloss/spark,kiszk/spark,chuckchen/spark,taroplus/spark,xuanyuanking/spark,BryanCutler/spark,rednaxelafx/apache-spark,witgo/spark,kiszk/spark,jkbradley/spark,actuaryzhang/spark,zuotingbing/spark,pgandhi999/spark,pgandhi999/spark,milliman/spark,kiszk/spark,holdenk/spark,zzcclp/spark,wangmiao1981/spark,wangyum/spark,holdenk/spark,guoxiaolongzte/spark,Aegeaner/spark,aosagie/spark,shaneknapp/spark,highfei2011/spark,techaddict/spark,facaiy/spark,zzcclp/spark,rednaxelafx/apache-spark,LantaoJin/spark,nchammas/spark,rednaxelafx/apache-spark,zuotingbing/spark,highfei2011/spark,hhbyyh/spark,chuckchen/spark,wangyum/spark,goldmedal/spark,aosagie/spark,milliman/spark,holdenk/spark,skonto/spark,guoxiaolongzte/spark,dongjoon-hyun/spark,goldmedal/spark,wzhfy/spark,darionyaphet/spark,ueshin/apache-spark,gengliangwang/spark,WeichenXu123/spark,yanboliang/spark,skonto/spark,aosagie/spark,apache/spark,WeichenXu123/spark,hvanhovell/spark,maropu/spark,dongjoon-hyun/spark,zuotingbing/spark,xuanyuanking/spark,jiangxb1987/spark,wzhfy/spark,WeichenXu123/spark,taroplus/spark,HyukjinKwon/spark,ueshin/apache-spark,holdenk/spark,jkbradley/spark,xuanyuanking/spark,wzhfy/spark,HyukjinKwon/spark,wangyum/spark,darionyaphet/spark,caneGuy/spark,facaiy/spark,taroplus/spark,ueshin/apache-spark,hvanhovell/spark,rezasafi/spark,dongjoon-hyun/spark,jiangxb1987/spark,dbtsai/spark,ConeyLiu/spark,guoxiaolongzte/spark,BryanCutler/spark,icexelloss/spark,maropu/spark,highfei2011/spark,taroplus/spark,aosagie/spark,goldmedal/spark,ptkool/spark,kiszk/spark,highfei2011/spark,xuanyuanking/spark,gengliangwang/spark,xuanyuanking/spark,highfei2011/spark,holdenk/spark,cloud-fan/spark,icexelloss/spark,vinodkc/spark,ConeyLiu/spark,LantaoJin/spark,goldmedal/spark,aosagie/spark,mahak/spark,ConeyLiu/spark,mahak/spark,bdrillard/spark,jkbradley/spark,rezasafi/spark,pgandhi999/spark,shaneknapp/spark,wangyum/spark,actuaryzhang/spark,caneGuy/spark