[SPARK-3355][Core]: Allow running maven tests in run-tests
[SPARK-3355][Core]: Allow running maven tests in run-tests

Added an AMPLAB_JENKINS_BUILD_TOOL env. variable to allow differentiation between maven and sbt build / test suites. The only issue I found with this is that, when running maven builds I wasn't able to get individual package tests running without running a `mvn install` first. Not sure what Jenkins is doing wrt its env., but figured its much better to just test everything than install packages in the "~/.m2/" directory and only test individual items, esp. if this is predominantly for the Jenkins build. Thoughts / comments would be great!

Author: Brennon York <brennon.york@capitalone.com>

Closes #4734 from brennonyork/SPARK-3355 and squashes the following commits:

c813d32 [Brennon York] changed mvn call from 'clean compile
616ce30 [Brennon York] fixed merge conflicts
3540de9 [Brennon York] added an AMPLAB_JENKINS_BUILD_TOOL env. variable to allow differentiation between maven and sbt build / test suites

srowen/spark,ddna1021/spark,pgandhi999/spark,techaddict/spark,UndeadBaneGitHub/spark,goldmedal/spark,wangyum/spark,hvanhovell/spark,huang1900/spark,map222/spark,wangmiao1981/spark,setjet/spark,publicRoman/spark,xflin/spark,darionyaphet/spark,michalsenkyr/spark,jkbradley/spark,zzcclp/spark,aray/spark,sahilTakiar/spark,setjet/spark,wzhfy/spark,hhbyyh/spark,dbtsai/spark,nlalevee/spark,UndeadBaneGitHub/spark,JerryLead/spark,aokolnychyi/spark,yanboliang/spark,xflin/spark,poffuomo/spark,alunarbeach/spark,hvanhovell/spark,xuanyuanking/spark,SnappyDataInc/spark,haowu80s/spark,JoshRosen/spark,andrewor14/iolap,milliman/spark,tengpeng/spark,kiszk/spark,BryanCutler/spark,ptkool/spark,dhruve/spark,ptkool/spark,jkbradley/spark,pgandhi999/spark,ptkool/spark,taroplus/spark,tejasapatil/spark,rednaxelafx/apache-spark,dongjoon-hyun/spark,stanzhai/spark,ericvandenbergfb/spark,gengliangwang/spark,tengpeng/spark,esi-mineset/spark,someorz/spark,michalsenkyr/spark,rekhajoshm/spark,rezasafi/spark,kissmetrics/spark,taroplus/spark,JoshRosen/spark,Panos-Bletsos/spark-cost-model-optimizer,nilsgrabbert/spark,esi-mineset/spark,nilsgrabbert/spark,dhruve/spark,publicRoman/spark,mdespriee/spark,ericvandenbergfb/spark,cin/spark,goldmedal/spark,brad-kaiser/spark,szhem/spark,michalsenkyr/spark,Aegeaner/spark,tengpeng/spark,tejasapatil/spark,panfengfeng/spark,poffuomo/spark,zero323/spark,shuangshuangwang/spark,jianran/spark,dbtsai/spark,patrick-nicholson/spark,ConeyLiu/spark,mahak/spark,haowu80s/spark,mike0sv/spark,stanzhai/spark,brad-kaiser/spark,joseph-torres/spark,jiangxb1987/spark,dhruve/spark,eyalfa/spark,yanboliang/spark,ptkool/spark,apache-spark-on-k8s/spark,zuotingbing/spark,ahnqirage/spark,mdespriee/spark,shaneknapp/spark,zero323/spark,skonto/spark,alunarbeach/spark,dbtsai/spark,rekhajoshm/spark,xflin/spark,JerryLead/spark,metamx/spark,guoxiaolongzte/spark,apache-spark-on-k8s/spark,loneknightpy/spark,UndeadBaneGitHub/spark,andrewor14/spark,metamx/spark,apache/spark,icexelloss/spark,milliman/spark,nilsgrabbert/spark,SnappyDataInc/spark,gengliangwang/spark,loneknightpy/spark,kissmetrics/spark,byakuinss/spark,someorz/spark,ibm-research-ireland/sparkoscope,Panos-Bletsos/spark-cost-model-optimizer,holdenk/spark,lvdongr/spark,rednaxelafx/apache-spark,lxsmnv/spark,vinodkc/spark,jiangxb1987/spark,nlalevee/spark,jrshust/spark,SnappyDataInc/spark,sachintyagi22/spark,rekhajoshm/spark,jkbradley/spark,witgo/spark,MLnick/spark,adrian-ionescu/apache-spark,bOOm-X/spark,shaneknapp/spark,mike0sv/spark,nchammas/spark,taroplus/spark,publicRoman/spark,aosagie/spark,wangmiao1981/spark,dhruve/spark,mike0sv/spark,patrick-nicholson/spark,liutang123/spark,guoxiaolongzte/spark,apache/spark,metamx/spark,dhruve/spark,highfei2011/spark,bOOm-X/spark,alunarbeach/spark,panfengfeng/spark,apache/spark,ibm-research-ireland/sparkoscope,ueshin/apache-spark,lxsmnv/spark,gioenn/xSpark,icexelloss/spark,zzcclp/spark,michalsenkyr/spark,MLnick/spark,techaddict/spark,markhamstra/spark,liyichao/spark,dhruve/spark,tejasapatil/spark,BryanCutler/spark,apache/spark,saltstar/spark,lvdongr/spark,xflin/spark,facaiy/spark,sachintyagi22/spark,setjet/spark,jianran/spark,rednaxelafx/apache-spark,ddna1021/spark,zhouyejoe/spark,lxsmnv/spark,patrick-nicholson/spark,vinodkc/spark,facaiy/spark,aray/spark,SHASHANKB/spark,spark-test/spark,eyalfa/spark,dongjoon-hyun/spark,andrewor14/iolap,chuckchen/spark,spark-test/spark,shaneknapp/spark,maropu/spark,jiangxb1987/spark,zuotingbing/spark,WeichenXu123/spark,JerryLead/spark,big-pegasus/spark,mdespriee/spark,esi-mineset/spark,narahari92/spark,pronix/spark,sachintyagi22/spark,rednaxelafx/apache-spark,rezasafi/spark,huang1900/spark,bdrillard/spark,tengpeng/spark,bOOm-X/spark,panfengfeng/spark,kimoonkim/spark,stanzhai/spark,bdrillard/spark,ddna1021/spark,mdespriee/spark,zzcclp/spark,akopich/spark,patrick-nicholson/spark,minixalpha/spark,skonto/spark,minixalpha/spark,map222/spark,shuangshuangwang/spark,lvdongr/spark,saturday-shi/spark,WindCanDie/spark,rezasafi/spark,sachintyagi22/spark,saturday-shi/spark,JoshRosen/spark,markhamstra/spark,wangmiao1981/spark,haowu80s/spark,rekhajoshm/spark,sahilTakiar/spark,ptkool/spark,map222/spark,jkbradley/spark,aosagie/spark,caneGuy/spark,andrewor14/spark,esi-mineset/spark,Aegeaner/spark,WeichenXu123/spark,poffuomo/spark,apache/spark,shaneknapp/spark,alunarbeach/spark,techaddict/spark,darionyaphet/spark,ericvandenbergfb/spark,dongjoon-hyun/spark,kimoonkim/spark,akopich/spark,someorz/spark,JoshRosen/spark,byakuinss/spark,UndeadBaneGitHub/spark,narahari92/spark,zero323/spark,huang1900/spark,jlopezmalla/spark,ConeyLiu/spark,lxsmnv/spark,joseph-torres/spark,holdenk/spark,bOOm-X/spark,eyalfa/spark,huang1900/spark,stanzhai/spark,narahari92/spark,hvanhovell/spark,jlopezmalla/spark,saturday-shi/spark,SHASHANKB/spark,minixalpha/spark,zhouyejoe/spark,brad-kaiser/spark,liyichao/spark,shubhamchopra/spark,nlalevee/spark,akopich/spark,bdrillard/spark,HyukjinKwon/spark,pronix/spark,shaneknapp/spark,SnappyDataInc/spark,JerryLead/spark,brad-kaiser/spark,aray/spark,janewangfb/spark,darionyaphet/spark,markhamstra/spark,maropu/spark,ioana-delaney/spark,WindCanDie/spark,nchammas/spark,kimoonkim/spark,zhouyejoe/spark,wangyum/spark,szhem/spark,rezasafi/spark,byakuinss/spark,BryanCutler/spark,mike0sv/spark,ajaysaini725/spark,ajaysaini725/spark,ioana-delaney/spark,WeichenXu123/spark,nlalevee/spark,LantaoJin/spark,lxsmnv/spark,pronix/spark,icexelloss/spark,techaddict/spark,spark-test/spark,shuangshuangwang/spark,actuaryzhang/spark,metamx/spark,patrick-nicholson/spark,zzcclp/spark,apache-spark-on-k8s/spark,LantaoJin/spark,rednaxelafx/apache-spark,rikima/spark,mzl9039/spark,lvdongr/spark,maropu/spark,kiszk/spark,kissmetrics/spark,pgandhi999/spark,BryanCutler/spark,map222/spark,HyukjinKwon/spark,tejasapatil/spark,esi-mineset/spark,hvanhovell/spark,vinodkc/spark,xuanyuanking/spark,guoxiaolongzte/spark,WeichenXu123/spark,highfei2011/spark,xuanyuanking/spark,kevinyu98/spark,ron8hu/spark,joseph-torres/spark,icexelloss/spark,skonto/spark,kissmetrics/spark,publicRoman/spark,facaiy/spark,bdrillard/spark,poffuomo/spark,MLnick/spark,nilsgrabbert/spark,xflin/spark,setjet/spark,liutang123/spark,stanzhai/spark,ron8hu/spark,JoshRosen/spark,gioenn/xSpark,szhem/spark,witgo/spark,matthewfranglen/spark,huang1900/spark,milliman/spark,aokolnychyi/spark,dbtsai/spark,kissmetrics/spark,chuckchen/spark,wzhfy/spark,tengpeng/spark,jrshust/spark,techaddict/spark,MLnick/spark,andrewor14/iolap,actuaryzhang/spark,zuotingbing/spark,cloud-fan/spark,icexelloss/spark,dhruve/spark,hhbyyh/spark,hhbyyh/spark,guoxiaolongzte/spark,ajaysaini725/spark,kiszk/spark,ConeyLiu/spark,xuanyuanking/spark,goldmedal/spark,yanboliang/spark,jrshust/spark,liyichao/spark,jiangxb1987/spark,rikima/spark,cin/spark,liutang123/spark,1haodian/spark,bdrillard/spark,zhouyejoe/spark,pgandhi999/spark,darionyaphet/spark,actuaryzhang/spark,tejasapatil/spark,sachintyagi22/spark,nchammas/spark,shubhamchopra/spark,nchammas/spark,maropu/spark,kevinyu98/spark,someorz/spark,poffuomo/spark,metamx/spark,kevinyu98/spark,skonto/spark,ioana-delaney/spark,witgo/spark,holdenk/spark,ron8hu/spark,chuckchen/spark,esi-mineset/spark,nchammas/spark,janewangfb/spark,lvdongr/spark,dbtsai/spark,milliman/spark,bravo-zhang/spark,Aegeaner/spark,pgandhi999/spark,spark-test/spark,rekhajoshm/spark,hvanhovell/spark,gioenn/xSpark,gioenn/xSpark,michalsenkyr/spark,tejasapatil/spark,metamx/spark,bravo-zhang/spark,szhem/spark,caneGuy/spark,rednaxelafx/apache-spark,markhamstra/spark,haowu80s/spark,witgo/spark,lvdongr/spark,ConeyLiu/spark,joseph-torres/spark,1haodian/spark,cin/spark,BryanCutler/spark,wangmiao1981/spark,jiangxb1987/spark,Panos-Bletsos/spark-cost-model-optimizer,lxsmnv/spark,SnappyDataInc/spark,caneGuy/spark,debugger87/spark,zero323/spark,holdenk/spark,yanboliang/spark,SHASHANKB/spark,adrian-ionescu/apache-spark,zzcclp/spark,caneGuy/spark,eyalfa/spark,holdenk/spark,milliman/spark,zuotingbing/spark,mike0sv/spark,WindCanDie/spark,setjet/spark,rekhajoshm/spark,srowen/spark,big-pegasus/spark,jiangxb1987/spark,jlopezmalla/spark,icexelloss/spark,rezasafi/spark,kevinyu98/spark,rikima/spark,apache-spark-on-k8s/spark,jianran/spark,HyukjinKwon/spark,byakuinss/spark,maropu/spark,shaneknapp/spark,ddna1021/spark,wangmiao1981/spark,mike0sv/spark,kiszk/spark,mdespriee/spark,cin/spark,zhouyejoe/spark,ajaysaini725/spark,ueshin/apache-spark,srowen/spark,liutang123/spark,haowu80s/spark,debugger87/spark,szhem/spark,mahak/spark,mahak/spark,ptkool/spark,sahilTakiar/spark,andrewor14/iolap,someorz/spark,ddna1021/spark,goldmedal/spark,dongjoon-hyun/spark,cloud-fan/spark,HyukjinKwon/spark,MLnick/spark,WindCanDie/spark,holdenk/spark,sachintyagi22/spark,zero323/spark,witgo/spark,SHASHANKB/spark,apache-spark-on-k8s/spark,loneknightpy/spark,byakuinss/spark,ahnqirage/spark,kiszk/spark,huang1900/spark,pgandhi999/spark,hhbyyh/spark,liutang123/spark,ptkool/spark,highfei2011/spark,kissmetrics/spark,kevinyu98/spark,big-pegasus/spark,witgo/spark,wangmiao1981/spark,xflin/spark,jiangxb1987/spark,adrian-ionescu/apache-spark,szhem/spark,saltstar/spark,gengliangwang/spark,adrian-ionescu/apache-spark,JoshRosen/spark,gioenn/xSpark,sureshthalamati/spark,apache/spark,kimoonkim/spark,facaiy/spark,brad-kaiser/spark,liyichao/spark,narahari92/spark,mahak/spark,kiszk/spark,ahnqirage/spark,hhbyyh/spark,kimoonkim/spark,jkbradley/spark,debugger87/spark,guoxiaolongzte/spark,cloud-fan/spark,panfengfeng/spark,brad-kaiser/spark,shubhamchopra/spark,nlalevee/spark,joseph-torres/spark,wangyum/spark,cin/spark,andrewor14/spark,UndeadBaneGitHub/spark,andrewor14/iolap,loneknightpy/spark,goldmedal/spark,shuangshuangwang/spark,cin/spark,liutang123/spark,wzhfy/spark,Panos-Bletsos/spark-cost-model-optimizer,tengpeng/spark,actuaryzhang/spark,ioana-delaney/spark,yanboliang/spark,SHASHANKB/spark,witgo/spark,ueshin/apache-spark,mzl9039/spark,milliman/spark,bOOm-X/spark,gengliangwang/spark,poffuomo/spark,MLnick/spark,janewangfb/spark,lvdongr/spark,alunarbeach/spark,sahilTakiar/spark,jlopezmalla/spark,jlopezmalla/spark,cloud-fan/spark,apache-spark-on-k8s/spark,Panos-Bletsos/spark-cost-model-optimizer,pgandhi999/spark,debugger87/spark,ddna1021/spark,zero323/spark,patrick-nicholson/spark,zuotingbing/spark,sureshthalamati/spark,skonto/spark,gengliangwang/spark,apache/spark,bOOm-X/spark,tengpeng/spark,pronix/spark,1haodian/spark,panfengfeng/spark,bdrillard/spark,spark-test/spark,panfengfeng/spark,aosagie/spark,BryanCutler/spark,highfei2011/spark,sahilTakiar/spark,janewangfb/spark,SnappyDataInc/spark,saturday-shi/spark,kimoonkim/spark,bravo-zhang/spark,wangyum/spark,mzl9039/spark,caneGuy/spark,mahak/spark,vinodkc/spark,jkbradley/spark,andrewor14/iolap,minixalpha/spark,shubhamchopra/spark,aokolnychyi/spark,facaiy/spark,dongjoon-hyun/spark,byakuinss/spark,ajaysaini725/spark,markhamstra/spark,zuotingbing/spark,publicRoman/spark,someorz/spark,ConeyLiu/spark,highfei2011/spark,andrewor14/iolap,joseph-torres/spark,aokolnychyi/spark,ericvandenbergfb/spark,adrian-ionescu/apache-spark,Aegeaner/spark,andrewor14/spark,ron8hu/spark,liyichao/spark,ahnqirage/spark,ioana-delaney/spark,icexelloss/spark,shubhamchopra/spark,maropu/spark,taroplus/spark,jianran/spark,wangyum/spark,andrewor14/spark,bdrillard/spark,bravo-zhang/spark,publicRoman/spark,kiszk/spark,gioenn/xSpark,highfei2011/spark,eyalfa/spark,srowen/spark,stanzhai/spark,rekhajoshm/spark,kevinyu98/spark,mzl9039/spark,LantaoJin/spark,loneknightpy/spark,saltstar/spark,Aegeaner/spark,nilsgrabbert/spark,szhem/spark,UndeadBaneGitHub/spark,jrshust/spark,kissmetrics/spark,wzhfy/spark,huang1900/spark,mzl9039/spark,nilsgrabbert/spark,skonto/spark,LantaoJin/spark,taroplus/spark,wzhfy/spark,techaddict/spark,michalsenkyr/spark,Panos-Bletsos/spark-cost-model-optimizer,narahari92/spark,ibm-research-ireland/sparkoscope,saltstar/spark,aray/spark,metamx/spark,eyalfa/spark,chuckchen/spark,ajaysaini725/spark,dbtsai/spark,someorz/spark,Aegeaner/spark,ericvandenbergfb/spark,actuaryzhang/spark,ahnqirage/spark,mahak/spark,hvanhovell/spark,aosagie/spark,bOOm-X/spark,ConeyLiu/spark,wangyum/spark,pronix/spark,mdespriee/spark,vinodkc/spark,nlalevee/spark,sureshthalamati/spark,brad-kaiser/spark,xflin/spark,cloud-fan/spark,akopich/spark,loneknightpy/spark,JerryLead/spark,rikima/spark,zzcclp/spark,JerryLead/spark,srowen/spark,actuaryzhang/spark,BryanCutler/spark,big-pegasus/spark,cin/spark,map222/spark,dongjoon-hyun/spark,HyukjinKwon/spark,jrshust/spark,JerryLead/spark,zhouyejoe/spark,guoxiaolongzte/spark,big-pegasus/spark,saturday-shi/spark,WindCanDie/spark,andrewor14/spark,mike0sv/spark,LantaoJin/spark,narahari92/spark,map222/spark,janewangfb/spark,nlalevee/spark,michalsenkyr/spark,nilsgrabbert/spark,guoxiaolongzte/spark,LantaoJin/spark,cloud-fan/spark,1haodian/spark,andrewor14/spark,ron8hu/spark,1haodian/spark,chuckchen/spark,debugger87/spark,chuckchen/spark,ioana-delaney/spark,highfei2011/spark,UndeadBaneGitHub/spark,jianran/spark,aokolnychyi/spark,SHASHANKB/spark,yanboliang/spark,jlopezmalla/spark,shaneknapp/spark,stanzhai/spark,ueshin/apache-spark,apache-spark-on-k8s/spark,shuangshuangwang/spark,debugger87/spark,actuaryzhang/spark,goldmedal/spark,aray/spark,taroplus/spark,esi-mineset/spark,maropu/spark,setjet/spark,jlopezmalla/spark,panfengfeng/spark,srowen/spark,minixalpha/spark,ddna1021/spark,wangmiao1981/spark,rikima/spark,sahilTakiar/spark,wzhfy/spark,MLnick/spark,hhbyyh/spark,saltstar/spark,bravo-zhang/spark,saltstar/spark,liyichao/spark,aosagie/spark,liyichao/spark,pronix/spark,ueshin/apache-spark,mahak/spark,goldmedal/spark,sureshthalamati/spark,alunarbeach/spark,debugger87/spark,gioenn/xSpark,darionyaphet/spark,1haodian/spark,skonto/spark,jkbradley/spark,caneGuy/spark,1haodian/spark,ConeyLiu/spark,byakuinss/spark,minixalpha/spark,akopich/spark,SHASHANKB/spark,sahilTakiar/spark,lxsmnv/spark,facaiy/spark,aosagie/spark,SnappyDataInc/spark,ericvandenbergfb/spark,rednaxelafx/apache-spark,spark-test/spark,aokolnychyi/spark,akopich/spark,sureshthalamati/spark,mzl9039/spark,loneknightpy/spark,saltstar/spark,dongjoon-hyun/spark,darionyaphet/spark,publicRoman/spark,kevinyu98/spark,ahnqirage/spark,sureshthalamati/spark,xuanyuanking/spark,wangyum/spark,spark-test/spark,aokolnychyi/spark,haowu80s/spark,rikima/spark,ibm-research-ireland/sparkoscope,liutang123/spark,srowen/spark,vinodkc/spark,Panos-Bletsos/spark-cost-model-optimizer,adrian-ionescu/apache-spark,markhamstra/spark,Aegeaner/spark,hhbyyh/spark,jianran/spark,gengliangwang/spark,big-pegasus/spark,caneGuy/spark,JoshRosen/spark,milliman/spark,ajaysaini725/spark,joseph-torres/spark,ericvandenbergfb/spark,markhamstra/spark,akopich/spark,gengliangwang/spark,techaddict/spark,rezasafi/spark,setjet/spark,taroplus/spark,shuangshuangwang/spark,narahari92/spark,aosagie/spark,big-pegasus/spark,ibm-research-ireland/sparkoscope,patrick-nicholson/spark,jrshust/spark,ibm-research-ireland/sparkoscope,minixalpha/spark,holdenk/spark,bravo-zhang/spark,ioana-delaney/spark,map222/spark,jrshust/spark,WeichenXu123/spark,janewangfb/spark,zero323/spark,kimoonkim/spark,WeichenXu123/spark,zhouyejoe/spark,dbtsai/spark,saturday-shi/spark,mzl9039/spark,janewangfb/spark,aray/spark,shuangshuangwang/spark,ron8hu/spark,tejasapatil/spark,ibm-research-ireland/sparkoscope,rezasafi/spark,darionyaphet/spark,ueshin/apache-spark,zuotingbing/spark,vinodkc/spark,HyukjinKwon/spark,xuanyuanking/spark,jianran/spark,ueshin/apache-spark,yanboliang/spark,facaiy/spark,sachintyagi22/spark,shubhamchopra/spark,sureshthalamati/spark,alunarbeach/spark,adrian-ionescu/apache-spark,bravo-zhang/spark,eyalfa/spark,WeichenXu123/spark,rikima/spark,shubhamchopra/spark,hvanhovell/spark,xuanyuanking/spark,chuckchen/spark,ahnqirage/spark,poffuomo/spark,WindCanDie/spark,LantaoJin/spark,aray/spark,cloud-fan/spark,zzcclp/spark,WindCanDie/spark,HyukjinKwon/spark,nchammas/spark,saturday-shi/spark,mdespriee/spark,wzhfy/spark,ron8hu/spark,nchammas/spark