[SPARK-9974] [BUILD] [SQL] Makes sure com.twitter:parquet-hadoop-bundle:1.6.0 is in SBT assembly jar
[SPARK-9974] [BUILD] [SQL] Makes sure com.twitter:parquet-hadoop-bundle:1.6.0 is in SBT assembly jar

PR #7967 enables Spark SQL to persist Parquet tables in Hive compatible format when possible. One of the consequence is that, we have to set input/output classes to `MapredParquetInputFormat`/`MapredParquetOutputFormat`, which rely on com.twitter:parquet-hadoop:1.6.0 bundled with Hive 1.2.1.

When loading such a table in Spark SQL, `o.a.h.h.ql.metadata.Table` first loads these input/output format classes, and thus classes in com.twitter:parquet-hadoop:1.6.0.  However, the scope of this dependency is defined as "runtime", and is not packaged into Spark assembly jar.  This results in a `ClassNotFoundException`.

This issue can be worked around by asking users to add parquet-hadoop 1.6.0 via the `--driver-class-path` option.  However, considering Maven build is immune to this problem, I feel it can be confusing and inconvenient for users.

So this PR fixes this issue by changing scope of parquet-hadoop 1.6.0 to "compile".

Author: Cheng Lian <lian@databricks.com>

Closes #8198 from liancheng/spark-9974/bundle-parquet-1.6.0.

JerryLead/spark,hhbyyh/spark,setjet/spark,apache/spark,lxsmnv/spark,lxsmnv/spark,huang1900/spark,ConeyLiu/spark,hvanhovell/spark,tengpeng/spark,publicRoman/spark,sureshthalamati/spark,sahilTakiar/spark,highfei2011/spark,techaddict/spark,mdespriee/spark,Panos-Bletsos/spark-cost-model-optimizer,bdrillard/spark,map222/spark,stanzhai/spark,gengliangwang/spark,rekhajoshm/spark,szhem/spark,WeichenXu123/spark,WindCanDie/spark,xflin/spark,skonto/spark,ahnqirage/spark,tengpeng/spark,nilsgrabbert/spark,minixalpha/spark,actuaryzhang/spark,milliman/spark,joseph-torres/spark,adrian-ionescu/apache-spark,MLnick/spark,yanboliang/spark,aosagie/spark,sachintyagi22/spark,jkbradley/spark,xflin/spark,zuotingbing/spark,mahak/spark,mdespriee/spark,ptkool/spark,liutang123/spark,facaiy/spark,kiszk/spark,wangyum/spark,srowen/spark,icexelloss/spark,wangmiao1981/spark,WeichenXu123/spark,chuckchen/spark,liutang123/spark,tengpeng/spark,ibm-research-ireland/sparkoscope,jlopezmalla/spark,wangmiao1981/spark,adrian-ionescu/apache-spark,zhouyejoe/spark,darionyaphet/spark,highfei2011/spark,ptkool/spark,techaddict/spark,BryanCutler/spark,shubhamchopra/spark,loneknightpy/spark,ddna1021/spark,dbtsai/spark,bdrillard/spark,liyichao/spark,HyukjinKwon/spark,nilsgrabbert/spark,zhouyejoe/spark,kissmetrics/spark,shaneknapp/spark,narahari92/spark,kiszk/spark,ioana-delaney/spark,caneGuy/spark,lvdongr/spark,LantaoJin/spark,jrshust/spark,sachintyagi22/spark,ericvandenbergfb/spark,szhem/spark,mzl9039/spark,apache-spark-on-k8s/spark,cloud-fan/spark,alunarbeach/spark,kevinyu98/spark,highfei2011/spark,panfengfeng/spark,jiangxb1987/spark,ron8hu/spark,wangmiao1981/spark,sachintyagi22/spark,ajaysaini725/spark,ron8hu/spark,caneGuy/spark,liutang123/spark,hhbyyh/spark,wzhfy/spark,hvanhovell/spark,BryanCutler/spark,ueshin/apache-spark,minixalpha/spark,LantaoJin/spark,akopich/spark,xuanyuanking/spark,ron8hu/spark,tejasapatil/spark,narahari92/spark,jlopezmalla/spark,hvanhovell/spark,ddna1021/spark,UndeadBaneGitHub/spark,chuckchen/spark,wzhfy/spark,kevinyu98/spark,kimoonkim/spark,facaiy/spark,panfengfeng/spark,mdespriee/spark,big-pegasus/spark,debugger87/spark,jianran/spark,bdrillard/spark,shaneknapp/spark,Panos-Bletsos/spark-cost-model-optimizer,dhruve/spark,lvdongr/spark,ericvandenbergfb/spark,nchammas/spark,dhruve/spark,WeichenXu123/spark,stanzhai/spark,zzcclp/spark,xuanyuanking/spark,sahilTakiar/spark,saltstar/spark,mike0sv/spark,mdespriee/spark,ueshin/apache-spark,huang1900/spark,ddna1021/spark,publicRoman/spark,lvdongr/spark,mdespriee/spark,haowu80s/spark,facaiy/spark,cloud-fan/spark,sachintyagi22/spark,srowen/spark,adrian-ionescu/apache-spark,ioana-delaney/spark,JerryLead/spark,icexelloss/spark,szhem/spark,shubhamchopra/spark,akopich/spark,1haodian/spark,huang1900/spark,milliman/spark,gioenn/xSpark,holdenk/spark,HyukjinKwon/spark,gioenn/xSpark,icexelloss/spark,loneknightpy/spark,icexelloss/spark,bdrillard/spark,nchammas/spark,brad-kaiser/spark,ericvandenbergfb/spark,dhruve/spark,actuaryzhang/spark,markhamstra/spark,haowu80s/spark,wzhfy/spark,xflin/spark,minixalpha/spark,andrewor14/spark,janewangfb/spark,JerryLead/spark,actuaryzhang/spark,matthewfranglen/spark,zzcclp/spark,ueshin/apache-spark,zhouyejoe/spark,ibm-research-ireland/sparkoscope,michalsenkyr/spark,ron8hu/spark,lvdongr/spark,byakuinss/spark,vinodkc/spark,saltstar/spark,ioana-delaney/spark,chuckchen/spark,highfei2011/spark,esi-mineset/spark,nchammas/spark,andrewor14/spark,JerryLead/spark,mike0sv/spark,witgo/spark,zuotingbing/spark,publicRoman/spark,BryanCutler/spark,Aegeaner/spark,narahari92/spark,SnappyDataInc/spark,kissmetrics/spark,saturday-shi/spark,milliman/spark,SHASHANKB/spark,JoshRosen/spark,gengliangwang/spark,dbtsai/spark,xuanyuanking/spark,metamx/spark,aray/spark,JoshRosen/spark,lvdongr/spark,MLnick/spark,jiangxb1987/spark,loneknightpy/spark,darionyaphet/spark,brad-kaiser/spark,loneknightpy/spark,bOOm-X/spark,cin/spark,eyalfa/spark,dongjoon-hyun/spark,WeichenXu123/spark,LantaoJin/spark,liyichao/spark,sachintyagi22/spark,nchammas/spark,sachintyagi22/spark,byakuinss/spark,andrewor14/spark,saturday-shi/spark,JoshRosen/spark,rezasafi/spark,saturday-shi/spark,SnappyDataInc/spark,icexelloss/spark,gengliangwang/spark,nilsgrabbert/spark,aosagie/spark,hvanhovell/spark,dbtsai/spark,caneGuy/spark,apache/spark,someorz/spark,stanzhai/spark,srowen/spark,markhamstra/spark,HyukjinKwon/spark,zhouyejoe/spark,taroplus/spark,bOOm-X/spark,joseph-torres/spark,tejasapatil/spark,poffuomo/spark,kimoonkim/spark,JerryLead/spark,saltstar/spark,tengpeng/spark,guoxiaolongzte/spark,map222/spark,rekhajoshm/spark,poffuomo/spark,shubhamchopra/spark,zhouyejoe/spark,bOOm-X/spark,JerryLead/spark,pgandhi999/spark,sahilTakiar/spark,brad-kaiser/spark,tengpeng/spark,lxsmnv/spark,setjet/spark,liyichao/spark,vinodkc/spark,cin/spark,SHASHANKB/spark,wangyum/spark,alunarbeach/spark,rezasafi/spark,big-pegasus/spark,vinodkc/spark,wzhfy/spark,gengliangwang/spark,metamx/spark,hhbyyh/spark,nlalevee/spark,yanboliang/spark,rednaxelafx/apache-spark,ron8hu/spark,highfei2011/spark,ConeyLiu/spark,sahilTakiar/spark,mzl9039/spark,shuangshuangwang/spark,bOOm-X/spark,lxsmnv/spark,zuotingbing/spark,ConeyLiu/spark,zzcclp/spark,janewangfb/spark,markhamstra/spark,xflin/spark,rikima/spark,ConeyLiu/spark,techaddict/spark,cloud-fan/spark,saltstar/spark,ibm-research-ireland/sparkoscope,cloud-fan/spark,wangmiao1981/spark,byakuinss/spark,1haodian/spark,setjet/spark,cin/spark,shaneknapp/spark,ahnqirage/spark,MLnick/spark,joseph-torres/spark,zero323/spark,xuanyuanking/spark,eyalfa/spark,spark-test/spark,map222/spark,brad-kaiser/spark,aray/spark,rednaxelafx/apache-spark,chuckchen/spark,goldmedal/spark,publicRoman/spark,facaiy/spark,Aegeaner/spark,mike0sv/spark,alunarbeach/spark,facaiy/spark,Panos-Bletsos/spark-cost-model-optimizer,publicRoman/spark,andrewor14/spark,zero323/spark,bravo-zhang/spark,holdenk/spark,spark-test/spark,ueshin/apache-spark,skonto/spark,actuaryzhang/spark,spark-test/spark,skonto/spark,zzcclp/spark,ericvandenbergfb/spark,hvanhovell/spark,jiangxb1987/spark,hhbyyh/spark,highfei2011/spark,ibm-research-ireland/sparkoscope,taroplus/spark,nilsgrabbert/spark,dongjoon-hyun/spark,shubhamchopra/spark,stanzhai/spark,esi-mineset/spark,debugger87/spark,rednaxelafx/apache-spark,WeichenXu123/spark,poffuomo/spark,pgandhi999/spark,apache/spark,shuangshuangwang/spark,akopich/spark,SHASHANKB/spark,dhruve/spark,alunarbeach/spark,techaddict/spark,rikima/spark,witgo/spark,szhem/spark,witgo/spark,tejasapatil/spark,mahak/spark,zhouyejoe/spark,metamx/spark,ahnqirage/spark,dongjoon-hyun/spark,dhruve/spark,metamx/spark,joseph-torres/spark,mzl9039/spark,liyichao/spark,big-pegasus/spark,spark-test/spark,techaddict/spark,yanboliang/spark,MLnick/spark,caneGuy/spark,kiszk/spark,esi-mineset/spark,sureshthalamati/spark,huang1900/spark,aosagie/spark,mahak/spark,debugger87/spark,byakuinss/spark,byakuinss/spark,rikima/spark,kimoonkim/spark,maropu/spark,kiszk/spark,rednaxelafx/apache-spark,janewangfb/spark,markhamstra/spark,eyalfa/spark,MLnick/spark,Aegeaner/spark,mzl9039/spark,HyukjinKwon/spark,ueshin/apache-spark,spark-test/spark,UndeadBaneGitHub/spark,cloud-fan/spark,aosagie/spark,mahak/spark,SHASHANKB/spark,wangmiao1981/spark,UndeadBaneGitHub/spark,brad-kaiser/spark,taroplus/spark,stanzhai/spark,liutang123/spark,poffuomo/spark,kevinyu98/spark,LantaoJin/spark,mdespriee/spark,michalsenkyr/spark,1haodian/spark,ericvandenbergfb/spark,SHASHANKB/spark,zuotingbing/spark,esi-mineset/spark,ibm-research-ireland/sparkoscope,big-pegasus/spark,maropu/spark,WindCanDie/spark,ajaysaini725/spark,ajaysaini725/spark,mdespriee/spark,map222/spark,SnappyDataInc/spark,jianran/spark,actuaryzhang/spark,kiszk/spark,michalsenkyr/spark,apache/spark,kiszk/spark,dongjoon-hyun/spark,akopich/spark,darionyaphet/spark,taroplus/spark,markhamstra/spark,cin/spark,saturday-shi/spark,ahnqirage/spark,UndeadBaneGitHub/spark,tengpeng/spark,shaneknapp/spark,kimoonkim/spark,ron8hu/spark,dbtsai/spark,michalsenkyr/spark,BryanCutler/spark,shaneknapp/spark,zero323/spark,ptkool/spark,ddna1021/spark,minixalpha/spark,Panos-Bletsos/spark-cost-model-optimizer,akopich/spark,jkbradley/spark,holdenk/spark,Aegeaner/spark,setjet/spark,alunarbeach/spark,bOOm-X/spark,sureshthalamati/spark,minixalpha/spark,milliman/spark,Panos-Bletsos/spark-cost-model-optimizer,maropu/spark,hvanhovell/spark,joseph-torres/spark,ahnqirage/spark,zuotingbing/spark,rezasafi/spark,ptkool/spark,michalsenkyr/spark,apache-spark-on-k8s/spark,jkbradley/spark,holdenk/spark,liutang123/spark,huang1900/spark,pronix/spark,BryanCutler/spark,rikima/spark,apache-spark-on-k8s/spark,kimoonkim/spark,narahari92/spark,jianran/spark,cin/spark,ptkool/spark,guoxiaolongzte/spark,rezasafi/spark,goldmedal/spark,mike0sv/spark,taroplus/spark,wangyum/spark,lxsmnv/spark,janewangfb/spark,bravo-zhang/spark,darionyaphet/spark,wzhfy/spark,shubhamchopra/spark,HyukjinKwon/spark,sureshthalamati/spark,WindCanDie/spark,wangmiao1981/spark,apache/spark,SnappyDataInc/spark,nlalevee/spark,aokolnychyi/spark,bdrillard/spark,dhruve/spark,gioenn/xSpark,jlopezmalla/spark,ibm-research-ireland/sparkoscope,ron8hu/spark,haowu80s/spark,aray/spark,1haodian/spark,stanzhai/spark,gengliangwang/spark,ioana-delaney/spark,HyukjinKwon/spark,caneGuy/spark,kissmetrics/spark,SHASHANKB/spark,poffuomo/spark,narahari92/spark,ajaysaini725/spark,lvdongr/spark,jrshust/spark,caneGuy/spark,setjet/spark,setjet/spark,hhbyyh/spark,mike0sv/spark,srowen/spark,xuanyuanking/spark,ConeyLiu/spark,gioenn/xSpark,milliman/spark,kissmetrics/spark,aray/spark,ericvandenbergfb/spark,mahak/spark,actuaryzhang/spark,ddna1021/spark,apache-spark-on-k8s/spark,markhamstra/spark,panfengfeng/spark,UndeadBaneGitHub/spark,mzl9039/spark,bdrillard/spark,ajaysaini725/spark,LantaoJin/spark,goldmedal/spark,Panos-Bletsos/spark-cost-model-optimizer,someorz/spark,ConeyLiu/spark,LantaoJin/spark,maropu/spark,patrick-nicholson/spark,map222/spark,spark-test/spark,xflin/spark,SnappyDataInc/spark,liutang123/spark,tejasapatil/spark,someorz/spark,adrian-ionescu/apache-spark,bravo-zhang/spark,ahnqirage/spark,milliman/spark,ioana-delaney/spark,dbtsai/spark,vinodkc/spark,UndeadBaneGitHub/spark,rekhajoshm/spark,sahilTakiar/spark,caneGuy/spark,rekhajoshm/spark,rezasafi/spark,wzhfy/spark,ioana-delaney/spark,gioenn/xSpark,BryanCutler/spark,rezasafi/spark,chuckchen/spark,jlopezmalla/spark,nlalevee/spark,loneknightpy/spark,minixalpha/spark,tejasapatil/spark,bOOm-X/spark,saltstar/spark,highfei2011/spark,jlopezmalla/spark,bravo-zhang/spark,narahari92/spark,JoshRosen/spark,kimoonkim/spark,pronix/spark,publicRoman/spark,srowen/spark,guoxiaolongzte/spark,pgandhi999/spark,bravo-zhang/spark,brad-kaiser/spark,milliman/spark,pronix/spark,szhem/spark,jkbradley/spark,sureshthalamati/spark,shubhamchopra/spark,tejasapatil/spark,nilsgrabbert/spark,witgo/spark,setjet/spark,goldmedal/spark,bOOm-X/spark,MLnick/spark,aray/spark,rikima/spark,pronix/spark,saltstar/spark,dongjoon-hyun/spark,mzl9039/spark,xflin/spark,cloud-fan/spark,patrick-nicholson/spark,nilsgrabbert/spark,nchammas/spark,mahak/spark,Aegeaner/spark,liyichao/spark,aosagie/spark,UndeadBaneGitHub/spark,ueshin/apache-spark,dbtsai/spark,alunarbeach/spark,maropu/spark,zero323/spark,kissmetrics/spark,pronix/spark,MLnick/spark,publicRoman/spark,rikima/spark,icexelloss/spark,holdenk/spark,pronix/spark,someorz/spark,ajaysaini725/spark,liutang123/spark,yanboliang/spark,WindCanDie/spark,aokolnychyi/spark,bravo-zhang/spark,metamx/spark,kevinyu98/spark,icexelloss/spark,eyalfa/spark,panfengfeng/spark,pgandhi999/spark,1haodian/spark,zhouyejoe/spark,adrian-ionescu/apache-spark,minixalpha/spark,BryanCutler/spark,goldmedal/spark,ahnqirage/spark,1haodian/spark,someorz/spark,facaiy/spark,jianran/spark,lvdongr/spark,andrewor14/spark,wangyum/spark,janewangfb/spark,ddna1021/spark,kevinyu98/spark,patrick-nicholson/spark,jianran/spark,map222/spark,jkbradley/spark,shubhamchopra/spark,hhbyyh/spark,ptkool/spark,ddna1021/spark,zero323/spark,zzcclp/spark,bdrillard/spark,aokolnychyi/spark,pgandhi999/spark,jiangxb1987/spark,jiangxb1987/spark,nlalevee/spark,debugger87/spark,sureshthalamati/spark,1haodian/spark,nlalevee/spark,panfengfeng/spark,big-pegasus/spark,jkbradley/spark,debugger87/spark,loneknightpy/spark,cin/spark,shuangshuangwang/spark,vinodkc/spark,haowu80s/spark,ptkool/spark,zzcclp/spark,witgo/spark,eyalfa/spark,esi-mineset/spark,saturday-shi/spark,big-pegasus/spark,pgandhi999/spark,ioana-delaney/spark,poffuomo/spark,nchammas/spark,rikima/spark,chuckchen/spark,hhbyyh/spark,ConeyLiu/spark,esi-mineset/spark,kissmetrics/spark,szhem/spark,nlalevee/spark,shuangshuangwang/spark,ueshin/apache-spark,liyichao/spark,akopich/spark,wzhfy/spark,saltstar/spark,patrick-nicholson/spark,patrick-nicholson/spark,haowu80s/spark,janewangfb/spark,witgo/spark,mike0sv/spark,aokolnychyi/spark,nchammas/spark,lxsmnv/spark,vinodkc/spark,lxsmnv/spark,actuaryzhang/spark,liyichao/spark,WindCanDie/spark,WeichenXu123/spark,brad-kaiser/spark,WindCanDie/spark,markhamstra/spark,adrian-ionescu/apache-spark,zuotingbing/spark,shuangshuangwang/spark,gengliangwang/spark,xflin/spark,map222/spark,WeichenXu123/spark,andrewor14/spark,yanboliang/spark,yanboliang/spark,skonto/spark,nlalevee/spark,haowu80s/spark,holdenk/spark,janewangfb/spark,apache-spark-on-k8s/spark,cloud-fan/spark,gioenn/xSpark,bravo-zhang/spark,guoxiaolongzte/spark,rednaxelafx/apache-spark,zero323/spark,debugger87/spark,stanzhai/spark,aosagie/spark,apache-spark-on-k8s/spark,wangyum/spark,panfengfeng/spark,goldmedal/spark,skonto/spark,rekhajoshm/spark,metamx/spark,JoshRosen/spark,JerryLead/spark,tejasapatil/spark,sachintyagi22/spark,JoshRosen/spark,huang1900/spark,SnappyDataInc/spark,aosagie/spark,taroplus/spark,akopich/spark,JoshRosen/spark,nilsgrabbert/spark,big-pegasus/spark,wangyum/spark,jiangxb1987/spark,darionyaphet/spark,dongjoon-hyun/spark,shaneknapp/spark,jrshust/spark,aokolnychyi/spark,gengliangwang/spark,techaddict/spark,szhem/spark,eyalfa/spark,ericvandenbergfb/spark,ajaysaini725/spark,maropu/spark,aray/spark,sureshthalamati/spark,apache/spark,hvanhovell/spark,yanboliang/spark,someorz/spark,jianran/spark,kiszk/spark,SHASHANKB/spark,vinodkc/spark,darionyaphet/spark,kimoonkim/spark,rednaxelafx/apache-spark,jrshust/spark,jrshust/spark,jiangxb1987/spark,byakuinss/spark,spark-test/spark,cin/spark,mahak/spark,maropu/spark,jrshust/spark,poffuomo/spark,eyalfa/spark,sahilTakiar/spark,sahilTakiar/spark,michalsenkyr/spark,gioenn/xSpark,SnappyDataInc/spark,holdenk/spark,LantaoJin/spark,dongjoon-hyun/spark,rekhajoshm/spark,guoxiaolongzte/spark,rekhajoshm/spark,chuckchen/spark,zzcclp/spark,xuanyuanking/spark,zuotingbing/spark,saturday-shi/spark,srowen/spark,guoxiaolongzte/spark,xuanyuanking/spark,aray/spark,jkbradley/spark,jianran/spark,apache/spark,rezasafi/spark,wangyum/spark,tengpeng/spark,witgo/spark,jlopezmalla/spark,debugger87/spark,shaneknapp/spark,mzl9039/spark,zero323/spark,narahari92/spark,huang1900/spark,michalsenkyr/spark,Aegeaner/spark,andrewor14/spark,goldmedal/spark,skonto/spark,shuangshuangwang/spark,patrick-nicholson/spark,mike0sv/spark,joseph-torres/spark,dhruve/spark,aokolnychyi/spark,srowen/spark,kevinyu98/spark,joseph-torres/spark,alunarbeach/spark,HyukjinKwon/spark,shuangshuangwang/spark,jlopezmalla/spark,someorz/spark,kissmetrics/spark,skonto/spark,loneknightpy/spark,rednaxelafx/apache-spark,panfengfeng/spark,ibm-research-ireland/sparkoscope,kevinyu98/spark,Aegeaner/spark,esi-mineset/spark,saturday-shi/spark,techaddict/spark,wangmiao1981/spark,guoxiaolongzte/spark,facaiy/spark,jrshust/spark,darionyaphet/spark,taroplus/spark,apache-spark-on-k8s/spark,dbtsai/spark,WindCanDie/spark,patrick-nicholson/spark,metamx/spark,aokolnychyi/spark,pgandhi999/spark,byakuinss/spark,adrian-ionescu/apache-spark,Panos-Bletsos/spark-cost-model-optimizer