default task number misleading in several places
default task number misleading in several places

  private[streaming] def defaultPartitioner(numPartitions: Int = self.ssc.sc.defaultParallelism){
    new HashPartitioner(numPartitions)
  }

it represents that the default task number in Spark Streaming relies on the variable defaultParallelism in SparkContext, which is decided by the config property spark.default.parallelism

the property "spark.default.parallelism" refers to https://github.com/apache/spark/pull/389

Author: Chen Chao <crazyjvm@gmail.com>

Closes #766 from CrazyJvm/patch-7 and squashes the following commits:

0b7efba [Chen Chao] Update streaming-programming-guide.md
cc5b66c [Chen Chao] default task number misleading in several places

patrick-nicholson/spark,chuckchen/spark,rednaxelafx/apache-spark,shuangshuangwang/spark,someorz/spark,tengpeng/spark,LantaoJin/spark,cin/spark,aray/spark,stanzhai/spark,tengpeng/spark,ddna1021/spark,sureshthalamati/spark,poffuomo/spark,ptkool/spark,bravo-zhang/spark,ptkool/spark,rikima/spark,sachintyagi22/spark,zhouyejoe/spark,highfei2011/spark,akopich/spark,apache-spark-on-k8s/spark,zuotingbing/spark,1haodian/spark,wangyum/spark,UndeadBaneGitHub/spark,mdespriee/spark,dbtsai/spark,darionyaphet/spark,cin/spark,ioana-delaney/spark,apache-spark-on-k8s/spark,janewangfb/spark,actuaryzhang/spark,ajaysaini725/spark,zhouyejoe/spark,zuotingbing/spark,narahari92/spark,darionyaphet/spark,ConeyLiu/spark,adrian-ionescu/apache-spark,xflin/spark,patrick-nicholson/spark,xuanyuanking/spark,lvdongr/spark,Panos-Bletsos/spark-cost-model-optimizer,setjet/spark,pronix/spark,bravo-zhang/spark,LantaoJin/spark,pronix/spark,ibm-research-ireland/sparkoscope,rekhajoshm/spark,narahari92/spark,eyalfa/spark,zero323/spark,WindCanDie/spark,map222/spark,yanboliang/spark,apache/spark,eyalfa/spark,jkbradley/spark,spark-test/spark,andrewor14/spark,maropu/spark,adrian-ionescu/apache-spark,kimoonkim/spark,liyichao/spark,joseph-torres/spark,wzhfy/spark,actuaryzhang/spark,kiszk/spark,Aegeaner/spark,wangyum/spark,esi-mineset/spark,sureshthalamati/spark,tengpeng/spark,janewangfb/spark,jrshust/spark,tengpeng/spark,akopich/spark,joseph-torres/spark,WeichenXu123/spark,hhbyyh/spark,mdespriee/spark,caneGuy/spark,jianran/spark,zhouyejoe/spark,akopich/spark,setjet/spark,hvanhovell/spark,tejasapatil/spark,hhbyyh/spark,HyukjinKwon/spark,huang1900/spark,HyukjinKwon/spark,patrick-nicholson/spark,map222/spark,1haodian/spark,maropu/spark,sureshthalamati/spark,sahilTakiar/spark,ioana-delaney/spark,byakuinss/spark,debugger87/spark,publicRoman/spark,mike0sv/spark,bdrillard/spark,saltstar/spark,skonto/spark,wangmiao1981/spark,JerryLead/spark,darionyaphet/spark,andrewor14/iolap,bOOm-X/spark,nilsgrabbert/spark,UndeadBaneGitHub/spark,huang1900/spark,zzcclp/spark,witgo/spark,sachintyagi22/spark,dongjoon-hyun/spark,liutang123/spark,big-pegasus/spark,WindCanDie/spark,sachintyagi22/spark,poffuomo/spark,guoxiaolongzte/spark,kevinyu98/spark,jlopezmalla/spark,BryanCutler/spark,maropu/spark,hhbyyh/spark,loneknightpy/spark,bdrillard/spark,joseph-torres/spark,eyalfa/spark,bravo-zhang/spark,xuanyuanking/spark,ron8hu/spark,wzhfy/spark,vinodkc/spark,lxsmnv/spark,milliman/spark,icexelloss/spark,rekhajoshm/spark,milliman/spark,bravo-zhang/spark,nlalevee/spark,stanzhai/spark,minixalpha/spark,alunarbeach/spark,someorz/spark,techaddict/spark,techaddict/spark,publicRoman/spark,lvdongr/spark,mdespriee/spark,nlalevee/spark,jlopezmalla/spark,SnappyDataInc/spark,rednaxelafx/apache-spark,techaddict/spark,janewangfb/spark,liutang123/spark,icexelloss/spark,saltstar/spark,skonto/spark,cloud-fan/spark,dbtsai/spark,shaneknapp/spark,kissmetrics/spark,yanboliang/spark,big-pegasus/spark,kevinyu98/spark,mahak/spark,ericvandenbergfb/spark,debugger87/spark,rikima/spark,kissmetrics/spark,kissmetrics/spark,aosagie/spark,someorz/spark,byakuinss/spark,MLnick/spark,pgandhi999/spark,saturday-shi/spark,aokolnychyi/spark,highfei2011/spark,mzl9039/spark,Panos-Bletsos/spark-cost-model-optimizer,ahnqirage/spark,andrewor14/iolap,kiszk/spark,janewangfb/spark,witgo/spark,loneknightpy/spark,mzl9039/spark,xflin/spark,ptkool/spark,UndeadBaneGitHub/spark,ueshin/apache-spark,WeichenXu123/spark,sureshthalamati/spark,Panos-Bletsos/spark-cost-model-optimizer,huang1900/spark,xflin/spark,liutang123/spark,yanboliang/spark,shubhamchopra/spark,debugger87/spark,jrshust/spark,dongjoon-hyun/spark,goldmedal/spark,metamx/spark,milliman/spark,icexelloss/spark,ioana-delaney/spark,big-pegasus/spark,brad-kaiser/spark,tejasapatil/spark,dhruve/spark,dhruve/spark,ioana-delaney/spark,ericvandenbergfb/spark,xflin/spark,aray/spark,xuanyuanking/spark,rikima/spark,taroplus/spark,mzl9039/spark,maropu/spark,bOOm-X/spark,gengliangwang/spark,markhamstra/spark,huang1900/spark,SHASHANKB/spark,guoxiaolongzte/spark,kiszk/spark,ajaysaini725/spark,patrick-nicholson/spark,haowu80s/spark,jianran/spark,yanboliang/spark,zzcclp/spark,Panos-Bletsos/spark-cost-model-optimizer,darionyaphet/spark,ron8hu/spark,JerryLead/spark,bdrillard/spark,techaddict/spark,ajaysaini725/spark,shaneknapp/spark,SnappyDataInc/spark,ueshin/apache-spark,1haodian/spark,ibm-research-ireland/sparkoscope,ConeyLiu/spark,alunarbeach/spark,joseph-torres/spark,holdenk/spark,setjet/spark,skonto/spark,ConeyLiu/spark,zhouyejoe/spark,andrewor14/iolap,maropu/spark,panfengfeng/spark,michalsenkyr/spark,mzl9039/spark,sahilTakiar/spark,hhbyyh/spark,jiangxb1987/spark,ajaysaini725/spark,akopich/spark,apache-spark-on-k8s/spark,jlopezmalla/spark,JoshRosen/spark,apache-spark-on-k8s/spark,holdenk/spark,joseph-torres/spark,loneknightpy/spark,Panos-Bletsos/spark-cost-model-optimizer,loneknightpy/spark,zzcclp/spark,szhem/spark,saltstar/spark,ahnqirage/spark,markhamstra/spark,cin/spark,highfei2011/spark,stanzhai/spark,BryanCutler/spark,BryanCutler/spark,lxsmnv/spark,aokolnychyi/spark,BryanCutler/spark,esi-mineset/spark,SnappyDataInc/spark,alunarbeach/spark,nilsgrabbert/spark,aosagie/spark,minixalpha/spark,SnappyDataInc/spark,andrewor14/iolap,rezasafi/spark,wangmiao1981/spark,JerryLead/spark,liyichao/spark,dbtsai/spark,srowen/spark,setjet/spark,ddna1021/spark,rekhajoshm/spark,tejasapatil/spark,WindCanDie/spark,jiangxb1987/spark,ptkool/spark,1haodian/spark,alunarbeach/spark,pronix/spark,icexelloss/spark,debugger87/spark,milliman/spark,mike0sv/spark,chuckchen/spark,rezasafi/spark,goldmedal/spark,kimoonkim/spark,nchammas/spark,brad-kaiser/spark,panfengfeng/spark,ibm-research-ireland/sparkoscope,guoxiaolongzte/spark,LantaoJin/spark,wzhfy/spark,goldmedal/spark,skonto/spark,xflin/spark,brad-kaiser/spark,caneGuy/spark,rikima/spark,cloud-fan/spark,nlalevee/spark,minixalpha/spark,JerryLead/spark,witgo/spark,patrick-nicholson/spark,kevinyu98/spark,someorz/spark,chuckchen/spark,nilsgrabbert/spark,janewangfb/spark,pgandhi999/spark,lvdongr/spark,andrewor14/iolap,gengliangwang/spark,minixalpha/spark,big-pegasus/spark,Aegeaner/spark,dongjoon-hyun/spark,joseph-torres/spark,srowen/spark,hvanhovell/spark,LantaoJin/spark,stanzhai/spark,cin/spark,michalsenkyr/spark,someorz/spark,caneGuy/spark,apache-spark-on-k8s/spark,andrewor14/iolap,wangmiao1981/spark,liyichao/spark,mdespriee/spark,bravo-zhang/spark,sachintyagi22/spark,saltstar/spark,aray/spark,eyalfa/spark,zuotingbing/spark,michalsenkyr/spark,poffuomo/spark,wangyum/spark,map222/spark,jrshust/spark,gengliangwang/spark,brad-kaiser/spark,HyukjinKwon/spark,stanzhai/spark,ericvandenbergfb/spark,jkbradley/spark,darionyaphet/spark,poffuomo/spark,kimoonkim/spark,rekhajoshm/spark,wangmiao1981/spark,hhbyyh/spark,mike0sv/spark,brad-kaiser/spark,eyalfa/spark,andrewor14/spark,gioenn/xSpark,liyichao/spark,rednaxelafx/apache-spark,rikima/spark,mdespriee/spark,vinodkc/spark,lvdongr/spark,nlalevee/spark,srowen/spark,aray/spark,apache/spark,ioana-delaney/spark,patrick-nicholson/spark,huang1900/spark,vinodkc/spark,ahnqirage/spark,kiszk/spark,szhem/spark,aokolnychyi/spark,cin/spark,kevinyu98/spark,michalsenkyr/spark,jiangxb1987/spark,kevinyu98/spark,WindCanDie/spark,janewangfb/spark,facaiy/spark,JoshRosen/spark,techaddict/spark,setjet/spark,shuangshuangwang/spark,skonto/spark,stanzhai/spark,rezasafi/spark,lxsmnv/spark,bOOm-X/spark,srowen/spark,tejasapatil/spark,icexelloss/spark,JoshRosen/spark,sahilTakiar/spark,andrewor14/iolap,ioana-delaney/spark,facaiy/spark,shuangshuangwang/spark,mahak/spark,kimoonkim/spark,gengliangwang/spark,kevinyu98/spark,liyichao/spark,ibm-research-ireland/sparkoscope,actuaryzhang/spark,kiszk/spark,saturday-shi/spark,map222/spark,LantaoJin/spark,aray/spark,Aegeaner/spark,dongjoon-hyun/spark,zero323/spark,metamx/spark,lvdongr/spark,SHASHANKB/spark,liutang123/spark,SnappyDataInc/spark,aokolnychyi/spark,guoxiaolongzte/spark,ddna1021/spark,UndeadBaneGitHub/spark,sachintyagi22/spark,tejasapatil/spark,lvdongr/spark,andrewor14/spark,witgo/spark,markhamstra/spark,jiangxb1987/spark,zuotingbing/spark,zero323/spark,adrian-ionescu/apache-spark,cloud-fan/spark,gioenn/xSpark,jkbradley/spark,hhbyyh/spark,kimoonkim/spark,someorz/spark,bravo-zhang/spark,bOOm-X/spark,szhem/spark,esi-mineset/spark,actuaryzhang/spark,wangmiao1981/spark,tejasapatil/spark,metamx/spark,ueshin/apache-spark,JerryLead/spark,pgandhi999/spark,vinodkc/spark,UndeadBaneGitHub/spark,MLnick/spark,SnappyDataInc/spark,mzl9039/spark,shubhamchopra/spark,loneknightpy/spark,jkbradley/spark,mdespriee/spark,xuanyuanking/spark,zuotingbing/spark,goldmedal/spark,ibm-research-ireland/sparkoscope,byakuinss/spark,setjet/spark,nchammas/spark,apache/spark,ahnqirage/spark,shuangshuangwang/spark,WeichenXu123/spark,goldmedal/spark,nlalevee/spark,nilsgrabbert/spark,nchammas/spark,andrewor14/spark,xuanyuanking/spark,shaneknapp/spark,adrian-ionescu/apache-spark,byakuinss/spark,publicRoman/spark,MLnick/spark,kimoonkim/spark,jrshust/spark,nilsgrabbert/spark,yanboliang/spark,facaiy/spark,taroplus/spark,wangyum/spark,publicRoman/spark,jkbradley/spark,narahari92/spark,ahnqirage/spark,xuanyuanking/spark,JerryLead/spark,spark-test/spark,brad-kaiser/spark,WindCanDie/spark,lxsmnv/spark,ron8hu/spark,SnappyDataInc/spark,loneknightpy/spark,esi-mineset/spark,publicRoman/spark,ConeyLiu/spark,aosagie/spark,taroplus/spark,kiszk/spark,liutang123/spark,brad-kaiser/spark,gengliangwang/spark,pgandhi999/spark,Panos-Bletsos/spark-cost-model-optimizer,poffuomo/spark,zhouyejoe/spark,1haodian/spark,HyukjinKwon/spark,UndeadBaneGitHub/spark,pgandhi999/spark,shubhamchopra/spark,aokolnychyi/spark,nchammas/spark,sahilTakiar/spark,rekhajoshm/spark,setjet/spark,chuckchen/spark,actuaryzhang/spark,SHASHANKB/spark,ericvandenbergfb/spark,apache/spark,jianran/spark,sahilTakiar/spark,dhruve/spark,aosagie/spark,ddna1021/spark,rezasafi/spark,bdrillard/spark,wangmiao1981/spark,1haodian/spark,facaiy/spark,debugger87/spark,ajaysaini725/spark,rezasafi/spark,actuaryzhang/spark,bravo-zhang/spark,milliman/spark,ptkool/spark,saturday-shi/spark,jianran/spark,mdespriee/spark,mahak/spark,JoshRosen/spark,akopich/spark,jianran/spark,narahari92/spark,wangyum/spark,wzhfy/spark,wangyum/spark,nilsgrabbert/spark,gioenn/xSpark,narahari92/spark,dbtsai/spark,tejasapatil/spark,BryanCutler/spark,bOOm-X/spark,BryanCutler/spark,zzcclp/spark,MLnick/spark,jiangxb1987/spark,markhamstra/spark,mike0sv/spark,caneGuy/spark,shuangshuangwang/spark,zhouyejoe/spark,actuaryzhang/spark,saturday-shi/spark,srowen/spark,liyichao/spark,shuangshuangwang/spark,debugger87/spark,ibm-research-ireland/sparkoscope,panfengfeng/spark,rezasafi/spark,panfengfeng/spark,byakuinss/spark,guoxiaolongzte/spark,chuckchen/spark,yanboliang/spark,lxsmnv/spark,apache/spark,aosagie/spark,ddna1021/spark,ahnqirage/spark,SHASHANKB/spark,ioana-delaney/spark,esi-mineset/spark,jrshust/spark,xuanyuanking/spark,zhouyejoe/spark,michalsenkyr/spark,Aegeaner/spark,szhem/spark,guoxiaolongzte/spark,ueshin/apache-spark,michalsenkyr/spark,haowu80s/spark,markhamstra/spark,debugger87/spark,adrian-ionescu/apache-spark,dbtsai/spark,hvanhovell/spark,milliman/spark,saturday-shi/spark,gioenn/xSpark,janewangfb/spark,rekhajoshm/spark,shaneknapp/spark,kissmetrics/spark,rikima/spark,eyalfa/spark,icexelloss/spark,jianran/spark,aosagie/spark,highfei2011/spark,ueshin/apache-spark,MLnick/spark,mahak/spark,esi-mineset/spark,adrian-ionescu/apache-spark,guoxiaolongzte/spark,xflin/spark,rikima/spark,szhem/spark,sachintyagi22/spark,jkbradley/spark,shaneknapp/spark,bdrillard/spark,patrick-nicholson/spark,dongjoon-hyun/spark,minixalpha/spark,haowu80s/spark,witgo/spark,nchammas/spark,taroplus/spark,ajaysaini725/spark,metamx/spark,mike0sv/spark,pronix/spark,taroplus/spark,holdenk/spark,kissmetrics/spark,ron8hu/spark,techaddict/spark,WindCanDie/spark,ddna1021/spark,apache/spark,jiangxb1987/spark,zero323/spark,bdrillard/spark,publicRoman/spark,tengpeng/spark,haowu80s/spark,hvanhovell/spark,panfengfeng/spark,mike0sv/spark,haowu80s/spark,techaddict/spark,saturday-shi/spark,matthewfranglen/spark,srowen/spark,jlopezmalla/spark,bOOm-X/spark,pgandhi999/spark,metamx/spark,jiangxb1987/spark,goldmedal/spark,zuotingbing/spark,ptkool/spark,aray/spark,apache-spark-on-k8s/spark,spark-test/spark,gioenn/xSpark,saltstar/spark,sachintyagi22/spark,saltstar/spark,zzcclp/spark,poffuomo/spark,rekhajoshm/spark,cloud-fan/spark,szhem/spark,facaiy/spark,gengliangwang/spark,zero323/spark,taroplus/spark,rednaxelafx/apache-spark,spark-test/spark,darionyaphet/spark,ron8hu/spark,caneGuy/spark,markhamstra/spark,highfei2011/spark,nlalevee/spark,WeichenXu123/spark,liutang123/spark,liyichao/spark,panfengfeng/spark,ericvandenbergfb/spark,xflin/spark,markhamstra/spark,big-pegasus/spark,lxsmnv/spark,joseph-torres/spark,pgandhi999/spark,cin/spark,narahari92/spark,skonto/spark,highfei2011/spark,kissmetrics/spark,ericvandenbergfb/spark,LantaoJin/spark,caneGuy/spark,holdenk/spark,bOOm-X/spark,zzcclp/spark,andrewor14/spark,saltstar/spark,zero323/spark,holdenk/spark,dhruve/spark,gioenn/xSpark,hhbyyh/spark,maropu/spark,tengpeng/spark,HyukjinKwon/spark,dhruve/spark,ahnqirage/spark,kevinyu98/spark,ron8hu/spark,mzl9039/spark,vinodkc/spark,milliman/spark,aosagie/spark,ConeyLiu/spark,jkbradley/spark,aray/spark,skonto/spark,lvdongr/spark,HyukjinKwon/spark,akopich/spark,shuangshuangwang/spark,andrewor14/spark,WeichenXu123/spark,shubhamchopra/spark,gengliangwang/spark,facaiy/spark,nilsgrabbert/spark,jianran/spark,jlopezmalla/spark,mahak/spark,dhruve/spark,hvanhovell/spark,big-pegasus/spark,goldmedal/spark,zzcclp/spark,minixalpha/spark,MLnick/spark,Panos-Bletsos/spark-cost-model-optimizer,wzhfy/spark,yanboliang/spark,hvanhovell/spark,metamx/spark,alunarbeach/spark,ron8hu/spark,shubhamchopra/spark,haowu80s/spark,aokolnychyi/spark,holdenk/spark,SHASHANKB/spark,HyukjinKwon/spark,Aegeaner/spark,sureshthalamati/spark,ptkool/spark,byakuinss/spark,sahilTakiar/spark,wzhfy/spark,dhruve/spark,michalsenkyr/spark,rednaxelafx/apache-spark,wangmiao1981/spark,spark-test/spark,shubhamchopra/spark,Aegeaner/spark,map222/spark,maropu/spark,big-pegasus/spark,kiszk/spark,spark-test/spark,wzhfy/spark,alunarbeach/spark,lxsmnv/spark,WeichenXu123/spark,ajaysaini725/spark,caneGuy/spark,jlopezmalla/spark,UndeadBaneGitHub/spark,JoshRosen/spark,apache/spark,ibm-research-ireland/sparkoscope,jrshust/spark,dbtsai/spark,sahilTakiar/spark,LantaoJin/spark,bdrillard/spark,shaneknapp/spark,metamx/spark,alunarbeach/spark,liutang123/spark,mahak/spark,JerryLead/spark,ddna1021/spark,stanzhai/spark,spark-test/spark,dongjoon-hyun/spark,vinodkc/spark,nlalevee/spark,MLnick/spark,ericvandenbergfb/spark,akopich/spark,apache-spark-on-k8s/spark,cloud-fan/spark,chuckchen/spark,chuckchen/spark,mzl9039/spark,WindCanDie/spark,loneknightpy/spark,szhem/spark,kissmetrics/spark,facaiy/spark,ueshin/apache-spark,shaneknapp/spark,Aegeaner/spark,rednaxelafx/apache-spark,JoshRosen/spark,JoshRosen/spark,sureshthalamati/spark,rezasafi/spark,icexelloss/spark,rednaxelafx/apache-spark,SHASHANKB/spark,minixalpha/spark,gioenn/xSpark,cloud-fan/spark,mike0sv/spark,zuotingbing/spark,darionyaphet/spark,publicRoman/spark,map222/spark,andrewor14/spark,SHASHANKB/spark,1haodian/spark,map222/spark,srowen/spark,mahak/spark,jlopezmalla/spark,saturday-shi/spark,esi-mineset/spark,cloud-fan/spark,kimoonkim/spark,nchammas/spark,eyalfa/spark,ueshin/apache-spark,pronix/spark,byakuinss/spark,jrshust/spark,tengpeng/spark,hvanhovell/spark,vinodkc/spark,ConeyLiu/spark,narahari92/spark,zero323/spark,highfei2011/spark,sureshthalamati/spark,someorz/spark,dongjoon-hyun/spark,aokolnychyi/spark,witgo/spark,pronix/spark,panfengfeng/spark,dbtsai/spark,taroplus/spark,witgo/spark,holdenk/spark,adrian-ionescu/apache-spark,wangyum/spark,huang1900/spark,poffuomo/spark,huang1900/spark,nchammas/spark,ConeyLiu/spark,shubhamchopra/spark,BryanCutler/spark,WeichenXu123/spark,cin/spark