[SPARK-4423] Improve foreach() documentation to avoid confusion between local- and cluster-mode behavior
[SPARK-4423] Improve foreach() documentation to avoid confusion between local- and cluster-mode behavior

Hi all - I've added a writeup on how closures work within Spark to help clarify the general case for this problem and similar problems. I hope this addresses the issue and would love any feedback.

Author: Ilya Ganelin <ilya.ganelin@capitalone.com>

Closes #4696 from ilganeli/SPARK-4423 and squashes the following commits:

c5dc498 [Ilya Ganelin] Fixed typo
07b78e8 [Ilya Ganelin] Updated to fix capitalization
48c1983 [Ilya Ganelin] Updated to fix capitalization and clarify wording
2fd2a07 [Ilya Ganelin] Incoporated a few more minor fixes. Fixed a bug in python code. Added semicolons for java
4772f99 [Ilya Ganelin] Incorporated latest feedback
448bd79 [Ilya Ganelin] Updated some verbage and added section links
5dbbda5 [Ilya Ganelin] Improved some wording
d374d3a [Ilya Ganelin] Merge remote-tracking branch 'upstream/master' into SPARK-4423
2600668 [Ilya Ganelin] Minor edits
c768ab2 [Ilya Ganelin] Updated documentation to add a section on closures. This helps understand confusing behavior of foreach and map functions when attempting to modify variables outside of the scope of an RDD action or transformation

kevinyu98/spark,cin/spark,jianran/spark,liutang123/spark,ConeyLiu/spark,setjet/spark,wangyum/spark,sureshthalamati/spark,apache/spark,brad-kaiser/spark,wzhfy/spark,ConeyLiu/spark,sachintyagi22/spark,adrian-ionescu/apache-spark,liyichao/spark,bravo-zhang/spark,kissmetrics/spark,HyukjinKwon/spark,vinodkc/spark,kimoonkim/spark,eyalfa/spark,ron8hu/spark,pronix/spark,akopich/spark,mzl9039/spark,xflin/spark,nlalevee/spark,byakuinss/spark,ioana-delaney/spark,saltstar/spark,jiangxb1987/spark,rezasafi/spark,icexelloss/spark,cloud-fan/spark,bOOm-X/spark,zuotingbing/spark,apache/spark,tejasapatil/spark,pronix/spark,setjet/spark,jkbradley/spark,SnappyDataInc/spark,wzhfy/spark,actuaryzhang/spark,dongjoon-hyun/spark,metamx/spark,JerryLead/spark,pronix/spark,map222/spark,hhbyyh/spark,someorz/spark,actuaryzhang/spark,tengpeng/spark,liyichao/spark,zzcclp/spark,xflin/spark,ibm-research-ireland/sparkoscope,andrewor14/iolap,panfengfeng/spark,jiangxb1987/spark,sureshthalamati/spark,sahilTakiar/spark,apache-spark-on-k8s/spark,xuanyuanking/spark,techaddict/spark,eyalfa/spark,gengliangwang/spark,hvanhovell/spark,shaneknapp/spark,SHASHANKB/spark,jrshust/spark,patrick-nicholson/spark,taroplus/spark,holdenk/spark,srowen/spark,WindCanDie/spark,aray/spark,alunarbeach/spark,WindCanDie/spark,mahak/spark,highfei2011/spark,dongjoon-hyun/spark,jianran/spark,JoshRosen/spark,jlopezmalla/spark,michalsenkyr/spark,holdenk/spark,gengliangwang/spark,milliman/spark,dongjoon-hyun/spark,apache/spark,caneGuy/spark,ptkool/spark,haowu80s/spark,apache-spark-on-k8s/spark,alunarbeach/spark,zuotingbing/spark,ajaysaini725/spark,taroplus/spark,minixalpha/spark,mike0sv/spark,nchammas/spark,aray/spark,setjet/spark,tejasapatil/spark,apache/spark,jrshust/spark,big-pegasus/spark,hvanhovell/spark,shaneknapp/spark,ioana-delaney/spark,wangmiao1981/spark,JerryLead/spark,1haodian/spark,mzl9039/spark,poffuomo/spark,goldmedal/spark,metamx/spark,haowu80s/spark,wangmiao1981/spark,BryanCutler/spark,dhruve/spark,sureshthalamati/spark,UndeadBaneGitHub/spark,actuaryzhang/spark,xflin/spark,darionyaphet/spark,xuanyuanking/spark,shuangshuangwang/spark,BryanCutler/spark,brad-kaiser/spark,jrshust/spark,wzhfy/spark,facaiy/spark,someorz/spark,mdespriee/spark,haowu80s/spark,liutang123/spark,wangyum/spark,adrian-ionescu/apache-spark,ueshin/apache-spark,jlopezmalla/spark,ddna1021/spark,nilsgrabbert/spark,MLnick/spark,janewangfb/spark,kissmetrics/spark,dhruve/spark,rekhajoshm/spark,bdrillard/spark,eyalfa/spark,witgo/spark,ron8hu/spark,kimoonkim/spark,actuaryzhang/spark,jrshust/spark,zuotingbing/spark,bdrillard/spark,ptkool/spark,lvdongr/spark,WindCanDie/spark,mike0sv/spark,rikima/spark,kevinyu98/spark,skonto/spark,facaiy/spark,maropu/spark,maropu/spark,metamx/spark,ron8hu/spark,gengliangwang/spark,goldmedal/spark,lxsmnv/spark,rednaxelafx/apache-spark,sureshthalamati/spark,kissmetrics/spark,cloud-fan/spark,big-pegasus/spark,nchammas/spark,UndeadBaneGitHub/spark,kevinyu98/spark,Panos-Bletsos/spark-cost-model-optimizer,ueshin/apache-spark,facaiy/spark,vinodkc/spark,joseph-torres/spark,ibm-research-ireland/sparkoscope,haowu80s/spark,huang1900/spark,zero323/spark,zzcclp/spark,darionyaphet/spark,saltstar/spark,ahnqirage/spark,zhouyejoe/spark,byakuinss/spark,guoxiaolongzte/spark,gengliangwang/spark,zzcclp/spark,ericvandenbergfb/spark,rikima/spark,poffuomo/spark,shubhamchopra/spark,cloud-fan/spark,JerryLead/spark,actuaryzhang/spark,ron8hu/spark,HyukjinKwon/spark,ptkool/spark,mdespriee/spark,adrian-ionescu/apache-spark,WindCanDie/spark,mdespriee/spark,aosagie/spark,stanzhai/spark,ajaysaini725/spark,Aegeaner/spark,metamx/spark,UndeadBaneGitHub/spark,alunarbeach/spark,rikima/spark,akopich/spark,andrewor14/spark,MLnick/spark,srowen/spark,guoxiaolongzte/spark,bOOm-X/spark,rekhajoshm/spark,ajaysaini725/spark,kevinyu98/spark,skonto/spark,shaneknapp/spark,kimoonkim/spark,goldmedal/spark,skonto/spark,rikima/spark,kiszk/spark,jiangxb1987/spark,saltstar/spark,ahnqirage/spark,yanboliang/spark,jlopezmalla/spark,mike0sv/spark,SnappyDataInc/spark,pronix/spark,publicRoman/spark,milliman/spark,liutang123/spark,xflin/spark,liyichao/spark,cin/spark,eyalfa/spark,SHASHANKB/spark,apache-spark-on-k8s/spark,BryanCutler/spark,guoxiaolongzte/spark,ddna1021/spark,narahari92/spark,SnappyDataInc/spark,hhbyyh/spark,lxsmnv/spark,spark-test/spark,srowen/spark,lvdongr/spark,panfengfeng/spark,witgo/spark,mzl9039/spark,andrewor14/iolap,joseph-torres/spark,jkbradley/spark,publicRoman/spark,bOOm-X/spark,goldmedal/spark,hhbyyh/spark,witgo/spark,wangmiao1981/spark,xuanyuanking/spark,akopich/spark,adrian-ionescu/apache-spark,kimoonkim/spark,esi-mineset/spark,jiangxb1987/spark,icexelloss/spark,saltstar/spark,wangmiao1981/spark,icexelloss/spark,BryanCutler/spark,darionyaphet/spark,publicRoman/spark,spark-test/spark,ueshin/apache-spark,kiszk/spark,map222/spark,zuotingbing/spark,milliman/spark,xuanyuanking/spark,michalsenkyr/spark,mzl9039/spark,JoshRosen/spark,hhbyyh/spark,chuckchen/spark,saturday-shi/spark,jlopezmalla/spark,highfei2011/spark,zuotingbing/spark,akopich/spark,zero323/spark,nchammas/spark,nchammas/spark,ahnqirage/spark,mdespriee/spark,alunarbeach/spark,gengliangwang/spark,spark-test/spark,JerryLead/spark,ddna1021/spark,kimoonkim/spark,setjet/spark,Aegeaner/spark,WeichenXu123/spark,ibm-research-ireland/sparkoscope,1haodian/spark,someorz/spark,mahak/spark,narahari92/spark,rezasafi/spark,witgo/spark,nlalevee/spark,tejasapatil/spark,narahari92/spark,Aegeaner/spark,esi-mineset/spark,liutang123/spark,maropu/spark,patrick-nicholson/spark,tengpeng/spark,setjet/spark,milliman/spark,aosagie/spark,aokolnychyi/spark,loneknightpy/spark,jkbradley/spark,WeichenXu123/spark,sahilTakiar/spark,techaddict/spark,tengpeng/spark,aray/spark,yanboliang/spark,lxsmnv/spark,adrian-ionescu/apache-spark,debugger87/spark,chuckchen/spark,alunarbeach/spark,LantaoJin/spark,srowen/spark,ibm-research-ireland/sparkoscope,zero323/spark,rekhajoshm/spark,apache-spark-on-k8s/spark,loneknightpy/spark,HyukjinKwon/spark,ajaysaini725/spark,UndeadBaneGitHub/spark,debugger87/spark,JerryLead/spark,kissmetrics/spark,big-pegasus/spark,techaddict/spark,chuckchen/spark,Panos-Bletsos/spark-cost-model-optimizer,minixalpha/spark,bdrillard/spark,taroplus/spark,JoshRosen/spark,liyichao/spark,dongjoon-hyun/spark,actuaryzhang/spark,LantaoJin/spark,kiszk/spark,big-pegasus/spark,ajaysaini725/spark,sachintyagi22/spark,SnappyDataInc/spark,wangmiao1981/spark,skonto/spark,sachintyagi22/spark,taroplus/spark,1haodian/spark,guoxiaolongzte/spark,shuangshuangwang/spark,tengpeng/spark,liyichao/spark,Panos-Bletsos/spark-cost-model-optimizer,aokolnychyi/spark,ptkool/spark,HyukjinKwon/spark,zhouyejoe/spark,brad-kaiser/spark,zhouyejoe/spark,pronix/spark,rezasafi/spark,ahnqirage/spark,SHASHANKB/spark,andrewor14/iolap,srowen/spark,shubhamchopra/spark,someorz/spark,cloud-fan/spark,andrewor14/iolap,wangmiao1981/spark,panfengfeng/spark,patrick-nicholson/spark,icexelloss/spark,nilsgrabbert/spark,sachintyagi22/spark,pgandhi999/spark,tejasapatil/spark,mdespriee/spark,ericvandenbergfb/spark,JoshRosen/spark,gioenn/xSpark,rezasafi/spark,aokolnychyi/spark,witgo/spark,bravo-zhang/spark,rednaxelafx/apache-spark,metamx/spark,ptkool/spark,jiangxb1987/spark,apache-spark-on-k8s/spark,Aegeaner/spark,publicRoman/spark,zzcclp/spark,zzcclp/spark,kiszk/spark,JoshRosen/spark,sahilTakiar/spark,rezasafi/spark,wangyum/spark,jkbradley/spark,nlalevee/spark,ahnqirage/spark,guoxiaolongzte/spark,someorz/spark,SHASHANKB/spark,lxsmnv/spark,jiangxb1987/spark,pgandhi999/spark,hhbyyh/spark,zero323/spark,facaiy/spark,cloud-fan/spark,andrewor14/spark,patrick-nicholson/spark,jianran/spark,janewangfb/spark,ioana-delaney/spark,mdespriee/spark,saturday-shi/spark,jkbradley/spark,dbtsai/spark,kiszk/spark,wzhfy/spark,taroplus/spark,bOOm-X/spark,stanzhai/spark,wzhfy/spark,byakuinss/spark,dbtsai/spark,shaneknapp/spark,pgandhi999/spark,mzl9039/spark,1haodian/spark,rikima/spark,lxsmnv/spark,esi-mineset/spark,apache-spark-on-k8s/spark,rikima/spark,joseph-torres/spark,stanzhai/spark,michalsenkyr/spark,facaiy/spark,bdrillard/spark,1haodian/spark,ddna1021/spark,bdrillard/spark,milliman/spark,cin/spark,janewangfb/spark,yanboliang/spark,kiszk/spark,JerryLead/spark,lvdongr/spark,bravo-zhang/spark,WeichenXu123/spark,highfei2011/spark,janewangfb/spark,ioana-delaney/spark,ConeyLiu/spark,kevinyu98/spark,brad-kaiser/spark,sureshthalamati/spark,spark-test/spark,joseph-torres/spark,gioenn/xSpark,lxsmnv/spark,byakuinss/spark,bOOm-X/spark,alunarbeach/spark,aray/spark,wangyum/spark,nchammas/spark,nlalevee/spark,jkbradley/spark,kimoonkim/spark,guoxiaolongzte/spark,lxsmnv/spark,hhbyyh/spark,nilsgrabbert/spark,huang1900/spark,maropu/spark,icexelloss/spark,poffuomo/spark,michalsenkyr/spark,ron8hu/spark,spark-test/spark,mdespriee/spark,kevinyu98/spark,srowen/spark,loneknightpy/spark,rekhajoshm/spark,hvanhovell/spark,apache/spark,rezasafi/spark,darionyaphet/spark,dbtsai/spark,HyukjinKwon/spark,sachintyagi22/spark,rednaxelafx/apache-spark,lvdongr/spark,adrian-ionescu/apache-spark,milliman/spark,MLnick/spark,ueshin/apache-spark,gioenn/xSpark,wzhfy/spark,szhem/spark,zuotingbing/spark,eyalfa/spark,gengliangwang/spark,ibm-research-ireland/sparkoscope,chuckchen/spark,facaiy/spark,BryanCutler/spark,panfengfeng/spark,Panos-Bletsos/spark-cost-model-optimizer,poffuomo/spark,highfei2011/spark,hvanhovell/spark,sahilTakiar/spark,ddna1021/spark,kissmetrics/spark,huang1900/spark,haowu80s/spark,someorz/spark,patrick-nicholson/spark,xuanyuanking/spark,nlalevee/spark,eyalfa/spark,sahilTakiar/spark,maropu/spark,UndeadBaneGitHub/spark,rednaxelafx/apache-spark,esi-mineset/spark,debugger87/spark,sahilTakiar/spark,pronix/spark,byakuinss/spark,jlopezmalla/spark,andrewor14/spark,alunarbeach/spark,Aegeaner/spark,WindCanDie/spark,nilsgrabbert/spark,WeichenXu123/spark,yanboliang/spark,JerryLead/spark,aosagie/spark,WindCanDie/spark,liyichao/spark,cloud-fan/spark,ajaysaini725/spark,jkbradley/spark,highfei2011/spark,cin/spark,publicRoman/spark,chuckchen/spark,joseph-torres/spark,zhouyejoe/spark,zhouyejoe/spark,ron8hu/spark,brad-kaiser/spark,hvanhovell/spark,patrick-nicholson/spark,yanboliang/spark,matthewfranglen/spark,wangyum/spark,zero323/spark,holdenk/spark,akopich/spark,ptkool/spark,ioana-delaney/spark,1haodian/spark,Panos-Bletsos/spark-cost-model-optimizer,rednaxelafx/apache-spark,zhouyejoe/spark,vinodkc/spark,kevinyu98/spark,dbtsai/spark,loneknightpy/spark,pgandhi999/spark,vinodkc/spark,setjet/spark,highfei2011/spark,michalsenkyr/spark,1haodian/spark,WeichenXu123/spark,sachintyagi22/spark,adrian-ionescu/apache-spark,haowu80s/spark,ibm-research-ireland/sparkoscope,jianran/spark,janewangfb/spark,darionyaphet/spark,MLnick/spark,dhruve/spark,kissmetrics/spark,map222/spark,panfengfeng/spark,mahak/spark,Panos-Bletsos/spark-cost-model-optimizer,michalsenkyr/spark,gioenn/xSpark,stanzhai/spark,pgandhi999/spark,shuangshuangwang/spark,shaneknapp/spark,apache/spark,goldmedal/spark,narahari92/spark,mzl9039/spark,kimoonkim/spark,ron8hu/spark,wangyum/spark,akopich/spark,markhamstra/spark,szhem/spark,patrick-nicholson/spark,goldmedal/spark,mahak/spark,map222/spark,nlalevee/spark,shuangshuangwang/spark,szhem/spark,darionyaphet/spark,aray/spark,jianran/spark,andrewor14/spark,stanzhai/spark,shuangshuangwang/spark,markhamstra/spark,metamx/spark,wangyum/spark,liutang123/spark,holdenk/spark,techaddict/spark,SnappyDataInc/spark,gioenn/xSpark,highfei2011/spark,liyichao/spark,zuotingbing/spark,dbtsai/spark,dongjoon-hyun/spark,WindCanDie/spark,big-pegasus/spark,bravo-zhang/spark,cin/spark,caneGuy/spark,ahnqirage/spark,szhem/spark,jianran/spark,aray/spark,SnappyDataInc/spark,aokolnychyi/spark,shuangshuangwang/spark,zhouyejoe/spark,minixalpha/spark,lvdongr/spark,shubhamchopra/spark,aosagie/spark,SHASHANKB/spark,stanzhai/spark,holdenk/spark,holdenk/spark,cloud-fan/spark,caneGuy/spark,dhruve/spark,bdrillard/spark,chuckchen/spark,saltstar/spark,rekhajoshm/spark,dhruve/spark,debugger87/spark,SnappyDataInc/spark,setjet/spark,ConeyLiu/spark,saturday-shi/spark,goldmedal/spark,andrewor14/iolap,markhamstra/spark,bdrillard/spark,rikima/spark,LantaoJin/spark,minixalpha/spark,caneGuy/spark,brad-kaiser/spark,szhem/spark,WeichenXu123/spark,map222/spark,loneknightpy/spark,ericvandenbergfb/spark,brad-kaiser/spark,BryanCutler/spark,skonto/spark,debugger87/spark,dbtsai/spark,WeichenXu123/spark,map222/spark,zzcclp/spark,hvanhovell/spark,pgandhi999/spark,esi-mineset/spark,zzcclp/spark,rednaxelafx/apache-spark,huang1900/spark,panfengfeng/spark,nchammas/spark,skonto/spark,shubhamchopra/spark,lvdongr/spark,vinodkc/spark,hvanhovell/spark,guoxiaolongzte/spark,yanboliang/spark,SHASHANKB/spark,caneGuy/spark,ericvandenbergfb/spark,mike0sv/spark,hhbyyh/spark,tejasapatil/spark,witgo/spark,vinodkc/spark,huang1900/spark,eyalfa/spark,ConeyLiu/spark,janewangfb/spark,andrewor14/iolap,ddna1021/spark,cin/spark,sureshthalamati/spark,saturday-shi/spark,nilsgrabbert/spark,saltstar/spark,panfengfeng/spark,tengpeng/spark,markhamstra/spark,lvdongr/spark,saltstar/spark,spark-test/spark,maropu/spark,ericvandenbergfb/spark,UndeadBaneGitHub/spark,jiangxb1987/spark,nchammas/spark,MLnick/spark,zero323/spark,rednaxelafx/apache-spark,mahak/spark,zero323/spark,xflin/spark,minixalpha/spark,ConeyLiu/spark,nlalevee/spark,andrewor14/spark,witgo/spark,caneGuy/spark,markhamstra/spark,saturday-shi/spark,publicRoman/spark,publicRoman/spark,techaddict/spark,mike0sv/spark,andrewor14/spark,debugger87/spark,narahari92/spark,aosagie/spark,ConeyLiu/spark,dongjoon-hyun/spark,debugger87/spark,liutang123/spark,janewangfb/spark,nilsgrabbert/spark,aokolnychyi/spark,xuanyuanking/spark,shaneknapp/spark,LantaoJin/spark,skonto/spark,tejasapatil/spark,tejasapatil/spark,ptkool/spark,aokolnychyi/spark,LantaoJin/spark,aosagie/spark,techaddict/spark,poffuomo/spark,jrshust/spark,map222/spark,ddna1021/spark,nilsgrabbert/spark,akopich/spark,icexelloss/spark,JoshRosen/spark,JoshRosen/spark,jianran/spark,shubhamchopra/spark,bravo-zhang/spark,jrshust/spark,apache/spark,someorz/spark,esi-mineset/spark,gioenn/xSpark,byakuinss/spark,apache-spark-on-k8s/spark,ueshin/apache-spark,minixalpha/spark,ibm-research-ireland/sparkoscope,loneknightpy/spark,mahak/spark,pgandhi999/spark,icexelloss/spark,xflin/spark,jrshust/spark,mzl9039/spark,LantaoJin/spark,facaiy/spark,HyukjinKwon/spark,rekhajoshm/spark,aosagie/spark,techaddict/spark,dbtsai/spark,dhruve/spark,MLnick/spark,Panos-Bletsos/spark-cost-model-optimizer,mike0sv/spark,jlopezmalla/spark,big-pegasus/spark,mahak/spark,darionyaphet/spark,liutang123/spark,joseph-torres/spark,BryanCutler/spark,stanzhai/spark,spark-test/spark,Aegeaner/spark,holdenk/spark,saturday-shi/spark,minixalpha/spark,bravo-zhang/spark,shuangshuangwang/spark,kiszk/spark,shubhamchopra/spark,taroplus/spark,szhem/spark,SHASHANKB/spark,aokolnychyi/spark,ericvandenbergfb/spark,poffuomo/spark,shaneknapp/spark,srowen/spark,sachintyagi22/spark,ajaysaini725/spark,caneGuy/spark,LantaoJin/spark,loneknightpy/spark,ioana-delaney/spark,ioana-delaney/spark,bOOm-X/spark,ahnqirage/spark,narahari92/spark,MLnick/spark,UndeadBaneGitHub/spark,bOOm-X/spark,dhruve/spark,esi-mineset/spark,big-pegasus/spark,yanboliang/spark,cin/spark,szhem/spark,jlopezmalla/spark,milliman/spark,aray/spark,mike0sv/spark,sureshthalamati/spark,kissmetrics/spark,joseph-torres/spark,chuckchen/spark,markhamstra/spark,gioenn/xSpark,huang1900/spark,poffuomo/spark,saturday-shi/spark,tengpeng/spark,taroplus/spark,bravo-zhang/spark,dongjoon-hyun/spark,huang1900/spark,byakuinss/spark,Aegeaner/spark,gengliangwang/spark,narahari92/spark,vinodkc/spark,rekhajoshm/spark,sahilTakiar/spark,ueshin/apache-spark,ericvandenbergfb/spark,xflin/spark,xuanyuanking/spark,actuaryzhang/spark,andrewor14/spark,tengpeng/spark,rezasafi/spark,wzhfy/spark,maropu/spark,michalsenkyr/spark,markhamstra/spark,wangmiao1981/spark,metamx/spark,ueshin/apache-spark,shubhamchopra/spark,HyukjinKwon/spark,andrewor14/iolap