[SPARK-30480][PYSPARK][TESTS] Fix 'test_memory_limit' on pyspark test
[SPARK-30480][PYSPARK][TESTS] Fix 'test_memory_limit' on pyspark test

### What changes were proposed in this pull request?

This patch increases the memory limit in the test 'test_memory_limit' from 1m to 8m.
Credit to srowen and HyukjinKwon to provide the idea of suspicion and guide how to fix.

### Why are the changes needed?

We observed consistent Pyspark test failures on multiple PRs (#26955, #26201, #27064) which block the PR builds whenever the test is included.

### Does this PR introduce any user-facing change?

No.

### How was this patch tested?

Jenkins builds passed in WIP PR (#27159)

Closes #27162 from HeartSaVioR/SPARK-30480.

Authored-by: Jungtaek Lim (HeartSaVioR) <6dfb5872701d21af1c4dd7add991df02fba61fbf@gmail.com>
Signed-off-by: HyukjinKwon <2680965ad5aae29275c8fb10150cb10f5554e78b@apache.org>

holdenk/spark,spark-test/spark,wangyum/spark,rednaxelafx/apache-spark,goldmedal/spark,dongjoon-hyun/spark,skonto/spark,ConeyLiu/spark,ueshin/apache-spark,hvanhovell/spark,nchammas/spark,darionyaphet/spark,hvanhovell/spark,kevinyu98/spark,wangmiao1981/spark,vinodkc/spark,chuckchen/spark,milliman/spark,kevinyu98/spark,maropu/spark,JoshRosen/spark,dongjoon-hyun/spark,goldmedal/spark,BryanCutler/spark,shuangshuangwang/spark,milliman/spark,wzhfy/spark,dbtsai/spark,zuotingbing/spark,ptkool/spark,rednaxelafx/apache-spark,BryanCutler/spark,wzhfy/spark,wzhfy/spark,jiangxb1987/spark,wangmiao1981/spark,shaneknapp/spark,mahak/spark,apache/spark,mahak/spark,xuanyuanking/spark,ptkool/spark,taroplus/spark,darionyaphet/spark,dbtsai/spark,dbtsai/spark,wangyum/spark,jiangxb1987/spark,gengliangwang/spark,nchammas/spark,rednaxelafx/apache-spark,ueshin/apache-spark,taroplus/spark,shuangshuangwang/spark,wangyum/spark,nchammas/spark,taroplus/spark,goldmedal/spark,spark-test/spark,wangmiao1981/spark,gengliangwang/spark,gengliangwang/spark,maropu/spark,zzcclp/spark,ptkool/spark,ueshin/apache-spark,skonto/spark,cloud-fan/spark,wangmiao1981/spark,cloud-fan/spark,shaneknapp/spark,darionyaphet/spark,apache/spark,witgo/spark,hvanhovell/spark,ConeyLiu/spark,darionyaphet/spark,dongjoon-hyun/spark,mahak/spark,chuckchen/spark,JoshRosen/spark,BryanCutler/spark,taroplus/spark,nchammas/spark,skonto/spark,jiangxb1987/spark,zzcclp/spark,milliman/spark,JoshRosen/spark,WeichenXu123/spark,wzhfy/spark,milliman/spark,BryanCutler/spark,zero323/spark,rednaxelafx/apache-spark,wangyum/spark,WeichenXu123/spark,srowen/spark,ueshin/apache-spark,apache/spark,ConeyLiu/spark,cloud-fan/spark,zzcclp/spark,wzhfy/spark,apache/spark,shuangshuangwang/spark,zuotingbing/spark,ueshin/apache-spark,dbtsai/spark,gengliangwang/spark,taroplus/spark,cloud-fan/spark,holdenk/spark,darionyaphet/spark,chuckchen/spark,dongjoon-hyun/spark,milliman/spark,gengliangwang/spark,xuanyuanking/spark,ptkool/spark,ptkool/spark,rednaxelafx/apache-spark,jiangxb1987/spark,zero323/spark,skonto/spark,maropu/spark,maropu/spark,matthewfranglen/spark,ueshin/apache-spark,vinodkc/spark,spark-test/spark,jiangxb1987/spark,zero323/spark,xuanyuanking/spark,witgo/spark,ConeyLiu/spark,shaneknapp/spark,srowen/spark,hvanhovell/spark,HyukjinKwon/spark,shuangshuangwang/spark,holdenk/spark,BryanCutler/spark,xuanyuanking/spark,wangmiao1981/spark,wangyum/spark,kevinyu98/spark,gengliangwang/spark,holdenk/spark,chuckchen/spark,dbtsai/spark,ConeyLiu/spark,HyukjinKwon/spark,zero323/spark,nchammas/spark,skonto/spark,chuckchen/spark,WeichenXu123/spark,goldmedal/spark,holdenk/spark,zuotingbing/spark,HyukjinKwon/spark,BryanCutler/spark,hvanhovell/spark,vinodkc/spark,skonto/spark,ueshin/apache-spark,shaneknapp/spark,darionyaphet/spark,hvanhovell/spark,shaneknapp/spark,milliman/spark,goldmedal/spark,witgo/spark,witgo/spark,wangmiao1981/spark,kevinyu98/spark,witgo/spark,JoshRosen/spark,WeichenXu123/spark,zero323/spark,vinodkc/spark,srowen/spark,cloud-fan/spark,ConeyLiu/spark,taroplus/spark,srowen/spark,zero323/spark,spark-test/spark,dongjoon-hyun/spark,nchammas/spark,taroplus/spark,wangmiao1981/spark,vinodkc/spark,skonto/spark,zuotingbing/spark,mahak/spark,rednaxelafx/apache-spark,srowen/spark,shaneknapp/spark,xuanyuanking/spark,nchammas/spark,jiangxb1987/spark,ConeyLiu/spark,cloud-fan/spark,dbtsai/spark,zzcclp/spark,JoshRosen/spark,dongjoon-hyun/spark,HyukjinKwon/spark,darionyaphet/spark,ptkool/spark,dbtsai/spark,jiangxb1987/spark,chuckchen/spark,hvanhovell/spark,xuanyuanking/spark,srowen/spark,wangyum/spark,mahak/spark,holdenk/spark,zzcclp/spark,JoshRosen/spark,zzcclp/spark,wangyum/spark,zuotingbing/spark,shuangshuangwang/spark,zuotingbing/spark,shuangshuangwang/spark,shuangshuangwang/spark,rednaxelafx/apache-spark,ptkool/spark,goldmedal/spark,cloud-fan/spark,zuotingbing/spark,BryanCutler/spark,goldmedal/spark,chuckchen/spark,witgo/spark,xuanyuanking/spark,srowen/spark,gengliangwang/spark,vinodkc/spark,maropu/spark,apache/spark,kevinyu98/spark,spark-test/spark,wzhfy/spark,HyukjinKwon/spark,mahak/spark,holdenk/spark,milliman/spark,maropu/spark,kevinyu98/spark,spark-test/spark,HyukjinKwon/spark,apache/spark,wzhfy/spark,WeichenXu123/spark,spark-test/spark,JoshRosen/spark,WeichenXu123/spark,apache/spark,shaneknapp/spark,witgo/spark,zzcclp/spark,zero323/spark,mahak/spark,vinodkc/spark,kevinyu98/spark,WeichenXu123/spark,maropu/spark,HyukjinKwon/spark,dongjoon-hyun/spark