[SPARK-35648][PYTHON] Refine and add dependencies needed for dev in dev/requirement.txt
[SPARK-35648][PYTHON] Refine and add dependencies needed for dev in dev/requirement.txt

### What changes were proposed in this pull request?

This PR proposes to update `dev/requirement.txt` file.

NOTE that:
- This file isn't used anywhere in Apache Spark CI. It's just for convenience
- To minimize the overhead of maintenance, I removed all lowerbounds of dependencies, which means that using the latest versions of them should work in the clean environment (e.g., you can reinstall all of them).

### Why are the changes needed?

To note the dependencies needed for Spark dev, and for easier env setting up.

### Does this PR introduce _any_ user-facing change?

No, dev-only.

### How was this patch tested?

Logically derived from setup.py, and other places like CI

Closes #32780 from HyukjinKwon/SPARK-35648.

Authored-by: Hyukjin Kwon <2680965ad5aae29275c8fb10150cb10f5554e78b@apache.org>
Signed-off-by: Hyukjin Kwon <2680965ad5aae29275c8fb10150cb10f5554e78b@apache.org>

jiangxb1987/spark,witgo/spark,cloud-fan/spark,vinodkc/spark,HyukjinKwon/spark,maropu/spark,cloud-fan/spark,ueshin/apache-spark,hvanhovell/spark,milliman/spark,WeichenXu123/spark,rednaxelafx/apache-spark,jiangxb1987/spark,shaneknapp/spark,HyukjinKwon/spark,witgo/spark,milliman/spark,srowen/spark,chuckchen/spark,gengliangwang/spark,HyukjinKwon/spark,HyukjinKwon/spark,chuckchen/spark,hvanhovell/spark,zzcclp/spark,apache/spark,wangmiao1981/spark,WeichenXu123/spark,taroplus/spark,cloud-fan/spark,apache/spark,zero323/spark,holdenk/spark,nchammas/spark,vinodkc/spark,zzcclp/spark,nchammas/spark,wangyum/spark,gengliangwang/spark,wangmiao1981/spark,maropu/spark,rednaxelafx/apache-spark,mahak/spark,rednaxelafx/apache-spark,milliman/spark,WeichenXu123/spark,shaneknapp/spark,dongjoon-hyun/spark,zzcclp/spark,apache/spark,wangyum/spark,zero323/spark,holdenk/spark,dongjoon-hyun/spark,xuanyuanking/spark,jiangxb1987/spark,nchammas/spark,nchammas/spark,ueshin/apache-spark,JoshRosen/spark,dongjoon-hyun/spark,jiangxb1987/spark,chuckchen/spark,wangmiao1981/spark,JoshRosen/spark,mahak/spark,jiangxb1987/spark,chuckchen/spark,zzcclp/spark,nchammas/spark,mahak/spark,gengliangwang/spark,holdenk/spark,wangmiao1981/spark,gengliangwang/spark,JoshRosen/spark,chuckchen/spark,zzcclp/spark,apache/spark,cloud-fan/spark,witgo/spark,mahak/spark,shaneknapp/spark,srowen/spark,vinodkc/spark,apache/spark,witgo/spark,wangyum/spark,taroplus/spark,cloud-fan/spark,gengliangwang/spark,milliman/spark,taroplus/spark,wangmiao1981/spark,zero323/spark,ueshin/apache-spark,jiangxb1987/spark,HyukjinKwon/spark,rednaxelafx/apache-spark,WeichenXu123/spark,srowen/spark,vinodkc/spark,apache/spark,holdenk/spark,srowen/spark,srowen/spark,xuanyuanking/spark,taroplus/spark,vinodkc/spark,xuanyuanking/spark,zero323/spark,witgo/spark,zero323/spark,WeichenXu123/spark,zero323/spark,apache/spark,holdenk/spark,hvanhovell/spark,wangmiao1981/spark,wangyum/spark,hvanhovell/spark,ueshin/apache-spark,chuckchen/spark,shaneknapp/spark,WeichenXu123/spark,vinodkc/spark,srowen/spark,dongjoon-hyun/spark,nchammas/spark,wangyum/spark,wangyum/spark,gengliangwang/spark,wangyum/spark,mahak/spark,JoshRosen/spark,JoshRosen/spark,milliman/spark,xuanyuanking/spark,holdenk/spark,rednaxelafx/apache-spark,hvanhovell/spark,xuanyuanking/spark,maropu/spark,ueshin/apache-spark,taroplus/spark,JoshRosen/spark,HyukjinKwon/spark,milliman/spark,rednaxelafx/apache-spark,vinodkc/spark,HyukjinKwon/spark,gengliangwang/spark,srowen/spark,hvanhovell/spark,WeichenXu123/spark,shaneknapp/spark,dongjoon-hyun/spark,zero323/spark,cloud-fan/spark,milliman/spark,xuanyuanking/spark,ueshin/apache-spark,maropu/spark,mahak/spark,maropu/spark,JoshRosen/spark,zzcclp/spark,taroplus/spark,shaneknapp/spark,zzcclp/spark,maropu/spark,maropu/spark,jiangxb1987/spark,shaneknapp/spark,witgo/spark,wangmiao1981/spark,cloud-fan/spark,mahak/spark,dongjoon-hyun/spark,holdenk/spark,ueshin/apache-spark,xuanyuanking/spark,dongjoon-hyun/spark,witgo/spark,hvanhovell/spark,taroplus/spark,chuckchen/spark,nchammas/spark,rednaxelafx/apache-spark