[SPARK-38189][K8S][DOC] Add `Priority scheduling` doc for Spark on K8S
[SPARK-38189][K8S][DOC] Add `Priority scheduling` doc for Spark on K8S

### What changes were proposed in this pull request?
Document how to set the priority class with the pod template.

### Why are the changes needed?
Currently, we didn't have a certain doc to help user enable priority scheduling

Related: https://github.com/apache/spark/pull/35716 https://github.com/apache/spark/pull/35639#issuecomment-1055847723

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
![image](https://user-images.githubusercontent.com/1736354/156696247-cf2cf566-57a0-4b8a-a18f-aa300a6f6a3d.png)

Closes #35728 from Yikun/SPARK-38189-doc.

Authored-by: Yikun Jiang <2206089eaa6cfa02a44178196e5767f367cbca91@gmail.com>
Signed-off-by: Dongjoon Hyun <6ff48b33cbb828151035af58a92f18a39f1d1036@apache.org>

zzcclp/spark,dongjoon-hyun/spark,JoshRosen/spark,gengliangwang/spark,ueshin/apache-spark,WeichenXu123/spark,JoshRosen/spark,HyukjinKwon/spark,nchammas/spark,rednaxelafx/apache-spark,mahak/spark,ueshin/apache-spark,srowen/spark,srowen/spark,mahak/spark,witgo/spark,zzcclp/spark,nchammas/spark,gengliangwang/spark,gengliangwang/spark,ueshin/apache-spark,WeichenXu123/spark,holdenk/spark,shaneknapp/spark,dongjoon-hyun/spark,vinodkc/spark,WeichenXu123/spark,JoshRosen/spark,HyukjinKwon/spark,hvanhovell/spark,apache/spark,srowen/spark,HyukjinKwon/spark,JoshRosen/spark,srowen/spark,zero323/spark,cloud-fan/spark,WeichenXu123/spark,gengliangwang/spark,hvanhovell/spark,srowen/spark,nchammas/spark,JoshRosen/spark,shaneknapp/spark,WeichenXu123/spark,vinodkc/spark,HyukjinKwon/spark,zzcclp/spark,holdenk/spark,mahak/spark,rednaxelafx/apache-spark,wangyum/spark,rednaxelafx/apache-spark,cloud-fan/spark,zero323/spark,cloud-fan/spark,hvanhovell/spark,witgo/spark,shaneknapp/spark,HyukjinKwon/spark,rednaxelafx/apache-spark,WeichenXu123/spark,mahak/spark,zero323/spark,holdenk/spark,vinodkc/spark,wangyum/spark,JoshRosen/spark,apache/spark,zzcclp/spark,shaneknapp/spark,apache/spark,zzcclp/spark,holdenk/spark,witgo/spark,wangyum/spark,zero323/spark,nchammas/spark,WeichenXu123/spark,zzcclp/spark,vinodkc/spark,mahak/spark,ueshin/apache-spark,ueshin/apache-spark,vinodkc/spark,nchammas/spark,rednaxelafx/apache-spark,dongjoon-hyun/spark,wangyum/spark,holdenk/spark,vinodkc/spark,vinodkc/spark,nchammas/spark,dongjoon-hyun/spark,cloud-fan/spark,JoshRosen/spark,witgo/spark,wangyum/spark,rednaxelafx/apache-spark,ueshin/apache-spark,rednaxelafx/apache-spark,holdenk/spark,nchammas/spark,witgo/spark,zzcclp/spark,mahak/spark,HyukjinKwon/spark,srowen/spark,cloud-fan/spark,apache/spark,zero323/spark,zero323/spark,dongjoon-hyun/spark,shaneknapp/spark,hvanhovell/spark,wangyum/spark,mahak/spark,witgo/spark,hvanhovell/spark,ueshin/apache-spark,wangyum/spark,srowen/spark,zero323/spark,holdenk/spark,hvanhovell/spark,shaneknapp/spark,apache/spark,apache/spark,dongjoon-hyun/spark,gengliangwang/spark,hvanhovell/spark,shaneknapp/spark,dongjoon-hyun/spark,apache/spark,cloud-fan/spark,HyukjinKwon/spark,gengliangwang/spark,gengliangwang/spark,cloud-fan/spark,witgo/spark