[SPARK-39155][PYTHON] Access to JVM through passed-in GatewayClient during type conversion
[SPARK-39155][PYTHON] Access to JVM through passed-in GatewayClient during type conversion

### What changes were proposed in this pull request?
Access to JVM through passed-in GatewayClient during type conversion.

### Why are the changes needed?
In customized type converters, we may utilize the passed-in GatewayClient to access JVM, rather than rely on the `SparkContext._jvm`.

That's [how](https://github.com/py4j/py4j/blob/master/py4j-python/src/py4j/java_collections.py#L508) Py4J explicit converters access JVM.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Existing tests.

Closes #36504 from xinrong-databricks/gateway_client_jvm.

Authored-by: Xinrong Meng <f8e847d07d7307743be32495e91478c80c8e0110@databricks.com>
Signed-off-by: Hyukjin Kwon <2680965ad5aae29275c8fb10150cb10f5554e78b@apache.org>

ueshin/apache-spark,srowen/spark,ueshin/apache-spark,HyukjinKwon/spark,mahak/spark,holdenk/spark,srowen/spark,witgo/spark,cloud-fan/spark,ueshin/apache-spark,WeichenXu123/spark,witgo/spark,srowen/spark,WeichenXu123/spark,rednaxelafx/apache-spark,dongjoon-hyun/spark,srowen/spark,gengliangwang/spark,rednaxelafx/apache-spark,mahak/spark,WeichenXu123/spark,mahak/spark,vinodkc/spark,zzcclp/spark,zzcclp/spark,HyukjinKwon/spark,apache/spark,dongjoon-hyun/spark,vinodkc/spark,wangyum/spark,dongjoon-hyun/spark,srowen/spark,JoshRosen/spark,witgo/spark,holdenk/spark,zero323/spark,hvanhovell/spark,WeichenXu123/spark,vinodkc/spark,zzcclp/spark,shaneknapp/spark,apache/spark,JoshRosen/spark,JoshRosen/spark,apache/spark,cloud-fan/spark,nchammas/spark,zero323/spark,cloud-fan/spark,holdenk/spark,dongjoon-hyun/spark,shaneknapp/spark,shaneknapp/spark,vinodkc/spark,zero323/spark,zzcclp/spark,wangyum/spark,mahak/spark,cloud-fan/spark,gengliangwang/spark,shaneknapp/spark,gengliangwang/spark,hvanhovell/spark,HyukjinKwon/spark,cloud-fan/spark,cloud-fan/spark,zero323/spark,rednaxelafx/apache-spark,gengliangwang/spark,rednaxelafx/apache-spark,rednaxelafx/apache-spark,zero323/spark,WeichenXu123/spark,JoshRosen/spark,apache/spark,apache/spark,HyukjinKwon/spark,nchammas/spark,nchammas/spark,witgo/spark,holdenk/spark,zzcclp/spark,ueshin/apache-spark,WeichenXu123/spark,srowen/spark,wangyum/spark,nchammas/spark,JoshRosen/spark,zzcclp/spark,cloud-fan/spark,ueshin/apache-spark,JoshRosen/spark,gengliangwang/spark,zero323/spark,shaneknapp/spark,zzcclp/spark,apache/spark,nchammas/spark,WeichenXu123/spark,shaneknapp/spark,witgo/spark,holdenk/spark,holdenk/spark,hvanhovell/spark,wangyum/spark,dongjoon-hyun/spark,mahak/spark,nchammas/spark,vinodkc/spark,gengliangwang/spark,wangyum/spark,zero323/spark,srowen/spark,wangyum/spark,vinodkc/spark,HyukjinKwon/spark,nchammas/spark,holdenk/spark,rednaxelafx/apache-spark,ueshin/apache-spark,hvanhovell/spark,mahak/spark,HyukjinKwon/spark,vinodkc/spark,wangyum/spark,dongjoon-hyun/spark,ueshin/apache-spark,hvanhovell/spark,gengliangwang/spark,shaneknapp/spark,witgo/spark,rednaxelafx/apache-spark,hvanhovell/spark,HyukjinKwon/spark,witgo/spark,dongjoon-hyun/spark,JoshRosen/spark,mahak/spark,apache/spark,hvanhovell/spark