[Build System] Workaround MacOS 10.13+ and XCode Python 3.8+ fork issue in pyspark
[Build System] Workaround MacOS 10.13+ and XCode Python 3.8+ fork issue in pyspark

From:

http://www.sealiesoftware.com/blog/archive/2017/6/5/Objective-C_and_fork_in_macOS_1013.html

> The rules for using Objective-C between fork() and exec() have changed
in macOS 10.13. Incorrect code that happened to work most of the time in
the past may now fail. Some workarounds are available.

This led to issues within the Python language, tracked at
https://bugs.python.org/issue33725. Any application that makes use of
e.g. multiprocessing (or in any way ends up calling MacOS system `fork`
or `exec`) is affected. According to the Python bug tracker, this issue
has been fixed in Python 3.8

The XCode Python 3.8 version begs to differ. In a very specific usecase,
that boils down to this simple reproducer:
```python
import pyspark

sparkconf = pyspark.SparkConf().setAll(
{"spark.app.name": "distrdf001_spark_connection",
 "spark.master": "local[4]", }.items())
sparkcontext = pyspark.SparkContext(conf=sparkconf)

def imp(partition):
import cppyy
return 1

count = sparkcontext.parallelize(range(1)).map(imp).reduce(lambda x,y:
x+y)
```

the issue is still triggered. Installing any other 3.8+ Python
executable on a MacOS node doesn't show this behaviour, with the exact
same pyspark version (3.2.1).

It is possible to workaround this problem, by setting the environment
variable OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES before running a test.
This effectively brings back the behaviour of MacOS10.12 and previous
versions.  This commit uses this workaround, since we cannot directly
modify how the system XCode Python on MacOS works.

To give an idea of the impact of this issue (and therefore the changes
in this commit), the situation where this occurs is as follows:

1. A user wants to run RDataFrame in distributed mode.
2. Their environment is MacOS 10.13+.
3. The application creates a pyspark mock cluster on the local Mac machine and
the computations happen strictly in the single local node, not in a
distributed cluster.

Thus, it probably interests only a very small fraction of use cases for
distributed RDataFrame, mainly regarding quick checks done locally before
actually sending the computations to a cluster.

Co-authored-by: Enric Tejedor Saavedra <8cdebc76d1dbdfa04c5a8fcb60c8a49335a107d5@cern.ch>

olifre/root,olifre/root,olifre/root,olifre/root,olifre/root,olifre/root,olifre/root,olifre/root,olifre/root,olifre/root,olifre/root