PEP: 229 Title: Using Distutils to Build Python Version: $Revision$
Last-Modified: $Date$ Author: A.M. Kuchling <amk@amk.ca> Status: Final
Type: Standards Track Content-Type: text/x-rst Created: 16-Nov-2000
Python-Version: 2.1 Post-History:

Introduction

The Modules/Setup mechanism has some flaws:

-   People have to remember to uncomment bits of Modules/Setup in order
    to get all the possible modules.
-   Moving Setup to a new version of Python is tedious; new modules have
    been added, so you can't just copy the older version, but have to
    reconcile the two versions.
-   Users have to figure out where the needed libraries, such as zlib,
    are installed.

Proposal

Use the Distutils to build the modules that come with Python.

The changes can be broken up into several pieces:

1.  The Distutils needs some Python modules to be able to build modules.
    Currently I believe the minimal list is posix, _sre, and string.

    These modules will have to be built before the Distutils can be
    used, so they'll simply be hardwired into Modules/Makefile and be
    automatically built.

2.  A top-level setup.py script will be written that checks the
    libraries installed on the system and compiles as many modules as
    possible.

3.  Modules/Setup will be kept and settings in it will override
    setup.py's usual behavior, so you can disable a module known to be
    buggy, or specify particular compilation or linker flags. However,
    in the common case where setup.py works correctly, everything in
    Setup will remain commented out. The other Setup.* become
    unnecessary, since nothing will be generating Setup automatically.

The patch was checked in for Python 2.1, and has been subsequently
modified.

Implementation

Patch #102588 on SourceForge contains the proposed patch. Currently the
patch tries to be conservative and to change as few files as possible,
in order to simplify backing out the patch. For example, no attempt is
made to rip out the existing build mechanisms. Such simplifications can
wait for later in the beta cycle, when we're certain the patch will be
left in, or they can wait for Python 2.2.

The patch makes the following changes:

-   Makes some required changes to distutils/sysconfig (these will be
    checked in separately)
-   In the top-level Makefile.in, the "sharedmods" target simply runs
    "./python setup.py build", and "sharedinstall" runs
    "./python setup.py install". The "clobber" target also deletes the
    build/ subdirectory where Distutils puts its output.
-   Modules/Setup.config.in only contains entries for the gc and thread
    modules; the readline, curses, and db modules are removed because
    it's now setup.py's job to handle them.
-   Modules/Setup.dist now contains entries for only 3 modules --_sre,
    posix, and strop.
-   The configure script builds setup.cfg from setup.cfg.in. This is
    needed for two reasons: to make building in subdirectories work, and
    to get the configured installation prefix.
-   Adds setup.py to the top directory of the source tree. setup.py is
    the largest piece of the puzzle, though not the most complicated.
    setup.py contains a subclass of the BuildExt class, and extends it
    with a detect_modules() method that does the work of figuring out
    when modules can be compiled, and adding them to the 'exts' list.

Unresolved Issues

Do we need to make it possible to disable the 3 hard-wired modules
without manually hacking the Makefiles? [Answer: No.]

The Distutils always compile modules as shared libraries. How do we
support compiling them statically into the resulting Python binary?

[Answer: building a Python binary with the Distutils should be feasible,
though no one has implemented it yet. This should be done someday, but
isn't a pressing priority as messing around with the top-level
Makefile.pre.in is good enough.]

Copyright

This document has been placed in the public domain.