PEP: 102 Title: Doing Python Micro Releases Author: Anthony Baxter
<anthony@interlink.com.au>, Barry Warsaw <barry@python.org>, Guido van
Rossum <guido@python.org> Status: Superseded Type: Informational
Content-Type: text/x-rst Created: 09-Jan-2002 Post-History:
Superseded-By: 101

Replacement Note

Although the size of the to-do list in this PEP is much less scary than
that in PEP 101, it turns out not to be enough justification for the
duplication of information, and with it, the danger of one of the copies
to become out of date. Therefore, this PEP is not maintained anymore,
and micro releases are fully covered by PEP 101.

Abstract

Making a Python release is an arduous process that takes a minimum of
half a day's work even for an experienced releaser. Until recently, most
-- if not all -- of that burden was borne by Guido himself. But several
recent releases have been performed by other folks, so this PEP attempts
to collect, in one place, all the steps needed to make a Python bugfix
release.

The major Python release process is covered in PEP 101 - this PEP is
just PEP 101, trimmed down to only include the bits that are relevant
for micro releases, a.k.a. patch, or bug fix releases.

It is organized as a recipe and you can actually print this out and
check items off as you complete them.

How to Make A Release

Here are the steps taken to make a Python release. Some steps are more
fuzzy than others because there's little that can be automated (e.g.
writing the NEWS entries). Where a step is usually performed by An
Expert, the name of that expert is given. Otherwise, assume the step is
done by the Release Manager (RM), the designated person performing the
release. Almost every place the RM is mentioned below, this step can
also be done by the BDFL of course!

XXX: We should include a dependency graph to illustrate the steps that
can be taken in parallel, or those that depend on other steps.

We use the following conventions in the examples below. Where a release
number is given, it is of the form X.Y.MaA, e.g. 2.1.2c1 for Python
2.1.2 release candidate 1, where "a" == alpha, "b" == beta, "c" ==
release candidate. Final releases are tagged with "releaseXYZ" in CVS.
The micro releases are made from the maintenance branch of the major
release, e.g. Python 2.1.2 is made from the release21-maint branch.

1.  Send an email to python-dev@python.org indicating the release is
    about to start.

2.  Put a freeze on check ins into the maintenance branch. At this
    point, nobody except the RM should make any commits to the branch
    (or his duly assigned agents, i.e. Guido the BDFL, Fred Drake for
    documentation, or Thomas Heller for Windows). If the RM screwed up
    and some desperate last minute change to the branch is necessary, it
    can mean extra work for Fred and Thomas. So try to avoid this!

3.  On the branch, change Include/patchlevel.h in two places, to reflect
    the new version number you've just created. You'll want to change
    the PY_VERSION macro, and one or several of the version subpart
    macros just above PY_VERSION, as appropriate.

4.  Change the "%define version" line of Misc/RPM/python-2.3.spec to the
    same string as PY_VERSION was changed to above. E.g:

        %define version 2.3.1

    You also probably want to reset the %define release line to
    '1pydotorg' if it's not already that.

5.  If you're changing the version number for Python (e.g. from Python
    2.1.1 to Python 2.1.2), you also need to update the README file,
    which has a big banner at the top proclaiming its identity. Don't do
    this if you're just releasing a new alpha or beta release, but /do/
    do this if you're release a new micro, minor or major release.

6.  The LICENSE file also needs to be changed, due to several references
    to the release number. As for the README file, changing these are
    necessary for a new micro, minor or major release.

    The LICENSE file contains a table that describes the legal heritage
    of Python; you should add an entry for the X.Y.Z release you are now
    making. You should update this table in the LICENSE file on the CVS
    trunk too.

7.  When the year changes, copyright legends need to be updated in many
    places, including the README and LICENSE files.

8.  For the Windows build, additional files have to be updated.

    PCbuild/BUILDno.txt contains the Windows build number, see the
    instructions in this file how to change it. Saving the project file
    PCbuild/pythoncore.dsp results in a change to PCbuild/pythoncore.dsp
    as well.

    PCbuild/python20.wse sets up the Windows installer version resource
    (displayed when you right-click on the installer .exe and select
    Properties), and also contains the Python version number.

    (Before version 2.3.2, it was required to manually edit
    PC/python_nt.rc, this step is now automated by the build process.)

9.  After starting the process, the most important thing to do next is
    to update the Misc/NEWS file. Thomas will need this in order to do
    the Windows release and he likes to stay up late. This step can be
    pretty tedious, so it's best to get to it immediately after making
    the branch, or even before you've made the branch. The sooner the
    better (but again, watch for new checkins up until the release is
    made!)

    Add high level items new to this release. E.g. if we're releasing
    2.2a3, there must be a section at the top of the file explaining
    "What's new in Python 2.2a3". It will be followed by a section
    entitled "What's new in Python 2.2a2".

    Note that you /hope/ that as developers add new features to the
    trunk, they've updated the NEWS file accordingly. You can't be
    positive, so double check. If you're a Unix weenie, it helps to
    verify with Thomas about changes on Windows, and Jack Jansen about
    changes on the Mac.

    This command should help you (but substitute the correct -r tag!):

        % cvs log -rr22a1: | python Tools/scripts/logmerge.py > /tmp/news.txt

    IOW, you're printing out all the cvs log entries from the previous
    release until now. You can then troll through the news.txt file
    looking for interesting things to add to NEWS.

10. Check your NEWS changes into the maintenance branch. It's easy to
    forget to update the release date in this file!

11. Check in any changes to IDLE's NEWS.txt. Update the header in
    Lib/idlelib/NEWS.txt to reflect its release version and date. Update
    the IDLE version in Lib/idlelib/idlever.py to match.

12. Once the release process has started, the documentation needs to be
    built and posted on python.org according to the instructions in PEP
    101.

    Note that Fred is responsible both for merging doc changes from the
    trunk to the branch AND for merging any branch changes from the
    branch to the trunk during the cleaning up phase. Basically, if it's
    in Doc/ Fred will take care of it.

13. Thomas compiles everything with MSVC 6.0 SP5, and moves the
    python23.chm file into the src/chm directory. The installer
    executable is then generated with Wise Installation System.

    The installer includes the MSVC 6.0 runtime in the files MSVCRT.DLL
    and MSVCIRT.DLL. It leads to disaster if these files are taken from
    the system directory of the machine where the installer is built,
    instead it must be absolutely made sure that these files come from
    the VCREDIST.EXE redistributable package contained in the MSVC
    SP5 CD. VCREDIST.EXE must be unpacked with winzip, and the Wise
    Installation System prompts for the directory.

    After building the installer, it should be opened with winzip, and
    the MS dlls extracted again and check for the same version number as
    those unpacked from VCREDIST.EXE.

    Thomas uploads this file to the starship. He then sends the RM a
    notice which includes the location and MD5 checksum of the Windows
    executable.

    Note that Thomas's creation of the Windows executable may generate a
    few more commits on the branch. Thomas will be responsible for
    merging Windows-specific changes from trunk to branch, and from
    branch to trunk.

14. Sean performs his Red Hat magic, generating a set of RPMs. He
    uploads these files to python.org. He then sends the RM a notice
    which includes the location and MD5 checksum of the RPMs.

15. It's Build Time!

    Now, you're ready to build the source tarball. First cd to your
    working directory for the branch. E.g. % cd .../python-22a3

16. Do a "cvs update" in this directory. Do NOT include the -A flag!

    You should not see any "M" files, but you may see several "P" and/or
    "U" files. I.e. you better not have any uncommitted changes in your
    working directory, but you may pick up some of Fred's or Thomas's
    last minute changes.

17. Now tag the branch using a symbolic name like "rXYMaZ", e.g. r212

        % cvs tag r212

    Be sure to tag only the python/dist/src subdirectory of the Python
    CVS tree!

18. Change to a neutral directory, i.e. one in which you can do a fresh,
    virgin, cvs export of the branch. You will be creating a new
    directory at this location, to be named "Python-X.Y.M". Do a CVS
    export of the tagged branch.

        % cd ~
        % cvs -d cvs.sf.net:/cvsroot/python export -rr212 \
                              -d Python-2.1.2 python/dist/src

19. Generate the tarball. Note that we're not using the 'z' option on
    the tar command because 1) that's only supported by GNU tar as far
    as we know, and 2) we're going to max out the compression level,
    which isn't a supported option. We generate both tar.gz tar.bz2
    formats, as the latter is about 1/6th smaller.

        % tar -cf - Python-2.1.2 | gzip -9 > Python-2.1.2.tgz
        % tar -cf - Python-2.1.2 | bzip2 -9 > Python-2.1.2.tar.bz2

20. Calculate the MD5 checksum of the tgz and tar.bz2 files you just
    created

        % md5sum Python-2.1.2.tgz

    Note that if you don't have the md5sum program, there is a Python
    replacement in the Tools/scripts/md5sum.py file.

21. Create GPG keys for each of the files.

        % gpg -ba Python-2.1.2.tgz
        % gpg -ba Python-2.1.2.tar.bz2
        % gpg -ba Python-2.1.2.exe

22. Now you want to perform the very important step of checking the
    tarball you just created, to make sure a completely clean, virgin
    build passes the regression test. Here are the best steps to take:

        % cd /tmp
        % tar zxvf ~/Python-2.1.2.tgz
        % cd Python-2.1.2
        % ls
        (Do things look reasonable?)
        % ./configure
        (Loads of configure output)
        % make test
        (Do all the expected tests pass?)

    If the tests pass, then you can feel good that the tarball is fine.
    If some of the tests fail, or anything else about the freshly
    unpacked directory looks weird, you better stop now and figure out
    what the problem is.

23. You need to upload the tgz and the exe file to creosote.python.org.
    This step can take a long time depending on your network bandwidth.
    scp both files from your own machine to creosote.

24. While you're waiting, you can start twiddling the web pages to
    include the announcement.

    1.  In the top of the python.org web site CVS tree, create a
        subdirectory for the X.Y.Z release. You can actually copy an
        earlier patch release's subdirectory, but be sure to delete the
        X.Y.Z/CVS directory and "cvs add X.Y.Z", for example:

            % cd .../pydotorg
            % cp -r 2.2.2 2.2.3
            % rm -rf 2.2.3/CVS
            % cvs add 2.2.3
            % cd 2.2.3

    2.  Edit the files for content: usually you can globally replace
        X.Ya(Z-1) with X.YaZ. However, you'll need to think about the
        "What's New?" section.

    3.  Copy the Misc/NEWS file to NEWS.txt in the X.Y.Z directory for
        python.org; this contains the "full scoop" of changes to Python
        since the previous release for this version of Python.

    4.  Copy the .asc GPG signatures you created earlier here as well.

    5.  Also, update the MD5 checksums.

    6.  Preview the web page by doing a "make" or "make install" (as
        long as you've created a new directory for this release!)

    7.  Similarly, edit the ../index.ht file, i.e. the python.org home
        page. In the Big Blue Announcement Block, move the paragraph for
        the new version up to the top and boldify the phrase "Python
        X.YaZ is out". Edit for content, and preview locally, but do NOT
        do a "make install" yet!

25. Now we're waiting for the scp to creosote to finish. Da de da, da de
    dum, hmm, hmm, dum de dum.

26. Once that's done you need to go to creosote.python.org and move all
    the files in place over there. Our policy is that every Python
    version gets its own directory, but each directory may contain
    several releases. We keep all old releases, moving them into a
    "prev" subdirectory when we have a new release.

    So, there's a directory called "2.2" which contains Python-2.2a2.exe
    and Python-2.2a2.tgz, along with a "prev" subdirectory containing
    Python-2.2a1.exe and Python-2.2a1.tgz.

    So...

    1.  On creosote, cd to ~ftp/pub/python/X.Y creating it if necessary.
    2.  Move the previous release files to a directory called "prev"
        creating the directory if necessary (make sure the directory has
        g+ws bits on). If this is the first alpha release of a new
        Python version, skip this step.
    3.  Move the .tgz file and the .exe file to this directory. Make
        sure they are world readable. They should also be group
        writable, and group-owned by webmaster.
    4.  md5sum the files and make sure they got uploaded intact.

27. the X.Y/bugs.ht file if necessary. It is best to get BDFL input for
    this step.

28. Go up to the parent directory (i.e. the root of the web page
    hierarchy) and do a "make install" there. You're release is now
    live!

29. Now it's time to write the announcement for the mailing lists. This
    is the fuzzy bit because not much can be automated. You can use one
    of Guido's earlier announcements as a template, but please edit it
    for content!

    Once the announcement is ready, send it to the following addresses:

        python-list@python.org
        python-announce@python.org
        python-dev@python.org

30. Send a SourceForge News Item about the release. From the project's
    "menu bar", select the "News" link; once in News, select the
    "Submit" link. Type a suitable subject (e.g. "Python 2.2c1 released"
    :-) in the Subject box, add some text to the Details box (at the
    very least including the release URL at www.python.org and the fact
    that you're happy with the release) and click the SUBMIT button.

    Feel free to remove any old news items.

Now it's time to do some cleanup. These steps are very important!

1.  Edit the file Include/patchlevel.h so that the PY_VERSION string
    says something like "X.YaZ+". Note the trailing '+' indicating that
    the trunk is going to be moving forward with development. E.g. the
    line should look like:

        #define PY_VERSION              "2.1.2+"

    Make sure that the other PY_ version macros contain the correct
    values. Commit this change.

2.  For the extra paranoid, do a completely clean test of the release.
    This includes downloading the tarball from www.python.org.

3.  Make sure the md5 checksums match. Then unpack the tarball, and do a
    clean make test.

        % make distclean
        % ./configure
        % make test

    To ensure that the regression test suite passes. If not, you screwed
    up somewhere!

Step 5 ...

Verify! This can be interleaved with Step 4. Pretend you're a user:
download the files from python.org, and make Python from it. This step
is too easy to overlook, and on several occasions we've had useless
release files. Once a general server problem caused mysterious
corruption of all files; once the source tarball got built incorrectly;
more than once the file upload process on SF truncated files; and so on.

What Next?

Rejoice. Drink. Be Merry. Write a PEP like this one. Or be like unto
Guido and take A Vacation.

You've just made a Python release!

Actually, there is one more step. You should turn over ownership of the
branch to Jack Jansen. All this means is that now he will be responsible
for making commits to the branch. He's going to use this to build the
MacOS versions. He may send you information about the Mac release that
should be merged into the informational pages on www.python.org. When
he's done, he'll tag the branch something like "rX.YaZ-mac". He'll also
be responsible for merging any Mac-related changes back into the trunk.

Final Release Notes

The Final release of any major release, e.g. Python 2.2 final, has
special requirements, specifically because it will be one of the longest
lived releases (i.e. betas don't last more than a couple of weeks, but
final releases can last for years!).

For this reason we want to have a higher coordination between the three
major releases: Windows, Mac, and source. The Windows and source
releases benefit from the close proximity of the respective
release-bots. But the Mac-bot, Jack Jansen, is 6 hours away. So we add
this extra step to the release process for a final release:

1.  Hold up the final release until Jack approves, or until we lose
    patience <wink>.

The python.org site also needs some tweaking when a new bugfix release
is issued.

2.  The documentation should be installed at doc/<version>/.
3.  Add a link from doc/<previous-minor-release>/index.ht to the
    documentation for the new version.
4.  All older doc/<old-release>/index.ht files should be updated to
    point to the documentation for the new version.
5.  /robots.txt should be modified to prevent the old version's
    documentation from being crawled by search engines.

Windows Notes

Windows has a GUI installer, various flavors of Windows have "special
limitations", and the Windows installer also packs precompiled "foreign"
binaries (Tcl/Tk, expat, etc). So Windows testing is tiresome but very
necessary.

Concurrent with uploading the installer, Thomas installs Python from it
twice: once into the default directory suggested by the installer, and
later into a directory with embedded spaces in its name. For each
installation, he runs the full regression suite from a DOS box, and both
with and without -0.

He also tries every shortcut created under Start -> Menu -> the Python
group. When trying IDLE this way, you need to verify that Help -> Python
Documentation works. When trying pydoc this way (the "Module Docs" Start
menu entry), make sure the "Start Browser" button works, and make sure
you can search for a random module (Thomas uses "random" <wink>) and
then that the "go to selected" button works.

It's amazing how much can go wrong here -- and even more amazing how
often last-second checkins break one of these things. If you're "the
Windows geek", keep in mind that you're likely the only person routinely
testing on Windows, and that Windows is simply a mess.

Repeat all of the above on at least one flavor of Win9x, and one of
NT/2000/XP. On NT/2000/XP, try both an Admin and a plain User (not Power
User) account.

WRT Step 5 above (verify the release media), since by the time release
files are ready to download Thomas has generally run many Windows tests
on the installer he uploaded, he usually doesn't do anything for Step 5
except a full byte-comparison ("fc /b" if using a Windows shell) of the
downloaded file against the file he uploaded.

Copyright

This document has been placed in the public domain.