Of frustration with build systems

Build Systems, Fortran & Python

Without question the most error-prone and dark corner-filled part of fwrap is the build system.  It’s based off of numpy’s distutils, but it is very fragile.  Fwrap demands a lot from a build system: (1) generate C, Fortran and Cython headers from an external python script, (2) compile Fortran 90 sources with module dependencies, (3) generate .c files from a .pyx, and (4) compile all the fortran and C source into a Python extension module.  Now that it is time to incorporate full-on Fortran 90 support in fwrap, distutils just plain doesn’t cut it.  I can either spend my time monkey patching things together or move to another build system.

Guess what I’ve decided to do.

One fundamental requirement is that the new build system not impose yet another external dependency.  So it either needs to already exist as part of every platform out there, or it needs to be distributed with fwrap.  This limits the choices (besides the already rejected distutils) to make or a new build tool named “waf”.

Waf is designed to be small and shipped as part of a larger project.  That way the host project has complete control over its build system and doesn’t have to worry about versioning issues.  Besides that, it is a proper build system (contrary to distutils), and it offers automatic dependency detection & resolution, especially desirable for Fortran 90/95 projects with modules.  I anticipate it taking fwrap’s most error prone component and making it much more robust.

In order to begin working on the new F90 stuff, waf needs to be incorporated into fwrap.  Once that is working, then I can set up the test suite to be module-aware, and move on to incorporating the new features.

Waf is suitable for any project, whether it be C, C++, Java, Go, Python (extensions), Cython, D, Fortran, whatever.  It easily handles mixed-language builds.  Adding support for new languages & compilers is pretty easy, too.  I’m somewhat new to it, but I plan on trying it out for some of my bigger codebases in the near future.

For more info:



3 Responses to “Of frustration with build systems”

  1. anonymous coward Says:

    Maybe worth checking David Cournapeau’s work on bento


    • cournape Says:

      I think waf is the most promising the solution for building complex software in the scipy community. Bento itself is developer with the idea that you will be able to use waf for building, and bento takes care of python-specific packaging (sdist, eggs, etc…).

  2. Victor Says:

    Looks like reinventing the wheel is open source community’s favorite way to kill time.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: