Archive for September, 2010

Of frustration with build systems

September 28, 2010

Build Systems, Fortran & Python

Without question the most error-prone and dark corner-filled part of fwrap is the build system.  It’s based off of numpy’s distutils, but it is very fragile.  Fwrap demands a lot from a build system: (1) generate C, Fortran and Cython headers from an external python script, (2) compile Fortran 90 sources with module dependencies, (3) generate .c files from a .pyx, and (4) compile all the fortran and C source into a Python extension module.  Now that it is time to incorporate full-on Fortran 90 support in fwrap, distutils just plain doesn’t cut it.  I can either spend my time monkey patching things together or move to another build system.

Guess what I’ve decided to do.

One fundamental requirement is that the new build system not impose yet another external dependency.  So it either needs to already exist as part of every platform out there, or it needs to be distributed with fwrap.  This limits the choices (besides the already rejected distutils) to make or a new build tool named “waf”.

Waf is designed to be small and shipped as part of a larger project.  That way the host project has complete control over its build system and doesn’t have to worry about versioning issues.  Besides that, it is a proper build system (contrary to distutils), and it offers automatic dependency detection & resolution, especially desirable for Fortran 90/95 projects with modules.  I anticipate it taking fwrap’s most error prone component and making it much more robust.

In order to begin working on the new F90 stuff, waf needs to be incorporated into fwrap.  Once that is working, then I can set up the test suite to be module-aware, and move on to incorporating the new features.

Waf is suitable for any project, whether it be C, C++, Java, Go, Python (extensions), Cython, D, Fortran, whatever.  It easily handles mixed-language builds.  Adding support for new languages & compilers is pretty easy, too.  I’m somewhat new to it, but I plan on trying it out for some of my bigger codebases in the near future.

For more info:

http://code.google.com/p/waf/