Archive for January, 2010

Fwrap update and compilation infrastructure

January 14, 2010

It’s been a while since an update.

Events out of my control have consumed my discretionary time, events that have conspired to delay fwrap’s delivery.  The usual story.  I lean on your patience, and I’m glad to know there is still interest out there.

A critical component of fwrap is its compilation infrastructure.  I’ve been thinking about this a lot lately; any input you care to share with me is very welcome.  At first I was using a simple Makefile for the automated integration tests on my box here.  That’s fine for testing, but won’t cut it for all the different systems out there.  I’ve since moved to a monkeypatched Cython-numpy distutils script that gets the job done, but isn’t the shiniest bit of code I’ve written.

The compilation system has to do a number of disparate things:

  1. Somehow generate the ISO C BINDING mappings from Fortran datatype to C datatype.  For some setups, this may require compiling and running a Fortran source file.  I’m informed by some (David Cournapeau, who has some expertise in this area) that doing this on some platforms is less than ideal.  There will be other setups, though.  More on this in a future post.
  2. Cythonize the Cython source files.
  3. Compile the Fortran source with the appropriate flags.
  4. Compile the C source.
  5. Create the extension module.

So the compilation script will, in an ideal world, be able to compile and run a Fortran source file, cythonize Cython code, compile C & Fortran code, and compile an extension module.  All while being intelligently adaptable to different command-line flags to customize different Fortran compilers & compilation flags, etc.  A tall order.

Here are the candidates, as I see it:

  1. Cython-numpy distutils — get this in shape and make it robust.  This encapsulates lots of work done in numpy.distutils.fcompiler  to automate all the different fortran compilers out there on all different platforms.  I’d be a fool to not make use of it.  On the other hand, distutils stuff can be ugly and error-prone, in my limited experience.  A benefit is that it would require just Cython and Numpy; Cython will be a dependency of fwrap, and Numpy very well might be.  So no extra dependencies, which is great.  All in all, though, I’d like something more robust out of the box, something like…
  2. Scons — I’ve been favorably disposed to what I’ve seen of scons, and I’d love to use it.  I’ve not experimented with its fortran support yet, although browsing its source shows some promise.  I see a number of different fortran compilers there that are supported, although not all (pathscale, xlf, others…)  It would be an extra dependency — is it worth it?  And how hard would it be to support other fortran compilers?  For that I’m looking into…
  3. Numscons — (  This likely is the bees’ knees of what I’m looking for, a combination of scons with numerical compilation.  We know it works for numpy so there’s one success already.  Is it mature?  Will it be supported in the future?
  4. Makefiles (on systems that would support it) — I’m Makefile literate, but no guru.  And it may be a minefield with all the different systems out there, i.e. it might require a guru.

If anyone has the competence to comment on the above I’m all ears.