Archive for June, 2009

Update

June 18, 2009

I’m glad to say that the f2cy part of the GSoC miterm evaluation is functional. By that I mean it can wrap functions & subroutine subprograms (not in modules, yet) with any integer or real argument. The testing framework is all in place, and all tests pass. There are still many things that I’d like to add before the midterm, but the basics are all there, and the additions should be fairly modular and easy to do. The code could use some cleanup, but it shouldn’t be too bad.

Here are the some additions I’d like to make before the midterm if time allows — nothing to be inferred by the ordering except where noted:

1) Support locally defined parameter kind-types (easy), e.g.:

subroutine foo(a,b)
    integer :: DP
    parameter  (DP = 8 )
    integer, parameter :: SP = 4
    real(kind=DP), intent(inout) :: a
    real(kind=SP), intent(in) :: b
    ! ...
end subroutine

2) Next, support use’d module kind-type parameters (easy):

module ktps
  integer :: KTP
  parameter (KTP = 10)
end module ktps
integer function bar(a)
  use ktps
  real(kind=KTP), intent(inout) :: a
  ! ...
end function

3) Next, support module-scoped functions (some corner cases to consider, a bit harder) — just to give you an idea…

module container
  use ktps, local_kpts => KTPS
  integer, parameter :: C_KTPS = local_ktps
  contains
  subroutine mod_subr(a)
    use othermod, rename_p => orig_p
    use yetanother, only: yetanother_p => xxx
    implicit none
    integer(kind=C_KTPS) :: a
    integer(kind=other_mod_param), intent(out) :: b
    integer(kind=rename_p) :: c
    ! ...
  end subroutine
end module

4) Logical <-> Integer conversions (probably before the above)

The ‘logical’ type is basically a specially typed integer — whenever a subprogram’s argument is a logical, its C wrapper should use the equivalently-sized integer and do a conversion before & after the call.

5) Character arguments:

It seems that these are functionally equivalent to one-dimensional arrays, with a few bugaboos to watch out for. I’m lumping them in with the second-half of the project, since after working with integer/real arrays I’ll have a better feel as to the best way to handle character arrays.

6) Complex arguments:

Since we can’t assume C99 _Complex support, these need to be passed in as structs, converted to the native complex type in fortran, and converted back on return. Once (4) is done, I’ll have a better feel for these. I’m putting them off until the second half, since they’re equivalent to passing derived types.

Cython side:

I’m turning my full attention to tickets 299, 300 & 178 now, provided that the recent intense discussion about a native Cython array type leave these tickets unchanged (see http://article.gmane.org/gmane.comp.python.cython.devel/6654).

Long-overdue Update — ‘f2cy’

June 11, 2009

Much progress over the past few weeks, I’m glad to say.  Here are the main threads that are out there.

First, the 10000 meter view:  For my GSoC, I’m developing, under the guidance of Dag Sverre Seljebotn, (1) a python package that wraps fortran so it can be easily called from a cython program; (2) improving cython’s buffer support to allow the passing of buffers/arrays to external code (C, fortran, or whatever can be linked in) or cdef functions.

The fortran-side:

Dag Sverre and I discussed a bit about the naming of the fortran wrapper project — right now it’s under the working title “f2cy,” in the long tradition of “<some-language/protocol/format>2<other-language/protocol/format>” converter names.  There are at least 85 similarly-named executables on my ubuntu system (apropos 2 | awk ‘{ print $1 }’ | grep ‘[a-zA-Z]2[a-zA-Z]’ | wc -l).  Since the wrapper and the code it generates has very little direct dependence on python/cython, and since it may be useful for those who want to automate wrapping fortran inside a C program, the name will likely change.  I’m thinking of something along the lines of ‘fwrap’ — Dag suggested ‘fbridge’ to emphasize the connectivity aspect.

f2cy, which is using the fortran parser ‘fparser’ from the latest version of f2py (http://code.google.com/p/f2py/) is able to parse pretty much everything required for the midterm review.  The rest of f2cy is still rapidly developing.  The package lives inside the Tools directory of the cython distribution package. To take a look (it’s still in pre-alpha stage) checkout the mercurial repo here: http://hg.cython.org/gsoc-kurt/.

After fparser generates the parse tree, f2cy makes a few passes on the AST (using the Visitor pattern); one pass adds type ‘annotations’ to the dummy arguments of the subprograms to be wrapped, the next pass generates the autoconfig fortran source file, the next one generates the fortran wrapper.  Only scalar-valued arguments are supported for now, but by August f2cy will support arrays (assumed-shape, assumed-size and explicitly shaped) and passing derived types (structs) end-to-end from cython/C.  How is this done?  Thanks to the iso_c_binding intrinsic module within the Fortran 2003 standard, a Fortran program can be portably linked to C code without too much pain — certainly less pain than was required before.  This module allows f2cy to easily wrap subprograms that take scalars, arrays and derived types — in the distant future, f2cy may support passing C callbacks.  There still is an issue of resolving the kind-type-parameters for the dummy arguments, but that is what the autoconfig file is for.

Our goal in the project is to generate fortran wrappers that are completely portable, in the same way that Cython itself is portable.  A number of projects will generate an extension module with Cython and ship it without a Cython dependence, so the source has to work with any C compiler.  In the same way, f2py will generate wrappers that do not have any compiler-specific knowledge (including the particular values for the kind-type-parameters for the various types).  This isn’t purely academic, since some compilers define integer type KTPs consecutively (1,2,3,…) while for others (gfortran, xlf) the KTPs correspond to the byte-size (1,4,8,10,…).  This makes for some complications and makes the autoconfig file necessary.  More on the autoconfig in a future post.

I’ll address the cython-side in a short while.