Wednesday, January 09, 2008

Compiling Python extension with GCC

In my previous post I've described how to make C code accessible from Python. I used Visual C++ compiler cl.exe to build an extension (or module) for Python. This follow-up shows how to compile the same extension for windows using GCC. I bet you already know what GCC is and that it is available from MinGW install as a result of install procedure described long ago.

Grab the source from the previous post - it won't change. Everything what is going to happen are just changes in .exe and its command line options. Saving source as farpython.c and starting GCC to compile it:

gcc farpython.c

As usual, this won't produce anything useful except errors.

farpython.c:14:20: Python.h: No such file or directory
farpython.c:18: error: syntax error before '*' token
farpython.c:19: error: syntax error before '*' token
...

Additional include search path is specified using -I option in GCC.

gcc -IE:\ENV\Python25\include farpython.c

A different picture, but the output is still grim.

D:\Temp/ccyOaaaa.o(.text+0x1c):farpython.c: undefined reference to `_imp__PyArg_ParseTuple'
D:\Temp/ccyOaaaa.o(.text+0x4c):farpython.c: undefined reference to `_imp__Py_BuildValue'
D:\Temp/ccyOaaaa.o(.text+0x88):farpython.c: undefined reference to `_imp__Py_InitModule4'
D:\Temp/ccyOaaaa.o(.text+0xbe):farpython.c: undefined reference to `_imp__Py_InitModule4'
E:/ENV/MSYS/mingw/bin/../lib/gcc/mingw32/3.4.2/../../../libmingw32.a(main.o)(.text+0x106):main.c: undefined reference to `
WinMain@16'
collect2: ld returned 1 exit status

Luckily these errors are not concerned with the code. They are from linker (ld) complaining it could not find library with binaries for functions defined in Python.h. The last one about undefined reference to WinMain is different though, but let's skip it until we deal with missing libraries. Python libraries are located at E:\ENV\Python25\libs and option to GCC is -L.

gcc -IE:\ENV\Python25\include -LE:\ENV\Python25\libs farpython.c

The output is still the same. The problem here is that linker doesn't know which specific library we need to link with to get binary bits for Python.h functions. cl.exe from Visual C++ was able to detect the correct library somehow, but for GCC we have to specify its name explicitly with -l option. Note that this option goes after a name of all compiled .c files. It is because params up to and including .c files are for compiler component and everything that goes after can be treated as linker's.

gcc -IE:\ENV\Python25\include -LE:\ENV\Python25\libs farpython.c -lpython25

Check the output.

E:/ENV/MSYS/mingw/bin/../lib/gcc/mingw32/3.4.2/../../../libmingw32.a(main.o)(.text+0x106):main.c: undefined reference to `
WinMain@16'
collect2: ld returned 1 exit status

WinMain is an entrypoint or starting point of any program on windows platform, but Python extension is not a program that starts execution itself. .pyd is a .dll, or shared library with functions to be called by other programs. To tell that to GCC we add -shared switch to command line.

gcc -IE:\ENV\Python25\include -LE:\ENV\Python25\libs farpython.c -lpython25 -shared

Now everything seems fine, but instead of far.pyd or farpython.pyd we've got a.exe Default output filename is easily corrected with yet another option -o

gcc -IE:\ENV\Python25\include -LE:\ENV\Python25\libs farpython.c -lpython25 -shared -o far.pyd

Test.

E:\>python
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import far
>>> far.example("echo")
ECHO is on.
0

4 comments:

  1. I'm using Python 2.6.5 on Windows 7, trying to build a C or C++ extension for Python. I followed your orders, but I needed to create a libpython26.a from python26.dll (using pexports and dlltool) in order to get rid of the "undefined reference" errors.
    But still after that, exaclty one error remains: `_imp__Py_InitModule4'.

    The same code works fine on linux with python 2.5.2.

    Any ideas?

    ReplyDelete
  2. @thesaint4cs:

    I realize your post was over a year old, but in case anyone else is searching:
    What a hassle. I finally found that it works (with python27), if you get the .def file elsewhere instead of extracting it from python27.dll with pexports. From what I've read, I think it's a problem with 64 bit installations.

    You can get the .def files from:
    http://wiki.cython.org/InstallingOnWindows (near the bottom of the page)

    If anyone wants instructions on pexports/dlltool:
    http://www.mingw.org/wiki/FAQ - scroll to "Other -> How do I create python extensions?"

    ReplyDelete