Showing posts with label extension. Show all posts
Showing posts with label extension. Show all posts

Wednesday, January 09, 2008

Compiling Python extension with GCC

In my previous post I've described how to make C code accessible from Python. I used Visual C++ compiler cl.exe to build an extension (or module) for Python. This follow-up shows how to compile the same extension for windows using GCC. I bet you already know what GCC is and that it is available from MinGW install as a result of install procedure described long ago.

Grab the source from the previous post - it won't change. Everything what is going to happen are just changes in .exe and its command line options. Saving source as farpython.c and starting GCC to compile it:

gcc farpython.c

As usual, this won't produce anything useful except errors.

farpython.c:14:20: Python.h: No such file or directory
farpython.c:18: error: syntax error before '*' token
farpython.c:19: error: syntax error before '*' token
...

Additional include search path is specified using -I option in GCC.

gcc -IE:\ENV\Python25\include farpython.c

A different picture, but the output is still grim.

D:\Temp/ccyOaaaa.o(.text+0x1c):farpython.c: undefined reference to `_imp__PyArg_ParseTuple'
D:\Temp/ccyOaaaa.o(.text+0x4c):farpython.c: undefined reference to `_imp__Py_BuildValue'
D:\Temp/ccyOaaaa.o(.text+0x88):farpython.c: undefined reference to `_imp__Py_InitModule4'
D:\Temp/ccyOaaaa.o(.text+0xbe):farpython.c: undefined reference to `_imp__Py_InitModule4'
E:/ENV/MSYS/mingw/bin/../lib/gcc/mingw32/3.4.2/../../../libmingw32.a(main.o)(.text+0x106):main.c: undefined reference to `
WinMain@16'
collect2: ld returned 1 exit status

Luckily these errors are not concerned with the code. They are from linker (ld) complaining it could not find library with binaries for functions defined in Python.h. The last one about undefined reference to WinMain is different though, but let's skip it until we deal with missing libraries. Python libraries are located at E:\ENV\Python25\libs and option to GCC is -L.

gcc -IE:\ENV\Python25\include -LE:\ENV\Python25\libs farpython.c

The output is still the same. The problem here is that linker doesn't know which specific library we need to link with to get binary bits for Python.h functions. cl.exe from Visual C++ was able to detect the correct library somehow, but for GCC we have to specify its name explicitly with -l option. Note that this option goes after a name of all compiled .c files. It is because params up to and including .c files are for compiler component and everything that goes after can be treated as linker's.

gcc -IE:\ENV\Python25\include -LE:\ENV\Python25\libs farpython.c -lpython25

Check the output.

E:/ENV/MSYS/mingw/bin/../lib/gcc/mingw32/3.4.2/../../../libmingw32.a(main.o)(.text+0x106):main.c: undefined reference to `
WinMain@16'
collect2: ld returned 1 exit status

WinMain is an entrypoint or starting point of any program on windows platform, but Python extension is not a program that starts execution itself. .pyd is a .dll, or shared library with functions to be called by other programs. To tell that to GCC we add -shared switch to command line.

gcc -IE:\ENV\Python25\include -LE:\ENV\Python25\libs farpython.c -lpython25 -shared

Now everything seems fine, but instead of far.pyd or farpython.pyd we've got a.exe Default output filename is easily corrected with yet another option -o

gcc -IE:\ENV\Python25\include -LE:\ENV\Python25\libs farpython.c -lpython25 -shared -o far.pyd

Test.

E:\>python
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import far
>>> far.example("echo")
ECHO is on.
0

Monday, May 28, 2007

How to compile pysqlite for Windows

If you are reading this page that means you know that pysqlite is and probably need updated version of this python extension for windows compiled against newer version of SQLite. Why would anyone need it? Perhaps because of fixed bugs or SQLAlchemy complaining that you need a better SQLite version as it happened in my case.

Original instructions are available for discontinued MS Toolkit Compiler at http://initd.org/pub/software/pysqlite/doc/install-source-win32.html and these are for MinGW compiler. First of all you need to install MinGW somewhere and preferably add directory with gcc.exe into %PATH%. Then unpack pysqslite sources, edit setup.cfg to look like this:

[build_ext]
define=
include_dirs=sqlite3
library_dirs=sqlite3
libraries=sqlite3

That means sqlite3/ directory should contain additional includes and libraries required to compile the extension. These are sqlite3.dll and sqlite3.h available at SQLite downloads from sqlite-source-x_x_x.zip and sqlitedll-3_3_17.zip archives. Unpack them into sqlite3/subdirectory of already unpacked pysqlite source. Now issue:

setup.py build --compiler=mingw32

If everything compiles Ok patch setup.py to carry sqlite3.dll inside to avoid messages about missing .dll

--- pysqlite-2.3.3/setup_old.py Sun Jan 14 02:56:12 2007
+++ pysqlite-2.3.3/setup.py Mon May 28 15:59:05 2007
@@ -109,7 +109,9 @@
+ glob.glob("doc/*.txt") \
+ glob.glob("doc/*.css")),
("pysqlite2-doc/code",
- glob.glob("doc/code/*.py"))]
+ glob.glob("doc/code/*.py")),
+ ("Lib/site-packages/pysqlite2",
+ ["sqlite3/sqlite3.dll"])]

py_modules = ["sqlite"]
setup_args = dict(


Then build an installer to save your work for the future. I saved mine at http://rainforce.org/sqlite/bindings/

setup.py bdist_wininst