This tutorial will try to teach and explain steps necessary to make a module in C for Python to call C code from Python program so that later it could be complemented with another tutorial to illustrate steps to call Python from C program. While the order of lessons could be reversed it is really better to start from writing a module (extending Python) to gain general understanding of how Python and C work together.
Let's start with example from http://docs.python.org/ext/simpleExample.html to build a module named "far". We'll define a function "example" that should be accessible from Python interpreter.
// Step 1: A simple example
#include <Python.h>
static PyObject *
far_example(PyObject *self, PyObject *args) {
const char* command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command)) {
return NULL;
}
sts = system(command);
return Py_BuildValue("i", sts);
}
Save example to farpython.c Now we need to compile it somehow. I use cl.exe - Microsoft Visual C++ compiler. It is possible to use other compilers (like GCC) too even though there are warnings http://www.python.org/doc/ext/win-cookbook.html that Python module should be compiled with the same version of compiler that was used to build Python itself. I am not sure if the official distribution of Python 2.5.1 was compiled with my version of VC++ but I gave it a try and it worked. Another day I tried to compile the same code with GCC and it worked too.
To compile we usually do:
cl.exe farpython.c
But in most cases this won't work,
farpython.c(4) : fatal error C1083: Cannot open include file: 'Python.h': No such file or directory
we need to specify where to find Python.h
cl.exe /IE:\ENV\Python25\include farpython.c
This won't produce anything usable either,
LINK : fatal error LNK1104: cannot open file "python25.lib"
because we also need to specify where to find accompanying library with binary code to link with. The required option below is used by the linker. cl.exe treats all options as "for the linker" if they come after /link parameter, which in turn comes after our .c filename.
cl.exe /IE:\ENV\Python25\include farpython.c /link /libpath:E:\ENV\Python25\libs
This won't work too. Nice, eh?
/out:farpython.exe
/libpath:E:\ENV\Python25\libs
farpython.obj
LINK : fatal error LNK1561: entry point must be defined
This means that another key should be passed to the linker. The one that'll tell it that we are creating module or dll and not executable program, so there is no entrypoint to look for.
cl.exe /IE:\ENV\Python25\include farpython.c /link /dll /libpath:E:\ENV\Python25\libs
No errors. Great!
/out:farpython.exe
/dll
/libpath:E:\ENV\Python25\libs
farpython.obj
well, almost. Resulting farpython.exe "can not be executed", because it is actually a dll. Python interpreter wouldn't be able to do anything with it even if it was named farpython.dll To be recognized as Python module the file should have .pyd extension. Let's check this by launching Python from the same directory.
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import farpython
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named farpython
>>> from shutil import copy
>>> copy("farpython.dll", "farpython.pyd")
>>> import farpython
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: dynamic module does not define init function (initfarpython)
While it is possible to rename the file manually, we will add another option for the linker to do the job automatically. The option is /out:farpython.pyd
cl.exe /IE:\ENV\Python25\include farpython.c /link /dll /libpath:E:\ENV\Python25\libs
Although we haven't managed to execute our C function from Python, at least we've convinced Python to recognize our output file as module (even though it was identified as being dead). It's about time to concentrate on our C code, which appears to be incomplete even claimed to be "simple example".
Quite obvious that we need some init function to be present in our C file together with our "example". I'll just make a stub to see what happens. Python looks for something that is called "initfarpython". Ok.
void
initfarpython(void) {
}
Recompile. Launch Python. Import module.
ImportError: dynamic module does not define init function (initfarpython)
Damn. On the second thought everything works as expected. We defined init function in our code, but didn't mention it should be accessible by other programs (i.e. by Python). This is usually done by telling compiler to "export" function. In MS VC++ case with __declspec(dllexport) construction.
__declspec(dllexport) void
initfarpython(void) {
}
Now there is something new from compiler
/out:farpython.exe
/dll
/libpath:E:\ENV\Python25\libs
/out:farpython.pyd
farpython.obj
Creating library farpython.lib and object farpython.exp
It says that our library has something useful for other programs and produces files to allow other programs link to it. But Python doesn't need these files to use the module. Moreover, it doesn't even require that other functions should be explicitly "exported". Quite the opposite - everything except init function should be "declared static", i.e. visible only in module source file.
Before we test and continue we add some options to compiler to make it less verbose. /nologo - removes copyright and compiler version info, /Fefarpython.pyd allows to specify output filename via compiler options rather than through linker.
cl.exe /nologo /IE:\ENV\Python25\include /Fefarpython.pyd farpython.c /link /dll /libpath:E:\ENV\Python25\libs
Starting Python. Importing.
>>> import farpython
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
SystemError: dynamic module not initialized properly
That's logical. What for is initialization function that does nothing? There should be the correct version in official manual. An example of it is in the middle of http://docs.python.org/ext/methodTable.html Adopting it for our case.
PyMODINIT_FUNC
initfarpython(void)
{
(void) Py_InitModule("spam", NULL);
}
There is also a good explanation of the role of init function, so I'll leave it where it belongs. It is also said that PyMODINIT_FUNC is compiler-independent way to define exported init function, a macros that evaluates to "__declspec(dllexport) void" in case of MS VC++.
Using the method above we can safely import the module. Great! Here is the whole listing that compiles and can be imported.
// Step 1: A simple example
#include <python.h>
static PyObject *
far_example(PyObject *self, PyObject *args) {
const char* command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command)) {
return NULL;
}
sts = system(command);
return Py_BuildValue("i", sts);
}
PyMODINIT_FUNC
initfarpython(void) {
(void) Py_InitModule("farpython", NULL);
}
But it's not finished, not yet. The goal is to call "example" function. If you are not already reading manual on http://docs.python.org/ext/methodTable.html then try it. It contains everything that this tutorial is assumed to avoid. This means that the tutorial is almost over.
The last step would be to tell Python what functions are available in module by filling and giving to snake a special structure called "method table". This is done in init function through a parameter that currently reads as NULL. As described on the aforementioned page it looks like:
static PyMethodDef FarMethods[] = {
{"example", far_example, METH_VARARGS,
"Execute a shell command."},
{NULL, NULL, 0, NULL} /* Sentinel */
};
This structure is well described in the manual. The piece of code above should be included after all methods, but before init function - names in C should be defined before they are used in source text. Sentinel here is not just a stub to avoid coding error and missing some params - it really means end of list - without it your module will crash on import.
Final source.
// Step 1: A simple example
#include <python.h>
static PyObject *
far_example(PyObject *self, PyObject *args) {
const char* command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command)) {
return NULL;
}
sts = system(command);
return Py_BuildValue("i", sts);
}
static PyMethodDef FarMethods[] = {
{"example", far_example, METH_VARARGS,
"Execute a shell command."},
{NULL, NULL, 0, NULL} /* Sentinel */
};
PyMODINIT_FUNC
initfarpython(void) {
(void) Py_InitModule("farpython", FarMethods);
}
Test it.
>>> import farpython
>>> dir(farpython)
['__doc__', '__file__', '__name__', 'example']
>>> farpython.example
<built-in function="" example="">
>>> farpython.example()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: function takes exactly 1 argument (0 given)
>>> farpython.example("echo")
ECHO is on.
0
>>> ^Z
That's all. The only minor P.S. in the end - the goal was to create "far" module, not "farpython". While I could rename everything from "farpython" to just "far", I need leave the name of the module source file to be "farpython.c" to clearly indicate its purpose inside a heap of other far related sources. But now you should know how to make "far" module out of "farpython.c" with two little fixes.
Great work.
ReplyDeleteThanks!
ReplyDelete