Open Source for many developers means getting the hands dirty in many projects simultaneously sending patches here and there as they polish their own way through the code to make sure the bug encountered today will not pester anyone in the future. Everybody seems to be interested in reporting the bugs and sharing patches, but in reality it doesn't happen too often, mostly because of lack of time necessary to contribute. Following modern enterprise tendency to define various measures and estimates let me call this parameter as Time-To-Contribute value. TTC is directly influenced by such factors as activity of developers and availability of the code as well as usability and knowledge of development tools.
It would be interesting to get deep into the details of code contributions, but to keep the long story short and justify the title let's compare SVN and CVS. These tools provide access to source code and poor decision may affect developers experience and desire to contribute in the future. This post based on RFC submitted to PHPDOC community a year ago, which may be useful for other conservative CVS parties out there like MinGW
Facts only.
++++ Accessibility:
CVS usually blocked by proxies (needs dedicated port)
SVN works over HTTP and HTTPS via WebDAV
CVS doesn't work behind a proxy
SVN works with proxies
CVS checkout is complicated, it is hard to remember all the prerequisites
cvs -d :pserver:cvsread@cvs.php.net:/repository login
cvs -z3 -d :pserver:cvsread@cvs.php.net:/repository checkout -P phpdoc
SVN project checkout command is easy to remember
svn co http://svn.php.net/repository/phpdoc
CVS is complicated to learn
SVN has a perfect book
CVS is abandoned by developers
SVN is supported
++++ Security:
CVS password is transmitted over network in cleartext with simple rot13-like translation
SVN works over HTTPS, supports Apache authentication schemes
++++ Usability:
Command set is mostly the same
CVS fetches previous revision online to build diff of changes
SVN builds diff offline
CVS takes twice less disk space
SVN stores full copies of checked out files for comparison
CVS screws linefeeds
SVN doesn't screw linefeeds
CVS is file based - history is separated for each file
SVN is atomic - modification of group of files is a whole
CVS maintains independent revision numbers for each file
SVN revisions are global for repository
CVS leaves deleted directories in repository tree
SVN keeps directory tree tidy
CVS has convenient concept of branches/tags
SVN branches/tags are just directory copies in repository
CVS branches/tags concept is complicated
SVN branches/tags are easy to understand
I think that for the most of us the choice is rather obvious if there are no practice to use $Id$ to track how many changes a file underwent since last time, or if people are not too addicted to CVS branches. Nevertheless there are still many projects that use CVS and the true reasons why people do this are: old habits, absence of time to learn something new and dependencies on CVS in hard-coded legacy scripts. Of course, laziness is also a reason and it's funny that this laziness sometimes pushes to seek more convenient tools.
Tuesday, April 29, 2008
Wednesday, April 09, 2008
503 Service Temporarily Unavailable
Just a typical 503 page source to know what to look for in web site monitoring scripts. This one is usually displayed by Apache when triggered by Tomcat.
--cut-[503.html]-
--cut-[503.html]-
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">-----------------
<HTML><HEAD>
<TITLE>503 Service Temporarily Unavailable</TITLE>
</HEAD><BODY>
<H1>Service Temporarily Unavailable</H1>
The server is temporarily unable to service your
request due to maintenance downtime or capacity
problems. Please try again later.
</BODY></HTML>
Friday, March 14, 2008
Binding PHP to Windows GUI
Great tool to create Windows GUI applications with PHP - WinBinder was once my favourite toy. Supplied with very easy to follow examples and a great-looking form designer, seemed like it is capable to not only become a standard for making Windows interfaces for PHP, but also the tool of choice for Rapid Application Development. But the progress goes on and Windows is no more preferred desktop system among developers. Many choose Linux, some folks stick with BSD and of course a lot of people fall in love with MacOS. Windows is no more as popular as before and everybody understand that making your application with PHP + WinBinder limits its widespread usage. It is a pity to know that such a great toy may have no future as every effort needs support and maintaining PHP to C bindings for Windows API takes a lot of time and requires a lot of specific knowledge. Although the code for WinBinder is very well structured, the less is developers base the less chances the project has its bugs to be fixed in time. Most WB users are PHP folks and there are not many with required C experience among them. Thanks Rubem Pechansky for inventing this fabulous toy and let's hope that one day cross-platform GUI building toolkits like wxWindows let us create similar nice, fast and nifty appications.
Tuesday, March 04, 2008
Bloodless partition backup
Sometimes it may happen that a system blackbox arrives at your desk together with a person begging to do something to make it work again. It may have been hardware failure, power outage, virus or just a prank that made this panicked person pull the plug out of the outlet and shamefully accept that "I am afraid to turn it on after that!!" What should be the first reaction? Backup what is left. There are tons of backup software available, but most of it just copy files from one place to another on a regular basis. What you need in this situation is to copy a whole disk to network drive without booting the system preferably on the same hardware to avoid mounting HDDs. This is a task for disk cloning tools.
And while it seems logical to think about Acronis True Image or Norton Ghost on the second thought they are not really great when you need quick burn-and-use ISO image that they are unlikely to provide in their downloads section. So the best way is to check what open source has to offer. There are several tools:
The list of 8 product seems enough to choose one. Let me remind that we need ready-to-run Live CD capable to save partition or disk image to a remote network location. Throw out Mondo - it doesn't provide Live CD. TRK 3.2 also goes to the shelf - this very nice Boot CD doesn't include tools to backup/restore partition to a network drive. A pity. Linbox Rescue Server - free GPLed version can't backup NTFS partitions - won't work. FOG - project named "Free Ghost" doesn't have Live CD! and even with such promising name is designed solely for distributing the same disk image to multiple workstations over the network. Removed. So what is left:
After browsing these sites for a bit the first intention was to get rid of tools with ugly web-pages and short feature list. And it is where g4u beats them all. How can two floppy images in total 3Mb long compete with the raw power of 90Mb image of Clonezilla? Well, let's see. It was very easy to download and run g4u 2.3 from VirtualBox. Boot message gives almost all necessary information to go on except it doesn't say that you need user "install" on your FTP server. FTP is the only supported method for upload/download, but it's ok. Because of simplicity this tool had all the chances to be the chosen one of this review. Even in Windows you can easily setup FTP server given Far Manager, Python and this plugin. g4u is a tool that does the job of backing up partition, whole drive and compressing it via GZIP, but!:
1. There is no command history - and you have to retype whole command every time you've made a mistake
2. There is no anonymous FTP login option - extra hassle to setup "install" FTP user
3. Empty password is not allowed
4. No progress bar with total MB/percentage completed, not speaking about estimated time
5. Rather obscure GZIP option, which basically means compression level (default is 9 - the biggest)
To end up with g4u - an ideal approach for this awesome and minimal tool with limited usability would be to grow its ISO to 10Mb disk image by including Python and provide some wizards in .py
Next is Clonezilla - wizard interface. No FTP. Only SSH, SAMBA or NFS server that needs to be mounted for an image file. For folks without Linux experience the most complicated stuff is probably to understand "mount clonezilla image directory" dialog which basically means to attach "share" where image files can be stored (sorry for the lame comment). Saving partition image worked like a charm - not only it displayed progress window, but also detected space in use! and saved only used part. Magic! I just wonder if I'll be able to restore image manually in case Clonezilla fails. In directory with image file there are text files with disk and partition parameters, but image file itself is in custom format. In case of NTFS system - in ntfsclone 2.0 To conclude - Clonezilla version 1.0.9-10 CD is nice and usable, but a single mistake during wizard requires to start the process anew. Let's skim over alternatives.
Ghost for Linux - doesn't have its own web-page - only standard SourceForge project template, but provides ISO image for downloads. The project itself is similar to g4u - the same FTP upload method with compression, but much better interface. Version 0.24 is GPLed and Linux-based. In comparison g4u is BSD-licensed and NetBSD-based. Ok, about the contents. Multiple boot options varied mostly by linux kernel. Weird. I do not want to try every one to find out which of them will do the job. Good reference is shown right after startup documenting two options, but lazy one would prefer only the first - to enter "g4l" at command line to start wizard interface. This preferably could be just one line reminder before the system proposes shell. The nice thing about g4l wizards is that they allow to go back to correct options. Nevertheless, the real test failed on VirtualBox, because there was no default 'img' directory on FTP, and even though g4l created it, it hung afterwards. After manual restart and mkdir 'img' on FTP the process successfully completed. It can be FTP or VirtualBox bug - no idea, but with this minimal correction the test passed. Resume: setup is minimal and intuitive, though partition selection doesn't give any info about partition type. Ideal for FTP backup, but for other network access types I would still prefer Clonezilla. g4l like Clonezilla includes ntfsclone utility for NTFS backup, but its version is slightly outdated and should be selected explicitly. If you remember - there is no partition information displayed.
PartImage doesn't have its own Live CD, but seems to be the tool of choice used on PING "Backup and Restore Disk Partitions" Live CD and on SystemRescueCd. PING 2.01.10 is even more simple than Clonezilla, includes wizard interface, for network storage supports only SMB shares and requires that a predefined set of directories is already created in the destination share. This predefined set is only described on official site and that's strange, because Clonezilla manages to create all required directories automatically. So PING is not an option for burn-and-boot solution. Another annoyance is that Alt-Tab pressed within VirtualBox ends PING session prematurely. Too bad. Let's see the last one. At last!
SystemRescueCd is a whole bunch of useful tools for plumbing works. However, even with the nice help displayed at startup knowledge of *nix and the tools is still required. If I haven't used Clonezilla or g4l I had no chance to know that I needed ntfsclone. In addition I must admit I still do not remember what cmd I have to type to mount SMB share. SystemRescueCd may be the best Rescue Live CD, but it is not as convenient and intuitive as more specialized tools. As a typical lame windows user I've failed to use it properly. I.e. for backup, but there is still awesome utility named 'gparted' for disk partitioning that I would recommend to everyone. In fact there is a very awesome project that combines SystemRescueCD and Clonezilla Live CD in one disk. Grab it here - you won't regret. =)
The verdict.
To backup your disk or partition via SSH, SAMBA or NFS use Clonezilla. For convenient backup to FTP use g4l.
That's all. Enjoy!
And while it seems logical to think about Acronis True Image or Norton Ghost on the second thought they are not really great when you need quick burn-and-use ISO image that they are unlikely to provide in their downloads section. So the best way is to check what open source has to offer. There are several tools:
- Clonezilla
- FOG - Free OpenSource Ghost
- g4u - Ghost for Unix
- G4L - Ghost for Linux
- LRS - Linbox Rescue Server
- Mondo Rescue
- PartImage
- Trinity Rescue Kit
The list of 8 product seems enough to choose one. Let me remind that we need ready-to-run Live CD capable to save partition or disk image to a remote network location. Throw out Mondo - it doesn't provide Live CD. TRK 3.2 also goes to the shelf - this very nice Boot CD doesn't include tools to backup/restore partition to a network drive. A pity. Linbox Rescue Server - free GPLed version can't backup NTFS partitions - won't work. FOG - project named "Free Ghost" doesn't have Live CD! and even with such promising name is designed solely for distributing the same disk image to multiple workstations over the network. Removed. So what is left:
- Clonezilla
- g4u - Ghost for Unix
- G4L - Ghost for Linux
- PartImage
After browsing these sites for a bit the first intention was to get rid of tools with ugly web-pages and short feature list. And it is where g4u beats them all. How can two floppy images in total 3Mb long compete with the raw power of 90Mb image of Clonezilla? Well, let's see. It was very easy to download and run g4u 2.3 from VirtualBox. Boot message gives almost all necessary information to go on except it doesn't say that you need user "install" on your FTP server. FTP is the only supported method for upload/download, but it's ok. Because of simplicity this tool had all the chances to be the chosen one of this review. Even in Windows you can easily setup FTP server given Far Manager, Python and this plugin. g4u is a tool that does the job of backing up partition, whole drive and compressing it via GZIP, but!:1. There is no command history - and you have to retype whole command every time you've made a mistake
2. There is no anonymous FTP login option - extra hassle to setup "install" FTP user
3. Empty password is not allowed
4. No progress bar with total MB/percentage completed, not speaking about estimated time
5. Rather obscure GZIP option, which basically means compression level (default is 9 - the biggest)
To end up with g4u - an ideal approach for this awesome and minimal tool with limited usability would be to grow its ISO to 10Mb disk image by including Python and provide some wizards in .py
Next is Clonezilla - wizard interface. No FTP. Only SSH, SAMBA or NFS server that needs to be mounted for an image file. For folks without Linux experience the most complicated stuff is probably to understand "mount clonezilla image directory" dialog which basically means to attach "share" where image files can be stored (sorry for the lame comment). Saving partition image worked like a charm - not only it displayed progress window, but also detected space in use! and saved only used part. Magic! I just wonder if I'll be able to restore image manually in case Clonezilla fails. In directory with image file there are text files with disk and partition parameters, but image file itself is in custom format. In case of NTFS system - in ntfsclone 2.0 To conclude - Clonezilla version 1.0.9-10 CD is nice and usable, but a single mistake during wizard requires to start the process anew. Let's skim over alternatives.
Ghost for Linux - doesn't have its own web-page - only standard SourceForge project template, but provides ISO image for downloads. The project itself is similar to g4u - the same FTP upload method with compression, but much better interface. Version 0.24 is GPLed and Linux-based. In comparison g4u is BSD-licensed and NetBSD-based. Ok, about the contents. Multiple boot options varied mostly by linux kernel. Weird. I do not want to try every one to find out which of them will do the job. Good reference is shown right after startup documenting two options, but lazy one would prefer only the first - to enter "g4l" at command line to start wizard interface. This preferably could be just one line reminder before the system proposes shell. The nice thing about g4l wizards is that they allow to go back to correct options. Nevertheless, the real test failed on VirtualBox, because there was no default 'img' directory on FTP, and even though g4l created it, it hung afterwards. After manual restart and mkdir 'img' on FTP the process successfully completed. It can be FTP or VirtualBox bug - no idea, but with this minimal correction the test passed. Resume: setup is minimal and intuitive, though partition selection doesn't give any info about partition type. Ideal for FTP backup, but for other network access types I would still prefer Clonezilla. g4l like Clonezilla includes ntfsclone utility for NTFS backup, but its version is slightly outdated and should be selected explicitly. If you remember - there is no partition information displayed. PartImage doesn't have its own Live CD, but seems to be the tool of choice used on PING "Backup and Restore Disk Partitions" Live CD and on SystemRescueCd. PING 2.01.10 is even more simple than Clonezilla, includes wizard interface, for network storage supports only SMB shares and requires that a predefined set of directories is already created in the destination share. This predefined set is only described on official site and that's strange, because Clonezilla manages to create all required directories automatically. So PING is not an option for burn-and-boot solution. Another annoyance is that Alt-Tab pressed within VirtualBox ends PING session prematurely. Too bad. Let's see the last one. At last!
SystemRescueCd is a whole bunch of useful tools for plumbing works. However, even with the nice help displayed at startup knowledge of *nix and the tools is still required. If I haven't used Clonezilla or g4l I had no chance to know that I needed ntfsclone. In addition I must admit I still do not remember what cmd I have to type to mount SMB share. SystemRescueCd may be the best Rescue Live CD, but it is not as convenient and intuitive as more specialized tools. As a typical lame windows user I've failed to use it properly. I.e. for backup, but there is still awesome utility named 'gparted' for disk partitioning that I would recommend to everyone. In fact there is a very awesome project that combines SystemRescueCD and Clonezilla Live CD in one disk. Grab it here - you won't regret. =)
The verdict.
To backup your disk or partition via SSH, SAMBA or NFS use Clonezilla. For convenient backup to FTP use g4l.
That's all. Enjoy!
Thursday, January 24, 2008
Make Java program work through the proxy
It is not surprising that not many users know how to make a Java program work through a domain proxy if there is no place to enter proxy settings. Just because they are not developers they do not know that it is enough to launch this program with the following command line:
This works for anonymous or domain proxies only, because there are no password settings. To work through password-authenticated proxy, you will have to setup additional local or personal proxy that uses user/pass settings to authenticate and pass traffic to upstream one. Unfortunately, I can't name any software for user/pass authentication because I've never had to work with this problem, but I know that at least Privoxy is capable to forward requests.
java -Dhttp.proxyHost=192.168.1.1 -Dhttp.proxyPort=3128 -jar SoftWare.jar %*
This works for anonymous or domain proxies only, because there are no password settings. To work through password-authenticated proxy, you will have to setup additional local or personal proxy that uses user/pass settings to authenticate and pass traffic to upstream one. Unfortunately, I can't name any software for user/pass authentication because I've never had to work with this problem, but I know that at least Privoxy is capable to forward requests.
Wednesday, January 09, 2008
Compiling Python extension with GCC
In my previous post I've described how to make C code accessible from Python. I used Visual C++ compiler cl.exe to build an extension (or module) for Python. This follow-up shows how to compile the same extension for windows using GCC. I bet you already know what GCC is and that it is available from MinGW install as a result of install procedure described long ago.
Grab the source from the previous post - it won't change. Everything what is going to happen are just changes in .exe and its command line options. Saving source as farpython.c and starting GCC to compile it:
As usual, this won't produce anything useful except errors.
Additional include search path is specified using -I option in GCC.
A different picture, but the output is still grim.
Luckily these errors are not concerned with the code. They are from linker (ld) complaining it could not find library with binaries for functions defined in Python.h. The last one about undefined reference to WinMain is different though, but let's skip it until we deal with missing libraries. Python libraries are located at E:\ENV\Python25\libs and option to GCC is -L.
The output is still the same. The problem here is that linker doesn't know which specific library we need to link with to get binary bits for Python.h functions. cl.exe from Visual C++ was able to detect the correct library somehow, but for GCC we have to specify its name explicitly with -l option. Note that this option goes after a name of all compiled .c files. It is because params up to and including .c files are for compiler component and everything that goes after can be treated as linker's.
Check the output.
WinMain is an entrypoint or starting point of any program on windows platform, but Python extension is not a program that starts execution itself. .pyd is a .dll, or shared library with functions to be called by other programs. To tell that to GCC we add -shared switch to command line.
Now everything seems fine, but instead of far.pyd or farpython.pyd we've got a.exe Default output filename is easily corrected with yet another option -o
Test.
Grab the source from the previous post - it won't change. Everything what is going to happen are just changes in .exe and its command line options. Saving source as farpython.c and starting GCC to compile it:
gcc farpython.c
As usual, this won't produce anything useful except errors.
farpython.c:14:20: Python.h: No such file or directory
farpython.c:18: error: syntax error before '*' token
farpython.c:19: error: syntax error before '*' token
...
Additional include search path is specified using -I option in GCC.
gcc -IE:\ENV\Python25\include farpython.c
A different picture, but the output is still grim.
D:\Temp/ccyOaaaa.o(.text+0x1c):farpython.c: undefined reference to `_imp__PyArg_ParseTuple'
D:\Temp/ccyOaaaa.o(.text+0x4c):farpython.c: undefined reference to `_imp__Py_BuildValue'
D:\Temp/ccyOaaaa.o(.text+0x88):farpython.c: undefined reference to `_imp__Py_InitModule4'
D:\Temp/ccyOaaaa.o(.text+0xbe):farpython.c: undefined reference to `_imp__Py_InitModule4'
E:/ENV/MSYS/mingw/bin/../lib/gcc/mingw32/3.4.2/../../../libmingw32.a(main.o)(.text+0x106):main.c: undefined reference to `
WinMain@16'
collect2: ld returned 1 exit status
Luckily these errors are not concerned with the code. They are from linker (ld) complaining it could not find library with binaries for functions defined in Python.h. The last one about undefined reference to WinMain is different though, but let's skip it until we deal with missing libraries. Python libraries are located at E:\ENV\Python25\libs and option to GCC is -L.
gcc -IE:\ENV\Python25\include -LE:\ENV\Python25\libs farpython.c
The output is still the same. The problem here is that linker doesn't know which specific library we need to link with to get binary bits for Python.h functions. cl.exe from Visual C++ was able to detect the correct library somehow, but for GCC we have to specify its name explicitly with -l option. Note that this option goes after a name of all compiled .c files. It is because params up to and including .c files are for compiler component and everything that goes after can be treated as linker's.
gcc -IE:\ENV\Python25\include -LE:\ENV\Python25\libs farpython.c -lpython25
Check the output.
E:/ENV/MSYS/mingw/bin/../lib/gcc/mingw32/3.4.2/../../../libmingw32.a(main.o)(.text+0x106):main.c: undefined reference to `
WinMain@16'
collect2: ld returned 1 exit status
WinMain is an entrypoint or starting point of any program on windows platform, but Python extension is not a program that starts execution itself. .pyd is a .dll, or shared library with functions to be called by other programs. To tell that to GCC we add -shared switch to command line.
gcc -IE:\ENV\Python25\include -LE:\ENV\Python25\libs farpython.c -lpython25 -shared
Now everything seems fine, but instead of far.pyd or farpython.pyd we've got a.exe Default output filename is easily corrected with yet another option -o
gcc -IE:\ENV\Python25\include -LE:\ENV\Python25\libs farpython.c -lpython25 -shared -o far.pyd
Test.
E:\>python
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import far
>>> far.example("echo")
ECHO is on.
0
Thursday, December 20, 2007
Entrypoint instructions to using C code from Python
Official Python documentation contains everything an experienced C developer needs to build an extension like a module. That means it doesn't cover some basics like compiling code that are essential for startup for beginner.
This tutorial will try to teach and explain steps necessary to make a module in C for Python to call C code from Python program so that later it could be complemented with another tutorial to illustrate steps to call Python from C program. While the order of lessons could be reversed it is really better to start from writing a module (extending Python) to gain general understanding of how Python and C work together.
Let's start with example from http://docs.python.org/ext/simpleExample.html to build a module named "far". We'll define a function "example" that should be accessible from Python interpreter.
Save example to farpython.c Now we need to compile it somehow. I use cl.exe - Microsoft Visual C++ compiler. It is possible to use other compilers (like GCC) too even though there are warnings http://www.python.org/doc/ext/win-cookbook.html that Python module should be compiled with the same version of compiler that was used to build Python itself. I am not sure if the official distribution of Python 2.5.1 was compiled with my version of VC++ but I gave it a try and it worked. Another day I tried to compile the same code with GCC and it worked too.
To compile we usually do:
But in most cases this won't work,
we need to specify where to find Python.h
This won't produce anything usable either,
because we also need to specify where to find accompanying library with binary code to link with. The required option below is used by the linker. cl.exe treats all options as "for the linker" if they come after /link parameter, which in turn comes after our .c filename.
This won't work too. Nice, eh?
This means that another key should be passed to the linker. The one that'll tell it that we are creating module or dll and not executable program, so there is no entrypoint to look for.
No errors. Great!
well, almost. Resulting farpython.exe "can not be executed", because it is actually a dll. Python interpreter wouldn't be able to do anything with it even if it was named farpython.dll To be recognized as Python module the file should have .pyd extension. Let's check this by launching Python from the same directory.
While it is possible to rename the file manually, we will add another option for the linker to do the job automatically. The option is /out:farpython.pyd
Although we haven't managed to execute our C function from Python, at least we've convinced Python to recognize our output file as module (even though it was identified as being dead). It's about time to concentrate on our C code, which appears to be incomplete even claimed to be "simple example".
Quite obvious that we need some init function to be present in our C file together with our "example". I'll just make a stub to see what happens. Python looks for something that is called "initfarpython". Ok.
Recompile. Launch Python. Import module.
Damn. On the second thought everything works as expected. We defined init function in our code, but didn't mention it should be accessible by other programs (i.e. by Python). This is usually done by telling compiler to "export" function. In MS VC++ case with __declspec(dllexport) construction.
Now there is something new from compiler
It says that our library has something useful for other programs and produces files to allow other programs link to it. But Python doesn't need these files to use the module. Moreover, it doesn't even require that other functions should be explicitly "exported". Quite the opposite - everything except init function should be "declared static", i.e. visible only in module source file.
Before we test and continue we add some options to compiler to make it less verbose. /nologo - removes copyright and compiler version info, /Fefarpython.pyd allows to specify output filename via compiler options rather than through linker.
Starting Python. Importing.
That's logical. What for is initialization function that does nothing? There should be the correct version in official manual. An example of it is in the middle of http://docs.python.org/ext/methodTable.html Adopting it for our case.
There is also a good explanation of the role of init function, so I'll leave it where it belongs. It is also said that PyMODINIT_FUNC is compiler-independent way to define exported init function, a macros that evaluates to "__declspec(dllexport) void" in case of MS VC++.
Using the method above we can safely import the module. Great! Here is the whole listing that compiles and can be imported.
But it's not finished, not yet. The goal is to call "example" function. If you are not already reading manual on http://docs.python.org/ext/methodTable.html then try it. It contains everything that this tutorial is assumed to avoid. This means that the tutorial is almost over.
The last step would be to tell Python what functions are available in module by filling and giving to snake a special structure called "method table". This is done in init function through a parameter that currently reads as NULL. As described on the aforementioned page it looks like:
This structure is well described in the manual. The piece of code above should be included after all methods, but before init function - names in C should be defined before they are used in source text. Sentinel here is not just a stub to avoid coding error and missing some params - it really means end of list - without it your module will crash on import.
Final source.
Test it.
That's all. The only minor P.S. in the end - the goal was to create "far" module, not "farpython". While I could rename everything from "farpython" to just "far", I need leave the name of the module source file to be "farpython.c" to clearly indicate its purpose inside a heap of other far related sources. But now you should know how to make "far" module out of "farpython.c" with two little fixes.
This tutorial will try to teach and explain steps necessary to make a module in C for Python to call C code from Python program so that later it could be complemented with another tutorial to illustrate steps to call Python from C program. While the order of lessons could be reversed it is really better to start from writing a module (extending Python) to gain general understanding of how Python and C work together.
Let's start with example from http://docs.python.org/ext/simpleExample.html to build a module named "far". We'll define a function "example" that should be accessible from Python interpreter.
// Step 1: A simple example
#include <Python.h>
static PyObject *
far_example(PyObject *self, PyObject *args) {
const char* command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command)) {
return NULL;
}
sts = system(command);
return Py_BuildValue("i", sts);
}
Save example to farpython.c Now we need to compile it somehow. I use cl.exe - Microsoft Visual C++ compiler. It is possible to use other compilers (like GCC) too even though there are warnings http://www.python.org/doc/ext/win-cookbook.html that Python module should be compiled with the same version of compiler that was used to build Python itself. I am not sure if the official distribution of Python 2.5.1 was compiled with my version of VC++ but I gave it a try and it worked. Another day I tried to compile the same code with GCC and it worked too.
To compile we usually do:
cl.exe farpython.c
But in most cases this won't work,
farpython.c(4) : fatal error C1083: Cannot open include file: 'Python.h': No such file or directory
we need to specify where to find Python.h
cl.exe /IE:\ENV\Python25\include farpython.c
This won't produce anything usable either,
LINK : fatal error LNK1104: cannot open file "python25.lib"
because we also need to specify where to find accompanying library with binary code to link with. The required option below is used by the linker. cl.exe treats all options as "for the linker" if they come after /link parameter, which in turn comes after our .c filename.
cl.exe /IE:\ENV\Python25\include farpython.c /link /libpath:E:\ENV\Python25\libs
This won't work too. Nice, eh?
/out:farpython.exe
/libpath:E:\ENV\Python25\libs
farpython.obj
LINK : fatal error LNK1561: entry point must be defined
This means that another key should be passed to the linker. The one that'll tell it that we are creating module or dll and not executable program, so there is no entrypoint to look for.
cl.exe /IE:\ENV\Python25\include farpython.c /link /dll /libpath:E:\ENV\Python25\libs
No errors. Great!
/out:farpython.exe
/dll
/libpath:E:\ENV\Python25\libs
farpython.obj
well, almost. Resulting farpython.exe "can not be executed", because it is actually a dll. Python interpreter wouldn't be able to do anything with it even if it was named farpython.dll To be recognized as Python module the file should have .pyd extension. Let's check this by launching Python from the same directory.
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import farpython
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named farpython
>>> from shutil import copy
>>> copy("farpython.dll", "farpython.pyd")
>>> import farpython
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: dynamic module does not define init function (initfarpython)
While it is possible to rename the file manually, we will add another option for the linker to do the job automatically. The option is /out:farpython.pyd
cl.exe /IE:\ENV\Python25\include farpython.c /link /dll /libpath:E:\ENV\Python25\libs
Although we haven't managed to execute our C function from Python, at least we've convinced Python to recognize our output file as module (even though it was identified as being dead). It's about time to concentrate on our C code, which appears to be incomplete even claimed to be "simple example".
Quite obvious that we need some init function to be present in our C file together with our "example". I'll just make a stub to see what happens. Python looks for something that is called "initfarpython". Ok.
void
initfarpython(void) {
}
Recompile. Launch Python. Import module.
ImportError: dynamic module does not define init function (initfarpython)
Damn. On the second thought everything works as expected. We defined init function in our code, but didn't mention it should be accessible by other programs (i.e. by Python). This is usually done by telling compiler to "export" function. In MS VC++ case with __declspec(dllexport) construction.
__declspec(dllexport) void
initfarpython(void) {
}
Now there is something new from compiler
/out:farpython.exe
/dll
/libpath:E:\ENV\Python25\libs
/out:farpython.pyd
farpython.obj
Creating library farpython.lib and object farpython.exp
It says that our library has something useful for other programs and produces files to allow other programs link to it. But Python doesn't need these files to use the module. Moreover, it doesn't even require that other functions should be explicitly "exported". Quite the opposite - everything except init function should be "declared static", i.e. visible only in module source file.
Before we test and continue we add some options to compiler to make it less verbose. /nologo - removes copyright and compiler version info, /Fefarpython.pyd allows to specify output filename via compiler options rather than through linker.
cl.exe /nologo /IE:\ENV\Python25\include /Fefarpython.pyd farpython.c /link /dll /libpath:E:\ENV\Python25\libs
Starting Python. Importing.
>>> import farpython
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
SystemError: dynamic module not initialized properly
That's logical. What for is initialization function that does nothing? There should be the correct version in official manual. An example of it is in the middle of http://docs.python.org/ext/methodTable.html Adopting it for our case.
PyMODINIT_FUNC
initfarpython(void)
{
(void) Py_InitModule("spam", NULL);
}
There is also a good explanation of the role of init function, so I'll leave it where it belongs. It is also said that PyMODINIT_FUNC is compiler-independent way to define exported init function, a macros that evaluates to "__declspec(dllexport) void" in case of MS VC++.
Using the method above we can safely import the module. Great! Here is the whole listing that compiles and can be imported.
// Step 1: A simple example
#include <python.h>
static PyObject *
far_example(PyObject *self, PyObject *args) {
const char* command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command)) {
return NULL;
}
sts = system(command);
return Py_BuildValue("i", sts);
}
PyMODINIT_FUNC
initfarpython(void) {
(void) Py_InitModule("farpython", NULL);
}
But it's not finished, not yet. The goal is to call "example" function. If you are not already reading manual on http://docs.python.org/ext/methodTable.html then try it. It contains everything that this tutorial is assumed to avoid. This means that the tutorial is almost over.
The last step would be to tell Python what functions are available in module by filling and giving to snake a special structure called "method table". This is done in init function through a parameter that currently reads as NULL. As described on the aforementioned page it looks like:
static PyMethodDef FarMethods[] = {
{"example", far_example, METH_VARARGS,
"Execute a shell command."},
{NULL, NULL, 0, NULL} /* Sentinel */
};
This structure is well described in the manual. The piece of code above should be included after all methods, but before init function - names in C should be defined before they are used in source text. Sentinel here is not just a stub to avoid coding error and missing some params - it really means end of list - without it your module will crash on import.
Final source.
// Step 1: A simple example
#include <python.h>
static PyObject *
far_example(PyObject *self, PyObject *args) {
const char* command;
int sts;
if (!PyArg_ParseTuple(args, "s", &command)) {
return NULL;
}
sts = system(command);
return Py_BuildValue("i", sts);
}
static PyMethodDef FarMethods[] = {
{"example", far_example, METH_VARARGS,
"Execute a shell command."},
{NULL, NULL, 0, NULL} /* Sentinel */
};
PyMODINIT_FUNC
initfarpython(void) {
(void) Py_InitModule("farpython", FarMethods);
}
Test it.
>>> import farpython
>>> dir(farpython)
['__doc__', '__file__', '__name__', 'example']
>>> farpython.example
<built-in function="" example="">
>>> farpython.example()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: function takes exactly 1 argument (0 given)
>>> farpython.example("echo")
ECHO is on.
0
>>> ^Z
That's all. The only minor P.S. in the end - the goal was to create "far" module, not "farpython". While I could rename everything from "farpython" to just "far", I need leave the name of the module source file to be "farpython.c" to clearly indicate its purpose inside a heap of other far related sources. But now you should know how to make "far" module out of "farpython.c" with two little fixes.
Subscribe to:
Posts (Atom)