Monday, January 25, 2010

Repacking library.zip from py2exe



intro
py2exe tool converts Python scripts to standalone .exe distributives. It isolates all required Python modules together with Python interpreter itself and wraps them into single library.zip file. This way application is not affected by Python modules that may be already installed on user system. Unfortunately, this also means that you can't add new modules to your standalone Python program.

For example, you need to add Hg-Git extension to Mercurial installed as standalone program. You can specify path to extension in Mercurial.ini, but Hg-Git depends on Dulwich module, which is not present in library.zip Attempt to use this extension will fail. The same problem is with converting Bazaar repositories using convert extension that is present in library.zip, but additionally requires installed bzr module.

solution
The script below helps to add Python modules to library.zip file made with py2exe. Latest version should be available at this bitbucket repository. The following command unpacks library.zip in current directory and makes library_unpacked.zip that you can edit with your favorite archiver:

python relibzip.py unpack
After you've finished, issue:

python relibzip.py pack
and script will create library_packed.zip from extracted resources. Copy this file over library.zip and you all set.

Source code (ugly, but works):

"""
pack/unpack library.zip created by py2exe to standard .zip archive

library.zip created with py2exe can not always be processed by standard
archivers. this script removes extra chunks added by py2exe and puts
them back when requested

MIT license, by techtonik // gmail.com
"""

import sys
import os
from optparse import OptionParser
import struct


PYTHONDLL = "<pythondll>"
PDNAME = "pythondll"
ZLIBPYD = "<zlib.pyd>"
ZDNAME = "zlibpyd"

UNPACKED = "library_unpacked.zip"
PACKED = "library_packed.zip"


def unpack(filename):
   f = open(filename, "rb")
   # looking for PYTHONDLL name
   pdname = f.read(len(PYTHONDLL))
   if pdname != PYTHONDLL:
       if pdname[:2] == "PK":
           sys.exit("Seems to be normal .zip archive, not unpacking")
       else:
           sys.exit("Unknown archive format")

   def save_section(secname, fname):
       print "Extracting %s section to %s" % (secname, fname)
       fsize = struct.unpack("i", f.read(4))[0]
       fpd = open(fname, "wb")
       fpd.write(f.read(fsize))
       fpd.close()
   save_section(PYTHONDLL, PDNAME)

   buf = ""
   zdname = f.read(len(ZLIBPYD))
   if zdname != ZLIBPYD:
       if zdname[:2] == "PK":
           print "No zlib.pyd section"
           buf = zdname
       else:
           sys.exit("Unknown archive format")
   else:
       save_section(ZLIBPYD, ZDNAME)
  
   flib = open(UNPACKED, "wb")
   flib.write(buf)
   flib.write(f.read())
   flib.close()
   f.close()
   print "Done. Unpacked .zip contents is available at %s" % UNPACKED
   sys.exit(0)


def pack(tofname):

   if not os.path.exists(PDNAME):
       sys.exit("%s section file %s is not found. Exiting" % (PYTHONDLL, PDNAME))
   if not os.path.exists(UNPACKED):
       sys.exit("Unpacked version %s is not found. Exiting" % UNPACKED)

   f = open(tofname, "wb")

   def write_section(secname, fname):
       print "Writing %s section from %s" % (secname,fname)
       f.write(PYTHONDLL)
       fsize = os.stat(PDNAME).st_size
       f.write(struct.pack("i", fsize))

       fpd = open(fname, "rb")
       f.write(fpd.read())
       fpd.close()
       print "Removing section file %s" % fname
       os.remove(fname)
   write_section(PYTHONDLL, PDNAME)

   # check for optional ZLIBPYD section
   if not os.path.exists(ZDNAME):
       print "No %s section file %s. Skipping" % (ZLIBPYD, ZDNAME)
   else:
       write_section(ZLIBPYD, ZDNAME)

   fzip = open(UNPACKED, "rb")
   f.write(fzip.read())
   fzip.close()

   f.close()
   print "Done. Packed .zip contents is available at %s" % tofname
   sys.exit(0)


parser = OptionParser(usage="usage: %prog ",
   description="update library.zip created by py2exe utility")
opt,arg = parser.parse_args()
  
if arg and arg[0] == 'unpack':
   unpack("library.zip")
elif arg and arg[0] == 'pack':
   pack(PACKED)
else:
   sys.exit(parser.format_help())

3 comments:

  1. Thanks for this script! It was quite useful for us today. One quick correction: it seems that the < and > symbols in and at the top of your script were not properly escaped by Blogger, so (at least) Firefox treats them as HTML tags and doesn't display them. That makes the script copied and pasted from your blog post text not work. A copy downloaded from the bitbucket worked beautifully, though!

    ReplyDelete
  2. Fixed < and > symbols and markup. Thanks for the notice.

    ReplyDelete
  3. foss.heptapod.net is the new bitbucket.org

    Would you mind to push that techtonik/py2exe-relibzip repo there?

    Thanks!

    ReplyDelete