Python based VNC client in browser

What I really like in open source is when people reuse existing stuff to bring up something new. Today Arkaitz Jimenez brought up a way to connect to VNC server with browser using hybrid solution from Python, JavaScript, gevent and WebSockets.

The exciting thing that this project uses patched version of python-vnc-viewer written by Chris Liechti that I've uploaded to Google Code, because it had to be updated to work with modern versions of required libraries. It is nice to see this effort brought up something cool today.

How to make RAM disk in Linux

UPDATE (2014-08-27): Exactly three years later I discovered that Linux already comes with RAM disk enabled by default, mounted as `/dev/shm` (which points to `/run/shm` on Debian/Ubuntu):

$ df -h /dev/shm
Filesystem      Size  Used Avail Use% Mounted on
tmpfs            75M  4.0K   75M   1% /run/shm

See detailed info here.

*RAM disk* is a term from the past when DOS was alive and information was stored on disks instead of internet. If you created image of some disk, it was possible to load it into memory. Memory disks were useful to load software from Live CDs. Usually software needs some space to write data during boot sequence, and RAM is the fastest way to setup one.

Filesystem space in memory can be extremely useful today too. For example, to run tests without reducing resource of SSD. While the idea is not new, there was no incentive to explore it until I've run upon tmpfs reference in Ubuntu Wiki.

For example, to get 2Gb of space for files in RAM, edit /etc/fstab to add the following line:

tmpfs     /var/ramspace       tmpfs     defaults,size=2048M     0     0

/var/ramspace is now the place to store your files in memory.

Audio output on Windows with pure Python + ctypes

I want to make a simple walkie-talkie for a local LAN to have fun with my friends. I've created Raw Audio Socket spec to keep it really simple. We have a lot of different operating systems, so it would be nice to have a single implementation that can be run cross-platform. I chose Python and started with getting the sound playing on Windows.

Python is known for its easy integration with C libraries and, indeed, there is a lot of Python bindings calling audio libraries written in C from Python. However, it requires the library to be compiled for your specific platform. I chose to avoid dependencies on any C code and instead call Windows WinMM Multimedia API  directly with ctypes module.

Thanks to an excellent tutorial by David Overton, here is the pure Python proof of concept that plays standard CD Audio PCM 44.1kHz 16bit Stereo sample from external file to the default sound device.

"""
Implementation of Raw Audio Socket server spec in pure Python
http://code.google.com/p/rainforce/wiki/RawAudioSocket

"""

import sys

#-- CHAPTER 1: CONTINUOUS SOUND PLAYBACK WITH WINDOWS WINMM LIBRARY --
#
# Based on tutorial "Playing Audio in Windows using waveOut Interface"
# by David Overton

import ctypes
from ctypes import wintypes


# 1. Open Sound Device

# --- define necessary data structures from mmsystem.h
HWAVEOUT = wintypes.HANDLE
WAVE_FORMAT_PCM = 0x1
WAVE_MAPPER = -1
CALLBACK_NULL = 0
MMSYSERR_NOERROR = 0

class WAVEFORMATEX(ctypes.Structure):
  _fields_ = [
    ('wFormatTag',  wintypes.WORD),
      # 0x0001 WAVE_FORMAT_PCM. PCM audio
      # 0xFFFE The format is specified in the WAVEFORMATEXTENSIBLE.SubFormat
      # Other values are in mmreg.h 
    ('nChannels',   wintypes.WORD),
    ('SamplesPerSec',  wintypes.DWORD),
    ('AvgBytesPerSec', wintypes.DWORD),
      # for WAVE_FORMAT_PCM is the product of nSamplesPerSec and nBlockAlign
    ('nBlockAlign', wintypes.WORD),
      # for WAVE_FORMAT_PCM is the product of nChannels and wBitsPerSample
      # divided by 8 (bits per byte)
    ('wBitsPerSample', wintypes.WORD),
      # for WAVE_FORMAT_PCM should be equal to 8 or 16
    ('cbSize',      wintypes.WORD)]
      # extra format information size, should be 0
# --- /define

# Data must be processes in pieces that are multiple of
# nBlockAlign bytes of data at a time. Written and read
# data from a device must always start at the beginning
# of a block. Playback of PCM data can not be started in
# the middle of a sample on a non-block-aligned boundary.

hwaveout = HWAVEOUT()
wavefx = WAVEFORMATEX(
  WAVE_FORMAT_PCM,
  2,     # nChannels
  44100, # SamplesPerSec
  705600,# AvgBytesPerSec = 44100 SamplesPerSec * 16 wBitsPerSample
  4,     # nBlockAlign = 2 nChannels * 16 wBitsPerSample / 8 bits per byte
  16,    # wBitsPerSample
  0
)

# Open default wave device
ret = ctypes.windll.winmm.waveOutOpen(
  ctypes.byref(hwaveout), # buffer to receive a handle identifying
                          # the open waveform-audio output device
  WAVE_MAPPER,            # constant to point to default wave device
  ctypes.byref(wavefx),   # identifier for data format sent for device
  0, # DWORD_PTR dwCallback - callback mechanizm
  0, # DWORD_PTR dwCallbackInstance - user instance data for callback
  CALLBACK_NULL # DWORD fdwOpen - flag for opening the device
)

if ret != MMSYSERR_NOERROR:
  sys.exit('Error opening default waveform audio device (WAVE_MAPPER)')

print "Default Wave Audio output device is opened successfully"


# 2. Write Audio Blocks to Device

# --- define necessary data structures
PVOID = wintypes.HANDLE
WAVERR_BASE = 32
WAVERR_STILLPLAYING = WAVERR_BASE + 1
class WAVEHDR(ctypes.Structure):
  _fields_ = [
    ('lpData', wintypes.LPSTR), # pointer to waveform buffer
    ('dwBufferLength', wintypes.DWORD),  # in bytes
    ('dwBytesRecorded', wintypes.DWORD), # when used in input
    ('dwUser', wintypes.DWORD),          # user data
    ('dwFlags', wintypes.DWORD),
    ('dwLoops', wintypes.DWORD),  # times to loop, for output buffers only
    ('lpNext', PVOID),            # reserved, struct wavehdr_tag *lpNext
    ('reserved', wintypes.DWORD)] # reserved
# The lpData, dwBufferLength, and dwFlags members must be set before calling
# the waveInPrepareHeader or waveOutPrepareHeader function. (For either
# function, the dwFlags member must be set to zero.)
# --- /define

class AudioWriter(object):
  def __init__(self, hwaveout):
    self.hwaveout = hwaveout
    self.wavehdr = WAVEHDR()

  def write(self, data):
    self.wavehdr.dwBufferLength = len(data)
    self.wavehdr.lpData = data
    
    # Prepare block for playback
    if ctypes.windll.winmm.waveOutPrepareHeader(
         self.hwaveout, ctypes.byref(self.wavehdr), ctypes.sizeof(self.wavehdr)
       ) != MMSYSERR_NOERROR:
      sys.exit('Error: waveOutPrepareHeader failed')

    # Write block, returns immediately unless a synchronous driver is
    # used (not often)
    if ctypes.windll.winmm.waveOutWrite(
         self.hwaveout, ctypes.byref(self.wavehdr), ctypes.sizeof(self.wavehdr)
       ) != MMSYSERR_NOERROR:
      sys.exit('Error: waveOutWrite failed')

    # [ ] calculate sleep delay based on sample length
    # iii [ ] Measure CPU usage spike during wait without delay
    import time
    time.sleep(1)

    # Wait until playback is finished
    while True:
      # unpreparing the header fails until the block is played
      ret = ctypes.windll.winmm.waveOutUnprepareHeader(
              self.hwaveout,
              ctypes.byref(self.wavehdr),
              ctypes.sizeof(self.wavehdr)
            )
      if ret == WAVERR_STILLPLAYING:
        import time
        time.sleep(1)
        continue
      if ret != MMSYSERR_NOERROR:
        sys.exit('Error: waveOutUnprepareHeader failed with code 0x%x' % ret)
      break


# [ ] it's no good to read all the PCM data into memory at once
data = open('95672__Corsica_S__frequency_change_approved.raw', 'rb').read()

aw = AudioWriter(hwaveout)
aw.write(data)


# x. Close Sound Device

ctypes.windll.winmm.waveOutClose(hwaveout)
print "Default Wave Audio output device is closed"

#-- /CHAPTER 1 --

Windows also provides DirectSound API, but it looks too complicated for me ATM. The code above is also available from https://bitbucket.org/techtonik/audiosocket and marked with 0.1 tag. You may expect to find further modifications there.

Why Subversion conflict resolution suxx?

Because conflict resolution choice is confusing (for non-English natives for sure):

$ svn up
U    bitten\slave.py
G    bitten\tests\queue.py
Conflict discovered in 'bitten/queue.py'.
Select: (p) postpone, (df) diff-full, (e) edit,
        (mc) mine-conflict, (tc) theirs-conflict,
        (s) show all options:

Should you select "theirs conflict" or "mine confict" to fix your code? In situation when you feel that incoming change is right you may want to select "mine conflict" as the change that conflicts and should be removed as a result. But the correct answer it to select "theirs conflict" as the change to be incorporated in your code. Subversion developers think that you're are not removing conflicts, but incorporating them into your code. If they could think of user action - it is removing or resolving conflict in high-level terminology, or selecting correct change in low-level description.

Let's see the help:

Conflict discovered in 'bitten/slave.py'.
Select: (p) postpone, (df) diff-full, (e) edit,
        (mc) mine-conflict, (tc) theirs-conflict,
        (s) show all options: s
  (e)  edit             - change merged file in an editor
  (df) diff-full        - show all changes made to merged file
  (r)  resolved         - accept merged version of file
  (dc) display-conflict - show all conflicts (ignoring merged version)
  (mc) mine-conflict    - accept my version for all conflicts (same)
  (tc) theirs-conflict  - accept their version for all conflicts (same)
  (mf) mine-full        - accept my version of entire file (even non-conflicts)
  (tf) theirs-full      - accept their version of entire file (same)
  (p)  postpone         - mark the conflict to be resolved later
  (l)  launch           - launch external tool to resolve conflict
  (s)  show all         - show this list

Frankly, I can't understand how (mc) is different from (mf) if both versions accept all changes. (same) marker doesn't add much sense either. But even if help doesn't describe anything - these options were still added for a reason, and if we meditate a bit on them starting from the bottom comment to (mf) help, it becomes clear that:

1. (mf) throws away any incoming changes for the given file even if some of them were merged successfully
2. that means (mc) probably keeps the changes that were merged successfully
3. that means (df) shows more changes than (dc) output

Recommendations? Introduce `change` and `conflict` terminology earlier, i.e. show info like `3 changes, 2 conflicts`. It will also allow incremental conflict resolution, like if you (e)dited the counter is changed to `4 changes, 1 conflict` etc. Final recommendation if to replace `(ignoring merged version)` with `(ignoring merged changes)` string. There are surely more ways to improve this.

Chrome logo was awesome

This is a Chrome 11 logo.

This one is the old variant.

Finding unused files in a SCons project with Process Monitor

New technologies are born and die, but one things remains in your project - their files. Quite often you have no idea about where are they used, and attempt to remove them may lead to serious consequences.

Fortunately, if your project is managed by fine grained build system such as SCons, if your build scripts are not globbing too much, there are chances you can find files that are not participating in the builds.

Here is how to do this on Windows using Process Monitor tool that intercepts all system calls including file access.

While build systems are usually common for C/C++ and Java projects, it is possible to add fine-grained file usage control for any project. For example, SCons itself is written entirely in Python, it could run directly from the source checkout or build distributives from checkout. But instead, it uses build procedure to copy all necessary files from checkout into separate directory and do stuff from there.

Thanks to that it is possible to see which files are no more actual. While it is possible to compare checkout source tree and copied directory trees, I'll go through the hells of monitoring system file access in a source tree during the build process using Process Monitor (FileMon in the past). Linux should have similar tools too - let me know how are they called.

The process is the following:

1. Start Process Monitor
2. Stop incoming event flood by (un)clicking Capture (Ctrl-E) button
3. Open Filter (Ctrl-L) dialog to add some filters


SCons build is started by bootstrap.py script from a root of SCons source checkout. The script is executed by python executable, so I add python.exe process name to the filter. I know that bootstrap.py copies files from src/ subdirectory, so it is the directory I need to monitor, so I add this dir to filters too.

4. Go Tools -> File Summary...


There is a list of paths catched by Process Monitor when listening to system calls. They are already filtered, but additional filters can be applied using bottom left button to make information even more useful.

5. Export to CSV using Save...

Exported CSV is not very useful without some postprocessing. I used the following a script to compare the list of paths in CSV to actual src/ directory contents. This gives me names of files that were not touched during build at all.

SRCDIR = "C:\\p\\python\\scons\\src"
CSVLIST = 'accessed_bootstrap_files.CSV'

import csv
import os

reader = csv.reader(open(CSVLIST))
header = reader.next()
pathidx = header.index("Path")
pathset = set([row[pathidx] for row in reader])

#for row in pathset:
#  print row

fileset = set()
for root, dirs, files in os.walk(SRCDIR):
  fileset.update( [os.path.join(root, f) for f in files] )
  if '.svn' in dirs:
    dirs.remove('.svn')  # don't visit .svn directories

if len(pathset & fileset) == 0:
  print 'Error: File sets do not intersect at all'

print "Files not found in source directory tree:"
for f in (pathset - fileset):
  if not os.path.isdir(f):
    print f

print
print "Untouched files in source directory tree:"
for f in sorted(fileset - pathset):
  if not os.path.isdir(f):
    print f

I've found a few interesting things about SCons. Core tests are mixed with source files in repository checkout. They are not copied during bootstrap build. There are also few setup.py files, post-install script and announcement that don't participate in the build.

Here is the output of the above script:

Files not found in source directory tree:
<Total>

Untouched files in source directory tree:
C:\p\python\scons\src\.aeignore
C:\p\python\scons\src\Announce.txt
C:\p\python\scons\src\engine\.aeignore
C:\p\python\scons\src\engine\SCons\.aeignore
C:\p\python\scons\src\engine\SCons\ActionTests.py
C:\p\python\scons\src\engine\SCons\BuilderTests.py
C:\p\python\scons\src\engine\SCons\CacheDirTests.py
C:\p\python\scons\src\engine\SCons\DefaultsTests.py
C:\p\python\scons\src\engine\SCons\EnvironmentTests.py
C:\p\python\scons\src\engine\SCons\ErrorsTests.py
C:\p\python\scons\src\engine\SCons\ExecutorTests.py
C:\p\python\scons\src\engine\SCons\JobTests.py
C:\p\python\scons\src\engine\SCons\MemoizeTests.py
C:\p\python\scons\src\engine\SCons\Node\.aeignore
C:\p\python\scons\src\engine\SCons\Node\AliasTests.py
C:\p\python\scons\src\engine\SCons\Node\FSTests.py
C:\p\python\scons\src\engine\SCons\Node\NodeTests.py
C:\p\python\scons\src\engine\SCons\Node\PythonTests.py
C:\p\python\scons\src\engine\SCons\Optik\.aeignore
C:\p\python\scons\src\engine\SCons\PathListTests.py
C:\p\python\scons\src\engine\SCons\Platform\.aeignore
C:\p\python\scons\src\engine\SCons\Platform\PlatformTests.py
C:\p\python\scons\src\engine\SCons\SConfTests.py
C:\p\python\scons\src\engine\SCons\SConsignTests.py
C:\p\python\scons\src\engine\SCons\Scanner\.aeignore
C:\p\python\scons\src\engine\SCons\Scanner\CTests.py
C:\p\python\scons\src\engine\SCons\Scanner\DirTests.py
C:\p\python\scons\src\engine\SCons\Scanner\FortranTests.py
C:\p\python\scons\src\engine\SCons\Scanner\IDLTests.py
C:\p\python\scons\src\engine\SCons\Scanner\LaTeXTests.py
C:\p\python\scons\src\engine\SCons\Scanner\ProgTests.py
C:\p\python\scons\src\engine\SCons\Scanner\RCTests.py
C:\p\python\scons\src\engine\SCons\Scanner\ScannerTests.py
C:\p\python\scons\src\engine\SCons\Script\.aeignore
C:\p\python\scons\src\engine\SCons\Script\MainTests.py
C:\p\python\scons\src\engine\SCons\Script\SConscriptTests.py
C:\p\python\scons\src\engine\SCons\SubstTests.py
C:\p\python\scons\src\engine\SCons\TaskmasterTests.py
C:\p\python\scons\src\engine\SCons\Tool\.aeignore
C:\p\python\scons\src\engine\SCons\Tool\JavaCommonTests.py
C:\p\python\scons\src\engine\SCons\Tool\PharLapCommonTests.py
C:\p\python\scons\src\engine\SCons\Tool\ToolTests.py
C:\p\python\scons\src\engine\SCons\Tool\f03.xml
C:\p\python\scons\src\engine\SCons\Tool\msvsTests.py
C:\p\python\scons\src\engine\SCons\UtilTests.py
C:\p\python\scons\src\engine\SCons\Variables\BoolVariableTests.py
C:\p\python\scons\src\engine\SCons\Variables\EnumVariableTests.py
C:\p\python\scons\src\engine\SCons\Variables\ListVariableTests.py
C:\p\python\scons\src\engine\SCons\Variables\PackageVariableTests.py
C:\p\python\scons\src\engine\SCons\Variables\PathVariableTests.py
C:\p\python\scons\src\engine\SCons\Variables\VariablesTests.py
C:\p\python\scons\src\engine\SCons\WarningsTests.py
C:\p\python\scons\src\engine\SCons\cppTests.py
C:\p\python\scons\src\engine\setup.cfg
C:\p\python\scons\src\engine\setup.py
C:\p\python\scons\src\script\.aeignore
C:\p\python\scons\src\script\scons-post-install.py
C:\p\python\scons\src\script\setup.cfg
C:\p\python\scons\src\script\setup.py
C:\p\python\scons\src\test_aegistests.py
C:\p\python\scons\src\test_files.py
C:\p\python\scons\src\test_interrupts.py
C:\p\python\scons\src\test_pychecker.py
C:\p\python\scons\src\test_setup.py
C:\p\python\scons\src\test_strings.py


Hope this helps clean up your projects too.

P.S. I wish there was a Python script replacement for Process Monitor, or at least that it could be controlled from command line.

Asynchronous input from Windows console

too long didn't read - https://bitbucket.org/techtonik/async-console-input - public domain/MIT example for Windows implemented in pure Python (ctypes)

What is asynchronous input?

Given: The program need to terminate immediately when a subprocess exits or user hits 'q'. It should not eat 100% CPU time.

Blocking synchronous input:

import sys, subprocess
from msvcrt import getch, kbhit

p = subprocess.Popen([r"notepad"], shell=True)

while True:
  char = getch()
  if char == 'q':
    sys.exit('terminated by user')
  if p.poll() != None:
    sys.exit('terminated by child exit')

This one doesn't work (doesn't exits immediately when child process terminates), because getch() blocks execution key is pressed, and even if child process exits, it won't be detected until some key is pressed.

Non-blocking synchronous input (polling):

import sys, subprocess
from msvcrt import getch, kbhit

p = subprocess.Popen([r"notepad"], shell=True)

while True:
  while kbhit():
    if getch() == 'q':
      sys.exit('terminated by user')
  if p.poll() != None:
    sys.exit('terminated by child exit')

This works ok, but constant polling uses 100% of CPU resources. It is possible to insert time.sleep() instructions to reduce the carbon footprint, but these crutches will greatly slow down the console when you add z-type to console to spend your time until the background process is finished.

Asynchronous console input on Windows with Python

Asynchronous input allows your program to receive notification from operating system when an input is available. This means that you define events your program needs to react (not necessary console events), send this list to operating system and put your program into wait mode. When system sees this event, your wait function returns and it's possible to inspect/filter event to a greater detail.

Windows provides WaitForMultipleObjects wait function, and in Linux I believe it is select call. If you've tried to understand how Twisted works, but couldn't - will it make more clear if I say that twisted reactor is a wait function? What you do when you code with twisted is just configuring events to react, making chains of them so that one event reacts to another event. This allow to build very interesting, complex and de-coupled asynchronous applications in a couple of days.

O.k. Getting back from Twisted to the asynchronous console input on Windows. Below is the full source code that uses WaitForMultipleObjects. I am afraid that's minimal complete example possible to build with ctypes using Windows API. Tested with Python 2.5

Non-blocking asynchronous input from console:
https://bitbucket.org/techtonik/async-console-input

"""
Example of non-blocking asynchronous console input using
Windows API calls in Python.  This can become handy for
async console tools such as IRC client.

Public domain or MIT license
by anatoly techtonik 


Notes:
  1. WaitForMultipleObjects is used to listen for the
     signals from process and stdin handles
  2. When handle is signalled it remains in this state
     until reset
  3. msvcrt.* keyboard functions don't clear signalled
     state from stdin handle, that's why console API
     functions are used to clear the input buffer
     instead of kbhit()/getch() loop
"""


import ctypes
import ctypes.wintypes
import subprocess

# open notepad in separate process and monitor its execution
# at the same time asynchronously processing events from
# standard input without wasting 100% CPU on looping

# OpenProcess desired access flag
#   "the right to use the object for synchronization. This
#    enables a thread to wait until the object is in the
#    signaled state"
SYNCHRONIZE=0x00100000L
# Constant to get stdin handle with GetStdHandle() call
STD_INPUT_HANDLE = -10
# Constant for infinite timeout in WaitForMultipleObjects()
INFINITE = -1

# --- processing input structures -------------------------
# INPUT_RECORD structure
#  events:
EVENTIDS = dict(
  FOCUS_EVENT = 0x0010,
  KEY_EVENT = 0x0001,      # only key event is handled
  MENU_EVENT = 0x0008,
  MOUSE_EVENT = 0x0002,
  WINDOW_BUFFER_SIZE_EVENT = 0x0004)
EVENTS = dict(zip(EVENTIDS.values(), EVENTIDS.keys()))
#  records:
class _uChar(ctypes.Union):
  _fields_ = [('UnicodeChar', ctypes.wintypes.WCHAR),
              ('AsciiChar', ctypes.wintypes.c_char)]
class KEY_EVENT_RECORD(ctypes.Structure):
  _fields_ = [
    ('keyDown', ctypes.wintypes.BOOL),
    ('repeatCount', ctypes.wintypes.WORD),
    ('virtualKeyCode', ctypes.wintypes.WORD),
    ('virtualScanCode', ctypes.wintypes.WORD),
    ('char', _uChar),
    ('controlKeyState', ctypes.wintypes.DWORD)]
class _Event(ctypes.Union):
  _fields_ = [('keyEvent', KEY_EVENT_RECORD)]
  #  MOUSE_EVENT_RECORD        MouseEvent;
  #  WINDOW_BUFFER_SIZE_RECORD WindowBufferSizeEvent;
  #  MENU_EVENT_RECORD         MenuEvent;
  #  FOCUS_EVENT_RECORD        FocusEvent;
class INPUT_RECORD(ctypes.Structure):
  _fields_ = [
    ('eventType', ctypes.wintypes.WORD),
    ('event', _Event)]
# --- /processing input structures ------------------------



np = subprocess.Popen([r"notepad"],
       stdin=subprocess.PIPE,
       stdout=subprocess.PIPE,
       stderr=subprocess.PIPE,
       shell=True)
# OpenProcess returns handle that can be used in wait functions
# params: desiredAccess, inheritHandle, processId
nph = ctypes.windll.kernel32.OpenProcess(SYNCHRONIZE, False, np.pid)
print("Started Notepad with pid=%s, handle=%s" % (np.pid, nph))

ch = ctypes.windll.kernel32.GetStdHandle(STD_INPUT_HANDLE)

handles = [ch, nph]

ctypes.windll.kernel32.FlushConsoleInputBuffer(ch)
eventnum = ctypes.wintypes.DWORD()
eventread = ctypes.wintypes.DWORD()
inbuf = (INPUT_RECORD * 1)()

print "[q]uit, [s]top console processing, launch bro[w]ser"
stopflag = False
while not stopflag and nph in handles:
    print "Waiting for handles %s.." % handles

    HandleArrayType = ctypes.wintypes.HANDLE * len(handles)
    handle_array = HandleArrayType(*handles)
    # params: count, handles, waitAll, milliseconds
    ret = ctypes.windll.kernel32.WaitForMultipleObjects(
            len(handle_array), handle_array, False, INFINITE)

    if handles[ret] == ch:
        """
        # msvcrt won't work, because it doesn't reset
        # signalled state of stdin handle
        import msvcrt
        while msvcrt.kbhit():
            print "key!"
            print msvcrt.getch()
        continue
        """
        # --- processing input ---------------------------
        ctypes.windll.kernel32.GetNumberOfConsoleInputEvents(
          ch, ctypes.byref(eventnum))
        for i in range(eventnum.value):
          # params: handler, buffer, length, eventsnum
          ctypes.windll.kernel32.ReadConsoleInputW(
            ch, ctypes.byref(inbuf), 2, ctypes.byref(eventread))
          if EVENTS[inbuf[0].eventType] != 'KEY_EVENT':
            print EVENTS[inbuf[0].eventType]
            pass
          else:
            keyEvent = inbuf[0].event.keyEvent
            if not keyEvent.keyDown:
              continue
            char = keyEvent.char.UnicodeChar.lower()
            #print char, keyEvent
            if char == 'q':
              print('[q] key pressed. Exiting..')
              stopflag = True
            elif char == 's':
              handles.remove(ch)
            elif char == 'w':
              import webbrowser
              webbrowser.open('http://techtonik.rainforce.org')
            #print char
        # --- /processing input --------------------------

    elif handles[ret] == nph:
        print("Notepad is closed. Exiting..")
        handles.remove(nph)
    else:
        print("Warning: Unknown return value '%s'" % ret)

ctypes.windll.kernel32.FlushConsoleInputBuffer(ch)
ctypes.windll.kernel32.CloseHandle(nph)
print "Done."

Where can it be useful?

Writing first Windows console IRC client in Python and make it cross-platform? Network log viewers with keyboard shortcuts? Real-time roguelikes? Matrix sniffer screensaver? I don't know - its your time. Hope I saved you some though.

Enjoy! If you want to enhance this stuff - feel free to join ever empty https://groups.google.com/forum/#!forum/rainforce for public discussion.

User Pattern: Button Status

Introduction into User Patterns

Many of you know about Design Patterns of software development, invented to make complicated solutions look less complicated. The problem with Design Patterns is that they describe solutions to coding problems - not for user problem. People abusing Design Patterns tend to align the functionality of application to patterns, sacrificing features users wanted. The application that looks well-designed in the eyes of developers, in fact makes even simple features hard to implement or just takes too much time. As time passes, users become dissatisfied and eventually lose all interest in the product.

People tried to approach the problem from the other end with User Interface Design guidelines. This often made developers unhappy, because their Design Patterns don't explain how to work with web and Ajax application where "business logic" becomes shared between client and server. Developers wanted to see how different frameworks approach this stuff and to make the development process more intuitive. For this reason there should be a way to see how some typical features/functionality are implemented in different frameworks and this should be more detailed than a wiki or a blog in 15 minutes.

So, let me introduce User Patterns. User Pattern is a very abstract story, missing details about design decisions, but detailed enough to provide implementation and see how _simple_, _extensible_ and _maintainable_ this implementation is.

Button Status User Pattern

User story: I want a page with a button and a status field displayed. When I click the button, the status should be updated.

Description: There are five levels in the quest for the implementation of this pattern.
Level 1. The status is blank - it is fetched/calculated only when the button is pressed.
Level 2. The status is initially available - and it is refreshed/calculated when the button is pressed.
Level 3. When refresh/recalculation fails, the error value is displayed in status field.
Level 4. When refresh/recalculation fails, the error value is displayed in status field along with user friendly message in separate field
Level 5. The status value is stored in application database - when the button is pressed, it is recalculated, saved and only when displayed to the user
Level 6. When recalculation/save or fetch operation fails, user should be presented with friendly message explaining the error and further action (i.e. report problem link)

I'm looking forward to create samples for AppEngine webapp, classic Django and Django-nonrel. Feel free to share your snippets.

[PySide] Python for Qt version 1.0.0~beta4 "I have altered the deal" released

I can't see this post on planet.python.org and can't resist to share the good news about new PySide release.

The PySide team is happy to announce the fourth beta release of PySide:
Python for Qt. New versions of some of the PySide toolchain components
apiextractor, generatorrunner, shiboken, libpyside, pyside-tools have been
released as well.

Like the others, this is a source code release only; we hope our community
packagers will be providing provide binary packages shortly. To acquire the
source code packages, refer to our download wiki page [1] or pull the relevant
tagged versions from our git repositories [2].

Major changes since 1.0.0~beta3
===============================

This is a bug fix release. Since beta3, a total of 47 high-priority
bugs have been fixed. See the list of fixed bugs at the end of this message.

Path towards 1.0 release
========================

There are still some outstanding bugs in our Bugzilla [3]. To have these
fixed, we plan to do other beta in two weeks. The beta cycle will continue
until we have all P2 bugs fixed.


About PySide
============

PySide is the Nokia-sponsored Python Qt bindings project, providing
access to not only the complete Qt 4.7 framework but also Qt Mobility,
as well as to generator tools for rapidly generating bindings for any
Qt-based libraries.

The PySide project is developed in the open, with all facilities you'd
expect from any modern OSS project such as all code in a git repository
[2], an open Bugzilla [5] for reporting bugs, and an open design
process [4]. We welcome any contribution without requiring a transfer of
copyright.

List of bugs fixed
==================

624     button click emit doesn't work
484     Error compiling QtContacts 1.1 (problems with const QList)
498     powerStateChanged-SIGNAL not emitted!
509     Can't use Shiboken when both Debug and Released are installed.
528     Connecting to SIGNAL positionUpdated fails
552     Segmentation fault when using QUiLoader and QTabWidget
553     A warning against using QUILoader is needed in the documentation
560     Lack of QtCore.Signal documentation
582     Python slots don't get called when they have a custom decorator
589     Crash related to QGraphicsProxyWidget and QVariant
592     shiboken.dll produces a segmentation fault when reloading a PySide module
608     Photoviewer example missing license boilerplates and shebang lines
609     Python site-packages path cannot be customized
610     QWidgetItemV2 not exposed to Python
626     Problem building PySide on OS X (qabstractfileengine_wrapper.cpp:
No such file or directory)
406     Unable to send instant messages using QMessageService
458     Doesn't build with QtMobility 1.1.0~beta2
487     Support QContactDetailFieldDefinition.setAllowableTypes
497     Miising __lt__ operators in QtMobility::QGeoMapObject
499     QFeedbackHapticsInterface and QFeedbackFileInterface are broken
511     QPainter doesn't respect Qt.NoPen
522     example/threads/mandelbrot.py crashes on exit
523     QWidget.winId() returns PyCObject (expected unsigned long)
530     Importing division from future breaks QPoint division
531     sessionProperty "ConnectInBackground" does not work
539     MCC and MNC interchanged
541     QTableWidget.itemAt(row, col) always returns item at 0, 0.
550     Can't call PySide slot from QtScript when the args are a list of anything.
556     QGraphicsScene.addItem performs very poorly when the scene has >10000 items
562     pyside-uic does not generate some layers properties
568     List insertion time grows with list size
574     In docs of QUuid there's documentation for a function called
"operator QString"
575     Strange behaviour of QTextEdit.ExtraSelection().cursor
584     python pickle module can't treat QByteArray object of PySide
591     QtCore.QRect has no attribute "getRect()" in Windows binary
611     enum values lack a tp_print implementation
614     FAil to register 2 objects in the same address
619     never automatically delete a QWidget that has no parent widget and
is visible
620     QAbstractItemModel.createIndex(int,int,PyObject*) does not
increment refcount
621     QGLWidget.bindTexture(QString) does not bind the texture correctly
622     PPA pyside is broken on Ubuntu 10.10
623     QGLWidget.bindTexture(QPixmap, GLenum, GLenum) is missing
625     QFileDialog return a tuple instead of a unicode
628     pyside-uic can't effect "headerVisible" attribute for QTreeView
and QTreeWidget
232     [FTBFS] Fails to build on hurd-i386 (Test "lock" hangs for more
than 191 minutes)
255     Test qtscripttools_debugger segfaults on ia64
298     Contact subtype not correctly set



References
==========

[1] http://developer.qt.nokia.com/wiki/PySideDownloads
[2] http://qt.gitorious.org/pyside
[3] http://bugs.openbossa.org/
[4] http://www.pyside.org/docs/pseps/psep-0001.html

Thanks
PySide team.

Python GUIniverse

I am on the PySide of the Force.

Injecting SVN bindings into Mercurial on Windows

If you need to add Subversion bindings for your Mercurial, which was installed using ordinary installer (and not TortoiseHg), then the following script can help to put all files in place.

https://bitbucket.org/techtonik/py2exe-relibzip/src/f15f8d032b5b/reinject.py

Unpack svn bindings, add all *.py files from svn/ and libsvn/ directories into the root of library.zip archive *with* these directories. Then call script with path to unpacked libsvn/. The script will generate some *.dll and *.py files in current directory. Put *.py into library.zip:/libsvn/ and *.dll into Mercurial installation directory. You may need to also copy libsvn_swig_py-1.dll into Mercurial dir.

The script can be tuned to insert C extensions to any py2exe distribution compiled with bundle_files=3