Friday, June 05, 2009

Code syntax hightlight for Blogger

Just add this code inside your template element:

<link href='http://rainforce.org/google/code/prettify/prettify.css' rel='stylesheet' type='text/css'/>
<script src='http://rainforce.org/google/code/prettify/prettify.js' type='text/javascript'></script>

and change <body> tag to be

<body onload="prettyPrint()">
In fact all above is well explained at Google Code Prettify site. Site of the JavaScript library used to colorize <pre> and <code> blocks that have class="prettyprint" attribute. I.e. wrap your code into following HTML block to make it colored:

<code class="prettyprint">
...
</code>

The source of this page contains CSS definition with colors for this blog, but you may copy/paste it from the block below:

<style type='text/css'>
/* desert scheme ported from vim to google prettify */
code.prettyprint { display: block; padding: 2px; border: 1px solid #888; background-color: #333; }
.str { color: #ffa0a0; } /* string - pink */
.kwd { color: #f0e68c; font-weight: bold; }
.com { color: #87ceeb; } /* comment - skyblue */
.typ { color: #98fb98; } /* type - lightgreen */
.lit { color: #cd5c5c; } /* literal - darkred */
.pun { color: #fff; } /* punctuation */
.pln { color: #fff; } /* plaintext */
.tag { color: #f0e68c; font-weight: bold; } /* html/xml tag - lightyellow*/
.atn { color: #bdb76b; font-weight: bold; } /* attribute name - khaki*/
.atv { color: #ffa0a0; } /* attribute value - pink */
.dec { color: #98fb98; } /* decimal - lightgreen */
</style>

Monday, March 02, 2009

Query to get MySQL table type

The following command can be used to get the type (engine) of MySQL table from current database in select query:

SELECT table_name, engine FROM information_schema.tables WHERE table_name="your_table" AND table_schema=DATABASE();

 

Tuesday, December 09, 2008

Google Translate for Trac timeline comments

JavaScript that translates changeset comments in Trac timeline using Google Language API. How to use it with Trac should be well-described at this Trac-Hacks page. Here I just want to state that this code can be reused for other .html pages.

This code is used at http://farmanager.rainforce.org/timeline

Just do not forget to add jQuery and the following line to your header.

<script type="text/javascript" src="http://www.google.com/jsapi"></script>

Here goes the main code:

// Translate changeset comments in Trac timeline
// Using Google Language API version 1

// Code in public domain
// techtonik // rainforce.org


// wrap the code to use $ as jQuery function name
(function($){


// setup language parameters

var srclang = "ru";
var tgtlang = "en";

var gtrans_btn_text = "Translate russian comments";

var srclang_a_z = Array();
srclang_a_z["en"] = /[a-z]+/i;
srclang_a_z["ru"] = /[Р°-С_]+/i;


// load Google API

google.load("language", "1");


// make this anonymous function executed by jQuery when document is loaded
jQuery(document).ready(function() {

// add "powered by Google" logo into footer
$('<p id="googleattr" class="left"></p>').insertAfter("#footer p.left:last")
google.language.getBranding('googleattr');

// add trac_timeline_translate() button to timeline menu
var gtrans_btn = $("<input type='button'/>")
.attr({
id: "gtrans_btn",
value:gtrans_btn_text
})
.click(function() {
trac_timeline_translate();
gtrans_btn.attr("disabled", "disables");
return false;
});

$("#prefs").append("<hr/>");
$("#prefs")
.append(
$("<div align='center'></div>").append(gtrans_btn)
)
})


function trac_timeline_translate() {
// translate changeset messages
g_trans_nodes( $("dd.changeset") );
}


function g_trans_nodes(nodes) {
// translate nodes one by one
// todo: group many small nodes into one packet < 5000 bytes

var transTexts = Array();
var transNodes = Array();

for (i=0; i<nodes.size();i++) {
var text = nodes[i].innerHTML;

if (text.match(srclang_a_z[srclang])) {
transTexts.push(text);
transNodes.push(nodes[i]);
}
}

for (i=0; i<transTexts.length;i++) {
function makeTransNodeClosure(target_node) {
// closure to save node parameter to be accessible
// in callback function
var node = target_node; // local variable
var trans_callback = function (result) {
node.innerHTML = result.translation;
}
return trans_callback; // return function
}

google.language.translate(transTexts[i], srclang, tgtlang, makeTransNodeClosure(transNodes[i]));
}
}


})(jQuery);

Friday, September 26, 2008

Python in Industial Automation

Before celebrating 10th Google birthday I would like to make a gesture of goodwill to fellow comrades from industrial automation field, who over years battled with a very special kind of the most conservative software in world.

Industrial Automation Software

It is the Software that directly puts people lives at stake. A little bug may kill people, inflict tremendous physical damage to buldings, vehicles and equipment. No wonder that Industrial Automation Software is among the most slowly developed technologies in the world. People fear to change things that work if the cost of error is so high. But using outdated software that nobody is capable to maintain carries even greater risks. Just as connected to Internet Windows without updates is brought down by viruses in minutes, the Enterprise is brought down by unmaintained legacy systems that just await the time to explode. Industrial software is ought to be stable, error free and resistant to all kind of threats, but it is not true in most cases. Most industrial software products lack public exposure, and so far I haven't seen any serious industrial product that didn't crash. Industrial software is the classic example of vendor lock-in. It happens just everythere. If you have some hardware sensors in the field and would like to collect signals from them - most likely you would have to buy industrial controller or PLC, which will be on 24/7/365/10 to gather your data. PLC's were mostly used solely for control automation without the need to share data with external systems, but in IT age you would need that data sooner or later. To get it from he PLC you would need to buy expensive OPC server that is capable to poll PLC using their own protocol and provide you with data in OPC format. OPC stood for OLE for Process Control and as you can guess from its name was vendor-locked to Microsoft from the start. It was based on OLE, COM and DCOM - its proprietary technology. To read one byte from your PLC you would have to buy Windows, buy OPC server and pay specialist to write an OPC client to access one byte of your sensor. Even after that some companies dared to include "Lower TCO" slogan in their markting campaigns. There was no alternative those days though. MS SQL is the only promoted "realtime database" even if there was a word "near" before the quotes, but it has support. Now there are plenty of open protocols and standards for data communication, many databases and time-tested solutions from other IT areas, but things are moving slowly in Industrial Automation world where "open protocol" still means that you are free to implement a specification, but need to pay for texts and it is ok. DCOM is slowly replaced by .NET, new consortiums appear just to put their label "approved" on community accepted standards. Industrial Automation is a business with enterprises that are primary sources of capital directly extracted from raw materials, so there is nothing exceptional in constant buzz that is generated by efforts of players to bite off a piece from this cake. But even more don't want to lose their share. The players are huge and so are the problems with Industrial Applications. It takes a lot of efforts to prove something different when it comes against mature and experienced players with a big sum of arguments in their bank accounts.

Industrial Automation applications are meant to be error free and stable, but there are still many errors, flakiness and unstability there. High cost of specialized instruments to debug and troubleshoot problems, low community response, closed protocols, formats and sale-oriented support that is not allowed to give any developers feedback - all these things are on the back side of the modern software development trends. Sooner or later the situation will change and industrial software will undergo the same change that affected internet and made it enjoy the fruits of open source developers collaboration. In the meanwhile let me show an example, how to "Lower your TCO" for the task of accessing data from Siemens S7 controllers.

Introducing Python for Automation World

You can control everything in Python. Starting from virtual game characters and application objects to real robots and trains. Just like you can control them in C. That means that Python is capable to directly use C libraries. Even if they are not really suitable to do such things it gives you the freedom of choice. Pulling slow snake's tail may be more efficient than guarding against fast mosCuitoes aiming at sucking you memory away. Python gives you a finer degree of control focusing your efforts on program logic while in C about 30% of code is dealing with memory allocation and type conversion. Python has more clear syntax than C and that is more important - the same syntax is used to control speedy C libraries.

Controller Data - Corner Stone of Automation

Data is everything. In Automation the primary source of data are sensors - any kind of device that is capable to measure something in common terms. Sensors are usually made to be simple to keep them free from errors, so they are often provide their data using electrical signals. These signals are usually transmitted in rough industrial environment where electrical disturbances are so common that direct connection of such sensors with computers (through COM port, for example) is a reason for frequent PC component replacements. That why specialized devices exist to poll sensor data. These are usually PLCs (programmable logic controllets) and distributed periphery devices that have a higher degree of protection against physical threats than common PC stations. Data inside PLC is the starting point to source of information for are used by industrial panels to visualize and control technical process parameters - link of PLC + visualization panel is commonly referenced as SCADA solution. Access to data hidden inside PLC from a panel is made using proprietary protocols even if the medium is Ethernet. Communication with upper levels require aforementioned Industrial Software provided for a price. But some protocols have alternative open source C libraries or drivers for PLC direct data access and I am going to show how to use such library with Python to run and stop PLC.

Parts of Assembly: libnodave, SWIG and Python

libnodave - communication library for exchanging data with Siemens PLCs released under LGPL license (you are welcomed to support this project if you find it useful). Python is a general-purpose scripting language. SWIG - tool to bind C and Python. Another required thing is C compiler. Because automation designers are mostly windows folks, we will use GCC from MinGW project Minimalistic user interface is done with wxWidgets library, which is also accessible from Python as wxPython library (created with SWIG, by the way) The total cost of all tools is zero. The performance of libnodave solution is still have to be measured. Siemens site contains performance data that can be useful for comparison.

libnodave allows to communicate with PLC with an API described on it's site. To start/stop controller you need to compile and execute the following C program:


#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include "nodavesimple.h"
#include "openSocket.h"

#ifdef LINUX
#include <unistd.h>
#include <sys/time.h>
#include <fcntl.h>
#define UNIX_STYLE
#endif

#ifdef BCCWIN
#include <time.h>
void usage(void);
void wait(void);
#define WIN_STYLE
#endif

#include <errno.h>

int main(int argc, char **argv) {
int useProtocol, useSlot;

daveInterface * di;
daveConnection * dc;
_daveOSserialType fds;

useProtocol=daveProtoISOTCP;
useSlot=2;

fds.rfd=openSocket(102, "192.168.1.244");
fds.wfd=fds.rfd;

if (fds.rfd>0) {
di =daveNewInterface(fds,"IF1",0, useProtocol, daveSpeed187k);
daveSetTimeout(di,5000000);
dc =daveNewConnection(di,2,0,useSlot);

if (0==daveConnectPLC(dc)) {
daveStart(dc);
return 0;
} else {
return -2;
}
} else {
return -1;
}
}


What I am trying to achieve is the following equivalent in Python:

import nodave
plc = nodave.NoDaveTCPConnect('192.168.1.244')
plc.start()


To make this happen the libnodave library should be made available to Python as a module. Module is imported to make its functions available for Python scripters. To make module for libnodave we need to compile library, generate source for module and compile module itself.

[To be continued..]

Wednesday, August 27, 2008

Bazaar Experience

Not so long ago I decided to play with launchpad - site for open source collaboration. It is built around decentralized Bazaar version control system that reached version 1.6 in its development up to this moment. I read that centralized VCS are bad and DVCS are good, but never had enough free time to check this in practice and finally..

The first thing I liked in Bazaar is that it is written in Python - that means that bugs can be found and fixed rather easily. This was also the thing that I disliked, because standalone bzr distribution includes Python and takes 14Mb and package for installed Python distribution doesn't add bzr to PATH

Accessibility. I often work through tunnels, proxies, firewalls and other stuff that kind people tend to put here and there in each Local Area Network. Not only I have to deal with them myself, but sometimes also need to teach others. In SVN proxy settings are stored in documented configuration file called 'servers'. In Bazaar there is no such file and all configuration is done with environment variables and Python settings that are not documented. To use Bazaar through a proxy - set environment variables:

set http_proxy=127.0.0.1:1080
set https_proxy=127.0.0.1:1080

These proxy settings are rather common for linux/unix machines, but cause particular pain with windows machines in a domain. If SVN is able to transparently authenticate against domain proxy without asking sensitive passwords then Bazaar lacks such feature and needs another authorization proxy server (NTLMAPS) middleware.

Usability. In general distributed version control systems should be better than centralized ones. A lot of articles and tutorials describe the theory very well, but in practice there are many things is are not that brilliant. For example, I needed to fix stubborn DevPak plugin for Code:Blocks that didn't want to install one specific version of a curl library. In SVN I would do a partial checkout, make modifications and upload patch to the forum. Not very convenient. In Bazaar I would have to make a branch of the whole Code::Blocks repository, start patching plugin and then merge the whole branch back. But branching the whole repository is an overkill for such little plugin and you can't make a branch out of subtree in Bazaar. More then that - Bazaar doesn't even allow to export subtree of repository. Download either the whole thing or nothing. Not too effective, especially when the traffic is 0.02 cents for Mb.

So, I manually created directory structure for the plugin similar to existing project and started a new Bazaar repository from scratch. After a couple of revisions I released fixed plugin for stable Code::Blocks and would like to switch to latest development code and merge changes made to plugin in trunk into my version. Again, Bazaar disappointed me. I knew that SVN won't allow merging with different repository, but at least I could grab files from the source tree and merge them manually. Bazaar didn't allow me even that. With almost 100 commands it just couldn't export some files.

It may be that Bazaar is a good system for some small projects, but in my case it appeared unwieldy and too expensive even being open and free.

Wednesday, July 02, 2008

Applying Unified Diffs with Python

Windows has a lot of annoyances for developers. One of these is that it lacks some precious tools - namely "diff" and "patch". They can be downloaded from the Internet, but when the latest patch binary provided by Win32 ports of version 2.5.9 refused to apply a patch built with "svn diff" and closed with an error, I decided to write my own version in python. If it will be included in standard python distributive as a logical complement to Scripts\diff.py utility then at least for people with python there will be no problem with applying patches in windows. One limitation though - the script parses only the most popular format of patches - unified diff.



To start out I've outlined a structure of unified diff using information from Guido van Rossum blog and wikipedia.











Parsing logic is implemented using brute-force regex parsing approach to avoid dependencies on parsing libraries (like pyparsing etc.). I took this approach to compare the code with the different techniques of Text Processing in Python by David Merz and learn how can I improve it.



Linefeeds are handled in automagic mode. Proper line ending is detected during scanning of source file. If source file has mixed line endings - lines from patch file are not transformed and written "as is". If lines in source files end with the same sequence - lines from patch file are stripped of their own line ends and applied.



The project doesn't have all UNIX patch options, but should be useful even without them. You may find it with sources (MIT license) at http://code.google.com/p/python-patch/

Tuesday, June 10, 2008

Yet another CVS to SVN transition

Some time ago I've compiled a short comparison of CVS vs SVN. It should not be hard to get the idea from it that SVN is better - easier to understand, easier to setup and to use. While there are many new decentralized systems around like Git, Darcs and Mercurial, SVN still has the best windows working clients that are even able to transparently authenticate against domain proxies. That's why SVN is the most accessible way of getting sources.

Many projects realize that and make switch to SVN to expose their public repositories to a wider audience even if some conservative developers are unwilling to change their habits and tools. Still the convenience that brought by SVN worth the efforts of rewriting tools and moving legacy codebase over to new rails. CVS deserves to be honored a special award in the history of software development, but perspective open source projects realize that evolution never stops and time comes to replace legacy CVS with tools that value developer's time. Here is a list of such projects that made the decision to help with yours:
To migrate from CVS there is cvs2svn tool written in Python hosted on tigris.org that can move CVS repository as big as Mozilla or FreeBSD to SVN or Git.