Why Git suxx

$ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.

$ git checkout https://github.com/gratipay/grtp.co/pull/178
 Checkout aborted
 There's no file https:\github.com\gratipay\grtp.co\pull\178 at HEAD

$ git switch https://github.com/gratipay/grtp.co/pull/178
git: 'switch' is not a git command. See 'git --help'.

Solution

Download https://github.com/tj/git-extras/blob/master/bin/git-pr to your git --exec-path.

Python Usability Bugs: Formatting argparse subcommands

Suppose you want to build a tool with a simple interface:

usage: sub <command>

commands:

  status -  show status
  list   -  print list

Python proposes to use argparse module. And if you follow documentation, the best you can get will be this output:

usage: sub {status,list} ...

positional arguments:
  {status,list}
    status       show status
    list         print list

And it you implement proper formatting, your code will look like this:

import argparse
import sys

class CustomHelpFormatter(argparse.HelpFormatter):
  def _format_action(self, action):
    if type(action) == argparse._SubParsersAction:
      # inject new class variable for subcommand formatting
      subactions = action._get_subactions()
      invocations = [self._format_action_invocation(a) for a in subactions]
      self._subcommand_max_length = max(len(i) for i in invocations)

    if type(action) == argparse._SubParsersAction._ChoicesPseudoAction:
      # format subcommand help line
      subcommand = self._format_action_invocation(action) # type: str
      width = self._subcommand_max_length
      help_text = ""
      if action.help:
          help_text = self._expand_help(action)
      return "  {:{width}} -  {}\n".format(subcommand, help_text, width=width)

    elif type(action) == argparse._SubParsersAction:
      # process subcommand help section
      msg = '\n'
      for subaction in action._get_subactions():
          msg += self._format_action(subaction)
      return msg
    else:
      return super(CustomHelpFormatter, self)._format_action(action)


def check():
  print("status")
  return 0

parser = argparse.ArgumentParser(usage="sub <command>", add_help=False,
             formatter_class=CustomHelpFormatter)

subparser = parser.add_subparsers(dest="cmd")
subparser.add_parser('status', help='show status')
subparser.add_parser('list', help='print list')

# custom help messge
parser._positionals.title = "commands"

# hack to show help when no arguments supplied
if len(sys.argv) == 1:
  parser.print_help()
  sys.exit(0)

args = parser.parse_args()

if args.cmd == 'list':
  print('list')
elif args.cmd == 'status':
  sys.exit(check())

Here you may see the failure of OOP (object oriented programming). The proper answer to this formatting problem is just to define a data structure for command line help in JSON or similar format and let people dump and process it with templates. Once option definition is parsed, the information there is static, so there is no need in those intertwined recursive method calls. So just do it in 2 pass - get dataset and render template.

Python Usability Bugs: subprocess.Popen executable

subprocess.Popen seems to be designed as a "swiss army knife" of managing external processes, and while the task is pretty hard to solve in cross-platform way, it seems the people who have contributed to it did manage to achieve that. But it still came with some drawbacks and complications. Let's study one of these that I think is a top one from usability point of view, because it confuses people a lot.

I've got a simple program that prints its name and own arguments (forgive me for Windows code, as I was debugging the issue on Windows, but this works the same on Linux too). The program is written in Go to get single executable, because subprocess has special handling for child Python processes (another usability bug for another time).

>argi.exe 1 2 3 4
prog: E:\argi.exe
args: [1 2 3 4]

Let's execute it with subprocess.Popen, and for that I almost always look up the official documentation for Popen prototype:

subprocess.Popen(args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=False, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0)

Quite scary, right? But let's skip confusing part and quickly figure out something out of it (because time is scarce). Looks like this should do the trick:

import subprocess

args = "1 2 3 4".split()
p = subprocess.Popen(args, executable="argi.exe")
p.communicate()

After saving this code to "subs.py" and running it, you'd probably expect something like this :

> python subs.py
prog: E:\argi.exe
args: [1 2 3 4]

And... you won't get this. What you get is this:

> python subs.py
prog: 1
args: [2 3 4]

And that's kind of crazy - not only the executable was renamed, but the first argument was lost, and it appears that this is actually a documented behavior. So let's define Python Usability Bug as something that is documented but not expected (by most folks who is going to read the code). The trick to get code do what is expected is never use executable argument to subprocess.Popen:

import subprocess

args = "1 2 3 4".split()
args.insert(0, "argi.exe")
p = subprocess.Popen(args)
p.communicate()

>python suby.py
prog: argi.exe
args: [1 2 3 4]

The explanation for former "misbehavior" is that executable is a hack that allows to rename program when running subprocess. It should be named substitute, or - even better - altname to work as an alternative name to pass to child process (instead of providing alternative executable for the former name). To make subprocess.Popen even more intuitive, the args argument should have been named command.

From the high level design point of view, the drawbacks of this function is that it *does way too much*, its arguments are not always intuitive - it takes *a lot of time to grok official docs*, and I need to read it *every time*, because there are too many little important details of Popen behavior (have anybody tried to create its state machine?), so over the last 5 years I still discover various problems with it. Today I just wanted to save you some hours that I've wasted myself while debugging pymake on Windows.

That's it for now. Bonus points to update this post with link when I get more time / mana for it:

• [ ] people who have contributed to it
• [ ] it came with drawbacks
• [ ] have anybody tried to create its state machine?
• [ ] subprocess has special handling for child Python processes

Open Source / Free Standards vs ISO/IEC

Intro

While trying to use Galaxy Note 10.1 as a tablet and remote control device for my Windows and Linux stations, I discovered awesome MIT licensed GfxTablet project (draw on your PC via your Android device):

GfxTablet shall make it possible to use your Android device (especially tablets) like a graphics tablet.

It consists of two components:

• the GfxTablet Android app
• the input driver for your PC

The GfxTablet app sends motion and touch events via UDP to a specified host on port 40118.

It was so awesome in its simplicity and protocol that I couldn't resist to build a Python client for it. It didn't take long (well, a day maybe), before I noticed two errors in its protocol. First is that byte order for fields with 2 bytes length was not described and appeared to be big endian (while I assumed the opposite). Second is that one of the fields described as 2 bytes ushort was actually 1 byte size octet. After reading the source, I found the mistakes and edited the protocol description from the web to fix them, which resulted in this (already merged, yay!) pull request.

Standard on Byte Size

I've got an interesting comment on my pull request:

"octet" is more accurate than "byte" because there are systems and programming languages which have a byte size that is not 8 bits

My natural reaction was "No way! That's can't be true.", but Wikipedia said I am wrong. Luckily, it also said that:

The de facto standard of eight bits is a convenient power of two permitting the values 0 through 255 for one byte. The international standard IEC 80000-13 codified this common meaning.

This was the first time I thought that ISO did something right, so I decided to take a look myself at this standard. I found two copies - IEC and ISO. ISO site has a better SEO department, so they've got a better Google position for their shop. Yes, the ISO and IEC are shops - the price to get official size of byte is ISO CHF 154,00 or IEC CHF 150. They really like to have accounts in Swiss banks for some reasons. I'd advice to sell those standard in Bitcoins instead - it is more profitable in a long term.

ISO/IEC as Commercial De-Facto Authorities

Why do we need some organizations like ISO/IEC that place their name and put limitations around access to de-facto standards that more like any other information want to be free? I'd say that our awesome decentralized and independent approach to develop what do you feel and support what do you want is just not widely exposed to those conservative oldschool bureaucrats, who still live in their own world of central authorities that should dictate people what to do.

Don't get me wrong. There are conflicting points when you DO need to set a standard, and an enforcing organization like ISO/IEC is required (enforcing, because market force business to comply no matter how "recommended" the standard is). The costs of dealing with conflicting parties and convincing them is high, and that's why they set price on papers (the calculation if the price is fair is thankfully out of scope for this post). But the thing that bothers me more is that they set price on assessing the facts that are de-facto standards and common knowledge.

The problem here that we separately don't have a tool to say something as whole. The problem here is that if anybody will try to speak for the whole net, the net would resist and that's natural. Because people tend to say too much in one phrase and they are too smart. The reason is to keep the facts short, clear voices from responsibility and make it all countable and strictly out of politics.

What can be fixed here?

Usually people sign petitions. I propose to extend this just for fun. Make a technical statement that countries should agree on (no politics, please), give people an opportunity to support these openly, say if they don't want to support openly and give ability to support in closed manner (?respect privacy), the same way for disapprovals. Disapprovals may carry a reason. Once this data is in place, let people upvote and see what will happen.

The statement - "the byte is 8 bits".

Then build a list of countries that nationally accepted this statement. Then name this initiative somehow - it is important to keep this strictly technical to shot the zombies.

Once a statement reaches some degree of exposure and human votes per country, country can decide to accept it by placing official signed statement online. Over the time, the statements can be combined into free will packs and signed too. This will allow to sync.

Arguments. I am not sure they are needed for de-facto standards, because you reach consensus not by persuasion, but by collecting overall feeling. However, if there are problems with de-facto ways, and people feel there is something wrong, there should have an ability to "opt-in for a change" to upvote/downvote such arguments too. People should not be ashamed to set a value of "my butthurt" meter when voting or proposing counter-arguments, because we are irrational by our design, and technical problems with standards need more feedback than any other area of development on the butthurt effect.

So, simple statements, public voting, open process, feddback, realtime status and summary on nation adoptions.

Who should do this?

I'd be interested in working on this if I had some place to live in of my own. I'd start from contacting guys from Stack Overflow to reuse open source parts of their experience. Quite boring, right? Well, I am not saying that I want and plan to work on this alone. I am just saying that I am not in a position to take a role of coordinator. I just want to says that if you like the idea, maybe even in some crippled variant, found the resources to go, and want to try, feel free to ping me.

One of the tools that was really close and impressed was (now defunct) http://hammerprinciple.com/ which helped me to discover bad things about my favorite version control and programming language without too much butthurt injury. Hopefully, it will strike back again.