Tuesday, November 20, 2012

Cinematic journey approach for Python development

Quotes page (fixed in stone) is silent about the one who said that Python, compared to other languages, allows to directly put thoughts into the code. I couldn't disagree with this, but taking idealistic approach, this was more true with Python 2 when coding on a system level, but not so true with the great coming of the web and i18n. So, what's wrong now? I don't have a clean and up to the point answer, because many people still think that there is nothing wrong with the Python. Probably the right question is: why Python is not better than it is now?

This one of the complicated questions nobody is able to answer fully. 42 is the answer, but does the question clear enough? The question is probably too complex for a good technical answer and should undergo decomposition. The decomposition can be achieved by clarifying. What means "better"? More easy to code in. Why is it hard to code? Here goes a list of problems...


Well, there is no list. Therefore there is no visibility, and without visibility no answer is possible. Gain visibility into the list of problems that make Python not-as-good as we want it to be is the primary step to take to make all subsequent steps reasonably grounded for a good party quest (and sane development roadmap for community to focus on).

Historically there were several driving forces behind Python development - mailing lists, bug reports and PEPs. PEPs more than the bugs. Mailing lists somewhere in between (YMMV).

ML were good until people had a lot of time to follow up. Bugs are good at tracking status of things, but they are tuned for fixing things and scratching issues, so language research naturally falls out of context in bug tracker interface. PEPs.

PEP is a good thing that helped to free Python core from featurecreep damage, provided a basis for discussions over a long period of time and insight into decisions over the language development. But PEPs start to fail, and the reason why they do this is the lack of time and energy to iterate over them. Most people can't say if technology is good or bad before testing it (version control as an example), and PEPs with lengthy pieces of design detail assume prior experience with the problem, require thorough imagination to see if the solution will play well.

PEPs require a lot of concentration - the resource of a big shortage nowadays, especially of professional grade. Which is not a surprise if you look at how good HR and management technologies are developed in modern world to keep people busy and involved. We can only hope that collective minds of big corp.s are somehow bugged with the problem and look for solutions to divert their resource flow to improve the grounds they are standing on. Let's hope that community can back up their support, and also somehow bugged with the problem about how to lower barriers of requirements, responsibility, experience and technical expertise for occasional community member, a student or elderly accountant, to be useful in Python development process. Lets's hope that both parties are interested enough to constantly improve ways to use the resource flow to the fullest extent possible.

There are two things that can be help here (and make Python better that it is now) - first one is to improve visibility. It takes its roots in cinematic industry and it's called scenario. Second one is to improve the process and it is a best practice developed over the time by user experience professionals. This one named customer journey map.

What keeps me away from putting my thoughts into code when I write Python?

"""Python forces me to maintain a lowest level structure of my writing - the indented layout, a good thing. Although this also comes with a pain while debugging, because Gangam style multiline comments require me to remember to indent them as well."""  - this is a scenario. You can add various metrics to it, such as:

  """I have only 7 operational attention slots in my mind, and one constantly falls out, because I have to pay attention to complicated commenting requirement.""" - the metrics here directly influences how deep one person can operate at any given moment. It is basically that multiline comments with strings are stealing concentration.

  """Those indentation errors are driving me mad every time I forget to indent multiline comment for debugging.""" - this says that a person uses iterative approach to debug problems, often commenting a lot, and probably in production environment using non-tuned editor. That's another scenario where Python comment hack doesn't play well.

Scenarios have two good qualities - they are short and can be conflicting between each other. PEP is on the other side - it is self-sufficient. To notice that PEP is contradictory - you need to attentively and thoroughly read it or write it yourself. It takes a lot of time. Scenarios are somewhat emotional, they are easy to remember and refer to. This makes it possible to concentrate on conflicting scenarios, outline conflicting points and concentrate all work around them rather than around vague opinions, which makes the whole process of looking for compromises (or good solutions) more fun and involving.

To summarize, the scenario is a good title to remember and a short story to tell. What is the difference between scenario and a StackOverflow question? Question may not have a story, scenario may not contain questions. What's the difference between scenario, use case and user story? "Use case" is an enterprise slang, "user story" is an agile term. Both may have some definitions. Scenario is just scenario, like in movie. You should replay it to see how it works. Scenario is for humans, it is less formalized and comes with emotions included (YMMV).

Let's skip to another example of problem with Python usability on a higher level - packaging - and present another tool from usability domain that can help with analyzing processes in general.

What's wrong with Python packaging that everybody constantly rewrites it?

I didn't intend to include it here first, but a half an hour ago I spotted this article - http://lucumr.pocoo.org/2012/6/22/hate-hate-hate-everywhere/ If distutils/setuptools had a scenario database for packaging, it could be possible to analyze limitations of Python in regard to each scenario. This analysis is similar to PEP, but not necessary a proposal and not necessary so extensive. Scenario may contain a history of the problem, a short description, summary and link to other conflicting scenarios. The role of scenario database is to aid decision making process and an easy reference for new people facing the same problems.

Scenarios can be universal and it is a good analysis tool. You can substitute Ruby for Python and look how good this specific workflow looks for the different system.

"""I can't list installed Python packages, why?""" - does anybody have a link? """I can't find the answer""", and that's another scenario about usefulness of scenarios.

So, to fix packaging there should be a way to operate with scenarios. There should be at least a list of scenarios (or better indented tree), so that (y-)hackers of a new packaging tool could go over it, think about their approach, tick checkboxes and hopefully, spot and bring to the surface this "Essential Packaging Restraint" that eats a whole generations of people. The point is to spot the problem before starting to code.

The scenario DB will help, but there is another usability tool that can make packaging, bug tracking and other development processes more streamlined (less time consuming, more fun and engaging). This tool is called Customer Journey Map, and it shows to people, who are not experiencing any problems with the process, where those problems are for somebody else. This map is also a good starting point in web site redesigns, conference organizations, all kinds of activities that involve people, or more specific, a single person named "Customer", barriers this guy is facing and steps to remove these barriers.

I can't extend to a great detail about CJM in this post due to time constraints. I was impressed by a presentation of awesome UXpresso team, there might be a video available, but it is likely Russian only, and I've heard of at least one major Python company (wargaming.net) that uses it extensively, so I can only give you a pointer for now. It will be interesting to make presentation of this technology for Python contribution process and talk about CJM at PyCon, but I am unlikely to afford the participation costs, so somebody else should do this.