Showing posts with label rainforce. Show all posts
Showing posts with label rainforce. Show all posts

Thursday, October 09, 2014

Domain disaster, identity, history, account balance and horizons


I've added Python tag to this post, because the traffic above is mostly Python related, so it might be interesting to those concerned about preservation on their posts (or about essential properties of well-known trolls in certain communities). Blogging is not an easy process, especially if you're not an English native and it takes about 4 hours on average to rewrite and reword the content until it satisfies you, so preserving this knowledge is somewhat important. Some of the most emotional and big posts are never published at all (like black hate of Python 3 default system encoding or drafts of Spyder plugin architecture) .

So, what happened with the graph above?

With zero income over the past 2+ years I couldn't keep up with my bills and the domain name registration expired after I couldn't make it. It doesn't cost much ~$10 a year, but it is on the bill with DreamHost web hosting plan that costs $120 yearly + minimal VPS instance for $5/month to run Bitbucket mirrors and other services. I got the money from a $500 gift that I wanted to make to my girlfriend after we broke apart. To be honest I never had a girlfrend, but it was a nice try. It was painful, but in the end these $500 helped to save on debts without sacrificing my desire to meet with friends. I must say it is really hard to sustain any kind of relationship (timely replies, calls, switching to the context of another person, sharing interests, job activities) when you have an ADHD mindset, especially in those moments when you "catch the flow" and just escape everything outside until the feeling passes. The escape is possible. Otherwise it would be a direct way nowhere, but it is always stressful and takes its toll on psychosomatic resilience (how much stress a person can handle until it starts to grow on person's in physical form). Other factors may be contributing too, such as wiring scheme in ADHD trained brain - which can be more flexible so that it is able to diverge resources from maintenance of body systems to cognitive abilities. If there is some virus or generic modification in your body (chronic disease is also an option) - it might (probably will) take this chance to capture more territory when under a stress condition. At a time when rewiring process switches off lights in corridors near the dark corners of immune system. But that's is just a theory, an indication how easy it is for me to become distracted, which in turn is the cause why the situation emerged at all.

Looking at the chart of traffic I think that if that traffic was in any way useful at all, I wish it could be directly converted into paying for domain name and hosting. I don't need extra and don't want to "monetize" content by introducing pestering ads and other nasty stuff from the "business end" - just need the power to keep the systems up and operational. It is quite sad knowing that only a year needs to pass after you die to erase traces of your work from the internet, and even web.archive.org don't help when new owner of your domain uploads robots.txt to block resource-sucking spiders. It will be much more comfortable knowing that what you do will persist to help somebody else save some time in future.

The easy answer would be to just blog on *.blogger.com *.wordpress.com or facebook if you like, and that's already an option for a newer generation, but you own domain is one of the achievements in the ladder of your technical skills. Or sometimes it is just matters for some reason.

One high tech idea to build a sustainable human-less domain preservation mechanism was to build a closed loop around automated recurring payments with bitcoins. That one needs a lot of time to develop and clean up roadblocks for an automatic chain that transforms useful traffic to Etherium or any other similar Bitcoin 2.0 concept. There are two concerns here. First is that useful needs to be defined more clearly. I am sure for some reason that Google already knows and calculates the metric even for posts without views, likes or +1 buttons. Second concern is that there is so much buzz about Bitcoin 2.0, that it looks like people are already fighting to capture this playfield, and I am not sure that these new technologies are really worthy. Investigation and review needs time and it is hard to tell how much exactly without trying.

Another option to sustain the costs was to use DreamHost referral system that through a referral link allows to get bonus for every newly registered user. rainforce.org is a very old account - it uses a deprecated recurring bonus scheme, that shares 10% of new user's payments instead of one-time bonuses, but so far I had only $5 from 10 people registered over last 5 years or so, and I don't think it is a good route to take unless you want to force people to hate you, because you're obsessed with selling them to DreamHost as referrals. rainforce.org also possesses a nice looking donation URL that looks like http://www.dreamhost.com/donate.cgi?id=3333 but it does not answer the questions "how much does this this blog needs to be alive and why?", "what part of a share should I pay?" and more import "why should I pay for this?". In other words there is no transparency and trust in "modern" economy, because openness and transparency hurts it. And if resources become scarce, the economy starts a killing cycle. It is not from economy - it is from ecosystems.

There are always ideas for new experiments and horizons in this area, such as marrying Gratipay concept with BitHub. These project are less obsessed with "doing money software for the money", but they can not escape the "preferential attachment" of people around it, meaning that people will support projects that support money movement more than projects that need actual support, such as PyPy, Spyder and PySide. It might be not even about project support, but more about people contributing to these projects. I don't think that I have a solution to the problem, because it complex, but I think I'd be happy to try if I had a chance.

I tried to submit a proposal for rainforce.org activities to H2020 program as a last chance to escape the corporate clutches and dedicate some time to experiment, but I could not find a single idea over which to subscribe myself to work over for the next five years. In the end I realized that I am alone and hardly the right the right person to do coordination and communication work even though it might be extremely interesting at some point (and hence extremely difficult to resist).

So, in the conclusion of this letter to make it more useful and worthy of 4 hour time, I'd really like to see more people from hard core Python projects that are somehow supported in Europe (such as PyPy and PySide), and especially people from outside who interested in helping them, to take a look at the Horizon 2020 program. It is a funding scheme for EU companies, but organizations from outside (including US and Russia) are welcome to participate with their own funds if they contribute something unique. I believe it is the only way for independent R&D to continue and it will be a pity to miss this chance to experiment with transparent economy schemes and trust systems to support people in open source and open science projects. And it requires more than just coding skills to handle. In fact, it may require a completely different set of skills, even art or design or cognitive sciences - everything that helps better understanding and communication. Because all problems in the world are the problems of communication.

Tuesday, January 07, 2014

Open Source / Free Standards vs ISO/IEC

Intro

While trying to use Galaxy Note 10.1 as a tablet and remote control device for my Windows and Linux stations, I discovered awesome MIT licensed GfxTablet project (draw on your PC via your Android device):
GfxTablet shall make it possible to use your Android device (especially tablets) like a graphics tablet.
It consists of two components:
  • the GfxTablet Android app
  • the input driver for your PC
The GfxTablet app sends motion and touch events via UDP to a specified host on port 40118.
It was so awesome in its simplicity and protocol that I couldn't resist to build a Python client for it. It didn't take long (well, a day maybe), before I noticed two errors in its protocol. First is that byte order for fields with 2 bytes length was not described and appeared to be big endian (while I assumed the opposite). Second is that one of the fields described as 2 bytes ushort was actually 1 byte size octet. After reading the source, I found the mistakes and edited the protocol description from the web to fix them, which resulted in this (already merged, yay!) pull request.

Standard on Byte Size

I've got an interesting comment on my pull request:
"octet" is more accurate than "byte" because there are systems and programming languages which have a byte size that is not 8 bits
My natural reaction was "No way! That's can't be true.", but Wikipedia said I am wrong. Luckily, it also said that:
The de facto standard of eight bits is a convenient power of two permitting the values 0 through 255 for one byte. The international standard IEC 80000-13 codified this common meaning.
This was the first time I thought that ISO did something right, so I decided to take a look myself at this standard. I found two copies - IEC and ISO. ISO site has a better SEO department, so they've got a better Google position for their shop. Yes, the ISO and IEC are shops - the price to get official size of byte is ISO CHF 154,00 or IEC CHF 150. They really like to have accounts in Swiss banks for some reasons. I'd advice to sell those standard in Bitcoins instead - it is more profitable in a long term.

ISO/IEC as Commercial De-Facto Authorities

Why do we need some organizations like ISO/IEC that place their name and put limitations around access to de-facto standards that more like any other information want to be free? I'd say that our awesome decentralized and independent approach to develop what do you feel and support what do you want is just not widely exposed to those conservative oldschool bureaucrats, who still live in their own world of central authorities that should dictate people what to do.

Don't get me wrong. There are conflicting points when you DO need to set a standard, and an enforcing organization like ISO/IEC is required (enforcing, because market force business to comply no matter how "recommended" the standard is). The costs of dealing with conflicting parties and convincing them is high, and that's why they set price on papers (the calculation if the price is fair is thankfully out of scope for this post). But the thing that bothers me more is that they set price on assessing the facts that are de-facto standards and common knowledge.

The problem here that we separately don't have a tool to say something as whole. The problem here is that if anybody will try to speak for the whole net, the net would resist and that's natural. Because people tend to say too much in one phrase and they are too smart. The reason is to keep the facts short, clear voices from responsibility and make it all countable and strictly out of politics.

What can be fixed here?

Usually people sign petitions. I propose to extend this just for fun. Make a technical statement that countries should agree on (no politics, please), give people an opportunity to support these openly, say if they don't want to support openly and give ability to support in closed manner (?respect privacy), the same way for disapprovals. Disapprovals may carry a reason. Once this data is in place, let people upvote and see what will happen.

The statement - "the byte is 8 bits".

Then build a list of countries that nationally accepted this statement. Then name this initiative somehow - it is important to keep this strictly technical to shot the zombies.

Once a statement reaches some degree of exposure and human votes per country, country can decide to accept it by placing official signed statement online. Over the time, the statements can be combined into free will packs and signed too. This will allow to sync.

Arguments. I am not sure they are needed for de-facto standards, because you reach consensus not by persuasion, but by collecting overall feeling. However, if there are problems with de-facto ways, and people feel there is something wrong, there should have an ability to "opt-in for a change" to upvote/downvote such arguments too. People should not be ashamed to set a value of "my butthurt" meter when voting or proposing counter-arguments, because we are irrational by our design, and technical problems with standards need more feedback than any other area of development on the butthurt effect.

So, simple statements, public voting, open process, feddback, realtime status and summary on nation adoptions.

Who should do this?

I'd be interested in working on this if I had some place to live in of my own. I'd start from contacting guys from Stack Overflow to reuse open source parts of their experience. Quite boring, right? Well, I am not saying that I want and plan to work on this alone. I am just saying that I am not in a position to take a role of coordinator. I just want to says that if you like the idea, maybe even in some crippled variant, found the resources to go, and want to try, feel free to ping me.

One of the tools that was really close and impressed was (now defunct) http://hammerprinciple.com/ which helped me to discover bad things about my favorite version control and programming language without too much butthurt injury. Hopefully, it will strike back again.

Tuesday, July 02, 2013

Code Review with Rietveld and Mercurial Queues

UPD 2014-07: On Windows, thanks to bug in subprocess that doesn't escape ^ character (caret) you need to escape it manually in all commands, like this ^^.

Teal Deer

    $ hg qser -v
    0 A supported-vcs
    1 U noul

    $ python upload.py --rev "'supported-vcs'^1:'supported-vcs'"

Also FAQ.

Rietveld, pre-commit and post-commit reviews


You probably know what Rietveld is - it allows you to send uncommitted changes for review with upload.py script that you grab from review server:
    $ python upload.py
    Upload server: codereview.appspot.com (change with -s/--server)
    New issue subject: Test
    Issue created. URL: http://codereview.appspot.com/10864043
    Uploading base file for README

This is a pre-commit review, where you discuss changes before they are committed, and add a link to passed review into commit message. In post-commit reviews you usually comment on existing revisions in external project history browser service, such as Google Code or GitHub. I you need a thorough examination of every change, then post-commit review process can become a challenge of catching a running train. If you just need to skim over the committed code and express doubts if anything is unclear for further discussion, the post-commit is ok. In fact we use this way for Spyder IDE development.

Post review is nice in the sense that development is not stopped while you're offline for a vacation, or your braincells are overwhelmed with more time pressing matters. Pre review is good if you want to ensure maximum stability for your system. Ideal development process should have convenient means to support both simultaneously.

--rev hack


But back to the Rietveld. Little is know that it supports reviewing existing changesets and revision ranges with the --rev argument of upload.py For example:
    $ python upload.py --rev "2869^1:2869"
    Upload server: codereview.appspot.com (change with -s/--server)
    New issue subject: Spyder IDE changeset 2869
    Issue created. URL: http://codereview.appspot.com/10866043
    Uploading base file for spyderlib/plugins/configdialog.py
    Uploading base file for spyderlib/utils/external/lockfile.py

This will upload revision 2869 (5a3d6821eabe) from Spyder repository for review. Why --rev is a hack? As you may see there is no original commit message, no revision number, no parent revision info. Actually, Rietveld doesn't know anything about that. What is gets from upload.py is a patchset as a plain SVN diff. Data loss could be prevented with extensible changeset format, which will greatly improve tool interoperability, but I don't know any entity that could support the time required for that development.

The --rev is a hack not only because of information loss, but also because it serves double purpose. The purpose is either:
  1. --rev REV specify the base revision to compare current change to
  2. --rev REV1:REV2 specify revision range to create the diff
To implement it the proper way, the upload.py needs -c CSET, --changeset CSET argument that will extract revision diff from the current version control history, propose to supply commit message as an issue description, save parent revision and changeset hash info and maybe cache the issue number for that changeset locally.

Uploading stuff from Mercurial Queue


Mercurial Queue is just a bunch of patch files in .hg/patches/ directory that can be applied or reverted in order specified by series file. This may change in future, but everybody knows and uses this fact. There is also a less obvious fact (that took long time for me to discover) that every applied patch is also a full fledged Mercurial revision. While it can be applied/reverted with qpush/qpop, as long as it applied, all other operations are working on it as well. In addition, every such revision gets a tag that is essentially this patch name.

So, to upload a patch from Mercurial Queue, make sure it is applied:
    $ hg qser -v
    0 A supported-vcs
    1 U noul

And use its name as --rev range argument:
    $ python upload.py --rev "'supported-vcs'^1:'supported-vcs'"

Make sure it is escaped as shown, because otherwise Mercurial will read is as a subtraction of tag named vcs from tag supported.

With Mercurial Queues it is also much-much easier to send patch updates. For that make sure your patch is active:
    $ hg qseries -v
    0 U supported-vcs
    1 U noul

It is not, so apply it:
    $ hg qpush 
    applying supported-vcs
    now at: supported-vcs

Do your modifications according to review comments and refresh the patch:
    $ hg qrefresh

Remember Rietveld issue number and use upload.py to update existing issue:
    $ python upload.py -i 10822044 --rev "'supported-vcs'^1:'supported-vcs'"

Stuff ToDo and Bitcoin Extortion


As I said Mercurial Queue integration can be improved. It is the 9th most requested feature in Rietveld at the moment and at least for me there is a clear reason why. The workflow for upload.py should look like the following:

(no arguments specified)
  • check that current VCS is Mercurial
  • check that hg diff output is empty
  • check that there is MQ patch applied
  • check if there is Rietveld issue number for the patch
    (needs local storage for this number)
  • check if the patch is not already uploaded
    (actual if issue number is found)
  • propose to upload a patch
  • propose to use commit message for patch description
    (propose to open editor to edit description)
  • submit

This is just an entrypoint for further enhancement to reach the ideal workflow, which will require more fixes not only in upload.py, but also on server side.

I am not sure I will be able to code this given the fact that I am to be sold into full time job slavery for my debts. That's why I leave this research here for further development. With that decomposition it should be easy to implement it and send a patch in exchange for some credits.

For those who'd appreciate the feature, but whose time/money ratio is leaning towards 0, there is a way to show the interest through a timeless fund (tracked here) with a total goal limit of 4 BTC for improved MQ support in Rietveld. I could name a few reasons why you should do this including the chance to trade money back for some life time in future, not only for yourself, but for the whole lot of open source projects that use Mercurial with Rietveld (such as core Python development). While all these are argument good, this is not a sponsored development, as I will likely to do this stuff sooner or later regardless of the goals met. I just need some Bitcoins to experiment with, and because I don't have anything else to trade, here is a good opportunity to give something back in exchange.