Please protect the right to create public domain works

snap is a package format for faster release cycle on Ubuntu. Recently its snapcraft.yml description file has got license field:

# License for the snap content, based on SPDX license expressions.
license: <expression>

Which led me to discover corporate looking SPDX website that aims to set standard on package metadata. And this metadata doesn't allow public domain code:

https://wiki.spdx.org/view/Legal_Team/Decisions/Dealing_with_Public_Domain_within_SPDX_Files

I expected that such project would be equally inclusive to protect the rights of people to release their code without copyright regulations. I would expect it to work with corporate and public services to enable this option, clarify its meaning and champion changes in legislation. But instead this Linux Foundation project decides that public domain code is too bothersome to deal with.

Most of my Python code is released as public domain. I like the total freedom of not being forced to deal with strings attached, and I want to give this freedom to people who use my code. I don't even want this code to be "my". I do not need copyright at all. Giving credits to people and supporting those who need support - this activity doesn't need copyright. Technology can make it automated. Being present in copyright header doesn't mean that people care. This problem copyright can not fix.

Copyright was invented before the internet, 300 hundred years ago, before git and Mercurial, and lawyers are still (ab)using it in their legal warfare, turning it against those who could show how we could use technology to build a better world, with less stress and more transparency. With the next generation mentality of the internet children, it is possible to create a next generation of ecosystem with cooperative gameplay and better ecological goals. We really need to ask each other personally - "What do you want? What do you need?" - and help each other grow internally - not externally. Collaborate on building better systems, better code, instead of squabbling on what's already been written.

Starting with myself - What did I want? I wanted recognition. I didn't want money. Copyright didn't help me to get recognition. Recognition came from my social activity, from interacting with people, building open source, collaborating together, trying to solve some problems in a better way. Many years later recognition came from inside. From mindfulness, and that was a huge relief - not trying to compete with people and show off to seem better on outside. Copyright didn't help me to get money - they came from solving problems for other people without adding new ones. Money came as a result of interacting with others over the things that I found challenging and interesting enough.

I've been paid money, because I can not survive without them, and my network was responsive to my need. I didn't write CV letters or applied to job fairs or went by advertisement in factories. Once I've become transparent to the network with open hands to be helpful, the money found their flow to compensate me for my time. In seven years I signed only one legal paper, one contract with NDA, and that was the only case when I was not paid after finishing the job. I don't want to live in your legal network - I want to be left in mine. In the network where we transparent and don't attack each other with long legal "electronic documents" that are just a scanned paper. I want to live in a netowrk where my will is a signed data structure on a public blockchain, where people are connected and understand what being a human is not easy, and appreciate our nature and support each other towards greater goal.

I don't mind people playing the copyright game. People need games to play. But I do not approve it. Suing programmers for their pet projects that got more popular than mundane office project, taking money for access to scientific papers and not paying scientists who produced them. I am tired, I am scared, because I am paranoid, and I want to opt-out from this system.

I needed support and I joined the Gratipay project. In Gratipay I didn't do much, but in the process I learned to support other people. People whose code I like, people I admire, or people who experienced difficulties, or just people who know how to have fun. I started contributing to Gratipay, because it was public domain and it was completely transparent down to the last penny - I could leave at any point and use the code that I learned in my other projects. I tried to make it better and contribute back. I learned a lot and the more questions I asked, the more interesting things I learned. About people, about economics, about financial gameplay. That was an awesome time and a great experience. I wish that everybody in their live could join a project like this. Helping completely open and transparent company. Gratipay could not survive in U.S., and now its public domain code is serving Liberapay in France.

Public domain was the only way to opt-out from copyright, and with the decision to abandon the fight for it, Linux Foundation closes the door for me to share my code with no strings attached. Treating public domain works "the same way as any license" is the worst decision the SPDX project could take. Public domain is in the "conflict of interests" zone of copyright lawyers, because it is about ignoring their domain, about the right not to care about their business and not being touched by their activities. The task for protecting public domain is for true legal hackers, the ones who go beyond compliance to see the people on the other end. Because there is always a person behind Twitter handle or GitHub account and the person means a little bit more than a "copyright holder".

SPDX raised an important point - Public Domain is not copyright. The right programmatic solution is it to add a separate copyright-optout: true field that is mutually exclusive with license, because this semantic is completely different from the paper based kludges and cargo cults of the past. It cleans up the space for the next generation of internet connected people to ask questions - how do we give credits and how do we support other people in the age of technology.

Please fight for Public Domain - it may appear more important for positive development of technology than we currently see, and it may provide a foundation for better bringing us together.

ANN: hexdump 3.2 - packaged and pretty executable

https://pypi.python.org/pypi/hexdump

The new version doesn't add much to the API, but completes a significant research about executable .zip packages with proof of concept distribution that is both an installable Python package that you can upload to PyPI and an executable .zip archive.

The benefit is that you may drop .zip package downloaded from PyPI into your source code repository and invoke it from your helper scripts as python hexdump-3.2.zip Keeping dependencies together with application source code in repository may seem like a bad idea, but we find it useful at Gratipay to setup project from checkout in offline mode with no internet connection.

The solution required three modifications to distutils setup.py (see sources):

• force sdist command to produce .zip file
• strip extra dir created at PyPI package root
• add __main__.py to make .zip executable

If you think that it is cool and should be promoted further, feel free to test Gratipay to send me 0.01+, and I'll get the signal to spend more time on reversing native distutils/packaging gotchas.

Shipping Python tools in executable .zip packages

UP201501: Found out how to force setup.py create .zip instead of .tar.gz

This is about how to make Python source packages executable. As a side effect, this also explains, how to run invoke's tasks.py with a copy of invoke .zip shipped with your source code, but without installing it.

You know, Python can execute .zip files. As simply as:

$ wget https://pypi.python.org/packages/source/h/hexdump/hexdump-3.1.zip
$ python hexdump-3.1.zip
/usr/bin/python: can't find '__main__.py' in 'hexdump-3.1.zip'

Well, you need to place '__main__.py' into the root. It will then import what it needs and do something. The only problem with .zip source packages is that their structure (if created with standard Python tools) includes one additional directory at the top:

`-- hexdump-3.1
    |-- PKG-INFO
    |-- README.txt
    |-- hexdump.py
    |-- hexfile.bin
    `-- setup.py

So, even if you place `__main__.py` into the root, it won't be able to import file from `hexdump-3.1` subdirectory.. unless you prepare for it. Here is the solution:

import os
import sys

# add package .zip to python lookup path
__dir__ = os.path.dirname(__file__)
path = os.path.join(__dir__, 'hexdump-3.1')
sys.path.insert(0, path)

import hexdump
msg = hexdump.dehex("48 65 6C 6C 6F 2C 20 57 6F 72 6C 64 21")
print(msg)

Now if you run it, it will print the obvious.

$ python hexdump-3.1.zip
Hello, World!

In Python 3, the result will be wrapped into b'' string. But that's it!

The next hexdump version will likely to ship as executable .zip package

Where you may need it?

It may be useful for scripts automation if you're "vendorizing" dependencies (ship them with your code). It started with need to fix a broken link on Gratipay site. The Gratipay codebase is in public domain clear of any NDA, CLA and all other BS, and that makes it great for people to learn, reuse and enhance. I wish Python internal projects possessed these properties, but PSF politics is another topic.

Gratipay inside site uses make. It doesn't need powerful build tool, such as scons, so I tried to replace it with some simple task automation utility to be more cross-platform. I chose invoke, because of my previous experience with fabric for remote control, and because I saw previous attempts in another Gratipay repository. The necessary condition for new tool is that user experience should not degrade for make users, so main invoke's tasks.py script needed to be self-executable.

I completed the proof of concept by making invoke .zip package executable with the method above, placed it into vendor/ directory, and tuned tasks.py to reexecute itself with invoke .zip This also solved a potential problem with currently unstable invoke API.

tasks.py was modified as following:

import sys
sys.path.insert(0, 'vendor/invoke-0.9.0.zip')

from invoke import task, run

# ... tasks go here

if __name__ == '__main__':
  import sys
  from invoke import cli
  cli.main()

Bonus points

There is one more thing, which is better explained with hexdump example. hexdump package ships `hexfile.bin` data file used for testing. hexdump.py looks for it in its directory (__file__ dirname), and this lookup fails when hexdump.py is in archive. I'd say Python could provide some kind of an API to deal with "virtual import filesystem" (with watchers to track who and when modifies sys.path), but I digress. If you modify the `__main__.py` from the first chapter above to run tests - just call hexdump.runtest() - the execution will finally fail with the following error:

Traceback (most recent call last):
  File "/usr/lib/python2.6/runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.6/runpy.py", line 34, in _run_code
    exec code in run_globals
  File "hexdump-3.1.zip/__main__.py", line 13, in
  File "hexdump-3.1.zip/hexdump-3.1/hexdump.py", line 311, in runtest
IOError: [Errno 20] Not a directory: '/root/hexdump-3.1.zip/hexdump-3.1/hexfile.bin'

(yes, I am hexdumping under root). The problem above was explained by Radomir, and I feel like the post is already tool long to write about it myself.

More bonus points

In theory it is possible to hack setup.py to produce executable source .zip package. By default, setup.py sdist command on Linux creates tar.gz files. So, first it needs to be forced to create .zip file. This can be done by overriding default --formats option:

# Override sdist to always produce .zip archive
from distutils.command.sdist import sdist as _sdist
class sdistzip(_sdist):
    def initialize_options(self):
        _sdist.initialize_options(self)
        self.formats = 'zip'

setup(
    ...
    cmdclass={'sdist': sdistzip},
)

I posted relevant documentation links to this StackOverflow question. The rest - how to inject __main__.py into the root of packed .zip - is yet to be covered, and I am not ready to research it just yet. Some hints may be provided by these answers.

Usability conclusion

I am actually quite happy with the ability to execute python .zip packages (which gave a lot of motivation to write this post), because previously I had to care about how to tell people that a tool is a single script, which can be downloaded from repository or from some dedicated download site (I mean Google Code, which does not support this anymore). I also don't trust my own server for downloads, because one day I found some "benzonasos" running there and I don't even have a slightest idea how it got there.

But now it becomes possible to just download and run the stuff from PyPI to test how it works without messing with creating and activating virtualenvs, and installation of package inside. Of course, if your .zip package has dependencies, they still need to be present on your system, but that's still a time saver. Oh, and that means that my tools can now consist of several modules inside.