Skip to content

Mob recruitment and leading: Python development

He says they are ‘hiring’ … and it’s an ad hoc group of volunteers trying to make Python better. The problem is significant: how do you encourage and motivate volunteers to join in and add to, rather than detract from, a project such as Python. Hettinger has some rather harsh words about the need to respect others in describing a few cases where good intentions tromped on others’ work unnecessarily. He covers the need for understanding a project identity and philosophy and some of the other factors that bring people together on a big project to make it better.

Raymond Hettinger, “Being a Core Developer in Python”, PyBay2016

What I’ve learned from being a maintainer and core developer for the past 15 years. Thoughts on channeling Guido, stability, hyper-generalization, Sturgeon’s law and [n]egativity, evaluating submissions, inability to predict the future, user centric design, treating mature code differently, believing in or doubting your predecessors, lever arguments and completers, problems of too many choices, implementation details, how much to document, needs of the standard library versus the needs of users, code that is dead on arrival and how PyPI changed everything, orthogonality, importance of skill and expertise, consistency and foolish consistency, optimization and premature optimization, security tautologies, argument ordering, operator abuse, avoiding race to implementation (we can all write working code), the naming of parts, economy of force and complexity balance, feature creep, developing for others, over reliance on Guido, great minds don’t think alike, preference for compactness, and aversion to deprecations. What it means to be completely reliant on long term unpaid volunteers.


Raymond has been a prolific contributor to the CPython project for over a decade, having implemented and maintained many of Python’s great features. He has been instrumental in modules like bisect, collections, decimal, functools, itertools, math, random, with types like namedtuple, sets, dictionaries, and in many other places around the codebase. He has contributed to the modification of nearly 90,000 lines of code in the CPython repository, and has made over 160 changes in the PEP repository.

Raymond has also served as a director of the Python Software Foundation, and has mentored many people over the years on their contributions to the python-dev community. He’s also well known for his contributions to the Python Cookbook, and shares many pieces of Python wisdom on Twitter. He received the Distinguished Service Award at PyCon 2014 for his exceptional contributions to the python community.

The talk is a good lesson on volunteer management and leadership and provides insight into what makes FOSS projects that are built on the work of tens or hundreds or even thousands of volunteer contributors successful.



The Z80 Membership Card

Last month, it was Wes putting together a retro-Pi project to run classic games from days past using emulation on the Raspberry Pi. This time it’s A Classic Retrocomputer Kit from the 1980’s. The idea is to build a classic Z80 computer to fit into an Altoids tin and call it a Z80 Membership Card.

Then in the 1970’s, the microprocessor was invented. Big companies viewed it as a primitive toy that could lure away customers and thus threaten their computing monopoly. But renegade groups of hackers saw it differently. Yes, it was a toy; but also a playground for unparallelled creativity, and a tool of unprecedented power. They sensed that the microcoprocessor was the next great invention that would change the world. The “lights” came on, the “wheels” started turning, and the “presses” quickly started churning out printed circuit boards that would revolutionize the world of computing.

These early microcomputers were pretty crude; like bicycles compared to sports cars. But that’s a good thing! A bicycle is vastly cheaper, and much easier to learn. Yet it can still take you anywhere a car can go, if you’re not in a hurry. And, a bicycle can take you to places that no car can ever go (with better scenery, too)!


The hobbyists that built them were often beginners and outsiders that didn’t know how computing was “supposed” to be done. They made their own rules, invented their own solutions, and came up with entirely new applications that were impossible with traditional computers. In the process, they wound up completely re-inventing the entire computing industry.

That was then. Hardware has changed but the ideas are still there. Build your own is a bit more sophisticated. Now you can build your own, too.

The Membership Card is a complete computer that fits in an Altoids tin. Inspired by classic 8-bit computers like the Altair 8800 and Heathkit H8, it is thoroughly documented and easy to build, with big parts, big pads, and big traces and spaces. It uses only generic parts common in the 1980s (and still available today) — no custom parts, and no surface mount. It’s fully self-contained: You don’t need PCs, Windows, megabyte compilers, or secret software to use it. Now you can learn about computers right from the ground up, and really understand how they work!

This is from the first era of the PC. That runs from the introduction of the first Apple and Radio Shack computers up to about 1982. That’s when the market opened up and the Commodore 64 picked up the home end (see the story in Distrita) and the IBM PC was struggling against Kaypro and Osborne and others for the ‘serious’ or business market. The second era in the mid 80’s was about a transition to disk based operating systems rather than ROM based. Hardware design and production improved dramatically and the competition was shaking out the loose leaves. 

The thing is, I already have several Z80 computers. Why build another? How can I use the ones I have? The TRS-80 Model 100 uses an 8085 and is an ideal size for a ham shack desk keyboard with a status display. The problem with these is illustrated by their serial ports. The RS-232 on these classic machines needed handshaking lines and carefully buffering even at 9600 baud speeds. The R-Pi handles serial communications at 115K baud without handshaking.

I am looking at the HP palmtops I have from the 90’s, too. There was a web site that was big on these a while back as there was a market for custom apps designed for them. The keyboards on these things was quite usable. The breaks in the case are probably the biggest inhibitor about trying to use the parts.

It is the keyboard and display that remain one of the biggest hurdles. When a modern tablet can be had for $50, retro projects lose some of any utility appeal they might have had. That just means that other motivations take the fore and that is a ripe field! There’s a lot of old hardware out there so the question is just what you can make with it.




The hidden elephant with the crypto currency fad

Some claim it is the Next Big Thing and going to overturn civilization. It sounds so wonderful with peace, love, and all that: technology is going to solve all human ills and whatnot. Technology dreams seem to invite a blindness.

Luboš Motl gets to the key issue with Bitcoin: Decentralized blockchain and subjectivity of the wave function – “I don’t find the key “virtue” of the cryptocurrencies – decentralization of the list of transactions – terribly important or practical.”

First, think about the blockchain. This concept is a key part of the Bitcoin network. It’s a list of records that remember which payers have moved the cryptocoins to which recipients. This list is growing. All the new transactions are added on top of the old ones. The new transactions are added in blocks. The blocks are finalized when some miner solves a mathematical problem. The solution is being verified by others.

The whole obsession of the cryptocoin community is the ability of the blockchain technology – and therefore the payment system based on the Bitcoin etc. – to decentralize the list of records. Normally, when you’re making payments through a bank, the bank’s computers remember the “official” balance of each account and the list of payments that have taken place.

There is a proposal to use these technologies to assure voting integrity, too. The implications of a distributed ledger of all transactions (or votes) don’t seem to register. That is one big chunk of data that is copied many times. Then there is the inequality thing about miners. These are the folks in the elite set that has the resources to use brute force to calculate security checksums for blocks of transactions which they leverage to gain income.

Jonah Goldberg found the word “ackamarackus” that seems to fit here.

Secure the pi

Good article at Make: Take These Steps to Secure Your Raspberry Pi Against Attackers. Pretty much standard stuff but good stuff well worth considering even on a project SBC [single board computer].

Why is the SBC a target? Make suggests it might be used for distributed computing to aid Bitcoin miners, as a host for I’net attacks, or as an entry into your LAN.

Besides disabling the default user and access and creating your own, there are instructions about adding security to SSH access, setting up iptables for a firewall, and other system management tasks.

Android updates and maturing technologies

Android is a Java based system running on a Linux Kernel. The inter-tangling between these parts is a conflict of the philosophies behind them and practical realities. That has meant that product evolution has been somewhat hampered and cell phones are static devices rather than living technology expressions. 

ArsTechnica has Android 8.0 Oreo, thoroughly reviewed — “We take a 20,000 word deep-dive on Android’s “foundational” upgrades” by Ron Amadeo. Here are some of the topics covered.

Project Treble gets the device and vendor specific code separated from the Android system code in order to facilitate Android updates. This way, Google can get out security and other necessary fixes promptly without having to depend upon a provider rebuilding their entire software suite.

Android’s Head of Engineering, Dave Burke, put it in his interview with Ars, “Today, it just costs too much to do an upgrade of Android. The amount of work and dependencies are too high.” The goal with Treble is to make it easier, faster, and—most importantly for OEMs—cheaper to pump out an Android update.
Google has a new set of tests, called the Vendor Test Suite (VTS), which ensures the Vendor Interface on a device is properly implemented and future proof. This is a hardware-focused analog to the Compatibility Test Suite (CTS), which ensures the Android app APIs are properly implemented on a device.

Notifications is cited as a killer app and it gets on overhaul most of which is in appearance. This is really a command and control user presentation and gets into background user data synchronization. There are other appearance and user interface changes but it’s the underneath stuff of most interest here. Consider the bloat problem as users add more and more apps and many of these want to do things in the background. That brings us back to “The Great Background Processing Lockdown.”

Android has always been, uh, generous when it comes to allowing apps to do stuff in the background. This leads to some really powerful apps, but Android’s background free-for-all also leads to some apps accidentally (or greedily) sucking down background resources
The goal of locking down background processing is both to not only improve battery life, but also to better manage memory so the device runs more smoothly. He said the team wants Android to have “consistent device performance” over time, which was definitely not the case in the past.

The control over what the machine is doing not only allows for managing a smooth user experience that is consistent over time but also to manage power consumption which should improve battery life.

Limiting scans for location and Wi-Fi is another way to manage battery use. Security is also addressed with a number of updates. See the article for more about one big upgrade of Android.




The system file API

Python libraries use the system API for fundamental file operations and I encountered one example of subtlety in this with the program to scan for duplicate files that had been merged by Fslint using links. A restored tarball had symbolic links that were no longer valid. That caused problems.

OSError: [Errno 2] No such file or directory:

The problem was using os.stat() to determine how many links there were to a file in a tree walkthrough. That particular call, though, barfs if the file it is given to check is a link to a file that doesn’t exist. os.stat() tries to follow the link to find the file properties and returns an error if the file doesn’t exist. To get around this problem, the call needs to be to os.lstat() which only looks at the given file name and not what it is linked to.

This is a system dependent thing so Python will convert an os.lstat() call to os.stat() call if on a system that doesn’t support the difference. In Linux, you can find out more by checking man pages (command ‘man stat’). Also see the Python Docs 16.1. os — Miscellaneous operating system interfaces and scroll down to “16.1.5. Files and Directories.”

not following symlinks: If follow_symlinks is False, and the last element of the path to operate on is a symbolic link, the function will operate on the symbolic link itself instead of the file the link points to. (For POSIX systems, Python will call the l… version of the function.)

You can check whether or not follow_symlinks is supported on your platform using os.supports_follow_symlinks. If it is unavailable, using it will raise a NotImplementedError.

Chasing this particular issue also produced other enlightenment. Python os.walk and symlinks at Stack Overflow provided an example of what Python offers in modern programming concepts. It walks a file tree to sum file sizes.

for subdir, dirs, files in os.walk(sys.argv[1]):
    paths = (os.path.join(subdir, f) for f in files)
    space = sum(os.stat(path).st_size for path in paths if os.path.isfile(path))

that is explained

@Serg It’s not sum, it is the use of iterators that result in space efficiency. You can see that paths is a genexp. So this declarative style ensures that you are dealing with one object at a time while essentially talking about all at the same time. There are no intermediate container data structures. But in this example I used “may be”, as you can see, because there aren’t intermediate lists anyway. But has a side effect, space is changing. – C Panda Apr 7 ’16 at 19:20

Generators and iterators, … Oh, My!

and don’t forget assigning tuples to multiple variables in a single assignment. Those ideas are what separate Python from Fortran or Basic, the high level programming languages of my youth.

It is a good point made. By iterating through a generator, memory is saved because only one file is at bat at a time. The straightforward for loop gathers a list of all the files in the entire tree and that can take up a lot of memory. For the remove links utility, I’d replace the sum() with a file delete and add the number of links condition to the isfile() condition. That could be interesting to play with.

Eclipse fallout

Zach Sutton has pictures of Rental Camera Gear Destroyed by the Solar Eclipse of 2017

With all of our rentals leading up to this event, we warned everyone to view the event with appropriate eyewear and to attach a solar filter to the end of their lenses to protect the lens elements and camera sensor.

But despite our warnings, we still expected gear to come back damaged and destroyed. And as evidence to our past posts of broken gear being disassembled and repaired, we figured you’d all want to see some of the gear that we got back and hear what went wrong.

amazing what can happen when the sun gets focused through a lens …

The most common problem we’ve encountered with damage done by the eclipse was sensors being destroyed by the heat. 

But mirrors and lens iris’s also suffered. Go check out the pictures!

Six Fresh Ways to Think About Retirement Savings

Trent Hamm has a good summary in his Six Fresh Ways to Think About Retirement Savings at the Simple Dollar. From the social security network to retirement fund withdrawal rules of thumb and from basic subsistence levels to choosing a location for cost of living reasons, he provides a good over-view of issues that many don’t consider until it’s too late to do anything.

Electric transport

Sven Gustafson reports that Tesla’s long-range Model 3 gets 80.5 kWh battery, per EPA. A cell phone has a battery in the 2-4 Wh range so this is about the battery power of 20,000 cell phones. A typical standard car battery is good for about 1 kWh so the Tesla, the “long range” version has the equivalent of 80 standard car batteries.

Looking at Gasoline Gallon Equivalent (wikipedia) indicates that the Tesla battery is the equivalent to a 2.4 gallon gas tank (80.5/33.4). Conversion of energy to motion is important to consider here as gasoline engines are perhaps 30% efficient while electrical battery charging is in the 80% range. That reduces the battery to 65 kWh effective and the gasoline equivalent to that to 8 gallons for the same transport capability.

The “long range” for the Tesla is stated to be a bit over 300 miles. Typical gas sedans may get 30 miles per gallon so this represents about ten gallons of fuel for them. That fits pretty well with the theoretical energy capacity and efficiency numbers.

This is why electric vehicles have limited utility. A ten gallon gas tank that takes hours to refill means a vehicle with restricted transport options. For daily commute or occasional errands, this might work. In town commercial service might start to push capabilities. There is no reserve capability to support any long haul uses.

Then there’s the battery costs. This might be offset to some degree by maintenance cost differences. The battery has a lifespan of 5 to 10 years and will cost several thousand dollars to replace. That’s a nearly $1000 per year maintenance cost just for batteries. 

The biggest problem many have with cars and money is that they don’t properly amortize initial and expected expenses. Ongoing costs such as registration and insurance often get separated from transportation. Depreciation is often ignored until its pain is felt at sale or trade-in. Major maintenance items such as tires tend to be surprises. 

There are reasons for the market being what it is. You can fight them or use them to your advantage. Make your choice.

Gnome approaches majority (legal age to vote and drink!)

Alan Day ponders The Gnome Way and that gains a special interest as Canonical is folding up Unity and its dreams of integration of the user interface across all platforms. Day notes that the Gnome project has been able to maintain its core identity, expressed in fundamental principles, for 20 years.

When I first got involved in GNOME, one of the things that struck me was how principled it was. The members of the project had a strong set of values, both about what they were doing and why they were doing it. It was inspiring to see this and it’s one of the things that really made me want to get more involved.
The principles that the members of the GNOME project hold in common play an important practical role. They make problem-solving more efficient, by providing a basis on which decisions can be made. They also help to coordinate activities across the project.


However, GNOME’s principles also have a far more important role: they define the project. They are what makes GNOME GNOME.
Another reason that I want to talk about GNOME’s principles is that the project is going to be 20 years old this year. That’s a long time in the software world! GNOME’s longevity is a great success and something to be hugely proud of. It also comes with an inherent risk: as the contributor base renews itself over time, the project’s collective memory could get eroded. As a result, GNOME could forget its principles and the lessons of the past. It is therefore important to find ways of communicating the project’s principles and values, so that they continue to inform successive generations of contributors.

This illustrates the importance of a coherent identity for an organization. Day describes the principles he considers important. The fact that Gnome has survived and continues a vital existence indicates just how important knowing who you are as an organization can be.