I saw Linotype, the Film last night, and thoroughly enjoyed it. It's ostensibly about the Linotype machine, the first real improvement in printing technology since Gutenberg. But it's also about the people who are still devoted to this marvel, well after it has been supplanted by digital technology, and a quirky and entertaining bunch they are.

[ http://player.vimeo.com/video/15032988?title=0

I heard about the movie when I went to the Boston Printing Office auction last year. The Linotype movie guys were there filming, and a number of shots from there are in the finished film, though I am not...

Afterwards, there was a Q&A panel with local interviewees from the movie, including the type designer Matthew Carter, who had designed faces for Linotype. He made an interesting comment about the freedom type designers enjoyed once the technology transitioned from metal to photo-typesetting. For example, the Linotype combined roman and italic, or roman and bold, so that the letters of each had to be the same width, a difficult constraint given the inherent differences between the three styles. Another constraint lifted was the need for the symbol to remain within its set-width, a release which enabled liberated scripts like Carter's Snell Roundhand.

I found the technological implications of the machine, its rise, and its demise, fascinating. Design is never completely unencumbered, it's always working within contemporary technological constraints.

I went looking today at other information about the machine itself, and found this extremely detailed instructional video about the workings of the Linotype, part 1, and part 2. It is awe-inspiring how intricate the Linotype was, and how many different functions had to be bundled into this one machine. There's even a small part involving binary numbers: the matrices are binary coded to deliver them back to the proper place in the magazine.

I think about the intricacies of the "machines" I build, and wish that I could lay them out the way these videos lay out the operation of the Linotype. Software has layers upon layers, which seems like a barrier to simple explanations, but if the Linotype can be explained like this, I'm sure software can be described more clearly also.

All in all, I've been thinking about the movie a lot. All I can say is, If you get a chance to see Linotype, the Film, do it.

A short recap of my history with Tabblo, a photo-sharing, storytelling site: I joined the startup in January 2006, we were acquired by Hewlett-Packard in May 2007, and I left HP in December of 2010.

I mention this because another milestone in my relationship with Tabblo was reached this month: not only are all of the original startup employees gone from HP (I was the last), but now all of the employees I hired to work on the server code are also gone. Now I literally don't know the people responsible for the site. In this case, "responsible" doesn't mean, "updates the code for the site," because nothing has been changed on the site in years. In this case, "responsible" means "will fix the servers if they fail."

As I figure it, there's only one milestone left to go: tabblo.com will eventually stop working, and no one will know how to fix it, and tabblo.com will be gone for good. Computers don't run indefinitely. Left alone, servers will go for a long time, but eventually something will break. I don't think anyone at Hewlett-Packard will miss tabblo.com, and I don't think anyone there would know how to fix it if it broke.

I loved Tabblo, both as a job and as a product, and I have a message for the current Tabblo users: leave Tabblo. I know there aren't similar alternatives, but Tabblo will not last forever. You should leave while it is still your choice.

If the past is any guide, some Tabblo users will want to do something to make HP care, to make them pay attention and take care of Tabblo. This is futile, HP won't care, not because HP is bad, but because Tabblo is not a viable business. True, that's partly due to HP's neglect of it over the years, but even when we were acquired, the site was not an interesting business proposition for HP.

When HP bought us, they already had a photo site, Snapfish. It isn't a community the way Tabblo is, and it doesn't allow for the same range of self-expression as Tabblo does. But none of that mattered. Whatever you think of Tabblo vs Snapfish, the fact remains: HP was never interested in Tabblo as a web site. Snapfish already had millions of customers, and generated revenue for HP. HP wasn't about to confuse people by running two photo sites, and there weren't enough paying customers on Tabblo to make merging them a priority. Tabblo was never a money-maker at the scale a company like HP needs.

HP didn't acquire Tabblo to get tabblo.com. They acquired Tabblo so that we could build other web sites that used Tabblo-like technology to make web content printable. Tabblo.com was left running because it was easier to let it run than to shut it down. As time went on, other web sites were run with the same code on the same servers, so shutting down Tabblo was tricky logistically. Now those other web sites are gone, and Tabblo just keeps on running. With the latest employee departures, no one at HP even knows how to shut it down, other than to simply pull the plug.

The Tabblo site is still running, but it won't indefinitely. When it fails, it will be gone. I'm not putting any more pictures on it, and I don't think anyone else should either.

The only reason I don't feel bad about saying this is that Tabblo stopped being a viable site a long time ago, and we confronted the possibility of it disappearing a long time ago too. There were various rumblings over the years of someone at HP finally deciding to shut down the site, but it never happened. Ironically, the reason the site will be allowed to run until it simply dies is because HP knows they shouldn't just shut off the servers. They know that the users deserve some advance notice, that a plan should be put in place for an orderly shut down. But that takes time and attention and focus, and Tabblo isn't important enough to HP to get time or attention or focus. So it will simply run until it dies.

Keep in mind: I don't work at HP, and I don't have any direct knowledge about anything happening there. Maybe things are better than I imagine. But I saw these forces at work while I was there, and I've been in touch with the last ones out the door to know that nothing had gotten visibly better.

My last Tabblo work was writing Tabblo Lifeboat, a tool you can use to download all of your tabblos along with their photos. If you have stuff on Tabblo, give it a try.

To the current users of Tabblo: find something else. Goodbye Tabblo, I love you in lots of ways, I wish it had turned out differently. It was fun. I did the best I could.

Hanging out in the #python IRC channel today, I learned something new about Python comparisons. It isn't so much a new detail of the language, as a way to make use of a detail, a clever technique that I hadn't seen before.

When defining a class, it's often useful to define an equality comparison so that instances of your class can be considered equal. For example, in an object with three attributes, the typical way to define __eq__ is like this:

When run, it shows what happens:

Here the __eq__ method compares the three attributes directly on the self and other objects, and returns a boolean, a simple direct comparison.

But on IRC, a different technique was proposed:

Now when we run it, something unusual happens:

Our __eq__ is being called twice! The first time, it's called with two Thing objects, and it tries to compare a tuple of (1, 2, 3) to other, which is y, which is a Thing. Tuples don't support comparison to Thing's, so it returns NotImplemented. The == operator handles that case, and relying on the commutative nature of ==, tries swapping the two arguments. That means comparing y to (1, 2, 3), which calls our __eq__ again. Now it compares (1, 2, 3) to (1, 2, 3), which succeeds, producing the final True result.

This is an interesting technique, but I'm not sure I like it. For one thing, the code doesn't read clearly. It's comparing a tuple to an object, which isn't supported. It only makes sense when you keep in mind the argument-swapping dance.

For another, it make operations work that maybe shouldn't:

I don't know that I want these comparisons to succeed. It exposes internals that should be hidden. Of course, why would a caller who didn't know the internals try a comparison like this? But things like this have a way of creeping out to bite you.

I'm glad to have a better understanding of the workings of comparisons, but I'm not sure I'll write them like this.

Like many, I use Fabric to write deploy procedures, but I feel like I'm doing it wrong. Fabric is fundamentally based on the ideas of "hosts" and "tasks". You write a Python file whose functions are tasks, and from the command line you can ask that a list of tasks be performed on a list of hosts.

Tasks can be decorated to affect their execution, for example, the @runs_once decorator will mean the function is only executed once, no matter how many hosts are specified. This can be useful for performing initial work, such as preparing a tarball to be copied to many hosts. So for example, I can write something like this:

Fabric will run this by running the deploy task for each host, which will call both make_tarball and copy_tarball, but the runs_once decorator on make_tarball means that it will only be executed for the first host, while copy_tarball will be executed for all of them.

This is great, and building on it for a multi-server deploy, I wanted to have functions that would be run on a subset of the hosts. I have servers divided into roles: app server vs. static content server, for example. Fabric provides a role system, and includes a @roles decorator to control what gets run where:

But we run into a problem: @roles only works on top-level tasks invoked from the command line. If I decorate my copy_tarball function with it, it will be ignored. This is because of how the decorator has been written: it annotates the function with role information, and the Fabric main loop knows how to read that annotation to decide what tasks to run on which hosts.

I wanted a deploy script that looked something like this:

So I wrote my own decorator to do roles the way I wanted:

But I felt funny about this: I saw something in the Fabric docs that sounded like just what I wanted, but it didn't work as I thought it would, so I had to write my own. This makes me think I'm using Fabric wrong.

The runs_once decorator is great for doing one-time initial work, and I found I wanted a book-end for it: a way to do one-time cleanup work. Fabric provided nothing, and I could see why: there's no global knowledge about all the hosts and tasks, and no way to specify work to be done after they are through. For that matter, there's no way to specify work to be done before they start, but @runs_once provides that effect.

So I wrote another decorator, this one more devious and risky:

Here we count the number of invocations, and guess at the number of the last one based on the hosts we know about. There are a variety of ways this could not work, but it was fine in my environment.

Since I'm sharing useful Fabric decorators, here's one that prevents repetitive work being done that won't have any extra effect:

Am I using Fabric wrong? It seems like maybe I'm expecting it to do too much, like the right way is to have my deploy() function in a larger script somehow. Or is Fabric fine like this, and I've just missed the right path?

For the last year, I've been doing a lot of freelance work for Threepress on Ibis Reader, their HTML5 e-book platform. The good news out this morning is that they have been acquired by Safari Books Online.

This is a great fit, because of their already-shared business, technology, and culture. Congratulations to Liza, Keith, Jonathan, and Chuck, who have all done great work, and will now be Safari employees. Xconomy has some sound bites from Liza and her new boss about the acquisition.

The down-side for me is that I will not be working as much with them now, but there may be some smaller projects, and I'm really glad to have had the year with them that I did.

PS: it isn't mentioned in the new blurbs, but Ibis Reader is a Django project, so this is a big plus for Django and Python also.

I like to walk to explore, even in my home neighborhood. So I like to find new places to go. I wanted to print map and draw a 1½-mile radius circle on it to find a good range.

But the set of points you can reach with a mile and a half of walking is not a circle, it will depend on where the streets are. Anyone know of a map app that can show me the actual range?

Mapnificent is a similar idea: it shows how far you can get in a given amount of time, using public transportation, walking, and maybe biking. But it also just draws circles.

Is what I want out there?

Happy New Year everyone, I hope your 2011 was good, and that 2012 will be even better. This year I hear people talking about "intentions" rather than "resolutions," is that a reflection of reality, or an early cop-out so that failure doesn't feel like "failure"?

When I think about the year past and the year ahead, my "intention" is to spend my time more mindfully. That doesn't dictate how I will spend it, or even that I should spend it purposefully, but that I should decide how to spend it, rather than falling into habits and ruts.

In Pico Iyer's relevant and recent widely-circulated piece, The Joy of Quiet, he notes our "progress:"

Luckily, the bulk of the essay is not on this same, "things are so different now" theme. In fact, he quotes Blaise Pascal:

Later, he calls on Thoreau:

Here's the full Thoreau paragraph:

So which is it? Are we in a bad state that has developed in the last generation, or are we merely in the same predicament as Pascal and Thoreau? I think it's easy for people to lament their situation and think their condition is a recent malady. Even Thoreau complains of "modern improvements," but if Thoreau complained in 1854, and Pascal in 1658, then how can we honestly blame 21st-century developments for our situation?

People are omnivores, and along with meat and vegetables, we consume information, attention and distraction, craving as much as we can get. Too much food will make us fat, too much distraction will make us, well, distracted. It's true now, and it was true in Thoreau's time.

The new technologies certainly contribute to the problem, but they didn't create it. Thoreau's quip about the "broad, flapping American ear" could have been written about tabloids, TV, and Twitter, but it wasn't. Iyer's solution is quiet, a respite from connection, but I think we can also use the tools we have in better ways.

I liked Alex Soojung-Kim Pang's conclusion:

So, along with the usual resolutions to eat more healthfully, here's to spending time more mindfully.

I saw this question this morning:

Naturally, the answer here is to make a dictionary keyed by first letter:

The question reminded me of others I've seen on Stack Overflow or in the #python IRC channel:

The thing all these have in common is trying to bridge the gap between two domains: the data in your program, and the names of data in your program. Any time this happens, it's a clear sign that you need to move up a level in your data modeling. Instead of 26 lists, you need one dictionary. Instead of N tables, you should have one table, with one more column in it.

These situations all seem really obvious, but there are more subtle situations where this dynamic appears. I just wish I could think of an example! :)

Tonight is the first night of Hanukkah. Of course, most calendars confusingly label tomorrow with "Hanukkah starts." I wish instead they would label today with "Hanukkah starts tonight."

I've chatted with plenty of friends over the years about the different natures of Christmas and Hanukkah, and finally hit on an apt metaphor: Christmas is like the Super Bowl, Hanukkah is like the World Series.

Like the Super Bowl, Christmas is one concentrated day. For celebrants, it is the only thing happening that day, and it's preceded by weeks of anticipation and preparation. Another similarity is the zeal with which businesses try to piggy-back on the excitement.

Hanukkah, on the other hand, is spread out over about a week, like the World Series. While the daylight passes as it normally would, the evenings are spent specially, and the entire week is tinged with a special feeling because of it. If you have to miss one night for another reason, that's OK, you've got others you can celebrate.

Each has its special feeling, either because of its intensity, or because of its calm. I enjoy them both.

If you celebrate Hanukkah, take pleasure in lighting your candles tonight!

The Stop Online Piracy Act is the latest battle in the war between the movie studios and the rest of us. It's a really bad bill that, depending on who you listen to, would either kill jobs in the US, or radically change the entire Internet, or both.

If you read this blog, you've probably already been pelted with calls to action. You may have figured you wouldn't do anything because it wouldn't have an effect. Do something anyway. It's the right thing to do.

When I was working for Hewlett-Packard, one of the advantages over being at a startup was that we could sign licensing deals with other really big companies, like Disney. I met once with a pair of HP guys who oversaw that sort of work. They used to work for the movie studios in a similar capacity. The day I met with them at our offices I happened to be wearing an EFF t-shirt, just because it was the top one on the pile that morning. When they saw me in it, they looked at me like I was carrying a sign that read, "I shoot babies."

They proceeded to tell me a story about a high-end TV console manufacturer that was building a $50k device that would let you feed in DVDs, and rip them to an internal hard-drive, then you'd have your entire collection of movies on a menu that you could access instantly. The point of the story was that the movie studios forced them to stop making this product, because although it was a closed system that would appeal to their best and richest paying customers, it involved ripping DVDs, and that was bad. So they were closing off new avenues of business because of the internal technology involved. So dumb.

One of them then launched into a rant about what would the world be like if the movie industry was put out of business by piraters? How would people be entertained? "The movie industry performs a valuable service, finding and producing the best entertainment!" Movies are great, it's true, but there's no constitutional right to the existence of a movie industry. It's existed for only about 100 years. Not only would people entertain themselves without it, just as they did before the movie industry existed, they may find better ways to do it.

Technology changes, and businesses sometimes suffer. The horse-buggy industry didn't try to write laws preventing automobiles. The vaudevillians didn't lobby their congressmen to stop movies. Radio didn't restrict freedom of speech in an effort to block TV.

I don't want the movie industry to go out of business, and I don't think it will. Their desperate actions to stop their perceived enemies are wrong-headed and technologically a bad idea.

SOPA is like burning down a house to get rid of the mice. How far are we willing to let these industries go in their desperate effort to stop piracy? Piracy's not a good thing, it's true, but the "solutions" in the SOPA are worse.

Call your congressman.

This is one of those conceptually easy tasks that seems frequently required, and yet needs a complex incantation to accomplish. I have a series of files, and it will grow over time, and I want to clean them up, but keep the most recent N files.

After poking around the Google, I found this for deleting PATTERN, but keeping the five most recent:

That's dash-t-one on the ls command. Or, in words:

That wasn't so hard, was it??

Django is easily the most popular Python web framework these days. For all of its features, and ease of use, though, sometimes it just seems misleading on purpose. This morning I fixed a mysterious problem, and once again I was reminded of how Django can seem simple until things go wrong, and then it's weirdly complex.

In particular, how the settings work is just odd. There are two ways that Django does two things when it would be better to do only one.

For Ibis Reader, our settings machinery is elaborate: the settings file imports from product_settings.py, then from a host-specific settings file, then from a local_settings.py which isn't committed to source control:

This scheme works great: you can put settings in the file that corresponds logically to why the setting needs the value.

But something odd was happening: if a setting was in both product_settings.py and the host settings file, then the value in product_settings won. How could this be? The host settings file is applied after product_settings!

Part of the answer is the first thing that Django does twice that should only happen once: the settings file is imported twice. This flies in the face of everything we know about Python modules, but it happens. So the actual order of imports for my settings files is:

I don't know why Django imports twice, but it's long been true, and I've had to rediscover it the hard way a few times.

But this still doesn't explain the mystery: every time product_settings is applied, host settings should then be applied over it, so why would a setting in product_settings take effect over one in host settings? The answer is in the second thing that Django does twice: adding directories to the Python path.

I don't know if this is really Django's fault, or something about the way people seem to always configure their Django projects, but it seems to very often be true: your source files are available through two different import paths, because your source tree has been added to the Python path twice at two different levels.

A Django project has a top level corresponding to the project ("ibis" in this case), and then apps beneath that. The Python path is constructed so that you can import a file as "my_project.my_app", or just as "my_app". Except that for some reason, this double-view of the source tree isn't always available, and it isn't during that second series of settings imports!? The path is being modified between the two import sequences!

So the import march actually looks like this:

The net result is that settings in both product_settings and host settings will keep the value from product_settings, even though host settings is imported second.

The fix is really easy: remove "ibis." from the host settings import line, taking advantage of the fact that either form will work, and in fact, the second form is more robust since it seems to always be available on the Python path. The settings files still get imported twice, but at least the same thing happens both times.

I still don't understand why all these things happen. I hope part of this is my fault, because then I can fix it for real.