Singly is hiring engineers
I’ve written before about joining Singly and a little bit about the open source software we’re building. There’s now a bit more content on our website too, or you could watch my colleague, Jabber founder Jeremie Miller, talk about it on stage.
Suffice to say, it’s pretty interesting stuff, and we need more interesting people to join the team in order to achieve our goals in the next year. In particular, we’re looking for engineers to build highly flexible, resilient and performant back-end infrastructure using Ubuntu, and innovative, usable and beautiful front-end interfaces using HTML5.
For full details and how to apply, check out the Singly blog or contact me directly. At this time, we have a preference for candidates in the San Francisco bay area, but are open to exceptional candidates from elsewhere.
Ada Lovelace Day 2011: Dr. Marian C. Diamond
For Ada Lovelace Day this year, I want to share my appreciation for Dr. Marian C. Diamond.
In years past, I’ve saluted women in the field of computing, which is my field as well. Dr. Diamond, however, is a biologist. Her research includes “neuroanatomy, environment, immune functions, and hormones. In particular, she is interested in studying the effects of the external environment, aging, and immune responses on the cerebral neocortex.” She has, in her words, had a love affair with the brain for about 70 years.
I know very little about biology. The content and methods of her research are, frankly, beyond me, though some of her results have garnered popular attention. She has inspired me by demonstrating that rare combination of gifts: a deep understanding of a technical subject, and the ability to explain it to other people in an accessible way.
In her interviews, articles and lectures, many of which are available online, Dr. Diamond displays these gifts in abundance. Her skill and enthusiasm for both learning and teaching is unmistakable. After applying her gifts in the classroom for many years, digital distribution has now enabled many more people to see and hear her, through millions of YouTube views.
In 1960, she became the first female graduate student in UC Berkeley’s anatomy department, and was apparently given the job of sewing a cover for a magnifying machine. I can only imagine the persistence required to continue from there to become a recognized leader in her field. She has gone on to help many other students along their way, and was named an “unsung, everyday hero” for the support she provided to students outside of the classroom or lab.
As if that weren’t enough, she has also traveled to Cambodia to apply her expertise in helping children injured by land mines. She still teaches today, just across the bay from where I write this, and will turn 85 next month.
Building a personal data locker
If you were building a digital container to store your personal data, what would it look like?
Personal data being information associated with you: your contacts, your photos, the web pages you’ve visited, the places you’ve been, the messages you’ve sent and received, and so on. In short, your stuff.
Here’s my personal wish list of technical requirements:
Of course, this isn’t merely an academic exercise, as my new day job at Singly is about building exactly this type of system. With a technical team including Jeremie Miller of Jabber and XMPP fame, our goal is to develop a personal data platform which meets these criteria and more.
There’s a lot of work to do, but today, you can check out the code and run a locker of your own, which can sync in data from Facebook, Twitter, Google, Foursquare, Github and dozens of other services. It’s a bit of a bear to set up, particularly if you don’t already have API keys for these services, but that’s fairly normal at this early stage of development.
If you try it, or have thoughts about what we’re doing, please let me know in the comments.
Where’s your data center?
Thanks to the tremendous growth of “social” applications over the past five years, we have our pick of services for collecting, saving and sharing our experiences online. We each have collections of photos, contacts, messages and more, spread across multiple popular services like Twitter, Facebook, LinkedIn, as well as many less popular services which address particular needs or preferences. We’re also producing a wealth of “exhaust data” through our browsing history, mobile sensors, transactions and other activity streams that we rarely if ever examine directly.
This ecosystem is becoming so complex that it’s easy to lose track of what you’ve created, shared or seen. We need new tools to manage this complexity, to make the most of the wealth of information and connections available to us through various services. John Battelle calls these “metaservices”, and points to growth in the number of connections between the services we use.
I expect that this next age of information tools will center around data rather than services. Data is a common denominator for these online experiences, a bridge across disparate services, technologies, social graphs, and life cycles. Personal data, in particular, has this property: the only thing that links together your photos on Flickr, Facebook, Picasa and Twitpic is…you.
So where’s your “data center”? I don’t anticipate the emergence of a single service where you do everything. There will continue to be innovation in the form of new and specialized services which meet a particular need very well. There won’t be a single service which is everything to everybody.
Instead, I foresee us wanting to track, save, use and control all of our “stuff” across the web. That’s why my new colleagues and I are working to make that possible.
There’s open source code available on github, a vibrant IRC channel (#lockerproject on Freeenode), and lots more I’d like to write about it. But it’s time to get back to work for now…
DEX finishes first batch of derivative patches for Debian
It’s been a few months since Zack and I announced the DEX project, which aims to improve the Debian ecosystem by working jointly with derivative distributions.
Our first milestone
The goal of our first project, nicknamed ancient-patches, was to clear out an old batch of a few hundred Ubuntu patches whose status was unclear. We couldn’t tell which ones had been merged into Debian, which were waiting in the BTS, and which had yet to be submitted to Debian. All of them were several years old.
I’m pleased to announce that this project is now complete. Thanks to help from David Paleino, Colin Watson, Nathan Handler and Steve Langasek, we were able to clear over 95% of the patches in a matter of days. These were the easy ones: patches which were obsolete, or had already been applied. We discussed the remainder, and resolved all of the patches whose status was still unclear. This left the harder ones: patches stalled in the BTS, and patches where there was no consensus about what to do with them.
One of the stalled patches was merged into Debian via an NMU, eliminating the delta between Debian and Ubuntu. Another had been submitted to Debian by a third party, but was no longer shipping in Ubuntu, so we considered it obsolete for purposes of this project.
This has left only two patches out of the original list of 277. Both of them are filed in the BTS and have been discussed with the relevant maintainer team. One of them is expected to be obsoleted when a new upstream version is packaged, which implements similar functionality. The other is being discussed with the upstream developers, but there is no conclusion yet about whether it can be merged upstream or in Debian.
Conclusions
Although we weren’t quite able to clear the whole list, we still consider the project to be a success because:
What’s next
In the most recent DEX update on debian-derivatives, I highlighted a few important events for DEX:
We’re looking forward to seeing DEX develop further. If you’d like to get involved, come and join us on the debian-derivatives mailing list or IRC (#debian-derivatives on freenodeOFTC).
Matt Zimmerman and Stefano Zacchiroli
Why I’m excited about joining Singly
This summer, I’ll be taking a bit of time off, moving back to San Francisco and starting a new job. I can’t wait to get back to work. Here’s why.
Me and my data
I have a singular relationship with my data. I have a copy of every email I’ve sent since I first got an Internet email address in 1994 (82,000 messages and counting). I have even older files downloaded from BBSes, and passed between friends on floppy disks. Chat logs, text messages, voicemail…I hold onto them all. Anything which is relevant to me personally, I tend to save.
This must seem banal to people who are first getting online today. In the age of Gmail and Flickr, it’s easy to assume that all of your data will be preserved indefinitely, with little or no effort on your part. But for me, it has been hard work over the years, because I’ve done it myself. I’ve carried my data with me to countless new computers, operating systems, storage technologies, file formats and cities over the years. Everywhere I’ve lived, I’ve brought it with me. Physically.
Really?
Why do I do this? Why have I gone to such trouble for a collection of bits? Especially now, why is most of my data still at home?
One pragmatic answer is that I can simply do more with my data when I have a copy. I can work with it using any software I want, including software that I write myself. I don’t have to worry about whether I can transfer it from one web service to another. I’m never stuck using yesterday’s services because my data is never trapped in them. My personal data is always available to me me, always raw, ready and waiting for the next wave of software to come along. When it does, I can load my data into it and keep going. The fact that Facebook and Google disagree over sharing their users’ data doesn’t bother me in the least.
Another reason is that I want to be in control of it. I decide who to share my data with, and when. Some of it, I prefer not to share at all, with any person or company, and I have that choice. Even if a powerful government wants to access my data, I am afforded certain protection under the law, at least in the countries where I’ve lived. If I turned over my data to service providers, my choices and protection would likely be much more limited.
I have a deeper emotional attachment to my data as well. Enfolded within that vast pattern of bits is some part of my self. By sharing my personal data with other people, I show them something of who I am. Increasingly, my personal data is part of my identity. This is more than just a state of mind: it’s been shown that even our “non-identifying” personal data can reveal who we are.
In other words, it’s not just “my data”—it’s “me data”.
Singly
I’m joining Singly because I want to take this concept much further, and combine people, data and software into a different shape with people at the center.
Today, we are creating vastly greater amounts of personal data, and it’s stored in many more places. We leave our trail on the Internet in the form of activity streams, messages and content, spread across different web sites, each with their own inscrutable terms of service and (if we’re lucky) their own API. These disconnected silos prevent us from using all of this information effectively.
Meanwhile, we want—and need—to connect with each other in more ways than ever before. We need applications which can connect us, through our personal data, to the services we need.
Singly is building the technology to make this possible. It will be designed with the deepest respect for the relationship that we have with our personal data, and with a vision for truly personal computing.
Singly is…
This opportunity is a great fit for my interests and experience. Singly aims to be the commercial part of a vibrant open source community, and I’m looking forward to building on what I’ve learned in Debian, Canonical and Ubuntu to help make it a success.
I’ll have lots more to say about it as time goes on. Meanwhile, if you’re interested in following what we’re doing, here’s where:
Moving on from Canonical (but not from Ubuntu)
This June, soon after my seventh anniversary with Canonical, I will finish my job as Ubuntu CTO at Canonical. You can read my official announcement on the Canonical blog.
It’s sometimes hard to believe that it’s been that long since we first envisioned a Linux desktop “for human beings”. When I look at how far we’ve come, at the difference we’ve made to so many people, it’s easy to see how the time passed so quickly.
It’s been an incredible experience for me to play a part in building Ubuntu and Canonical. Whether building a decentralized company of hundreds of people, to a global community project of tens of thousands of members, I’m grateful for all of the learning opportunities along the way. It has been a privilege to work with so many brilliant, passionate and thoughtful people under such auspicious circumstances. There is much I will miss, and I have many memories to enjoy, from all-night global hacking sessions driving toward a ship date, to casual singing and playing music at our many face-to-face events
Nonetheless, it is time for me to seek out new challenges and stretch myself in new ways. I’ll be moving back to the US, closer to old friends and family, and starting a new job with a different type of company. I am leaving behind a capable and dedicated team at Canonical, who I am confident will achieve even greater things in the years to come.
I will remain active in the free software community as a volunteer. I intend to continue to participate in Ubuntu, and to serve on the Technical Board. I will also be continuing my work with Debian and the DEX project. This year, I’ve accepted advisory positions with the Ada Initiative and the Freedom Box Foundation, and will continue to support those organizations and their missions.
I’ll be in Budapest at the Ubuntu Developer Summit for the next week, and look forward to seeing my friends in Canonical and the Ubuntu community.
I’ll have more to say soon about what I’ll be doing next professionally. Watch this space for updates!
DEX: Debian and its derivatives, getting things done together
Since I resumed active status in Debian, I’ve been thinking about how to bridge the gap between Debian and its derivatives*. I’ve spoken at length with Zack, the attendees of the Derivatives BoF at DebConf 10, and the fine folks at the Derivatives Front Desk about the technical and social issues affecting derivative projects, and could probably write a very thorough series of blog posts on the subject.
Instead, Zack and I decided to try doing something about it: we have begun a project to test out a new approach to the problem.
Introducing DEX
DEX is all about action: merging patches, fixing bugs, crunching data, whatever is necessary to get changes from derivatives into Debian proper. DEX doesn’t try to change the way any existing project works, but adds a “fast path” for getting code from one place to another.
DEX is a joint task force where developers from Debian and its derivatives work together on this common goal. As a pilot project, we’ve established an Ubuntu DEX Team focused on merging code from Ubuntu into Debian. With members from both projects, we hope to be able to resolve blockage anywhere in the pipeline. Whatever needs to get done in order to merge an Ubuntu patch, someone in the Ubuntu DEX team will know what to do. If we get good results with Ubuntu, we hope that other derivatives will follow. With thanks to David Paleino, we’re excited that the Utnubu project is merging into DEX as it aligns well with their goals. I’m very grateful to have Colin Watson and James Westby signed up to contribute as well.
Our first project is simple: turn this list green. This is an archive of quite old patches from Ubuntu, most of which have probably been merged already or made obsolete, but they pre-date any kind of tracking system so they need to be verified. Once that’s done, we’ll move on to a new project with a new todo list.
If you want to see Debian benefit from technical work done in derivatives, DEX is a chance for you to act together to make it happen. If you work on a derivative and want to carry a smaller delta, come and join us. I’m sure we’ll learn a lot from this experience.
* There are many instances of great cooperation between Debian and derivative distributions, including joint package maintenance teams, and some derivatives are even part of the Debian project. Nonetheless, there are areas were most people I’ve spoken to agree that we need to do better. This is what I’ve referred to as the “gap”.
Listening to users
In the software community, people hold strong opinions on the subject of listening to users. Some feel that users are an essential source of information for making successful products, as evidenced in the customer development methodology, and seek to involve users deeply in product development. Others believe that users don’t know what they want, invoking the quote attributed to Henry Ford, “If I’d asked customers what they wanted, they would have said ‘a faster horse’”. Some say that user needs are unknowable except through the lens of a marketplace, where people choose in aggregate which products suit them best, and customers “vote with their wallets” (anything else is “anecdata”).
Regular readers will not be surprised that I believe they are all right, but only in certain contexts. The right strategy for involving users in product decisions will depend on factors related to the product itself, the market, and the product development method being used.
One of the most important is the life cycle stage of the product: is it a new and rapidly evolving concept, or a mature commodity, or somewhere in between? Simon Wardley explains this well over on his blog, so I won’t rehash his points here, but will add a few of my own.
If what we’re looking for is inspiration for a new product, it’s here that Henry Ford was right: users generally won’t hand you a complete product vision on a silver platter. They’ll frame their input in terms of what they know, and the choices already available to them. However, this doesn’t mean that users don’t have a role to play in this instance: watching users can be a great source of inspiration. It’s the combination of domain knowledge and passionate imagination which triggers the creative spark. Henry Ford applied his engineer’s interests to a problem which was evident all around him.
If our goal is to test whether a new product is a good fit for its users, there is no substitute for user feedback. We can guess at whether there is a fit, and our intuition may be good, but users are the ultimate judges, and we don’t know if we’re right or wrong until users evaluate it. So ask them! By engaging in dialogue with individual users, we can learn unexpected things which will help to refine the idea. If we don’t find what they think until our new product is released, we risk making something that no one wants. Why wait until it’s too late? It can be challenging to extract useful feedback for a product which doesn’t yet exist, but this effort is well worth it to avoid wasting much more effort in software engineering.
When our objective is to incrementally improve an existing product, individual anecdotes can mislead us. A given change may be an improvement for one user, but a disaster for another. What we want to know is whether the new version is better for the population as a whole, and in this case, we do well to rely on data. There are pitfalls here as well, of course. We need to choose our questions carefully, and realize that users will often resist any change: not because they’re stodgy by nature, but because they have to invest effort in adapting to the change. I think of incremental improvement as a joint investment made between product developers and their users, to improve the whole system of people and technology for the better.
By choosing the right tool for the job, we can make better decisions, improve faster, and ultimately solve the right problem for our users.
A diversity statement for Ubuntu
The Ubuntu website states that “we aim to make Ubuntu a wonderful place to participate”. We developed the Ubuntu Code of Conduct to set a standard for participants to accept each other in the spirit of cooperation, and have improved it over time to state these principles more clearly.
It is implicit in our philosophy that these and other Ubuntu values should hold equally true for everyone. I would like to propose that we upgrade this to an explicit statement on behalf of the project.
I have spoken with many people who were interested in joining a free software project, but were put off because they felt unwelcome. I know various people who participate in Ubuntu today, but sometimes face difficult social obstacles in order to do so. Going forward, I would like for us, as members of the Ubuntu community, to make the extra effort to accept all kinds of people. This may sound simple, but it can be very difficult to put into practice. People often don’t even notice they’ve gotten it wrong, until the offended party points it out to them. We need tools and guidance to make this a reality.
To that end, I would like to propose a diversity statement for Ubuntu. This draft has already received support from a majority of the Community Council, but I’d like to take it a step further. Because I want this to be a commitment that we can all stand behind, I’m also calling for support from the community as a whole. Please give this issue your consideration, and let me know in the comments if you can get on board with an official statement like this. The more support we have, the more real this commitment can be.
Here’s the text. Many thanks to Mary Gardiner, Valerie Aurora and Benjamin Mako Hill for their review and input.
The Ubuntu project welcomes and encourages participation by everyone. We are committed to being a community that everyone feels good about joining. Although we may not be able to satisfy everyone, we will always work to treat everyone well.
Standards for behavior in the Ubuntu community are detailed in the Code of Conduct and Leadership Code of Conduct. We expect participants in our community to meet these standards in all their interactions and to help others to do so as well.
Whenever any participant has made a mistake, we expect them to take responsibility for it. If someone has been harmed or offended, it is our responsibility to listen carefully and respectfully, and do our best to right the wrong.
Although this list cannot be exhaustive, we explicitly honor diversity in age, culture, ethnicity, genotype, gender identity or expression, language, national origin, neurotype, phenotype, political beliefs, profession, race, religion, sexual orientation, socio-economic status, subculture, and technical ability.
Some of the ideas and wording for this statement were based on diversity statements from the Python community and Dreamwidth Studios (CC-BY-SA 3.0).
