Gumption (via Mowser)

Design for Health: Notes from a Multidisciplinary CSCW 2012 Workshop

I participated in an incredibly well organized and facilitated workshop on Brainstorming Design for Health: Helping Patients Utilize Patient-Generated Data on the Web on Saturday. The participants represented a diverse range of backgrounds and interests - even for a workshop at a Computer Supported Cooperative Work (CSCW) conference, already a particularly diverse community. Our workshop had representation from fields including computer and/or information science (especially data geeks), design (several flavors), anthropology, urology and even veterinary medicine.

After outlining the agenda and going around the room with brief introductions, we were treated to a remote keynote presentation by Paul Wicks, Director of Research & Development for PatientsLikeMe, a web service for compiling and provisioning patient-reported data for use in clinical trials. Paul offered an overview of the organization, highlighting some of its successes - including the discovery of the ineffectiveness of lithium for treating ALS and a more recent study revealing the positive user experiences of PatientsLikeMe users with epilepsy (55% of respondents consider PatientsLikeMe â€œmoderately or veryâ€ helpful in learning about the type of seizures they experience) - and some of the challenges they face with respect to complexities (ontologies for symptoms, diagnoses and treatments) and incentives (ensuring that patients who give something get something).

The epilepsy study was paticularly interesting to me, as I've explored the web 2.0 service on behalf of my wife, who suffers from a few chronic conditions, and we were both struck (and personally disincentivized) by how narrowly structured the interface for describing conditions was. Paul acknowledged the regimentation of the data and interface, but noted that very little progress has been made in using natural language processing techniques to effectively extract useful data from less structured patient descriptions of symptoms, diagnoses or treatments, despite a great deal of effort.

PatientsLikeMe takes a very pragmatic approach to serving as an intermediary between patients' data and organizations that are willing to pay for that data, and so they focus on the sweet spot of data that can be relatively easily collected and provisioned. I was glad to hear about the epilepsy study, as it provides evidence that some patients are also reaping benefits from sharing their data. More generally, Paul was forthright and even evangelical about the business orientation of PatientsLikeMe (they are a for-profit corporation) and encouraged all workshop participants to think about sustainability - beyond the scope of government grants, business contracts or other relatively short term fors of support - in our own work.

After the keynote, we were partitioned into four working groups, all of which were tasked with defining a problem, designing a solution and reporting back to the broader group. The focused small group activity provided a context for stimulating discussions about a range of issues involving health, data, users and design, and the time constraints provided an impetus to keep things flowing toward a goal.

Methods for processing narrative versus numeric data Depicting a diversity of opinions and experiences embedded within patient-generated information Working with "lay" concepts and language and their alignment with complex medical issues Being mindful with privacy-enhancing methods for data handling

I started off in Group 1, as I am interested in narrative data (the hard problem currently avoided by PatientsLikeMe), but it quickly became apparent that most of the other members in the group were primarily interested in the relatively short narratives that unfold on Twitter rather than the longer form patient narratives - such as one might find in blogs or online support forums - that I am primarily interested in.

During the report outs after breakout session 1, Group 3 described a persona, "Kelly", who was so remarkably similar to my wife, and her epic digestive health odyssey (which we described in a rather epic long form narrative blog post last August), that I decided to switch groups during the lunch break. Group 3 was particularly diverse - with two interaction designers, a graphics designer, an MD, an anthropologist, and a few folks (like me) with computer or information science backgrounds - and most of the members came were oustide of the traditional CSCW community (which itself is rather diverse). This diversity, coupled with the participation of people who could personally relate to the plight of the patient persona of "Kelly", enabled us to make good progress on our design for helping "Kelly".

The first - and in this case, probably final - design, "Health Tryst", was modeled - in both name and functionality - on Pinterest the increasingly popular online pinboard for "organizing and sharing the things you love", and included features for helping a patient with irritable bowel syndrome (IBS) navigate to relevant information and online support groups that might help her (or him) cope with a chronic condition, and to share these items with others.

I won't go into all the details of the design, as I believe the most valuable aspect of the process were the discussions that arose in the context of designing something that would be useful to a patient like "Kelly" suffering from IBS. The one feature I will highlight is a capability for Kelly to enter her own personal narrative using her own words, and the application would automatically seek out both synonyms used more commonly in the medical and/or patient support communities, as well as automatically link to resources associated with the themes and topics indicated in her narrative.

The unfolding design sketches and scenario offered effective props to keep those discussions focused and flowing. I'm not sure if anyone will carry the design forward, but suspect everyone involved came away with a keener awareness of some of the issues faced by "Kelly" and the medical providers and online community members who might help her.

There were several other interesting designs that emerged from the other groups. Unfortunately, I didn't take good notes on them, and so cannot report on the other designs. As was the case in our group, the designs served to spark interesting discussions within and across groups on issues relating to health, data and technology. However, I was struck by a general theme that emerged (for me), which I believe was particularly well summed up in a recent Forbes article by David Shaywitz on Medicine's Tech Future, from last week's Future Med 2020 conference:

thereâ€™s a huge gap between the way many technologists envision medical problems and the way problems are actually experienced by physicians and patients

Our group was extremely fortunate to have good representation of both physicians and patients, as well as technologists and designers. In fact, if I had to select the highlight of all the highlights of the entire day, it was the discovery of an incredibly powerful visualization of a medical history timeline created by Katie McCurdy to use as a prop in discussing her chronic conditions in an initial interview with yet another new physician, and the confirmation by the physician in our group that this was, indeed, exactly the kind of prop that he and other physicians would likely find extremely useful in such a context.

At the end of the workshop, we discussed a number of ways we might move forward in the future. I think one of the most effective ways to move discussions - and designs - forward will be to ensure broader participation from patient and physician communities, perhaps organizing or participating in workshops associated with some of these other communities.

I also think that health applications offer a perfect context within which to organize unconferences to bring together designers, developers, patients, physicians and business folk. I participated in a civic hacktivism project at Data Camp Seattle last February, but the Hacking 4 Health unconference in Palo Alto in September 2010 is a more relevant example that might be emulated to help move things forward. The Health Foo Camp this past July offers another unconference event that might be of interest to those who want to continue designing for health, and anyone interested in participatory design in the context of health might also want to check out the Society of Participatory Medicine and their blog, e-Patient.net.

The main conference is about to start, so I want to wrap this up. I am very grateful to the workshop organizers - Jina Huh and Andrea Hartzler - for bringing us all together and providing the perfect level of structure for promoting engaging discussions and designs on a topic that is of such great interest to all participants, and I look forward to future opportunities to practice designing for health.

Update: I'm including a couple of related blog posts, in case they help facilitate links across communities interested in this area (these and other related posts can be found in the Health category of this blog):

The Multidimensional Role of Social Media in Health Care (ACM interactions forum article) Social Media and Computer Supported Cooperative Healthcare (summary of some earlier CSCW papers in this area, and links to some leading voices in the Health 2.0 movement)

Posted by Joe McCarthy on February 13, 2012 at 08:19 AM in Conferences and Workshops, Health, Research, Web/Tech | Permalink | Comments (0) | TrackBack (0)

Technorati Tags: cscw, design, health, medicine, technology

On the Personal Philosophy of Carl Rogers

A while back, I was delighted to discover the source of one of my favorite quotes:

What is most personal is most general.

The quote is from psychologist Carl Rogers' 1956 essay "'This is Me': The Development of My Professional Thinking and Personal Philosophy", which can be found in the first chapter in his 1961 book, On Becoming a Person: A Therapist's View of Psychotherapy. I bought the entire book based on this one quote, and its elaborating paragraph. As I anticipated, the book is full of inspiring insights and experiences, far more than can be adequately expressed in a single blog post. I already applied and integrated some of his wisdom in a post on client-centered therapy, student-centered learning and user-centered design. In this post, I will excerpt some sections from his "This is Me" essay.

"This is Me" traces Rogers' personal and professional development, and the insights he gained into himself, his profession and the institutions and disciplines with which he was affiliated. Of particular relevance to me, in my own current professional context as a non-tenure track senior lecturer, is his judgment about the tenure process (based on his own context of having been hired, with tenure, at The Ohio State University):

I have often been grateful that I have never had to live through the frequently degrading competitive process of step-by-step promotion in university faculties, where individuals so frequently learn only one lesson - not to stick their necks out.

However, it is the more personal "significant learnings" that he shares that I find most inspiring (an example, perhaps, of the most personal being the most general). He is very careful to state at the outset that these learnings are true for him, and may or may not be true for others. Throughout the book, he carefully delineates data, feelings, judgments and wants, and he states each of his significant learnings with phrasing that makes the personal nature of his perspective clear. I will include the most direct statements of these learnings (with his emphasis) below, along with a few other excerpts that elaborate the learnings in ways I find personally useful.

In my relationships with persons I have found that it does not help, in the long run, to act as though I were something that I am not. I find that I am more effective when I can listen acceptantly to myself, and can be myself ... a decidedly imperfect person ... the curious paradox is that when I accept myself as I am, then I can change. I have found it of enormous value when I can permit myself to understand another person. ... understanding is risky. If I let myself really understand another person, I might be changed by that understanding. I have found it enriching to open channels whereby others can communicate their feelings, their private perceptual worlds, to me. I have found it rewarding when I can accept another person. ... it has come to seem to me that this separateness of individuals, the right o each individual to utilize his experience in his own way and to discover his own meanings in it - this is one of the most priceless potentialities of life. The more I am open to the realities in me and in the other person, the less do I find myself wishing to rush in and "fix things." I can trust my experience. ... when an activity feels as though it is valuable or worth doing, it is worth doing. Evaluation by others is not a guide for me. Experience is, for me, the highest authority. I enjoy the discovering of order in experience. ... Thus I have come to see both scientific research and theory construction as being aimed toward the inward ordering of significant experience. ... I have, at times, carried on research for other reasons - to satisfy others, to convince opponents and sceptics, to get ahead professionally, to gain prestige, and for other unsavory reasons. These errors in judgment and activity have only served to convince me more deeply that there is only one sound reason for pursuing scientific activities, and that is to satisfy a need for meaning which is in me [an example, perhaps, of irritation-based innovation] The facts are friendly. ... while I still hate to readjust my thinking, still hate to give up old ways of perceiving and conceptualizing, yet at some deeper level I have, to a considerable degree, com to realize that these painful reorganizations are what is known as learning. What is most personal is most general. It has been my experience that persons have a basically positive direction. ... I have come to feel that the more fully the individual is understood and accepted, the more he tends to drop the false fronts with which he has been meeting life, and the more he tends to move in a direction which is forward. Life, at its best, is a flowing, changing process in which nothing is fixed. ... It is always in process of becoming.

I hope this blog, and my personal and professional life will continue to evolve in a positive direction. Meanwhile, I will surely continue to incorporate other of Rogers' insights - implicitly or explicitly - in my future thinking and writing.

Posted by Joe McCarthy on January 31, 2012 at 10:58 AM in Books, Inspiration, People, Psychology, Relationships | Permalink | Comments (0) | TrackBack (0)

Technorati Tags: carl rogers, humanistic psychology, psychotherapy

Scott Berkun's Personal Insights on the Experience of User Experience Professionals

Scott Berkun shared some mistakes and lessons learned from his experience as and with user experience (UX) professionals last night at a meeting of the Puget Sound Special Interest Group on Human-Computer Interaction (SIGCHI). In a highly interactive session, he also invited the 40 or so other UX professionals who attended the meeting to share their own mistakes and lessons. The most prominent lesson I took away from the evening was one that applies much more broadly than to the UX (or HCI) profession: being a specialist generally means most people won't know what you do, so you must always be prepared to give a brief "101" explanation - and/or demonstration - to the uninitiated about what you do, how you contribute value and why others should care.

Scott began with a playful scree about the proliferation of titles - user experience researcher, usability engineer, interaction designer, etc. - that has led to a factionalization of UX, and suggested that we do away the variants and focus on the primary verb that unites the different roles: design. He then presented his list of UX mistakes and provoked a lively discussion that revealed that many experiences of many ~~user experience professionals~~ designers involve many of the mistakes (and lessons) he listed. Having recently written about the interrelationships between client-centered therapy, student-centered learning and user-centered design, it struck me that the root of many of the mistakes arise from a deficit in transparency, acceptance and/or deep empathic understanding on the part of one or more parties.

A large portion of the discussion revolved around issues of credibility, and the challenges designers face when working with teams composed primarily of software engineers and/or business folk. Many of these challenges arise from others' lack of transparency, acceptance or understanding of design[ers]. However, some challenges result from an unwillingness on the part of some designers to fully understand the needs of their other team members. Scott described one category of mistakes as "Vulcan pretension", an approach in which a designer focuses [only] on collecting, analyzing and reporting data, and is more concerned with the number of studies produced rather than how the results of those studies will be effectively applied to the problem(s) the team is trying to solve. Organizational and individual incentives that reward designers based on the count vs. impact of studies only serve to reinforce and exacerbate this issue.

Scott highlighted the multidimensional facets of usability involved in the design process: a designer might create an incredibly rich mockup of an interface that represents the epitome of usability for the eventual users of the product, without paying sufficient attention to how developers - another important set of users - will use that mockup to implement that interface. Taking care to specify details such as colors, fonts and sizes of different interface elements greatly eases the usability of the mockup for the developers who have to use it ... which also helps build credibility for the designer. Design is an inherently iterative process, but It is important to iterate with the developers - not just the intended end users - so as to become better acquainted with the developers' challenges as early as possible in the design process.

Another suggestion Scott had for designers to gain credibility was to find allies. Even if no one else on the team "gets" design, a designer can at least identify the one person who is least unappreciative, and invite that person to coffee or create another 1:1 interaction opportunity to help that person better appreciate the process and products of design. And if designers finds themselves in meetings without anything useful to contribute, it is best to be transparent, and talk with the manager about whether or how to set the stage to make contributions, or raise the prospect of not going to future meetings. This last suggestion sparked some interesting discussions about meetings, and about laptops in meetings making people more productive even when they cannot contribute [much] ... but Scott - fortunately - steered the discussion away from a potential rathole on meetings. Throughout this portion of the discussion, I found myself musing about Goethe's provocative insight in The Holy Longing:

Tell a wise person or else keep silent.

Another set of common UX mistakes involves what Scott calls a"Dionysian pretension" - being intoxicated with lofty ideas without sufficient concern for their applicability - and is also related to Scott's mistake category of "never get dirty". A willingness to roll up one's sleeves and do the dirty or disagreeable work the team must slog through - e.g., sticking around to participate in late night bug bashes - both yields a deeper empathic understanding [my words] of challenges faced by other members of the team and builds greater credibility among them. This wisdom aligns well with a post I recently encountered by David Heinemeier Hansson of 37signals arguing that There's No Room for the Idea Guy, in which he emphasizes the relative importance of execution vs. [only] ideation.

I would share more details of Scott's presentation, but I know he is planning to write his own blog post on the topic, and didn't want to steal too much of his [mind]fire. I will update this post with a link when his post is available.

[Update: Scott has posted a far more thorough writeup on the top 10 mistakes UX designers make.]

Posted by Joe McCarthy on January 27, 2012 at 11:19 AM in Events, Web/Tech, Work | Permalink | Comments (0) | TrackBack (0)

Technorati Tags: design, hci, sigchi, usability, user experience

Client-Centered Therapy, Student-Centered Learning and User-Centered Design

I recently finished Carl Rogers' 1961 book, On Becoming a Person: A Therapist's Guide to Psychotherapy, in which the renowned psychologist describes his approach to client-centered therapeutic relationships. Rogers makes a compelling case for extending his approach to cultivating relationships with his clients to all personal and professional relationships, including those between parents and children, managers and employees, and teachers and students. I'm currently teaching a senior-level undergraduate course on human-computer interaction (HCI), and believe that Rogers' approach is also well suited to relationships cultivated in the practice of user-centered design (UCD), which constitutes one of our primary lenses for the course.

Rogers states his guiding question in the second chapter of the book:

How can I provide a relationship which this person may use for his own personal growth?

He goes on to describe three conditions that characterize his approach to therapeutic relationships (and all relationships):

Transparency or congruence: "I have found that the more I can be genuine in the relationship, the more helpful it will be ... rather than presenting an outward facade ... It is only by providing the genuine reality which is in me, that the other person can successfully seek the reality in him." Acceptance: "I find that the more acceptance and liking I feel toward this individual, the more I will be creating a relationship which he can use. By acceptance I mean a warm regard for him as a person of unconditional self-worth - of value no matter what his condition, his behavior or his feelings." Deep empathic understanding: "I also find that the relationship is significant to the extent that I feel a continuing desire to understand - a sensitive empathy with each of the client's feelings and communications as they seem to him at the moment. Acceptance does not mean much until it involves understanding. It is only as I understand the feelings and thoughts which seem so horrible to you, or so weak, or so sentimental, or so bizarre - it is only as I see them as you see them, and accept them and you, that you feel really free to explore all the hidden nooks and frightening crannies of your inner and often buried experience."

Other characteristics of a facilitative relationship include "attitudes of warmth, caring, liking, interest, respect" or "unconditional positive regard", a recognition of personal boundaries and the separateness of the other person, an allowance for the other person to be utterly free to be himself or herself, and a willingness to see things from the other's perspective and to step into the other person's private world "so completely that I lose all desire to judge or evaluate it". Rogers believed that each person already has the potential solutions to their own problems somewhere within them, and so the goal of the therapist is to be a "midwife to a new personality", creating a safe container within which that internal knowledge can be discovered and applied by the client. This approach is in sharp contrast to the more traditional authoritarian approach to psychotherapy - and many other health care fields - wherein an enlightened therapist diagnoses problems and prescribes solutions for the unenlightened client.

Rogers applies these conditions to many other types of relationships, but of primary importance to me (in my current context) is the application to learner-centered education. In Rogers' view, the teacher should embody the characteristics above, and provide resources relevant to the domain of study (as well as being a "resource-finder"). Students are then allowed to use these resources however they see fit to discover, appropriate and apply the knowledge that they believe will be most relevant to them.

When Rogers taught a course, he would show up the first day of class with stacks of papers and tapes (e.g., of recorded therapy sessions), introduce himself, invite students to introduce themselves - if they felt so inclined - and then wait for them to structure the learning process in which they would participate throughout the course. He did not provide a syllabus, assign homework or readings, nor give any tests. This unorthodox approach sometimes entailed several awkward sessions at the outset, during which students would demand or implore him to impose structure, but he would kindly and resolutely refuse to do so, and they would eventually take the initiative.

Rogers, a rigorous empiricist, reported on some early findings about the gains realized by students who participate in student-centered educational processes: greater personal adjustment, self-initiated extra-curricular learning, creativity and self-responsibility. He was also a radical reformer - or, at least, an advocate of radical rethinking - as can be seen in a 1952 essay on Personal Thoughts on Teaching and Learning, in which he espouses doing away with teaching, examinations, grades, credits, degrees and even the exposition of conclusions.

Others have written far more extensively - and eruditely - about Rogers' approach to student-centered learning, and I've recently encountered a number of other inspired and inspiring resources that are aligned with this approach, including Emily Hanford's provocative American RadioWorks program on Don't Lecture Me, Cathy Davidson's bold experiment with Crowdsourcing Grading, and Howard Rheingold's evocatively named Peeragogy Handbook Project. While I feel a strong sense of alignment with Rogers' principles (and those articulated by others), I don't have the gumption to fully embrace his radically unstructured approach to student-centered learning in my own teaching practice, however I will strive to iteratively incorporate as many of the principles of learner-centered education as I can.

I want to conclude this post with a few thoughts about the connections between Rogers' perspective and the principles of User-Centered Design, a paradigm which prioritizes users over technologies, and places human needs, wants, skills and experiences at the center of the design process. A definition of user-centered design at the Usability First web site highlights the parallels with Rogers' thinking:

User-Centered Design (UCD) is the process of designing a tool, such as a websiteâ€™s or applicationâ€™s user interface, from the perspective of how it will be understood and used by a human user. Rather than requiring users to adapt their attitudes and behaviors in order to learn and use a system, a system can be designed to support its intended usersâ€™ existing beliefs, attitudes, and behaviors as they relate to the tasks that the system is being designed to support.

Like Rogers, user-centered design emphasizes rigorous empiricism, offering a variety of methods for observing, measuring and evaluating the effects of different potential design elements on the user experience. UCD also involves iterative design and experimentation, very much in keeping with the serial nature of client-centered psychotherapy sessions and the punctuated equilibrium that I believe characterizes the process of unfolding revelations. UCD methods work best, I think, when its practitioners approach users with transparency, acceptance and deep empathic understanding. While there is a significant emphasis on evaluation and judgment in UCD, it is focused on the methods and their results rather than the human subjects - aka users - under study.

One important question in user-centered design is who the users are. Are we designing for ourselves or designing for others? Reflecting on Rogers' observation that the most personal is the most general, a third option might be proposed: designing for others through designing for ourselves. Much of the emphasis in UCD has been on designing for others, focusing on methods of observation, measurement and evaluation that help designers better understand [other] users' perspectives. However, some recent developments suggest a growing openness - on the part of some designers - to the idea of designing for others through designing for ourselves.

I captured portions of a debate on this topic - involving accusations of arrogance and justifications for self-centeredness - in an earlier post on irritation-based innovation, which I will simply summarize here with an observation made by one of the participants, Jason Fried, co-founder of 37signals, in response to a critique by human-centered design advocate, Don Norman:

Solutions to our own problems are solutions to other peopleâ€™s problems too.

Personally, I believe there is room for both perspectives. While I think that the most innovative designs arise out of the effort to solve one's own problems, the solutions can be made more useful and usable through a greater understanding of how they might be perceived and used by others to solve their own problems. So, while the most personal may be the most general, UCD practices can help pave the way for maximizing that generalization.

Posted by Joe McCarthy on January 19, 2012 at 06:46 AM in Books, Psychology | Permalink | Comments (0) | TrackBack (0)

Technorati Tags: carl rogers, design, education, hci, humanistic psychology, therapy

The Gaps, Crap and Gumption Traps in Creative Work

The poster above reflects hard-won wisdom acquired and shared by Ira Glass, host of PRI's This American Life, emphasizing the importance of perseverance in developing mastery of creative production. While Glass focuses on storytelling for radio and television, his insights and experiences about the gaps between ambitions and realizations - and the connections between quantity and quality - relate to wisdom I've encountered from masters of the crafts of filmmaking and maintaining motorcycles. I believe this wisdom applies to any creative endeavor, and I would argue that storytelling is an essential ingredient in every creative enterprise, as the creative things we produce and consume comprise an integral part of the stories we make up about ourselves.

The poster is derived from a video interview posted in August 2009 (Ira Glass on Storytelling, Part 3 of 4) in which he describes both the frustration and importance of making stuff that is still "kind of crappy" as an unavoidable part of the apprenticeship required for the journey to master craftspersonship ... and, according to Sturgeon's Law, 90% of everything is crap anyway.

[ http://www.youtube.com/embed/BI23U7U2aUY ]

Ira Glass is my favorite interviewer, and so I was intrigued when he was interviewed by another experienced interviewer, Kathryn Schulz, author of Being Wrong. The interview, which appeared in a June 2010 Slate article, On Air and On Error: This American Life's Ira Glass on Being Wrong, offers some glimpses of the wisdom captured in the pithy poster above:

One of the reasons I was interested in doing this interview is because I feel like being wrong is really important to doing decent work. To do any kind of creative work well, you have to run at stuff knowing that it's usually going to fail. You have to take that into account and you have to make peace with it. We spend a lot of money and time on stuff that goes nowhere. It's not unusual for us to go through 25 or 30 ideas and then go into production on eight or 10 and then kill everything but three or four. In my experience, most stuff that you start is mediocre for a really long time before it actually gets good. And you can't tell if it's going to be good until you're really late in the process. So the only thing you can do is have faith that if you do enough stuff, something will turn out great and really surprise you.

In a recent American Masters documentary on Woody Allen, the prolific writer, actor and director shared a similar perspective on the need to produce lots of stuff. Although the documentary is no longer viewable online, an interview with Robert B. Weide, the documentary filmmaker - a filmmaker filming a filmmaker - is available, in which Weide shares Allen's Quantity Theory:

You ask him [Woody Allen] about his endurance and his longevity over 40 years, and how prolific he is, doing a film a year for 40 years, as a writer and a director, and in many of them, an actor. And he says, "You know, longevity and endurance have their place, those are accomplishments of a sort, but those aren't the accomplishments I care about, which is to make a really great film." He says that he's working on the quantity theory, which is that if you just keep knocking them out, one picture after another, just keep making them and making them, some of them won't be that great, but every now and then, one will come out good.

Allen's Quantity Theory brings to mind the Metaphysics of Quality, and the idea of a gumption trap that Robert Pirsig described in his classic 1974 book, Zen and the Art of Motorcycle Maintenance. Pirsig uses motorcycle maintenance as a metaphor for life, and explores a variety of gumption traps - externally induced out-of-sequence reassembly, intermittent failure and parts problems as well as internally induced traps arising from value rigidity, ego, anxiety, boredom and impatience - and ways of addressing and overcoming them. I won't include the full text of Pirsig's hypothetical course in Gumptionology 101 here, but the following passage gives a sense of his perspective, and its relevance to the views shared more recently by Ira Glass and Woody Allen:

Throughout the process of fixing the machine things always come up, low-quality things, from a dusted knuckle to an accidentally ruined "irreplaceable" assembly. These drain off gumption, destroy enthusiasm and leave you so discouraged you want to forget the whole business. I call these things "gumption traps."

There are hundreds of different kinds of gumption traps, maybe thousands, maybe millions. I have no way of knowing how many I donâ€™t know. I know it seems as though Iâ€™ve stumbled into every kind of gumption trap imaginable. What keeps me from thinking Iâ€™ve hit them all is that with every job I discover more. Motorcycle maintenance gets frustrating. Angering. Infuriating. Thatâ€™s what makes it interesting.

Pirsig's ideas about gumption were part of the inspiration for this blog, and I have consciously and unconsciously encountered some of these traps when writing - and not writing - posts here. When I look back on my early posts, many of them now seem like crap ... and I don't think any of the posts I've written - or anything I've produced in any other realm - have ever quite closed the gap between my ambitions and my realizations. I suppose blogging gives me a channel through which to work out - or at least work with - the ongoing tension between striving and acceptance.

Finally, speaking of blogging, I first encountered the poster at the top of this post a few weeks ago at the top of a post at Tim Kastelle's blog (which I always enjoy) on How to Make Things Look Simple. Tim found it amid one of the longest chains of Tumblr reblogs I've ever encountered, but further searching suggests that it was originally created by Sawyer Hollenshead. In digging around for the source, I also found a plain text version of the Ira Glass quote on a blog maintained by NPR's Fresh Air associate producer Melody Kramer, which I'll include - and conclude with - here, as I find it more readable (though less striking) than the poster:

Nobody tells this to people who are beginners, I wish someone told me. All of us who do creative work, we get into it because we have good taste. But there is this gap. For the first couple years you make stuff, itâ€™s just not that good. Itâ€™s trying to be good, it has potential, but itâ€™s not. But your taste, the thing that got you into the game, is still killer. And your taste is why your work disappoints you. A lot of people never get past this phase, they quit. Most people I know who do interesting, creative work went through years of this. We know our work doesnâ€™t have this special thing that we want it to have. We all go through this. And if you are just starting out or you are still in this phase, you gotta know its normal and the most important thing you can do is do a lot of work. Put yourself on a deadline so that every week you will finish one story. It is only by going through a volume of work that you will close that gap, and your work will be as good as your ambitions. And I took longer to figure out how to do this than anyone Iâ€™ve ever met. Itâ€™s gonna take awhile. Itâ€™s normal to take awhile. Youâ€™ve just gotta fight your way through.

Posted by Joe McCarthy on December 23, 2011 at 09:08 AM in Art, Film, Heard on NPR, Media, Television, Work | Permalink | Comments (2) | TrackBack (0)

Technorati Tags: creativity, ira glass, narrative, perseverance, robert pirsig, storytelling, woody allen

I Am Because We Are: African Wisdom in Image and Proverb

We are all interconnected and we have responsibility for each other.

This is the interpretation of the Swahili word, ubuntu, offered near the start of a short, inspiring interview with photographer Betty Press by NPR Weekend Edition Sunday host, Audie Cornish two weeks ago. The interview focused on the incredible photographs celebrating the lives of people in Africa compiled over a 20-year period in a new book by Press, I Am Because We Are: African Wisdom in Image and Proverb. The NPR web page for the segment, A Photographer Changes The Focus In Africa, includes a selection of 7 of the 125 black and white photographs from the book. The images are striking, but given that this was a radio interview - on NPR, no less - I was a bit disappointed that, with the exception of the quote above, the proverbial dimension of African Wisdom was largely omitted from the segment ... and, well, talking about photographs is like writing about music ... which, of course, is like dancing about architecture.

Although I have never been there, I have been interested in Africa since inadvertently becoming an unofficial spokesperson for Nokia's efforts to empower people in developing regions with mobile technologies at PopTech 2007 (and several subsequent events). And as a technology guy, I have long interpreted ubuntu as a reference to a Debian-derived version of the open-source Linux operating system. I was intrigued, though not entirely surprised, to discover the origin of the term, which does seem well-aligned with the philosophy embodied by this evolving software artifact. So after learning more about the broader - and deeper - interpretation of ubuntu from Betty Press, my appetite was whetted for more examples of proverbial African wisdom to be revealed during the course of the interview.

Unfortunately, there were no further examples of proverbs offered on NPR - during the interview or on its associated web page - and while the book's web site offers a gallery that include additional photos, there are no examples of the proverbs in the book ... although the its proverbial aspects are highlighted in the following endorsement by Joanne Veal Gabbin on the main page:

A wise one said Proverbs are the palm wine with which words are eaten. Proverbs, like poems, are concise, loaded with metaphors, wisdom, nuance, and the rhythms of lifeâ€¦

At $39.95, this is not an inexpensive book (well, at least, not in my book), and I wasn't sure I wanted to make the investment. As much as I am moved by visual images, words are my primary source of inspiration. The book cover says "Proverbs compiled by Annetta Miller", and so I don't know if the division of labor is, in part, responsible for the primacy of images vs. words in nearly all the marketing materials (I cannot find a web page for Ms. Miller, but her bio suggests she has been involved in compiling other collections of African wisdom).

Having purchased and now received a copy of the book, I can attest to the captivating imagery contained in the photographs. Many of the proverbs of the book reflect wisdom that I've encountered in proverbs arising in American, European and/or Asian cultures - perhaps reflecting the universal nature of many of the most meaningful insights and experiences we share as human beings - but a few stood out as particularly poignant pronouncements of perspicacity. I wanted to help compensate for what I see as a deficit of attention to the proverbial wisdom in the book by sharing a few of my favorites:

The world is a mirror; it looks at you the same way you look at it. [North African proverb]

Our children are living messages sent to a future we may never see. [Nigerian proverb]

What you help a child to love is more important than what you help her to learn. [Sengalese proverb]

If you educate a man you educate an invdividual, but if you educate a woman you educate a family (nation). [Ghanaian proverb]

If you can talk, you can sing; if you can walk, you can dance. [Zimbabwean proverb]

These last two are especially resonant, after having recently attended a David Whyte poetry reading, in which he and representatives of a local organization, Young Women Empowered, shared some proverbial wisdom about the importance of empowering [young] women and for the need for all humans to courageously speak out in the world. Whyte also spoke of embracing different forms of beauty, and the images and words in I Am Because We Are are a powerful illustration of beautiful forms that arise in the people and places of Africa.

Posted by Joe McCarthy on December 10, 2011 at 07:00 PM in Africa, Art, Books, Heard on NPR | Permalink | Comments (0) | TrackBack (0)

Technorati Tags: africa, books, photography, proverbs, wisdom

David Whyte on Feminine Wisdom, Courage and Power

David Whyte shared his inspired and inspiring wisdom about the feminine embodiments of power last night at Town Hall Seattle. At a benefit event for Young Women Empowered (Y-WE) - an organization co-founded by his wife, Leslie - he guided the audience on a journey exploring the "five forms of female courage" and revealed aspects of that courage through stories, poetry and an articulation of what he calls a philosophy of attention. Whyte suggested that these forms of courage are not restricted to women, but that men typically only arrive at these forms of courage - and wisdom - after they have tried all the more masculine forms of courage. I have often wrestled with an interior tension between the masculine and the feminine - most of my closest friends are women, and most of the artists who inspire me are women - and so Whyte's framing of the masculine vs. the feminine forms offers me a new perspective from which to contemplate this tension.

Whyte described the first form as the feminine relationship to the unknown or to mystery, perhaps best exemplified by a woman's central role in the miracle of birth ... and a man's role as an outsider looking in. After distinguishing a father's relationship with a son, whom he is supposed to teach, and his relationship with a daughter, to whom he is supposed to apprentice himself, Whyte recited My Daughter Asleep, which he composed over a course of several years for his daughter, Charlotte, beginning shortly after her birth. When she was five, he recited the poem for her, and asked what her favorite part was. Charlotte's favorite section is also my favorite ... and I wonder if my own daughter (who I suspect is very close to Charlotte's age) would also find a deep resonance with the lines:

May she find
in all this
day or night
the beautiful
centrality
of pure opposites

As a marine biologist who worked as a naturalist in the Galapagos, and who has always felt and expressed a keen appreciation of the natural world, Whyte introduced another poem by explaining that a collection of larks is called an exaltation of larks, and asserting that he considers the lark an emblem of humans' ability to speak out in the world. He then recited Song of the Lark, another poem that reveals - or perhaps more precisely, revels in - the mystery of the feminine, which was inspired by [a postcard of] a painting by Jules Breton from 1884:

What is called in her rises from the ground
and is found in her body,
what she is given is secret even from her.

The second form of feminine courage explored during the evening was a willingness to ground that mystery in the world. Whyte sees vulnerability as a source of strength rather than weakness, reflecting wisdom I have encountered in words written and spoken by empowered and empowering women such as Oriah Mountain Dreamer (through whom I first discovered David Whyte) and Brene Brown, who also advocate - and model - connection and compassion through courage, vulnerability & authenticity.

Whyte recited poetry exemplifying strength through vulnerability, including a poem by the courageous Russian poet Marina Tsvetaeva, I Know the Truth, remarking how much more authentic such a bold assertion seems when it comes from a woman vs. a man. He also recited his own poems The House of Belonging and Start Close In, the latter of which suggests a step-wise approach to grounding the mystery in the world:

Start close in,
don't take the second step
or the third,
start with the first
thing
close in,
the step
you don't want to take.

The third form of feminine courage is a willingness to say "no" to anything and everything that is not a full "yes". This courage is illustrated by Meera (aka Mira / Meerabai / Mirabai), an ecstatic Hindu poet and singer in 16th century India, and her poem, Why Meera Can't Go Back Home, which articulates firm boundaries:

Approve me or disapprove me;
I praise the Mountain Energy night
and day.

Whyte also recited Sweet Darkness, a poem with particularly deep personal resonance that helped me re-frame and re-interpret a misconstrued defeat during a period in which I was moving toward belonging, freedom and coming alive again after a period of soul-squelching darkness:

Sometimes it takes darkness and the sweet
confinement of your aloneness
to learn

anything or anyone
that does not bring you alive

is too small for you.

After an intermission, Jamie Rose-Edwards, the Executive Director of Young Women Empowered, and two of its recent graduates shared their hopes for and experiences with the program. The mission of Y-WE is to "empower young women from diverse communities to step up as leaders in their schools, communities, and the world." There was clear alignment between the kinds of courage and power Whyte was expressing and the characteristics modeled and cultivated by the staff and mentors of the program.

When Whyte returned to the stage, he articulated the fourth form of feminine courage, a willingness to live in different forms of beauty. The forms of beauty that a woman will exhibit and experience will change throughout her life, and the transitions will often involve disappointment and heartbreak, but the courage to work through the transitions open doors to new forms of beauty (and wisdom). Whyte recited Shakespeare's Sonnet 29, his own poem to (and, in a way, from) his mother, Farewell Letter, and a poem, Weathering, by New Zealand poet (and librarian), Fleur Adcock, composed during her year-long, late mid-life sabbatical in the English Lake District (one of the places through which Whyte leads tours):

But now that I am in love
with a place that doesnâ€™t care
how I look and if I am happy,
happy is how I look and thatâ€™s all.

At the end of the evening, Whyte shared the fifth form of feminine courage, the courage to accept the invitations of life and step out beyond yourself, noting that the most courageous conversation is the one we don't want to have. Throughout much of his prose and poetry, Whyte advocates adopting an investigative vulnerability in exploring the frontiers of experience, and his poem, The True Love - with which he concluded the event - is one of this most penetrating articulations of this truth, drawing upon the biblical account of Jesus inviting Peter to find the faith and courage get out of a boat amid stormy seas and walk on water toward him. A passage in the poem is especially poignant for me now, as I wrestle with my own vulnerable sense of power, worthiness and faith:

I am thinking of faith now
and the testaments of loneliness
and what we feel we are
worthy of in this world.

Most of the poems Whyte read - both his own and those by other poets - were from his collection, River Flow, a book I have read cover-to-cover dozens of times, and probably represents the closest approximation I have to a bible, serving as a personal perpetual source of wisdom and inspiration. I've also listened to his entire 6 CD set, Clear Mind, Wild Heart [also available in MP3] dozens of times, a series of recordings with a format similar to the event of last night: an intermingling of poetry, prose and philosophy. Although I was already familiar with most of the poems (and the stories behind them) shared last night, it was still a thrilling experience to see him live. This was my first poetry reading, and throughout the event, I experienced the same kinds of "goose-bump moments" I've often felt (and written about) during music concerts, perhaps most notably during the Indigo Girls Zoo Tunes concert that Amy and I attended a few years ago. What was qualitatively different between the poetry reading last night and music concerts I have attended was the frequent, collective sighs and other spontaneous shared expressions of recognition of deep truths that were articulated during Whyte's recitation of poems. I often experienced a double jolt of goose bumps, first from my own personal resonance, and shortly thereafter from the awareness that I was not alone in feeling that deep resonance.

I'll finish off this post with a passage from a poem that Whyte did not read last night, Revelation Must be Terrible, but which exquisitely captures the notions of shared resonance and aloneness:

Being far from home is hard, but you know,
at least we are exiled together.

Namaste.

Posted by Joe McCarthy on December 03, 2011 at 01:36 PM in Events, Poetry, Spirituality | Permalink | Comments (2) | TrackBack (0)

Technorati Tags: david whyte, empowerment, poetry

A Compelling, Compassionate, Critique of Conservative Extremism by David Frum

NewYorkMagazine_20111128_politicscvr_150

the [Republican] party is getting the big questions disastrously wrong

[David Frum on the GOPâ€™s Lost Sense of Reality (New York Magazine, 20 November 2011)]

David Frum, former economics speechwriter for former U.S. President George W. Bush, offers a sharp critique of the Republican Party in an interview with NPR's Steve Innskeep yesterday, David Frum asks "When did the GOP lose touch?". The interview was prompted by Frum's recent article in the current issue of New York Magazine, which is impressive in its breadth and depth ... and, I would argue, its compassion.

I often feel incensed at some of the things I hear and read conservatives saying and writing. Frum's article helps provide some context for some of the perspectives presumably felt and sometimes articulated by some conservatives, but does so largely without being condescending. I'm reminded of one of the central tenets of non-violent communication: communication designed to induce fear, shame and/or guilt in a listener often arises from conscious or unconscious fear, shame and/or guilt on the part of the speaker. I'm also uncomfortably reminded of my own tendencies toward projection and rejection ... which are [also] reflected in the subtitle of Frum's article:

Some of my Republican friends ask if Iâ€™ve gone crazy. I say: Look in the mirror.

I highly recommend reading the entire article, and its complementary companion article by Jonathan Chait, who until recently was senior editor at The New Republic, How Did Liberals Become So Unreasonable (although frankly, I did not find Chait's article as compelling ... or compassionate). Here, I wanted share a few excerpts highlighting some key observations Frum makes regarding the rightward GOP shift(s).

On Fiscal Austerity and Economic Stagnation:

... the big winners in the American fiscal system are the rich, the old, the rural, and veteransâ€”typically conservative constituencies. ... Any serious move to balance the budget, or even just reduce the deficit a little, must inevitably cut programs conservative voters do like: Medicare for current beneficiaries, farm subsidies, veteransâ€™ benefits, and big tax loopholes like the mortgage-interest deduction and employer-provided health benefits. The rank and file of the GOP are therefore caught between their interests and their ideologyâ€”intensifying their suspicion that shadowy Washington elites are playing dirty tricks upon them.

On Ethnic Competition:

[In a National Journal article based on a Gallup poll of Republican voters, Second Verse, Same as the First, Ron Brownstein reports that] "... noncollege whites are the gloomiest: Just one-third of them think their kids will live better than they do; an equal number think their children wonâ€™t even match their living standard. No other group is nearly that negative." Those fears are not irrational. ... It is precisely these disaffected whitesâ€”especially those who didnâ€™t go to collegeâ€”who form the Republican voting base.

On Fox News and Talk Radio:

Extremism and conflict make for bad politics but great TV. Over the past two decades, conservatism has evolved from a political philosophy into a market segment. An industry has grown up to serve that segmentâ€”and its stars have become the true thought leaders of the conservative world. The business model of the conservative media is built on two elements: provoking the audience into a fever of indignation (to keep them watching) and fomenting mistrust of all other information sources (so that they never change the channel).

On [what I would call] unenlightened self-interest:

We used to say â€œYouâ€™re entitled to your own opinion, but not to your own facts.â€ Now we are all entitled to our own facts, and conservative media use this right to immerse their audience in a total environment of pseudo-facts and pretend information. ... [sinister GOP] billionaires do exist, and some do indeed attempt to influence the political process. ... Yet, for the most part, these Republican billionaires are not acting cynically. They watch Fox News too, and theyâ€™re gripped by the same apocalyptic fears as the Republican base. In funding the tea-party movement, they are Âactually acting against their own longer-term interests, for it is the richest who have the most interest in political stability, which depends upon broad societal agreement that the existing distribution of rewards is fair and reasonable. If the social order comes to seem unjust to large numbers of people, what happens next will make Occupy Wall Street look like a street fair.

"a going-out-of-business sale for the baby-boom generation":

Some call this the closing of the conservative mind. Alas, the conservative mind has proved itself only too open, these past years, to all manner of intellectual pollen. Call it instead the drying up of conservative creativity. ... In the aftershock of 2008, large numbers of Americans feel exploited and abused. Rather than workable solutions, my party is offering low taxes for the currently rich and high spending for the currently old, to be followed by who-knows-what and who-the-hell-cares. This isnâ€™t conservatism; itâ€™s a going-out-of-business sale for the baby-boom generation.

Frum finishes off the article with a call for conservative moderates to speak up:

I refuse to believe that I am the only Republican who feels this way. If CNNâ€™s most recent polling is correct, only half of us sympathize with the tea party. However, moderate-minded people dislike conflictâ€”and thus tend to lose to people who relish conflict. The most extreme voices in the GOP now denounce everybody else as Republicans in Name Only. But who elected them as the GOPâ€™s membership committee?

During this period of increasing protests against inequality and injustice - on Wall Street and other streets in America, as well as on streets and squares in Egypt and elsewhere around the world - I'm reminded of earlier widespread protests against the Vietnam War ... and former Republican President Richard Nixon's claims during that period to be the spokesperson for what he called the silent majority, and his largely successful efforts to divide and polarize the American people ... and claims made by the more recently self-appointed Republican spokespeople of real Americans.

However, harking back to that earlier period of protest also reminds me of the wisdom of an inspiring liberal who, like Frum, [also] called for moderation in words and actions in the cause of promoting change: Dr. Martin Luther King, Jr:

History will have to record that the greatest tragedy of this period of social transition was not the strident clamor of the bad people, but the appalling silence of the good people.

Although Frum and King espouse different perspectives on the types of changes that are likely to lead to a greater Good, a vigorous, non-violent debate seems like the most likely course to lead toward improvements in politics and society.

Posted by Joe McCarthy on November 25, 2011 at 11:25 AM in Current Affairs, Heard on NPR, Non-violent Communication (NVC), Politics | Permalink | Comments (2) | TrackBack (0)

Technorati Tags: conservatives, David Frum, GOP, Republican Party, tea party

Usability and confusability in Health IT: doctor-computer interaction vs. doctor-human interaction

A segment on the Marketplace Tech Report, Health care providers having trouble with new technology, caught my ear yesterday. The story included health and safety concerns raised by one of the authors of a 197-page report, Health IT and Patient Safety: Building Safer Systems for Better Care, published by the Institute of Medicine this week:

Dr. Ashish Jha, a professor at Harvard's School of Public Health and an author of this report as well says, "One of things that happened in Pittsburgh, in the pediatric ICU was that when the electronic system was put in, it really changed the way doctors and nurses interacted and the way they worked together. Physicians started spending a lot less time at the bedside and they were spending a lot more time staring at the computer screen, and interacting less with nurses and interacting less with patients. And there's a lot of information you pick up when you speak directly with people that when you go to purely electronic communication, you miss."

Two diametrically opposed possible explanations for the increased inattention of doctors to their human colleagues and customers (aka patients) came to mind:

the health information technology (HealthIT) systems are so well designed that doctors are becoming engrossed in the wealth of information newly available to them the new HealthIT systems are so poorly designed that doctors are being needlessly distracted by confusing and unintuitive interfaces that require significant attention to navigate effectively.

After downloading and skimming the report [a pre-publication version of which is available as a free PDF], it appears that the latter explanation is most on point. Chapter 4 of the report, Opportunities to Build a Safer System for Health IT, includes observations, analysis and recommendations for usability, workflow and other human-computer interaction (HCI) issues involved in the design of effective Health IT systems. The report includes chapters on other topics involving information and technology (many of which incorporate elements of HCI), but it is encouraging to see such a strong human-centered focus on what is, by definition, a very human-centered domain.

Among the observations shared by the committee who authored the report is a strident call for usability as a mission-critical factor:

The committee expressed concerns that poor [Health IT] usability ... is one of the single greatest threats to patient safety. On the other hand, once improved, it can be an effective promoter of patient safety. [emphasis added]

Among the relevant references they recommend for improving usability:

NIST Guide to the Processes Approach for Improving the Usability of Electronic Health Records (29 November 2010) Customized Common Industry Format Template for Electronic Health Record Usability Testing (16 November 2010) National Center for Cognitive Informatics and Decision Making in Healthcare (NCCD) Rapid Usability Assessment NCCD's unified framework incorporating task, user, representation & function (TURF) for defining, designing and assessing usability for Electronic Healthcare Records (EHR)

Unfortunately, I don't currently have the time to dig more deeply into the report or the wealth of references it cites (I have 75 midterm Operating Systems exams to grade in the next 36 hours). However, in my [re]new[ed] role as educator, I will be teaching a class on HCI next quarter and am pondering how I might indulge my increasing interest in computer supported cooperative healthcare and find ways of focusing on healthcare issues as a stimulating and worthwhile problem domain for undergraduate students learning about HCI.

Having recently read a compelling NYTimes article on Why Science Majors Change Their Minds (It's Just So Darn Hard) - which emphasized the increasing importance of design, projects, problem solving and social service to motivate students studying Science, Technology, Engineering and Math (STEM) - I'm eager to put some of these ideas into practice.

[Update: The New York Times has a related article on doctor-computer interaction, which focuses on the distraction caused by highly engaging IT devices and services that are not designed specifically for healthcare: As Doctors Use More Devices, Potential for Distraction Grows (found via a post on Distraction at Josephine Ensign's "Medical Margins" Blog).]

Posted by Joe McCarthy on November 11, 2011 at 07:15 AM in Health, Heard on NPR, Web/Tech | Permalink | Comments (0) | TrackBack (0)

Technorati Tags: hci, health it, healthcare, medicine, stem, usability

Hadoop, Apache and the Benefits of Contributing to Open Source Projects

Jake Homan, a Senior Software Engineer at LinkedIn and UW Bothell CSS graduate, gave a recent guest lecture at UWB on Apache Hadoop: Petabytes and Terawatts, offering an overview and applications of Hadoop as well as related distributed computing tools developed within the Apache Software Foundation. The presentation offered a great balance of breadth and depth that was very well suited to the audience, primarily composed of senior undergraduate and Master's-level computer science students (and a few faculty). One of the most valuable insights shared by Jake was the enormous value that contributing to open source software projects can offer CS students - and other interested in software engineering career opportunities - to develop and demonstrate both their technical skills and their ability to work and play well with others.

Jake explained that Hadoop has two primary components: a distributed file system and a framework to support distributed computation. The Hadoop Distributed File System (HDFS) divides files into 128 MB blocks, makes 2 copies - yielding 3 replicas - of all the blocks, and then distributes the blocks on different DataNodes (computers). A NameNode manages the DataNodes and, among other tasks, regenerates the file blocks stored on a DataNode when that DataNode dies - and given enough DataNodes and enough time, a DataNode is sure to die - to ensure that 3 replicas of every file block are always available.

Hadoop provides a Java implementation of the MapReduce framework to support distributed computation. Using the prototypical example of a word count program - which Jake described as the "hello, world" program for distributed computing - he showed how to break down a computation into a Mapper and a Reducer. Generally speaking, a Mapper takes a <key, value> pair and generates zero or more <key, value> pairs; a Reducer takes all the values of one key and generates zero or more <key, value> pairs.

Applying this framework to the problem of counting words in a text (or collection of texts), a Hadoop program might start by splitting the text into lines or sentences where the keys represent the sequence positions of lines or sentences and the values represent the segments of text, e.g.,

<0, "Four score and seven years ago ...">
...

Hadoop would distribute these <key, value> pairs acrross DataNodes, where a TaskTracker on each DataNode would use a Mapper to split its line or sentence into a sequence of words and counts (where all counts are initially 1), yielding

<"Four", 1>
<"score", 1>
<"and", 1>
<"seven", 1>
...

During the Reduce phase, the outputs of Mappers are aggregated and sorted by key, yielding <key, list-of-values> pairs:

<"a", [1, 1, 1, 1, 1, 1, 1]>
<"above", [1]>
<"add", [1]>
...

These are then reduced [again] to <key, value> pairs, yielding the final sequence of word and frequency counts:

<"a", 7>
<"above", 1>
<"add", 1>
...

Distributed systems are increasingly the norm rather than the exception in companies providing any kind of web services - or involving any other kind of non-trivial computation - and so knowledge and experience in working with distributed systems is an increasingly important component of computer science education. However, even with knowledge of distributed systems, writing programs that can take advantage of distributed system architecture is still difficult and error-prone.

Jake said that if programmers can learn to think in terms of MapReduce, they can use Hadoop to manage many of the logistical and coordination aspects of distributed system programming; if programmers want to think or work with relational databases (SQL), they can use Hive; and if they want to think or work with higher level scripting languages, they can use Pig. Both of these are among the many Apache tools that can be layered on top of Hadoop. [I wrote about several of these tools in a post last August on Hadoop Day in Seattle: Hadoop, Cascading, Hive and Pig.]

One of the most useful pieces of knowledge that Jake shared during his presentation concerned the often underappreciated second-order benefits of contributing to open source projects, i.e., above and beyond the intrinsic value of improving software tools which, in many cases, programmers are using themselves. The first question he asks a software engineer candidate is "Have you done open source?" Open source software projects typically make all the code and the online conversations about the code publicly available, so Jake can do some background investigation to learn about both the open source code the candidate has written and the way the candidate has interacted with other contributors and stakeholders (e.g., the way a candidate has responded to bug reports or feature requests). The candidacy of any software engineer who has not contributed to any open source software projects may be considerably diminished by a deficit in this area.

Getting involved in an open source project can be intimidating, so Jake shared a link to the Apache Software Foundation list of ASF newbie issues that would be appropriately scoped projects for someone who wants to test the waters. I have not contributed directly to any Apache project - yet - but I did engage in some civic hacktivism at Data Camp Seattle in February, and some random hacks of kindness at RHOK 3 in June. I would like to organize an appropriately and inspiringly themed open source hackathon at UWB for students, faculty and other interested parties sometime in the near future ... but it will have to wait until after the fall quarter, as the three classes I'm teaching now are consuming nearly all time and energy. I'm glad I at least took an hour off last week for Jake's engaging and educational presentation.