All in the <head>

– Ponderings & code by Drew McLellan –

– Live from The Internets since 2003 –

About

How A Missing Favicon Broke My App for Chrome Users

3 days ago

Excuse me while I let off some steam. I’ve just spent many hours debugging an authentication issue that was preventing Google Chrome users from logging into a PHP web app we’re currently working on. Here are the gory details for your amusement.

The app uses OAuth to authenticate against the client’s central auth service. That means instead of logging in to our app directly, the process is more like signing into Twitter from a third party – we send the user away to log in, they authenticate on the central server as normal and get sent back to us with a token which we can then use to log them in to our app.

The issue we were seeing is that Chrome users were clicking the login link, being sent out to authenticate, coming back and still being told they weren’t logged in. In all other browsers, returning from a successful login allowed access the app as expected. Puzzling.

As I went through debugging, it became clear that the point at which the process was failing was when the CSRF tokens were being compared. When we create the log in link, we add a CSRF token to the URL and at the same time store the same token in the user’s PHP session. On returning from authenticating, the token is handed back to us and we can compare it to the copy in the session storage to make sure there’s no funny business going on.

Poking further, it became clear that the reason the CSRF tokens didn’t match was due to the user being given a new PHP session when they returned. So when we compared the token, there was no stored value to compare it to. Very odd indeed.

Usually, if sessions go awry, the best thing to do is turn to the raw HTTP headers. A PHP session is of course based on a cookie – the server issues a session ID in a cookie and the browser presents it back to the server with each request. If the session ID was changing, that had to mean that either the browser wasn’t sending the cookie, or the server was issuing a new one. Either way, that would show up in the request and response headers.

Using the Network tab of the Chrome web inspector, I could see the headers for each file. I could see the cookie being set with the page response, and then presented back with the stylesheet request. Clicking out to log in and coming back to my page, I could see the cookie being presented again… but wait! With a different session ID. When did that change?

Double checking the headers again, I could see that no new cookie was being sent in the response to any of the items in the Network inspector. But there are requests that happen that the network inspector doesn’t show you. Like requesting a favicon.

As this app is in development, it doesn’t have a favicon, so the request should result in a 404. Except in this case, it didn’t. The server is configured (and to be fair, misconfigured) to route any requests that are not for real files or directories to my index.php page. This is so the app can have /any/url/it/pleases/ and it just gets routed through to my app to parse internally. A front controller.

This meant that instead of getting a 404 from Apache, the favicon request was being handed back the app’s home page. Niiice. But that shouldn’t be an issue, as even a request for a favicon is sent along with the cookie information, so PHP should pick up the session and carry on as normal. But the plot thickens. Apache is sitting behind Varnish, which I’d configured to strip cookies from any request for static files such as images, CSS, JavaScript, and, you know, favicons.

That meant that by the time the request got passed through to Apache, the cookie had been stripped and the request looked like a new user. Who was then given a brand new PHP session. As soon as I put a favicon in place, bingo, Chrome users could log in.

I’m not sure if Chrome requests the favicon at the beginning of the request, or right at the end, but either way, it was enough to change the user’s session cookie between the end of one page load and the beginning of the next.

A perfect storm of a bit of misconfiguration on my part, a lack of transparency from Chrome’s network inspector, and a missing favicon.

- Drew McLellan

The Best Forms Implementation I've Ever Built

182 days ago

The one thing that will really kick your developers’ butts when building an interaction-heavy web app or site is the forms. Forms can be a lot of work to implement. Get the technical design of your form generation/handling/validation system right and your project can fly along. Get it wrong, and you’re sunk in work that becomes tedious and demotivates everyone involved.

Now, before you tell me that this is a non-issue because no one builds individual forms anymore and that they’re all auto-generated by frameworks, I’m talking about the bit that goes on inside the framework. My consideration is how you design a system that outputs forms without resulting in a sucky literal representation of a database row in HTML.

Anyone who cares about interaction design and gathering accurate data carefully designs and tunes forms individually to suit the task in hand. Auto-generated data entry forms are fine for routine back-office jam-this-data-in-a-database-table tasks, but they suck for anything that matters.

So the problem becomes one of how to feed the form engine with the data it needs whilst still giving designers control of the markup and user experience.

Attributes of a form handling system

There are a few core things that a form handling system needs to do.

Firstly, it needs to generate the HTML for the form itself, including all input fields, labels, default values, surrounding help and tips and error messages.

Secondly, it needs to be able to detect that the form has been submitted, and validate the data for required fields, check the format of any data, and verify any co-dependancies between fields (such as two password fields matching). If the validation fails, errors need to be set and the fields all need to be repopulated with the data that was just submitted.

Lastly, when the form passes validation checks, it needs to collect all the data up and pass it along the line for the next step of the application to deal with.

The processing of this generally falls into two places – MVC-types would say in the Controller and the View – but can simply be thought of as before any browser output and down in the page.

What happens down in the page could be dismissed as basic template conditionals, but there’s such a level of complexity with repopulated vs. default values, messages, errors and such that I really consider it to be worthy of more detailed consideration than just basic templating. If the goal is to reduce work, then it needs more thought.

Bad designs are easy

Due to this split nature of forms, the usual design for a system of this nature is declare all the fields in code, and then have some templating system handle the output down in the page.

A system I used for a while was PHP’s HTML_QuickForm (now QuickForm2) which defines fields like this (from their hello-world tutorial):

$fieldset = $form->addElement('fieldset')->setLabel('QuickForm2 tutorial example');
$name = $fieldset->addElement('text', 'name', array('size' => 50, 'maxlength' => 255))
                 ->setLabel('Enter your name:');
$fieldset->addElement('submit', null, array('value' => 'Send!'));

The shape of the form is defined in code and then piped into a standard template for output. If you want to customise the HTML output, you can code up your own custom renderer. I’m an experienced PHP developer, and that makes my toes curl.

Designers need control

In my ideal world, a designer should be able to put a form together on the fly, working in HTML as much as is possible. When every form could have a small area of uniqueness, custom renderers or overriding templates isn’t the way to go – let the app deal with the input tags, but the designer needs direct control over the HTML.

That’s the principal I’ve stuck with for the last five or six years (since deciding QuickForm wasn’t the way to go) and have designed my form systems around it. I say systems, but really it’s just one which has evolved and finds use in both the edgeofmyseat.com web app and CMS frameworks, and ultimately in the control panel of Perch.

That system took me part way there, with the layout of a form being generated directly in the HTML, but with validation and processing rules being declared up in the PHP code before output. So it was a good step forward, but still required form declaration be split across two places.

Adding forms to Perch

When it came to designing a way for Perch users to add forms to their websites, I knew I’d need to do better still. One of our design principals is that we try not to abstract the designer away from the page. If you want to add something to a page during site build, you go into the page and add a region.

We also wanted designers to be able to throw in a form into pretty much any situation without needing to think too much about the technical implications. If you’re listing out products, you should be able to throw in an “add to cart” form, or a booking form for an event. That sort of thing. Wherever you’re outputting content, you should also be able to output a form.

I quickly came to the conclusion that our forms would have to be completely declarative. Rather than specifying a form in code and have a template turn that into HTML, we’d let designers create the form in as-close-to-HTML as we could and let the code figure the rest out.

Enter HTML5

One of the best things about HTML5 for me is the improvements that have been made to forms. You can now specify a field as, e.g.

<input id="email" name="email" type="email" required="true" />

and a supporting browser will prevent the form being posted until the field is completed and contains a valid email address. No fuss, no tangle of ugly server-side code, just simple, easy to use declarations. I thought this was the perfect model for making forms simple, so I copied it. I wrote a complete server-side implementation of HTML5 forms.

Inevitably, we had to add a bit of magic around any forms, and it was never going to be a case of just using straight HTML, but I tried to make things as natural-feeling as possible.

<perch:label for="email">Email</perch:label>
<perch:input id="email" type="email" required="true" />

Outputs:

<label for="form1-email">Email</label>
<input id="form1-email" name="email" type="email" required="true" />

Enhanced with the magic needed to pre-fill and re-fill field values, automatically ensure that IDs are unique in the page and so on. If you want to generate an error message:

<perch:error for="email" type="required">
        Please enter your email address.
</perch:error>

Change type="required" to type="format" and specify an error if the format isn’t correct. These can include any markup you need, and go anywhere in the form – near the field or at the top, whichever is your preference. And when the form is successfully submitted,

<perch:success>Thank you for filling out the form!</perch:success>

specifies the response. (Of course, as this is all in content templates, any of the text or even form attributes can be content managed.)

Making it work

It’s one thing to have forms, but you need to be able to process them with something. Perch has a system of ‘apps’ (add-on functionality, e.g. Blog, Events, Gallery etc) which now all have the opportunity to make use of forms.

When specifying a form, the designer adds an app attribute:

<perch:form id="add_to_cart" app="perch_products">

When a form is submitted for a named app, the app is notified and handed a prepared, validated set of fields and files that have been uploaded, along with the ID and a copy of the form template all loaded up and ready for further inspection if required. Creating a app using forms is about as trivial as it gets, which is great.

So what’s your point?

My point is this. I’ve built a lot of different form handling systems over the years, in a few different languages, and they’re hard to get right. If experience has taught me anything, it’s that a design that doesn’t put the web designer in control of the output is going to end up being a burden to your project.

This design shifts form configuration into the template and I think it really works well. Writing forms is fast and simple because by the time you’ve built your template you’ve defined the form. I really do think this is the best form implementation I’ve ever built, and so I thought it would be useful to share.

- Drew McLellan

The Lure of On-page Editing

326 days ago

I get asked quite often why Perch doesn’t offer any sort of on-page editing. It’s something we’ve given a lot of consideration to, and as it’s an interesting software design issue I thought I would document some of my reasoning here.

So what do I mean by on-page editing? In the context of a content management system, we’re talking about the ability to make edits to a page directly by clicking on the content (or a nearby Edit link) and manipulating it in place on the front end of a website. The alternative approach in contrast to this — and the approach we currently take with Perch — is for the user to go to a dedicated control panel to edit the content away from the page.

There are a couple of issues with on-page editing in my mind.

The technical

The big technical hurdle with on-page editing is the necessity for the CMS to inject its UI into the front-end page. At first that doesn’t sound too bad, but the more you think about it, the bigger a challenge that is. The pages own CSS and JavaScript are all going to influence the editing UI in unpredictable ways. The CSS you might think can just be dealt with by using very specific selectors and some sort of local CSS-reset, and you might be right, but what about z-index, overlapping absolutely positioned elements, dealing with Flash? JavaScript is another issue – if the editing UI uses JavaScript, will it conflict with anything on the page? Will the page’s own JavaScript conflict with it? What if the page attaches event listeners to the editing UI by mistake? What happens if two versions of the same library get loaded? What about sIFR?

These are all thorny technical issues that require effort to work around. Are they insurmountable? No – they can all be addressed. However, it becomes a Red Queen’s race to address issues as they arise, requiring a considerable amount of development time (not to mention the associated support) in order to maintain the status quo. If you have limited development resources, taking on that sort of burden could really inhibit your ability to move the product forward. As a developer I want to spend my time making a product better, not keeping it the same.

The touchy-feely stuff

More important than the technical considerations (which can all be dealt with if you choose to take them on) are the usability considerations. Will this be good for my clients editing their site? On-page editing certainly gives good demo, but does it go further than that?

Obviously, the immediate usability issue that has to be overcome is how clients edit content that’s not always visible on the page. Things like modal overlays and JavaScript carousels all requires a lot of thought, especially when you have no idea of the sort of sites where the CMS will be implemented.

That’s not my main concern with on-page editing, however. My main concern is this. One of the primary benefits of a CMS is being able to reuse bits of content around your site whilst maintaining it in one central place. A client can create, for example, a listing of news items on a News page, and also have the latest item or headline display on the home page. This is where on-page editing falls down.

On-page editing creates a model whereby the client is told they are directly manipulating the content on the page. It tells them they are taking a ‘physical’ page and remoulding it to their requirements before setting it down. That’s a really powerful metaphor, and I can clearly see why clients would like it.

However, as soon as you attempt any content reuse across the site, the metaphor is broken. The physical model becomes metaphysical, as a change to content in one place and affect content in another. This leads to unpredictability in the interface, which is the very worst thing for less technically-confident clients. Unpredictability leads to a distrust of the software, which leads to it just not getting used.

The control panel editing experience creates an abstraction from the page quite deliberately. That abstraction is needed in order to be able to make use of the technology in the way that best serves the client. I don’t believe the direct-manipulation metaphor (i.e. on-page editing) can be maintained whilst still offering the labour-saving benefits of managed content.

It’s about giving users confidence

Much of what we do with Perch is designed to give those less technically-confident clients confidence in editing their sites. We don’t want to see designers handing off a CMS to their clients only to be getting a phone call every time some content needs changing. The abstraction of using a control panel for editing enables content reuse around the site – that means the client doesn’t need to remember a long task checklist each time they make a change.

Features like preview, drafts and undo are all designed to give clients the confidence that they can’t make a mess of the site, or if they do, they can back out of it.

On-page editing is a technical and a design challenge, but we’re not afraid of those. There are plenty of instances where we’ve taken on difficult problems because it will make the user experience much better. It’s more that I believe that direct-manipulation is a flawed approach because it is in direct conflict with the goal of using software to reduce tedious, repetitive tasks.

If we can find a way in the future of implementing on-page editing without those downsides, we’ll certainly give it some attention. I want the editing experience with Perch to be the best it can be for clients. At the moment, although on-page editing looks great in demos, I don’t think that implementing it is at all in the client’s interests. In fact, I believe it’s against their interests.

- Drew McLellan

Photographs

Work With Me

edgeofmyseat.com logo

At edgeofmyseat.com we build custom content management systems, ecommerce solutions and develop web apps.

Follow me

Recent Links

Affiliation

Web Standards Project Britpack 24 ways

I made

Perch - a really little cms

About Drew McLellan

Photo of Drew McLellan

Drew McLellan (@drewm) has been hacking on the web since around 1996 following an unfortunate incident with a margarine tub. Since then he’s spread himself between both front- and back-end development projects, and now is Director and Senior Web Developer at edgeofmyseat.com in Maidenhead, UK (GEO: 51.5217, -0.7177). Prior to this, Drew was a Web Developer for Yahoo!, and before that primarily worked as a technical lead within design and branding agencies for clients such as Nissan, Goodyear Dunlop, Siemens/Bosch, Cadburys, ICI Dulux and Virgin.net. Somewhere along the way, Drew managed to get himself embroiled with Dreamweaver and was made an early Macromedia Evangelist for that product. This lead to book deals, public appearances, fame, glory, and his eventual downfall.

Picking himself up again, Drew is now a strong advocate for best practises, and stood as Group Lead for The Web Standards Project 2006-08. He has had articles published by A List Apart, Adobe, and O’Reilly Media’s XML.com, mostly due to mistaken identity. Drew is a proponent of the lower-case semantic web, and is currently expending energies in the direction of the microformats movement, with particular interests in making parsers an off-the-shelf commodity and developing simple UI conventions. He writes here at all in the head and, with a little help from his friends, at 24 ways.


You are viewing a mobilized version of this site...
View original page here

Mobilized by Mowser Mowser