The mobile web: Fast Pipe, Always On, Get Out the Way

There have always been tensions in between mobile operators and manufacturors, because both want control over phone services like email, news and sports readers.

Now, a third sector has entered the mix - Silicon Valley. Google, Yahoo and Microsoft are all racing to get their own services and applications - such as search engines or mapping tools - installed on phones.

Consequently, there's confusion. Amid much publicity, Vodafone recently annouced they were automatically reformatting web pages to fit mobile screens - and it quickly became clear that they were removing every advert and breaking many sites.

Fast Pipe, Always On, Get Out the Way

There's only one clear approach to solving this, best summarised by Tim Bray, and it relates to mobile operators: Fast pipe. Always on. Get out the way.

In other words, mobile operators must resign themselves to being pure utilities, providing wireless data at ever-increasing speeds. They're wasting time and money creating their own software services for consumers, because Silicon Valley understand better what to provide, can innovate faster, and can scale better.

Back to the future...

The mobile industry will go the same way as the PC industry. Initially, the only applications available were installed locally on the client. Then, basic web browsers became available. As network speeds increase, web applications became ever more popular. Eventually, people will live in the browser, wondering why any other applications even exist.

Even Silicon Valley seems to be forgetting some of these lessons. Yahoo! Go is a java download for mobiles that displays email, internet search, and maps. Why aren't they just re-formatting their websites instead, especially as mobile browsers like Safari 3 and Opera 9 support features like Ajax?

Mobile operators ARE broadband providers

The recent rush of mobile operators into providing home broadband is no surprise. It's a chance for them to learn the ropes before the mobile industry becomes fully IP-based too.

Unfortunately, they've failed to learn a key lesson of the web: IP is stupid, and intentionally so. All the power lies at the edge, with the content. Providing good content and services is a very different business to managing an IP network, and I think they'll struggle to do both.

New types of computer: display and desk

For years, there have been three basic form factors for computers - the desktop, the laptop, and the PDA / smartphone. Now, finally, Moore's law and new display technology are changing things.

I'm not referring here to the iMac, Tablet PC or iPhone - these follow traditional factors, albeit in a new way. I'm also not talking about the underlying functionality, which is converging for all factors towards internet access, phone, and camera.

Instead, I'm saying computers will start to flourish in new environments. There are two in particular I've got in mind - the display, and the desk.

Display

We'll soon see many display computers for the home. These are computers used for displaying information or art - usually wall-mounted, or sat on a table or mantelpiece.

The first examples are digital photo frames. Already, some are adding Wifi, to display photos from a home PC or from a photo sharing website like Flickr.

Within a few years, they'll have touch screen web browsers too, enabling them to display any website.

Imagine a display in your kitchen showing a clock, your up to date calendar, the latest weather information, and the news headlines.

Or imagine a display in the living room connected to the British National Gallery website, rotating through their art collection.

Or even imagine a display on the bedside table, showing a clock during the night, and tuning in to a TV to wake you up with the morning news.

Although they'll be standard web browsers, people won't use them much to browse - mostly, they'll be connected to a single website, refreshing regularly to show the latest content.

Desk

When you're in the office, you want more than a computer desktop - you want a computer desk. Watch the Microsoft Surface demonstration to get the idea, and imagine if your entire office desk was a touch screen computer. This replicates the physical paper and files scattered around your desk for the virtual world.

Personally, I can imagine a computer desk being easier to use at an angle rather than horizontal, so you could reach documents further away.

I'm sure the software and processing power for this is here today. We might have to wait a while for the display technology, though - after all, it requires massive, durable, high resolution touch screens. But the first prototypes are coming out now.

Fitting our lifestyles better

People have been talking about the digital home for a long time. Finally, the vision is becoming clearer - touch screen web browsers, connected to personalized services in the cloud (the home server was a red herring).

Now we have this vision, and most of the technology required to achieve it. It's a question of fitting it to our lifestyles - whether it's in the office, the kitchen, or the living room.

SVG as image format

There are two common methods for adding SVG to a page - inline, via <svg>, and externally, via <object>.

The <object> element is bad. It's not semantic - it may as well be called <other> or <miscellaneous>. Although useful in the short term for displaying SVG, I would hope that this use will diminish.

The inline <svg> element is also bad, for the same reason - it's not semantic. It's the equivalent of having a <jpg> element, rather than using <img> - it's named after the format, rather than the purpose.

From the semantic perspective, there are three potential uses of SVG.

• Foreground images: use <img src="x.svg"> to point to an SVG file
• Background images: use CSS "background-image" to point to an SVG file
• Inline with connected DOM: use <iframe> to point to an SVG file.

These are much better because they re-use existing semantic elements.

Unlike foregrounds or inline images, backgrounds should not enable any user interaction - events (e.g. mousedown), hyperlinks, pseudo-classes (e.g. :hover), etc. Some people say javascript should be turned off - this might be a rough and ready first implementation, but some javascript might be appropriate (e.g. random placement of shapes, or animation), so long as the "no user interaction" rule is followed.

The other advantage of <img> and CSS background-image over <svg> is that you don't need to use XHTML. Standard HTML gets round a whole series of issues with mime types, browser control, and backwards compatibility.

Advantages of SVG as image format

SVG images fill a lot of gaps with HTML styling:

• rounded rectangles, circles, and any polygon
• fancy borders (arcs, swirls, etc)
• opacity, color gradients and filters
• shape hyperlinks and :hover, rather than pixel maps
• interaction via the DOM (for foreground images)
• scaling of background images multiple backgrounds (in one SVG)
• background text (e.g. graffitti, murals, etc)
• intricate website 'themes' to each page

The possibilities for graphical designers are huge.

Browser support

I'm very pleased to see that the next version of Opera will support SVG images via <img> and background-image. Unfortunately, it's not on the schedule for either Firefox 3 or Safari 3, although it's an aspiration for both teams.

There are four possible methods of using SVG in a webpage - <svg>, <object>, <img>, and CSS backgrounds.

The SVG implementation status for Firefox and Safari is marked at around 55%. Personally, while they only support two of these four methods, I'd hold them at half this - 22%.

Microsoft still don't get it

quote this week from their CEO:

Ballmer explained that Microsoft already is rearchitecting its core platform to be more of a Web-centric one. As he told the Partner Conference audience, “the programming model stays .Net and Windows.” But beyond that, Microsoft is is redoing its products and business models from scratch.

How do they expect to be Web-centric, if they're using tools for programming client-server? What's wrong with HTML, javascript, RSS, and dynamic server-side languages like PHP? Google must be laughing all the way to the bank.

Free your data: use HTML tables

Recently I've concluded that spreadsheets are not an optimal way to manage tabular data.

That's quite a claim - Excel earns billions for Microsoft every year, and the recent upsurge for OpenOffice is about standards and open source, not about form and functionality. Even Google has faithfully represented the spreadsheet as an online application.

So, let's review the problems with spreadsheets:

1. Page layout, text, and multimedia. You can't simply relegate this to MS Word - people want to annotate, explain and display their data professionally. Spreadsheets are awful at this.
2. WYSIWYG. Spreadsheets display endless rows and columns, no matter the amount of data.
3. Semantic. You can't distinguish headers, footers, or captions, except through styling. That prevents spreadsheets from being properly computer readable.
4. Storing data. it's hidden in a binary or zip file, among heaps of formatting, configuration data, etc.
5. Linking to external data especially on the internet - if you can only analyse your own data, you're missing a lot out - e.g. mashups.
6. Publishing data If you give someone a spreadsheet, they can edit all the cells - unless you rely on hopelessly insecure password protection!
7. Collaboration - online discussions, versioning, and synchronous editing

The first three issues are fundamental and inherent with any spreadsheet. The final three are inherent with client-based spreadsheets, but could be partially solved using online tools like Google Spreadsheets.

Using web standards for tabular data

Let's take a step back and categorize everything that spreadsheets do with tabular data, and whether there are any internet technologies with equivalent functionality:

Spreadsheet capabilities, Web technologies

Capability	Description	Web technology
Store	Visually in cells, semantically in data format	HTML tables
Transform	Sort, group, filter, pivot, consolidate, chart	DOM / XSL
Style	Borders, shading, text formatting	CSS
Model	Functions, calculated values, goal seek	Javascript, XPath

All these use cases - with the possible exception of conditional formatting, where CSS falls short - can be simply achieved using HTML, CSS, XSL, and a bit of javascript. The browser can beat both MS Excel and OpenOffice - we can free our data.

So what does it look like?

The solution is an editable HTML table inside a web page. The table contains all the spreadsheet functionality you need - sorting, grouping, functions, etc - but rather than taking up the whole page, it's just part of the page, and only contains the amount of cells you need. This allows analysts to surround their data with website text, images, or video (solving problem 1).

Only the appropriate number of rows and columns are displayed in the table - if you want more, you can add them (solving problem 2). This makes the page much more natural and avoids existing problems with people getting lost at the 64,000th row.

HTML tables are the best semantic way to store tabular data (solving problem 3), since there are a range of elements - from rows, to columns, to header and footers and captions - to label the contents. And because it's a web page, all sorts of collaboration, linking, and publishing techniques are immediately available (solving problems 4-7).

New use cases

There are plenty of new opportunities once you use the web to manage tabular data. None are particularly feasible using spreadsheets:

• mashups
• new widgets, e.g. maps
• embedded microformats (e.g. addresses, calendars, etc)
• version management (rather than endless versions on corporate C: drives)
• publishing and read-only tables
• extensibility - new functions & transformations
• using the web as a database, e.g. DabbleDB

Web Data Management

I don't think this even requires a new online application. You can easily imagine it being part of a blogging website - when you insert a table into your blog entry, the spreadsheet functionality immediately becomes available.

It's just a matter of imagination and time before this happens - and personally, I can't wait!

Semantic text editors and Wikis

Most text editors nowadays (including Microsoft Word) provide a huge range of different styles for the user to present their documents - bold, alignment, indent, border, background-color, spacing, font color, and many more.

As a result, their range of semantic elements is poor. As Ian Hixie says, "People think visually. Trying to ask a Web designer to think in terms of (e.g.) headers instead of font sizes is just something that WYSIWYG implementers and UI researchers simply haven't solved yet."

Too much style, too little semantics

This doesn't matter too much in a stand-alone Word Document. But in a collaborative environment - like a corporate portal, or a web community - it's much more important. It enforces a consistent look and feel, it allows for re-styling of the entire site if required, it reduces storage size, and it aids search engines.

The solution may be the Wiki. The Wiki designer selects site-wide CSS styles for each HTML element, with some options where necessary. For example, they may select:

• standard paragraph font type, color, size
• three types of <emp> element, with bold, italic and underline
• standard header elements
• standard list types
• various different table options, e.g. header rows and cells, data rows and cells

Then end users must choose the relevant elements, rather than using arbitrary styling. By removing unnecessary options - for example, enforcing the Arial font - Wikis are also actually easier to use.

The semantic text editor is an old vision. But new ideas about content management and user collaboration - specifically, Wikis, which will grow massively in usage - offer a solution.

Functional Animation, revisited

Since my blog post on Functional Animation, there's been some movement towards the idea of setting DOM nodes to equal functions of other (potentially varying) nodes.

In fact, this idea already exists in XSL - see the code below, where the second paragraph is set to contain the same text as the first paragraph.

<?xml version="1.0" encoding="ISO-8859-1"?>
<html xmlns="http://www.w3.org/1999/xhtml/" 
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<body>
<p id="p1">p1 content</p>
<p id="p2" title="{//p[@id='p1']}">
          <xsl:value-of select="//p[@id='p1']"/>
</p>
<a href="javascript:document.getElementById('p1').textContent 
  = 'test'">Test</a>
</body>
</html>

When you click on the "Test" link, the value of p1 is changed - and p2 should automatically update too (both its content and its title attribute).

Unfortunately, p2 doesn't update - it seems that browser makers don't automatically re-execute XSL stylesheets. Interestingly, CSS stylesheets do re-execute when you add or change nodes, and there's no mention of what should happen in the XSL spec. I don't see why browser makers couldn't extend their code.

An idea whose time has come

This idea of Functional Animation is so useful and intuitive - probably replacing most awkward DOM Events code - that it's a real shame there aren't any widescale implementations.

Having said that, I recently saw the idea cropping up in several places:

• the bind keyword in Java FX scripts
• the "{ }" binders in Flex 2
• Functional Reactive Programming
• the <output> form element in HTML 5
• the <bind> element in XFoms

Personally, I like the XSL approach with inline XPath, because it's much more natural at selectors. On the other hand, some might prefer javascript due to its familiarity and extra features (e.g. Math and String functions).

Either way, it's clearly an idea whose time has come!

REST, Ajax, and personalization

I didn't realise it until recently, but if you follow REST principles, then you're pretty much forced towards Ajax - the two come together. That's due to page personalization.

When I navigate to www.amazon.com, several parts of the page are personalized. It says "Hello, Chris Jay" at the top, and offers recommendations for me, presumably based on purchases I've made in the past.

Although it's a very nice touch, this bothers me. At first glance, following REST, shouldn't the URL www.amazon.com be the same for everyone? Otherwise it can't be cached!

Option 1: Separate URLs for everyone

One option would be to redirect me immediately to something like www.amazon.com/ChrisJay/, containing my personalized homepage. This would require permissions, as set by the cookie - otherwise, other people could access this URL too. But that also holds for the current Amazon site.

There are several benefits to this approach. Amazon and my ISP can now both cache my homepage, so when I access it again, I see it much more quickly. And it's obvious from the URL that the page is personalized.

Unfortunately, following this option, the main body of the page - the bit that's not personalized - is not cached separately. It has to be produced again for every user.

Option 2: Same URLs for everyone, use Ajax

Another approach would be to keep using www.amazon.com for everyone, but include some Ajax that scans the HTTP header, reads the current user id, uses it to GET another REST URL (e.g. /recommendations/ChrisJay), and displays it inline.

That way, the homepage can be cached for anyone (including the Ajax code), the personalized bits can also be cached separately, and the REST architecture is still followed.

For this reason, Option 2 is the better approach.

Designing your URLs

The first step to building REST sites is to select your URLs.

You might think the rule is "one URL for every page". But that's not necessarily true, as Option 2 shows, especially if you're offering personalized sections to a page.

A better rule would be "one URL for every page and for every personalized section".

Then you can use Ajax to tie the two up.