Ajax is the beginning of the end for relational databases

On the web, every page is a text document. The text documents are interpreted and displayed by browsers according to a standard contract – HTML.

The classic client-server programming model works very differently – a Visual Basic application, for example, will contain all manner of binary files and libraries that directly call operating system functions. It’s like writing a new browser for each website!

The choice of model has far-reaching consequences – for example in security, in user interface design, in software distribution, and in application integration. But, surprisingly, one part of the classic model remains – the relational database. Mixing a relational database with online text documents should really not work:

• Database tables are not documents, so they can’t be referenced by URLs or appear in hyperlinks in a standard way
• There is no standard file format for database tables, so they can’t be transported in a standard way across HTTP
• Cacheing models for databases and web servers are inconsistent
• Relational database best practice is for data normalization, whereas HTML documents are naturally un-normalized and hierarchical
• Security works differently in the database layer to the web server layer, leading to potential inconsistencies and gaps
• Systems integration for databases is much more difficult, due to the lack of a common extensible approach to table structures and their transformations

As a workaround, developers borrow old client-server tools like ADO.net and JDBC links in object-oriented code, and mismatch them with the text document. This creates a complicated intermediate layer and doesn’t really address the issues above.

Since the biggest use of relational databases nowadays is to supply HTML documents, their massive take-up is something of a mystery. It’s explained by three factors – they are fast, they are stable, and they haven’t had much competition!

But now some competition is arriving at the low end – Ajax. Ajax offers the ability to look up reference data sources during runtime, which is exactly the function of a relational database. And by using URLs, HTTP, XML and javascript, Ajax overcomes each of the relational database flaws above.

Viewing Ajax as a competitor to a relational database might be unconventional, but it’s revealing. In Ajax, the XML file itself represents the database table. There is less focus on a query language like SQL - DOM javascript carries out the bare minimum, with XSLT transformations filling some gaps – but a lot more focus on getting access to the data via URLs and HTTP and XmlHttpRequest.

Apart from data access, the key advantage of Ajax is that it returns data in the same text document format as HTML. So it can be readily inserted without any intermediate layer – in fact sometimes with pretty much no code, using something like < div src="database.xml" >

Of course, you would never use Ajax to download an XML file with a million rows. But you could do some pre-processing on the server, and return a brief XML results file via Ajax. And this pre-processing step is becoming one of the main functions of relational databases – they are being relegated to storing data and returning query results in XML via HTTP, which his not their natural position.

Recently several groups have extended this approach. On the W3C side, an ambitious attempt to replace lines of javascript code with declarative XML binding has been defined – XForms. Although of great theoretical benefit, XForms is hampered by its lack of adoption by the browser manufacturers. And as a grassroots movement, the Atom Publishing Protocol is seeking to standardize the structure of XML databases. It does not require any browser enhancements and has already attained significant adoption, most notably by Google as the foundation for Gmail, Google Calendar and Google Docs & Spreadsheets.

My contrarian prediction for 2007 is that Ajax will take over from ADO.net and JDBC as the middle ground between web pages and data sources. The Atom Publishing Protocol will become a popular way to manage data sources – taking over even more of the relational database scope.

Relational databases will get decoupled from web application development. They will be increasingly relegated to XML storage and output, plus their traditional role in the declining client-server world. After more than 20 years, this is the start of their demise as the all-conquering database technology.

The Mobile Phone

Over the years, the question "what is a mobile phone" has raised different responses. From a rich man's toy, it evolved into an essential device for making calls. Now it's a bit more confusing - "phone" doesn't seem right for a device that plays music and provides email access, and the British slang "mobile" seems a bit vague.

There's a revealing connection with the misnomer "Personal Computer". PCs are not personal - we share them at work, at home, and on our travels. They used to be personal, but then the internet happened, and with it the ability to log on to any website from any PC.

It’s far more appropriate to call mobile phones “Personal Computers”, because they are truly personal; we feel naked without them, wherever we are. And due to that same factor - the internet - they are becoming general purpose computers.

In the next couple of years the internet will take pride of place in the mobile phone. For example, imagine if your mobile phone was synchronized with your email account, so you could visit Hotmail from any PC and update your phone address book, view your missed calls, listen to voicemails, and phone someone up, as if from your phone. And it would work the other way round, too - you could read your Hotmail emails and instant messages from your phone. This would provide a unified communications service, with a single ‘inbox’ containing email, instant messages, SMS text messages and voicemails, accessible from anywhere.

How many of us have lost their phone address book along with their phone (or even when upgrading phones)? The unified communications service avoids all these issues by storing the details on a website and synchronizing with the phone.

And what if you could automatically access all your phone's photos and music via your email account? And if you uploaded more to your email account, you could view them too via your phone?

During its path to becoming a truly personal computer, the mobile phone will become a device for accessing your Hotmail or Skype or iTunes account while on the move.

From a business perspective, this creates a huge dilemma for the mobile industry. Does Vodafone become just a mobile ISP? Or does it build a website to compete with Hotmail, Gmail, Skype and iTunes all at once?

From a technical perspective, the vision highlights the absurdity of the .mobi domain and other attempts to create a 'walled garden' - they are betting against the internet. Once you allow phones to access the web, they are exposed to creativity that will overcome any attempt at central control. Small screen sizes will always be a limitation, but there’s a lot you can do given a limited real estate if you really try!

From the customer's perspective, the mobile internet will finally enable the new world promised (but not delivered) of 3G - 'killer' new products and services enabled via websites. Many of these take advantage of physical proximity – see below for some ideas - but all of them rely on the web.

For example, what if your phone had swipe-card functionality? This would enable you to make simple purchases without carrying your wallet around. And it would be internet-based – during the swipe, the phone would fetch a web page that asked for authentication and confirmed details of the transaction. Rather than typing a website into the address bar, the browser is activated by physical proximity. Creative informative and transactional uses will result from following web standards.

The personal nature of the phone could also be used to secure web transactions. Banks are introducing two-factor authentication to improve security, which augments the traditional password with a read-out from a physical device. Why should the phone not play this role?

Internet access will also be key to finally introducing mapping services on your phone. Why not use Google Earth on your phone? This would allow you to easily find directions, locate the nearest Chinese restaurant, or even receive warnings when your children wander.

And finally, one of the major gripes of mobile phones is their high international charges. The mobile internet would eliminate this, as there is no geography on the internet. All calls are the price of local calls.

The mobile phone is turning into the truly personal computer, and mobile web access will enable its functionality, from phone calls to emails to swipe card.

Creative Destruction in the Media

Much has been made of the 'long tail' of the internet - the fact that on the web, niche, amateur producers of content, like YouTube film makers or unsigned MySpace bands, can collectively account for a significant proportion of the industry.

But in reality this presents no danger to existing professional producers - as disposable incomes rise, people will want more and better entertainment, and professionals will comprise the 'quality' end of a much bigger industry. There will always be demand for the Sopranos, or Beyonce, or even TV gameshows, and it's funny how those unsigned bands don't stay unsigned for long!

The real revolution will occur in content aggregation and distribution. The fundamental principle will be 'universal access' - anyone in the world using any internet connection will be able to access every piece of content ever created, whenever they want to. At home, on your phone, at any PC; films, music, live sports, chat shows, home videos & photos, the whole lot, using just a credit card!

It's happening already - witness the flurry of deals that YouTube is signing with media companies, or the success of iTunes, or TV companies rushing to post shows on their websites.

From a business perspective, this means creative destruction. It will turn the industry upside down, destroying many business models and bringing riches to those quick to change. There will be three main effects: globalisation of aggregation, pay-by-advert, and the fall of the conglomerates.

First, globalisation. National media companies will find themselves in competition - if, sitting in the UK, I can log on to a US website to watch sitcoms, why should I bother with ITV? If I can watch the Champions League on a French website, why should I view it on Sky? The only reason we currently have excusive geographical deals (like the rights to show "Friends" in Germany) is because TV distribution has always been split geographically too. But on the internet, companies can set up as global distributors, in competition with national operators, and reap massive economies of scale (including better negotiating leverage with the producers). ITV will have to decide whether it can compete with HBO in providing US sitcoms, or whether it should stick to national content like the XFactor.

The second effect is that the producers will be paid directly per advert. On the internet, it is possible to target adverts personally to each individual watching a show. Google pays advertisers per click, but TV is less interactive and not likely to generate many 'clicks'. But it could pay per view of each advert, and this revenue could be divvied up with the producers. In this model, episodes of 'Friends' would be posted to YouTube, advertisers would bid on a per-viewer basis, and the revenue would be shared among the parties. No exclusive deals, no geographical boundaries, just an open online marketplace.

The third effect is the fall of the conglomerates. There will still be large media companies, but they won't cover the whole value chain (production, aggregation, distribution). People purchase music from iTunes, that was produced by a band working for EMI, and the distributor is their local broadband provider. In the same way, people will view films on YouTube, that were produced by Universal Studios, and the distributor is the broadband provider. The aggregator (like YouTube) will be interested in maximising coverage fairly for the consumer, which is a conflict of interest from promoting the output of a single producer. And broadband providers have tried and failed to push their own media - once you've got someone on the internet, you can't force them to visit your website. So the industry will split into producer (BBC / HBO), aggregator (YouTube / iTunes), and distributor (ISP / telco).

The benefits for the consumer will be huge - the goal of 'universal access' is now achievable.

But is the business world ready?

It seems incredible that this enormous industry is being shaken to its foundation by the web - a technology dreamt up by a physicist only 17 years ago (http://www.w3.org/History/1989/proposal.html). The participants are still grasping the scale and direction of change, and there is plenty of active resistance by the media companies. But the world is changing around them - led by Silicon Valley.

The future of telecoms

Silicon Valley has upturned many industries in its time - now its 'creative destruction' is at work again. The telecom industry is currently undergoing a huge strategic shift:

• Landline phone companies merging with mobile companies
• Mobile companies merging with broadband companies
• The rise of Voice Over IP (VOIP), using data networks to efficiently transport phone calls

Where is all this heading?

The reason landline companies are merging with broadband and mobile companies is because it's vastly less expensive for landline companies to run voice over an IP data network than over a traditional circuit. And thanks to rapid technology advances, it will quickly become even less expensive (and higher quality).

And the mobile industry recognises that 3G is a failure - a massively expensive minor enhancement - but it can gain redemption by offering high speed mobile data access and internet services.

But once you've given your customers access to the internet, you can't make them use your content! This is the lesson all ISPs have learnt - think of the failed old AOL portal abandoned by customers in favour of Google, the 'free email addresses' abandoned in favour of Hotmail, and 'walled gardens' on the mobile web that have failed to attract customers.

Current phone companies - like AT&T or BT - will just sell access to the internet (fixed or mobile). No-one will have a phone number with them, just like no-one uses their ISP's provided email address, or visits their ISP's internet portal for news. People will use Google Talk or Skype for phone calls, and Hotmail or Yahoo mail for email addresses, and cnn.com or bbc.co.uk for news.

The reason? Skype (like its competitors Google Talk and Vonage) understands internet technology, moves at internet speed, and will have more customers than a broadband provider ever can, because it is open to anyone globally on the internet. So it can take advantage of massive efficiences of scale in providing voice services, and it can integrate them with other services like blogging or instant messaging.

The same will happen in the mobile world (albeit more slowly given lagging technology) - people will purchase mobile access to the internet. But they won't use the mobile phone number they were given; instead they will use their Skype or Google Talk account.

The industry will be split in three:

• Device manufacturers, selling mobile devices and PCs. Three or four globally, based in Far East.
• Broadband providers - offering fixed and mobile broadband internet access. Three or four based in each country.
• Internet Communications companies - websites offering email and phone services over the internet. Three or four globally, based in Silicon Valley.

Existing phone companies, landline and mobile, will become simply broadband providers. And they will remain utilities, despite all their attempts to break out of this market. Their best option is to receive commission from internet communications companies, by referring new customers. This is what happened this week in the UK, with BSkyB white labelling Google's products to its broadband customers.

And internet communications companies will offer an integrated suite including email, phone with voicemail, instant messaging, RSS reader, blogging, buddies, community features, personal web-pages as per MySpace, etc. These will be served to any device, with any screen size, whether fixed or mobile. And they will come in two flavours - free but advert supported for consumers, and subscription-based for enterprises. It's a whole lot more than just phone calls, and the telcos can't compete!

I'm sure we'll see many attempts at vertical integration across these industries. But the cultures and business models are so different that this will be an anomaly. Telcos will become utility internet access companies, and all 'content' - including phone calls - will be handled via separate communications companies.