Multi-touch and DOM events

The iPhone has highlighted a weakness with the HTML DOM - it has keyboard events and mouse events, but nothing for touch screens.

You can try to kludge touch screens using the mouse interface, or if you're simply looking to code drag-and-drop or resize, you can wait until HTML 5 comes along and use its new attributes "draggable" and "resize".

But if this doesn't work, you're stuck. And for other user interfaces, such as accelerometers in the Wii, the situation is even worse.

Sensors

So it's time to resuscitate my sensors proposal, which provides a framework for any sensor, not just a touch screen.

What it takes is a single new HTML DOM javascript function - document.sensors() - which returns an XML document of all sensors that the page has access to, e.g.:

<sensors xmlns="http://sensors.org/namespace">
<keyboard shift="" ctrl="" alt="" ins="" value="ab"/>
<mouse x="20px" y="30px" left="down" right="none" middle="none"/>
<touch pressure="30" x="150px" y="50px"/>
<temperature value="23C"/>
<video src="file://c/program%20files/webcam/webcam.mov"/>
<accel x="2" y="0" z="0"/>
<location latitude="37.386013" longitude="-122.082932"/>
</sensors>

This document is constantly being updated with the latest values - here, you can see there is a keyboard, mouse, touch screen, thermometer, webcam, accelerometer, and GPS. The namespace would define a standard basic set of sensors, but I'm sure that extra ones could evolve over time.

So, any web designer can access the latest status of the environment, in an extensible, standard way, using javascript.

In the case of a touch screen, you could place an onchange() listener on the element, to be notified of any movement. You could also watch out for multiple elements, if several fingers hit the screen.

Security

Some of the sensory information is private, so various security measures should be in place to protect prying eyes from getting access to it. I've listed a few ideas:

• Only pages with focus can see the sensory data
• Browsers allow site-specific permission settings for each sensor
• Webcam and background sound files can't be posted back to web servers

Conclusion

The simple suggestion above provides a much more comprehensive approach to human computer interaction than today's HTML DOM events. It handles not just keyboards, mice and touch screens, but all types of sensors - from thermometers to accelerometers to webcams - in a standard, extensible way. And it brings the web into many new areas - from security cameras to production line control to satellite navigation.