Iluminating, exhaustive article and illustrated history of multitouch. I wonder how many of these papers, prototypes and devices the patent examiners came across when they were deciding on the prospective multitouch patent now owned by Apple. Money quote:
----
Multi-touch technologies have a long history. To put it in perspective, my group at the University of Toronto was working on multi-touchin 1984 (Lee, Buxton & Smith, 1985), the same year that the first Macintosh computer was released, and we were not the first. Furthermore, during the development of the iPhone, Apple was very much aware of the history of multi-touch, dating at least back to 1982, and the use of the pinch gesture, dating back to 1983. This is clearly demonstrated by the bibliography of the PhD thesis of Wayne Westerman, co-founder of FingerWorks, a company that Apple acquired early in 2005, and now an Apple employee
Westerman, Wayne (1999). Hand Tracking,Finger Identification, and Chordic Manipulation on a Multi-Touch Surface. U of Delaware PhD Dissertation: http://www.ee.udel.edu/~westerma/main.pdf
In making this statement about their awareness of past work, I am not criticizing Westerman, the iPhone, or Apple. It is simply good practice and good scholarship to know the literature and do one's homework when embarking on a new product. What I am pointing out, however, is that "new" technologies - like multi-touch - do not grow out of a vacuum. While marketing tends to like the "great invention" story, real innovation rarely works that way. In short, the evolution of multi-touch is a text-book example of what I call "the long-nose of innovation."
----
Interesting examples relating to pinch-to-zoom:
[o] 1983: Video Place / Video Desk (Myron Krueger)
His use of many of the hand gestures that are now starting to emerge can be clearly seen in the following 1988 video, including using the pinch gesture to scale and translate objects: http://youtube.com/watch?v=dmmxVA5xhuo
[o] 1991: Digital Desk(Pierre Wellner, Rank Xerox EuroPARC, Cambridge)
Clearly demonstrated multi-touch concepts such as two finger scaling and translation of graphical objects, using either a pinching gesture or a finger from each hand, among other things.
This page makes it quite clear how improvements in this subfield of tech, like in many other fields, are evolutionary and build upon existing ideas in the literature. Many potential implementations have been proposed and experimented with. What role do patents play in this picture? What is the breadth of Apple's pinch-to-zoom patent - how far does it extend beyond smartphones? Are any of these devices close enough to the patent to legally count as prior art?
Thanks for this great submission. Buxton has been referenced on HN a few times before, but it's my first time reading his page.
He makes a pinch gesture, and the circle on the CRT gets bigger and smaller. If that's not pinch to zoom, or the precursor to it, I don't know what is.
Which is the whole problem with Apple's case. They seem to like to pretend that they invented everything that's used on the iPhone, but this sort of research has been going on for decades.
update: I think the most striking thing about watching that pinch to zoom thing is that he's doing it idly while thinking/talking about something else - it's no big deal for him. I suspect that he's been using gestures like it for quite a while at that point.
You're right about one thing - you see what you want to see. I'd swear if asked that he's using his thumb and finger to draw a rectangle shape on the desk representing the keyboard he describes at the same time, but there's no denying the circle on the screen, even if just a shape bounding his touch points, rather than an object he selected and zoomed.
The later clip with the two fingered multitouch zoom of the person that recoiledsnake links to is more compelling, even if the gesture doesn't quite match that we know as pinch to zoom.
How different is "scaling" an image from "zooming" a browser page? Are they different at all? Because scaling a figure is exactly what Kruger demonstrates in the linked video at 4:30, as the other commenters have pointed out.
Furthermore, it looks like he may be using a touch surface in addition to IR/video cameras, because he can draw strokes on-screen with his finger while his whole silhouette is projected. (A capacitative touchscreen or not, I don't know. Maybe the cameras have good depth perception.) Let's not mince words, "pinching" is equivalent to moving two fingertips together, and "zooming" is scaling.
I also find striking the little crawling/flying sprite his silhouette plays with, starting at 3:34. We're seeing something that should be in a Kinect game, 20+ years before the Kinect. He speaks eloquently about how natural it is for humans to expect behaviours in the real world to translate to the virtual world, when they interact with images on a TV screen.
His exposition is full of so many casually demonstrated possibilities, each of which could probably be patented in our current environment. This guy (probably) didn't - leaving them open to free, public use - and we should be thankful for that.
Look in the 1998 video (http://youtube.com/watch?v=dmmxVA5xhuo) around 4:30. You could perhaps argue that's 'scaling' not 'zooming', but that's a very pedantic position, and not one I personally would entertain.
Devil's advocate: It is neigh impossible to judge from the video, but to me, the video shows less of a direct link between hand movement and what happens on the display. It is not as if the hand really grabs the screen. I also got the impression that he had to issue a command, chosen from the top of the screen (outside the video image) to switch from scaling to rotation. Another argument could be that this user does not actually seem to touch the screen (at least, that is my impression). It is likely that all these are due to technological limitations, but if I had to defend this as "not prior art" in court, I would try and work from those observations. There is little else to argue about, as a lot of the interactions are there, in a primitive (technologically) way.
----
Multi-touch technologies have a long history. To put it in perspective, my group at the University of Toronto was working on multi-touchin 1984 (Lee, Buxton & Smith, 1985), the same year that the first Macintosh computer was released, and we were not the first. Furthermore, during the development of the iPhone, Apple was very much aware of the history of multi-touch, dating at least back to 1982, and the use of the pinch gesture, dating back to 1983. This is clearly demonstrated by the bibliography of the PhD thesis of Wayne Westerman, co-founder of FingerWorks, a company that Apple acquired early in 2005, and now an Apple employee
In making this statement about their awareness of past work, I am not criticizing Westerman, the iPhone, or Apple. It is simply good practice and good scholarship to know the literature and do one's homework when embarking on a new product. What I am pointing out, however, is that "new" technologies - like multi-touch - do not grow out of a vacuum. While marketing tends to like the "great invention" story, real innovation rarely works that way. In short, the evolution of multi-touch is a text-book example of what I call "the long-nose of innovation."----
Interesting examples relating to pinch-to-zoom:
[o] 1983: Video Place / Video Desk (Myron Krueger)
His use of many of the hand gestures that are now starting to emerge can be clearly seen in the following 1988 video, including using the pinch gesture to scale and translate objects: http://youtube.com/watch?v=dmmxVA5xhuo
[o] 1991: Digital Desk(Pierre Wellner, Rank Xerox EuroPARC, Cambridge)
Clearly demonstrated multi-touch concepts such as two finger scaling and translation of graphical objects, using either a pinching gesture or a finger from each hand, among other things.
This page makes it quite clear how improvements in this subfield of tech, like in many other fields, are evolutionary and build upon existing ideas in the literature. Many potential implementations have been proposed and experimented with. What role do patents play in this picture? What is the breadth of Apple's pinch-to-zoom patent - how far does it extend beyond smartphones? Are any of these devices close enough to the patent to legally count as prior art?
Thanks for this great submission. Buxton has been referenced on HN a few times before, but it's my first time reading his page.