Programming Windows: Tablet PC (Premium)

Bill Gates fervently believed that Tablet PCs and their stylus-based handwriting capabilities together represented the future of portable computing.

He was wrong about that, as it turns out. But the Tablet PC capabilities that Microsoft developed in the early 2000s would be improved, and over time they were incorporated into the Windows versions used by mainstreamed portable PCs. And it’s fair to say that this initiative would ultimately impact Windows as a platform to a much higher degree than did contemporary XP technologies like Freestyle (Windows Media Center) and Mira (Windows Powered Smart Displays).

As Microsoft’s final Windows deliverable to customers in 2002—Microsoft had circled November 7 for the launch date several months earlier—Windows XP Tablet PC Edition represented years of research, competitive analysis, and development across a wide range of hardware, software, and services. It was a pet project of Bill Gates, who oversaw the entire process at a high level and spearheaded its final push to release.

It wasn’t the industry’s first attempt to digitize handwriting, of course. Earlier portable computing devices called personal digital assistants (PDAs) had employed simplistic and small styluses to let users enter data on their tiny displays. Apple’s Newton had infamously been lampooned by cartoonist Garry Trudeau in his “Doonesbury” strip for its comically bad handwriting recognition. And so latecomer Palm bypassed this issue by inventing a simplified notation system called Graffiti that users had to learn. It was inelegant, but it worked.

It wasn’t even Microsoft’s first attempt. After a failed WinPad initiative in the early 1990s, Microsoft created and then evolved Windows CE, which ran on non-PC portable devices like Handheld PCs, Palm-sized PCs, and then Pocket PCs, the latter of which saw some measure of success, thanks largely the popular and expandable Compaq iPAQ. Windows CE would never rise to the sophistication or power of Windows XP Tablet PC Edition, but it got better and better over time and perhaps should have remained the primary focus when you consider how the market changed years later when Apple introduced the iPhone in 2007 and then the iPad in 2010. But in Microsoft’s defense, even the most forward-looking prognosticator could never have predicted such a turn of events. And the Microsoft of 2000 was very much obsessed with its Windows cash cow.

In any event, Microsoft would solve the problems with PDAs via “the magic of software,” as Gates called it, and the steady advance of the PC platform. As with Media Center, the core strength of Tablet PC—it was a PC—was also its core weakness. But the central genius of Tablet PC was a combination of capabilities that together represented a major leap forward.

The first was a technology called ClearType, which Gates announced at COMDEX 1998 in Las Vegas. ClearType solved the problem of “jaggy” onscreen text by using sub-pixel rendering techniques on color LCDs to triple the perceived resolution. ClearType was great for reading applications, of course, but it also impacted productivity applications like Word and Excel, and Microsoft made it a core part of Windows, hastening the move from CRT displays to flat-panel LCDs.

The second required an advanced display digitizer that would accept input from a digital pen and do so at such a high resolution that handwriting and drawings would look and feel natural, as if the user was writing on paper with a real pen. Even more impressive, these digitizers would be active, meaning that they could sense the digital pen in real time as the user moved it above the screen; an onscreen cursor could, for example, follow the movement of the pen above the screen and then immediately begin displaying a digital ink when it was set down on the display.

The third and perhaps most impressive advance was a software system that could recognize handwritten notes reliably, adapt to a user’s unique writing over time, learning as it went, and then store those notes as “a first-class data type,” alongside text, Word documents, and other data. That handwriting could then be scanned, understood, indexed, copied to—and pasted from—the Clipboard, and more. And while it could be converted to text, it didn’t need to be. It simply lived in the system as handwriting.

This system was so advanced that it required hardware that didn’t exist yet, not just the display digitizers, but also a new generation of microprocessors that were somehow both powerful and energy efficient. And so it took Microsoft several years to come up with a prototype that in any way resembled a real tablet with wireless connectivity that one might carry around, untethered to a desk or wall outlet.

But it slowly started coming together. And excited by the advances he was seeing internally, Gates began mentioning “new form factors like tablet PCs” throughout 1999 and 2000.

At the Fusion 99 show July 1999, he noted that ClearType “actually had better readability than paper” thanks to new 200 DPI displays “enabling these tablet-type form factors to show up.” Combined with a wireless network, you’d be able to walk around your home or the office and get work done. “Knowledge workers will be the ones [who] get the natural interfaces, as we get the speech recognition and handwriting recognition, they’ll be the ones who will take advantage of that,” he said. “And so every meeting you’ll have the full PC or this tablet PC and not just a tablet of paper.”

“This is the idea of a tablet PC,” he said at Microsoft’s Financial Analyst Meeting that year. “This is one that, although it’s in the charging station, you just take it out, you have the stylus to work with. We’re working with a number of people, all the leading companies who do portable machines. We’re doing tablet form factors.” He showed off early prototypes of different sized tablets, one that was basically a smaller e-reader and one that was a larger tablet PC with an “8 by 10” display, like a piece of paper.

Tablets came up again during his COMDEX 1999 speech, with Gates noting that they could be used to access and record information in meetings. “Sitting in a meeting, you can take notes, as soon as you go into that meeting room, it recognizes everybody who is there,” he predicted. “You can put information up on their screens and annotate that as you go along. And so really taking the time that’s spent in meetings and making it quite different than it is today, getting rid of the paper forms so that information can be navigated up on any of those screens.”

At TechEd 2000 in June, Gates pushed Tablet PCs on the IT pros in the audience.

“One of the new form factors for the PC will be a tablet form factor, a device you can take to a meeting, take your notes on, and that becomes natural to use the same way that a paper tablet would have been used in the past,” he said. “The readability of these tablet devices will mean that for the first time we don’t have a split between our paper world and our digital world. We don’t have to go back after meetings and try and decide which notes to type in, or when we go to our terminal to read electronic mail decide which emails are long enough that we need to print them out. And only in that form can we do the kind of annotation that we’re used to with paper …  taking your notes with this new tablet PC form factor, having those be something that are easy to go back and find, and easy to share in their digital form.”

And then, finally, the first demonstration arrived. At Forum 2000 that June, Gates turned the spotlight on the tablet work that Microsoft’s researchers had been doing. He brought out Burt Keylie to show off a large, orange slate called the Tablet PC.

“This is the device that combines the visual qualities of a magazine with the handiness of a paper notebook and all the power of a PC,” Mr. Keylie said. “So what we’re going to get to talk about today and show today actually is the actual first demonstration of what Microsoft calls our vision for the Tablet PC. Within the next year or two you’re going to see Microsoft’s partners bringing out hardware like this. This is actual prototype hardware running real software, including real Windows 2000.”

Keylie explained that the Tablet PC would enable the reading of electronic books, with animated page turning and zoom capabilities that took advantage of ClearType. He used a stylus to select on-screen text to highlight it, create bookmarks, and look up definitions of selected text. “There are no visual artifacts, no clutter,” he said. “These are the kinds of qualities of paper that we really think that a tablet PC should be able to have.”

Keylie also demonstrated an online bookstore that worked with the Microsoft Reader software and could manage subscriptions to publications like Slate. It used .NET services on the back end and worked with Digital Rights Management (DRM) technologies to protect content creators. He also showed off productivity applications like Outlook to check email, download an attachment, and then open and read that attachment.

Most impressively, Keylie showed that his handwritten notes were interactive. He selected the ink and highlighted it, as he had earlier with text. He resized his notes and they reflowed normally. He searched his handwritten notes for specific terms. And manipulated drawings.

“I’m certainly excited about getting one of those devices,” Gates intoned. “There’s really an amazing amount we’ll be able to do with notetaking and sharing information … A key point there is, that was a full-powered PC. That was not just a limited companion device. That was a Windows 2000 device that ran all the existing applications and the new ones that take advantage of that platform.”

At the Professional Developers Conference (PDC) that July, Gates promised that Microsoft would have prototype Tablet PCs by the end of the year, and he expected “to see those out in the marketplace in the next couple of years.” The Tablet PC, he said, “alone is going to be a market that we feel will be even bigger than today’s portable PC market.”

At COMDEX 2000 in November, Gates again carted out Mr. Keylie to demonstrate the Tablet PC. New to this demo was the notion of inserting space in handwritten notes to add new notes midstream. The term “ink” had by then evolved to “electronic ink,” but it would soon come to be called Digital Ink.

By 2001, Microsoft would reveal and then launch Windows XP, and so its Tablet PC work moved on to the new platform. And at WinHEC 2001 in March, Gates noted that Microsoft had “five leading system manufacturers committed to the Tablet PC project,” with Intel delivering a new generation of microprocessors to meet its unique needs.

“The Tablet PC, you can think of it as revolutionary or evolutionary,” he said. “It is evolutionary in the sense that it runs Windows XP and it runs all the applications. It’s evolutionary in that when you buy it, of course it will have either a wireless or a connected keyboard. But it’s revolutionary because of your ability to take it in your hands, sit there and read and annotate, take notes, things that were not possible before. So the scenarios, the breadth of use and the way that the software will take advantage of the pen and that direct manipulation, that is very profound. With the pen you can do things like editing marks and move things around in a way that even the mouse isn’t nearly as good at.”

Charlton Louis provided the demo this time, again using a Microsoft prototype that he described as an evolution of the laptop.

“There’s a lot of computer science going on here,” he said, demonstrating an application he alternatively called Microsoft Notebook and MSN Notebook. “When I lift the pen, for one we’re actually sampling at 133 samples per second as opposed to the 30 or 40 samples per second for a typical mouse. In addition, we’re turning the poly-lines into Bezier curves and providing anti-aliasing to make the ink even look better. So ultimately the tablet makes ink rock.” He noted that Microsoft was also working with third-party software makers to create Tablet PC-specific apps.

At COMDEX 2001 in November, Gates made several Tablet PC revelations. The product, he said, would be called Windows XP Tablet PC Edition. And it would arrive via two basic hardware designs in 2002, the slate-like Tablet PC that Microsoft’s researchers had designed for use with an external keyboard when docked, and a new form factor, created initially by Acer, called a convertible PC.

The convertible was a laptop with a display lid that could swivel after you released two latches, so that you could cover the keyboard with the display and use it like a thick tablet. Gates showed off early Tablet PC designs from Acer, Compaq, Fujitsu, Toshiba, and several others during his speech.

The participation of seasoned PC makers improved Tablet PC immeasurably. Where Gates and Microsoft saw the platform as the real-world successor to Alan Kay’s Dynabook, a tablet concept from the late 1960s that Xerox had refused to entertain building, Microsoft’s hardware partners were more pragmatic. And most saw that users would need the familiarity and productivity of always-available keyboards and trackpads for this platform to make any sense. Some opted for slate designs with clip-on keyboards. Others, like Acer, preferred convertible designs.

The launch was finally getting close. In his CES keynote in January 2002, Gates promised that the Tablet PC would “be coming out in the second half” of the year.

“The Tablet PC to me is probably the most important thing to come along for the last four or five years on the client,” he said at WinHEC 2002 in April. “Reading, notetaking, that simplicity is very important. We’ve reached a threshold where the size and weight of that device are very reasonable, but I’m sure that there will be constant improvements there. The first-generation devices look very good. They’ll be out from five different [PC makers] late this year, but I’m sure there will be second-generation devices where you get the weight, the size down, you customize the LCDs even more for the kind of reading experience, so a very rich vein of improvements that will come along with that.”

I had been allowed to briefly play with Microsoft’s prototype Tablet PCs before, but at WinHEC I had my first extensive experience with various pre-release Tablet PCs from Acer, Fujitsu, and other firms. The Acer convertible, with its latches and swiveling display, was particularly interesting, and so I was happy to receive a review unit a few months later when Microsoft held a Tablet PC Reviewers Workshop in Seattle. That workshop was interesting for several reasons. Bill Gates was on hand to describe his personal role in bringing the product to market, and he explained that he used a Tablet PC for reading, notetaking, and other tasks.

“I will say, and this has been a positive surprise, that once you get the PC into a natural form factor, people start doing digital imaging more,” he told us. “It just becomes more personal. It makes the photo experience so much better. The PC isn’t designed for two people to sit there and share. Even things as simplistic as buying books online are better with the Tablet PC. It’s going to make computing more personal. These activities were previously limited, Now, with this form factor, they’re natural.”

Gates described how the platform had evolved, how important it was that the PC makers had gotten involved, and how their designs had already evolved since the CES reveal. He noted that one of the hardware advances Microsoft had wanted to include in the Tablet PC, solid-state storage, had not advanced as it had hoped, and they were surprised to discover hard drives that were small enough to use instead. But solid-state storage would come, he said, and future Tablet PCs would be thinner still.

Gates didn’t mention this specifically, but the hoped-for advances in mobile PCs didn’t happen quickly enough either. Most Tablet PCs shipped with inefficient Intel Pentium III microprocessors, though a coming generation of Centrino-based Pentium-M CPUs promised better battery life. One esoteric Tablet PC, from Compaq (later HP) would use a Transmeta CPU that translated the Intel x86 instruction set on the fly and promised terrific battery life. But the performance was terrible.

During the Q&A, things got interesting.

Reviewers questioned whether Tablet PCs would ever go mainstream, with Gates simply stating that his “personal adoption cycle” was immediate and that he “wouldn’t go to a meeting without it.” One reviewer asked about the pervasiveness of wireless technologies, which was seen as a key requirement to make the Tablet PC feasible. One questioned why Microsoft would even launch this product when a fully compatible version of Office wasn’t even available yet.

But one question caused the biggest disruption I’ve ever experienced at a Microsoft event to this day. Stephen Manes, the co-author of the highly regarded book “Gates,” demanded to know how well the handwriting recognition worked. And he wanted a specific figure. Was it 60 percent of the time? 65 percent? What?

Alexandra Loeb, Microsoft’s Tablet PC vice president, explained that the company didn’t have a specific number like that because there wasn’t one: the results varied from person to person. For some people, the recognition was excellent and worked well immediately. For others, the system needed a bit of time to learn the nuances of their handwriting. Oddly, the success rate didn’t seem to be tied to how well someone wrote.

This didn’t satisfy Manes, and his demands got more and more irrational. “You have the number,” he demanded. “Tell us the number.” Gates, Loeb, and the others on stage exchanged confused glances. There was no number, they insisted. But Manes wouldn’t back down. So I finally turned back towards the imbecile and said, “sit down and shut up, you jackass.” And he did.

(Months later at CES 2003, I had the weird experience of hearing Manes tell this story from his perspective to a handful of clueless fans while we were all waiting to get into a Showstoppers event. They were behind me, and I could hear Manes bragging about how he had stood up to the suits and demanded—demanded—that Microsoft tell him the truth about Tablet PC handwriting recognition. So I turned around and asked, “Did he tell you the part where I called him a jackass?” They all looked at me, stunned. So I smiled and walked away.)

My experience with the review device—that Acer convertible PC that I felt represented the best that the PC makers had to offer at that time—was that the handwriting recognition was surprisingly good. It wasn’t perfect, not even close. But that it worked as well as it did and could learn as you went without requiring you to sit through a formal teaching session was impressive.

“You’re about to have a big treat,” Gates told us, ignoring the Manes embarrassment. “Enjoy your Tablet PCs.”

The review unit was an Acer TM100 with an 800 MHz Pentium III, 256 MB of SDRAM, 20 GB HDD, 10.4-inch 1024 x 768 TFT LCD display, two USB ports, Firewire, external VGA, PCMCIA, LAN Port, and “pre-installed internal 802.11b on a mini-PCI card. It had a 26-watt-hour battery. There was a USB-based optical drive, but like other Tablet PCs, no internal optical option.

Later that month, Microsoft revealed more PC makers had come on board to ship Tablet PC designs, including Motion, which had been started by some former Dell engineers. Adobe, Corel, FranklinCovey/Agilix, SAP, and others were busy adapting their software for the new platform as well. And Microsoft would ship Microsoft Reader 2.5 to take advantage of the Tablet PCs’ bigger and higher resolution displays.

In August, Microsoft released Windows XP Service Pack 1 (SP1), which would form the basis for both Windows XP Media Center Edition and Windows XP Tablet PC Edition. Media Center was released to manufacturing in early September and shipped in October. And then Microsoft launched Windows XP Tablet PC Edition, as promised, on November 7, in New York.

“The launch of the Tablet PC marks an exciting new era of mobile computing that is limited only by the imagination of its users,” Bill Gates said at the launch. “The Tablet PC is a great example of how computers are adapting to how people really work, whether they’re taking notes in a meeting, collaborating wirelessly with colleagues, or reading on screen. We’re just scratching the surface of what is possible.”

Tablet PC slate and convertible designs from Acer, FIC, Fujitsu, HP, Motion Computing, NEC, Tatung, Toshiba, and ViewSonic were on hand at the launch event, and Microsoft revealed that a new entry from Panasonic would soon be available today. I was particularly impressed by three of the new Tablet PCs. The Viewsonic V1100 was a 3-pound slate design that let you swap out its battery without losing data, a nice touch since battery life was only 3 hours. The HP Tablet PC TC1000 was a former Compaq design that featured a detachable keyboard base and an optional docking station. And the Toshiba Portégé 3500 was a convertible like the Acer but without the screen latches; it had a 12.1-inch display. I would go on to review all three of them.

Historical sidenote: The Toshiba Portégé 3500—and its successor, the M200–would go on to become the Microsoft corporate standard, and for a few years there it was unusual to see a Microsoft employee without one. Indeed, it was unusual to see anyone using a Toshiba Tablet PC who wasn’t a Microsoft employee. One time, I was flying from Boston to Seattle, and as I walked towards the bathroom on the plane, I spied someone using one in his seat. So I asked him what he did for Microsoft. He started to answer and then said, “hey, wait a second. How did you know I worked for Microsoft?” I pointed at the Toshiba and smiled. It was a dead give-away.

As for the software, Windows XP Tablet PC Edition was a superset of Windows XP Professional that included a surprisingly thin range of new features. The most obvious was a new Input Panel (later, Text Input Panel, or TIP), which was at first accessed via a new icon next to the Start button in the taskbar.

This panel could be displayed at any time, letting the user toggle between an on-screen keyboard and a writing pad, which could be used to handwrite onscreen with a digital pen. The handwriting would be recognized by the system, converted to text, and then sent to an open application, such as a web form in Internet Explorer, an email, a Word document, or similar.

XP Tablet PC Edition also came with a handful of Tablet PC-specific utilities and applications, including Windows Journal (previously Microsoft/MSN Notebook) for notetaking and ink document storage management.

There was also an ink-based version of Windows Sticky Notes, several tutorial walkthroughs and pen and Tablet PC feature configuration utilities, and a fun little game called Inkball.

Users could also install a free Tablet PC Office Pack for Microsoft Office XP that added digital ink capabilities to Word, Outlook, and PowerPoint only. Oddly, some Windows apps one would expect to be tablet-enabled, like Paint, were not initially.

Despite Microsoft’s optimism, the Tablet PC didn’t set the world on fire. But some of its problems were solved over time as the Pentium III gave way to the Centrino-based Pentium-M, solid-state storage became viable as a hard drive replacement, and the digitizers required for pen input got thinner and more sophisticated. On the software front, Microsoft followed up its initial Tablet PC release with “Lonestar,” which went on to be released as Windows XP Tablet PC Edition 2005. This version of the system offered several important improvements, including an improved TIP that was no longer locked, inconveniently, to the bottom of the display but would instead pop up as a floating window wherever it was needed.

Lonestar also brought a real-time handwriting recognition feature that Microsoft had planned for the original release but had been nixed by Bill Gates because it was only 95 percent reliable. With this feature, Microsoft’s handwriting technology would now translate your chicken scratches on the fly, using a small space at the bottom of the TIP’s Writing Pad area to preview each letter as you wrote.

It also benefitted from the release of a new office productivity suite, Microsoft Office 2003, that included more pervasive Tablet PC features: Word, Outlook, PowerPoint, and Excel 2003 all featured native support for text insertion and annotation via the Tablet PC’s stylus, and the new OneNote 2003 natively supported Digital Ink throughout. “With OneNote 2003, you can take notes in your normal handwriting, more closely emulating the way you take notes in a real notebook,” I wrote at the time. “You can also move around its virtual pages at will, scribble diagrams and drawings, and utilize all the pen types of colors to which you’re accustomed.”

Windows XP Tablet PC Edition 2005 was made available for free to all Tablet PC users via Windows XP Service Pack 2 (SP2), about which I’ll be writing more soon. But it was clear, even in 2003, that the Tablet PC had run its course and that this functionality would simply be rolled into more mainstream Windows versions soon, most likely in the Longhorn timeframe.

Looking back on this time, it is ironic and sad that all of Microsoft’s Tablet PC efforts amounted to little more than a footnote in personal technology history. But we can blame this on hubris. As Walter Isaacson explains in his official biography of Steve Jobs, a Microsoft engineer was among those who attended Jobs’ 50th birthday party in 2005, and the engineer wouldn’t stop bragging about the Tablet PC.

“This guy badgered me about how Microsoft was going to completely change the world with this Tablet PC software and eliminate all notebook computers, and Apple ought to license his Microsoft software,” Jobs told Isaacson. “But he was doing the device all wrong. It had a stylus. As soon as you have a stylus, you’re dead. This dinner was like the tenth time he talked to me about it, and I was so sick of it that I came home and said, ‘Fuck this, let’s show him what a tablet can really be’.”

And then Jobs went into the office the next day and told his team at Apple to create a tablet that “can’t have a keyboard or a stylus.” Instead, users would interact with it using the ten styluses built into their hands, their fingers. The resulting feature, called multi-touch, was not first invented by Apple. But it was likewise not a feature of Microsoft’s Tablet PC software, which had relied on digital pen, keyboard, and mouse interactions.

Apple would employ multi-touch first with the iPhone, in 2007. And then, most damaging to Microsoft’s tablet efforts, with the iPad, in 2010. That the iPad would go on to add both keyboard and stylus functionality is, of course, also ironic. Or perhaps just hypocritical. Regardless, the iPad was much more successful than Microsoft’s Tablet PC efforts ever were.