Nestor: putting neural nets to work

“What we are fighting against is forcing you to go back to kindergarten,” says David Fox, president of Nestor. “‘Too many errors? Go back and rewrite it!’ You don’t want a handwriting recognition system that teaches you to write, you want one that learns from you.” Fox maintains two criteria for handwriting recognition systems: the first is that they should offer walk-up accuracy and the second is that they should be adaptive. Fox: “What we call walk-accuracy means that a system shouldn’t make too many errors and shouldn’t require long training sessions. It should learn as you work. The beauty of the neural network technology on which Nestor’s system is based is that the more you use it the better it gets. One tenacious misconception that we are constantly confronted with is the myth that somehow neural networks saturate. This is simply untrue. The more data we offer them, the more accurate they become.”
    Nestor, a small company based in Providence, Rhode Island (USA), was established in 1975 by two Brown University professors, one of whom received a Nobel prize in physics, to develop and exploit algorithms based on models of biological processes. While Nestor has been quietly working on handwriting recognition and other applications of neural network technology for a number of years (it offered a 2000 character kanji recognizer as early as 1985), the company has not been able to exploit its full commercial potential. The recent explosion of interest, however, in pen-computers is offering Nestor a golden opportunity to introduce its software to a vast new market. In some ways, Nestor has been waiting for the hardware to catch up.
    Fox says that three standard input technologies for computers should evolve during the coming decade: keyboard, pen, and voice. Each will be suited to different activities. Nestor is striving to establish its handwriting recognition software as the “Dolby of computing,” as standard a feature of computers as Dolby noise reduction is for cassette decks.

A data-driven approach
Earlier this year, Nestor demonstrated its handwriting recognition software for Language Industry Monitor, running on a pre-production unit of the new NCR 3125 pen-computer. Initially, it was hard to judge just how accurate the recognition software was; you do not instinctively know how hard to press the stylus onto the screen, and unless the pressure is even, the system has difficulty knowing when one stroke stops and another begins. Within a few minutes, however, it began to pick up many if not most of the letters we “wrote” within the on-screen grid. NestorWriter is in principle able to handle variances in geographical penmanship; specific recognizers could also be developed for European users, for example. “It’s a data-driven approach,” says Fox. “The neural net can adapt to variations. This means it is implicitly superior to the rule-based approach of other recognition systems; those very quickly stumble on irregularities, such as overlapping areas or segmented characters.”
    When PenWindows was announced, Fox vigorously petitioned Microsoft to allow it to be extendable with recognizers from third parties, arguing that its ultimate success would be closely linked to the quality of the recognition software. Microsoft acquiesced and provided an api for Nestor and other developers. Nonetheless, Nestor feels it is in a rather vulnerable posi-tion, as its software will be heavily dependent on whatever support Microsoft chooses to offer. The api, for example, will only allow a single character to be passed to Pen Windows, not a word, meaning the recognizer will be completely dependent on the host system for word-hypothesization. Nestor also feels it burned its fingers by zealously demonstrating to Microsoft the merits of adaptability in response to Microsoft’s confident assertion that its recognition software would be so good it would not need to learn. The about-face was so rapid that Nestor vowed not to demonstrate its software to another living soul before PenWindows shipped. According to Fox, evaluation packages of NestorWriter running under PenWindows have now been shipped to select customers.
    PenWindows, however, is just one of the fronts being covered by Nestor. Poquet, the maker of “palmtop” computers which is owned by Fujitsu, started shipping a new model which includes Nestor’s own PenShell software running on top of DOS.
    Nestor has also developed a related product, NestorReader, for deciphering handwriting on scanned material, such as paper forms or faxes. While perhaps less exciting than a real-time recognizer, this batch processor has enormous potential as an input device for the document image processing systems many companies are now installing. Document Access, a young company in Rotterdam, is currently testing a modified OEM version of NestorReader with fifty-thousand pages of scanned material for a number of potential customers in the Dutch banking and insurance sectors. Document Access is confident that the system will prove accurate and robust enough for online forms processing in an “industrial” setting.
    Is Nestor also working on cursive recognition? Fox says his company is, but cannot be specific about progress. He notes, however, that the ideal recognizer should take both cursive and block handwriting in its stride, for that is the way people really write; a combination of both. Cursive recognition would probably require higher signal-processing bandwidth than block letters; it would most certainly require word-hypothesization, as easily fifteen percent of the letters in cursive handwriting can not be recognized in isolation, even by humans.

Nestor, Inc. One Richmond Square, Providence, RI 02906, USA Tel +1 401 331 9640, Fax +1 401 331 7319