A brief technical introduction to Anoto technology

This page is intended to provide a brief technical introduction to how the Anoto system works. It is intended for people with at least moderate technical and mathematics knowledge. I hope that it is clear; please let me know if there are any parts of this that are unclear or require further clarification.

I am working with Anoto technology as part of my research for the Integrated Systems Laboratory at Karlsruhe University. All of the code here has been released under the GNU GPL. It is reasearch code, not end-user code. It is intended to assist people wishing to understand this technology, and is not intended for any other purpose. It is minimally documented, and may have bugs. I am releasing it so that those reading this document may understand in more detail how this system works. If you want any more information, or have any comments, etc. then please feel free to contact me:

Important Disclaimer:

Various parts of the Anoto system are, presumably, covered by patents in various jurisdictions. I am not an attorney. I can not advise you on legal matters. All of the information contained in this document has been gathered from public sources. Your use of this information may require a patent liccense from Anoto. Please consult an attorney to determine if that is the case or not.

What is it?

A pen using Anoto technology is a pen that uses standard ink, but writes on paper with a special pattern of tiny dots that allow the pen to know where it is on the page, and to do various things, such as:

Save all the pages of your notebook, in electronic form, to your desktop computer so you can look at them there, back them up, etc.
Send a GIF version of a page you've written over Bluetooth or E-mail , via your phone, and then send it to somebody else
Allow companies that use custom paper forms to keep electronic copies of them without having to scan the original page

You can take a look at a raster version of a page, or the SVG vector XML version of the same thing.

You can get various pens with Anoto technology:

Ericsson Chatpen CHA-30 (Bluetooth only. Apparently discontinued, never really hit the mass market. Currently available from Sysnet, who also handle Penmail subscriptions)
Logitech io (USB only, two versions, Bluetooth version announced but does not yet seem to have shipped)
Nokia SU-1B (Bluetooth and USB, reasonably widely available in Europe)
Hitachi Maxell PenIt (Appears to only be available in the Japanese domestic market.)

Who Made It?

Anoto is the original designer of the system, and all the current products that use the system use a core engine designed by Anoto. There is little, if any, significant customization on the part of the pen vendors. While there are definitely differences between pens from the various vendors, the differences are primarily a question of whether they included a specific feature or not - they choose to drop features they think are too expensive or unnecessary, rather than innovating themselves.

So, really, what is it?

At its simplest level, the pen contains an optical sensor that analyzes the paper while you write. You must use special paper for this to work. The paper has a non-repeating dot pattern on it. This means that:

The pen knows where on the page it is the moment you touch it to the page
In most notebooks, every single page is unique - the pen will know that it is one centimeter from the left edge, and three centimeters from the top of page 34 of an "Esselte A5 Notebook for Anoto." Note that, of course, every single notebook Esselte produces has the same "page 34." If you have a pair of "Esselte A5 Notebook for Anoto" notebooks, the pen can't tell which is which.
This means you can write on any page in the notebook at any time, without having to worry.

When you touch the pen to the paper, it can do one of two things, based on where on the page you're writing:

Decide that you're writing a stroke of some kind (letter, diagram, etc.)
Notice that your pen is inside a "magic box" that does something special; the pen will vibrate for a fraction of a second to indicate that it knows you're in a magic box, and then will do whatever the box asks it to do.

Magic boxes can do various things:

Ask the pen to convert the page to a GIF, then send that GIF over Bluetooth to your cellphone
Change the color of the ink or the thickness of the line - of course, you'll only see this if you send a GIF over bluetooth, since the pen still only writes with a single color of ink on the page. There are 16 different colors and four different line thicknesses. Some notebooks have a page in the back with a collection of colors and line thicknesses, others do not. Some of the notebooks only have a subset of the possible colors. However, if the pen is writing with thick, purple ink, it will write like this on every page, in every notebook - you can tear out the color pallette from the back of a notebook, and use it with any other notebook. When you put the cap back on the pen, it reverts to standard color and thickness automatically.
Check the battery life of the pen: There's a line, from 0% to 100%. Touch the pen at 100%, and move it left towards 0%. It will vibrate when you're at its current battery life.
Troubleshoot your connectivity: A series of checkboxes that tell the pen to connect to your phone over Bluetooth, try out GPRS, and so on, to see why you can't send E-mail
A virtual keyboard, for sending E-mails. Touch the pen to the letter you want. There's no way to see the address, unfortunately.
New notebook: This tells the pen that you've just opened up a brand new notebook. The pen stores all your strokes in internal memory - this means that it knows everything I've ever written on page 5. When you check "New Notebook," it knows that next time you send a GIF of page 5, it should only send the strokes you've added. Note that it still stores them internally, and if your pen has a USB interface, you can download the strokes to your Windows computer.

Differences between the various pens

With few exceptions, all the Anoto pens are fundamentally the same. What I mean by that is, the only reason one pen won't do something another pen can do is that it doesn't have that hardware - some pens have Bluetooth, some pens have USB, one of them has both. If you check the "send GIF via bluetooth" box on a USB-only pen, it can't do that. Similarly, if you have a Bluetooth-only pen, and you have paper without a "send via Bluetooth" magic box, the pen will keep storing the strokes, waiting for something to do with them - but you have no way of telling the pen what to do with them.

What does the Magic Box do?

Most notebooks are a collection of pages, each of which has the same set of magic boxes on them. Magic boxes can do various things:

Send a GIF version of the page via Bluetooth to your phone. Some notebooks give you a choice of high or low resolution for the image.
Tell your Windows desktop software that you would like to send an e-mail. When you plug the pen in via USB, the Windows software will open up your e-mail program with a new message, containing a bitmap of that page, ready for you to send.
Send an e-mail to yourself, or to somebody else, via GPRS and your phone.
Change the color of the ink or the line thickness
Connect to your phone, and check whether the communications link works - some paper has a series of checkboxes, each of which tells the pen to check a different part of the link.
Pair a Bluetooth pen with your phone. Note that the pen is discoverable normally, but when you check this magic box, it searches for a phone and tries to pair.
My favorite checkbox, only present on a few notebooks: Establish a GPRS link via your phone, and for every page that has been modified in the notebook, send an e-mail with a vector version of the page in SVG format. There would be a single checkbox like this for the entire notebook, and it would send one e-mail for each modified page. The SVG file would contain only the new strokes, not all the existing strokes on the page. Very few notebooks have this magic box, because it was intended to be used with the Ericsson Chatpen CHA-30, which did not have USB. GPRS was the only way to get data out of the pen. The old version of the Anoto Windows software would log in to your POP3 e-mail account and download these individual page deltas, and merge them into your existing SVG file for that particular notebook.

How the dot pattern works

Every dot on an Anoto page is in one of four positions: up, down, left or right. The pen is capable of determining its absolute position from a 6x6 grid of these dots. The size of the patternspace is extremely large - literally larger than Europe and Asia combined. Each dot encodes an X and a Y bit - these two bits are independant.

The dots in a particular axis are what is known as a cyclic single-track gray code. There is a 52 bit long sequence. If you look at six bits in a row from this sequence, you know precisely where you are in the sequence - in other words, no 6 bit pattern appears more than once. This means that when you look at a 6x6 grid of Anoto dots, you don't need any special alignment markers to tell you where you are - there's no "this way up" marking.

What this means is that if you know where the sequence starts at one end, you can keep repeating it across the entire page. For example, at the upper left edge of the page, the first row might begin with bit 5. The second row might begin with bit 23 of the sequence. Since the sequence repeats across the entire page, I can just keep repeating it. The same property is true vertically as well.

The effect of this is that if you know the top and the left edge, you can regenerate an entire page perfectly. This is useful for analyzing how well the pen handles errors in dots, scaling, dot size variations, etc. Virtually all of the pre-printed Anoto pages have lines or grids of some kind on them; personally, I prefer different line and grid types than the pages on offer, so regenerating a page is useful.

What these tools do

NOTE: These tools are incomplete. They are not end-user tools. The source here demonstrates some concepts, and elaborates upon the text, rather than being a useful system to work with.

Dot Analysis

Analyze the dot pattern from a scanned page:

Take a series of scans, figure out where the dots on the page are, and produce a grid of chunks that represent the bits that the dots represent
Take this series of recognized, overlapping blocks, and stitch them together to form a single, large block

This two-stage tool consists of a C++ application that takes a PNG, and outputs a Python module containing the dots that were recognized on the page. Specifically, it looks at an MxM sized square, and analyzes what the X and Y positions are. It then move down D dots, and repeats. It does this in the X and Y axes. As long as there is some overlap somewhere, this contains enough information to re-create the page.

The next component of the tool is a python script that takes each of these blocks, and figures out where it belongs on the page. It places one block in the middle of the page, then goes through each block in sequence and checks where it fits, if it fits. It doing this over and over until it finds no more that fit. Then it goes to the next scanned image, and repeats itself.

The output of the second tool is a pair of arrays, one for the left, one for the top. These arrays represent the start coordinates for the appropriate row or column. The python code also looks at probabilities - specifically, every extra dot is used to increase the probability that a particular row or column starts at a certain point. This means that if you have errors, you can scan more sections and thus decrease the error rate. I've been working with a 2400x4800 dpi optical resolution scanner. I didn't have such good results with a 1200dpi scanner, but I believe that better algorithms and image processing would fix that.

Dot Regeneration

There's python code that takes a pair of arrays for the left and the top, and produces Postscript code that generates those dots. It actually creates a Postscript Type 3 font with four characters: an up dot, a left dot, a right dot and a down dot. Then, for each horizontal row, it draws a long string. This takes under a minute per page to print on many printers. It is much faster than the initial technique of drawing a circle for each point - the printer rasterizes each dot a single time when you select the font, rather than creating a bézier curve for every single dot on the page.

Stroke storage and rasterization

Once the pen sends your strokes out, you need to do something with them. This code will read your e-mail inbox over POP3, and download any Anoto attachments, and then do something with them. It will join strokes together into a single page - in other words, if you write, send the strokes, and repeat, i'll know that both sets of strokes belong to the same page, and will merge them together. It'll send the strokes to a piece of software as appropriate - the idea is that somebody can subscribe to a page and then get the strokes from that appage. A program can request a sequence of Anoto pages, use them for awhile, then give them back to the system so that another program can have them.

Download the Free Dot Toolkit from its Sourceforge project page