Synthesizing queries for handwritten word image retrieval
José A. Rodriguez, Florent Perronnin
We propose a method to perform text searches on handwritten word image
databases when no ground-truth data is available to learn models or select
example queries. The approach proceeds by synthesizing multiple images of the
query string using different computer fonts. While this idea has been successfully
applied to printed documents in the past, its application to the handwritten
domain is not straightforward. Indeed, the domain mismatch between queries
(synthetic) and database images (handwritten) leads to poor accuracy.
Our solution is to represent the queries with robust features and use a model
that explicitly accounts for the domain mismatch. While the model is trained
using synthetic images, its generative process produces samples according to the
distribution of handwritten features. Furthermore, we propose an unsupervised
method to perform font selection which has a significant impact on accuracy.
Font selection is formulated as finding an optimal weighted mixture of fonts
that best approximates the distribution of handwritten low-level features. Experiments demonstrate that the proposed method is an effective way to perform
queries without using any human annotated example in any part of the process.
Will be published in the journal "Pattern Recognition"
2011-068preprint.pdf (293.12 kB)