In this paper, we propose an effective approach for a PDA-based sign system, and it presents user the sign translator. Its main functions include 3 parts: detection, recognition and translation. Automatic detection and recognition of text in natural scenes is a prerequisite for automatic sign translator. In order to make the system robust for text detection in various natural scenes, the detection approach efficiently embeds multi-resolution, adaptive search in a hierarchical framework with different emphases at each layer. We also introduce an intensity-based OCR method to recognize character in various fonts and lighting condition, where we employ Gabor transform to obtain local features, and LDA for selection and classification of features. The recognition rate is 92.4% for the testing set got from the natural sign. Sign is different from the normal used sentence. It is brief, with a lot of abbreviations and place nouns. We here only briefly introduce a rule-based place name translation. We have integrated all these functions in a PDA, which can capture sign image, auto segment and recognize the Chinese sign, and translate it into English.

In this paper, we propose an approach for detecting signs from natural scenes. The approach efficiently embeds multiresolution, adaptive search, and affine rectification algorithms in a hierarchical framework, with different emphases at each layer. We combine multi-resolution and multi-scale edge detection techniques to effectively detect text in different sizes. Different from the existing approaches, by using the cues from text inside the image, we introduce affine rectification transformation to recover deformation of the text region caused by an inappropriate camera view angle. This procedure can significantly improve text detection rate and OCR (Optical Character Recognition) accuracy. Experimental results have demonstrated feasibility of the proposed algorithms. We have applied the proposed approach to a Chinese sign translation system, which can