Text Segmentation

Reliable automatic semantic annotation systems do not exist for many languages. Their creation depends in many respects on construction of gold standard corpora. In this paper we present a system for supporting the semi-automatic construction of such corpora. The ...

Current methods of coding recall, summarization, talk-aloud, and question-answering data are inherently unreliable and not effectively documented. If the process of coding protocol data could even be partially automated, this would be an im- portant scientific advance in the field of text comprehension. Twenty-four human subjects read and recalled each of four short texts. Half of the human recall data

In this paper, we describe a new unsupervised sentence boundary detection system and present a comparative study evaluating its performance against different systems found in the literature that have been used to perform the task of automatic text segmentation into sentences for English and Portuguese documents. The results achieved by this new approach were as good as those of the previous systems, especially considering that the method does not require any additional training resources.

CAPTCHA is a test that can tell humans and computer programs apart automatically. The aim is to allow the server to identify the visitor is a human or a computer, and only provide services to human. It can improve the current server system and user information security. The static plane visual CAPTCHA based on OCR problems with the advantages of implementation and operation[1] become the mainstream of the current CAPTCHA technology application form. However, with a variety of targeted text segmentation technologies merging, such CAPTCHA based on OCR problems is faced with increasing security threats. In this paper, a new CAPTCHA based on the moving object identification and tracking problems is proposed, which is referred to biological motion vision model. An Innovative Single-frame Zero-knowledge rule is also put forward to make the CAPTCHA generation algorithm based on Edge Mutation. An attacker can log on the test service system, only after he solves the moving object recognition problem. Such animation CAPTCHA will be able to resist the attacks of all the static OCR technology, and resist the mainstream of attacks against the moving object detection.

Translators play an important role in conveying ideas and thoughts from one language to another. A contextual based approach to translation of English words from images to the equivalent Hindi words is described in this paper. Text segmentation is an integral and critical process in text extraction which have the major control over the accuracy of any translator. The paper describes a study on text segmentation algorithm with Maximally Stable Extremal Regions (MSER) which is implemented as a preprocessing module of the translation system. Initially the system learns through user disambiguation of appropriate words in the sentence. However, once the system has learned enough contextual information, it performs the automatic translation. The experimental results of segmentation are compared and analyzed. Such a system has a number of uses, including elearning of the English language as well as for Indian tourists from overseas countries. It can be applied for learning any courses. The system can also be extended to include other Indian languages.

В статье обсуждаются критерии выделения неканонических подлежащих с признаками нулевых местоимений и обсуждается корреляция между сегментацией текста и параметрами грамматики, лицензирующими употребление разных видов нулевых подлежащих в финитных клаузах. Изучение нулевых подлежащих и параметра нулевого субъекта является областью, где применение реляционной типологии А. Е. Кибрика вполне оправдано, поскольку факторы, обуславливающие выбор конструкций с нулевым подлежащим в ряде случаев связаны с комбинированным применением формально-синтаксических, коммуникативно-синтаксических и референциально-дейктических механизмов. В синтаксически аккузативных языках, для которых был ранее предложен параметр нулевого подлежащего, имеются разные типы нулевых подлежащих, в том числе, ‘нули Мельчука’, т. е. нулевые подлежащие с ролевой семантикой, контролирующие согласовательную форму предиката. Встречаются языки, использующие в разных типах клауз более одного типа нулевых подлежащих с ролевой семантикой. Корреляция между семантикой и согласовательной морфологией нулевого подлежащего не является принципом общей грамматики, но зависит от сочетания морфосинтаксических параметров, сложившегося в конкретном языке. Прямолинейное отождествление ‘нулей Мельчука’ с малым pro непродуктивно, так как эти единицы имеют разные категориальные свойства и разную дистрибуцию. Вместе с тем, все типы нулевых подлежащих, по-видимому, связаны со стратегией на экономное использование синтаксического материала клаузы и на опущение информационно избыточных элементов дискурса/текста. При этом частое использование pro, т. е. нулевого тематического местоименного подлежащего в позиции повторной номинации, в ряде случаев создает благоприятные условия для воспроизводства конструкций с ‘нулями Мельчука’, имеющими ролевую семантику Агенса, в ситуациях, где уточнение референции Агенса нерелевантно для развертывания текста. Эта связь была продемонстрирована на примере разных языков, в том числе, на базе оригинальных текстов древнескандинавских языков и на базе текстов субнормативного русского языка.

In this paper, we exploit natural language processing techniques to build a system that automatically segments Hadith into its two main components, Isnad and Matn. We evaluate the previous attempts to segment Hadith and identified the limitations in these studies. Then a Hadith segmentation system is built and tested with Hadith collections extracted from six different Hadith books. The result demonstrates that bi-grams are effective in identifying Hadith segments with 92.5% accuracy.

Handwritten text line segmentation is an important task of Optical Character Recognition. The proposal discusses a novel technique for Segmentation of Lines for the handwritten text document written in Kannada Language. The algorithm imbibes the approach of finding the Components, bounding box and computing the coefficient of variance. Preprocessing steps such as correction of skewness due to improper scanning as well as removal of noise is assumed to have been performed on the data. Our technique of line segmentation addresses the key complexity issue of variations in the gaps between and across lines as well as curly and skew character text of handwritten document. The purpose of this paper is to segment line by connecting the centroids within the bounding box and employing the coefficient of variance. The method has been tested on unconstrained Kannada scripts. An experimental result obtained by executing the proposed method of line segmentation has achieved high degree of accuracy and high performance.

https://sites.google.com/site/ijcsis/

Recognition of dimensioning text in engineering drawings is an essential part of the drawing understanding process, as this text provides the exact dimensions and tolerances of the object described in the drawing. We consider engineering drawings produced according to either ISO or ANSI drafting standards. Text segmentation and recognition are preceded by orthogonal zig-zag vectorization, arc segmentation, and arrowhead pair recognition. Initial textbox extraction is done by a region growing process, performed on text-wire candidates. On the basis of textbox context (neighboring
annotation wires) the drafting standard is detected. Raw
textboxes are divided into logical textboxes, which are further decomposed into basic textboxes. A neural network based OCR algorithm is applied to each basic textbox. Finally, the OCR recognition results are verified by using contextual information and comparing the results with the measurements made directly on the drawing

Abstract This article reviews the role of relevance in text processing. It argues that relevance instructions provided by instructors and texts help readers identify text segments that are germane to a reading goal. A taxonomy of relevance instructions is presented and four basic types ...

In this paper the problems of deriving a taxonomy from a text and concept-oriented text segmentation are approached. Formal Concept Analysis (FCA) method is applied to solve both of these linguistic problems. The proposed segmentation method offers a conceptual view for text segmentation, using a context-driven clus-tering of sentences. The Concept-oriented Clustering Segmentation algorithm (COCS) is based on k-means linear clustering of the sentences. Experimental results obtained using COCS algorithm are presented. 1.

A reading-based CAPTCHA, called ‘ScatterType, ’ designed to resist character–segmentation attacks, is described. Its challenges are pseudorandomly synthesized images of text strings rendered in machine-print typefaces: within each image, characters are fragmented using horizontal and vertical cuts, and the fragments are scattered by vertical and horizontal displacements. This scattering is designed to defeat all methods known to us for automatic segmentation into characters. As in the BaffleText CAPTCHA, English-like but unspellable text-strings are used to defend against known-dictionary attacks. In contrast to the PessimalPrint and BaffleText CAPTCHAs (and others), no physics-based image degradations, occlusions, or extraneous patterns are employed. We report preliminary results from a human legibility trial with 57 volunteers that yielded 4275 CAPTCHA challenges and responses. ScatterType human legibility remains remarkably high even on extremely degraded cases. We speculate that...

An optical character recognition (OCR) system may be the solution to data entry problems for saving the printed document as a soft copy of them. Therefore, OCR systems are being developed for all languages, and Kurdish is no exception. Kurdish is one of the languages that present special challenges to OCR. The main challenge of Kurdish is that it is mostly cursive. Therefore, a segmentation process must be able to specify the beginning and end of the characters. This step is important for character recognition. This paper presents an algorithm for Kurdish character segmentation. The proposed algorithm uses the projection-based approach concepts to separate lines, words, and characters. The algorithm works through the vertical projection of a word and then identifies the splitting areas of the word characters. Then, a post-processing stage is used to handle the over-segmentation problems that occur in the initial segmentation stage. The proposed method is tested using a data set consisting of images of texts that vary in font size, type, and style of more than 63,000 characters. The experiments show that the proposed algorithm can segment Kurdish words with an average accuracy of 98.6%.

In this paper the Naive Bayes Classifier (NBC) is introduced for text segmentation. A set of training data is generated from a wide category of document images for learning the NBC. The images used for generating the training data include both machine-printed and handwritten text with different fonts, sizes, intensity values and background models. A small subset of the coefficients of a discrete cosine transformed image block is used to classify the block as text or non-text. The NBC decision threshold is optimized on a test set. Experiments carried out on unseen documents show promising results. A comparison with a well-established method for text segmentation indicates advantages of the proposed method.

Over the past several years, researchers have applied different methods of text segmentation. Text segmentation is defined as a method of splitting a document into smaller segments, assuming with its own relevant meaning. Those segments can be classified into the tag, word, sentence, topic, phrase and any information unit. Firstly, this study reviews the different types of text segmentation methods used in different types of documentation, and later discusses the various reasons for utilizing it in opinion mining. The main contribution of this study includes a summarisation of research papers from the past 10 years that applied text segmentation as their main approach in text analysing. Results show that word segmentation was successfully and widely used for processing different languages.

When presented with a retrieved document, users of a search engine are usually left with the task of pinning down the relevant information inside the document. Often this is done by a time-consuming combination of skimming, scrolling and Ctrl+F. In the setting of a digital library for scientific literature the issue is especially urgent when dealing with reference works, such as surveys and handbooks, as these typically contain long documents. Our aim is to develop methods for providing a “go-read-here” type of retrieval functionality, which points the user to a segment where she can best start reading to find out about her topic of interest. We examine multiple query-independent ways of segmenting texts into coherent chunks that can be returned in response to a query. Most (experienced) authors use paragraph breaks to indicate topic shifts, thus providing us with one way of segmenting documents. We compare this structural method with semantic text segmentation methods, both with respect to topical focus and relevancy. Our experimental evidence is based on manually segmented scientific documents and a set of queries against this corpus. Structural segmentation based on contiguous blocks of relevant paragraphs is shown to be a viable solution for our intended application of providing “go-read-here” functionality.

Reliable automatic semantic annotation systems do not exist for many languages. Their creation depends in many respects on construction of gold standard corpora. In this paper we present a system for supporting the semi-automatic construction of such corpora. The ...