4
4
4
Article
Talk
Read
Edit
View history
Tools
From Wikipedia, the free encyclopedia
(Redirected from Corpus of text)
In linguistics and natural language processing, a corpus (pl.: corpora) or text
corpus is a dataset, consisting of natively digital and older, digitalized,
language resources, either annotated or unannotated.
Annotated, they have been used in corpus linguistics for statistical hypothesis
testing, checking occurrences or validating linguistic rules within a specific
language territory.
Overview
A corpus may contain texts in a single language (monolingual corpus) or text data
in multiple languages (multilingual corpus).
In order to make the corpora more useful for doing linguistic research, they are
often subjected to a process known as annotation. An example of annotating a corpus
is part-of-speech tagging, or POS-tagging, in which information about each word's
part of speech (verb, noun, adjective, etc.) is added to the corpus in the form of
tags. Another example is indicating the lemma (base) form of each word. When the
language of the corpus is not a working language of the researchers who use it,
interlinear glossing is used to make the annotation bilingual.
Applications
Corpora are the main knowledge base in corpus linguistics. Other notable areas of
application include:
vte
Natural language processing
General terms
AI-completeBag-of-wordsn-gram BigramTrigramComputational linguisticsNatural-
language understandingStop wordsText processing
Text analysis
Argument miningCollocation extractionConcept miningCoreference resolutionDeep
linguistic processingDistant readingInformation extractionNamed-entity
recognitionOntology learningParsingPart-of-speech taggingSemantic analysisSemantic
role labelingSemantic decompositionSemantic similaritySentiment analysisSyntactic
parsingTerminology extractionText miningTextual entailmentTruecasingWord-sense
disambiguationWord-sense induction
Text segmentation
Compound-term processingLemmatisationLexical analysisText chunkingStemmingSentence
segmentationWord segmentation
Automatic summarization
Multi-document summarizationSentence extractionText simplification
Machine translation
Computer-assistedExample-basedRule-basedStatisticalTransfer-basedNeural
Distributional semantics models
BERTDocument-term matrixExplicit semantic analysisfastTextGloVeLanguage model
(large)Latent semantic analysisSeq2seqWord embeddingWord2vec
Language resources,
datasets and corpora
Types and
standards
Corpus linguisticsLexical resourceLinguistic Linked Open DataMachine-readable
dictionaryParallel textPropBankSemantic networkSimple Knowledge Organization
SystemSpeech corpusText corpusThesaurus (information retrieval)TreebankUniversal
Dependencies
Data
BabelNetBank of EnglishDBpediaFrameNetGoogle Ngram ViewerUBYWordNet
Automatic identification
and data capture
Speech recognitionSpeech segmentationSpeech synthesisNatural language
generationOptical character recognition
Topic model
Document classificationLatent Dirichlet allocationPachinko allocation
Computer-assisted
reviewing
Automated essay scoringConcordancerGrammar checkerPredictive textPronunciation
assessmentSpell checkerSyntax guessing
Natural language
user interface
ChatbotInteractive fictionQuestion answeringVirtual assistantVoice user interface
Related
HallucinationNatural Language ToolkitspaCy
Categories: Discourse analysisCorpus linguisticsComputational linguisticsWorks
based on multiple worksTest items