Update (July 13, 2014): I’ve been informed that I should be looking at hierarchical topic models (see Blei’s papers here and here). Thanks to Reddit users /u/GratefulTony and /u/EdwardRaff for bringing this to my attention. However, Redditor /u/NOTWorthless says HDPs do not provide a ‘posterior on the correct number of topics in any meaningful sense’. I’ll do more research and do a follow-up post.
Natural Language Processing is a field that covers computer understanding and manipulation of human language, and it’s ripe with possibilities for newsgathering. You usually hear about it in the context of analyzing large pools of legislation or other document sets, attempting to discover patterns or root out corruption. I decided to take it into the kitch
NLTK 2.2 - Conditional Frequency Distributions
The summer is over, and it's time to go back to school and continue our exploration of the NLTK book. We're finally getting at the last part of the second chapter - conditional frequency distributions. What is a conditional frequency distribution? From a statistical point of view, it is a function of 2 arguments - a condition and a concrete outcome - producing an integer result that is a frequency
Combining Scikit-Learn and NTLK
A collection of sloppy snippets for scientific computing and data visualization in Python. In Chapter 6 of the book Natural Language Processing with Python there is a nice example where is showed how to train and test a Naive Bayes classifier that can identify the dialogue act types of instant messages. Th classifier is trained on the NPS Chat Corpus which consists of over 10,000 posts from instan
One of the difficulties inherent in machine learning techniques is that the most accurate algorithms refuse to tell a story: we can discuss the confusion matrix, testing and training data, accuracy and the like, but it’s often hard to explain in simple terms what’s really going on. Practically speaking this isn’t a big issue from an engineering perspective, but in a general political sense it is-
“Natural Language Processing with Python” (read my review) has lots of motivating examples for natural language processing, focused on NLTK, which among other things, does a nice job of collecting NLP datasets and algorithms into one library. Let’s take one of Shakespeare’s sonnets and see if we can recommend alternate rhymes: import nltk from nltk.corpus import cmudict sonnet = ['O thou, my lovel
Using Python to see how the Times writes about men and women¶Neal Caren - University of North Carolina, Chapel Hill mail web twitter scholar Do men and women come up in different contexts in the newspaper? One quick way to answer that question is to compare the words in sentences that discuss women with the words in sentences that discuss men. Here's an example of how to do this sort of analysis u
Kneser-Ney Smoothing は高性能な言語モデルである。と、よく聞かされて知っているつもりだけど、まだ一度も試したことがなかったので、試してみた。 コードはここ。 https://github.com/shuyo/iir/blob/master/ngram/knsmooth.py 実験用にべったり書いているのでコピペは多いし、速度やメモリの効率も悪いが、まあ気にしないで。 コーパスは最初手元の適当なニュースコーパスを使っていたんだけど、それだと再現検証できないので、nltk のコーパスを使うように変更した。 nltk.corpus の下にあるコーパスモジュール名を -c オプションで与えると、そのコーパスを使って additive smoothing と Kneser-Ney smoothing の perplexity を出してくれる。 デフォルトでは Brown コーパス
Domena fjavieralba.com została zarejestrowana w serwisie AfterMarket.pl. Zainteresowany tą domeną? Jeśli chcesz skontaktować się z abonentem domeny fjavieralba.com, naciśnij poniższy przycisk aby wysłać wiadomość: Kontakt z abonentem domeny » Czym jest AfterMarket.pl? To: Jeden z największych rejestratorów domen w Polsce. Dla naszych 110000 klientów utrzymujemy ponad 410000 domen. — Zarejestruj sw
NLTK 2.0.1, a.k.a NLTK 2, was recently released, and what follows is my favorite changes, new features, and highlights from the ChangeLog. New ClassifiersThe SVMClassifier adds support vector machine classification thru SVMLight with PySVMLight. This is a much needed addition to the set of supported classification algorithms. But even more interesting… The SklearnClassifier provides a general inte
1
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
NLTK 2.1 - Working with Text Corpora
NLTK 2.3 - Working with Wordnet
