This library does not actually compute TF-IDF

[kensens]

See the description of TF-IDF on Wikipedia, and in particular the example: https://en.wikipedia.org/wiki/Tf%E2%80%93idf#Example_of_tf%E2%80%93idf.

In the code, doc_dict correctly computes the "TF" (term frequency) part. However, the "IDF" part (inverse document frequency, or the log of the inverse fraction of documents containing the word) is not computed. At first glance, corpus_dict seems close, but it actually stores the number of times a word appears across all documents, so it is not a fraction of documents at all. There is also no "log" in the code, and the normalization of the query seems foreign to me.

The README's examples also do not match a manual computation of TF-IDF values. For example, the score for "alpha" and document "foo" should be:

(number of times "alpha" appears in document "foo") / (number of words in document "foo") * log( (number of documents) / (number of documents containing "alpha") ) = 1/8 * log(3/2) = 0.02201.

Because "alpha", "bravo", and "charlie" appear the same number of times in the same documents, they all have identical TF-IDF scores. Thus, the score for the entire query would be 3 * 0.02201 = 0.06603, which is not close to the README's value of 0.6875.