How do I efficiently compute the pairwise similarity scores among a very large set of n-grams on a huge corpus?

Isn’t simhash/minhash what you want?

svcs.cs.pdx.edu Git – simhash.git/summary

See also How do you find high-similarity pairs of sentences in a large corpus?

Answered 2016-01-07

[Originally posted on http://quora.com/How-do-I-efficiently-compute-the-pairwise-similarity-scores-among-a-very-large-set-of-n-grams-on-a-huge-corpus/answer/Nick-Nicholas-5]

Leave a Reply Cancel reply