Bag of Words and TF–IDF: How Computers Turn Text into Mathematics
This article explains Bag of Words and TF–IDF, two fundamental text representation methods. It covers how they work mathematically, their historical origins, and provides a complete implementation in pure PHP for document search.
The article describes how computers convert text into numerical representations, focusing on Bag of Words (BoW) and TF-IDF. BoW represents text as a vector of word counts, ignoring word order, while TF-IDF weighs words by their frequency within a document and rarity across a corpus. The author provides a historical context, noting that BoW originated in the 1950s-60s and TF-IDF was developed by Karen Spärck Jones in 1972, later popularized by Gerard Salton. Limitations include ignoring word order and semantics. The article includes a step-by-step implementation in pure PHP, demonstrating tokenization, term frequency calculation, inverse document frequency, TF-IDF vectors, and cosine similarity for document similarity search.
- Abbreviations
- BoW = Bag of Words — мешок слов
- TF = Term Frequency — частота термина
- IDF = Inverse Document Frequency — обратная частота документа
- NLP = Natural Language Processing — обработка естественного языка
- SMTP = Simple Mail Transfer Protocol — простой протокол передачи почты
Source: Хабр — Data Mining —
original