Elasticsearch from the Inverted Index Up

Elasticsearch tutorials love to start with PUT /index/_doc/1 and a search that works on the first try. Which is why, six weeks later, the same developers are in my DMs asking why "Türk Telekom" doesn't match "turk telekom", why relevance ordering looks drunk, and why changing one field type requires reindexing forty million documents. All three answers live in the fundamentals the tutorial skipped — the inverted index, analyzers, and mappings. Let's actually learn them.

The inverted index: why search is fast and updates are weird

A relational index maps row → values. An inverted index maps term → list of documents containing it, like the index at the back of a book. Searching for laravel queue means: look up the posting list for laravel, the list for queue, intersect. That's why full-text search over millions of documents answers in milliseconds — it's set intersection over pre-built lists, not scanning.

The same structure explains Elasticsearch's personality quirks. Documents aren't updated in place — segments are immutable; an "update" writes a new version and tombstones the old, and background merges compact things later. That's why heavy single-document update churn is Elasticsearch's least favorite workload, and why it's a search engine fed by your database, not a primary store. Repeat that last clause at parties; it prevents more architectural damage than any config setting.

Analyzers: the machine that decides what a "term" even is

Between your text and the index sits the analyzer pipeline: character filters → tokenizer → token filters. "Türk Telekom'un Fiber Kampanyası!" might become [turk, telekom, fiber, kampanya] — lowercased, ASCII-folded, apostrophe-split, maybe stemmed. Search only matches what analysis produced, and the query text goes through (usually) the same pipeline. Every "why doesn't X match Y" mystery is answered by one API call:

GET my_index/_analyze
{ "analyzer": "my_turkish", "text": "Türk Telekom'un kampanyası" }
// → the exact tokens in the index. No more guessing.

And a Turkish-aware analyzer, since half my readers need one:

PUT products
{
  "settings": {
    "analysis": {
      "analyzer": {
        "turkish_clean": {
          "tokenizer": "standard",
          "filter": ["apostrophe", "turkish_lowercase", "turkish_stemmer", "asciifolding"]
        }
      },
      "filter": {
        "turkish_lowercase": { "type": "lowercase", "language": "turkish" },
        "turkish_stemmer":  { "type": "stemmer", "language": "turkish" }
      }
    }
  }
}

(The dedicated turkish_lowercase matters because of the dotted/dotless i — the same İ/i issue that bites CSS uppercase bites search twice as hard.)

Mappings: schema-on-write wearing a schema-less costume

Elasticsearch will happily infer field types from your first document — and that inference is a trap, because mappings are permanent. The first document's "zip": "01234" becomes a number, and next week's "0A1 2B3" is a mapping conflict; fixing it means a new index and a full reindex. So: explicit mappings, always, and learn the two text types by heart:

• text — analyzed, for full-text matching. You cannot sensibly sort, aggregate or exact-match on it.
• keyword — stored verbatim, for filters, sorting, aggregations, exact match.

"title": {
  "type": "text",
  "analyzer": "turkish_clean",
  "fields": { "raw": { "type": "keyword" } }   // title for search, title.raw for sorting
}

That multi-field pattern — analyzed for search, keyword twin for everything else — is 80% of practical mapping design. The other 20%: dates with explicit formats, scaled_float for money, and resisting nested fields until you truly need per-object matching (they multiply document count under the hood).

Relevance: BM25 in one honest paragraph

Default scoring is BM25, and its intuition fits in a sentence: a document scores higher when the query terms are rare across the corpus (matching "kubernetes" means more than matching "the"), frequent within the document (with diminishing returns), and the document is short (five words containing your term beat a novel containing it once). Practical control comes not from tuning BM25's constants but from structure: query multiple fields with boosts (title^3, body), combine a match for recall with a match_phrase boost for precision, and add signals like recency via function_score. When ordering looks wrong, "explain": true shows the arithmetic per document — relevance debugging is reading that output, not shuffling boosts by vibes.

The mental model to leave with

Elasticsearch is three machines bolted together: an analysis machine that turns text into terms (you configure it per language and field), an inverted index that makes term lookup instant (and updates awkward), and a scoring machine that ranks by statistical surprise. Every production issue I've debugged maps to misunderstanding one of the three. Master them and the rest — the query DSL's hundred keywords, the cluster sizing, the application wiring — is vocabulary on top of grammar.

Next in the series: wiring it into a real Laravel app without the sync bugs. And if your search "works but feels dumb", that's a fixable feeling.