The Desi Architect
Notes / Docker Sandboxes / Elasticsearch Near-Real-Time Search, Lucene Segments & Refresh
Docker Sandboxes · Tier 2: Data & Scale · Lab 35

Elasticsearch Near-Real-Time Search, Lucene Segments & Refresh

Load a small catalogue and search it. See the best match rank first, a typo still find the iPhone, a text field and a keyword field behave differently, a new document wait to become searchable, and a request for page 2,000 get refused, with the fix for each.

Time35 minutes
StackElasticsearch 8.19Ports 19200
MemoryAbout 1400 MB
Before thisLab 03HTTP Caching Headers & DNS Resolut
Download lab (.zip) View steps
Verified runWindows, Mac, LinuxBash and PowerShell
  1. 1PrepareWhat you will learn
  2. 2PredictGuess before you run
  3. 3RunStep by step
  4. 4ObserveWhat you should see
  5. 5Break itChange one thing
  6. 6Clean upLeave nothing running
On this page
  1. 00 / What You Will Learn
  2. 01 / Why It Matters
  3. 02 / What We Are Building
  4. 03 / Words You Need
  5. 04 / Predict First
  6. 05 / Step by Step
  7. 06 / What You Should See
  8. 07 / Look Inside
  9. 08 / Break It, Fix It
  10. 09 / In Real Projects
  11. 10 / Interview Corner
  12. 11 / Clean Up

00 /What You Will Learn

By the end you will be able to
  • Explain an inverted index and what BM25 ranks by.
  • Tell a text field from a keyword field.
  • Explain near real time: why a search does not see a write at once, and what refresh and wait_for do.
  • Explain why deep pagination is refused and use search_after.

Before you start

Finish Lab 03 (HTTP, Caching and DNS) first. You should know what a JSON request over HTTP looks like. Docker Desktop needs about 1 GB of free RAM for this lab, and Python 3 is needed.