Elasticsearch

Indices

Data in a index is partitioned acrossed shards to make storage more managable. An index may be too large to fit on a single disk, but shards are smaller and can be allocated across different nodes as needed

• Searches can be run across different shards in parallel speeding up query processing
• Each shard may have a number of replicas
• The replica shards process queries but do not index documents
• They are allocated at to a different node from the primary shard

Frozen indices

For very old indices that are rarely accessed, we should free up the memory that they use

Frozen indices | Elasticsearch Reference [7.9] | Elastic

• When an index is frozen, it becomes read-only, and its resources are no longer kept active.
• The tradeoff is that frozen indices are slower to search

Index lifecycle management

• Hot indices: Actively receiving data index and are frequently serving queries. Typical actions for this phase includes
  • Setting high priority for recovery
  • Specifying rollover policy to create a new index when the current one becomes too large, too old, or has too many documents.
• Warm indices are no longer having data indexed in them, but they still serve read request.
  • Setting medium priority for recovery.
  • Optimizing the indices by shrinking them, force-merging them, or setting them to read-only.
  • Allocating the indices to less performant hardware.
• Cold indices are rarely queried at all.