Understanding Acceleration Instructions

An acceleration instruction defines the materialization type Varada will perform on a column in a table in the Varada Catalog‍ in order to warm up the data in the column. The actual warmup takes place when a query hits the column.

Instructions are created automatically by the query acceleration engine. You can also define acceleration instructions from the Varada Control Center, or using REST API commands.

Each instruction is associated with a list of partition values. Varada will only perform column materialization on data in the specified partitions.

Acceleration Instruction Attributes

An acceleration instruction includes the following attributes:

  1. The column to which the instruction applies.

  2. The table partitions to be materialized (optional).

  3. For each column in the instruction, the materialization type Varada will perform. The materialization type can be any combination of the following:

    • Index
    • Data
    • Lucene
    • Bloom High
    • Bloom Medium
    • Bloom Low
  4. The priority of the instruction. There are five possible priority levels:

    • Always (only users can define)
    • High (user or query acceleration engine can define)
    • Medium (user or query acceleration engine can define)
    • Low (user or query acceleration engine can define)
    • Never (only users can define)

    You can override a priority assigned by the query acceleration engine, but the query acceleration engine cannot override a priority you assign. Once the priorities are set, they behave according to their rating, regardless of whether they were assigned by a user or by the query acceleration engine.

  5. For each column in the instruction, the Time to Live (TTL). Only in cases where no available storage and when the TTL threshold is met without the data associated with the instruction being accessed, the associated column materialization is reverted.

👍

Acceleration Instruction Best Practice

To ensure a column is accelerated even if the storage capacity is exceeded, create an acceleration instruction for the column with the Priority set as Always. Similarly, to ensure a column is never accelerated, even if storage capacity is available, create an acceleration instruction for the column with the Priority set as Never.