An acceleration instruction defines the materialization type Varada will perform on a column in a table in the Varada Catalog in order to warm up the data in the column. The actual warmup takes place when a query hits the column.
Each instruction is associated with a list of partition values. Varada will only perform column materialization on data in the specified partitions.
An acceleration instruction includes the following attributes:
The column to which the instruction applies.
The table partitions to be materialized (optional).
For each column in the instruction, the materialization type Varada will perform. The materialization type can be any combination of the following:
- Bloom High
- Bloom Medium
- Bloom Low
The priority of the instruction. There are five possible priority levels:
- Always (only users can define)
- High (user or query acceleration engine can define)
- Medium (user or query acceleration engine can define)
- Low (user or query acceleration engine can define)
- Never (only users can define)
You can override a priority assigned by the query acceleration engine, but the query acceleration engine cannot override a priority you assign. Once the priorities are set, they behave according to their rating, regardless of whether they were assigned by a user or by the query acceleration engine.
For each column in the instruction, the Time to Live (TTL). When the TTL threshold is met without the data associated with the instruction being accessed, the associated column materialization is reverted.
When Varada has fulfilled all the prioritized instructions, if unused space remains, it will bring the data for any index-only columns, and add additional data and indexes for the next columns in line. These are the most popular columns based on recent queries.
Updated 3 months ago