Varada's provides index and cache resiliency that expedites index and data cache creation, as described in Index and Cache Resiliency. This feature is disabled by default, and can be enabled as follows:
- In the in varada.properties configuration file, for each of the cluster worker nodes, define the value of import-export.enabled as
You can override the import-export.enabled value with the import_export session parameter.
The config.store.path value is the path to the bucket in your data lake in which to save the shared indexes and data repository. You can override the config.store.path value with the import_export_s3_path session parameter.
Storing the index and the cache can fill up your S3 storage. You can limit and control the S3 footprint by setting a lifecycle policy.
Updated about 1 month ago