Varada as a Data Platform

You can deploy Varada as a data platform on one or more dedicated clusters in your own Virtual Private Cloud (VPC). Either manually spin up a new Varada cluster and then connect it to the Varada Control Center for monitoring, or quickly spin up a new cluster from the Varada Control Center.

When you spin up your Varada cluster from the Varada Control Center, it is automatically connected to the Varada Control Center, which has full control over the cluster, and immediately begins monitoring workloads.

962962

Your Varada cluster leverages the Trino open-source distributed SQL query engine, which is designed to query large data sets distributed over one or more heterogeneous data sources. This means that you can run any SQL query without the need for modeling or optimizations.

Varada also includes out-of-the-box native support for all community-supported Trino SQL connectors to access a wide array of data sources, enabling you to use Varada to join data from your data lake with different data sources.

High Availability

Varada supports high availability (HA) as follows:

  • If the coordinator falls, all running queries are terminated, and the cluster automatically switches over to a new coordinator to accept new queries. You will need to explicitly retry any failed queries. If required, a query retry mechanism should be implemented to handle queries that fail as a result of a high availability event.

  • If any of the worker nodes fall, all running queries are terminated, and the index and cache on that node are lost. The cluster immediately begins accepting and executing new queries, and spins up a new worker node to auto-scale to the required cluster size.

📘

A worker node restart (not a node replacement) of any kind does not result in the loss of the index and cache on that node.