Architecture
Built for Efficiency

Firebolt offers the versatility to handle both structured and semi-structured data efficiently. For structured data, it facilitates easy migration of both normalized and denormalized models, using JOIN accelerators and indexes to optimize data storage and retrieval. Primary indexes enhance data pruning and compression, while aggregating indexes pre-compute and automatically update aggregations at ingestion, simplifying analytics workloads.

For semi-structured data, Firebolt supports complex data types and provides a rich library of functions and Lambda expressions for data processing. It enables a range of processing methods, from raw text processing to flattening and indexing techniques, providing flexible options.

In Firebolt, data lifecycle starts right at ingestion, utilizing schema inference and parallel processing for rapid batch data onboarding. Data is then sorted, compressed, indexed, and stored efficiently as tablets. Similarly, trickle inserts, updates and deletes are handled efficiently to ensure that the data warehouse is always up to date.

Crucially, Firebolt enables ACID transactions, coordinating data management activities through its metadata service for strong, global consistency across its distributed system. This guarantees data integrity and consistency even in complex, multi-engine environments.

Firebolt's query processing service is tailor-made to offer low latency and scalable processing, using a resource-aware admission control layer that scales to manage high concurrency. The process begins with an optimizer that uses historical and cost-based methods to create efficient execution plans, considering data distribution and index availability and learning from past queries to enhance consistency.

The execution of these plans occurs on a distributed runtime system that employs multithreading and vectorized processing, optimizing operations through tiered caching, sub-graph reuse, and resource-aware scheduling. This runtime is designed for efficient memory use and leverages distributed shuffle techniques to maximize the use of cluster resources.

Pervasive security and governance starts with a framework designed around Organizations & Accounts, integrating Single Sign-On (SSO), Role-Based Access Control (RBAC), and Network Policies. This structure allows for precise control over user access and permissions, ensuring that only authorized personnel can access specific resources or data. Together, these features form a secure foundation for managing and safeguarding your data and resources.

Firebolt enhances observability by providing insights into workload patterns, consumption and billing, facilitating efficient resource management. Observability can be extended with Open Telemetry integration to existing data collection and analysis tools. This streamlined approach enables DevOps/Site Reliability Engineering (SRE) teams to effectively monitor and manage the environment.

Delivering insights or data products requires multiple personas to collaborate together such as data architect, data engineer, analytics engineer, developer and DevOps. Firebolt workspace is designed to foster collaboration and provide visibility into the data lifecycle. Security and governance, data modeling, data exploration, SQL development, consumption and performance management are all performed within dedicated areas.