FAQ

Monitoring the status of your Firebolt engine using the REST API is a key step to ensure smooth operations. Firebolt provides a way to programmatically check engine status by querying the system engine. This article explains how to authenticate, retrieve the system engine URL, and query the engine status using Firebolt's REST API.To begin, ensure that you have an active service account with the necessary permissions. You will need the service account credentials to generate an access token for API authentication.

After obtaining an access token, use the following request to retrieve the system engine URL:

curl https://api.app.firebolt.io/web/v3/account/<account_name>/engineUrl \
-H 'Accept: application/json' \
-H 'Authorization: Bearer <access_token>'

Once you have the system engine URL, you can query it to check the engine's status with a simple SQL query, as shown below:

curl --location 'https://<system_engine_URL>/query' \
--header 'Authorization: Bearer <access_token>' \
--data "select status from information_schema.engines where engine_name = '<your_engine_name>'

This will return the current status of your engine, helping you monitor its activity and health.

For more information please refer to the Using the API Documentation.

Yes. The engine_running_querie and engine_query_history tables provide insights into current workloads. For more information see our Information Schema Documenation.

Use the engine_metering_history information schema view to track FBU consumption for each engine.

Firebolt provides three different observability views that provide insight into the performance of your engine.

1/ engine_running_queries - This view provides Information about currently running queries. This includes whether a query is running or in the queue. For queries that are currently running, this view also provides information on how long it has been running.

2/ engine_query_history - This view provides historical information about past queries - for each query in history, this includes the execution time of the query, amount of CPU and Memory consumed and amount of time the query spent in queue, among other details.

3/ engine_metrics_history - This view provides information about the utilization of CPU, RAM and Storage for each of the engine clusters. You can use these views to understand whether your engine resources are being utilized optimally, whether your query performance is meeting your needs,  what percentage of queries are waiting in the queue and for how long. Based on these insights, you can resize your engine accordingly.

For more information, please refer to the Sizing Engines article in our Documentation.

If the engine has the AUTO_START option set to True, an engine in a stopped state will be automatically started when it receives a query. By default, this option is set to True. If this option is set to False, you must explicitly start the engine using the START ENGINE command. For more information, please refer to the Work with Engines Using DDL article in the Firebolt Documentation.

You can use the AUTO_STOP feature available in Firebolt engines to make sure that your engines are automatically stopped after a certain amount of idle time. Engines in stopped state will not be charged, hence do not incur any costs. As with other engine operations, this can be done via SQL or the UI. For example, while creating an engine, you can specify the idle time, using AUTO_STOP, as below:

CREATE ENGINE IF NOT EXISTS MyEngine WITH
TYPE = “S” NODES = 2 CLUSTERS =1 AUTO_STOP = 15;

The above command will ensure that MyEngine will be automatically stopped if it has been idle for 15 minutes continuously. Alternatively, you can achieve the same after an engine has been created.

ALTER ENGINE MyEngine SET AUTO_STOP = 15;

For mor information, please see the Engine Consumption Documentation.

To start an engine:

sql
START ENGINE MyEngine;

To stop an engine:

vbnet
STOP ENGINE MyEngine;

For more information, please refer to the Work with Engines Using DDL article in the Firebolt Documentation.

Yes. By default, creating an engine would result in the creation of the underlying engine clusters and start the engine. This would enable the engine to be in a running state where it is ready to start serving the queries. However, you have the option to defer the creation of the underlying clusters for an engine by setting the property “INITIALLY STOPPED” to True while calling CREATE ENGINE. You can start the engine at a later point, when you are ready to start running queries on the engine. Note that you cannot modify this property after an engine has been created.

CREATE ENGINE IF NOT EXISTS MyEngine WITH

TYPE = “S” NODES = 2 CLUSTERS =1 START_IMMEDIATELY = FALSE;

Your queries will continue to run uninterrupted during a scaling operation. When you perform horizontal or vertical scaling operations on your engine, Firebolt adds additional compute resources per your new configuration. While new queries will be directed to the new resources, the old compute resources will finish executing any queries currently running, after which they will be removed from the engine.

For more information, check out our Engine Consumption Documentation.

No. Scaling operations in Firebolt are dynamic and do not require stopping the engine, so your applications will not experience downtime.

For more information, check out our Engine Fundamentals Documentation.

In Firebolt, you can scale an engine across multiple dimensions. All scaling operations in Firebolt are dynamic, meaning you do not need to stop your engines to scale them.

Scale Up/Down You can vertically scale an engine by using a different node type that best fits the needs of your workload.

Scaling Out/In You can horizontally scale an engine by modifying the number of nodes per cluster in the engine. Horizontal scaling can be used when your workload can benefit by distributing your queries across multiple nodes.

Concurrency Scaling Firebolt allows you to add or remove clusters in an engine. You can use concurrency scaling when your workload has to deal with a sudden spike in the number of users or number of queries. Note that you can scale along more than one dimension simultaneously. For example, the command below changes both the node type to “L” and the number of clusters to two.

ALTER ENGINE MyEngine SET TYPE = “L” CLUSTERS = 2;

All Scaling operations can be performed via SQL using the ALTER ENGINE statement or via UI. For more information on how to perform scaling operations in Firebolt, see the Guides section in documentation.

All operations in Firebolt can be performed via SQL or UI. To create an engine, you can use the “CREATE ENGINE” command (shown above), specifying a name for the engine, number of clusters the engine will use, number of nodes in each cluster and the type of the nodes used in the engine. After the engine is successfully created, users will get an endpoint that they can use to submit their queries. For example, you can create an engine named MyEngine with two clusters, each with two nodes of type “M” as below:

CREATE ENGINE IF NOT EXISTS MyEngine WITH TYPE = “M” NODES = 2 CLUSTERS = 2;

This creates an engine named "MyEngine" with two clusters, each containing two nodes of type "M". For more details, see the documentation.

The typical start-up time for a Firebolt engine is 10-15 seconds, but this is not guaranteed due to potential resource constraints on AWS.

For more information, check out our Sizing Engines Documentation.

No. While there is no theoretical limit on the number of databases you can use with a given engine, note that the configuration of your engine will determine the performance of your applications. Based on the performance demands of your applications and the needs of your business, you may want to create the appropriate number of engines.

For more information, check out our Engine Permissions Documentation.

No. Engines and databases are fully decoupled in Firebolt. A given engine can be used with multiple databases, and conversely, multiple engines can be used with a given database. On Firebolt, all engines can write to the same database. No need to segregate engines as read-write and read-only.

For more information, check out our Engine Permissions Documentation.

You can use up to 10 clusters per engine.

For more information, check out our documentation.

You can use anywhere from 1-128 nodes per cluster in a given engine.

For more information, check out our documentation.

There are four node types available in Firebolt: Small, Medium, Large, and X-Large. Each node type provides a certain amount of CPU, RAM, and SSD. These resources scale linearly with the node type. For example, an "M" type node provides twice as much CPU, RAM, and SSD as a "S" type node.

For more information, check out the Engine Fundamentals article in our documentation.

An engine has three key dimensions:

Type - This refers to the type of nodes used in an engine.

Cluster - A collection of nodes of the same type.

Nodes - The number of nodes in each cluster.

An engine comprises one or more clusters. Every cluster in the engine has the same type and the same number of nodes.

In Firebolt, an “engine” refers to a virtual compute resource that provides the processing power to execute queries, load data, and perform various SQL driven tasks. Unlike traditional cloud data warehouses, Firebolt engines can be resized, paused, and resumed in a much more granular, and cost effective way to optimize performance and cost.