Articles
In the world of cloud data warehousing, it’s typical that compute cost accounts for the lion’s share of the overall operating cost. Getting that under control goes a long way towards achieving the promise of cloud, which is to save cost and only pay for what you use. Snowflake provides an example of a pricing model that supports this approach.
How are Snowflake warehouse costs calculated?
Snowflake’s pricing model is one that separates storage and compute costs, offering flexibility and control over cloud data warehousing expenses. The costs for virtual warehouses are measured in Snowflake Credits, which are consumed based on the size of the warehouse and the time it is running. Snowflake offers various sizes of warehouses, from X-Small to 4X-Large. Larger warehouses consume more credits per hour but provide more computing power. Compute costs are incurred only when the warehouse is active and billing is per-second, with a minimum of 60 seconds (one minute), meaning that, once a warehouse starts, you are billed for at least one minute of usage.
What is an ‘idle’ warehouse?
A warehouse is ‘idle’ when it’s running but not actively executing any queries or performing any data processing tasks. Essentially, it is a compute resource that is on standby, ready to execute tasks but currently unused. This is akin to leaving the lights on in an empty room—energy is being consumed without providing value.
The challenge is to balance the readiness of the warehouse to execute queries (which favors keeping the warehouse running) with the need to minimize costs (which favors shutting down the warehouse when it is not needed). This is where features like Snowflake’s Auto Suspend come into play.
Snowflake’s Auto Suspend setting
Snowflake offers many robust features to manage costs. A simple and often overlooked option is the “Auto Suspend” setting in which the CREATE WAREHOUSE command specifies the number of minutes a warehouse stays active after a query has been serviced. Let’s remember that a compute warehouse incurs cost as long as it is active, although the number of credits per hour varies based on the size of the warehouse and the number of clusters.
As of this writing, the default for Auto suspend is 5 minutes, which means charges will be incurred for 5 minutes after the last query completes running.
Usage patterns show that typical user warehouses don’t get used continuously. A user session is spread over a period that includes breaks for data analysis, query tweaks, and so forth. Factoring in five to ten breaks in a working day, we are looking at 20 to 40 minutes of unused compute time that is being billed per warehouse. This will add up quickly .
How can we use this option to our benefit?
For warehouses that do not need to run continuously, set the Auto Suspend to 1 minute. If the UI doesn’t allow it, use the command or, better yet, an IaC tool. The shutdown process takes a minute to complete so anything less than that will have no impact. Do not set it to 0 as it will cause the warehouse not to suspend.
What are the downsides of auto suspending a warehouse
If you’ve worked with other cloud data warehouse products, you might think this short suspend duration would cause a degradation in the user experience when the warehouse resumes and must warm up the cache or take time to bring up the cluster. With this Snowflake feature, the user rarely notices this resume and suspend cycle. Query performance in Snowflake is very consistent.