Marketers / Analytics / Data Cloud

Data Synchronization to Snowflake: The Complete Guide

By Ryan Goodman

Salesforce and Snowflake are two powerful centers of gravity when it comes to enterprise cloud platforms. Together, they create a robust suite of business applications and analytics. 

Salesforce offers native data integration with Snowflake designed to facilitate many descriptive and predictive analytics use cases. Without getting lost in the branding and product names, the goal for integrating Snowflake and Salesforce is to transform data quickly and easily into actionable information.

With several methods to connect Salesforce and Snowflake, navigating the various options available can be challenging. This comprehensive guide was designed to help you understand the different ways you can connect Snowflake to Salesforce, along with a few real-world use cases – helping you to make informed decisions for your organization.

What is Snowflake?

Snowflake is a cloud-native data warehouse that has rapidly grown in adoption. What makes Snowflake special is the way it separates compute, storage, and database administration as a service (more about Snowflake architecture here).

Snowflake runs natively on Google Cloud, Azure, and AWS, and is designed to operate as a fully-fledged data platform while supporting structured and unstructured data together.

If these technical details don’t mean much to you, what makes Snowflake popular for a Salesforce team is the speed and ease of setup. Setting up, connecting, and prototyping end-to-end connectivity between Salesforce and Snowflake can be accomplished in hours.

READ MORE: Snowflake: The Hottest Data Lake for Salesforce

Why Integrate Snowflake With Salesforce?

Historically, companies would extract data from Salesforce into a data warehouse or directly into Business Intelligence platforms to build reports, analytics, and predictive models. As the Salesforce ecosystem has grown and more business processes are being built on the platform, there is an increasing need for more data-intensive processes and insights. 

Snowflake provides a true plug-in-and-play solution to fill this void and doesn’t require specialized database and development operations. If you are vested in the Salesforce ecosystem for analytics, Snowflake is the perfect supercharged data platform to get more value out of your investments.

Snowflake to Salesforce Connector Options

Salesforce provides multiple Snowflake connectors for CRM Analytics (formerly known as Tableau CRM). Here are four different options you can use, depending on your business requirements…

1. Snowflake Connector

The Snowflake connector is designed to pull data from Snowflake and import it into Salesforce Data Manager. Copying data from Snowflake into Salesforce allows you to blend and harmonize your Snowflake and Salesforce data together. 

For example, you can build recipes, visualizations, or models using the Salesforce CRM Analytics and then embed that Snowflake powered information directly in Salesforce objects for business consumption.

How Does the Snowflake Connector Work?

Data pulled into Salesforce using this connector is pulled into the Salesforce Data Management data store and not into Salsforce objects like Leads and Accounts. This Salesforce Data Management store is designed for ‘slicing and dicing’ your data and then reporting it through tabular or visual formats like Salesforce CRM Dashboards. 

To pull data directly into your Salesforce objects, you will need to look at a 3rd party connector.

Example Snowflake Connector Use Case

In my role as head of data and analytics, we used the Snowflake connector to pull payment and balance data to build historical trends for individual accounts and validate balances. Because balances were calculated once per day, we could run the data sync process once per day and process the data with Salesforce using a Salesforce CRM Analytics recipe.

Recommendations

When using the Snowflake connector, you should focus on use cases where data changes on an infrequent basis. This will reduce your Snowflake utilization costs but more importantly ensure users are not using stale data to drive decisions.

Step by Step Configuration 

Estimated time to complete is roughly 20-30 minutes.

2. Snowflake Direct Connector

Unlike the standard Snowflake connector, the direct connector provides live access to data in Snowflake. This option provides real-time access to data to your end users where they work. The direct connection does not rely on scheduling data movement into Salesforce. 

In addition to live embedded analytics, Einstein Discovery can also utilize the direct connector for Snowflake to deliver features into predictive models, opening the door for features that do not flow through your Salesforce instance.

Example Use Case

We leveraged the Snowflake direct connector to deliver embedded fees history in Snowflake so sales and operations teams could operate without having to leave Salesforce. Because the data could change throughout the day, the direct connector displayed Snowflake data securely and accurately, without having to move any data. The Direct Connector was formatted in a visual way and placed directly onto the Accounts Lighting layout.

When using the Direct Connector you should consider the frequency of data updates and the implications for cost on your Snowflake instance. Because Snowflake is consumption based, you want to choose use cases that require up to the minute data where the return on investment is understood. We recommend preparing a view in Snowflake, and using that view to deliver data into Snowflake Direct Connector.

Step by Step Configuration 

Estimated time to complete is roughly 15 minutes.

3. Snowflake Output Connector

The Snowflake Output connector is designed to push data from Salesforce out to your Snowflake data warehouse on a scheduled basis. This provides a Salesforce native data pipeline to power your analytics use cases outside of Salesforce, including in the Tableau cloud. 

You can schedule a sync from Salesforce objects, or use a Salesforce recipe that transforms and blends data before pushing to Snowflake. This is a powerful solution for organizations that have made investments in Salesforce and Snowflake – even if you don’t want to use the full CRM Analytics suite or Einstein Discovery feature set. 

The output connector provides complete control over the fields you export and will automatically create the tables and fields for you. This eliminates the database administrative and data pipeline management tasks needed to get data from Salesforce into your Snowflake data lake / data warehouse.

Example Use Case

In our case, the Snowflake output connector was particularly useful for streamlining our Leads database which had millions of records. Third-party data pipeline and ETL tools required configuration and often took 10-20 minutes to stream data – however, the native Snowflake output connector reduced that time down to less than 2 minutes while putting all of the control into our Salesforce Admin team. From the data extracted from Salesforce into Snowflake, we were able to increase the frequency of our bulk export to hourly on millions of records.

One of the limitations of the output connector is an inability to automatically select all fields. This means that the output connector will require some change management when adding new fields that are required for your data team that uses Snowflake. 

Step by Step Configuration 

Estimated time to complete is roughly 20-30 minutes.

Notes on Snowflake Permissions

To ensure your data ends up in Snowflake, I found that the Snowflake Schema needs the following permissions for the role used within my Salesforce configuration:

  • OWNERSHIP
  • CREATE EXTERNAL TABLE
  • CREATE STAGE
  • CREATE TABLE
  • SELECT – FUTURE EXTERNAL TABLE
  • SELECT – FUTURE TABLE
  • TRUNCATE – FUTURE TABLE
  • UPDATE – FUTURE TABLE
  • USAGE
  • USAGE – FUTURE STAGE

4. Third-Party Connectors to Snowflake

Snowflake and Salesforce have a seemingly infinite number of data solutions for moving, transforming and analyzing data. This article may mention vendors but does not serve as an endorsement or recommendation. Your own team skills, business requirements, and technology investments should drive the right solution. 

Streaming Data Synchronization 

For the purpose of this guide, streaming data synchronization occurs when events happen in Salesforce (i.e create or update), or when other apps result in data streaming out of Salesforce objects. Mulesoft or Zapier are two popular examples of streaming data synchronization solutions.

Data Pipelines 

A data pipeline is intended to move data from Salesforce objects out to a third-party data store (file or database). Conversely data pipelines can move data from third-party sources directly into Salesforce objects. 

While the Data Output Connector is considered a native Salesforce pipeline, third-party data pipelines allow you to pull data directly into Salesforce objects. Data pipelines can be used for analytics and data lakes or move data into other third-party cloud applications.

ETL

There is some overlap between ETL and data pipelines, but the key differentiator is that ETL is designed mostly for data warehousing and analytics. It provides a lot of power and control over the data structure in between Salesforce and the application where you are moving the data.

Summary

Connecting Salesforce and Snowflake can be a powerful solution for organizations looking to streamline their data management and analytics capabilities. By following the step-by-step configurations provided, you can quickly build a proof of value and make informed decisions for your organization’s data needs. 

Whether you’re using the Snowflake connector, the direct connector, or the output connector, it’s important to consider doing a complete proof of value before purchasing the software licenses. Additionally, there are many third-party connectors available for moving, transforming, and analyzing data, so it’s important to choose the solution that aligns best with your business requirements and skills.

If you run into issues or need help figuring out the Salesforce data synchronization conundrum in your organization, feel free to reach out!

The Author

Ryan Goodman

Ryan has managed data and analytics at Reliant Funding for the last 4 years, and recently, he founded DataToolsPro.com as a free resource to share tools and techniques and tools that helped his team succeed.

Comments:

    Penny Cooper
    June 28, 2023 8:12 am
    Great article - thanks!
    Ryan Goodman
    July 17, 2023 2:28 am
    Thanks Penny!
    Ryan Goodman
    July 17, 2023 2:29 am
    Super excited to update this article for the next release of Salesforce Data Cloud which features a zero-clone copy. This is a game changer for "zero-etl", real-time access from Snowflake to Salesforce data. Stay tuned for more details!

Leave a Reply