Share post
Share post
Share post
Blog: Product
Omnata as an alternative to Salesforce Data Cloud
Omnata as an alternative to Salesforce Data Cloud
Snowflake |
Analytics
By: Chris Chandler
12 July 2024
The TLDR
For customers where Snowflake is the centre of gravity and Salesforce is just another source/target, Salesforce Data Cloud is a big financial lift for a product architecture that delivers few benefits. If you are in this bucket and weighing up Salesforce Data Cloud primarily for the native integration features, in our opinion, it's not worth it.
On the other hand, if Salesforce is the centre of gravity with customer apps and experiences built across the product set, with Snowflake is just another source/target, it might be worth it. But you need to primarily get value from data-modelling and customer unification in Salesforce, not just integration features.
The key challenges of Data Cloud
If the native integration features to Snowflake are the primary attraction, Salesforce Data Cloud has some key challenges that make it a big lift for little gain:
Product core capabilities
What is the source of truth?
Pricing & costs
Integration architecture - 'real-time'?
Product core capabilities - CDP
Salesforce Data Cloud is, at its heart, a Customer Data Platform (CDP). As such, its internal data architecture is rigidly designed around the concept of unifying a single customer record and all of the features are built to achieve this objective. Data Cloud is also a separate data-store. It transforms data to create its own modelled view of a customer and it becomes a data silo.
On the other hand, Snowflake is also where many customers are naturally building their unified view of a customer. However, architected primarily as a utility data-platform, Snowflake can offer customer data unification as just one of many possible workloads.
Snowflake and Salesforce are actually in competition to own these workloads.
What do you want to be the source of truth?
Before we consider the technical and financial pros and cons, whether to use Salesforce Data Cloud as an ‘add-on’ for Snowflake is mostly philosophical. Both platforms claim that they can be the ‘source of truth’, but there can be only one source of truth. So, which bucket do you fall into?
Snowflake is the source of truth and Salesforce is just another source/target
OR
Salesforce is the source of truth and Snowflake is just another source/target
Many technology platforms seek to be the centre of gravity in the enterprise and what’s ‘correct’ for you depends on your vision for enterprise architecture.
OK, there is a third option…
You are from a large enterprise and there is a divide between technology ownership; there's the Salesforce people and the Snowflake people. In these cases, Snowflake and Data Cloud can be operated in tandem, but this goes along with the acceptance that there will never be a ‘single source of truth’ and each side will invest a high six-figure budget to do the same work. There's got to be a better way.
We've picked a side…
We think Snowflake should be the single source of truth
At Omnata, we have a strong opinion; Snowflake should be the single source of truth and everything else is a source/target.
Our native app, Omnata Sync for Snowflake is architected with this in mind. It directly replicates data to/from many sources and targets, including the Salesforce APIs. Compared to the leading SaaS ETL options, Omnata is unique in that it can both ingest Salesforce data into Snowflake for analytics and push modelled data out to Salesforce for activation.
With this in mind, let's unpack the challenges of Salesforce Data Cloud.
Pricing & costs
Salesforce Data Cloud starts at $100,000/year before any usage-billing charges. Then, reading through the add-on price list, Data Cloud pricing sheets and billing event help docs, I was honestly stumped...
The pricing is incredibly confusing but it appears to set customers on a path of ever increasing usage costs for the product’s core functionality. Every record processed incurs a cost in credits that multiplies depending on what happens to that record. Then, credits are purchased in fixed contracts.
My best bet at estimating the usage billing events needed to replicate data from Salesforce CRM to Snowflake is this:
Batch data pipeline - 2000 credits /million records
Profile unification - 100,000 credits /million records
Data share rows shared - 800 credits /million records
Now, this may or may not wind up being a reasonable cost… 🤷♂️
But I’ll guess that $100k/year + whatever the usage fees are leaves a fairly large window for an ISV to operate.
The alternative: Omnata’s native app pricing
Omnata Sync runs natively on Snowflake and is available self-service on the Snowflake Marketplace. Connectors are priced at a flat daily rate for each sync-direction in use, billed monthly by Snowflake.
If you have multiple production Salesforce orgs, you follow the same pricing structure for each org. For sandbox orgs, manual test syncs are free so you can validate data structures before moving to prod.
Overall, Omnata provides a fully predictable and easily calculated cost for your integration use cases that will knock zeroes off your integration bill, whether Data Cloud or another ETL tool. The total cost of Omnata is that much lower than standalone ETL tools because the native app architecture is so much simpler than standalone SaaS. You can read more about ETL tool costs, and see our pricing page or on our marketplace listings.
Integration Architecture
Salesforce Data Cloud creates an extra hop in data pipelines that are not actually real-time
Salesforce Data Cloud has a real-time, Zero-ETL integration to Snowflake. This is not a lie.
And, they use Iceberg tables to automatically land data in Snowflake at-scale, then use Snowflake data-sharing. It's all super cool sounding.
However, Salesforce Data Cloud does not have a real-time, Zero-ETL integrations to the rest of the Salesforce ecosystem products. As such, the true latency of the data-pipeline is in the replication jobs between Salesforce products.
The vast majority of customers want to pull and push data to/from Salesforce CRM, Marketing Cloud or Commerce Cloud endpoints. Each of these becomes a replication job to Data Cloud, before the real-time sharing step to Snowflake, and vice versa.
The true latency of sync jobs from Data Cloud to the rest of the stack is no better than any external ETL tool can achieve. The Salesforce docs detail that most objects are unremarkable Daily and Hourly syncs, with 10mins is the lowest latency for an incremental sync from Salesforce CRM. It's all overly complex for no gain.
Snowflake & Salesforce integration use cases with Omnata
When customer data unification and modelling is being done in Snowflake, the integration layer can just be a straight mapping of data. No transformation in-flight needs to take place. In these cases, the data-modelling features of Data Cloud are not valuable.
Omnata uses Snowflake's External Access features to sync data bi-directionally between Snowflake and Salesforce.
It never made sense to us that the leading SaaS tools in the ELT and Reverse-ETL categories only work in one direction. Omnata doesn't handle data or credentials so it's a far easier to get approved and implemented. You can schedule a sync as often as one-minute. However, the actual sync run time will depend on the API overheads and limits that everyone is subject to, even Salesforce themselves.
Use cases
There are two primary Snowflake<>Salesforce use cases that most customers are trying to solve:
Ingesting Salesforce CRM data into Snowflake for analytics
Pushing modelled data, insights and actions from Snowflake to Salesforce CRM
Inbound and outbound syncs with Salesforce Marketing Cloud is the other most common use case. As with all Salesforce products, it is another separate architecture and data-store. Omnata has a dedicated connector for Marketing Cloud which I’ll blog about in the coming weeks…
1. Ingesting Salesforce CRM data into Snowflake for analytics
Data Cloud’s architecture with Snowflake is built around data-sharing. However, it is key to understand that the data-sharing integration is specifically between Data Cloud and Snowflake, not Salesforce CRM, Marketing Cloud, or any other Salesforce products.
Omnata connects directly to the Salesforce REST and Bulk APIs. You can choose which objects you want to pull into Snowflake, the data fetch and storage behaviours, and finely tune the scheduling. So if you have a handful of high-priority objects, you can put these on the lowest latency and refresh everything else hourly or daily. Omnata automatically switches between the REST and Bulk API, depending on the size of the load to optimize performance.
2. Pushing modelled data, insights, and actions from Snowflake to Salesforce CRM
As above, Data Cloud has a direct share with Snowflake as a source. Once the Snowflake data is available in Data Cloud it can be used as an ‘Enrichment’ at the field level. This would be performed via another batch replication from Data Cloud to CRM objects.
On the other hand, Omnata enables you to push Snowflake tables, mapped to fields of CRM objects, directly via the Salesforce CRM API. Commonly referred to as Reverse-ETL, this makes sense where CRM data like a lead or a contact has a one-to-one mapping of external data, for example, a Lifetime Customer Value metric, Lead Score, Status or Segment.
Omnata Sync supports the standard replication strategies; create, upsert, mirror, delete.
For more advanced customer 360 use cases, Omnata supports actions like merging of records. This enables customers to action unification modelling built out in Snowflake and clean up duplicate records in Salesforce.
See Omnata in action
Conclusion
Wrapping up, we think that using Salesforce Data Cloud primarily as an integration layer between Snowflake and Salesforce is not worth the cost and does not deliver architectural benefits. If you already have lots of Salesforce products and experiences, and you are not setting out to centralize enterprise data in Snowflake, then it will probably do the job.
The TLDR
For customers where Snowflake is the centre of gravity and Salesforce is just another source/target, Salesforce Data Cloud is a big financial lift for a product architecture that delivers few benefits. If you are in this bucket and weighing up Salesforce Data Cloud primarily for the native integration features, in our opinion, it's not worth it.
On the other hand, if Salesforce is the centre of gravity with customer apps and experiences built across the product set, with Snowflake is just another source/target, it might be worth it. But you need to primarily get value from data-modelling and customer unification in Salesforce, not just integration features.
The key challenges of Data Cloud
If the native integration features to Snowflake are the primary attraction, Salesforce Data Cloud has some key challenges that make it a big lift for little gain:
Product core capabilities
What is the source of truth?
Pricing & costs
Integration architecture - 'real-time'?
Product core capabilities - CDP
Salesforce Data Cloud is, at its heart, a Customer Data Platform (CDP). As such, its internal data architecture is rigidly designed around the concept of unifying a single customer record and all of the features are built to achieve this objective. Data Cloud is also a separate data-store. It transforms data to create its own modelled view of a customer and it becomes a data silo.
On the other hand, Snowflake is also where many customers are naturally building their unified view of a customer. However, architected primarily as a utility data-platform, Snowflake can offer customer data unification as just one of many possible workloads.
Snowflake and Salesforce are actually in competition to own these workloads.
What do you want to be the source of truth?
Before we consider the technical and financial pros and cons, whether to use Salesforce Data Cloud as an ‘add-on’ for Snowflake is mostly philosophical. Both platforms claim that they can be the ‘source of truth’, but there can be only one source of truth. So, which bucket do you fall into?
Snowflake is the source of truth and Salesforce is just another source/target
OR
Salesforce is the source of truth and Snowflake is just another source/target
Many technology platforms seek to be the centre of gravity in the enterprise and what’s ‘correct’ for you depends on your vision for enterprise architecture.
OK, there is a third option…
You are from a large enterprise and there is a divide between technology ownership; there's the Salesforce people and the Snowflake people. In these cases, Snowflake and Data Cloud can be operated in tandem, but this goes along with the acceptance that there will never be a ‘single source of truth’ and each side will invest a high six-figure budget to do the same work. There's got to be a better way.
We've picked a side…
We think Snowflake should be the single source of truth
At Omnata, we have a strong opinion; Snowflake should be the single source of truth and everything else is a source/target.
Our native app, Omnata Sync for Snowflake is architected with this in mind. It directly replicates data to/from many sources and targets, including the Salesforce APIs. Compared to the leading SaaS ETL options, Omnata is unique in that it can both ingest Salesforce data into Snowflake for analytics and push modelled data out to Salesforce for activation.
With this in mind, let's unpack the challenges of Salesforce Data Cloud.
Pricing & costs
Salesforce Data Cloud starts at $100,000/year before any usage-billing charges. Then, reading through the add-on price list, Data Cloud pricing sheets and billing event help docs, I was honestly stumped...
The pricing is incredibly confusing but it appears to set customers on a path of ever increasing usage costs for the product’s core functionality. Every record processed incurs a cost in credits that multiplies depending on what happens to that record. Then, credits are purchased in fixed contracts.
My best bet at estimating the usage billing events needed to replicate data from Salesforce CRM to Snowflake is this:
Batch data pipeline - 2000 credits /million records
Profile unification - 100,000 credits /million records
Data share rows shared - 800 credits /million records
Now, this may or may not wind up being a reasonable cost… 🤷♂️
But I’ll guess that $100k/year + whatever the usage fees are leaves a fairly large window for an ISV to operate.
The alternative: Omnata’s native app pricing
Omnata Sync runs natively on Snowflake and is available self-service on the Snowflake Marketplace. Connectors are priced at a flat daily rate for each sync-direction in use, billed monthly by Snowflake.
If you have multiple production Salesforce orgs, you follow the same pricing structure for each org. For sandbox orgs, manual test syncs are free so you can validate data structures before moving to prod.
Overall, Omnata provides a fully predictable and easily calculated cost for your integration use cases that will knock zeroes off your integration bill, whether Data Cloud or another ETL tool. The total cost of Omnata is that much lower than standalone ETL tools because the native app architecture is so much simpler than standalone SaaS. You can read more about ETL tool costs, and see our pricing page or on our marketplace listings.
Integration Architecture
Salesforce Data Cloud creates an extra hop in data pipelines that are not actually real-time
Salesforce Data Cloud has a real-time, Zero-ETL integration to Snowflake. This is not a lie.
And, they use Iceberg tables to automatically land data in Snowflake at-scale, then use Snowflake data-sharing. It's all super cool sounding.
However, Salesforce Data Cloud does not have a real-time, Zero-ETL integrations to the rest of the Salesforce ecosystem products. As such, the true latency of the data-pipeline is in the replication jobs between Salesforce products.
The vast majority of customers want to pull and push data to/from Salesforce CRM, Marketing Cloud or Commerce Cloud endpoints. Each of these becomes a replication job to Data Cloud, before the real-time sharing step to Snowflake, and vice versa.
The true latency of sync jobs from Data Cloud to the rest of the stack is no better than any external ETL tool can achieve. The Salesforce docs detail that most objects are unremarkable Daily and Hourly syncs, with 10mins is the lowest latency for an incremental sync from Salesforce CRM. It's all overly complex for no gain.
Snowflake & Salesforce integration use cases with Omnata
When customer data unification and modelling is being done in Snowflake, the integration layer can just be a straight mapping of data. No transformation in-flight needs to take place. In these cases, the data-modelling features of Data Cloud are not valuable.
Omnata uses Snowflake's External Access features to sync data bi-directionally between Snowflake and Salesforce.
It never made sense to us that the leading SaaS tools in the ELT and Reverse-ETL categories only work in one direction. Omnata doesn't handle data or credentials so it's a far easier to get approved and implemented. You can schedule a sync as often as one-minute. However, the actual sync run time will depend on the API overheads and limits that everyone is subject to, even Salesforce themselves.
Use cases
There are two primary Snowflake<>Salesforce use cases that most customers are trying to solve:
Ingesting Salesforce CRM data into Snowflake for analytics
Pushing modelled data, insights and actions from Snowflake to Salesforce CRM
Inbound and outbound syncs with Salesforce Marketing Cloud is the other most common use case. As with all Salesforce products, it is another separate architecture and data-store. Omnata has a dedicated connector for Marketing Cloud which I’ll blog about in the coming weeks…
1. Ingesting Salesforce CRM data into Snowflake for analytics
Data Cloud’s architecture with Snowflake is built around data-sharing. However, it is key to understand that the data-sharing integration is specifically between Data Cloud and Snowflake, not Salesforce CRM, Marketing Cloud, or any other Salesforce products.
Omnata connects directly to the Salesforce REST and Bulk APIs. You can choose which objects you want to pull into Snowflake, the data fetch and storage behaviours, and finely tune the scheduling. So if you have a handful of high-priority objects, you can put these on the lowest latency and refresh everything else hourly or daily. Omnata automatically switches between the REST and Bulk API, depending on the size of the load to optimize performance.
2. Pushing modelled data, insights, and actions from Snowflake to Salesforce CRM
As above, Data Cloud has a direct share with Snowflake as a source. Once the Snowflake data is available in Data Cloud it can be used as an ‘Enrichment’ at the field level. This would be performed via another batch replication from Data Cloud to CRM objects.
On the other hand, Omnata enables you to push Snowflake tables, mapped to fields of CRM objects, directly via the Salesforce CRM API. Commonly referred to as Reverse-ETL, this makes sense where CRM data like a lead or a contact has a one-to-one mapping of external data, for example, a Lifetime Customer Value metric, Lead Score, Status or Segment.
Omnata Sync supports the standard replication strategies; create, upsert, mirror, delete.
For more advanced customer 360 use cases, Omnata supports actions like merging of records. This enables customers to action unification modelling built out in Snowflake and clean up duplicate records in Salesforce.
See Omnata in action
Conclusion
Wrapping up, we think that using Salesforce Data Cloud primarily as an integration layer between Snowflake and Salesforce is not worth the cost and does not deliver architectural benefits. If you already have lots of Salesforce products and experiences, and you are not setting out to centralize enterprise data in Snowflake, then it will probably do the job.
The TLDR
For customers where Snowflake is the centre of gravity and Salesforce is just another source/target, Salesforce Data Cloud is a big financial lift for a product architecture that delivers few benefits. If you are in this bucket and weighing up Salesforce Data Cloud primarily for the native integration features, in our opinion, it's not worth it.
On the other hand, if Salesforce is the centre of gravity with customer apps and experiences built across the product set, with Snowflake is just another source/target, it might be worth it. But you need to primarily get value from data-modelling and customer unification in Salesforce, not just integration features.
The key challenges of Data Cloud
If the native integration features to Snowflake are the primary attraction, Salesforce Data Cloud has some key challenges that make it a big lift for little gain:
Product core capabilities
What is the source of truth?
Pricing & costs
Integration architecture - 'real-time'?
Product core capabilities - CDP
Salesforce Data Cloud is, at its heart, a Customer Data Platform (CDP). As such, its internal data architecture is rigidly designed around the concept of unifying a single customer record and all of the features are built to achieve this objective. Data Cloud is also a separate data-store. It transforms data to create its own modelled view of a customer and it becomes a data silo.
On the other hand, Snowflake is also where many customers are naturally building their unified view of a customer. However, architected primarily as a utility data-platform, Snowflake can offer customer data unification as just one of many possible workloads.
Snowflake and Salesforce are actually in competition to own these workloads.
What do you want to be the source of truth?
Before we consider the technical and financial pros and cons, whether to use Salesforce Data Cloud as an ‘add-on’ for Snowflake is mostly philosophical. Both platforms claim that they can be the ‘source of truth’, but there can be only one source of truth. So, which bucket do you fall into?
Snowflake is the source of truth and Salesforce is just another source/target
OR
Salesforce is the source of truth and Snowflake is just another source/target
Many technology platforms seek to be the centre of gravity in the enterprise and what’s ‘correct’ for you depends on your vision for enterprise architecture.
OK, there is a third option…
You are from a large enterprise and there is a divide between technology ownership; there's the Salesforce people and the Snowflake people. In these cases, Snowflake and Data Cloud can be operated in tandem, but this goes along with the acceptance that there will never be a ‘single source of truth’ and each side will invest a high six-figure budget to do the same work. There's got to be a better way.
We've picked a side…
We think Snowflake should be the single source of truth
At Omnata, we have a strong opinion; Snowflake should be the single source of truth and everything else is a source/target.
Our native app, Omnata Sync for Snowflake is architected with this in mind. It directly replicates data to/from many sources and targets, including the Salesforce APIs. Compared to the leading SaaS ETL options, Omnata is unique in that it can both ingest Salesforce data into Snowflake for analytics and push modelled data out to Salesforce for activation.
With this in mind, let's unpack the challenges of Salesforce Data Cloud.
Pricing & costs
Salesforce Data Cloud starts at $100,000/year before any usage-billing charges. Then, reading through the add-on price list, Data Cloud pricing sheets and billing event help docs, I was honestly stumped...
The pricing is incredibly confusing but it appears to set customers on a path of ever increasing usage costs for the product’s core functionality. Every record processed incurs a cost in credits that multiplies depending on what happens to that record. Then, credits are purchased in fixed contracts.
My best bet at estimating the usage billing events needed to replicate data from Salesforce CRM to Snowflake is this:
Batch data pipeline - 2000 credits /million records
Profile unification - 100,000 credits /million records
Data share rows shared - 800 credits /million records
Now, this may or may not wind up being a reasonable cost… 🤷♂️
But I’ll guess that $100k/year + whatever the usage fees are leaves a fairly large window for an ISV to operate.
The alternative: Omnata’s native app pricing
Omnata Sync runs natively on Snowflake and is available self-service on the Snowflake Marketplace. Connectors are priced at a flat daily rate for each sync-direction in use, billed monthly by Snowflake.
If you have multiple production Salesforce orgs, you follow the same pricing structure for each org. For sandbox orgs, manual test syncs are free so you can validate data structures before moving to prod.
Overall, Omnata provides a fully predictable and easily calculated cost for your integration use cases that will knock zeroes off your integration bill, whether Data Cloud or another ETL tool. The total cost of Omnata is that much lower than standalone ETL tools because the native app architecture is so much simpler than standalone SaaS. You can read more about ETL tool costs, and see our pricing page or on our marketplace listings.
Integration Architecture
Salesforce Data Cloud creates an extra hop in data pipelines that are not actually real-time
Salesforce Data Cloud has a real-time, Zero-ETL integration to Snowflake. This is not a lie.
And, they use Iceberg tables to automatically land data in Snowflake at-scale, then use Snowflake data-sharing. It's all super cool sounding.
However, Salesforce Data Cloud does not have a real-time, Zero-ETL integrations to the rest of the Salesforce ecosystem products. As such, the true latency of the data-pipeline is in the replication jobs between Salesforce products.
The vast majority of customers want to pull and push data to/from Salesforce CRM, Marketing Cloud or Commerce Cloud endpoints. Each of these becomes a replication job to Data Cloud, before the real-time sharing step to Snowflake, and vice versa.
The true latency of sync jobs from Data Cloud to the rest of the stack is no better than any external ETL tool can achieve. The Salesforce docs detail that most objects are unremarkable Daily and Hourly syncs, with 10mins is the lowest latency for an incremental sync from Salesforce CRM. It's all overly complex for no gain.
Snowflake & Salesforce integration use cases with Omnata
When customer data unification and modelling is being done in Snowflake, the integration layer can just be a straight mapping of data. No transformation in-flight needs to take place. In these cases, the data-modelling features of Data Cloud are not valuable.
Omnata uses Snowflake's External Access features to sync data bi-directionally between Snowflake and Salesforce.
It never made sense to us that the leading SaaS tools in the ELT and Reverse-ETL categories only work in one direction. Omnata doesn't handle data or credentials so it's a far easier to get approved and implemented. You can schedule a sync as often as one-minute. However, the actual sync run time will depend on the API overheads and limits that everyone is subject to, even Salesforce themselves.
Use cases
There are two primary Snowflake<>Salesforce use cases that most customers are trying to solve:
Ingesting Salesforce CRM data into Snowflake for analytics
Pushing modelled data, insights and actions from Snowflake to Salesforce CRM
Inbound and outbound syncs with Salesforce Marketing Cloud is the other most common use case. As with all Salesforce products, it is another separate architecture and data-store. Omnata has a dedicated connector for Marketing Cloud which I’ll blog about in the coming weeks…
1. Ingesting Salesforce CRM data into Snowflake for analytics
Data Cloud’s architecture with Snowflake is built around data-sharing. However, it is key to understand that the data-sharing integration is specifically between Data Cloud and Snowflake, not Salesforce CRM, Marketing Cloud, or any other Salesforce products.
Omnata connects directly to the Salesforce REST and Bulk APIs. You can choose which objects you want to pull into Snowflake, the data fetch and storage behaviours, and finely tune the scheduling. So if you have a handful of high-priority objects, you can put these on the lowest latency and refresh everything else hourly or daily. Omnata automatically switches between the REST and Bulk API, depending on the size of the load to optimize performance.
2. Pushing modelled data, insights, and actions from Snowflake to Salesforce CRM
As above, Data Cloud has a direct share with Snowflake as a source. Once the Snowflake data is available in Data Cloud it can be used as an ‘Enrichment’ at the field level. This would be performed via another batch replication from Data Cloud to CRM objects.
On the other hand, Omnata enables you to push Snowflake tables, mapped to fields of CRM objects, directly via the Salesforce CRM API. Commonly referred to as Reverse-ETL, this makes sense where CRM data like a lead or a contact has a one-to-one mapping of external data, for example, a Lifetime Customer Value metric, Lead Score, Status or Segment.
Omnata Sync supports the standard replication strategies; create, upsert, mirror, delete.
For more advanced customer 360 use cases, Omnata supports actions like merging of records. This enables customers to action unification modelling built out in Snowflake and clean up duplicate records in Salesforce.
See Omnata in action
Conclusion
Wrapping up, we think that using Salesforce Data Cloud primarily as an integration layer between Snowflake and Salesforce is not worth the cost and does not deliver architectural benefits. If you already have lots of Salesforce products and experiences, and you are not setting out to centralize enterprise data in Snowflake, then it will probably do the job.
subscribe
Deliver the goods to your inbox
subscribe
Deliver the goods to your inbox
subscribe