Salesforce - To Push or to Pull

So you want to integrate with Salesforce! This usually means you have outside data you want in Salesforce, or Salesforce data you want out, or both. Interestingly, there are two ways to get stuff in and two ways to get stuff out, and some ways can be better than others. To get data out of Salesforce, for example, I can be in Salesforce and push it out, or be outside Salesforce and pull it out. Who'd have thought!  

Before we look at which is best, I've already dug a hole because I've not described the nature of the data we want to transact (that's a great word). In saying this I've hinted at the problem - potential volume, frequency, and concurrency constraints. Because Salesforce runs in a shared, multitenant environment, limits are required to ensure code and external systems don't monopolize shared resources. Actually, I made it sound trivial when, in fact, there is a seemingly limitless number of limits.

First, let's look at pushing data to an external system versus pulling data, essentially the same thing from a data perspective - getting the data out of Salesforce! Some of the push options are:

• Data Export with Data Loader - no development effort required
• Outbound messages - no development effort required
• Callouts - development effort in Salesforce required
• Use FuseIT SFDC Explorer

Now, consider the pull options, where the external system drags data out of Salesforce:

• REST and SOAP API's - development effort in Salesforce required
• Streaming API - development effort in Salesforce required
• Bulk API - development effort in Salesforce required

So which is best, in terms of data volume, frequency, concurrency, and reliability? Annoyingly, the answer depends on your needs. In enterprise situations, where everything is big, busy, and important, we have found pulling data from Salesforce using APIs is better. Rather than push the data out with a callout, we use the callout to send a simple request to the external service to pull data from Salesforce. By doing this we can shift larger volumes out of Salesforce.

It's not all roses and chocolates though. There are a few gotchas to pulling data out of Salesforce. For example, each call pulls only 2000 records from Salesforce so you need to keep checking if you have got all the records. You should be suspicious if you see a message saying "2000 records returned"! The other potential showstopper is that Salesforce counts the number of times you hit the API each day. It is a big number but if you exceed the allowed limit (which relates to the number of licenses) your API will be closed down for 24 hours.

The challenges associated with pushing data from Salesforce are generally more complex. For callouts, a 6MB call size is imposed by Salesforce which means large documents cannot be pushed. There are also callout frequency limits and other challenges that collectively make this approach more challenging in an enterprise situation. 

If you need to learn more, this excellent article by Patrick Connelly goes into more detail.