Build An Integration Strategy: Like A Salesforce Customer Success Architect (CSA)

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 26

Build an Integration Strategy

Like a Salesforce Customer Success Architect (CSA)


By: Richard Booth and Tom Leddy
Customer Success Architects, Salesforce.org
As Customer Success Architects, we work with nonprofit and
higher education organizations as they define their IT landscapes.
Introduction Determining how to keep data and processes in sync across multiple
systems is one of the biggest challenges most organizations face.

Accessing data from an outside system requires an integration, and


building integrations requires an integration strategy. Without a
defined strategy, integration projects can be time-consuming, costly,
and ineffective. This whitepaper will walk you through how to build
an integration strategy step-by-step so you have a clear vision of what
systems to integrate, when to integratee, and what the return on
investment for building your integrations will be.

Integration Overview 3

Building an Integration Strategy 6

Further Reading 24

About the Authors 25


3

Integration Regardless of the size of your organization, you probably use more
than one software application to run your day-to-day operations.

Overview
Even if you consider Salesforce to be your organization’s primary
application, you may still use other applications for things like event
management, payment processing, and online giving. This means
that you also have data stored in all of these systems. In some cases,
you may even have similar data stored in more than one application–
for example, a history of online donations by constituents or alumni
in both Salesforce and an online giving platform.

Sometimes having data in more than one system is necessary;


however, trouble arises when bits and pieces of your data are spread
out across multiple, disconnected applications. This prevents you
from a full, 360-degree view of your operations and leads to data
inconsistency that is expensive and time-consuming to rectify.
4
What Exactly is an Integration?
An integration is the process of linking together different computing systems and software applications physically
or functionally to act as a coordinated whole. There are both technical and business aspects to building an
integration.

• From the technical side, an architect or developer will need to think about how to physically connect the
systems and make sure that the correct data gets exchanged.
• From the business side, a business analyst and other stakeholders (and probably an architect as well) will
need to think about process related details like the type of data to be shared and frequency of sharing.

The Salesforce platform has a robust set of options for building integrations that should allow you to share
data and create end-to-end processes that span multiple systems in almost any scenario, regardless of your
oragnization’s size and the number or type of applications you’re trying to connect.

The table on the next page contains a high level overview of some of the different
integration approaches available within the Salesforce platform.

A partner or Customer Success Architect can work with you to


determine which approaches are best suited to your
requirements.
5

WAYS TO CREATE AN INTEGRATION


6
Having a well-defined, clear integration strategy ensures your
organization is aware of the systems that are integrated and the

Building an frequency at which they exchange data with each other.

Integration It may be tempting to think that you need to integrate all of your
applications and that you want all updates to your data shared in real

Strategy
time. But that isn’t always the smartest or most achievable goal.

For example, if you have a small handful of records in one system


that need to be updated once a year with data from another system,
is an integration into your CRM worth the time and money to build?
Perhaps the better and more cost effective option is for someone to
manually update those records annually.

Cost and the return on your investment (or ROI) are usually the
biggest limiting factors when it comes to deciding which applications
to integrate, but there are others to consider as well. Security,
compliance, and projected data growth all play a role. There’s no right
or wrong answer-it’s based on the needs of your organization.

The best way to make integration decisions is to start by looking


at your IT landscape holistically and developing an organizational
integration strategy. In the next few sections of this document, we’re
going to walk through a step-by-step process. At the end, you’ll have a
worksheet that will help you identify and prioritize your organization’s
integration needs.
7
Step 1: Know Your Systems and Data
The first step is to compile a list of all data elements that are currently shared between multiple applications or will
need to be shared in the future. We’re not thinking about how to technically build any of these integrations yet–
we’re simply identifying them. Start by creating a spreadsheet with these five columns:

1 Data Element: The information that’s being shared between systems. You can keep this high level for now.
Eventually, we’ll get down into granular details like filters and field mappings. At this point, using general terms
like “Contacts” or “Donations” is fine.

2 Source System/System of Record: The source system is where data originates. The system of record is the
authoritative source for this data (i.e. if the same data exists in multiple systems but doesn’t match, which
one would you consider to be “correct”?). The source system and system of record are often the same, but
they don’t have to be. If you have any that are different, list them both and identify which is which.

3 Target System: The system the data gets copied to (either manually or via an automated process).

4 New Records Per Month: On average, the amount of new records that get created each month. This doesn’t
have to be exact, but try to get your estimate as close as you can. If your organization operates in cycles where
some months have a significantly higher number of records created than others (i.e. increased numbers of
donations at the end of a quarter), use the higher end for your estimates.

5 Changed Records Per Month: On average, the amount of records that get modified each month. This can
be any type of update ranging from a simple change to a field value to a wholesale record and relationship
change. Also note that this number might be 0. Some transactional records like processed payments are
rarely updated unless they need to be corrected because of an error in the original entry.
8
EXAMPLE DATA ELEMENT TABLE
9
Best Practices For Building Your Data Elements Table

• You may have more than one line for the same data element. Notice that in the table above, donors and
donations have two lines each with different source and target systems. This is how to represent data
that needs to be transferred to more than one application. In this example, donors and donations get
transferred from an online giving platform to Salesforce and then to an accounting system.

• The same application can be a source system for some data elements and a target system for others. It all
depends on where your data is being entered initially and where it needs to go afterwards.

• Some of your integrations might be bi-directional, which means that you need to be able to add or update
records in either system and have the changes flow into the other one. If you have any scenarios like this,
you should create two separate line items with the source and target systems reversed.

• You might also have a scenario where you have more than one source system for the same data element.
For example, let’s say you have mailing list sign-up form on your website and an events registration app
that people use to register for your events. The data from these sources are all stored their own databases,
which means that each one of those applications would be considered a source system. If combining all
of that data together into a central location would make it easier for you to keep track of your constituents
and report on your organization’s activities, then create separate line items for each source system.

• It’s not uncommon for fundraising and event management systems to be owned by third parties, so you
may also have occasions where you’ll want to share data between one of your internal systems and a
system that’s owned by someone else. While there are some additional considerations for systems like this
that we will discuss later in this document, an integration is usually still possible, so go ahead and list them.
10

If you’re having trouble thinking of systems that might need to share data, refer to the list of examples below. These
are commons systems that nonprofits and higher education institutions typically integrate:

COMMON NONPROFIT AND HIGHER EDUCATION SYSTEMS

• Student Information Systems


• Event Management Systems
• Learning Management Systems
• Online Giving Platforms
• Point of Sale and Payment Processing Systems
• Accounting Systems
• Content Management Systems
• Marketing Automation Systems
• External Databases
• Data Warehousing and Business Intelligence Systems
• Other Salesforce Orgs

At this point it’s also worth looking for existing integration technologies
or platforms. You never know what might already be available for you to
use. On-premise, hosted externally, or iPaaS (integration platform as a
service), you might find ETL or API focussed technologies ready to go.
These might save you time and money through re-use.
11
Step 2: Determine Your Update Frequencies
Once you’ve identified your data elements, the next step is to think about how often the data needs to be updated
in your target systems. Just like in the last step, we aren’t going to talk about technical details yet. All we’re going
to do right now is add an extra column to our spreadsheet and think about timing - how often your target system
needs to be updated and when the updates should occur.
12
As a note, it might be tempting to put “real time” in this field for every integration. As soon as data is created or
updated in one system, you want that information to be immediately available in every other system, right?

Maybe, or maybe not. Real time integrations are nice because they ensure that data is instantly available in a
target system whenever it’s updated in a source system, but they’re also more complex to build and more costly
to maintain. There may be some scenarios where you really do need real time integrations to ensure that your
business processes flow smoothly, so be honest: if your data is going to a reporting system and you only look at
your reports once a week, does that system really need to be updated in real time? Probably not. If you’re ok with
grouping updates together and sending them to a target system all at once as part of a batch process, then specify
that along with the frequency that you’d like these updates to occur (hourly, daily, weekly, etc.).
13
Step 3: Identify Your System Owners
Once you’ve identified your data elements, systems and timing, the next thing to think about is ownership. This
is important for long-term maintenance. If an integration fails, the failure can be caused by an issue in the source
system, in the target system or somewhere within the integration itself. Troubleshooting may require a coordinated
effort between multiple groups.

Remember those third party integrations we talked about in Step 1? Here’s where we’ll list that information as well.
Building and maintaining integrations with third party systems often requires coordination with whoever owns the
system you’re integrating with. For this step, we’ll add three additional fields to our spreadsheet:

• Source System Owner and Target System Owner: This may be an individual, a department or a third party. Your
answer will most likely depend on the size of your organization. Larger organizations with big IT departments
sometimes have entire teams dedicated to supporting individual applications, whereas smaller organizations are
more likely to have a small group of individuals that are responsible for supporting most of an organization’s IT
systems. A couple additional tips regarding system ownership:

• If you have a large organization, your applications might have both a business owner and an IT owner. The
business owner is generally a non-technical person who understands how the system is used as part of your
organization’s business processes, whereas the IT owner is an administrator who understands the technical
details around how the system is configured. If this is the case for your organization, list both owners.
• All of your system owners should be full time permanent employees, not partners or contractors.

• Technical Requirements: Does your data need to be encrypted while in transit? If you’re integrating on-premise
applications with Salesforce, do you need to change any firewall settings? Enter answers to these and related
questions. Another thing to consider is user access. If a particular group of users isn’t authorized to access data
in the source system, appropriate precautions should be put in place to ensure that they can’t view the data
while it’s being transferred (especially in cases of file-based integrations) or in the target system.
14
While filling out this information, an architect should also be thinking about a holistic approach to building the
integrations in the list. Some things to consider during this step are: Does it make sense to purchase an ETL tool
instead of building a series of point to point integrations? Will integrations involving larger data volumes be spaced
out enough so as not to cause performance issues? What types of skills will be needed to build and maintain these
integrations? Do you already have employees who have these skills or will you need to hire additional staff (or use a
third party to augment your current staff)?
15
Step 5: Calculate ROI
The last column we’re going to add to our spreadsheet requires a bit of math. Part of deciding whether or not
to build an integration is to ensure that you receive a return on your investment to develop and maintain the
integration. Use the steps below to figure this out.

Determine Manual Maintenance Costs

1 Multiply the average amount of time that it takes your employees to create a new record by the number of
records that are created each month from your worksheet. Example: 200 new donors per month x 5 minutes
to create a donor record = 1000 minutes or about 16.7 hours per month to create new donor records.

2 Multiply the amount of time it takes for your employees to update a record by the number of records that are
updated per month from your worksheet. Example: 100 updated donor records per month x 5 minutes to
update a record = 500 minutes or about 8.3 hours per month to update existing donor records.

3 Add those two numbers together and that’s how many hours are spent per month maintaining that particular
piece of data manually. Example: 16.7 + 8.3 = 25 total hours per month spent on manual maintenance of
donor records.

4 Multiply the number of hours you calculated by the average hourly rate for the employees who are responsible
for entering the data and that will give you a rough estimate of how much it will
cost per month to not build an integration. Example: 25 x $15 = $375
per month to maintain this data manually.
16
Determine One Time Build Costs
Next, we’ll calculate the cost to build the integration initially. This is a one time cost and later we’ll determine how
long it will take to recoup the money you spent on it.

1 First calculate the cost of any software you may need to purchase (i.e. middleware tools, paid connectors,
etc.). An important note is that you should only include this in your calculation if you’re purchasing the
software specifically for the integration you’re doing the calculation for. If you already own it or were planning
on purchasing it for some other reason, it shouldn’t be included in this calculation. Example: For this
example, we’ll assume you have the software you need and don’t have to make any purchases.

2 Next, work with a developer or middleware expert to put together an estimate for the number of hours it
will take to connect the two applications and do any necessary data mapping. Multiply the estimate by that
person’s hourly rate. Example: 20 hours x $50 per hour = $1000.00

3 Add those two numbers together. This will be your one time setup fee. Example: $1000.00 + 0 = $1000.00
one time fee to build this integration.
17
Determine Ongoing Maintenance Costs
The next thing we’re going to do is figure out how much it will cost to maintain your information on an ongoing
basis. These costs include things like recurring software license fees and costs related to the time it will take to
identify and correct data transfer errors which will inevitably occur (though hopefully infrequently).

1 First calculate any software maintenance costs (i.e. monthly subscription fees, etc.). Just like in the previous
section, you should only include this in your calculation if you’re purchasing the software specifically for the
integration you’re doing the calculation for. If you already own it or were already planning on purchasing it for
some other reason, there’s no need to include it in this calculation. Example: For this example, we’ll assume
you have the software you need and don’t have to make any purchases.

2 Next, work with a developer or middleware expert to put together an estimate for the number of hours that
will be required per month to troubleshoot and fix any potential errors that may occur. This should be a low
number and will most likely be a rough estimate based on the complexity of the interface. Example: 5 hours
per month at $50 per hour = $250.00 per month.

3 Add those two numbers together. Example: $250.00 + 0 = $250.00 per month of ongoing maintenance
costs for this integration.
18
Final ROI Calculations
Now we’ll calculate two things: the amount of money we’ll save (or won’t save) each month if we decide to build a
particular integration and how many months it will take to recover the initial build costs.

1 Subtract the ongoing maintenance costs you just calculated from the manual maintenance costs you
calculated earlier. This is your monthly ROI for this integration. Example: $375.00 - $ 250.00 = $125.00

2 If the result is negative, you can stop here. This means that it’s not cost effective build this integration and
unless you have some other compelling reasons to build it (i.e. you’re anticipating a large increase in the
number of records that will need to be processed over the next couple years), you should probably hold off
on it. Example: Our result is positive so we’ll continue to the final step.

3 If the result is positive, then divide the one time build costs you calculated earlier by your monthly ROI to
determine how many months it will take you to recover your one time setup costs. Example: 1000 / 125 = 8
months to recover your one time costs.

So in this example, we’ll save $125 per month by building an integration and it will take about 8 months to recover
the money we spent to build the integration in the first place. If we also consider the fact that once the integration
is built, our employees will be able to do something else with the time they were spending on data maintenance,
we can safely say that at least from a financial perspective, it makes sense to build this integration.
19

The last thing we’ll do for this step is add ROI and Recovery information to each integration in our spreadsheet.
Notice that not all integrations have a positive ROI.
20
Step 6: Organize and Prioritize
In our last step we calculated the ROI for each of our investments. Now, we’re going to sort our existing rows by
each integration’s ROI, with the highest at the top and the lowest at the bottom.

Now we know which of our planned integrations will give us the biggest bang for our buck. That means we’re all
finished, right? Not quite….

ROI calculations are a great tool to help determine which integrations make the most sense to build from a
financial perspective, but like we mentioned earlier, things are a little bit more complicated than that. From a
purely financial perspective, it makes sense to build the integrations with the highest ROI first. But from a business
process perspective, it might not.

This is where someone with a thorough understanding of your business processes should take a deeper look and
make sure that the order you’re planning on building your integrations in makes sense, and that those integrations
are being built at the right time to support your organization’s change initiatives.

There might also be other reasons to rearrange the order in which integrations get built, such as grouping related
integrations together so that they can be built as part of the same project. Technical debt is also something to think
about. If you’re planning to retire one of the systems you’ve identified as a source or target system, it might not
make sense to integrate it with other applications unless you have a business critical need.

Remember that this worksheet is just to help you with your overall integration strategy and you don’t have to build
all of these integrations at once. If you have an upcoming project that’s relevant to some of the integrations in your
list, it might make sense to include them in that project’s scope. Or you may want to plan a project specifically to
build your integrations. Again, the final decisions around how to approach this will be up to you, but this prioritized
list should help.
21
Step 7: Detailed Interface Design
After your integration strategy is complete and you’ve identified the integrations you want to build first, the next
step will be to document the additional details required to build each one. We won’t be adding these details
to our worksheet. The deliverables for this step include business process flows, data flow diagrams, functional
specifications and technical specifications. This step will also most likely require input from different resources,
including Business Analysts and Developers/Architects.

Business Analysts Developers/Architects


• Document the business process that includes the • Determine the best integration pattern. As a note,
integration (including any related integrations and you can use this guide as a starting point.
dependencies). • Determine whether the interface should fully refresh
• Specify the timing for the integration and any all of the data (full refresh) in the target application or
triggers (i.e. transfer the data as soon as a new should only include deltas (changed records).
record is saved). • Document connectivity information and identify any
• Help to identify the specific fields that need to be relevant middleware / ETL tools.
included from each record. • Document any field mappings and required
• Identify any filters that need to be put in place to transformations.
limit the types of records that get sent from one • Document any required configuration changes or
system to another. customizations in the source and target systems.
• Document any detailed security requirements.
• Document any required firewall rule changes.
• Determine what monitoring tools will be used to
check for integration errors.
22
Step 8: Build and Test
Once your detailed design work is complete, the next step will be for a developer or middleware expert to build
the integrations based on the specifications that were created in the previous step. The type of resources that are
required for this step will vary based on the types of integrations being built, whether an ETL tool is being used and
whether or not changes are required in the source or target systems. The architects and developers who helped
with the detailed design documentation should be able to provide some additional information about any required
skill sets and whether you already have people with these skills available or will have to work with a partner.

After the integrations are built, they should be thoroughly tested prior to being moved to your production
environment. Putting a test strategy in place is a separate topic but there are some great resources available to
help you get started. The key is to make sure you have enough test cases to cover creating and modifying existing
records with a variety of different values in each field in order to make sure that you don’t stumble across a specific
combination that will cause the interface to fail. Integrations that require encryption or additional security should
undergo additional tests to make sure that your data can’t be compromised while it’s in transit.
23
Step 9: Prepare, Deploy, and Maintain
Once your integrations have been built and you’re satisfied with your test results, you can plan your deployment.
This may also require multiple resources, particularly if you had to make changes in your source or target systems.

Alongside technical preparations, remember to get your system owners and technical team members up to
speed on the integrations you’re implementing. Think about what training and documentation is needed. It’s
also important to arm your support staff with knowledge of processes and activities that will be impacted if an
integration fails. Impact is what your business colleagues will be concerned about, so be sure to know who to notify
in the event of failure.

The system owners you identified in your worksheet should be stakeholders in the deployment process and a
cutover plan should be drafted to ensure that changes are moved to production in the correct order and that the
new interfaces aren’t turned on until all of the required pieces are in place.

Once a new interface has been moved into production, monitor it regularly to ensure it is running properly and
all necessary records make it to the target system. Most middleware tools have their own reporting and event
monitoring features. There are also some tools available within Salesforce, such as Event Monitoring that can help
with troubleshooting. There are a number of third party tools available on the AppExchange that can help as well.

What if you need to make changes to an integration after it’s been built? This can happen for a variety of reasons.
Two common examples would be the addition of new fields, or if the timing of an integration needs to be changed
to support a new business process. If a need like this arises, refer back to the integration’s original detailed design
documentation (Step 7) and make the appropriate adjustments. Follow this with another build and test phase
(Step 8) and then deploy the updated interface to production. It’s also helpful to establish a Center of Excellence
that is responsible for long term maintenance of your organization’s integration strategy.
24

Further Trailhead
• Lightning Platform API Basics

Reading
• Platform Events Basics
• Salesforce Connect

Help Documents
• Integration and APIs
• Which API Do I Use?
• Introducing Lightning Platform REST API
• Introducing SOAP API
• Introduction to Bulk API and Bulk API 2.0
• Understanding Metadata API
• Streaming API
• Integration Patterns and Practice

Blog Posts
• Integration & APIs Archive
25

About the
Authors
Richard Booth
Richard Booth is a Senior Principal Customer Success Architect
at Salesforce.org based in the London area in the UK. He helps
nonprofit and higher education organisations make the best
possible use of Salesforce technologies to deliver impact and
support their mission. You can connect with Richard on LinkedIn.

Tom Leddy
Tom Leddy is a Director of Education Services at Salesforce.org.
As a certified Salesforce Architect, he advises education industry
customers that are undergoing digital transformation efforts to
help reshape their organizations and serve their constituents more
effectively. He also leads multiple Salesforce.org community efforts,
including the Ask An Architect content series and Architect Academy
Roundtables series. Tom is a published author, public speaker and
marathon runner based in the Chicago area. You can connect with
him on LinkedIn, Instagram and Twitter
Thank you.
Learn more about Salesforce CSAs.

You might also like