Professional Documents
Culture Documents
The Forrester New Wave - Standalone Chatbots For IT Operations Q2 2019
The Forrester New Wave - Standalone Chatbots For IT Operations Q2 2019
This PDF is only licensed for individual use when downloaded from forrester.com or reprints.forrester.com. All other distribution prohibited.
FORRESTER.COM
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS
2 Standalone Chatbots For IT Operations Get Control Over Your Bots With Forrester’s
Evaluation Overview Automation Framework
15 Supplemental Material
Vendors from varied backgrounds are offering chatbot solutions for IT operations in three major
forms: chatbot toolboxes, platform-specific chatbots, and virtual agents.1 Most of these solutions
currently require some degree of manual development for chatbot language models, intent, entity, and
workflows.2 Leading vendors provide more domain-specific assets, such as prefabricated IT operations
workflows, and greater levels of intelligence, such as self-optimizing language training. Some vendors
are experimenting with actively pathing interactions for even more automated language and workflow
development.3 In the future, the process of setting up, training, and optimizing chatbots will be further
automated and simplified.
We included eight vendors in this assessment: Avaamo, Botpress, EdgeVerve (Infosys), Inbenta
Technologies, IPsoft, Kore.ai, Serviceaide, and SnatchBot (see Figure 2 and see Figure 3). Each of
these vendors has:
›› Advanced language capabilities. The product has advanced chat capabilities such as natural
language processing (NLP) and natural language understanding (NLU) provided natively or through
integration with more than one external NLP/NLU engine out of the box.
›› Net-new chatbot creation. The product can create and customize a net-new chatbot
persona through tooling.
›› Chatbot development GUI. The product provides the capabilities to create and customize
complex workflows through GUI-based tooling.
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 2
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
›› Learning capabilities. The product has the capability to continuously learn and improve actions
through advanced analytics, machine learning, or AI.
›› English language support. The product provides English language support out of the box.
›› Active customers. The product has more than 10 active, paying customers using it for IT
operations and was generally available as of January 1, 2019.
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 3
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
Chatbot How well does the solution take advantage of advanced language capabilities
learning (NLU/NLP)? How are intent and language models trained, modified, and managed?
How are data sets imported? Does the chatbot make use of any predictive
capabilities?
Chatbot What tooling/workflow engine capabilities are provided natively or via integrations?
automation How are complex workflows supported? Can the chatbot be configured to take
action, independent of invocation? What automation partnerships does the vendor
have? How are these integrations handled?
Architecture What integration facilities are available in or for the product? How can users create
and integrations integrations? What API management tools are provided? How are connections into
other systems mapped? What prebuilt IT ops integrations are provided? What is the
technical architecture of the product?
Chatbot What predefined intent models and language sets are available? What
readiness domain-specific and general out-of-the box (OOB) workflows are provided, and how
are these modified? What degree of OOB understanding of IT ops requirements is
demonstrated thorough language, intent, and workflows?
Development How is chatbot development supported? What type of tooling is provided, and how
and tooling easy is it to use? How robust is the GUI provided? At what level of granularity can bot
functionality be modified? How does the solution facilitate the creation and
management of responses? How are data sets managed?
Deployment What solution deployment options are available? What administration efforts are
and security required? How are user permissions granted and managed? How is security and user
data privacy handled? How granular are personally identifiable information (PII)
protection tools? What is the vendor’s data handling and storage strategy?
Bot What does the solution provide in terms of bot operations and reporting? How are
management bots managed? Does it provide visibility into user journey? What is the process for
bot deployment? What optimization tools are available? Does reporting provide any
proactive optimization suggestions or capabilities?
Vendor How well does the product vision and strategy align for the standalone chatbots for
positioning IT operations market? How is the vendor positioning its product in relation to other
vendors? What differentiating capabilities is the solution or vendor offering?
Road map How well-defined are planned enhancements, and how effectively do they address
the standalone chatbots for IT operations market?
Market Is the company executing a successful go-to-market approach? Does the vendor
approach have a well-thought-out partner strategy?
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 4
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
Strong
Challengers Contenders Performers Leaders
Stronger
current
offering IPsoft
Serviceaide
Avaamo
EdgeVerve
Kore.ai
Inbenta Technologies
SnatchBot Botpress
Weaker
current
offering
Market presence*
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 5
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
ns
ity
in
io
ur
ol
at
c
to
n
se
gr
io
ng
s
nd
es
h
at
te
ng
en
nd
ni
ac
ta
m
in
in
tio
ni
em
ta
ro
to
ad
en
ar
re
si
pp
en
au
ag
re
le
ap
po
m
ct
ta
ym
an
op
ot
ot
ot
m
ite
or
ke
tb
tb
tb
tm
lo
el
d
nd
ch
ep
ha
ha
ha
oa
ar
ev
Company
Bo
Ve
Ar
M
C
R
IPsoft
Serviceaide
Avaamo
EdgeVerve
Kore.ai
Botpress
Inbenta Technologies
SnatchBot
Vendor QuickCards
Forrester evaluated eight vendors and ranked them against 10 criteria. Here’s our take on each.
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 6
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
›› Leads the pack with robust IT operations chatbot features. With a library of granular prefabricated
IT operations workflows and language models, detailed reporting, an automation platform, and
extensive language training, IPsoft’s Amelia leads the market in IT operations readiness.
›› Needs to work on appealing to more than just large enterprises. Today, customers view IPsoft’s
Amelia is as a significant investment in money and resources. To address a broader market, the
firm should continue to work on simplifying cost and bot training requirements.
›› Is the best fit for enterprises looking to make a commitment to cognitive technology. With its
workflow automation and active language development features, IPsoft provides an ideal solution
for those looking for advanced functionalities such as creating workflows for more-sophisticated
help desk interactions.
Overall, IPsoft’s customers are satisfied with the product and seek to continuously expand their use
cases. They noted that the product has significantly improved in usability over time.
Wave position
IPsoft LEADER
Products evaluated
Amelia
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 7
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
›› Leads the pack with IT operations orientation. Due to Serviceaide’s heritage in IT operations, its
chatbot strategy and road map strongly align to supporting the needs of day-to-day IT operations,
including granular out-of-the-box IT ops workflows.
›› Needs to expand its visual development capabilities. Missing from the Serviceaide platform are
graphical relationship models, e.g., wireframe graphs. Introducing these features will further assist
developers in gaining an understanding of the user journey through the chat interface.
›› Is the best fit for organizations pursuing an augmented IT operations team. With its IT
operations specificity and easy-to-configure follow-up interactions, Serviceaide’s Luma provides an
attractive platform for augmenting IT service delivery capabilities.
Serviceaide customers are exceptionally pleased with their experiences, noting the solution’s fast time-
to-implementation and time-to-value, thanks to its out-of-the-box capabilities and workflows.
Wave position
Serviceaide LEADER
Products evaluated
Serviceaide Luma
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 8
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
›› Provides robust integration and training capabilities. With prebuilt integrations, including legacy
solutions, Avaamo can address non-API-enabled environments. It also provides data-cleaning tools
model training, using unformatted data for added flexibility.
›› Needs to streamline elements of the development tool interface. While Avaamo provides a
GUI-based chatbot development environment, managing multiple language and workflow elements
is cumbersome and needs to better support designing multidirectional chatbot conversations.
›› Is best suited for multiple generations of IT systems. Through the platform’s capabilities relating
to training and integration, coupled with full security features, Avaamo provides an attractive option
for organizations with legacy IT elements.
Avaamo’s customer references are enthusiastic about their implementations and look to further expand
their uses of the solution. They noted, however, that managing user expectations regarding the current
bot capabilities is critical.
Wave position
Avaamo LEADER
Products evaluated
Avaamo Conversational AI Platform
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 9
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
›› Provides granular language model development tools and RPA coupling. EdgeVerve gives
users a powerful language model development and optimization tool set. Additionally, it natively
integrates with EdgeVerve’s own robotic process automation (RPA) platform, providing a versatile
automation suite.
›› Needs to better market its IT operations capabilities. EdgeVerve provided limited publicly
available information related to Nia and its capabilities in the context of Nia’s functionality for IT
operations. This may cause organizations to overlook the solution when developing their strategies.
›› Is the best fit for customers looking to expand EdgeVerve functionality. Through its
connectivity with the EdgeVerve RPA platform and flexible language models, Infosys’ Nia provides
users with a powerful and customizable automation package.
EdgeVerve was a nonparticipating vendor, and Forrester was unable to speak with EdgeVerve
customers for this evaluation.
Wave position
Products evaluated
EdgeVerve NIA
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 10
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
›› Provides superior language model visibility and control. Kore.ai provides one of the most
intuitive chatbot language management suites we evaluated. It provides a comprehensive intent
management portal, multicontextual conversation support, NLU decision-tree visibility and training,
and native data sanitation tools.
›› Needs to expand IT ops support. While the platform provides a prefabricated ITSM “smartbot,”
the solution assumes an IT environment accessible mostly through open APIs with already-defined
workflows to use as templates for automation beyond base capabilities.4
›› Is the best fit for companies that need a transparent language model. Supporting a variety of
general use cases and granular language learning visibility, Kore.ai provides a well-rounded solution
to pursue a department-agnostic chatbot strategy.
Kore.ai customer references reported overall satisfaction with their experiences and are looking to
expand their uses of the platform. They did note that their initiatives have taken longer than anticipated,
including some unexpected delays due to integration complexities.
Wave position
Products evaluated
Kore.ai Bots Platform
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 11
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
›› Provides a developer-friendly platform. With its modular platform, code-level access to all
elements of a bot, insight into NLU decision making, and overall open source strategy, Botpress
provides developers with a comprehensive set of tools to rapidly develop a chatbot solution.
›› Needs to expand its out-of-the-box support for IT ops. Botpress currently provides a limited
set of prefabricated assets, in general, and requires more IT operations-centric workflows and
language models, a prerequisite for most organizations looking to rapidly implement a chatbot for
those purposes.
›› Is best suited for customers looking to pursue a developer-led chatbot strategy. With its
developer tools, open source community, and customizability, Botpress provides developers with
the materials they need to get a chatbot rapidly up and running.
Botpress’ customers are eager to expand their use of the tool but note that further prebuilt assets
would help them more readily expand beyond their initial implementations.
Wave position
Botpress CONTENDER
Products evaluated
Botpress
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 12
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
›› Provides tight knowledge-base integrations and functionality. With native integration to its
separate knowledge-base solution, Inbenta’s strengths lie in its capabilities to navigate, extract, and
summarize data. This makes the solution ideal for developing a robust information-retrieval chatbot.
›› Is the best fit for organizations pursuing information-focused chatbot support. With strong
knowledge-base capabilities and broad capabilities for internal organization, Inbenta is a good
candidate for helping employees rapidly find the information they need.
Inbenta Technologies was a nonparticipating vendor, and Forrester was unable to speak with Inbenta
customers for this evaluation.
Wave position
Products evaluated
Inbenta Chatbot
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 13
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
›› Provides unique in-chatbot features and marketplace. SnatchBot features unique capabilities
such as in-chatbot variable calculation capabilities and provides users with a broad marketplace
for sharing premade chatbots.
›› Is the best fit for organizations curious about chatbots. With its free-to-try model and tooling
focused on business analysts, SnatchBot provides a fast way to experiment with chatbots.
SnatchBot’s reference customers are satisfied with their implementations and are slowly expanding
their use of the tool but noted that more pretrained models are needed.
Wave position
SnatchBot CHALLENGER
Products evaluated
SnatchBot
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 14
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
To help you put research Translate research into Join our online sessions
into practice, connect action by working with on the latest research
with an analyst to discuss an analyst on a specific affecting your business.
your questions in a engagement in the form Each call includes analyst
30-minute phone session of custom strategy Q&A and slides and is
— or opt for a response sessions, workshops, available on-demand.
via email. or speeches.
Learn more.
Learn more. Learn more.
Supplemental Material
We conducted primary research to develop a list of vendors that met our criteria for the evaluation and
definition of this emerging market. We evaluated vendors against 10 criteria, seven of which we based
on product functionality and three of which we based on strategy. We also reviewed market presence.
We invited the top emerging vendors in this space to participate in an RFP-style demonstration and
interviewed customer references. We then ranked the vendors along each of the criteria. We used a
summation of the strategy scores to determine placement on the x-axis, a summation of the current
offering scores to determine placement on the y-axis, and the market presence score to determine
marker size. We designated the top-scoring vendors as Leaders.
For some vendors, Forrester was only able to interview one customer reference. For non-participating
vendors, customers were not interviewed, and scoring was based on a briefing, previous analyst
experiences, and public supplementary information.
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 15
Citations@forrester.com or +1 866-367-7378
FOR INFRASTRUCTURE & OPERATIONS PROFESSIONALS April 15, 2019
The Forrester New Wave™: Standalone Chatbots For IT Operations, Q2 2019
The Eight Providers That Matter Most And How They Stack Up
Integrity Policy
We conduct all our research, including Forrester New Wave evaluations, in accordance with the
Integrity Policy posted on our website.
Endnotes
1
For more on the specifics of these categories and an overview of the wider market of chatbots for IT operations, see
the Forrester report “Now Tech: Chatbots For IT Operations, Q1 2019.”
2
In the context of chatbots, “intents” and “entities” refer to specific concepts. Intents, and intent management, help
the language-processing side of chatbots determine what a user is attempting to do or requesting and help make
a connection between a user’s sentence and what action or workflow to execute. Entities are keywords or types of
keywords, such as a name, a proper noun, or a personal or possessive pronoun, that modify an intent and identify
specific components or variables in a sentence. For example, in the sentence “I would like to reset my password,” the
intent of the sentence would indicate that someone wishes to kick off a password reset workflow, with the entities of
“I” and “my” identifying the user as well as the subject of the password reset.
3
Actively pathing interactions refers to the process of capturing and creating workflows based on agent/chatbot user
interactions in an automated fashion.
4
ITSM is IT service management.
Kore.ai is currently cooperatively developing connections into RPA platforms to supplement its automation
capabilities, with some customers already leveraging these connections.
© 2019 Forrester Research, Inc. Unauthorized copying or distributing is a violation of copyright law. 16
Citations@forrester.com or +1 866-367-7378
We work with business and technology leaders to develop
customer-obsessed strategies that drive growth.
PRODUCTS AND SERVICES
›› Core research and tools
›› Data and analytics
›› Peer collaboration
›› Analyst engagement
›› Consulting
›› Events
CLIENT SUPPORT
For information on hard-copy or electronic reprints, please contact Client Support at
+1 866-367-7378, +1 617-613-5730, or clientsupport@forrester.com. We offer quantity
discounts and special pricing for academic and nonprofit institutions.
Forrester Research (Nasdaq: FORR) is one of the most influential research and advisory firms in the world. We work with
business and technology leaders to develop customer-obsessed strategies that drive growth. Through proprietary
research, data, custom consulting, exclusive executive peer groups, and events, the Forrester experience is about a
singular and powerful purpose: to challenge the thinking of our clients to help them lead change in their organizations.
For more information, visit forrester.com. 144395