A F i gure Ei ght e B o o k

How to Create
an AI Center
of Excellence
for Enterprise

Lead business and technology expertise within your organization to deliver

on artificial intelligence that drives revenue, customer experience, and more,
powered by high-quality training data.
Introduction
Organizations are beginning to invest heavily Remembering that it will take some time before
in artificial intelligence (AI) initiatives. Some are realizing a return on your AI investment is
spending millions of dollars on AI strategies, crucial, as well as acknowledging the fact that
and with good reason. Companies that have you’ll need buy-in from other stakeholders of
already adopted AI report that it has allowed the value of AI.
them to edge ahead of competitors.
To convince stakeholders to embark upon AI
If you’re reading this, chances are your journeys, you must find and attract internal
organization has already made inroads on an AI customers for artificial intelligence. These
strategy. You may have one project under your individuals or teams are interested in AI, but
belt but find yourself asking, “Now what?” The probably aren’t sure where to begin.
first thing to remember is patience. According
to Gartner’s senior director analyst Chirag Whether they know it or not, these stakeholders’
Dekate, “Delivering business impact from AI journeys begin with training data. Training data
initiatives takes much longer than anticipated.” is crucial to AI. In fact, AI can’t exist without
high-quality training data. You need to develop
AI will be used by 40% of Infrastructure a resilient data fabric in order to collect, store,
and Operations teams by 2023 and annotate data that will be used to train
(Source: Gartner) machine learning models.

What is the best way to ensure your machine learning is

accurate in production?
Build a training data practice in your AI center of excellence (CoE), as part of your resilient data fabric

DEFINITION: DATA FABRIC

Data fabric enables frictionless access and sharing of data in a distributed data environment. It enables a
single and consistent data management framework, which allows seamless data access and processing by
design across otherwise siloed storage.

According to Gartner, through 2022, bespoke data fabric designs will be deployed primarily as a static
infrastructure, forcing organizations into a new wave of cost to completely re-design for more dynamic
data mesh approaches.

LEARN MORE

2
What Is a Center of Excellence?
Some organizations have it, some are still getting around to building
it. It’s important to define a Center of Excellence.

DEFINITION: CENTER OF EXCELLENCE

Team of people that serves as the nexus of expertise for a given field of study or subject.

The concept of a center of excellence (CoE) ■■ NTT Data has an AI center of excellence,
has been around for a while. This team manages geared toward developing “further expertise
resources, provides counsel, and offers best in AI and train[ing] engineers engaged
practices for any other team or individual that worldwide in the AI field.”
wishes to learn more about the CoE’s given
subject. Firms in many industries have already
Services organizations, such as Forrester and
developed their own centers of excellence,
McKinsey offer advice on how clients can
which can provide a blueprint for organizations
develop CoEs for any number of purposes.
that wish to develop their own centers.

While the purpose of CoEs runs the gamut, AI

■■ The Infosys automation CoE was built using and its related fields are arenas where centers
a “detailed process analysis for automation can produce plenty of competitive edge. AI has
suitability and adopted a hybrid delivery become so important to modern organizations
model” that has improved efficiencies that more than one-third of large firms,
between 30% and 80%. including non-technology firms, have already
■■ Microsoft has a Data Insights Center established a CoE for the explicit purpose of
of Excellence and offers its services furthering AI.
and technology to organizations that
wish to establish their own CoE.

3
Thomas Davenport, an MIT Digital Economy research fellow, professor at Babson College, and
senior adviser at Deloitte Analytics, and a senior adviser at Deloitte Analytics; and Shivaji Dasgupta,
Managing Director and Head of Data Architecture and Smart Analytics for Deutsche Bank’s Private
and Commercial Bank, provide, via Harvard Business Review, guidelines for establishing a CoE.

 Create a vision for AI in the company

 Identify business-driven use cases

 Determine the appropriate level of ambition

 Create a target data architecture

 Develop and maintain a network of AI champions

 Spread success stories

While the specifics of the organizations’

approaches differ, the basic direction is similar:

Building the AI Center of Excellence

Make the Case Get Executive Build your Launch your

for AI in your Buy-In for the CoE Team & Initiative by
Organization AI CoE Architecture Building a
Flywheel

Figure Eight eBook | How to Create an AI Center of Excellence for Enterprise www.figure-eight.com 4
Make the Case for AI
in your Organization
Identify the key business and customer use cases, the data you
have, the data you need, and establish the scope of your AI Center
of Excellence

To identify the first issue to tackle with your discovery process will change from organization
CoE, it’s better to think of the exercise as to organization. However, it’s important to
one of discovery. Discovery is a process best meet with product teams to understand their
approached through questioning and iterating. roadmaps; engineering teams to uncover their
The end goal is to best understand how the CoE efficiency issues; data science teams to uncover
can act as a go-to resource for any machine available data, and customer success teams, as
learning-related questions the company has. they’ll be attuned to what customers need and
want out of your products.
There will be many internal stakeholders
with whom to meet — such as: Data Science, With a strong grasp of various internal
Product, Engineering, Professional Services, roadmaps and who owns their direction, there’s
Customer Service, Success, and Business a much greater chance you’ll identify where your
Development leadership. The individuals company’s data challenges exist. Now it’s time
and teams whom you consult as part of the to think about executive buy-in.

Adobe gives everyone — from emerging artists to global brands — everything they need to design
and deliver exceptional digital experiences.

One of Adobe’s flagship offerings is Adobe Stock, a curated collection of high-quality stock imagery. The
library itself is staggeringly large: there are over 120,000,000 photos, diagrams, videos, and graphics with
roughly 150,000 uploaded each and every day. And every one of those images needs to be found.

LEARN MORE

5
Get Executive Buy-In for
the AI CoE
Team effort both operationally, and strategically, gets AI executed
at scale

Getting executive buy-in early is the key to In many cases, someone like the CTO or VP of
getting an AI initiative kicked off the ground. Engineering or Product is responsible for AI
Demonstrating your project’s effectiveness can initiatives, given that it usually leverages both
help you build the business case for a CoE and engineering and data resources, for training
also establish credibility for AI as a useful tool. and deployment. Part of securing buy-in will
Here, it’s important to remember that you’re not be affirming that the CTO or other technology
building a business case for just one specific leader is willing to lead the CoE team.
problem or problems.
If, rather than a product leader, line-of-business
You’re focused instead on showing how AI can owner, or technical practitioner, you are the
scale to positively impact many facets of your CTO or executive responsible for a CoE, then
business. The fact is, making the shift to an AI- it is imperative you prioritize this initiative, fuel
powered organization (which is the ostensible it with relevant and sufficient data, and build
goal of creating a CoE in the first place) requires the right team, and the right infrastructure, to
building an infrastructure that can support deliver on the AI goals.
interdisciplinary functions.

Artificial Intelligence consultancy and solutions provider

“
An AI CoE can significantly improve the likelihood of AI providing true value back to the business by
introducing standardization of tools, data architectures, processes, and knowledge sharing. It ensures
day 1 value focus by enforcing the identification of measurable KPIs and success criteria in early stage
PoCs. The CoE, combined with the best of breed tools, like the Figure Eight platform, enable different
business units to securing quick wins and create massive momentum across the organization for
AI programs.”

PAT MCDERMOTT, HEAD OF STRATEGY & BUSINESS DEVELOPMENT AT PROVECTUS

6
Build your CoE Team &
Architecture
Under the CTO, most AI CoE teams will likely look similar to the
following composition

7
AI CoE Teams
Despite the fact data scientists and ML
engineers are up to date about the latest
Machine Learning industry trends and technologies, their primary
focus is on the data itself, rather than R&D-
type activities. Identifying the appropriate data
Machine learning teams are specialized
to use, ingesting that data, and cleaning and
and built to help improve the processes that
annotating it are crucial to building a successful
support the core of the business. Typically,
model, and our research suggests most data
data scientists and machine learning
scientists spend at least one-quarter of their
engineers make up the bulk of the team.
time managing, cleaning, and/or labeling data.

Data scientists are often responsible for

While research projects may be more appealing
creating new data architectures to solve
for their excitement, the current truth is that
a given business problem. These team
much of a technical practitioner’s job is in the
members are typically up to date on the
more tedious production work.
latest technologies and how to best leverage
them. Scientists also often help engineers
troubleshoot the ML model.

Data Analyst
ML engineers understand which type of
architecture they need to build and train the
Data Analyst teams use business intelligence
model or series of models. They should be well-
(BI) tools to understand data’s impact, unlike
versed in working with different data types,
data scientists, who focus on building and
testing and tuning the model, and figuring
explore algorithms. Both data scientists and
out how to get the model into production.
data analysts can be data mungers, who work
Furthermore, engineers help make sure the
with the data on their own to identify issues and
entire pipeline can support rapid development
devise strategies to fix them, either internally or
and iterative increments after launch.
with trusted vendors.

Amazon’s mission is “to be Earth’s most customer-centric company”.

“
We believe that machine learning leaders have the opportunity and responsibility to leverage the latest
advancements in AI technology to deliver a step-change improvement in customer experience. Figure
Eight provides the required collaboration between technologists and business leaders to solve vexing
challenges and embrace ambitious opportunities. Through our partnership, we are striving to offer the
best selection to our customers with high-quality training data at the core of their initiatives.”

KRISTOF SCHUM, GLOBAL SEGMENT LEADER OF MACHINE LEARNING IN AWS PARTNER NETWORK

8
 DevOps Project Management

DevOps teams ensure everything runs smoothly A technical project manager liaises between the
on the company’s infrastructure. Organizations product team and engineers, data scientists,
need DevOps support to launch a model and and DevOps teams. They manage the entire
manage the continuous delivery pipeline. process and evaluate and curate the tools the
DevOps also helps the team understand if team uses. This role should be filled by someone
the model is working as intended, as well as who is able to lead those conversations and
oversees and maintains the technology behind understand the requirements of the product
deploying the model. side to figure out feasibility and, just as
importantly, prevent the model creation from
getting too research-oriented or too costly.

Engineering Product Management

Product managers bring in the ability to

Engineers can and often do interact with
understand the workflow in terms of how
DevOps engineers, but you want to make
it’s affecting the business case and how it’s
sure the interface between the ML team
making money. They are a critical part of
and traditional departments is seamless.
the CoE group, especially for product-first
For example, maybe a mobile app is talking
organizations, where AI can become a key
to the model, but it wasn’t previously — you
part in delivering business value.
need the right people in place to make sure
the API is integrated and working.
The product manager should also have
expertise in the specific space in which the
If your organization uses a cloud
company is developing algorithms. A large
infrastructure provider, such as Amazon
part of a scientist’s time is spent in only one
Web Services, Google Cloud, or Microsoft
field because the speed at which various forms
Azure, then it may also invest in a cloud
of AI-enabling techniques, like deep learning or
infrastructure manager who is an expert
computer vision, are evolving requires them to
in whichever data lake the company uses.
specialize. Accuracy for video is very different
than that for text, for example.

9
AI CoE Architecture
With buy-in secured and a team in place, you can hone in on which problems the CoE will tackle
first. The goal is to create scalable practices, but it doesn’t make sense to try to apply AI to every
single process in one fell swoop. To build scalable practices, start small. Identify and prioritize an ML
opportunity that should be a relatively quick win, and build momentum from there.

In order to generate momentum, you’ll need the architecture necessary to scale an AI practice.
To manage this architecture successfully, organizations might turn to AIOps, DevOps’ artificial
intelligence-infused evolution. According to Andrew Lerner of Gartner, AIOps’ platforms “utilize big
data, modern machine learning and other advanced analytics technologies to directly and indirectly
enhance IT operations (monitoring, automation and service desk) functions with proactive, personal
and dynamic insight.”

In effect, AIOps leverages AI to keep product and operations infrastructure running smoothly. If
you’re building scalable AI, why not continue to leverage its benefits for your incident management
and other operations-related activities, like getting high-quality training data?

Se
ve) rv
ic
r e
se
b

D
(O

es
ng

k(
Eng
Monitori

Machine Learning
age)

TD
Big Data

AIOps
Platform

Au
to m t)
a t i o n ( Ac Training Data for
Machine Learning

B u s i n e s s Va l u e Internal / Vendor Managed

Annotation Platform
Source: Gartner

10
Launch your Initiative
by Building a Flywheel
Scalable AI relies on strong training data practices, machine learning
algorithm selection, and business analytics

Once you uncover the projects best suited For example, organizations may want to
for an AI-enabled makeover, you can prioritize determine metrics that measure the business
which to look into first. This is where your impact they expect from AI, such as:
project manager on the CoE team will come in.
They will help you keep your AI initiative on task
and remind everyone it’s a marathon, despite
the fact development cycles are referred to as
“sprints”.
SAVING GENERATING
The training data center of excellence is MONEY REVENUE

primarily responsible for acquiring and

annotating the high-quality training data
needed to build the algorithms for your
project. Before you begin to sift through reams
of information, your team must implement
success metrics.
SAVING IMPROVING
TIME SPECIFIC
EFFICIENCIES

GumGum teaches machines to see in order to solve hard problems across the media industry.

“
Working with Figure Eight has made our model development process 10 times faster, allowing us
to get to the next step much faster, focus on research and think about new possibilities such as
audio and video analysis.”

LANE SCHECHTER, PRODUCT MANAGER AT GUM GUM

LEARN MORE

11
These metrics will help guide the launch a smaller project and are now ready to elevate
process. They will help you determine they types AI-enabled technology into production as a core
of data needed to build the algorithm and what part of your business.
kind of partners/vendors you need to engage
with as part of the CoE workstreams. To turn AI into a core part of your business, you’ll
need to build a flywheel—a mechanism that
If launching an initiative, it’s likely that your allows you to scale your training data-creation
organization already has become comfortable and model-building processes across other
with a certain level of AI within its business facets of your business.
processes. You’ve probably tinkered with AI for

The AI CoE Flywheel

With a clear definition of success and your initial It will create an engine that encourages people
project underway, you can turn some of your to come to you for AI-related projects. When
attention toward building a self-fulfilling process. you create a CoE that acts like a flywheel,
By now, the CoE team should feel empowered you’re creating a machine that is an integral
to go out and find other opportunities along the part of the organization. You’re moving out of
product roadmap. The ideal state for the CoE the experimental group to core piece of driving
should be that AI becomes a core part of your revenue for the company.
company’s products and services, not a series
of one-off projects or experiments. Creating Building a training data flywheel requires
momentum for AI inside the organization is ingesting the appropriate data, annotating
critical for elevating the AI function to a table- it, and developing an algorithm or a set of
stakes member of the technology stack and algorithms that address a business case. When
product core feature list. Once you get the first developing an algorithm, you’ll have a number of
use case permanently in production, the next choices, each with its own set of pros and cons,
ones will follow naturally alongside, leveraging depending on the size of your organization, the
that momentum. This process is like creating an resources available for a CoE, and the purpose
AI flywheel within your organization. But what of the algorithm.
does success here look like, exactly?

Digit analyzes spending and automatically saves the perfect amount every day.

Digit uses Figure Eight to constantly build and iterate on intelligent transaction models.
■■ Categorize transaction data
■■ Identify transaction synonyms
■■ Evaluate recurrence in transaction data

READ MORE

12
 PROS CONS

■■ The models provided by ■■ Despite the powerful APIs

vendors are generally these models contain,
PAYING cheaper than building a they’re very generic and offer
for a vendor-produced model.
model in house. limited, specific functionality.
There is a growing market for ML If your use case does not
model providers. AWS offers an ■■ The APIs that these vendors
align well with what one of
extensive market with models offer typically work with a
for a variety of purposes. Google, these models offers, then it
number of use cases.
Microsoft, and IBM Watson are just a will be of little help.
few of the big-name enterprises that ■■ Finally, purchasing a
offer machine learning-as-a-service model typically allows an
solutions for sale.
organization to get AI up and
running faster.

■■ Building a proprietary model ■■ Building a model in house is

offers an organization control likely the most expensive—at
over the IP. This route will also least at the outset—option of
yield a model that fits very the three.
BUILDING specifically the use case in
a model in house. ■■ It requires an appropriate
question.
team in place, as well as the
Working on a process that’s core
to your business? Then building
■■ Finally, if more models raw resources necessary to
a model your organization owns are required in the future, build from scratch, train, test,
and can tweak when necessary is building an initial test case and maintain in production.
probably the best scenario.
in house can offer a strong
foundation from which
to work when developing
subsequent models.

■■ Outsourcing a model allows ■■ Paying a third party to

OUTSOURCING for a more customized model create a model can end up
model creation.
without the initial overhead being more expensive than
This is the option that perhaps required of building a model anticipated if said party
best suits an organization
from scratch. doesn’t produce a model
that needs a more tailored model
than what providers offer but adequate for your purposes.
■■ Outsourcing is a more flexible
doesn’t have the resources to
option than using pre-
build one from scratch. You can
partner with organizations like built algorithms and allows
Provectus to tap into their existing for more agile long-term
AI expertise for model creation.
planning.

13
Regardless of which approach to model
building you select, you’ll need the appropriate Learn more about the
tools to guide you from data ingestion to model
Build vs Buy tradeoffs when
deployment. Figure Eight offers a combination
it comes to AI Data Labeling
of a global crowd of annotators for creating
high-quality training data and AI training data
solutions designed to accelerate customers’
artificial intelligence initiatives. While it’s possible
to build data-annotation tools in-house, it’s
often more efficient to think outside of the
organization for certain parts of a CoE.

In the case of a CoE built for AI training data

creation, it’s likely you’ll want to lean on
successful and proven vendors to augment
the infrastructure you’ve built. Focus internal
energy and resources on developing cohesion
among stakeholders and determining the vision
LEARN MORE
and goals that the CoE will support. AI promises
immense opportunity, and it’s worth exploring
existing tools to help leverage that potential.

Figure Eight eBook | How to Create an AI Center of Excellence for Enterprise www.figure-eight.com 14
Critical Success Factors
for Enterprise AI
Build Healthy Data Work Streams within AI Center of Excellence

Even after you understand the process This type of tech stack, coupled with
necessary for building a center of excellence, professional services to guide the entire CoE-
you may find yourself asking, “How am I going to creation journey, is uniquely positioned to
do this by myself?” Finding the right people and deliver on the increasing volume, quality, and
technology partners to make these processes speed requirements for training data to support
more efficient and act as a guide along your business innovations and efficiencies which use
journey will be invaluable. ML and AI.

It’s imperative you build a team with domain •

expertise and leverage solutions that enable
you to annotate data at scale with high levels of
accuracy and a large-enough crowd.

15
The Figure Eight platform is the most comprehensive
solution for collecting and labeling images, text, speech,
audio, and video. It combines Figure Eight’s machine
learning ML-enabled annotation tools and self-serve
client workspaces with Figure Eight’s global multilingual
crowd of skilled contractors, and a wide range of
managed services to ensure delivery of high-quality
training data at scale, with the speed and security
required by customers.

Figure Eight’s ML-enabled annotation tools allow human

annotators to create appropriate training data faster
than humans alone can. With a training data annotation
platform readily available, taking the data and turning
it into a useful model will be a much smoother process.

In 2019, Appen acquired Figure Eight. Both of our

companies have shared the same goal for many years;
to provide our customers with high-quality training
data needed to scale their machine learning and AI
programs. By bringing Appen and Figure Eight together,
we’re offering our customers a single source for high-
quality training data, supported by leading technology,
an extensive global crowd, and a team that is committed
to their success.