Forecasting with SAP APO DP?

The Gap betweenThe State

Theory ofPractice?
and the Art?
30 October 2013, The Work Foundation, London, UK

Dr. Sven F. Crone

Assistant Professor - Director
 Slagging of IT is
always in fashion

but its too simple!
 Disclaimer
Objectives of ERP Systems
Relative importance Relative importance
to value chain activities to support activities

Data Integration
External buy

Overall importance
New ways of
doing Business in of knowledge
Corporate
benefits
& processes
Global capacities

Flexibility / Agility

IT IT purchasing
benefits benefits
IT architecture
cost reduction

[Martin et al. (2002) p.182]

Functionality is only one of many aspects in ERP/APS implementation

 Disclaimer
Objectives of ERP Systems
System Design Objectives
ERP / APS systems offer >3000% functionality
Functionality is accessible in 1000+ best practice processes customise
normal implementation aims at 80%-85% coverage!

Standard ERP system

Current functionality
Target functionality

Lacking functions Unwanted functions

100% (or more) functionality is NOT the aim of standard ERP/APS systems!
 Disclaimer
Objectives of SAP APO
SAP APO is more than just a forecasting tool
Holistic integrated planning suite
Communication with legacy systems

need to evaluate overall SAPAPO based on total functionality

 you can always
Customise!

Customised Mercedes-Benz SL600s, studded with 300,000 Swarovski crystal glass,

 Disclaimer
Cases of misuse of SAP APO

Buying the most expensive

Hammer

does not make you

the best Carpenter!

Aforecasting system is only a tool in the hands of a forecaster!

Yields very different results by Crafstman vs. Novice (DP/IT)
Not today!
 Agenda

Forecasting with SAPAPO DP

1. Disclaimer

2. How do companies use APO DP?

Evidence from a survey
System(s), Orga, Setup, Methods !

3. Best Practices in APO DP?

Forecasting Science as benchmark
Data Exploration, Model Selection
Promo Forecasting, New Products
 Study Methodology
Survey of Practitioners
Questionaire Design
Pilot study in 2011 (to ensure validity)
Final version pre-tested with 18 FMCG forecasters
Questionnaire implemented online
Conducted January 2012-August 2012

Survey Sample Design

Specified target group: demand planning &
forecasting professionals (in manufacturing)
LCF Mailing list, forecasting lists / blogs (ISF, SAS)
100s of LinkedIn Groups
2000+ personalised LinkedIn invites
Multiple reminders sent

Response
540 responses
260 incomplete (reminders send, to only speculative interest, unwilling
to give email address (although not mandatory), Atrophy (number of
repeated questions), unsuitable respondent (industry sector &
position)
15 complete responses discarded (Consultants/academics, rushed
surveys (10-15 mins), highly inconsistent answers, middle-clicking {
same answer for every question in groups)

263 complete surveys with usable forecasting systems information

200 surveys from forecasters in Manufacturing representative!
 Study Results
Systems use of Respondents
Group responses by a forecasting software plus average response!

APO and APO+ (incl. specialists FSS add-ons) dominate todays market!
 Study Results
Systems use of Respondents
Group responses by a forecasting software plus average response!

Interpret general
forecasting
Breakdown of responses by sector practices
and use of systems?
Food & Beverage
Classification:
Industrial/other
Industry Sector

Other consumer goods APO

Retail APO+
Electronics & computing ERP/APS
Consumer healthcare & beauty
Excel
Wholesaler/Distributor
Pharmaceuticals SAP
Logistics/3PL Specialist
Clothing & Apparel Interpret whether
0 5 10 15 forecasting
20 25 practices
are influenced by
Responses
SAP APO DP system?

mostly large manufacturers active in the FMCG / CPG industry

Allows insight into different forecasting practices by systems APO DP!?
Which Systems?
 Study Results
Systems use of Respondents
Excel only
17% SAP APO only
21% SAP based solutions
dominate all of ERP
48 users apply
usage (100 v. 89 users
only SAP APO DP
or 38% v. 34% in %)
FSS Specialist Only
11%

SAP APO + Other FSS

12%

23 respondents
Majority (72%) of use APO DP plus
respondents use an SAP (non APO) an extra FSS
ERP/APS system 5%
Specialist system
(incl. SAP APO etc.)
ERP/APS14 respondents
34%
still use SAP with
no APO modues
(e.g. R/3 MM )
Heterogeneous market of ERP/APS vs. stand-alone FSS & combination
 Study Results
Systems use of APO+ Respondents
Arkieva/Zemeter,
Demand SAP APO+ FSS
SAP APO
Excel only
17%
only Solutions, Oliver JDA
21%
Wight eSOP tool, 16%
FSS
Servigistics,
Specialist
Only
SO99+, SPSS,
11% SAP APO +
Other FSS
Steelwedge,
12% TOOLBELT,
SAP (non
Vanguard
APO)
5%
29%
ERP/APS
34%

Forecast X
16%
23 users apply
SAP APO DP
plus an additional
Mercia
Specialist FSS 7%
Finmatica
INFOR (2005)
Forecast Pro SAS
16% 16%

Additional tool indicate shortcomings of SAP APO DP

No clear market leader / equal share of additional FSS tools used
 Study Results
Systems use of Respondents
add*one, ASW,
Excel only
SAP APO COMPASS, Epicor ERP/APS SYSTEMS (NON-SAP)
only
17%
21% iScala, GPAO/Oracle,
Kinaxis, M3
FSS (Lawson), MXB
Specialist JDA
Only PRMS, NeoGrid,
11% SAP APO + 23%
Other FSS Reflex, Rockysoft,
12%
Ross/scp,
SAP (non
APO)
Servigistics, Syspro,
5% Baan TecCMI
ERP/APS
34% 2% 17%
Steelwedge
2%
Futurmaster
2%
Microsoft Dynamics
BUT: 4 non SAP AX
3%
users of ERP/APS Infor
3%
also use extra FSS
Oracle
specialist software 3% Demand Solutions
14%
(ForecastPro, eTXT, Arkieva/Zemeter
5%
Steelwedge etc.)
MfgPro/QAD Demantra
Logility
7% 10%
9%

Large variety of ERP/APS systems led by JDA & Demand Solutions

 Study Results
Systems use of FSS Specialist Respondents

Excel only
SAP APO SPECIALIST FSS SYSTEMS
only
17%
21%

Forecast Pro
FSS 21%
Specialist
Only
11% SAP APO +
Other FSS
12%

SAP (non
APO)
5%
ERP/APS
34%

Apollo Data
Technologies,
Applix Tm1, SAS
Avercast, Demand 10%
Solutions, Eviews,
GMT, Infor,
Predicast/Aperia,
Project i, Remira
LogoMate, Retail Forecast X
Express AMP2, 7%
SAF, SPSS, Stata,
TORA, TXT
R
7%

Large variety of software packages, led by ForecastPro

 Study Results (all industries)
Use of Softare Packages
Excel & FSS users
dont use ERP/ APO DP users use
APS systems MS Excel as well
to make forecasts!

APO DP users use

APO DP most
frequently to
Demand Sensing Few companies make forecasts
& CPFR systems today still use a
are rarely used by custom made (in-
anyone! house) solution

Companies use multiple software together lack of functionality!

Adoption of standard packages (ERP/APS/FSS) individual dies out
 Planners use SAP APO and Excel and FSS
all in parallel!
(requires additional data exchange, process coordination, extra time & effort )

indicates some lack

of functioality!
What systems do you use?
Which Organisation?
 Study Results (all industries)
Organisational Setup

Most companies
Smaller number predict 1000
of items likely to items across
drive use of systems
MS Excel Small companies Large companies
more likely to more likely to
use MS Excel
Number of use SAP APO DP
forecasters is
independent of
APO+ users with
use of system(!?)Large companies
more #items
more likely to
than APO users
use SAP APO DP
drives system?

# of items & company turnover drives APO DP adoption

# of Forecasters seems independent of #SKUs and # of revenue?
 Study Results (all industries)
Forecasting Setup

APO DP normally
forecast at item
or account level,
never store / DC
APO DP+ FSS
forecast also at
Daily forecasts DC level
are still rare (!) (VMI functions?)

Users forecast at
multiple time
buckets at the
same time!

Similar setup: companies forecast on multiple levels & time buckets

How is this (best) supported by SAP APO DP?
 Study Results (all industries)
Drivers of Forecasting

Users of SAP R/3

(MM etc.) face
the least impacts
(lucky them!)

Most companies
& system users
face the same
relative impact

Market complexities impact all business equally, regardless of system

What organisation do you use?
Which Tools?
 Study Results (all industries)
Use of Forecasting Methods
APO DP users
forecast with
Stats + Judgment
(S&OP process?)

Specialist FSS
Excel users apply Excel users apply
users apply
mainly judgment mainly judgment
mainly Stats
with any Stats! (without
with any Stats!)
Stats!
(without Judg.!)

Typical SAP APO usage employs 25% only stats, 25% only judgment
and 50% a combination of Statistics & Judgment
 Study Results (all industries)
Use of Forecasting Methods

72% use simplest

43%
methods only for
Only marginal Exp.Smoothing
level timeisseries
Only 10% of the (28% MA+15%
use of Linear
methods used can
the single most Naive)
Regression
integrate causal in used method
driversSAP APODCDP
(Promos, family in APO DP
stock )

Only limited use

of advanced
methods across
any systems!

APO DP: 78% (45% ETS + 22% Av+ 11% Nave) use simple methods!
APO drives use of simpler methods than average software packages
What algorithms do you use?
 Study Results (all industries)
Satisfaction with Forecast Accuracy

using Specialist
Software slightly
increases user
satisfaction

SAP R/3 users SAP APO users

are the least are (moderately)
satisfied (but still satisfied - as are
only moderate) most other users

All users are only moderately happy with their forecast accuracy
APO DP not worse off than SAP R/3, ERP or Specialist software
 Study Results (all industries)
Satisfaction with Forecast Accuracy

Excel & SAP users

consider better
systems more
More resources important, not
& Training are SAP APO users!
considered least
important

Users seem satisfied that better systems will not improve accuracy!
BUT: will more customer data not require new system (fucntions)?
 Study Results (all industries)
Satisfaction with Forecast Software

APO & APO+ users

are generally
satisfied with their
forecast software

Users seem generally satisfied with their systems

Tricky interpretation, as ERP/APS/Excel can be customised!
 Agenda

Forecasting with SAPAPO DP

1. Disclaimer

2. How do companies use APO DP?

Evidence from a survey
System(s), Orga, Setup, Methods !

3. Best Practices in APO DP?

Forecasting Science as benchmark
Data Exploration, Model Selection
Promo Forecasting, New Products
NON-
 Forecasting &
its technology
are well established!
 Advances in Forecasting
25 years of Scientific Insights!
Founding of the International Institute of Forecasters
unify the field and bridge the gap between theory & practice

Publication of 2 core interdisciplinary journals Peer Reviewed!

International Journal of Forecasting 1985, Journal of Forecasting 1982
Various publications in Operational Research, Management Science,
Finance, Econometrics, Economics, Marketing, Retailing journals

Publication of Practitioner oriented journals Peer Reviewed!

Foresight The Journal of Practical Forecasting (JBF to a lesser extent)

Annual conferences
The International Symposium on Forecasting (ISF) Peer Review
Professional conferences (Forecasting Summit, commercial
IBF, iE, SAS, Terra etc.) are typically non-peer review

4 Nobel Prices for research in forecasting & related areas

Clive Granger & Robert Engle
Daniel Kahneman

Forecasting has developed as a scientific discipline of its own right

State of the art know-how from 25 years of R&D plus practice !!!
 The Forecasting Process
Statistical Modelling: Model Application
Initialization via overall analysis

Monthly
Maintain Analyse & Decision on Analyze
statistical
master data understand automated time series forecasting
& link with assortment forecasting Identify
& clean data
predecessors (ABC / XYZ) suitable
models &
parameters
Evaluate
forecast
quality
Apply
statistical
/ manual
Adjust & forecasting
Finalization release
of forecast forecast
 The Forecasting Process
Statistical Modelling: Model Application
Initialization via overall analysis

Monthly
Maintain Analyse & Decision on Analyze
statistical
master data understand automated time series forecasting
& link with assortment forecasting Identify
& clean data
predecessors (ABC / XYZ) suitable
models &
parameters
Evaluate
forecast
quality
Apply
statistical
/ manual
Adjust & forecasting
Finalization release
of forecast forecast
 The Forecasting Process
Statistical Modelling: Model Application
Initialization via overall analysis

Monthly
Maintain Analyse & Decision on Analyze
statistical
master data understand automated time series forecasting
& link with assortment forecasting Identify
& clean data
predecessors (ABC / XYZ) suitable
models &
parameters
Evaluate
forecast
quality
Apply
statistical
/ manual
Adjust & forecasting
Finalization release
of forecast forecast
 Analysing Assortments
ABC-XYZ Analysis

41.4% of
profit
comes
from 20%
of
products

59.8% of
forecasting
error
comes
from 20%
of
products

Now also available in SAP APO DP in master data (VARCOEFF)

 Analysing Assortments
ABC-XYZ Analysis

A-Narts
Important for the business
Interactive MoSel for decision support Fewer NARTs (20%)
Cannot use Neural Nets as
Exponential Smoothing & SLR it cant be explained in e.g.
ISF

Automatic Automatic
X-Narts
Simple to forecast
Exponential Exponential Smoothing & SLR incl. Neural
BY, BZ, CY, & CZ -Narts
NARTs Smoothing & SLR Networks Less important
Avoid Neural Nets Many time series (50%)
similar high Neural Nets can be used
accuracy use the
simplest method

No provision of process know-how on how to use & apply it

 The Forecasting Process
Data Analysis & Exploration

Maintain Analyse & Decision on Analyze

master data understand automated time series
& link with assortment forecasting Identify
& clean data
predecessors (ABC / XYZ) suitable
models &
parameters
Evaluate
forecast
quality
Apply
statistical
/ manual
Adjust & forecasting
release
forecast
 Baseline Data Analysis & Exploration
Very flexible Data Exploration in SAP APO DP
customization of
key-figures to be
displayed on
screen

Data analysis and exploration in SAPAPO DP is

 Baseline Data Analysis & Exploration
Graphics in Interactive Forecasting

Standard colours What is this?

Level time series No distinction
are poor & often
not changeable between Null
due to setup and 0 distorts

Seasonal time series

Trend time series

Visualisation is needed for Model Selection & Outlier Detection

 Baseline Data Analysis & Exploration
Grahs Missing

Seasonal Plots ACF Plots Spectral Plots

Seasonal Diagram Amplitude
5.1 60
5

4.9 40
4.8

4.7 20

4.6
seasonal
0
4.5 0 0.1 0.2 0.3 0.4 0.5
2 4 6 8 10 12
frequency (Hz)

Statistical tests, e.g. F-Test Compare distributions

Seasonal Diagram
5.1 using F-test (parametric)
5 or
5 5 Freidmans (nonparametric)
4.9
Many others exist: 5
4.8
4.8 Trend (Kendalls Test, Spearman's
4.8 Rho, Cox-Stuart), 4.9

4.8

4.7 Linear Coefficient , Noether's Cyclical) 4.7

Seasonality (Kruskal Wallis Test, Chi-Squared-Mod- 4.6
4.6 4.6
Test, F-Test, ACF-Heuristic) 4.5 4.6
4.5 Noise (Cox-Stuart Dispersion, Runs (Mean), Runs
2 4 6 (Median),
8 10 Runs 12
(Up-Down) 1 2 3 4 5 6 7 8 9 10 11 12
Model form (Level Shifts, Outliers, nonlinearity ) 1 2
Series Characteristics (Zero Values Intermittent
Demand, Length New Products)
Together with data transformations (detrend / deseasonalise, log transforms etc.)
 Baseline Data Analysis & Exploration
Data Exploration in SAP APO DP

Create simple
seasonal diagram as
planning book

Program interactive
graphics container
with multiple
diagrams for APO

Together with data transformations (detrend / deseasonalise, log transforms etc.)

 Baseline Data Analysis
Show prediction
Understanding Regular Patterns
intervals UK Gross Domestic Product: chained volume measures
650 2000000

600
1500000
550
Units

GDP
500 1000000

450
500000
400

350
Show multiple
0
20 40
step60ahead80
Month
1948 1958 1968 1978 1988 1998 2008 2018
Year
UK forecasts
UK Hourly Electricity Demand
Demand UK Hourly Electricity Demand
UK
110000 110000

100000 100000

90000 90000
Demand

Demand
80000 80000
Demand

Demand
70000 70000

60000 60000

50000 50000

40000 40000
10/26/08 10/27/0810/28/0810/29/08
10/26/0810/27/08 10/28/08 10/29/08 10/30/08 10/31/08
10/30/0810/31/08 10/26/08 10/26/0810/27/08
10/26/08 10/27/0810/28/0810/29/08 10/30/0810/31/08
10/28/08 10/29/08 10/26/08
10/30/08 10/31/08
Day Day

Together with data transformations (detrend / deseasonalise, log transforms etc.)

 Baseline Data Analysis & Exploration
Graphics in other software packages

Forecast Pro

Not every fancy visualisation is helpful!

Underdeveloped
Graphics!
 The Forecasting Process
Model Selection & Parameterisation

Maintain Analyse & Decision on Analyze

master data understand automated time series
& link with assortment forecasting Identify
& clean data
predecessors (ABC / XYZ) suitable
models &
parameters
Evaluate
forecast
quality
Apply
statistical
/ manual
Adjust & forecasting
release
forecast
Baseline Model Selection
Why Model Selection?
 Baseline Model Selection
Model Selection in APO DP

Various selections required in SAP APO DP:

Methods: Exponential Smoothing: Single, Trend, Seasonal, Trend-Seasonal,
Linear regression, Seasonal Linear Reg.,
Choice of 98 forecasting methods in SAP APO DP!

Supports mainly
a manual Model
2 options for Selection by Forecaster
afully automaticAlpha [0,,1] , Beta [0,,1], Gamma [0,,1]
Parameters:
Model Selection Choice of 100x100x100 parameter combinations!

14ETS models x 100 x 100 x 100 parameters

offer 14.000.000 choices per time series for each planner!
 Baseline Model Selection
Model Selection in APO DP
2 Options for Automatic Model Selection in SAP APO DP (2 magic buttons?)
Option 1 NOR Option 2 always work ( after testing)
Mean Improvement average
Judgmental FC 0% for X
FC Pro -1% for X
Intelligent Forecaster -1% for X
AMSP2 SAP APO -3% for X
Judgmental FC 3% for Y
FC Pro 1% for Y
Automatic Model Intelligent Forecaster -9% for Y
Selection in SAP APO DP AMSP2 SAP APO -81% for Y
does NOT work robustly! Judgmental FC 33% for Z
Intelligent Forecaster 21% for Z
FC Pro 15% for Z
AMSP2 SAP APO -65% for Z

Works sometimes, but not always cannot trust the system!

Cannot run automatically in the background!
 Baseline Model Selection
Model Selection in APO DP

Solutions exist
Train demand planners in model selection
Use bolt-on systems for model selection APO-DP

Forecast Profile Model Type Mosel-specific Settings

Can train demand planners (preferred!)

Can integrate systems to do model selection for you
Poor Model Selection!
 The Forecasting Process
Model Application
Initialization via overall analysis

Monthly
Maintain Analyse & Decision on Analyze
statistical
master data understand automated time series forecasting
& link with assortment forecasting Identify
& clean data
predecessors (ABC / XYZ) suitable
models &
parameters
SAPAPO DP
Evaluate
includes some
forecast
qualitymodels which
dont work Apply
statistical
SAPAPO DP / manual
excludes
Adjust some
& forecasting
Finalization release
of forecast
important
forecast
models
Baseline Model Application
Models in APO not functional
Forecasting Methods

Statistical Judgemental

Time Series Causal Individual

Double ETS
Naive Methods Expert Opinion
(2nd order ETS)
Linear Trend ETS Moving Average

Linear Regression Single ETS

Causal Dynamic
Trend-Seasonal Seasonal ETS Regression (ARX)
ETS (multip.) (multiplicative)
Seasonal Linear
Regression

Median Method
 Baseline Model Application
Models in APO not functional

Exponential Smoothing require an Initialisation to forecast

poor Nave Initialisation will impair forecast

Out-of-sample Out-of-sample
140 140

130
130
120
120
GDP

GDP
110
110
100
100
90
Data Data
80 APO 90 APO
External External
70 80
5 10 15 20 25 30 35 40 45 5 10 15 20 25 30 35 40 45
Month Month

3 years in-sample not enough to forget bad initialisation, requires higher smoothing
parameters Filter noise adequately?

Initialisation problem for ETS trend models is significant better avoid them!
Similarly: avoid 2nd order Exponential Smoothing
 Baseline Model Application
Models in APO missing

Exponential Smoothing Models in APO DP

-Seasonal Models

In APO DP there is only multiplicative seasonal exponential smoothing. When there

is no trend or level shifts there is little difference between additive and multiplicative
models; however, this is not the case for trended time series.
Additive Seasonality Multiplicative Seasonality
Out-of-sample Out-of-sample

APO
1000 cannot run 1000

additive seasonal
800 800
model
600 600

400 400

200 200
Seasonality
0 0
expands as level
0 10 20 30 40 50 60 0 10 20 30 40 50 60
increases

This restricts the valid usage of the seasonal trend model in APO DP
 Baseline Model Application
Models in APO missing
Exponential Smoothing Models in APO DP
-Seasonal Models
Because APO has only multiplicative seasonal EXSM, any time series that has
values close to zero becomes impossible to forecast.
At
Multiplicative Seasonality
Out-of-sample
Lt (1 ) Lt 1
350 St s
300
At
250 St 1 St s
200 Lt
Level Component
150 1500

100
1000
50
0 10 20 30 40 50 60
Multiplicative Seasonality 500
Out-of-sample
2000
0
10 20
Seasonal 30
Component 40
1500
Effect of 2

zero
1000 1.5

1
500
0.5

0 0
0 10 20 30 40 50 60 10 20 30 40

Very small values break down multiplicative seasonal models use

additive
 Baseline Model Application
Models in APO missing

Forecasting Methods

Statistical Judgemental

Time Series Causal Individual Group

available Jury of Exec.

Naive Methods Expert Opinion
in SAP Opinion
APO DP Structured
Averages Delphi
Analogies
Exponential Intervention Models Prediction
PERT
Smoothing (IMA) (Smoothing IMAX) Markets
Dynamic Causal Dynamic Market
Regression (AR) Regression (ARX) Surveys
available in ARIMA ARIMAX available in
ForecastPro and SARIMA and SARIMAX Autobox
Comp. Intelligence & Causal CI & Choose method according to
Neural Networks Neural Networks forecasting problem & data
Simple methods often

perform equally well
Problem dependent
 Causal Modelling
That can get more complicated...
250
Sales
Forecast
200 32%
150
Units

68%
100

50

0
10 20 30 40 50 60 70 80 90 100
Period

y 25.36 0.40 yt 1 0.42 yt 2 0.24 yt 3 98.23(Pr omo1) 46.98(Pr omo2)

1

Promotion 1 0.5

0
10 20 30 40 50 60 70 80 90 100

Promotion 2 0.5

0
10 20 30 40 50 60 70 80 90 100
 The ScanPRO model
n p rj 3 T K
kjt
Qkjt krt lrjDlkrt jtXt Zk kj e
r 1 pkrt l 1 t 1 k 1
Cross-effects between products Interactions & cannibalism
Cross-effects between promotions Interactions, support & cannibalism
Cross-effects between stores Spatial cannibalisation & competition
6000

5000

4000
Sales

3000

2000

1000

0
5 10 15 20 25 30 35 40 45 50 55 60
2000
Period

+ Effect 1000
+ Effect
0

- Effect -1000 - Effect

-2000
5 10 15 20 25 30 35 40 45 50 55 60
Period
 APO DP Best Practices and Limitations
Croston Intermittent Demand

Croston method is designed to deal with these type of products.

Demand Only 5 months of data!

20 20

15 15

10 10

5 5

Intermittent Time Series 0 0

1 2 3 4 5 1 2 3 4 5 6
20
15 Forecast them with the same single
10
exponential smoothing model
5
0
Time Interval
Mar-05
May-05

May-06
Jan-05

Jan-06
Mar-06
Nov-05

Nov-06
Jul-05
Sep-05

Jul-06
Sep-06

8
8
6
6
12 months of data 4
4
2 2

0 0
1 2 3 4 5 1 2 3 4 5 6

Count every how many

months there is demand

Advanced Intrmittent methods are missing, e.g. neural nets,

Bootstrapping, Zero-inflated demand
Limited Model
Functionality!
 The Forecasting Process
Statistical Modelling: Model Application
Initialization via overall analysis

Monthly
Maintain Analyse & Decision on Analyze
statistical
master data understand automated time series forecasting
& link with assortment forecasting Identify
& clean data
predecessors (ABC / XYZ) suitable
models &
parameters
Evaluate
forecast
quality
Apply
statistical
/ manual
Adjust & forecasting
Finalization release
of forecast forecast
 Baseline Data Analysis & Exploration
Allows various Data Exploration in SAP APO DP
overrides for
promo planning,
history correction,

Can add notes

No explicit systems support of judgmental decision making in system!

 HUMAN CAUSES THE PROBLEM THE REMEDIES

Rational
Human
Causes
e.g. biases Sub-optimal Side effects:
Training Lower user
from company sales forecast Decompose Judgment
acceptance
policy
accuracy from Dont adjust
Human Use Analogies
Irrational Develop systems
Human Causes
Judgment
FSS features to
limited support judgment
information FSS features to
processing How to avoid sys- constrain judgement
capacity tematic errors in
final forecast?

Support Judgement with Analogous Information

Improve Forecasting Support Systems (FSS)
 Adaptation
judgment
support
Similarity
judgment
support

Memory
support

Test what type of support works best using ANOVA

Interactively select & store similar promotional cases
Limited Support
of Judgment!
 The Forecasting Process
Statistical Modelling: Model Application
Initialization via overall analysis

Monthly
Maintain Analyse & Decision on Analyze
statistical
master data understand automated time series forecasting
& link with assortment forecasting Identify
& clean data
predecessors (ABC / XYZ) suitable
models &
parameters
Evaluate
forecast
quality
Apply
statistical
/ manual
Adjust & forecasting
Finalization release
of forecast forecast
 Model Evaluation
Misleading Error Measures

650

600
In-sample data Out-of-sample 1 forecast origin
data only 1 measurement
550
low confidence in the
Units

500
measurement accuracy
450
BUT: more data (24) is
400
Forecast origin available
350
10 20 30 40
40 50
50 60
60 70
70 80
80
Month

Assess accuracy correctly: out of sample, rolling, good measures!

 Model Evaluation
Model Wrappers
Fit all possible models and measure the errors:
1 n
MSE Ai Fi Robust versions
2

n i 1 of MAPE exist
1 n Ai Fi 1 n Ai Fi
MAPE sMAPE
n i 1 Ai n i 1 ( Ai Fi ) 2
1 n
MAE Ai Fi
n i 1

At Ft
REt
At FNaive use Information Criteria
1 Relative Errors
n Ai Fi n for model selection?
GMRAE missing
AIC 2k 2 ln( L)
i 1 Ai FNaive
BIC 2 ln( L) k ln(n)
1 n At Ft
MASE
n i 1 MAEin sample1 stepahead Naive

Assess accuracy correctly: out of sample, rolling, good measures!

 Model Evaluation
Model Wrappers

Visualisation of error
distributions to check for
errors and bias for
relevant lead time!

Assess accuracy correctly: out of sample, rolling, good measures!

 The Forecasting Process
Triage of Baseline | Promo | New

Baseline Planning

Promotions Planning

New Product Planning

 Take aways
Companies are uisng SAP APO withmultiple other
systems and still use simple methods & little data
does not reflect Marketing hype & S.o.t.A.!
Data exploration can be enhanced
poor graphics & visualisation customisable?
Model Selection can be enhanced
enough model selection & models for you?
Judgmental Adjustments are not supported well
smart notes to support & costrain judgment!

Review your SAP APO DP setup

possibly new releases / customisation flawed?

making SAP APO DP work for you!

 Discussion?
Questions?

Dr. Sven F. Crone

Assistant Professor, Director
Lancaster University Management School
Research Centre for Forecasting

Lancaster, LA1 4YX

Tel. +44 1524 5-92991
s.crone@lancaster.ac.uk
Copyright
These slides are from a workshop on forecasting.

You may use these slides for internal purpose, so long as they are
clearly identified as being created and copyrighted by Sven F.
Crone, Lancaster Centre for Forecasting.

You may not use the text and images in a paper, tutorial or external
training without explicit prior permission from the centre.

Dr. Sven F. Crone

Assistant Professor, Director
Lancaster University Management School
Research Centre for Forecasting

Lancaster, LA1 4YX

Tel. +44 1524 5-92991
s.crone@lancaster.ac.uk