Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                

TDL Vs TDH

Download as pdf or txt
Download as pdf or txt
You are on page 1of 17

Temenos Data Lake vs.

Transact Data Hub


• No confusion with “Data
Why are we introducing TDH? Lake”
• TDH optimized for Transact
• Transact data only
• Integrated architecture
In 2019 we launched the Temenos Data Lake as an enterprise data • Integrated security
lake product which is meant to allow a bank to integrate multiple • Data ExStore + event driven
data sources into a banking specific big data platform to enable + multi-db + cloud, etc. etc.
reporting, analytics, and AI use cases. After launching Temenos
Data Lake we received many client to satisfy Transact needs only,
thus we decided to launch a product which focusses specifically to
the requirements of Transact. This new product, Transact Data Hub,
works on an optimized version of the Temenos Data Lake
technology platform but has tightly integrated capabilities with
Transact.

Transact Data Hub makes Transact more attractive to prospects


and easier to differentiate from the competition.
What is Transact Data Hub?

Transact Data Hub

}
Semantic Query Layer

Data Stores (Operational/Snapshot)


Major components
Data Engineering

Data Event Streaming

Metadata
Management
Master Data
Management
Data Quality

XACML Security
Data
Governance

Scheduler
Customer Data
Protection

Reconciliation } Platform Capabilities

Transact

3
Transact Data Hub

Transact Data Hub

Operational
Data Store
Data
Data
Event
Transact Engineering
Stream
Snapshot
Data Store

API Layer Other Access


2. Operational Data Store 3. Snapshot Data Store APIs 4. Snapshot Data Store
1. Streaming data Data files ODBC
APIs (Near Real-Time) (Time -1 day) APIs (Time – n days)

Bank
Cloud and/or
On-Prem
ETL Reporting APIs Apps
AI
} Bank
Requirements

4
Transact Data Hub Architecture

API Browser User Agents UXP Query & Reporting Embedded Analytics

Interaction Framework
APIs

Record Temenos Analytics


Keeping
Data Engineering Semantic Query Reporting KPIs
Layer
Data Event

Transact Analytics Embedded


Stream

Data Data
Transform Parsing
Operational Snapshot
Data Data Data Store Data Store
Cleansing Enrichment Analytical
Data Store

Transact
Data Hub

5
Temenos Data Lake Architecture
Browser Interaction Framework APIs User Agents
Embedded
Queries Channels Query and Reporting Analytics

Temenos Data
Other Analytics
Engineering
Record Sources
Data Storage and
Reference Master &
Keeping Data Derived Data Reporting
CBS Data Data
Raw Data
Artificial Intelligence
Framework

Digital Raw Cleansing Enrichment


Machine Learning
Data Temenos ODS Relational
Data

Wealth Data Prescriptive


Data Event

ETL Replication
Kafka Blending and
Funds Admin In-memory
Snapshots
Predictive
Publish/ User Defined
Data Quality Analytics
Payments Subscribe Functions Diagnostic

FCM Java Database Descriptive


Connectivity Data Lineage
Read Reporting
Relational Micro
Replication
ETL
Services Apps
DLM NuoDB

ESB B2B Other


Systems

Data Governance
Security
Temenos Data Lake
6
T24 Transact + Infinity Components
What is Transact Data Hub?
Transact Data Hub (TDH) is a packaged data platform solution for Temenos Transact banks to stream,
transform, store, optimize, and distribute Transact data into a variety of formats and to solve
numerous data driven problems which banks face.

TDH provides banks with access to 4 main formats of Transact data:


1. Streaming Data – Access to real-time data events and integration messages via a stream
processing platform such as Apache Kafka, Amazon Kinesis, and Azure Event Hub
2. Near Real-time Relational Data – Access to a relational replication of Transact data which is
updated in near-real time via streaming data messages
3. End of Day Relational Data – Access to a snapshot replication of Transact data which has an
“end of day” state with data that has been updated via the close of business processes
4. Temporal Snapshot Data – Historical snapshot retention which stores end of day snapshots in an
optimal format which is critical for business to perform reconciliation, regulatory and audit
requirements.

TDH provides access to data in several methods


▪ Via APIs – TDH data is available via IRIS or Analytics API Services
▪ Direct database access – For on-premise and TDH on client cloud direct ODBC access to TDH
data sources is available. For TSS clients access data via APIs
▪ Direct stream access – Clients can subscribe to streaming data topics such as Apache Kafka,
Amazon Kinesis, or Azure Event Hub
▪ Files – TDH can produce files in any format for bank consumption
What is Transact Data Hub?

TDH provides banks with important data platform tooling functionality allowing them to take full
advantage of data within Transact by providing capability such as:
1. Data Engineering – in-memory distributed data processing layer allowing banks to manage
existing or create new data and ETL pipelines and easily integrate with other systems such as
data warehouses or data lakes, and provide curated high quality datasets to any consuming
system
2. Metadata Management – full data catalog and data lineage capabilities allowing for better
data governance and regulatory compliance
3. Data Quality Management – solution to ensure the highest accuracy of Transact data which
enhances both Transact itself and all consuming reporting, analytics, or AI applications
4. Semantic Query Layer – GraphQL based semantic layer allowing for performant APIs to be easily
created and managed from TDH databases
5. Integrated Security – XACML based security mechanism is fully integrated with Transact ensuring
integrated authentication and authorization
6. Reconciliation – TDH provides multi-level reconciliation, ensuring data integrity and fulfilling
regulatory requirements
What is Transact Data Hub?

▪ TDH is based on a scaled down version of Temenos Data Lake technology. This allows customers
to upgrade to the capabilities of the Temenos Data Lake in the future with no additional software
installation

▪ TDH can run on multiple database including (PostgreSQL, SQL Server, Azure SQL, nuoDB, and
Oracle)

▪ TDH can be deployed on premise or on Azure, AWS, and Google cloud platforms. TDH uses
container technology to enable compatibility across different cloud platforms

▪ TDH can be deployed in as low as 60 days. This assumes a standard implementation and existing
setup of underlying infrastructure

▪ TDH can be deployed on commodity hardware, as low as 8 CPUs and 32GB of memory

▪ TDH can be combined with data hubs from other Temenos products (coming soon), allowing
data from Transact to be blended with multiple Infinity data sources for added value
What is provided with TDH Out of the Box?

TDH ships with the following packaged functionality out of the box:

▪ Pre-configured data flows which export 122 tables from Transact and populate 2 optimized data
stores (Operational Data Store and Snapshot Data Store)
▪ Operational Data Store is a near real-time replication of Transact data into a relational
database format with pre-parsing of multi-value, multi language and local reference
fields
▪ Snapshot Data Store provides an end of day incremental/full snapshot of Transact data
including data fields which are calculated in the close of business processes such as
financial accounts, accrued interest, and effective interest rates. Snapshot Data Store
also retains history of end of day snapshots
▪ 175 banking APIs which are based on data held within the data stores. These APIs are highly
performant and can optimize Transact enquiries or other consuming user agents
What is provided with TDH Out of the Box?
TDH contains robust data platform functionality including:

▪ Data Event Streaming – A streaming data application that exports data messages from Transact onto a streaming platform such as Apache
Kafka
▪ Data Engineering - An in-memory data processing application that parses, cleanses, and optimizes Transact data and loads it into downstream
databases
▪ Operational Data Store (ODS) - replicates data from Transact, in near real-time, into a separate database in relational database format which
can be used for APIs, enquiries, or other reporting and analytics use cases
▪ Snapshot Data Store – provides a snapshot of data at the end of day, immediately post Transact close of business, which contains calculated
data fields such as financial accounts, accrued interest, and effective interest rates
▪ Metadata Management – this includes 2 components which are essential for data regulation compliance: 1) a complete data catalog which
inventories all data held within the TDH system 2) a data lineage capability which provides traceability of all data from source to destination
▪ Data Quality Management – an ability to identify data quality issues within Transact and update Transact data in near real-time.
▪ Semantic Query Layer – a GraphQL based query layer allowing for metadata driven queries of TDH data sources which form the basis of data
for APIs
▪ Customer Data Protection – aligned with the Transact CDP module this replicates CDP configuration for GDPR data compliance within TDH
▪ Integrated Security – aligned with the Transact CDP module this replicates CDP configuration for GDPR data compliance within TDH
▪ Administration – Provides comprehensive administration capability around Transact data pipelines.
▪ Scheduling – Enables scheduling of extracts on different data hub data stores with complete automation.
▪ Deployment – One click deployment of workflows across DEV, SIT, UAT and PROD.
▪ Reconciliation – TDH provides multi-level reconciliation, ensuring data integrity and fulfilling regulatory requirements

11
What does TDH not include?

TDH does not include:

▪ Reporting or Analytical Content - TDH does not include any reporting, dashboards, BI
applications, or analytics. This can be added by the purchase of Temenos Analytics. With
Temenos Analytics the bank receives the Analytical Data Store, the Analytics web application,
and various reporting and analytics content packs. Temenos Analytics can be sold on top of TDH.

▪ Ability to integrate and process “big data” such as unstructured data sources like video, audio,
images, etc. This functionality can be added by upgrading to Temenos Data Lake.

▪ Direct integration with XAI. The Data Engineering components which connect to XAI for training
or scoring are included with an upgrade to the full Temenos Data Lake product

▪ Direct integration with Infinity. We will introduce a new product called Infinity Data Hub in 2021
which will include the integration of Infinity sources.

▪ Ability to integrate non-Transact data. TDH is used for Transact data only. The ability to integrate
non-Transact data is included with an upgrade to the full Temenos Data Lake product.*

*Currently some exceptions are made for ingesting legacy core banking data
What are the main use cases for TDH?
Transact Enquiry Performance – Certain Transact enquiries run slow on the main Transact (OLTP) database. TDH includes the Operational Data Store,
which is optimized for query performance and can optimize enquiry performance up to 10 times.

Performant Banking APIs - Certain Transact APIs run slowly on the main Transact (OLTP) database. These API’s now can be directly consumed by
Operational data store through semantic query layer which can improve performance up to 10 times.

Data Extraction – Banks require Data held within Transact to populate a myriad of other systems within the bank. TDH allows for real-time extraction,
preparation, and optimization of Transact data to support many data extraction use cases.

CSV creation – TDH includes a capability to product CSV files from Transact data which is often a requirement for batch integration with external systems
inside the bank and to external vendors

Population of bank Data Warehouse – Bank data warehouse interfaces are often very complex and require extensive data transformation to meet
the target data warehouse schema. TDH, with Data Engineering, can push data to bank data warehouses or banks can use their own ETL tool to connect to
data streams or TDH data stores to populate the data warehouse.

Reporting – TDH data stores, Operational Data Store and Snapshot Data Store, can be a source for bank reporting. However, the Analytical Data Store,
which sells with Temenos Analytics, further optimizes Transact data for many types of bank reporting and analytics. **

Legacy Core Data – TDH can import data from the legacy (pre-conversion) core banking system into a TDH data store. This allows for many use cases
such as historical customer statements, reconciliation, post-live retrieval of legacy core data within Transact, internal audit review of historical fraud, and
others.

AI – Data scientists can connect to TDH data streams or data stores to access the Transact data they require for loading into their feature engineering
environments and model training environments.**

Data Archiving – TDH keeps historical daily snapshots of Transact data which can be used for any use case where historical end of day data is required
such as reporting, audit, integration, customer enquiries, etc.

Internal Audit – Bank audit groups will often need to access historical data when investigating various fraud or other audit scenarios.

** - Additional licenses might be required to achieve OOB data for these use cases. 13
Transact Data Hub vs. Temenos Data Lake
Feature Comparison
Temenos Data Hub Temenos Data Lake Details
Main Use Case Transact Data Export and Preparation Big Data Platform for Digital Banking Analytics
Technology Stack TDL Technology Stack TDL Technology Stack
Transact Data Integration Yes Yes
Multi-source data integration No Yes Legacy core data can be integrated into Transact Data Hub
Data Profiling Yes Yes
Data Blending Yes Yes TDH supports data blending of Transact data only
Big Data Solution No Yes
Support Hadoop platform No Yes
Unstructured data support No Yes
Dedicated XAI integration components for model training and scoring are
XAI Integration No* Yes
included in TDL
Dedicated Infinity integration components for marketing campaigns and
Infinity Integration No* Yes
notifications are included in TDL
Cloud Ready Yes Yes
Multi-cloud support Yes Yes
TDH should allow at least ODS databases write for a few selection, e.g. SQL,
Multi-database support Yes Yes NUODB etc. This is so that client can then integrate to their other ETLs on other
sources systems.
Minimum Infrastructure requirements 8+ CPU, 32GB Memory 16+ CPU, 64GB Memory
Operational Data Store Yes Yes
Snapshot Data Store Yes Yes
ADS is included with Temenos Analytics and is deployed on the TDH / TDL
Analytical Data Store Yes Yes
technology stack
Transact Integration Support R17+ Yes Yes Required DES
Transact Integration Support R16- Yes Yes Requires DW.Export Online purchase and or upgrade
Temenos Analytics can run on TDH or TDL however on TDH reporting and
Data platform for Temenos Analytics Yes Yes
analytics is restricted to Transact data
Can be upgraded from Data ExStore Yes Yes
Can be upgraded from Relational Replication Yes Yes
Metadata Management Yes Yes
Master Data Management No Yes
Reference Data Management No Yes
Data Quality Management Yes Yes Data quality assured for Trsnact ingestion but not the tooling to manage
XACML Integration Yes Yes
DES Design Time Yes* Yes
TDE Design Time Yes* Yes On Transact data only, with write to select DB in ODS and SDS.
Workbench integration Yes* Yes

* For full comparison see “Feature List” spreadsheet 14


TDH Implementation Estimate
Man Days ▪ High level estimate assumes:
Discovery Phase ▪ Out of box configuration (as defined in OOB configuration
Identify/Get the TDL candidates 2 document)
Perform Health Check on TDL Candidates 2 ▪ Data Event Stream already implemented and tested
Apply Data Profiling 2
▪ Qualified and certified implementation teams
▪ Estimates valid for both cloud and on-premise deployments
Build Phase
Infrastructure Setup 5
Deploy TDH Artefacts 2
Parametrisation/Configuration of TDE 2
Add Tables via Deploy Data Store 2

Execution Phase
Perform Initial Load Process 5
Reconcile ILP Data 2
Process Online Transaction 2
Reconcile Online Data 2
Process COB & CSV Extraction 2
SIT Phase 15
UAT Phase 15

Total 60

15
N – Not Available

TDH Feature Matrix NA – Not applicable


Y – Available
NR – Not Required
TDH Features R10 R11 R12 R13 R14 R15 R16 R17 R18 R19 202003 R20 202009 202010 Dev
Transact Data Integration N Y N N N N N Y N Y Y Y Y Y
Cloud Agnostic N Y N N N N N Y N Y Y Y Y Y
Multi-database support N Y N N N N N Y N Y Y Y Y Y
Data platform for Analytics N Y N N N N N Y N Y Y Y Y Y
Operational Data Store N Y N N N N N Y N Y Y Y Y Y
Snapshot Data Store N Y N N N N N Y N Y Y Y Y Y
Semantic Layer N N N N N N N N N Y Y Y Y Y
Metadata Management N Y N N N N N Y N Y Y Y Y Y
XACML Integration N N N N N N N N N N Y Y Y Y
Not Supported
Operational Data Store API’s N N N N N N N N N N Y Y Y Y
Retry and Recovery N Y N N N N N N N N N N Y Y
Initial Load Processing N Y N N N N N Y N Y Y Y Y Y
Local Reference Support N Y N N N N N Y N Y Y Y Y Y
IRIS Integration N N N N N N N Y N N Y Y Y Y
Schema extension N Y N N N N N Y N Y Y Y Y Y
DW Online Integration N Y N N N N N N N N N N Y Y
DES Design Time NR NR NR NR NR N N Y N Y Y Y Y Y
TDE Design Time N Y N N N N N Y N Y Y Y Y Y
Workbench integration N N N N N N N N N N N N N 2021
Reconciliation N N N N N N N N N N N N N 2021
Metrics dashboard N N N N N N N N N N N N N Y
CDP Compliance N N N N N N N N N N N N N 2021

* In the case of N back-patching can be achieved but will incur additional commercial proposal to prospect / client 16
N – Not Available

TDH Stack Matrix NA – Not applicable


Y – Available
NR – Not Required

TDL Stack R10 R11 R12 R13 R14 R15 R16 R17 R18 R19 202003 R20 202009 202010 Dev

Azure Cloud N Y N N N N N Y N Y Y Y Y Y

AWS Cloud N N N N N N N N N N N N N 2021

GCP Cloud N N N N N N N N N N N N N 2021

Oracle Support – ODS/SDS N Y N N N N N Y N N Y Y Y Y

Not Supported
MSSQL Support – ODS/SDS N Y N N N N N Y N Y Y Y Y Y

PostgreSQL Support – ODS/SDS N Y N N N N N Y N N Y Y Y Y

Azure SQL Support – ODS/SDS N Y N N N N N Y N Y Y Y Y Y

NuoDB Support – ODS/SDS N N N N N N N N N N N N N Y

Open shift Container N N N N N N N N N N N N N 2021

On Premise Deployment N Y N N N N N Y N Y Y Y Y Y

Kubernetes Deployment N Y N N N N N Y N Y Y Y Y Y

* In the case of N back-patching can be achieved but will incur additional commercial proposal to prospect / client 17

You might also like