Talend Data Catalog

Business Glossary Workshop

TOPICS
• Talend Data Catalog (TDC) Glossary overview
• The Glossary Metamodel
• Multiple Glossary models

• Semantic Mappings & Data Models

• Glossary Governance
• Options for Glossary edit
• Glossary Workflow
• State and Transition options
• Security controls
• Stewards & Notification

2
Glossary Overview
TDC GLOSSARY “METAMODEL”

• The metamodel defines:

• the types of items that may be created in the Glossary model,
• the types of relationships that may be created between the items, and
• the attributes of the items.
• The following two slides illustrate the TDC Glossary metamodel
• Note that TDC also permits any number of additional “custom attributes” to
be defined against any of the types, in order to hold customer specific
information
• E.g. for WOW, the “Formula” attribute will be implemented as a custom attribute
• Such new custom attributes are set up by a TDC administrator, but may be
populated by any user with update rights to an item

4
GLOSSARY METAMODEL
• ISO 11179 standard – based
Glossary
1
• A Glossary Model consists of:
• * 1
A Glossary, containing
• A hierarchy of Categories, containing or referencing Category *
• Terms
•
1 *
and their relationships
Owns References
* *
More 1 *
General
More
Term Represented By
- type Represents
Specific * *
* 1 * See *
Contains Contained Also Synonym
By

5
METAMODEL PROPERTIES
Glossary
Comment - name
- Comment - model name
-
* 1 - description *
Endorsement *
- Warning comment - last modified * Label
- Certification
* number
1 Category datetime
boolean
- name * * List binary
* - definition
- char
last modified
User and/or * text
steward int
User Group undefined
* Business
Term Entity
- name Attribute
Domain Candidate
1 - definition Draft
- data type Possible Value
Under Review
* - type
* Pending Approval
Documentation - abbreviation
- text 1 1 - alternative abbreviation New
Approved
Published
- status Published Deprecated
- state Deprecated
- used
- last modified false
true 6
7
GLOSSARY MODELS
• Multiple Glossaries may be implemented
• These may be:
• Either within one TDC Glossary Model, under different top-level Categories
• Necessary if you want Terms/Categories of one Glossary to reference Terms of the other
• But restricts you to using the same Glossary state model for all glossaries in that model
• Or as separate TDC Glossary Models
Glossaries Model
Corporate

WOW Glossary Jenius Glossary Jenius

EQ Glossary
a b c
WOW EQ Jenius
WOW EQ

a b c x y c x y c
a b c x y c x y c8
Semantic Mappings & Data Models
MAPPING TERMS TO TECHNICAL METADATA ITEMS
• Terms may be mapped to the physical metadata items which are implementations of that
concept e.g. mappings to:
• A database column or table
• A file field or record
• An Excel file column or worksheet
• Report fields
• Etc.
• The mappings are created from the physical metadata items to the Terms they implement
• And so typically will be set up by technical users, NOT by business users
• Typically the mappings will be create to the items of one physical metadata model (e.g. a relevant
data warehouse or data mart)
• These mappings enable business users to have access to relevant technical metadata item
definitions, including seeing the lineage and usage of these technical items (see diagram on
following page).

10
 Business Users

Talend Data Catalog

Glossary

lineage usage (= impact analysis)

Flat
file
ETL ODS ETL DWH ETL Mart BI

xml

Technical Users

11
SEMANTIC MAPPINGS
Mapping across the conceptual levels Enterprise
• down Term/Term ’more specific’ relationships Term
Glossary
• down Term/Term ‘represents’ relationships
• between (Terms of) two Glossaries
Sem. Map
• between a Glossary and a physical model
• E.g. database schema, flat file, BI model, PDM
• between a Glossary and a Logical Data Model Project Term
Glossary Term
• E.g. ERwin, ER Studio, PowerDesigner etc.
• from the LDM to its physical model
Glossary Sem. Map

More 1 *
General Represented By
More Term LDM
Specific * Represents
Table
DB Model Column

Database 12
TRACING SEMANTIC USAGE
Enterprise
Term
Glossary

Project Term
Glossary more general
represented by

represents more specific

Term Term

Dimension Column
Column
pass thru
transform Column pass thru Column

transform

13
TRAINING SCENARIO

14
POPULATING THE GLOSSARY
• Import from a formatted CSV file, via the
Metadata Explorer interface UI CSV
• Manually via the Metadata Explorer import

interface
• From an existing data model OWL
Glossary
• from tools such as ERwin, PowerDesigner, import import
UML
ER/Studio, Oracle Designer Model
• From a UML modelling tool import UML
import
• From a W3C Web Ontology Language (OWL) file Data
Model Data
• From a physical data model import Model
• Manually, via classification of a physical import
Model
model, to fill gaps detected in the Glossary
UI TDC
15
Glossary Governance
SECURITY SETTINGS – CONCEPTS RECAP

Repository Models & Glossaries

• Users • Stewards
• Groups

Model, Configuration & Folder roles

• Viewers Glossary Workflow roles
• Editors • Editors
• Data Viewers • Reviewers
• Managers
• Approvers
• Security Managers
• Data Managers
• Publishers

17
OPTIONS FOR GLOSSARY TERM EDITING
1. All users granted ‘Editor’ rights on a
Glossary may edit any item
2. Only user(s) assigned as ‘Steward’ of a Term
or Category may edit it
3. Enable Glossary Workflow:
• users with ‘Editor’ role on a Category which
contains a Term may edit that Term

18
1. ALL USERS WITH EDITOR RIGHTS
• Loosest update control mechanism
• Users may be assigned Editor rights to the whole TDC Glossary model
• Either directly
• Or preferably (best practice) via a User Group

• It is possible to monitor updates to Terms and Categories:

• Every Term has an Audit Log, capturing who made what changes
• Each Term/Category may be assigned one or more ‘Stewards’ (users or groups),
who will be notified via email every time that Term is edited.

19
2. ALLOW ONLY STEWARDS TO EDIT
• Tight control on update, but NO validation by a separate approver
• Select, for the Glossary model, to allow editing only by Stewards
• All users/groups to be assigned as Stewards must also be given Editor rights on
the Glossary model
• Applies to edit of Category items as well as to Terms
• Creating a new Term in a Category will, by default, assign the Steward(s) of the
owning Category to be the Stewards of the new Term also
• Only a Steward of a Term/Category is permitted to edit it.

20
3. WORKFLOW CONTROLLED UPDATE
• Tightest update control, with configurable workflow process steps
• Choose which steps are appropriate for your governance process
• Each step has an associated role to which users/groups are assigned
• Editor / Reviewer / Approver / Publisher
• These roles are assigned at the Category level, with inheritance down the category
hierarchy. The roles apply to update of all Terms owned by the category.

21
WORKFLOW STEP CONFIGURATION
• New Terms may, optionally, be proposed Allow approval of terms
by any user with Editor permission on Allow review of terms
the whole Glossary model: Publish on Approval

Allow approval of terms

Allow any user to propose terms Allow review of terms
Publish on Approval

Allow approval of terms

• Five possible step configurations: Allow review of terms
Publish on Approval

Allow approval of terms

Allow approval of terms Allow review of terms
Publish on Approval 22
ROLE RESPONSIBILITIES THROUGH THE WORKFLOW (1)
Editor Reviewer Approver Publisher
Submit for
Option 1:
Approval
Draft Allow approval of terms
Delete /
Discard Start Allow review of terms
Review Under Publish on Approval
Review Submit for Approval /
Recommend Approval
Send to Draft /
Mark for Request Change Pending Approve
Deprecation
Approval
Send to Draft /
Reject
Approved Publish Published
Send to Draft /
Reject
Deprecated
23
ROLE RESPONSIBILITIES THROUGH THE WORKFLOW (2)
Editor Approver Publisher
Submit for
Option 2:
Approval
Draft Allow approval of terms
Delete /
Discard Allow review of terms
Publish on Approval

Mark for Pending Approve

Deprecation
Approval
Send to Draft /
Reject
Approved Publish Published
Send to Draft /
Reject
Deprecated
24
ROLE RESPONSIBILITIES THROUGH THE WORKFLOW (3)
Editor Reviewer Approver
Submit for
Option 3:
Approval
Draft Allow approval of terms
Delete /
Discard Start Allow review of terms
Review Under Publish on Approval
Review Submit for Approval /
Recommend Approval
Send to Draft /
Mark for Request Change Pending
Deprecation
Approval Publish

Send to Draft /
Reject
Published

Deprecated
25
ROLE RESPONSIBILITIES THROUGH THE WORKFLOW (4)
Editor Approver
Submit for
Option 4:
Approval
Draft Allow approval of terms
Delete /
Discard
Allow review of terms
Publish on Approval

Mark for Pending

Deprecation
Approval Publish

Send to Draft /
Reject
Published

Deprecated
26
ROLE RESPONSIBILITIES THROUGH THE WORKFLOW (5)
Editor Publisher
Option 5:
Approve
Draft Allow approval of terms
Delete /
Discard

Mark for
Deprecation

Approved Publish Published

Send to Draft /
Reject
Deprecated
27
CONSIDERATIONS

• Too many steps makes process execution cumbersome, and completion of

the process slow (waiting on input from multiple users in series)
• Note that the “Under Review” state, if used, is always optional.
• From “Draft” the Editor may choose transition
• “Start Review”, to go to Under Review, or
• “Submit for Approval” to transition direct to Pending Approval.

28
EXAMPLE SECURITY
SET-UP, CONTROLLING,
FOR EACH ROLE, AT
THE CATEGORY LEVEL
OF GRANULARITY

Is this low level of

granularity required?
Can the assignments to
roles be at the level of
the whole glossary,
rather than for each
separate Category?

29
GLOSSARY WORKFLOW OVERVIEW

• Workflow can be turned on (or not) for each Glossary independently

• You choose which Glossaries to put under workflow control, and which not
• Typically you will only apply workflow control once the contents of the Glossary
have reached a certain level of maturity and adoption within the organisation
• The exact workflow steps to be followed by all workflows of a Glossary are
configurable for each Glossary independently
• Once saved, the above settings cannot be changed for that Glossary
• For each Category within a Glossary, the exact users involved, and the roles
each may play in the workflow, can be set up independently and changed at
any time
• Workflow control applies to Terms, but not to Category objects
30
STEWARDS
• Users or user groups can be assigned as Stewards of a Term or a whole
Category of terms, as their domain expert(s)
• Stewards play no specific role in a workflow, other than as experts who
may be consulted about changes
• (… unless they are otherwise specifically assigned to be
Reviewers/Editors/Approvers/Publishers)
• Stewards are notified of changes to their Terms

31
NOTIFICATION
• Users may also receive notification emails concerning workflow state
transitions that invite their participation in the next step of a workflow
process execution

• To enable this:
• SMTP connection must have been set up
• a user/group email address must be set
• the user must be a Steward (on the Tools -> Admin Users/Groups tabs)
• Stewardship can be defined directly in the user account definition, or
in the definition of a group they belong to that is involved in the
workflow (i.e. has been defined as an Editor of the Glossary model)
32
QUESTIONS

• How do you wish to control Glossary updates, now and in the future?
• Application Administrators only?
• Glossary model Editors only?
• What User Group(s)?
• Term/Category Stewards only?
• What User Groups are needed for the Steward assignments on each Term and Category?
• Workflow?
• What workflow states do you wish to include?
• What User Groups are needed for the role assignments, for each Category in each Glossary?

• Do technical users need update access as well as business users?

• Do we wish to integrate via SMTP with email to notify stewards of updates to their Terms?

33