Developer Flat File Schema Developer's Guide: Title Page

Title Page
Developer Version 7.1 Flat File Schema Developer’s Guide

Copyright
& Docu‐
ment ID
Cerebra, Glue, Infravio X‐Broker, Infravio X‐Registry, Infravio, My webMethods Server, My webMethods, webMethods Access, webMethods Administrator,
webMethods Broker, webMethods Central Configuration, webMethods Dashboard, webMethods Designer, webMethods Developer, webMethods Fabric,
webMethods Glue, webMethods Infrastructure Data Collector, webMethods Infravio X‐Broker, webMethods Infravio X‐Registry, webMethods Installer,
webMethods Integration Server, webMethods logo, webMethods Mainframe, webMethods Manager, webMethods Modeler, webMethods Monitor,
webMethods Optimize for Infrastructure, webMethods Optimize for Process, webMethods Optimize, webMethods Portal, webMethods Process Engine,
webMethods Servicenet, webMethods Task Engine, webMethods Trading Networks, webMethods Workflow, and webMethods are either registered
trademarks or trademarks of webMethods, Inc.
Acrobat, Acrobat, and Reader are registered trademarks of Adobe Systems Incorporated. Amdocs and ClarifyCRM are registered trademarks of Amdocs.
Ariba is a registered trademark of Ariba, Inc. BEA, BEA WebLogic Server, Jolt, and Tuxedo are registered trademarks, and BEA WebLogic Platform is a
trademark of BEA Systems, Inc. Action Request System, BMC Software, PATROL, and Remedy are registered trademarks of BMC Software, Inc. BroadVision
is a registered trademark of BroadVision, Inc. Chem eStandards and CIDX are trademarks of CIDX, The Chemical Industry Data Exchange. SiteMinder and
Unicenter are registered trademarks of CA, Inc. PopChart is a registered trademark of CORDA Technologies, Inc. Kenan and Arbor are registered trademarks
of Alcatel‐Lucent. Data Connection and SNAP‐IX are registered trademarks of Data Connection Corporation. D&B and D‐U‐N‐S are registered trademarks of
Dun & Bradstreet Corporation. Eclipse is a trademark of Eclipse Foundation, Inc. Entrust is a registered trademark of Entrust, Inc. papiNet is a registered
trademark of the European Union and the United States. Financial Information eXchange, F.I.X, and F.I.X Protocol are trademarks of FIX Protocol Ltd.
UCCnet and eBusinessReady are registered trademarks, and 1SYNC and Transora are trademarks of GS1 US. Hewlett‐Packard, HP, HP‐UX, OpenView, PA‐
RISC, and SNAplus2 are trademarks of Hewlett‐Packard Company. i2 is a registered trademark of i2 Technologies, Inc. AIX, AS/400, CICS, ClearCase, DB2,
Domino, IBM, Informix, Infoprint, Lotus, Lotus Notes, MQSeries, OS/390, OS/400, RACF, RS/6000, SQL/400, S/390, System/390, VTAM, and WebSphere, and
z/OS are registered trademarks; and Communications System for Windows NT, DB2 Universal Database, IMS, MVS, and SQL/DS are trademarks of IBM
Corporation. InnoDB is a trademark of Innobase Oy. Itanium is a registered trademark of Intel Corporation. Linux is a registered trademark of Linus
Torvalds. W3C is a registered trademark, and X Window System is a trademark of the Massachusetts Institute of Technology. MetaSolv is a registered
trademark of Metasolv Software, Inc. ActiveX, Microsoft, Outlook, Visual Basic, Visual SourceSafe, Windows, Windows NT, and Windows Server are
registered trademarks of Microsoft Corporation. Six Sigma is a registered trademark of Motorola, Inc. Firefox and Mozilla are registered trademarks of the
Mozilla Foundation. MySQL is a registered trademark of MySQL AB. nCipher is a trademark of nCipher Corporation Ltd. Eclipse is a trademark of Eclipse
Foundation, Inc. Entrust is a registered trademark of Entrust, Inc. papiNet is a registered trademark of the European Union and the United States. Financial
Information eXchange, F.I.X, and F.I.X Protocol are trademarks of FIX Protocol Ltd. UCCnet and eBusinessReady are registered trademarks, and 1SYNC and
Transora are trademarks of GS1 US. Hewlett‐Packard, HP, HP‐UX, OpenView, PA‐RISC, and SNAplus2 are trademarks of Hewlett‐Packard Company. i2 is a
registered trademark of i2 Technologies, Inc. AIX, AS/400, CICS, ClearCase, DB2, Domino, IBM, Informix, Infoprint, Lotus, Lotus Notes, MQSeries, OS/390,
OS/400, RACF, RS/6000, SQL/400, S/390, System/390, VTAM, and WebSphere, and z/OS are registered trademarks; and Communications System for Windows
NT, DB2 Universal Database, IMS, MVS, and SQL/DS are trademarks of IBM Corporation. InnoDB is a trademark of Innobase Oy. Itanium is a registered
trademark of Intel Corporation. Teradata is a registered trademark of NCR Corporation. Netscape is a registered trademark of Netscape Communications
Corporation. ServletExec is a registered trademark, and New Atlanta is a trademark of New Atlanta Communications, LLC. SUSE is a registered trademark
of Novell, Inc. Appia is a registered trademark and Javelin Technologies is a trademark of NYFIX, Inc. CORBA is a registered trademark of Object
Management Group, Inc. JD Edwards, OneWorld, Oracle, PeopleSoft, Siebel, and Vantive are registered trademarks; and Infranet, PeopleSoft Pure Internet
Architecture, Portal, and WorldSoftware are trademarks of Oracle Corporation. Perforce is a trademark of Perforce Software. JBoss and Red Hat are
registered trademarks of Red Hat, Inc. PIP and RosettaNet are trademarks of RosettaNet, a non‐profit organization. SAP and R/3 are registered trademarks
of SAP AG. PVCS is a registered trademark of Serena Software, Inc. SWIFT and SWIFTNet are registered trademarks of Society for Worldwide Interbank
Financial Telecommunication SCRL. SPARC and SPARCStation are registered trademarks of SPARC International, Inc. BAAN and SSA are registered
trademarks; and SSA Global is a trademark of SSA Global Technologies, Inc. EJB, Enterprise JavaBeans, Java, JavaServer, JDBC, JSP, J2EE, Solaris, Sun, and
Sun Microsystems are registered trademarks; and Java Naming and Directory Interface, JavaServer Pages, SOAP with Attachments API for Java, and SunSoft
are trademarks of Sun Microsystems, Inc. Sybase is a registered trademark of Sybase, Inc. VERITAS is a registered trademark, and VERITAS Cluster Server is
a trademark of Symantec Corporation. UNIX is a registered trademark of The Open Group. Unicode is a trademark of Unicode, Inc. VeriSign is a registered
trademark of Verisign, Inc.
Software AG and all Software AG product names are either trademarks or registered trademarks of Software AG.
Other product and company names mentioned herein may be the trademarks of their respective owners.
Copyright © 2007 webMethods, Inc. All rights reserved.
Copyright © 2007 Software AG and/or its suppliers, Uhlandstrasse 12, 64297 Darmstadt, Germany. All rights reserved.
Document ID: DEV-FFS-DG-71-20070831

Contents
Contents
About This Guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Document Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Additional Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Chapter 1. Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
What Is a Flat File? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
What Types of Flat Files Does webMethods Support? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
What Is a Flat File Schema? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
What Type of Flat File Constraints Can You Define? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
How Is a Flat File Schema Used to Parse Records? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Record Parsers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Record Identifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Extractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Undefined Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Default Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
What Is a Flat File Dictionary? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
When Should I Create a Flat File Dictionary? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
What Is the Flat File Schema Editor? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Flat File Structure Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Chapter 2. Processing Flat Files Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Formatting Inbound and Outbound Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Processing Inbound Flat Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Step 1: Create Appropriate Flat File Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Step 2: Receive the Flat File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Step 3: Parse, Convert, and Validate Flat File Data to an IS Document . . . . . . . . . . . . . . . . . . . 27
Step 4: Process the IS Document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Mapping Flat File Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Handling Large Flat Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Processing Outbound Flat Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Step 1: Create Appropriate Flat File Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Step 2: Convert the IS Document to a Flat File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Step 3: Send/Process the Flat File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Flat File Schema Developer’s Guide Version 7.1 3

Contents
Chapter 3. Creating and Editing Flat File Schemas and Dictionaries . . . . . . . . . . . . . . . . 35

Creating a Flat File Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Step 1: Add and Configure the Flat File Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Step 2: Specify Flat File Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Floating Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Examples: Parsing Inbound Floating Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Examples: Parsing Outbound Floating Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Step 3: Define the Structure of the Flat File Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Flat File Schema Property Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Extractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Validators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Conditional Validators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Field Validators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Length Validator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Code List Validator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Byte Count Validator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Format Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Creating Format Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Working with Format Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Disabling Format Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Managing Dictionary Dependencies on Format Services . . . . . . . . . . . . . . . . . . . . . . 67
Displaying Element Descriptions and Alternate Names in the Flow Editor . . . . . . . . . . . . . 67
Step 4: Test the Flat File Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Creating a Flat File Dictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Editing a Flat File Schema or Dictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Managing Flat File Dictionary Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Setting Access Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Customizing the Flat File Configuration Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Chapter 4. Sending and Receiving Flat Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Flat File Content Handler and Content Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Choosing the Service to Receive the Flat File from the Content Handler . . . . . . . . . . . . . . . . . . 79
Submitting a Flat File to Integration Server in a String Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Submitting a Flat File to Integration Server via HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Building a Client that Posts a Flat File to a Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Submitting a Flat File to Integration Server via FTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
FTPing a File From a webMethods Integration Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Getting Results from an FTP’d Document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Contents
Submitting a Flat File to Integration Server via File Polling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Submitting a Flat File to Integration Server via an E–mail Message . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Requirements for Submitting a Flat File Document via an E-mail Message . . . . . . . . . . . . . . . . 84
Getting Results from an Emailed Document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Chapter 5. WmFlatFile Built–in Services Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Flat File Conversion Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
pub.flatFile:FormatService . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
pub.flatFile:convertToString . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
pub.flatFile:convertToValues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
pub.flatFile:getSupportedEncodings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Flat File Generation Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
pub.flatFile.generate:createDocumentType . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
pub.flatFile.generate:createFFDictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
pub.flatFile.generate:deleteFFDictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
pub.flatFile.generate:deleteFFDictionaryEntry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
pub.flatFile.generate:deleteFFSchema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
pub.flatFile.generate:findDependants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
pub.flatFile.generate:findReferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
pub.flatFile.generate:getFFDictionaryAsXML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
pub.flatFile.generate:getFFDictionaryEntryAsXML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
pub.flatFile.generate:getFFSchemaAsXML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
pub.flatFile.generate:listFFDictionaryEntries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
pub.flatFile.generate:saveXMLAsFFDictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
pub.flatFile.generate:saveXMLASFFSchema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
pub.flatFile.generate:updateFFDictionaryEntryFromXML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Flat File Generation IS Document Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
pub.flatFile.generate:FFDictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
pub.flatFile.generate:FFSchema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Flat File Generation Sample Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
sample.flatFile.generateFFSchema:delimited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
sample.flatFile.generateFFSchema:fixedLength . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Appendix A. Validation Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Validation Error Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
General Error Entries in the errors Array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Entries for Conditional Validator Errors in errorDetails Array . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Validation Error Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

Contents
Appendix B. Flat File Byte Count Parser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Handling Partial Characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Reading Partial Characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Writing Partial Character Encodings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Stateful Encodings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Writing Stateful Encodings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Appendix C. Programmatically Creating Flat File Schemas and Dictionaries . . . . . . . . . 129

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Creating Flat File Dictionary Entries, Dictionaries, and Schemas . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Creating Flat File Dictionary Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Creating an Entire Flat File Dictionary with Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
Creating an Empty Flat File Dictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
Creating a Flat File Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
Modifying Flat File Dictionary Entries, Dictionaries, and Schemas . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
Modifying an Existing Flat File Dictionary Entry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
Modifying an Existing Flat File Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Deleting Flat File Dictionary Entries, Dictionaries, and Schemas . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
Sample Flow Services for Working with XML Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
Creating a Service that Retrieves the XML File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
Retrieving Namespace Data to Write to an XML File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

About This Guide
About This Guide
The webMethods Integration Server provides the WmFlatFile package to enable you to
exchange and process flat files using its built–in services. This guide is for users who want
to use webMethods Developer to create flat file schemas and dictionaries to parse and
validate inbound flat files and produce outbound flat files.
To use this guide effectively, you should be familiar the basic concepts described in the
webMethods Integration Server Administrator’s Guide and the webMethods Developer User’s
Guide.
Document Conventions
Convention Description
Bold Identifies elements on a screen.
Italic Identifies variable information that you must supply or change
based on your specific situation or environment. Identifies terms the
first time they are defined in text. Also identifies service input and
output variables.
Narrow font Identifies storage locations for services on the webMethods
Integration Server using the convention folder.subfolder:service.
Typewriter Identifies characters and values that you must type exactly or
font messages that the system displays on the console.
UPPERCASE Identifies keyboard keys. Keys that you must press simultaneously
are joined with the “+” symbol.
\ Directory paths use the “\” directory delimiter unless the subject is
UNIX‐specific.
[ ] Optional keywords or values are enclosed in [ ]. Do not type the [ ]
symbols in your own code.

About This Guide
Additional Information
The webMethods Advantage Web site at http://advantage.webmethods.com provides you
with important sources of information about webMethods products:
Troubleshooting Information. The webMethods Knowledge Base provides
troubleshooting information for many webMethods products.
Documentation Feedback. To provide feedback on webMethods documentation, go to
the Documentation Feedback Form on the webMethods Bookshelf.
Additional Documentation. Starting with 7.0, you have the option of downloading the
documentation during product installation to a single directory called
“_documentation,” located by default under webMethods installation directory. In
addition, you can find documentation for all webMethods products on the
webMethods Bookshelf.

Chapter 1. Concepts
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
What Is a Flat File? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
What Is a Flat File Schema? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
What Type of Flat File Constraints Can You Define? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
How Is a Flat File Schema Used to Parse Records? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
What Is a Flat File Dictionary? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
When Should I Create a Flat File Dictionary? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
What Is the Flat File Schema Editor? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1. Concepts
Overview
The webMethods Flat File package (WmFlatFile) is a set of services that enables you to
translate documents into and from flat file formats. To set up the translation, you use a flat
file schema to define how to identify individual records within a flat file and what data is
contained in each of those records.
What Is a Flat File?

When different applications attempt to communicate with one another, they may not
speak the same “language.” Flat files enable you to send data to any application in a
mutually agreed upon format so that the data in the files can be read and processed.
Flat files present complex hierarchical data in a record–based storage format, which
unlike XML does not embed structural information (metadata) within the data. In other
words, the metadata of a flat file is separated from the data and contained in a flat file
schema. A single logical record of application data is externalized as a set of records
without any structural information. Therefore, the application receiving the flat file must
have knowledge of the structure of the flat file, through the flat file schema, to read the flat
file. This structural information identifies the parent–child relationships between different
records in the flat file. The actual data has been “flattened” by removing this hierarchical
relationship. By nesting records in the flat file schema (adding a record to a record) you
can represent the hierarchical structure of the data in the flat file schema.
All flat files consist of a list of records containing fields and composites:
Fields are atomic pieces of data (for example, postal code).
Composites contain multiple fields (for example, ID and ID qualifier, Date and time).
The fields within a composite are referred to as subfields. A composite definition
contains the same information as a record definition, except that composites are not
allowed inside composites.
Records (also known as segments) are sequences of fields and/or composites.
For example, the following flat file data and its list of delimiters enables you to see how
elements (records, composites, and fields) within a flat file can be identified.
Field record delimiter = !
Rec1*C1Field1+C1Field2*FieldA*C2Field1+C2Field2! field delimiter = *

subfield delimiter = +
Composite Composite
Record
To communicate using flat files, users must create a flat file schema that contains a
particular flat file’s structural information, including how to identify records and separate
those records into fields. This enables receivers of flat file documents to parse and validate

1. Concepts
inbound flat files for use by their back–end systems, and senders of documents to convert
outbound documents to flat file format for delivery to another back–end system.
What Types of Flat Files Does webMethods Support?

The WmFlatFile package can exchange all types of flat files but can process only certain
types of flat files. The WmFlatFile package can process flat files in which:
The records in the flat file are defined using one of the following methods:
Delimiters. Each record in the flat file is separated by a delimiter.
Fixed length. Each record is a fixed number of bytes (for example, mainframe
punch or print records).
Variable length. Each record is preceded by two bytes that indicate the length of the
record. Records in the flat file can have different lengths.
Each distinct record structure has a record identifier, or if no record identifier is
present, the record must be parsed using a default record definition. For more
information about default records, see “Default Records” on page 18.
If the flat file contains record identifiers, the record identifiers must be located in the
same location in all records in the file.
The WmFlatFile package then can identify fields in these records based on either:
Delimiters. Each field is separated by a delimiter, and the Nth delimited field is
identified in the record. This can be used only when a field delimiter (and, if necessary,
subfield delimiter) has been specified.
Byte position. Each field is defined by 1) the number of bytes from the beginning of the
record and 2) the field length. This can be used regardless of whether a field delimiter
has been specified.
To parse a flat file that does not conform to these restrictions, you can create a custom
service using webMethods Developer. For more information, see the webMethods Developer
User’s Guide.

1. Concepts
What Is a Flat File Schema?

A flat file schema is the blueprint that contains the instructions for parsing or creating a
flat file and is created as a namespace element in the webMethods Integration Server. This
blueprint details the structure of the document, including delimiters, records, and
repeated record structures. A flat file schema also acts as the model against which you can
validate an inbound flat file.
A flat file schema consists of hierarchical elements that represent each record, field, and
subfield in a flat file. Each element is a record, composite, or field, and either a definition or
reference. You then configure each element with the necessary constraints.
For instructions to create a flat file schema, see Chapter 3, “Creating and Editing Flat File
Schemas and Dictionaries”, in this guide.
What Type of Flat File Constraints Can You Define?

Flat file schemas contain information about the document, record, and field levels of flat
files. A flat file schema can define three types of constraints:
Structural constraints describe the sequence of records, composites, and fields in a flat
file. For example, a flat file schema might describe an address record that consists of a
Street Address field, followed by a City field, State field, and ZIP Code field. You
define structural constraints for each parent element in the flat file schema.
Content type constraints describe the possible values or minimum/maximum values for
elements in a flat file. For example, the State field might have a content constraint
specifying that its value must be exactly two digits. These types of constraints are
referred to as validators. In flat file dictionaries, you can define validators only for
parent records.
Conditional constraints describe the valid combinations of fields and subfields allowed
in records and composites. For example, you can use the Required (R) conditional
validator to require that at least one of the specified fields be present.
During data validation, the validation engine in the WmFlatFile package compares the
records, composites, and fields in a flat file with the constraints described for those
records and fields in the flat file schema. An inbound flat file is considered valid when it
complies with the constraints outlined in its related flat file schema. For more information
about setting flat file schema constraints, see Chapter 3, “Creating and Editing Flat File
Schemas and Dictionaries” in this guide. For more information about data validation, see
Appendix A, “Validation Errors” in this guide.

1. Concepts
The following table identifies the types of flat file schema constraints you can set for the
document, record, and field levels.
Level Constraint
Document How the records are parsed
What records can appear in the document
Whether the records are record definitions or record references
The order in which the records can appear
Whether a record is mandatory or optional
Whether a record can repeat, and if so, how many times
Record/ How fields and composites are extracted from a record
Composite
Delimited field. Extracts the Nth delimited field from the record.
This can be used only when a field delimiter has been specified.
Fields within a character–delimited composite are separated by
a subfield delimiter.
Positional field. Each field is defined by an offset from the
beginning of the record and length. This can be used regardless
of whether a field delimiter has been specified.
Whether a field or composite is mandatory or optional
How to conditionally validate each field
Field Minimum and maximum field lengths
List of valid values
Range of valid values
Partition of valid values
Important! All lengths and positions specified in a flat file schema refer to byte lengths and
positions, not character lengths and positions. This is important to note because the byte
count can differ from the character count for encodings that have multi–byte characters.

1. Concepts
How Is a Flat File Schema Used to Parse Records?

The WmFlatFile package provides the pub.flatFile:convertToValues service, which uses a flat
file schema to parse flat files inbound to the Integration Server. The convertToValues service
uses a record parser (Delimited, Fixed Length, or Variable Length) to parse the records in a
flat file.
After each record in a flat file has been parsed, each record must be identified. This is done
using the record identifier. Identifying the record provides the definition of that record, as
defined in the flat file schema using the Flat File Schema Editor. The field values then are
pulled from the record using the extractors defined in the flat file schema.
If the record cannot be identified (known as undefined data) and you have selected a default
record, the field values are extracted from the record using the definition of the default
record. If the record cannot be identified and you have not selected a default record, the
field values are not extracted from the record. You also can format flat file data to meet the
requirements of your back–end system or application using the Format Service property on
the Flat File Structure tab of the Flat File Schema Editor. For information about how to
specify field format services, see “Format Services” on page 66.
Finally, the parsed record is placed in the output data based on the structure defined in
the flat file schema. This process is repeated until all records in the flat file have been
parsed.
Record Parsers
A record parser breaks a flat file into individual records. In the WmFlatFile package, you
can choose from one of its four record parsers:
Delimited Record Parser. This parser expects a specific delimiter to indicate the end of a
record. For a record delimiter, you can specify:
A character (for example, !) or character representation (for example, \r\n for
carriage return)
Hexidecimal value (for example, 0X09)
Octal value (for example, 009)
Unicode characters (for example, \uXXXX, where XXXX represents the unicode
value of the character)
The occurrence of the record delimiter signals the end of one record and the
beginning of the next record.

1. Concepts
Fixed Length Record Parser. This parser splits a file into records of the same

pre‐specified length.
Note: This parser measures the lengths and positions of records in terms of bytes,
rather than characters. Prior to 6.5, Service Pack 2, the parser measured lengths and
positions in terms of characters. This change does not affect users who currently
parse/compose flat files using single‐byte encodings because one byte equals one
character. In this case, there is no functional difference between specifying bytes or
characters.
The parser supports both single‐byte and multi‐byte encodings. Multi‐byte encoded
files must run on JVM version 1.4 or later. (The behavior of delimited fields and
records remain the same.) For details, see Appendix B, “Flat File Byte Count Parser”.
Variable Length Record Parser. This parser expects each record to be preceded by two

bytes that indicate the length of the record. Each record may be a different length.
EDI Document Type Record Parser. This parser is used only for EDI flat files and provides
additional functionality needed to properly parse EDI documents. Documents using
this record parser should be created using the webMethods EDI Module, not the
WmFlatFile package. For more information, see the webMethods EDI Module User’s
Guide.
Record Identifiers
A record identifier looks at a record and extracts an identifier out of the data. The
WmFlatFile package uses that identifier to connect the record definition in a flat file
schema with a particular record in the flat file. The name of the record definition must
match the value obtained by the record identifier. In the WmFlatFile package, you can
choose from one of two methods of record identification:
Starts at position record identifiers compare the value that occurs in the record, at the
specified offset, to all the record names defined in the flat file schema. Note that the
Starts at position identifier cannot distinguish between all types of record names. For
example, if you name records “Rec1” and “Rec,” some instances of “Rec1” may be
identified as “Rec,” because “Rec1” begins with “Rec.”
Nth Field record identifiers use the value of the specified field as the record identifier.
These identifiers count from zero (0). For example, if 2 is specified, the third field is
used as the record identifier.
The WmFlatFile package does not perform any kind of normalization on input provided
in the flat file schema or when comparing or processing values retrieved from a file. You
must enter carefully the exact Unicode character values you want to search for in an
instance of the flat file you are describing. For example, you should differentiate wide
(sometimes called multi–byte or zenkaku) characters from their narrow (or single–byte)
equivalents when processing Asian characters. Another example is the use of Unicode

1. Concepts
combining and pre–composed sequences. In all cases, fields are matched using a strict
binary comparison of Unicode character values in the separate fields.
You also should exercise care when selecting the encoding of the file being processed.
Some encodings are very similar to one another. For example, the Shift–JIS and MS932
encodings commonly used with Japanese language text are very similar, but they map
some characters differently. This can result in the WmFlatFile package not finding a match
where you otherwise would expect one to exist.
Extractors
Extractors take data out of a parsed record and place it in the output of the
pub.flatFile:convertToValues service. If extractors are not defined in the flat file schema, the
parser returns a series of records that contain no fields or composites.
Fields and composites are extracted from a record based on the position of the field
delimiter. From the beginning of the record to the first field delimiter is the first field in
the record. From the first field delimiter to the second field delimiter is the second field in
the record, and so on. Everything from the last field delimiter to the record delimiter (the
end of the record) is considered part of the final field.
Note: Although fixed length and variable length record parsers do not use record
delimiters, they can have field delimiters.
Fields also can be extracted from a record based on a substring of the original record
starting at a particular byte count and ending at a particular byte count. The user specifies
the start of the field by specifying the number of bytes from the beginning of the record to
the start of the field and specifying the number of bytes from the beginning of the record
to the end of the field.
If a value is a composite, it is simply a field that can be further parsed into more fields.
Instead of a field delimiter, a subfield delimiter is used to separate a composite into fields.
Fields are extracted from a composite based on either the position of the subfield delimiter
in the composite or on a substring of the composite. Remember that all positions used in
extracting fields from a composite are referenced from the beginning of the composite, not
the start of the record.
Undefined Data
In some cases, the data resulting from the pub.flatFile:convertToValues service might contain
records that were not recognized, known as undefined data. This could be a result of the
complexity of the flat file, or of the lack of detail in the flat file schema (such as no record
identifier). When the convertToValues service processes an unrecognized record, it puts a

1. Concepts
placeholder named unDefData in the resulting IS document (IData object) and stores the
record as a string in the pipeline.
Note: If the file is encoded using a multi‐byte encoding, and if you use a fixed length or
variable length parser, the service puts two placeholders into the pipeline: unDefData and
unDefBytes.
You can select a default record when creating a flat file schema. Any record that cannot be
recognized will be parsed using this default record. If a default record is not selected,
unrecognized records will be placed into the output IS document with the undefined data
tag, which might produce errors.
Note: If your flat file does not contain record identifiers, you must select a default record.
By selecting a default record, a CSV (comma separated values) file can be parsed as a
special case of record with no record identifier, but with fixed field and record delimiters.
For some types of documents, undefined data should not be allowed in the document and
if encountered should generate validation errors in the output. In other cases, undefined
data is expected and should not generate validation errors. In still other cases, all
unidentified records should be parsed using a default record definition, as described in
“Default Records” on page 18.
You can choose whether you want errors to be generated for a flat file schema when
undefined data is encountered. You do so using a combination of two fields:
The Allow Undefined Data check box on the Settings tab
The Allow Undefined Data check box for each record in the flat file schema on the Flat File

Structure tab
When the Allow Undefined Data check box on the Settings tab is selected, which it is not the
default, and the Allow Undefined Data check box for a particular record is selected, no errors
will be generated if the record contains undefined data. If either the Allow Undefined Data
check box on the Settings tab or the Allow Undefined Data check box for a particular record is
cleared, errors will be generated if the record contains undefined data.
Working with undefined data involves three steps:
1 Configuring the flat file schema and record–level Allow Undefined Data settings.
2 Ignoring undefined data, OR parsing the undefined data in each record based on the
document structure.
3 Writing logic to handle the parsed data, such as logic to map the parsed data to
another service or perform error handling.

1. Concepts
Default Records
If the pub.flatFile:convertToValues service cannot recognize a record (for example, the record
does not have a record identifier), the record will be treated as undefined data unless you
specify a default record definition to be used to parse all unrecognized records of a
particular type. In fact, if your flat file schema does not contain any record identifiers, you
must specify a default record. A default record can only be defined in a flat file dictionary.
For example, if you have a CSV (comma separated values) flat file that simply lists the
name, address, and phone number of a group of customers, none of the records would
have a record identifier:
John Doe, 123 Main St Washington DC, 888–555–1111;
Jane Smith, 321 Oak Dr Memphis TN, 800–555–2222;
If you specified a default record definition to parse this particular type of record, the file
can be properly parsed and converted to an IS document (IData object) of the name
recordWithNoID. By default, each recordWithNoID record appears as a child of the record
above it, in an array. To modify this default behavior, see “Customizing the Flat File
Configuration Settings” on page 74.
If a default record has not been specified, the record will be treated as undefined data, as
described in “Undefined Data” on page 16.
What Is a Flat File Dictionary?

A flat file schema can contain either record definitions or references to record definitions
that are stored elsewhere in the namespace in a flat file dictionary. A flat file dictionary is
simply a repository for elements that you reference from flat file schemas. This allows you
to create record definitions in a dictionary that can be used across multiple flat file
schemas. Reusing record definitions reduces the amount of memory consumed by a flat
file schema.
Flat file dictionaries are created as namespace elements in the Integration Server and
contain definitions of records, composites, and fields. When you change a definition in a
flat file dictionary that is referenced in multiple flat file schemas, the element definition is
updated automatically in all of the flat file schemas.
Note: You can reference a flat file dictionary definition in any flat file schema regardless of
whether the dictionary and schema are located in the same package.
When creating an element definition in a flat file dictionary, you specify only certain
properties. You then specify the remaining properties in the instance of the element
definition in a particular flat file schema.

1. Concepts
When Should I Create a Flat File Dictionary?

The decision to define records in a flat file dictionary versus in a flat file schema depends
on the type of flat files that you intend to parse. The Electronic Document Interchange
(EDI) ANSI X12 standard defines a large set of document structures that reuse the same
record, field, and composite definitions many times. Defining these records, fields, and
composites in a dictionary allows for them to be reused throughout the entire set of EDI
ANSI X12 document flat file schemas. Reusing definitions reduces the amount of memory
consumed by the webMethods Integration Server.
EDI ANSI X12 also has different versions of these documents (for example, 4010). Each
version of the document set should have its own dictionary. In this way, you can be certain
that changes to a record, field, or composite between versions are maintained.
A more complex scenario would involve multiple families of documents and multiple
versions of those families. An example of this is EDI ANSI X12 and UN/EDIFACT
documents. One dictionary should be created for each version of EDI ANSI X12
documents and one dictionary should be created for each version of EDI UN/EDIFACT
documents. A separate dictionary would not be required for each flat file schema in the
same version. All flat file schemas in one version of the same family should use the same
dictionary.
In a scenario in which you intend to parse only one flat file, or flat files that do not share
record, composite, or field definitions, you can define these elements directly in the flat
file schema. This allows for the entire document to be edited in a single view, without
referencing a flat file dictionary.
If a clear choice does not exist between these two scenarios, the best approach is to create
the definitions in the flat file dictionary and reference them in a flat file schema. The
definitions then can be reused at a later time.
What Is the Flat File Schema Editor?

When you select a flat file schema or dictionary from the Navigation Panel of
webMethods Developer, Developer displays the contents of the flat file schema or
dictionary in the Flat File Schema Editor.

1. Concepts
Identifying Flat File Schemas and Dictionaries

Name of Flat File Schema
Navigation Flat File

Panel Schema Editor
The following table identifies the symbols for flat file schemas and dictionaries in the
Navigation Panel of webMethods Developer.
Symbol Description
Flat File Schema. Identifies a flat file schema in the Navigation Panel.
Flat File Dictionary. Identifies a flat file dictionary in the Navigation Panel.
Note: A green check mark next to a flat file schema or dictionary ( ) indicates that the

flat file schema or dictionary is locked and available to be edited. When the check mark is
absent, the schema or dictionary is unlocked and cannot be edited.
The Flat File Schema Editor is divided into the following sections:
Flat File Definition tab. This tab enables you to specify what type of flat file you are
parsing or creating.
Flat File Structure tab. This tab enables you to specify the structural, content type, and
conditional constraints of your flat file schema. For detailed information, see “Flat File
Structure Tab” below.

1. Concepts
Properties panel. This panel enables you to configure a default record, allow undefined
data, add areas, and set up the ACL (access control lists) permissions for your flat file
schema or dictionary.
Flat File Structure Tab

When creating your flat file schema, you will create new records, composites and fields.
The Flat File Schema Editor uses unique symbols to represent the components of flat file
schemas and dictionaries. The following table identifies the symbols for each component
that can appear in a flat file schema.
Symbol Description
Record Definition. Identifies a record definition in a flat file schema.
Record Reference. Identifies a record reference in a flat file schema.
If “Reference not found” appears after the reference name, your reference is
broken. A broken reference indicates that the referenced element no longer
exists. Flat file schemas that contain broken references will not parse
correctly. To fix a broken reference, either delete the reference or re–create
the definition in the appropriate dictionary.
Composite Definition. Identifies a composite definition in a flat file schema. A
composite definition contains the same information as a record definition,
except that composites are not allowed inside of a composite.
Composite Reference. Identifies a composite reference in a flat file schema.
See Record Reference in this table for information about “Reference not
found.”
Field Definition. Identifies a field definition in the flat file.
Field Reference. Identifies a field reference in a flat file schema. See Record
Reference in this table for information about “Reference not found.”
An element definition is an element that is contained in a flat file schema or dictionary. An
element reference refers to an element definition in a flat file dictionary. For the element
name you have selected in the Flat File Schema Editor, when you add a new element it
will be added as a child of the selected element. The appropriate new element selections
will be displayed for that element level. For example, when you have a record selected,
you can add record, composite, or field references or definitions to this record. If you
create a composite and then select that composite, you can add only a field reference or
field definition.

1. Concepts
You use the buttons at the top of this tab to edit the structure of your schema. The
following table provides a description of each of these buttons.
Button Description
Select the desired element, and then click to delete the selected
element.
Select the desired element, and then click to move the element up or
down in the flat file schema structure.
Select the desired element, and then click to move the element left or
right in the flat file schema structure.
Creates an IS document type for this flat file schema. The IS document
type is used to map the output of the convertToValues service to another
format. For more information, see “Mapping Flat File Data” on
page 27 of Chapter 2, “Processing Flat Files Overview” of this guide.

Chapter 2. Processing Flat Files Overview
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Processing Inbound Flat Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Processing Outbound Flat Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2. Processing Flat Files Overview
Overview
As discussed in Chapter 1, “Concepts” in this guide, the webMethods WmFlatFile
package enables you to exchange all types of flat files and process certain types of flat
files. You create flat file schemas to convert and validate inbound flat files and to create
outbound flat files.
The following diagram summarizes the services that the WmFlatFile package provides to
process flat files: pub.flatFile:convertToValues and pub.flatFile:convertToString. For descriptions, see
the table that follows the diagram below.
Note: If you also are using Trading Networks, see the sections about flat file TN document
types in the webMethods Trading Networks Concepts Guide and webMethods Trading Networks
Administrator’s Guide for information about how Trading Networks processes flat files.
Processing Flat Files
webMethods Integration Server

Flat File Package
Conversion/Validation Services
Flat File
(ffData) Inbound Receive flat file
convertToValues IS Doc
(optional validation) (ffValues)
Process IS Document
Flat File Schema
IS Doc
Process IS Document
(ffValues)
convertToString
Flat File Send/Process
(ffData) Outbound Flat File
Item Description
Flat File (ffData) Flat file input with type of String, InputStream, or ByteArray
(received by the Integration Server via File Polling, HTTP, FTP, etc.
For more information, see Chapter 4, “Sending and Receiving Flat
Files” in this guide.)

Item Description
Conversion/ WmFlatFile package provides services (pub.flatFile:convertToValues
Validation Services and pub.flatFile:convertToString) that parse, validate, and convert
inbound flat files to IS documents (IData objects), and construct
outbound documents from IS documents. For more information
about these services, see Chapter 5, “WmFlatFile Built–in Services
Reference” in this guide.
Flat File Schema Flat File schema containing the structure of the flat file.
IS Doc (ffValues) An IS document (IData object) represents the structure and values
of your flat file for eventual mapping or other processing. All
inbound documents must be converted to IS documents before
being processed further.
Formatting Inbound and Outbound Data

You can format flat file data to meet the requirements of your back–end system or
application. The Format Service property in the Flat File Schema Editor enables you to
specify the service you want to invoke for a particular field when convertToValues or
convertToString is called. For example, when using the convertToValues service to convert a
date field in a flat file into an IS document, you can specify that a particular format service
be invoked for that field during parsing. To alter the format of the date field, you might
invoke a service that changes the format of the field from YYYYMMDD to MMDDYYYY.
For information about how to specify field format services, see “Format Services” on
page 66.
Processing Inbound Flat Files

The WmFlatFile package provides the pub.flatFile:convertToValues service, which uses a flat
file schema to parse flat files inbound to a webMethods Integration Server. The input of
this service is a flat file and the name of a flat file schema, and the output of this service is
an IS document (IData object). The output IS document is created based on the structure
defined in the flat file schema. You can also specify to validate the flat file against the flat
file schema, which generates errors as output based on the constraints placed on the
records and fields in the flat file schema.
Note: Validation is turned off by default. To enable validation, you must set the validate
variable of the convertToValues service to true.
You should use this service as part of customized flow services. You then can use
webMethods Developer services to process the resulting data as necessary, (for example,
map data from one format to another). For more information about creating flow services,

see the webMethods Developer User’s Guide. For more information about mapping data, see
“Mapping Flat File Data” on page 27.
The general process for inbound flat files is as follows:
Inbound Flat File Process
Conversion/Validation
Flat File convertToValues IS Doc

(ffData) (optional validation) (IData)
Process IS Document Is Valid

Flat File Schema
errors
The flat file schema provides the convertToValues service with the information necessary to
detect the boundaries between records and fields. After a record boundary is detected, the
service matches the record in the flat file with a record defined in the flat file schema using
the record identifier.
Note: When parsing a file, the flat file parser ignores any data that appears before the first
valid record in the file.
Step 1: Create Appropriate Flat File Schema

Before you start working with an inbound flat file, you must create a flat file schema to be
used for conversion and validation of the document. Use the webMethods Developer to
create the flat file schemas used to parse, convert, and validate inbound flat files. For steps
to create flat file schemas, see Chapter 3, “Creating and Editing Flat File Schemas and
Dictionaries” in this guide.
Step 2: Receive the Flat File

The webMethods Integration Server provides a number of ways to send and receive flat
files. For more information, see Chapter 4, “Sending and Receiving Flat Files” in this
guide.

Step 3: Parse, Convert, and Validate Flat File Data to an IS

Document
To parse a flat file using a flat file schema, and to convert the data into IS documents
(IData objects), you call the convertToValues service.
During flat file parsing, it is an option to validate the flat file against the document, record,
and/or field constraints you defined in a flat file schema. For a list of the constraints you
can set, see “What Type of Flat File Constraints Can You Define?” on page 12. To set or
change the constraints for a particular flat file schema, see Chapter 3, “Creating and
Editing Flat File Schemas and Dictionaries” in this guide.
To validate a flat file against the flat file schema used to parse the file, you must set the
validate variable of the convertToValues service to true, which returns errors describing how
the given ffData violates the constraints described in the flat file schema. For a list of
validation error codes and messages, see Appendix A, “Validation Errors” in this guide.
For more information about the convertToValues service, see Chapter 5, “WmFlatFile Built–
in Services Reference” in this guide.
Step 4: Process the IS Document

webMethods Developer provides tools for processing IS documents (IData objects),
including mapping data into other formats.
Mapping Flat File Data

In many cases it is necessary to map the output of the convertToValues service to another
format. Creating an IS document type from your flat file schema enables you to reference
and view the document structure in a flow service.
These steps enable you to map data into other formats, such as your back–end, or user–
defined format.

To map flat file data
1 In the Developer Navigation Panel, select the saved flat file schema, click the Flat File
Structure tab, and then click . This creates an IS document type in the same folder as
the schema.
This IS document type is named the same as the flat file schema, with “DT”
appended. For example, as shown in the figure above, if your flat file schema name is
“PurchaseOrder,” the related IS document type name is “PurchaseOrderDT.”
2 Click to save the IS document type.
3 For the flow service you are creating or editing, on the Flow Editor of the Developer
select the INVOKE pub.flatFile:convertToValues flow operation.

4 On the Pipeline tab under Service In, in the ffSchema variable specify the location and
name of the flat file schema (for example, “Test:PurchaseOrder”). Specify other inputs
as necessary.
5 Perform the following to add a Document Reference to the IS document type that is
based on the flat file schema.
a On the Pipeline tab click under Pipeline Out, then click . Select Document
Reference to add a new document reference variable.
b In the Select dialog box, navigate to the IS document type created in step 1 of this
procedure (for example, “PurchaseOrderDT”). The name appears in the Name
box. Click OK.

6 On the Pipeline tab under Service Out, map the value of the ffValues variable to the
document reference variable created in this procedure.
7 Click to save the service.
Handling Large Flat Files

As installed, the WmFlatFile package processes all flat files in the same manner,
regardless of their size. That is, the WmFlatFile package receives a flat file and keeps the
file content in memory during processing. However, if you receive large files, the
WmFlatFile package can encounter problems when working with these files because the
system does not have enough memory to hold the entire parsed file.
If some or all of the files that you process using the WmFlatFile package encounter
problems because of memory constraints, you can set the iterator variable in the
pub.wmflatFile:convertToValues service to true to process top level records (children of the
document root) in the schema one at a time. After all child records of the top level record
are parsed, the convertToValues service returns and the iterator moves to the top level of the
next record in the schema, until all records are parsed. This parsing should be done in a
repeat step where each time the convertToValues service returns, the results are mapped and
dropped from the pipeline to conserve memory. If the results were kept in the pipeline,
out–of–memory errors might occur.
The service generates an output object (ffIterator variable) that encapsulates and keeps
track of the input records during processing. When all input data has been parsed, this
object becomes null. When the ffIterator variable is null, you should exit from the repeat
step to discontinue processing.

Processing Outbound Flat Files

The WmFlatFile package provides the pub.flatFile:convertToString service, which uses a flat file
schema to create a flat file outbound from a webMethods Integration Server. The input of
this service is an IS document (IData object) and a flat file schema, and the output of this
service is a flat file.
Note: The flat file resulting from convertToString will be only as complete as the information
translated from the original flat file to the IS document. If you specified only certain parts
of the flat file to translate to the IS document (via the flat file schema), of course, only those
parts will be translated back to the flat file.
The general process for outbound documents is as follows:
Outbound Flat File Process
Conversion
IS Doc Flat File
(IData) convertToString (ffData)
Send/Process
Flat File
Flat File Schema
Step 1: Create Appropriate Flat File Schema

Before you start working with an IS document (IData object), you must create a flat file
schema to be used for conversion of the object. You use webMethods Developer to create
the flat file schemas used to create outbound flat files. For steps to create flat file schemas,
see Chapter 3, “Creating and Editing Flat File Schemas and Dictionaries” of this guide.
Step 2: Convert the IS Document to a Flat File

The WmFlatFile package provides the convertToString service to convert flat file IS
documents (IData objects) to outbound flat files using the flat file schema you created.
This service enables you to specify delimiters to be used as record, field, and subfield
separators as well as optionally specify an outputFileName so that the output also will be
written to a file.

Example
In many cases it is necessary to map the output of the convertToString service to another
format. Creating an IS document type from your flat file schema enables you to reference
and view the document structure in a flow service.
The following steps provide an example of how you might convert an IS document to a
flat file.
To convert an IS document to flat file data
1 In the Developer Navigation Panel, select the saved flat file schema, click the Flat File
Structure tab, and then click . This creates an IS document type in the same folder
as the schema.
This IS document type is named the same as the flat file schema, with “DT”
appended. For example, as shown in the figure above, if your flat file schema name is
“PurchaseOrder,” the related IS document type name is “PurchaseOrderDT.”
2 Click to save the IS document type.
3 For the flow service you are creating or editing, on the Flow Editor of the Developer
select the INVOKE pub.flatFile:convertToString flow operation.
4 On the Pipeline tab under Service In, in the ffSchema variable specify the location and
name of the flat file schema (for example, “Test:PurchaseOrder”). Specify other inputs
as necessary.

5 Perform the following to add a Document Reference to the IS document type that is
based on the flat file schema.
a On the Pipeline tab, click under Pipeline In, then click . Select Document Reference

to add a new document reference variable.
b On the Select dialog box, navigate to the IS document type created in step 1 of this
procedure (for example, “Test:PurchaseOrder”). The name appears in the Name
box. Click OK.
6 On the Pipeline tab, map the document reference variable created in this procedure to
the value of the ffValues variable under Service Out.
7 Click to save the service.
Step 3: Send/Process the Flat File

After the flat file is created, you can send or process the file as necessary. You can send a
flat file using many methods, including via HTTP, SMTP, and FTP. For more information,
see Chapter 4, “Sending and Receiving Flat Files” in this guide.


Chapter 3. Creating and Editing Flat File Schemas and
Dictionaries
Creating a Flat File Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Step 1: Add and Configure the Flat File Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Step 2: Specify Flat File Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Step 3: Define the Structure of the Flat File Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Step 4: Test the Flat File Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Creating a Flat File Dictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Editing a Flat File Schema or Dictionary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Managing Flat File Dictionary Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Setting Access Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Customizing the Flat File Configuration Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

3. Creating and Editing Flat File Schemas and Dictionaries
Creating a Flat File Schema

Creating a flat file schema consists of a number of steps. To simplify the steps needed to
create a flat file schema, this procedure is broken down into sections:
“Step 1: Add and Configure the Flat File Schema” on page 36
“Step 2: Specify Flat File Properties” on page 41
“Step 3: Define the Structure of the Flat File Schema” on page 48
“Step 4: Test the Flat File Schema” on page 68
Step 1: Add and Configure the Flat File Schema
To add a new flat file schema
1 Create a package with a dependency on the WmFlatFile package by clicking and
typing WmFlatFile in the Package box and its version number in the Version box. If you
need instructions for creating packages within the Developer, see the webMethods
Developer User’s Guide.
2 Create a new folder within the new package.
3 If you want to reuse element definitions in other flat file schemas, create a flat file
dictionary. (Note that any flat file schemas sharing records, fields, or composite
definitions should use a common dictionary.) For more information about creating
and using a flat file dictionary, see “Creating a Flat File Dictionary” on page 69.
4 In webMethods Developer, click File New in the menu bar.
5 In the New dialog box, select Flat File Schema, and then click Next.
6 Type the name of the new flat file schema, select its location (the folder you created in
the WmFlatFile–dependent package), and then click Finish.
The flat file schema you created is automatically selected, and the Flat File Schema
Editor displays in the right pane.
7 On the Flat File Definition tab under Record Parser, select the type of parser for this flat
file schema:
Delimiter. Each record is separated by a delimiter.
Fixed Length. Each record is of a fixed length (for example, mainframe punch or
print records).
Variable Length. The first two bytes of the input data stream contains the length of
each record.

EDI Document Type. Do not select this option.
Important! If you are using the webMethods EDI Module to process EDI
documents, you should use the wm.b2b.edi services to create your flat file schemas.
This guide does not provide information about creating EDI flat file schemas for
use with the webMethods EDI Module. For more information and steps, see the
webMethods EDI Module User’s Guide. The EDI Document Type option is displayed for
you to view existing EDI flat file schemas. Although you can create EDI flat file
schemas using the Flat File Schema Editor, Software AG does not recommend it.
Note: In subsequent steps you can specify delimiters using the Character property.
When you specify delimiters, use the following syntax:
One character or character representation (for example, *, \n for line terminator,
\t for tab). For the Record delimiter, you can also select carriage return line feed (or
specify \r\n) if a record ends with the two–character CRLF.
Hexidecimal value with prefix “0X” (for example, 0X09, 0X13)
Octal value with prefix “0” or decimal value (for example, 009, 013)
Unicode characters (for example, \uXXXX, where XXXX represents the unicode
The space character is allowed as a delimiter.
8 For all types of record parsers, specify the following fields:
a For Field or Composite:
Property Description
Character Optional – Character that separates fields in a flat file
document.
–OR–
Character position Optional – Starting from the beginning of the document and
counting from zero (0), the character position at which the
field delimiter for this document is located.
For example, if you specify 4 as the Character position, you
have indicated that the field delimiter appears in the fifth
character position from the beginning of the document.

b For Subfield:
Character Optional – Character that separates subfields in a flat file
document.
–OR–
subfield delimiter for this document is located.
have indicated that the subfield delimiter appears in the
sixth character position from the beginning of the
document.
c For Release Character:
Specify either Release Character –or– Quoted Release Character. If you specify both,
the convertToString service uses Quoted Release Character.
Character Optional – Character used to enable a delimiter to be used for
its intended, original meaning. The character following the
release character will not be treated as a delimiter.
For example, your field delimiter is + and your release
character is \. When using + within a field as text, you must
preface it with your release character. When using the
convertToValues service to create the strings a+b+c and d+e+f,
the record would appear as a\+b\+c+d\+e\+f. When using
the convertToString service to create a\+b\+c+d\+e\+f, the
value of the record would be a+b+c and d+e+f.
–OR–
Character Optional – Starting from the beginning of the document and
position counting from zero (0), the character position at which the
release character for this document is located.
have indicated that the release character appears in the sixth

d For Quoted Release Character:

Specify either Release Character –or– Quoted Release Character. If you specify both,
the convertToString service uses Quoted Release Character.)
Character Optional – Character used to enable a section of text within a
field to be represented as its literal value. Any delimiter
characters that appear within this section will not be treated
as delimiters.
For example, your field delimiter is (,) and your release
character is “. When you want to use (,) within a field as
text, you must preface it with your quoted release character.
When using the convertToValues service to create the strings
Doe, John and Doe, Jane, the record would appear as
“Doe, John”,“Doe, Jane”. When using the convertToString
service to create “Doe, John”,“Doe, Jane”, the value of
the record would be Doe, John and Doe, Jane.
When using the convertToString service, if you have specified
both the Release Character and the Quoted Release Character,
the Quoted Release Character will be used.
–OR–
quoted release character for this document is located.
have indicated that the quoted release character appears in
the sixth character position from the beginning of the
document.
9 If you selected Delimiter for Record Parser, complete the following Record fields:
Character Character that separates records in a flat file document.
–OR–
Character position Starting from the beginning of the document and counting
from zero (0), the character position at which the record
delimiter for this document is located.
For example, if you specify 3 as the Character position, you have
indicated that the record delimiter appears in the fourth

Important! If records use a fixed position extractor, the Delimiter record parser does not
include the record delimiter character in the parsed record. You will not be able to
extract the record delimiter character in a fixed position field.
Important! If you use the webMethods EDI Module, set the
useReadableDelimiterReporting variable to false. This variable controls how the
Delimiter record parser parses non‐readable characters, such as \r (carriage return),
\n (line feed), or \t (tab). For details, see “Customizing the Flat File Configuration
Settings” on page 74.
10 If you selected Fixed Length for Record Parser, complete the following Record Parser

field:
Record length Length, in characters, of each record in the flat file.
11 To define the record identifier for this flat file schema, complete the appropriate
Record Identifier property:
Starts at Position Counting from zero (0), the character position at which the

record identifier is located.
For example, if you specify 5 as the Starts at Position, you have
indicated that the record identifier appears in the sixth
character position in each record. The default for this field is
zero (0).
–OR–
Nth Field Counting from zero (0), indicates which field in each record you
want used as the record identifier.
This value cannot be null and must be an integer greater than or
equal to zero (0). This field is available only if you have
specified a field delimiter.

Step 2: Specify Flat File Properties
To specify properties for the flat file schema
1 View the Properties panel.
2 Under Default Record, configure the following properties:
Set Optional – Under Value, click Set to select the default record for this flat
file schema from a flat file dictionary. This record is used to parse an
undefined data record when the convertToValues service fails to find a
match between the flat file and the flat file schema.
If the default record is specified when creating the flat file schema,
any record that cannot be recognized will be parsed using this default
record. If a default record is not selected, the record will be treated as
undefined data.
If the Undefined Data property is set to False and the validate variable of
the convertToValues service is set to true, convertToValues will generate
errors when it encounters undefined data. For more information
about the Undefined Data property, see page 42.
Note: If the flat file you are parsing does not contain record identifiers,
you must select a default record. By selecting a default record, a CSV
(comma separated values) file can be parsed as a special case of record
with no record identifier, but with fixed field and record delimiters.
Delete Under Value, click Delete to delete the default record for this flat file
schema. The actual record definition still exists, but is no longer
assigned to this flat file schema.
Dictionary Displays the name of the flat file dictionary that contains the default
record.
Name Displays the name of the record reference that will be used as the
default record.
Floating Optional – The name of the record that is defined in the schema as a
Record floating record.
Note: If the floating record has an alternate name, specify the alternate
name in this field.
For details about how the parser handles floating records, see
“Floating Records” on page 43.

3 Under Settings, the Undefined Data property helps control the generation of undefined
data errors at any location in the flat file schema.
If you want to allow undefined data, select True. You now can choose whether to allow
undefined data at the record level. For more information about the record–level
property, see the Allow Undef Data property on page 57.
If you do not want to allow undefined data in any location in this flat file schema, select
False. This is the default. If the validate variable of the convertToValues service is set
to true, the convertToValues service will generate errors when undefined data is
encountered.
When the convertToValues service processes an undefined record, it puts a placeholder
named unDefData in the resulting IS document and stores the record as a string in the
pipeline.
Note: For a file that is encoded using a multi‐byte encoding, the behavior is different.
When you use a fixed length or variable length parser (but not a delimited parser) to
parse such a file, the service places unDefData and unDefBytes into the pipeline. If
you want to re‐parse such a file to extract fixed byte position fields, use unDefBytes.
4 Under Settings, the Areas property enables you to add and edit areas for this flat file
schema. Click Choices to display the Areas dialog box, which enables you to use the
following buttons to add and edit areas.
Click to add a new area to this flat file schema. You must save the flat
file schema before the new area will be available for use in your flat file
structure.
An area is a way to associate an arbitrary string with a given record. For
example, you may have an address record that needs to specify the
shipping address in the document header, but needs to specify the
billing address in the document detail. To differentiate these two
records, you would create “Header” and “Detail” areas. Then when
defining a record in “Step 3: Define the Structure of the Flat File
Schema” on page 48, you would assign the appropriate area to a record
to identify in which section of the document the record belongs.
Note: Areas are used primarily for EDI document parsing.
Click to insert a new area in a specific location in the list.
Click to delete an existing area.

Floating Records
You can use the Floating Record property to designate any single record of a given flat file
schema to be a floating record. By designating a floating record, you enable that record to
appear in any position within a flat file without causing a parsing validation error.
If you do not use this property, validation errors will occur if the record structure of an
inbound document does not match the record structure defined in its flat file schema. For
example, consider the following EDI document and its flat file schema (where all records
are mandatory except the record NTE):
Flat file schema Input EDI

definition document Output IS document
Rec1 Rec1 Rec1
Rec2 NTE NTE
Rec3 Rec2
The parser generates a validation error because it
NTE Rec3 expects NTE to be the last record, not the second
record. In addition, it ignores Rec2 and Rec3.
To solve this problem, you would designate NTE to be a floating record so it may appear
anywhere in the EDI document without causing a validation error. (The position in which
it appears in the parsed record is explained in the following sections.) The parser creates a
container for NTE records, and places each NTE record that appears in an EDI document
into this container at run time. If a flat file schema contains multiple instances of the
designated floating record at multiple record levels, the system creates a container for
each record level. This property appears in the Properties panel of the Flat File Schema
Editor (see the table in step 8 on page 55).
This problem can also occur while converting outbound IS documents (IData objects) into
string documents. The parser supports floating records for both inbound and outbound
parsing (i.e., while executing the convertToValues and convertToString services, respectively).
For examples, see “Examples: Parsing Inbound Floating Records” on page 44 and
“Examples: Parsing Outbound Floating Records” on page 47.

Examples: Parsing Inbound Floating Records

When the convertToValues service receives a document that includes floating records, the
parser handles them as shown in the following examples. Assume that the Floating Record
property is set to the record NTE.
Example 1:
Flat file schema definition Input EDI document Output IS document

Rec1 Rec1 Rec1
NTE [] Max Occurs 99 Rec2 NTE []
Rec2 Rec3 NTE (1)
Rec3 Rec4 NTE (2)
Rec4 NTE (1) NTE (3)
Rec5 NTE (2) Rec2
NTE (3) Rec3
Rec5 Rec4
Rec5
In accordance with the schema definition, the parser creates an NTE array as the first
record, and places NTE (1), NTE (2), and NTE (3) into it. A validation error is generated
if more than 99 NTE records appear in the document. NTE records are not allowed prior
to the Rec1 record.
Example 2:
Flat file schema definition Input EDI document Output IS document

Rec1 Rec1 Rec1
NTE [] Max Occurs 99 NTE (1) NTE []
Rec2 [] Max Occurs 99 Rec2 NTE (1)
NTE [] Max Occurs 99 Rec3 NTE (5)
Rec3 NTE (2) Rec2 []
Rec4 Rec4 Rec2
Rec5 NTE (3) NTE []
Rec2 NTE (2)
NTE (4) NTE (3)
Rec5 Rec3
NTE (5) Rec4
Rec2
NTE []
NTE (4)
Rec5

How Example 2 was Parsed

1 The first six records of the EDI document were parsed as follows:
Input EDI document Output IS document

Rec1 Rec1
NTE (1) NTE [] <== NTE container for Rec1.
Rec2 NTE (1)
Rec3 Rec2 []
NTE (2) Rec2
Rec4 NTE [] <== NTE container for Rec2 array.
NTE (2)
Rec3
Rec4
a The positions of the first three records (Rec1, NTE (1), and Rec2) match the
schema. In accordance with the schema definition, the parser:
Creates an NTE container (an array) under Rec1, and places NTE (1) into it.
Creates a Rec2 array, and places Rec2 into it.
b Even though the schema definition says the next record should be an NTE
(followed by Rec3), the next record in the document is Rec3 (followed by NTE
(2)). This is valid; the parser simply:
Places Rec3 into the Rec2 array, as specified in the schema definition.
Creates another NTE container (an array) under Rec2, as specified in the
schema definition. To determine which container to put NTE (2) in, the
parser looks for the last defined container at the current level. Thus, the
parser places NTE (2) in this container.
c The parser places the next record, Rec4, in the Rec2 array, in accordance with the
schema definition.

2 Parsing NTE (3):
Input EDI
document Output IS document
Rec1 Rec1
NTE (1) NTE []
Rec2 NTE (1)
Rec3 Rec2 []
NTE (2) Rec2
Rec4 NTE []
NTE (2)
NTE (3) NTE (3) <== added to this container at
current level
Rec3
Rec4
To determine which container to put NTE (3) in, the parser looks for the last defined
container at the current level. Thus, the parser places NTE (3) in the same container as
NTE (2).
3 Parsing Rec2 and NTE (4):
Input EDI
Rec1 Rec1
NTE (1) NTE []
Rec2 NTE (1)
Rec3 Rec2 []
NTE (2) Rec2
Rec4 NTE []
NTE (3) NTE (2)
Rec2 NTE (3)
Rec3
NTE (4) Rec4
Rec2
NTE [] <== creates an NTE container for second
Rec2
NTE (4)
In accordance with the schema definition, the parser adds another Rec2 to the Rec2
array. Just as with the first Rec2 in the array, the parser also creates another NTE
container, for the second Rec2.

4 Parsing Rec5 and NTE (5):
Input EDI
Rec1 Rec1
NTE (1) NTE []
Rec2 NTE (1)
Rec3 NTE (5) <== this container is same level as Rec5
NTE (2) (current level)
Rec4 Rec2 []
NTE (3) Rec2
Rec2 NTE []
NTE (4) NTE (2)
Rec5 NTE (3)
Rec3
NTE (5) Rec4
Rec2
NTE []
NTE (4)
Rec5
a In accordance with the schema definition, the parser places Rec5 under Rec1.
b To determine which container to put NTE (5) in, the parser looks for the last
defined container at the current level. Thus, the parser places NTE (5) in the same
container as NTE (1).
Examples: Parsing Outbound Floating Records

When the convertToString service converts an IS document (an IData object) to a string, the
parser places each floating record immediately following its parent record (as defined in
the IS document).
In the following examples, assume that the Floating Record property is set to the record
NTE.
Example 1:
Flat file schema

definition Input IS document Output string
Rec1 Rec1 Rec1!Rec2!Rec3!NTE(1)!NTE(2)!Rec4
Rec2 Rec2
NTE [] Rec3
Rec3 NTE []
Rec4 NTE(1)
NTE(2)
Rec4

Because NTE(1) and NTE(2) are children of Rec3, the parser places them after Rec3 in the
string.
Example 2:
Flat file schema

definition Input IS document Output string
Rec1 Rec1 Rec1!NTE(1)!NTE(2)!Rec2!Rec3!Rec4
Rec2 Rec2
NTE [] Rec3
Rec3 Rec4
Rec4 NTE []
NTE(1)
NTE(2)
Because NTE(1) and NTE(2) are children of Rec1, the parser places them after Rec1 in the
string.
Step 3: Define the Structure of the Flat File Schema

Important! Validation is turned off by default. To enable validation, you must set the validate
variable of the convertToValues service to true. This enables errors to be generated for the
Ordered, Mandatory, Validator, and Undefined Data properties. For more information about this
service, see Chapter 5, “WmFlatFile Built–in Services Reference” of this guide.
To define the structure of a flat file schema
1 With the flat file schema name selected in webMethods Developer Navigation Panel,
click the Flat File Structure tab of the Flat File Schema Editor.

2 To add the first record, click the name of the flat file schema in the Name column, and

then click . The Select New wizard appears. You also can right–click an element and
select Add.
3 Select one of the following.
Record Definition Select to create a record definition to be used one time in this

flat file schema. After selecting Record Definition, click Next.
Record Reference Select to create a reference to a record definition in a flat file
dictionary.
After selecting Record Reference, click Next.
Navigate to the flat file dictionary in which the record is
located, select the dictionary, and then click Next.
Select the element that you want to reference and then
click Next.

4 In the Enter Record Definition Name(s) dialog box, type the name of the record.
Important! This name must match the value of its record identifier exactly as it will
appear in the flat file. The name of a record reference does not have to match the name
of the record definition in the flat file dictionary. The name of a record reference will
be matched to the record identifier in the record. The name of the record definition in
the flat file dictionary does not need to match the record identifier that appears in the
flat file.
To add another record definition at this time, click and type the record name.
When you have added all of the records for this level, click Finish.
The record(s) is added to the flat file schema structure. The Referring To field indicates
the record definition to which the record reference refers. The Dictionary field indicates
the flat file dictionary to which the record reference refers. If the element is a
definition, these two fields are empty.
5 In the Properties panel of the Flat File Schema Editor, specify the particular properties
for this record. For a list and further explanation of these properties, see the table in
step 8 on page 55.

For element definitions, you see only one line of detail. For element references, you
will see two lines of detail: one for the instance of that element in this flat file schema,
and one for the element definition in the flat file dictionary. In a flat file dictionary you
can specify only certain fields for each element. The remaining fields must be set in
the particular instance of the element in the flat file schema. You cannot edit the
dictionary definition detail line. For information about creating flat file dictionaries,
see “Creating a Flat File Dictionary” on page 69.
6 To add more elements, select the element to which you want to add a child and click
. You also can right–click an element and select New. The valid element selections
display for the element you have selected.
If you are adding a record definition, type the Name of the record in the Enter
Record Definition Name(s) dialog box.
Note: The name(s) must match the value of the identifier for each record exactly as
it will appear in the flat file.

To add additional record definitions at this level, click .
If you are adding a composite definition, specify the Name, Position, and Mandatory
properties in the Enter Composite Definition Name(s) dialog box.
Name Type the name of the composite.
Position Type the field number in the record that contains the composite
you want to extract. This pulls the subfield data from the
composite.
If you leave this property empty, the composite will not be
extracted. For a description of composite extractors, see
“Extractors” on page 61.
Note: The extractor works only if a field and subfield delimiter
have been defined for this flat file schema.

Mandatory Optional – Select the check box to require that this composite
appear in the flat file.
If it is not selected, the composite is not required to appear in the
flat file. If it is selected and the convertToValues service validate
variable is set to true, errors will be generated if the composite
does not appear in the flat file.
To add additional composite definitions at this level, click .
If you are adding a field definition, specify the Extractor Type, Name, Start, End, and
Mandatory properties in the Enter Field Definition Name(s) dialog box.
Extractor Type Property Description
Fixed Position Counting from zero (0), extracts a fixed number of

bytes from a record. For more information about
field extractors, see “Extractors” on page 61.
Name Type the name of the field.
Start Type the first byte to extract from the record.
End Type the first byte that is not included in the
extraction. If you enter a negative number (for
example, –1), the extractor returns all bytes from
the byte specified in Start to the last byte in the
record or composite.
Mandatory Optional – Select the check box to require that this
field appear in the flat file.
If is not selected, the field is not required to
appear in the flat file. If it is selected and the
convertToValues service validate variable is set to
true, errors will be generated if the field does not

Extractor Type Property Description
Nth Field Counting from zero (0), indicates the field that

you want to extract from the record. For more
information about field extractors, see
“Extractors” on page 61.
Name Type the name of the field.
Position Type a value to indicate the position of the field
that you want to extract from the record.
This value cannot be null and must be an integer
greater than or equal to zero (0). For example, if
you type 1, the second field will be extracted. This
option is available only if you specified a field
delimiter when configuring your flat file schema.
This extractor returns the field as a key–value
pair. The key is the name of the field. The value is
the String value of the field.
Mandatory Optional – Select the check box to require that this
field appear in the flat file.
If is not selected, the field is not required to
appear in the flat file. If it is selected and the
convertToValues service validate variable is set to
true, errors will be generated if the field does not
To add additional field definitions at this level, click .
If you are adding an element reference, complete the following steps:
After selecting Reference, click Next.
Navigate to the flat file dictionary in which the element is located, select the
dictionary, and then click Next.
Select the element that you want to reference and then click Next.
7 When you have added all of the elements for this level, click Finish.

8 For each of the following element levels and type, you can specify the following
properties in the Properties panel of the Flat File Schema Editor:
Element Property For details, see ...
Record Ordered page 60

Definition Mandatory page 60
Max Repeat page 60
Validator page 61
Area page 57
Position page 61
Allow Undef Data page 57
Check Fields page 58
Alternate Name page 57
Description page 58
Record Ordered page 60
Reference Mandatory page 60
Max Repeat page 60
Area page 57
Position page 61
Allow Undef Data page 57
Local Description page 59
Composite Mandatory * page 60
Definition Extractor page 58
Validator * page 61
Check Fields * page 58
Alternate Name * page 57
Description * page 58
ID Code * page 59
Local Description ** page 59
* not available for composite definitions in record references
** not available for composite definitions in flat file schemas
Composite Mandatory * page 60
Reference Extractor page 58
Alternate Name * page 57
* not available for composite references in record references

Field Mandatory page 60

Definition Extractor page 58
Validator page 61
Format Service page 59
DataType page 58
Description page 58
ID Code page 59
Local Description * page 59
* only available for field definitions in composite references or record
references
Field Mandatory * page 60
Reference Extractor * page 58
* not available for field definitions in composite references or record
references
9 After you have added all of the necessary elements to this flat file schema, click .
10 To map the output of the convertToValues service to another format, click on the Flat
File Structure tab. For more information about mapping flat file schemas, see “Mapping
Flat File Data” on page 27 of Chapter 2, “Processing Flat Files Overview” of this
guide.
You now can use this flat file schema to parse and convert inbound flat files to IS
documents (IData objects) and convert outbound documents to flat files.

Flat File Schema Property Descriptions

The flat file schema properties that you can define using the Flat File Schema Editor are
described below, in alphabetic order.
Allow Undef Data Optional. Use in record definitions and record references in flat file

schemas.
If the schema–level Undefined Data property is set to True, select True to
allow undefined data at the record level and not generate errors
when undefined data is encountered. The default is True. For more
information about the Undefined Data property, see page 42.
Select False to not allow undefined data at the record level. If the
validate variable of the convertToValues service is set to true, the
convertToValues service will generate errors when undefined data is
encountered.
Alternate Name Optional. Use in: record definitions and composite definitions in flat
file schemas; composite references in record definitions; field
definitions in composite definitions and record definitions; record,
composite, and field definitions in flat file dictionaries.
An alternate name for the record definition. This name will appear
as the document name in the document type that is generated by this
flat file schema. This name will also appear in the Flow Editor. Use
this field to provide a more meaningful name that can be helpful
when mapping the record. For example, instead of using the record
name ST, you could specify the alternate name Transaction Start. It
will also appear as a tool tip comment in the Flow Editor. For details,
see “Displaying Element Descriptions and Alternate Names in the
Flow Editor” on page 67.
Area Optional. Use in record descriptions and record references in flat file
schemas.
If you have created areas for the flat file schema on the Settings tab,
from the Area drop–down list, select the area you want to use for this
record.
This field is used primarily for EDI document parsing. For more
information, see the Areas property on page 42.

Check Fields Optional. Use in: record definitions and composite definitions in flat

file schemas; record, composite, and field definitions in flat file
dictionaries.
Select True to report as errors more fields at the end of the record or
composite than are defined in the flat file schema.
Select False to allow extra fields at the end of a record or composite.
The default is False.
DataType Use in: field definitions in composite definitions and record
definitions; fields in flat file dictionaries.
Select the appropriate data type. It will also appear as a tool tip
comment in the Flow Editor. For details, see “Displaying Element
Descriptions and Alternate Names in the Flow Editor” on page 67.
Description Optional. Use in: record definitions and composite definitions in flat
file schemas; field definitions in composite definitions and record
definitions; record, composite, and field definitions in flat file
dictionaries.
Describe the purpose of the record, composite, or field. This
description will appear in the comments section of the document
type that is generated for this flat file schema. It will also appear as a
tool tip comment in the Flow Editor. For details, see “Displaying
Element Descriptions and Alternate Names in the Flow Editor” on
page 67.
Extractor Use in: composite definitions in flat file schemas and record
references; composite references in record definitions and record
references; field definitions and field references in composite
definitions or record definitions.
Click the Extractor field, and then type the field number in the record
that contains the composite you want to extract. This pulls the field
or composite data from the record, or pulls the subfield data from
the composite.
If you leave this property empty, the composite will not be extracted.
For more information, see “Extractors” on page 61.
Note: The extractor works for a field only if field delimiters have been
defined for this flat file schema. The extractor works for a composite
only if field delimiters and subfield delimiters have been defined for
this flat file schema.

Format Service Use in: field definitions in composite definitions or record

definitions; fields in flat file dictionaries.
Click the Format Service field to select the service you want to invoke
for a particular field when convertToValues or convertToString is called,
and then click OK. For example, to alter the format of the date field,
you might invoke a service that changes the format of the field from
YYYYMMDD to MMDDYYYY. To clear the format service, simply
clear the text box.
For the service to be visible in the browser, the template specification
for the service on its Input/Output tab must be pub.flatFile:FormatService.
For information about format services, see “Format Services” on
page 66.
ID Code Use in: composite definitions in flat file schemas; field definitions in
composite definitions or record definitions; composite and field
definitions in flat file dictionaries.
The IDCode, which is provided by your SEF file, is used by the
WmEDI package. It will appear in the comments section of the
document type that is generated for this flat file schema. It will also
appear as a tool tip comment in the Flow Editor. For details, see
“Displaying Element Descriptions and Alternate Names in the Flow
Editor” on page 67.
Local Description Optional. Use in: record references in flat file schemas; composite
references in record definitions or record references; composite
definitions in record references; field definitions and field references
in composite references or record references.
Describe the record, composite, or field as it is used in this particular
flat file schema. Since the record that is being referenced is stored in
a flat file dictionary, it may be used differently in different flat file
schemas.
Local descriptions will also appear as a tool tip comment in the Flow
Editor. For details, see “Displaying Element Descriptions and
Alternate Names in the Flow Editor” on page 67.

Mandatory Optional. Use in: record definitions, record references, and
composite definitions in flat file schemas; composite references in
record definitions; field definitions and field references in composite
definitions or record definitions.
Select True to require that this record, composite, or field appear in
the flat file.
Select False to not require the record, composite, or field to appear in
the flat file. If you select True and the convertToValues service validate
variable is set to true, errors will be generated if the record does not
Max Repeat Optional. Use in record definitions and record references in flat file
schemas.
Maximum number of times this record can repeat within the flat file.
You can select Unlimited or type any positive integer. The default
value is 1, except for default records. For all default records, the
value is always Unlimited and cannot be changed.
If you set this property and the convertToValues service validate variable
is set to true, errors will be generated if the number of record
instances exceeds the maximum repeat value.
Ordered Optional. Use in record definitions and record references in flat file
schemas.
Select True to require that the child records of this record appear in the
flat file in the order that they appear in the flat file schema. This
property applies only to records, not composites or fields.
Select False to allow the child records to appear in any order in the
flat file. If you select True and the convertToValues service validate
variable is set to true, errors will be generated if the records do not
appear in this order.

Position Reserved for future use.
Validator Optional. Use in: record definitions and composite definitions in flat
file schemas; field definitions in composite definitions or record
definitions; record definitions
Click the Validator field, and then type the desired validation you
want to be performed on this record, composite, or field in the flat
file. For a list of valid validators, see “Validators” on page 62.
Important! Validation is disabled by default. To enable validation, you
must set the validate variable of the convertToValues service to true.
Extractors
Extractors represent the location of a field within a record, or of a subfield within a
composite. You extract fields from a composite using a Nth field extractor, while you
extract fields from a record using Nth field, ID Node, or Fixed Position extractors.
Note: To use Nth Field or ID Node extractors for a field or composite in a record, the field

delimiter must be specified. To use these extractors for a field in a composite, the subfield
delimiter must be specified. If these extractors are used without the appropriate delimiter
set, the field values will not be returned correctly.
Nth Field. Counting from zero (0), Position indicates the position of the field that you

want to extract from the record. This value cannot be null and must be an integer
greater than or equal to zero (0). For example, if you type 1, the second field will be
extracted. This option is available only if you specified a field delimiter when
configuring your flat file schema. This extractor returns the field as a key–value pair.
The key is the name of the field. The value is the String value of the field. Software AG
recommends that you use this extractor instead of ID node.
ID Node. This extractor is a variation of the Nth Field extractor and is available for
backward compatibility for users of the webMethods EDI Module. Counting from
zero (0), Position indicates the position of the field that you want to extract from the
record. This value cannot be null and must be an integer greater than or equal to zero
(0). This extractor returns the field as a key–value pair. The key is the name of the
field. The value is an IS document (IData object) containing a key–value pair in which
the key is always “code,” and the value is the String value of the field. Software AG
does not recommend that you use the ID node extractor.

Fixed Position. Counting from zero (0), extracts a fixed number of bytes from a record.
Start Position. First byte to extract from the record.
End Position. First byte that is not included in the extraction. If you enter a negative
number (for example, –1), the extractor returns all bytes from the byte specified in
the Start Position to the last byte in the record or composite.
For example:
record = webMethods
Start Position = 1
End Position = 7
Extracted bytes = ebMeth
Validators
Different types of validators are available for records, composites, fields, and multi‐byte
encoded records. You validate:
Records and composites using conditional validators.
Fields using either length or code list validators.
Multi‐byte encoded records, you use the byte count validator.
Conditional Validators
Regardless of what type of flat file you are processing, you follow the syntax rules created
by the EDI Standards bodies ANSI X12 or UN/EDIFACT to specify conditional validators
for a flat file record or composite. In these rules, 0n and 0n0 represent the number of the
field defined in the record or composite, where n is the number of the field in the
composite or record. For example, in the following flat file:
Record1*FieldA*FieldB*FieldC*FieldD*FieldE;
To validate fields using the ANSI X12 conditional validator, you might enter the following
in the Validator property:
C0102C030405
A B C D E
Code Field
To validate the appropriate fields using the UN/EDIFACT conditional validator, you
might enter the following in the Validator property:
D5(010,020) D5(030,040,050)
A B C D E
Code Field
The record definition has five fields defined. Each with an Nth Field extractor, the first
extracting field A, the second field B and so on. Both of these validators indicate that if

Field A is present, Field B is required. If Field C is present, Field D and Field E are
required. However, if Field D and Field E are present, Field C need not be present.
The conditional validator refers to the location of the field in the definition, not the
location of the data in the flat file. For example, if you switched the extractors for the first
and second field such that the first field extracted the value Field B and the second field
extracts the value Field A, the conditional validator would indicate that if Field B is
present Field A is required.
Example:
Record1 – conditional validator C0102
Field A – Nth field extractor, position 1
FieldB – Nth field extractor, position 2
This indicates that if FieldA is present Field B also must be present. If you switched the
position of these fields in the record, you would get the following.
Example:
Record1 – conditional validator C0102
FieldB – Nth field extractor, position 2
FieldA – Nth field extractor, position 1
This now indicates that if Field B is present Field A also must be present.
ANSI X12 Rule UN/EDIFACT Rule

Name (and Code) Name (and Code) Definition
Required (R) One or more (D3) At least one of the specified fields must be
present.
ANSI X12 Example: R030405
UN/EDIFACT Example: D3(030,040,050)
Conditional (C) If first, then all If a specified field is present, all of the fields
(D5) specified in a dependent set of fields also must
be present. If a dependent field is present, the
first specified field need not be present.
ANSI X12 Example: C030405
Exclusion (E) One or none (D4) At most, only one of the specified fields can be
present.
ANSI X12 Example: E030405

ANSI X12 Rule UN/EDIFACT Rule

Name (and Code) Name (and Code) Definition
List If first, then at If a specified field is present, at least one of the
Conditional (L) least one (D6) fields specified in a dependent set of fields also
must be present. If a dependent field is present,
the first specified field need not be present.
ANSI X12 Example: L030405
Paired (P) All or none (D2) All of the specified fields must be present or
absent.
ANSI X12 Example: P030405
UN/EDIFACT Example: D2(010,020)
N/A One and only one Exactly one field may be present.
(D1)
N/A If first, then none If the first field is present, all others cannot be
(D7) included.
Field Validators
You validate fields using either a length validator or a code list validator.
Length Validator
For a length validator, you specify the maximum and minimum number of characters this
field can contain to be considered valid.
Minimum length. Minimum number of characters that this field can contain to be
considered valid. If the field contains fewer characters than the number entered, an
error will be generated. You can enter zero (0).
Maximum length. Maximum number of characters that this field can contain to be
considered valid. If the field contains more characters than the number entered, an
error will be generated. If you enter a negative number (for example, –1), a maximum
length error will never be generated for this field, regardless of its value.
Code List Validator

In the Conditions box, type the appropriate code list conditions. The code list is a comma–
separated list of the allowed values for this field. If the value of the field is not contained
in the code list, errors will be generated.
Note: This validator validates only in ascending order and according to the ASCII values.

You can specify three different types of code lists:
Simple Code List. A comma separated list of valid codes.
Range List. A range list, which can occur within any type of code list, validates that a
string value is between a given start string and end string. The range is specified as
start string:end string. Values are considered to be valid for a given range if all
of the following conditions are true:
The value is the same length as the start and end strings of the range.
The character value in the same position in the start and end string comes from
the same character set (a–z, A–Z or 0–9).
The value is lexically greater than or equal to the start string and lexically less
than or equal to the value of the end string.
For example A0:Z9 is valid, because the first character of each string comes from the
set A–Z, and the second character of each string comes from the set 0–9. A2:2B is not
valid because the first character of the start string does not come from the same
character set as the first character of the end string.
Partition List. A partition joins together multiple code lists within one field, separated
by a pipe (|). For example, if you enter ABC,DEF|1,2, the allowed values of the field
are ABC1, ABC2, DEF1, and DEF2. Each set of code lists must have the same number
of bytes.
The code list validator allows for the right hand side of the partition to be optional. For
example, if you enter ABC,DEF|1,,2, the allowed values of the field are:
ABC1, ABC2, ABC, DEF1, DEF2, and DEF.
The extra comma in 1,,2 indicates that the values 1 and 2 are optional.
Byte Count Validator

You use the byte count validator for multi‐byte encoded records. You specify the
maximum and minimum number of bytes this field can contain to be considered valid.
Minimum length. Minimum number of bytes that this field can contain to be considered
valid. If the field contains fewer bytes than the number entered, an error will be
generated. You can enter zero (0).
Maximum length. Maximum number of bytes that this field can contain to be considered
valid. If the field contains more bytes than the number entered, an error will be
generated. If you enter a negative number (for example, –1), a maximum length error
will never be generated for this field, regardless of its value.

Format Services
You can use the Format Service property on the Flat File Structure tab of the Flat File Schema
Editor to specify the service you want to invoke for a particular field when convertToValues
or convertToString is called.
Important! If a particular field does not have a value (that is, a value is not returned in an IS
document (IData object) for convertToValues or is not present in the input data for
convertToString) the format service assigned to that field will not be executed.
A format service formats the string and optionally ensures that the value of the string
meets the restrictions specified in the format service. For example, when using the
convertToValues service to convert a date field in a flat file into an IS document (IData object),
you can specify that a particular format service be invoked for that field during parsing.
To alter the format of the date field, you might invoke a service that changes the format of
the field from YYYYMMDD to MMDDYYYY.
Note: If you are using the webMethods EDI Module, you can use the built–in format
services provided in the WmEDI package to format fields. For more information, see the
webMethods EDI Module User’s Guide and the webMethods EDI Module Built‐in Services
Reference.
Creating Format Services

To create a format service for use with a field in a flat file schema or dictionary, the service
must implement the pub.flatFile:FormatService service as its template specification for the
format service (located on its Input/Output tab).
Working with Format Error Messages

If the validate input variable of your format service was set to true to validate the format
and value of a field, and that field does not meet the criteria specified in the service, you
will receive an error message in the errorMessage output variable of the format service
indicating why this field could not be validated.
Disabling Format Services

To disable the format service for a particular field, you simply clear the text box and save
the flat file schema or dictionary. To disable all format services within the WmFlatFile
package regardless of what is specified in each field definition, open
IntegrationServer_directory\packages\WmFlatFile\config\ff.cnf and set the
enableFormatting variable to false.
For more detailed information about format services, see “pub.flatFile:FormatService” on
page 88 in Chapter 5, “WmFlatFile Built–in Services Reference”.

Managing Dictionary Dependencies on Format Services

If you move or rename a format service, the Integration Server’s dependency manager will
ask whether it should update all the flat file schemas and flat file dictionaries that use that
format service with the new location or name of the format service.
If you delete a format service, the dependency manager will list all schemas and dictionaries
that will be impacted by the deletion, and prompts you to confirm the deletion.
Displaying Element Descriptions and Alternate Names in the Flow Editor

The webMethods Developer Flow Editor will display the values of several properties that
you specified for the elements of your schema. They are visible as tool tips when you
move the cursor over the elements. Use these optional properties to provide helpful
comments and more meaningful names when mapping elements in flow services.
The Flow Editor will display the following properties:
Element Property
Record definition Description
Alternate Name
Record reference Local Description
Composite definition Description
Alternate Name
ID Code
Composite reference Alternate Name
Local Description
Field definition Description
Alternate Name
IDCode
DataType
Field reference Local Description
To display properties in the Flow Editor
1 Define values for the properties as described in step 8 of “Step 3: Define the Structure
of the Flat File Schema” on page 48.
2 Enable the Show Variable’s Comment option in the webMethods Developer’s Flow Editor
as follows:
a Click File New in the menu bar.
b Select Flow Editor in the navigation area.
c Click Show Variable’s Comment.

Step 4: Test the Flat File Schema

Testing a flat file schema involves viewing a flat file schema in a readable structure. This
enables you to locate errors in your flat file schema that are keeping your flat file
documents from being parsed correctly. The test response you receive contains the same
information that would result from processing and validating a flat file.
Note: Although validation is turned off by default, testing a flat file schema always
performs validation regardless of you settings. To enable validation in the processing of
an actual flat file schema, you must set the validate variable of the convertToValues service to
true. For more information about this service, see Chapter 5, “WmFlatFile Built–in
Services Reference” of this guide.
To test a flat file schema
1 In webMethods Developer Navigation panel, double–click the flat file schema that
you want to test.
2 Click . The Select file and encoding dialog box appears.
3 To select a flat file with which to test the flat file schema, click , navigate to and
select the test flat file, and then click Open. The name of your test flat file appears in the
top portion of the Test tab.
4 From the Encoding drop–down list, select the encoding for the flat file you are testing.
5 To run the test, click OK. The parsed and converted IS documents appear in the Results
panel. Any errors found when parsing the flat file appear in the Errors section of the
Flat File Schema Editor.
Note: If you make a change to the test file you are parsing, you must reload the file
before running a test.
For information about validation errors, see Appendix A, “Validation Errors” of this
guide.
6 After you have reviewed the results, you may find that it is necessary to edit your flat
file schema.

Creating a Flat File Dictionary

You can create a flat file dictionary for use with flat file schemas. Like a flat file schema, a
flat file dictionary contains definitions of records, composites, and fields that can be used
across multiple flat file schemas. Reusing these definitions reduces the amount of memory
consumed by a flat file schema. Flat file dictionaries can also contain references to
composite and field definitions in this dictionary and/or in other dictionaries.
Flat File Dictionary
After you have created elements, you edit the element and its child elements just as you
would elements in a flat file schema, except that you cannot add a record to a record.
For more information about the circumstances for which you might want to create and use
a flat file dictionary, see “When Should I Create a Flat File Dictionary?” on page 19 in
Chapter 1, “Concepts” of this guide.

To create a flat file dictionary
1 In webMethods Developer, click File New in the menu bar.
2 In the New dialog box, select Flat File Dictionary, and then click Next.

3 Type the name of the new flat file dictionary, select its location (a folder in a
WmFlatFile–dependent package), and then click Finish.
The flat file dictionary you created is automatically selected, and the Flat File Schema
Editor displays in the right pane of the Developer. You now can use the Flat File
Schema Editor to configure the flat file dictionary.
4 To add elements, select the element type to which you want to add a child (Record
Definition, Composite Definition, or Field Definition) and click . You also can right–click
an element and select New.

The valid element selections display for the element you have selected. For each of the
following element levels, you can specify the following properties in the Properties
panel of the Flat File Schema Editor:
Record Definition Validator page 61

Check Fields page 58
Description page 58
Define all other record definition properties in the record
reference instance in the flat file schema. (For a list of these
properties, see “Flat File Schema Property Descriptions” on
page 57.) You define child composite and field definitions and
references within a record the same way you define them in a
flat file schema.
Composite Validator page 61
Definition Check Fields page 58
Description page 58
ID Code page 59
Define all other composite definition properties in the
composite reference instance in the flat file schema. (For a list
of these properties, see “Flat File Schema Property
Descriptions” on page 57.) You define child field definitions
and references within a composite the same way you define
the elements in a flat file schema.
Field Definition Validator page 61
Format Service page 59
DataType page 58
Description page 58
ID Code page 59
Define all other field definition properties in the field reference
instance in the flat file schema. You cannot define child
elements for a field definition.
To add additional element definitions at this level, click .
5 After you have created elements, you edit the element and its child elements just as
you would elements in a flat file schema, except that you cannot add a record to a
record.

6 After you have added all of the elements to this flat file dictionary, click to save
the dictionary.
Note: Elements added the dictionary cannot be referenced until the dictionary has
been saved.
You now can create flat file schemas based on this flat file dictionary.
Editing a Flat File Schema or Dictionary

Important! To edit a flat file schema or dictionary, you must have the proper access
permissions to do so and have locked the flat file schema. For information about access
control lists (ACLs) see “Setting Access Permissions” on page 73.
For flat file schemas and dictionaries, you can do the following:
Edit any element properties. To do so, make the necessary changes on the Flat File Structure
tab and then save the flat file schema or dictionary. For a list of these settings and
properties, see “Creating a Flat File Schema” on page 36.
Rename existing elements. To do so, right–click the element, and then select Rename.
Re–structure the elements. To rearrange the structure of your flat file schema, use the

buttons at the top of the Flat File Structure tab.
Button Description
Select the element you want to delete, and click to delete.
Select the element you want to move, and click to move the element
up or down in the flat file schema structure.
Select the element you want to move, and click to move the element
left or right in the flat file schema structure.
Managing Flat File Dictionary Dependencies

If you move or rename a dictionary, the Integration Server’s dependency manager will ask
whether it should update all the flat file schemas and dictionaries that use this dictionary
with the new location or name of the dictionary.
If you delete a dictionary, the dependency manager will list all flat file schemas and
dictionaries that will be impacted by the deletion, and prompts you to confirm the
deletion. However, it does not identify the names of the records, fields, or composites that
reference the dictionary; that is your responsibility.

The dependency manager also manages dictionary dependencies on format services in a
similar way; see “Managing Dictionary Dependencies on Format Services” on page 67.
Setting Access Permissions

Administrators of the Integration Server can set access control list (ACL) permissions to
control user access and editing privileges for each flat file schema or flat file dictionary.
To set access privileges for a flat file schema or dictionary
1 From the webMethods Developer Navigation Panel, double–click the flat file schema
for which you want to set permissions.
2 In the Properties panel, under Permissions, from the List ACL drop–down box, select
the ACL level to which you want to grant read access. This enables the ACL you select
to view this flat file schema or dictionary.
3 From the Write ACL drop–down box, select the ACL level to which you want to grant
write access. This enables the ACL you select to view and edit this flat file schema or
dictionary.
4 Click to save the flat file schema permissions.

Customizing the Flat File Configuration Settings

The WmFlatFile package provides the following flags you can use to customize some
aspects of the package’s default behavior. You implement these flags in the configuration
file:
IntegrationServer_directory\packages\WmFlatFile\config\ff.cnf
Flag Value
alwaysSortFields Determines whether the pub.flatFile:convertToString service checks
the order of the fields in composites and records that it receives
as input (in ffValues) against the flat file schema that is specified
in the ffSchema input parameter.
false (the default): Assume that the fields in composites
and records are in the same order as the flat file schema.
true: Attempt to match the order of the fields in composites
and records to the order of the fields in composites and
records in the flat file schema.
Example: alwaysSortFields=true
useAlternateCodeSets Determines whether to enable the use of alternate code sets.
true (the default): Enable the use of alternate code sets.
false: Do not enable the use of alternate code sets.
Example: useAlternateCodeSets=true

Flag Value
useReadableDelimiter Controls how the Delimiter record parser parses non‐readable
Reporting delimiters, such as \r (carriage return), \n (line feed), or \t
(tab).
true (the default): The parser reports the delimiters as
human‐readable characters, e.g., the carriage return
character is represented as \r. This variable does not impact
human‐readable delimiters such as | or *.
false: The parser reports the delimiters as the actual
delimiter character. For example, if the delimiter is a
carriage return, the value reported will be the non‐readable
carriage return code (decimal 13 or hexidecimal OD).
Note: If you use the webMethods EDI Module, set this variable
to false.
This impacts how you can use the delimiter information in flow
mapping. If you use the delimiters directly in flows with
variable substitution, non‐printable characters will not be
substituted correctly when useReadableDelimiterReporting=true.
For example, when the output of the convertToValues service
shows that the record delimiter is \n and a user flow has an
input set to the following:
abc%@delimiters\record%def
the resulting value will be as follows:
If useReadableDelimiterReporting=true, the resulting value is
abc\ndef
If useReadableDelimiterReporting=false, the resulting value is
abc
def

Flag Value
spacePadJustifies Controls how the input parameter spacePad justifies fixed length
fields handled by the services pub.flatFile:convertToString and
wm.b2b.edi:convertToString.
If spacePadJustifies=true and:
spacePad is set to left, fixed length fields are left justified.
spacePad is set to right, fixed length fields are right
justified.
If spacePadJustifies=false (the default) and:
spacePad is set to left, fixed length fields are right
justified.
spacePad is set to right, fixed length fields are left
justified.
spacePadJustifies has no effect when spacePad is set to none.
Example: spacePadJustifies=false
dontTrimControl Determines how whitespace and control characters are handled
Characters by the service wm.b2b.edi:convertToString when the parameter
spacePad is set to left or right.
If dontTrimControlCharacters=true, and spacePad is set
to left or right, the service does not truncate control
characters from delimited fields.
If dontTrimControlCharacters=false, and spacePad is
set to left or right, control characters are treated as
whitespace and truncated.
Example: dontTrimControlCharacters=true

Chapter 4. Sending and Receiving Flat Files
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Flat File Content Handler and Content Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Choosing the Service to Receive the Flat File from the Content Handler . . . . . . . . . . . . . . . . . . 79
Submitting a Flat File to Integration Server in a String Variable . . . . . . . . . . . . . . . . . . . . . . . . . 79
Submitting a Flat File to Integration Server via HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Submitting a Flat File to Integration Server via FTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Submitting a Flat File to Integration Server via File Polling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Submitting a Flat File to Integration Server via an E–mail Message . . . . . . . . . . . . . . . . . . . . . . 84

4. Sending and Receiving Flat Files
Overview
The webMethods Integration Server provides the following automated mechanisms for
sending and receiving any type of flat file.
Submit a flat file to a service in a String variable (of any name), which the target
service can convert into an IData object using pub.flatFile:convertToValues.
Post a flat file to a service via HTTP.
FTP a flat file to a service.
Submit a flat file to a service via File Polling.
Send a flat file to a service as an e–mail message attachment.
For general steps to configure ports, which is not included in this guide, see the
“Configuring Ports” section of the “Configuring the Server” chapter in the webMethods
Integration Server Administrator’s Guide.
Note: If you use Trading Networks, you can send flat files directly to Trading Networks.
For more information about how Trading Networks processes flat files, see sections about
flat file TN document types in the webMethods Trading Networks Concepts Guide and
webMethods Trading Networks Administrator’s Guide
Flat File Content Handler and Content Type

You submit a flat file to a flow service using one of the methods discussed in this chapter.
The WmFlatFile package registers a content handler for the content type “application/x–
wmflatfile.” This content handler passes the contents of the flat file to the service specified
in the submit method in an InputStream named ffdata. If an InputStream or ByteArray
named ffreturn is present when the service completes execution, the InputStream is read
and the results are returned to the invoker of the service, for example, HTTP, FTP, etc.
The WmFlatFile package provides services that can be called within customized flow
services or the File Polling processing service to initially accept and consume inbound flat
files. These services are described in “Choosing the Service to Receive the Flat File from
the Content Handler” in this chapter. For more information about File Polling, see
“Submitting a Flat File to Integration Server via File Polling” on page 84.

Choosing the Service to Receive the Flat File from the

Content Handler
Those methods that use the flat file content handler must do the following:
Accept an InputStream object called ffdata.
Return data in an InputStream or a ByteArray called ffreturn.
The WmFlatFile package provides the pub.flatFile:convertToValues service to be used within
flow services to initially accept inbound flat files. Details about using this service to
process inbound documents are provided in “Processing Inbound Flat Files” on page 25
of Chapter 2, “Processing Flat Files Overview” of this guide.
Submitting a Flat File to Integration Server in a String Variable

One way to submit a flat file to the Integration Server is to pass the entire document to a
service in a String variable. This approach does not use the flat file content handler, so the
restrictions outlined in the previous section do not apply. Instead, the invoked service
must expect the contents of the flat file to be in a specific pipeline variable. The java client
must place the source file in the pipeline as a String. The invoked service then processes
the input data and can place parameters in the pipeline that will be returned to the java
client as determined by the specific scenario.
When you use this approach, you should code the target service to execute
pub.flatFile:convertToValues to convert the String variable (i.e., the flat file) to an IS document
(IData object). For more information about using convertToValues, see Chapter 5,
“WmFlatFile Built–in Services Reference” of this guide.
The following code fragment illustrates how a Java client might submit a flat file to the
purch:postOrder service on the Integration Server. In this example, the client 1) loads the flat
file into a String, 2) puts the String into the element orders in an IS document (IData
object) named “inputs,” and 3) invokes purch:postOrder on the server at localhost:5555.
import com.wm.app.b2b.client.*;
import com.wm.util.*;
import com.wm.data.*;
import java.io.*;
public class ArbitraryFlatFileClient
.
.
//––Load FF into orders string
String orders = YourLoadFlatFileMethod(orderFile);
//––Put input values into IData object
IData inputs = IDataFactory.create();
IDataCursor inputsCursor = inputs.getcursor();
inputsCursor.last();
inputsCursor.insertAfter(“orders”, orders);
inputsCursor.insertAfter(“authCode”, authCode);

//––Submit request to server

c.connect(“localhost:5555”, “null”, null);
IData outputs = c.invoke(“purch”, “postOrder”, inputs);
c.disconnect();
if (inputsCursor.first(“response”))
{
//...process response
}
The Integration Server invokes postOrder to pass the flat file to convertToValues. This will
produce an “orders” string that can be mapped to the ffdata input variable in the
convertToValues service. As discussed in Chapter 2, “Processing Flat Files Overview” of this
guide, this service will produce an IS document (IData object).
Submitting a Flat File to Integration Server via HTTP

A client can post a flat file to a service via HTTP. To use this approach, you must have a
client that can do the following:
Send a string of data (a flat file) to the Integration Server using the HTTP POST
method.
–AND–
Set the value of the Content–Type request–header field to “application/x–wmflatfile.”
When the Integration Server receives an HTTP POST request where Content–Type is
application/x–wmflatfile, it passes the body of the request as an InputStream to the service
specified in the request’s URL. Because most browsers do not allow you to modify the
Content–Type header field, they are not suitable clients for this type of submission. Clients
that you might use to submit a flat file in this manner include PERL scripts (which allow
you to build and issue HTTP requests) or the Integration Server service, pub.client:http.
Building a Client that Posts a Flat File to a Service

Regardless of which client you use, if you want to post a flat file to a service through
HTTP, you must specify certain information.
To build a client
1 Submit a POST request to the Integration Server.
2 Address the request to the URL of an service (for example,
http://rubicon:5555/invoke/purch/postOrder).
3 Set the Content–Type header field to “application/x–wmflatfile.”
4 Contain flat file data in the body of the message. The data must be the only text that
appears in the body of the request. Do not assign it to a name=value pair.

The following example describes the values that you set if you use pub.client:http to POST a
flat file to a service.
Set this Variable... To...
url This is the URL of the service that you want to invoke.
webMethods services can be invoked via a URL. The format for
specifying the URL is http://hostname:port/invoke/
folder.subfolder/service name, where hostname is the name of the
machine running the Integration Server, port is the port on which
the Integration Server is running, folder.subfolder is the path to the
service folder names, and service name is the name of the service to
invoke. The following value would invoke the “ProcessFlatFile” flow
service located in the “MyFFProcessing” package on the “rubicon”
server with port number “5555.”
Example: http://rubicon:5555/invoke/MyFFProcessing/
ProcessFlatFile
method “post”
headers.Content– Name: Content–type
Type
Value: The specific content type for the document (application/
x–wmflatfile)
data.string This is the flat file that you want to post.
–OR–
data.bytes
You also will set any optional HTTP variables, such as authorization information, that are
required by your application. The invoked service will have an InputStream ffdata object
in the pipeline that contains the flat file data.
Submitting a Flat File to Integration Server via FTP

You can FTP a flat file to the Integration Server’s FTP listening port. By default the FTP
port is assigned to port “8021.” However, this assignment is configurable, so you should
check with your Integration Server administrator to see which port is used for FTP
communications on your Integration Server.
When the Integration Server receives a flat file on the FTP listening port, it passes it as an
InputStream to the service in the directory to which the file was FTP’d.
To submit a flat file to the Integration Server via FTP, the service to which you want to pass
the document must take an InputStream as input for the ffdata variable.

If you want to submit a flat file to a service through FTP, the application must specify
certain information.
To submit a flat file via FTP
1 Initiate an FTP session on the Integration Server’s FTP listening port.
2 Point to the directory that contains the service to which you want to pass the flat file.
Example: cd \ns\Purchasing\SubmitOrder
Important! Note that the root directory for this operation is your Integration Server’s
namespace directory (ns), not the root directory of the target machine. Therefore, if
you want to FTP a file to a service in the Purchasing folder, you use
\ns\Purchasing\ServiceName as the path to that service, not
IntegrationServer_directory\ns\Purchasing\ServiceName. Each folder in the Developer
is treated as a directory when FTPing to the Integration Server. For example, for the
purchasing.purchaseRequests:rfq service, the directory name would be
\ns\purchasing\purchaseRequests\rfq.
3 Copy the flat file to this directory using the following command:
put sourceflatfile.txt destinationflatfile.txt;file content type
Where sourceflatfile.txt is the name of source file that you want to pass to the
Integration Server, destinationflatfile.txt is the name of the destination file, and
application:x–wmflatfile indicates the content type of the file submitted. For example, to
put a document called PurchaseOrder.txt on the Integration Server as
Partner1PO.txt, you would use the following FTP command:
Example: put PurchaseOrder.txt Partner1PO.txt;application:x–wmflatfile

Note that the file you FTP to the Integration Server is never actually written to the
server’s file system. The file you send and the output file it produces (see below) are
written to a virtual directory system maintained in your Integration Server session.
FTPing a File From a webMethods Integration Server

The pub.client folder contains built–in services that you can use to FTP a file from the
Integration Server. To use the pub.client:ftp service, you must set the variables listed in the
following table.
Set this variable... To set...

serverhost This is the name of the machine running the Integration Server.
serverport This is the port on the Integration Server that is listening for FTP
traffic.
username This is the valid user name on the target machine.

Set this variable... To set...

password This is the valid password for the username in username.
command “put”
dirpath If you are sending to the Integration Server, this is the local path to
the service that should be invoked after receiving the flat file. For
example, \ns\Purchasing\SubmitOrder. If sending to another
type of server, this is the working directory of the FTP server.
localfile This is the name of the source file.
remotefile This is the name of the remote file in which you want to save the
data you sending. For example, destinationflatfile.txt. If
you are sending to the Integration Server, the file name should be
followed by the content–type. For example,
destinationflatfile.txt;application:x–wmflatfile.
For more information about these services, see the webMethods Integration Server Built‐In
Services Reference.
Getting Results from an FTP’d Document

The results from a service executed by an FTP request are written to the same virtual
directory to which the flat file was initially FTP’d. The name of the output file to which
results are written is sourcefile.txt.out.
You retrieve this document using the FTP “get” command. For example, if you put a
document called “PurchaseOrder.txt” on the Integration Server, you would use the
following FTP command to get its results:
Example: get PurchaseOrder.txt.out
“PurchaseOrder.txt.out” is the name of the flat file initially FTP’d to the service. This file
contains the value of the output variable ffreturn of the invoked service.
It is a good practice to make each file name that you FTP to the Integration Server unique
(perhaps by attaching a timestamp to the name) so that you do not inadvertently
overwrite other FTP’d documents or their results during an FTP session.
Important! When you end the FTP session, the Integration Server automatically deletes the
original file and its results from your session.

Submitting a Flat File to Integration Server via File Polling

You can send a flat file to the Integration Server via a File Polling processing service. For
general steps to configure the File Polling listener port, see the “Configuring Ports”
section of the “Configuring the Server” chapter in the webMethods Integration Server
Administrator’s Guide.
To poll specifically for flat files, you must specify the Default Content Type as “application/x–
wmflatfile”. When a file with this content type is posted to the directory on the Integration
Server that you are monitoring for flat files (Monitoring Directory), the Integration Server
executes the service that you specified as the Processing Service.
Because the content type is “application/x–wmflatfile”, when the Integration Server
receives the flat file in the appropriately configured Monitoring Directory, it passes the
document as an InputStream to the service configured in the pipeline variable ffdata.
Submitting a Flat File to Integration Server via an E–mail

Message
You can send a flat file document as an attachment to an e–mail message to an e–mail
mailbox and have the Integration Server automatically retrieve the e–mail message and
process the flat file it contains. To do this, your Integration Server must be configured with
an e–mail port that monitors the mailbox to which the flat file will be sent. (Consult your
Integration Server administrator to see whether an email port has been set up on your
Integration Server.)
When a flat file arrives in the e–mail port’s mailbox, the Integration Server automatically
retrieves the message and passes that document as an InputStream to the service specified
on the e–mail’s subject line (or, if a service is not specified on the subject line, the e–mail
port’s default service).
Requirements for Submitting a Flat File Document via an

E-mail Message
To submit a flat file to the Integration Server via an e–mail message, your client program
must specify certain information.
To submit a flat file via email
1 Put the flat file in an e–mail attachment.
2 Set the e–mail’s Content–Type header to “application/x–wmflatfile.”
3 Specify the name of the service that will process the file in the e–mail’s subject line. If
you leave the subject line empty, the document will be processed by the global service
if one is defined or, if not, by the default service assigned to the e–mail port (if one has

been assigned). For more information about specifying the port’s default service, see
the webMethods Integration Server Administrator’s Guide.
The service that will process the flat file must take an InputStream as input in the ffdata
variable.
The following example describes the values that you would set if you used pub.client:smtp to
email a flat file to a service. For more information about using this service, see the
webMethods Integration Server Built‐In Services Reference.
Set this Variable... To...
to A String specifying the e–mail address monitored by the
Integration Server’s e–mail port.
subject A String specifying the fully qualified name of the service to
which the Integration Server will pass the attached document.
Example: orders:ProcessPO
If you do not specify subject, the e–mail port invokes its default
service (if one has been assigned to it).
from A String containing the e–mail address to which the results of the
service will be sent.
attachments. The specific content type for the document (“application/x–
contenttype wmflatfile”).
attachments. A string specifying the fully qualified name of the file containing
filename the flat file.
Getting Results from an Emailed Document

If your e–mail port has been configured to return results, the results from a service
invoked through the port are written to sourcefile.txt.out, and then sent as an attachment of
an e–mail message to the sender of the original message.
This file contains the value of the output variable ffreturn of the invoked service. ffreturn
can be either an InputStream or a byte array.
Important! By default, the e–mail port does not return any results from requests that it
receives. If you want the port to return results, you must explicitly configure it to do so.
For more information about configuring the e–mail port to return results, see your
webMethods Integration Server Administrator’s Guide.


Chapter 5. WmFlatFile Built–in Services Reference
Flat File Conversion Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
Flat File Generation Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Flat File Generation IS Document Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Flat File Generation Sample Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

5. WmFlatFile Built–in Services Reference
Flat File Conversion Services

Use the services in the pub.flatFile folder when converting between Flat File documents and IS documents
(IData objects).
pub.flatFile:FormatService
Use this specification when you create format services for fields in a flat file schema or dictionary. The
format service is invoked for a field when the pub.flatFile:convertToValues and pub.flatFile:convertToString services
are invoked. You create a format service to format the field String and ensure that the value of the String
meets the format restrictions of the format service. When creating a particular format service for use with
the Format Service property in a flat file schema or dictionary, the service you select must implement the
pub.flatFile:FormatService specification (located on its Input/Output tab).
Important! If a particular field does not have a value (that is, a value is not returned in the IS document
(IData object) for the pub.flatFile:convertToValues service or is not present in the input data for the
pub.flatFile:convertToValues service) the format service assigned to that field will not be executed.
Input Variables
value String The field value to format.
direction String Indicates the type of formatting to be applied to the field. Specify one of the
following:
Value Description
convertToString This field is in an outbound document and needs its internal
format converted to its external format.
convertToValues This field is in an inbound document and needs its external
format converted to its internal format.
validate String The value of the input parameter validate from the pub.flatFile:convertToValues
service.
true The value of the convertToValues validate parameter is true
(validate).
false The value of the convertToValues validate parameter is false (do
not validate). This value is always false when the value of the
direction parameter is convertToString.
minLength String Enables you to validate the minimum length of a field. If the field is extracted via
a Fixed Position Extractor, this is the number of bytes that are extracted. If the field is
not extracted via the Fixed Position Extractor and a Length Validator is associated with
this field, this is the minimum length that will be considered valid. Otherwise, this
parameter will not be present in the pipeline.

maxLength String Enables you to validate the maximum length of a field. If the field is extracted via
a Fixed Position Extractor, this is the number of bytes that are extracted. If the field is
not extracted via the Fixed Position Extractor and a Length Validator is associated with
this field, this is the maximum length that will be considered valid. If the maximum
length is unlimited (–1) or there is no Length Validator, this parameter will not be
present in the pipeline.
FormatInfo Document Information that can be used by individual formatting services. This
information can be obtained from one of 3 locations:
convertToString – You can specify FormatInfo in addition to the delimiter information
for a call to this service.
convertToValues – If delimiter information is explicitly passed into the convertToValues
service, FormatInfo can be specified.
From the UNEDIFACT UNA segment – The EDI document type automatically
extracts the decimal separator from the UNA segment.
The only format services that use this feature are the decimal formatting services (for
implied decimal and decimal formats). The FormatInfo IS document should contain a
string called DecimalCharacter. If the decimal character is ‘,’ the number would be
formatted as 100,10 (European format) instead of 100.10, as is common in the US.
Note: Changes to the data in this object will be reflected in all other format services that
are invoked during execution of convertToString and convertToValues.
Output Variables
formattedValue String The field value with appropriate formatting applied.
meetsFormat String Whether the value could be formatted properly.
Value Description
true Indicates that the value could be properly formatted.
false Indicates that the value could not be properly formatted.
errorMessage String If meetsFormat is false, this parameter provides a text message describing the
formatting error.
valueToValidate String The value that will be used by the validator for this field. If this value is not
present, the value passed in the input variable value will be validated. This field is
used only for situations in which the input variable validate is set to true.

pub.flatFile:convertToString
Converts an IS document (IData object) to a flat file document based on the flat file schema that you
specify. By default, this service returns the document as a string, but you can set a flag to optionally return
the document as a byte array instead.
Note: This service does not validate the document.
Input Variables
ffValues Document The IData object representing the flat file.
ffSchema String Namespace name of the flat file schema to use to convert the given IS
document to a string.
spacePad String (optional) How to position the records in the flat file.
Value Meaning
left Left justify the records (add blank spaces to the right of the records)
before the records are written to the output. This is the default.
right Right justify the records (add blank spaces to the left of the records)
before the records are written to the output.
none No spaces added.
signalError String Whether to create errors in the output.
Value Meaning
false Do not create errors in output.
true Create errors in output.
Important! If you are upgrading from webMethods Integration Server version 4.6, to
enable left or right justification you must add the following line to the
IntegrationServer_directory\packages\WmFlatFile\config\ff file:
spacePadJustifies=false
Then, reload the WmFlatFile package so that this configuration setting will take
effect. For details, see “Customizing the Flat File Configuration Settings” on page 74.

noEmptyTrailing String Whether trailing empty fields are to be removed from the output. Used only
Fields with records that have delimited fields.
Value Description
true Trailing empty fields will be removed from the output. For
example, if it is set to true, the output for a record with empty
trailing fields looks like the following: AAA*01*02! (where ! is used
as segment terminator). This is the default.
false A field separator remains to denote an empty field. For example, if
it is set to false, the output for a record with empty trailing fields
looks like the following: AAA*01*02********! (where ! is used as
segment terminator).
delimiters Document (optional) The separator characters used to construct the output string. To
specify a delimiter, you can specify:
\t for tab)
Unicode characters (for example, \uXXXX where XXXX represents the unicode
Value Meaning
record String Character to use to separate records. If you want to specify
the two–character carriage return line feed (CRLF) characters,
specify \r\n.
field String Character to use to separate fields.
subfield String Character to use to separate subfields.
release String Character to use to ignore a record, field, or subfield delimiter in
a field. If a release character occurs in a field or subfield before the
delimiter, it will be prefixed with release before being written to the
output string.
quotedRelease String Character to use to ignore a record, field, or subfield delimiter in
a field. If a quoted release character occurs in a field or subfield
before the delimiter, it will be prefixed with quotedrelease before
being written to the output string. The string is pre‐ and appended
with the quoted release character.
For example, if * is a delimiter, the field value is a*b, and the quoted
release character is “, the string appears as “a*b”.

FormatInfo Document Any values mapped to the FormatInfo variable will be
passed unmodified to all format services invoked by convertToString
and convertToValues.
outputFileName String (optional) If you want the output returned in a file instead of in the string
output variable, provide the name of the file you want created as a result of this
service.
Encoding String The type of encoding used to write data to the output file. The default
encoding is UTF–8.
sortInput String (optional) Whether you want the service to sort the input records to match the
flat file schema specified in ffSchema. You should specify true for sortInput if the data
in ffValues is not in the same order as defined by ffSchema.
Value Description
true You want the service to sort the input records to match the flat file
schema.
Important! If you select to sort the input records, note that:
The service will run slower.
All undefined records will be sorted after the defined records.
The order of the undefined records appear in the final
document is random.
If there are multiple records at the same level with the same name,
the order they appear in the final document is random.
false You do not want the service to sort the input records to match the
flat file schema. The input records must match the order of the flat
file schema. This is the default.
returnAsBytes
Value Description
false Returns the document as a string. This is the default.
true Returns the document as a byte array instead of a string. This
setting is useful (but optional) when parsing multi‐byte encodings.

Output Variables
string String Data that represents the flat file document.
bytes Object If the input variable returnAsBytes=true, returns the output as a byte array
encoded using the specified encoding. The string value is not returned.
errorArray Object String array containing messages pertaining to errors that occurred during
conversion. If no errors are encountered, this contains a value of null.
pub.flatFile:convertToValues
Converts a flat file document to an IS document (IData object) based on the input flat file schema.
Input Variables
ffData Object The flat file input with type of String, InputStream, or ByteArray.
ffSchema String The full name of the flat file schema object used to parse the ffData object.
ffIterator Object (optional) An object that encapsulates and keeps track of the input data during
processing. It is used only when the iterate variable has been set to true.
encoding String (optional) The encoding of the InputStream passed in to ffData. The default
encoding is UTF–8.
delimiters Document (optional) An IData object that contains the segment terminator and the
field and subfield separators. If the delimiter is null, it will be located using the
information defined in the flat file schema. To specify a delimiter, you can specify:
\t for tab)
Unicode characters (for example, \uXXXX where XXXX represents the unicode
The space character.
Important! If you specify one delimiter value, you must specify all values. Specifying
one of these values will override any information set in the flat file schema.
Variable Description
record String Character used to separate records. If you want to specify
the two–character carriage return line feed (CRLF) characters,
specify \r\n.

field String Character used to separate fields.
subfield String Character used to separate subfields.
release String Character used to ignore a record, field, or subfield delimiter
in a field. If a release character occurs in a field or subfield before
the delimiter, it will be prefixed with the release before being
written to the output ffValues.
quotedRelease String Character to use to ignore a record, field, or subfield delimiter
in a field. If a quoted release character occurs in a field or subfield
before the delimiter, it will be prefixed with quotedrelease before
being written to the output string. The string is pre‐ and appended
with the quoted release character.
For example, if * is a delimiter, the field value is a*b, and the
quoted release character is “, the string appears as “a*b”.
FormatInfo Document Any values mapped to the FormatInfo variable will be
passed unmodified to all format services invoked by convertToString
and convertToValues.
iterate String (optional) Whether you want to process the input all at one time or not.
Value Description
true Processes top level records (children of the document root) in the
flat file schema one at a time. After all child records of the top
level record are processed, the iterator moves to the top level of
the next record in the flat file schema, until all records are
processed.
false Processes all input data at one time. This is the default.
createIfNull String (optional) Whether to create the IData object if all the fields are null.
Value Description
true No IS document (IData object) will be created if all the fields are
null. This is the default.
false Always create IS document even though all the fields are null.

skipWhiteSpace String (optional) Whether white space at the beginning of records will be ignored.
Note: The fixed length record parser ignores skipWhiteSpace; it preserves white space.
Value Description
true Ignore white spaces at the beginning of a record. This is the
default.
false Record is used as it is identified (useful for positional data record).
keepResults String (optional) Whether to return the parsed data in the ffValues output parameter.
Value Description
true The parsed ffData will be returned in the output ffValues. This is the
default.
false ffValues will not return data. Use this option when validating the
structure of the ffData against the given flat file schema.
validate String (optional) Whether to return error messages that describe how ffData differs
from the flat file schema.
Value Description
false Do not return error messages describing how ffData differs from
the specified flat file schema. This is the default.
true Return errors describing how the given ffData violates the
constraints described in the flat file schema.
returnErrors String (optional) Whether to return the validation errors. Validation errors are
returned only if validate is set to true.
Value Description
asArray Return any validation errors with the ffData in an array called
errors. This is the default.
inResults Return validation errors in the ffValues object.
both Return validation errors in both errors and ffValues.
maxErrors String (optional) The maximum number of errors that can be returned from one
record. When the flat file parser encounters more than the maximum number of
errors within a record, the parser will stop parsing and return the parsed data and
errors processed up until that point. Validation errors are returned only if validate is
set to true.

flags Document (optional) Flags that you can set to govern convertToValues options.
addRecordCount String Whether you want the service to add an additional field
(@record–count) to each parsed record in the resulting IData object
(ffValues). The @record–count field is used to identify the record
number of each parsed record.
Value Description
true The @record–count field is added to each parsed
record. This field contains the number of the parsed
record. The first parsed record is 1, the second is 2,
etc.
If there are records that are undefined data, the count
of the next defined record will reflect the undefined
data. For example, if the @record–count field for a
record is 2 and that record contains 5 undefined
records, the @record–count field for the next defined
record will be 8.
false The @record–count field is not added to each parsed
record. This is the default.
detailedErrors String Whether you want detailed conditional validation error
information. This flag is only used when validate is true.
Value Description
true When a conditional validation error occurs, the
output errors variable will contain detail information
about all the conditions that were violated. For more
information, see Appendix A, “Validation Errors”.
false When a conditional validation error occurs, the
service does not provide detail error information.
Conditional validators report only whether a
condition failed validation with no additional
information about the conditions that were violated.
This is the default.

skipToFirst String Whether you want the service to wait until it finds the first
Record valid record before reporting invalid records as errors.
Value Description
true The service will wait until it finds the first valid
record before reporting invalid records as errors. This
is the default.
false The service will report invalid records as errors prior
to locating the first valid record.
trimWhitespace String Whether you want the service to delete any blank spaces at
the beginning of fields, at the end of fields, or both.
Value Description
none The service will not delete any blank spaces from
fields. This is the default.
left The service will delete all blank spaces at the
beginning of all fields.
right The service will delete all blank spaces at the end of
all fields.
both The service will delete all blank spaces at the
beginning and end of all fields.
Output Variables
ffValues Document The IData object that represents the input flat file data.

ffIterator Object (optional) An object that encapsulates and keeps track of the input records
during processing. It is used only when the iterate variable has been set to true. When
all input data has been processed, the object becomes null. When the ffIterator variable
is null, you should exit the LOOP to discontinue processing.
isValid String Whether flat file contains validation errors.
Value Description
true The validate input variable was set to true and no errors were found.
false The validate input variable was set to true and errors were found, or
the validate input variable was set to false.
errors String (optional) An array containing the validation errors, if any, that were found in
ffData. For more information about validation error codes see Appendix A, “Validation
Errors” of this guide.

pub.flatFile:getSupportedEncodings
This service returns a list of supported encodings. This service will only report webMethods encodings,
not Java defaults. That is, if you do not have converters.jar installed, it returns null.
Input Variables
None
Output Variables
encodings String List A list of supported encodings.
Flat File Generation Services

Use the services in the pub.flatFile.generate folder to create, modify, and delete flat file dictionary entries,
entire flat file dictionaries, and flat file schemas. For more information about how to use these services, see
Appendix C, “Programmatically Creating Flat File Schemas and Dictionaries”.
The sample.flatFile.generateFFSchema folder, which is in the WmFlatFileSamples package, contains sample
services that illustrate how to use the services in the pub.flatFile.generate folder. The sample services are the
sample.flatFile.generateFFSchema:delimited service and the sample.flatFile.generateFFSchema:fixedLength service,
which are described in “Flat File Generation Sample Services” on page 111.
pub.flatFile.generate:createDocumentType
Creates an IS document type that defines the XML representation of a flat file schema.
Input Variables
FlatFileSchema String The fully–qualified name of the flat file schema for which you want to
generate an IS document type.
PackageName String The name of the Integration Server package in which you want the created
IS document type to be placed.
DocumentTypeName String The fully–qualified name that you want to assign to the created IS
document type.
Output Variables
None

pub.flatFile.generate:createFFDictionary
Creates an empty flat file dictionary. This service throws an exception is if the flat file dictionary you want
to create already exists when the service is invoked.
Input Variables
FFDictionaryName String The fully–qualified name of the flat file dictionary you want to create.
PackageName String The name of the Integration Server package in which you want the created
flat file dictionary to be placed.
Output Variables
None
pub.flatFile.generate:deleteFFDictionary
Deletes a flat file dictionary. Before deleting the dictionary, the Integration Server determines if other
dictionaries depend on the dictionary being deleted, and gives the user the option of cancelling the
deletion.
Input Variables
FFDictionaryName String The fully qualified name of the flat file dictionary that you want to delete.
Output Variables
deleted String Whether the flat file dictionary was successfully deleted; deleted will be either
true or false.
Value Description
true The flat file dictionary was successfully deleted.
false The flat file dictionary was not successfully deleted.
Usage Note
Before you run this service, you should run the pub.flatFile.generate:findDependants service to return the names
of all flat file schemas and dictionaries that are dependent on the dictionary you are deleting.

pub.flatFile.generate:deleteFFDictionaryEntry
Deletes a single entry from a flat file dictionary.
Input Variables
FFDictionaryName String The fully–qualified name of the flat file dictionary that contains the entry
that you want to delete.
EntryName String The name of the entry that you want to delete.
EntryType String The type of entry that you are deleting. Specify Record, Composite, or
Field.
Output Variables
deleted String Whether the flat file dictionary entry was successfully deleted; deleted will be
either true or false.
Value Description
true The flat file dictionary entry was successfully deleted.
false The flat file dictionary entry was not successfully deleted.
pub.flatFile.generate:deleteFFSchema
Deletes a flat file schema.
Input Variables
FFSchemaName String The fully–qualified name of the flat file schema that you want to delete.
Output Variables
deleted String Whether the flat file schema was successfully deleted; deleted will be either
true or false.
Value Description
true The flat file schema was successfully deleted.
false The flat file schema was not successfully deleted.

pub.flatFile.generate:findDependants
Returns the names of all flat file schemas and dictionaries that are dependent on a given flat file dictionary.
Input Variables
ffDictionaryName String The name of the flat file dictionary whose dependants you want to find.
Output Variables
dependants Document List The dependant objects and the packages that contain them.

packageName String The name of the package that contains the dependant
object.
name String The name of the dependant object.
pub.flatFile.generate:findReferences
Returns the names of all flat file dictionaries that are referenced by a given flat file dictionary or flat file
schema.
Input Variables
name String The name of the flat file dictionary or flat file schema whose references you
want to find.
Output Variables
references Document List The referenced objects and the packages that contain them.

packageName String The name of the package that contains the referenced
object.
name String The name of the referenced object.
pub.flatFile.generate:getFFDictionaryAsXML
Returns a dictionary as an XML string.
Input Variables
FFDictionaryName String The fully–qualified name of the flat file dictionary that you want returned
as XML.

Output Variables
FFXML String The returned flat file dictionary as an XML string. The returned XML
string conforms to the pub.flatFile.generate:FFDictionary IS document type.
pub.flatFile.generate:getFFDictionaryEntryAsXML
Returns a single dictionary entry as an XML string.
Input Variables
FFDictionaryName String The fully–qualified name of the flat file dictionary that contains the entry
that you want returned as XML.
EntryName String The name of the entry that you want to returned as XML.
EntryType String The type of entry that you want returned. Specify Record, Composite, or
Field.
Output Variables
FFXML String The returned flat file dictionary entry as an XML string. The returned XML
string conforms to the pub.flatFile.generate:FFDictionary IS document type.
pub.flatFile.generate:getFFSchemaAsXML
Returns the specified flat file schema as an XML string.
Input Variables
FFSchemaName String The fully–qualified name of the flat file schema that you want returned as
XML.
Output Variables
FFXML String The returned flat file schema as an XML string. The returned XML string
conforms to the pub.flatFile.generate:FFSchema IS document type.

pub.flatFile.generate:listFFDictionaryEntries
Lists all entries in a specified flat file dictionary that are of a specified type.
Input Variables
FFDictionaryName String The fully–qualified name of the flat file dictionary that contains the entries
that you want listed.
EntryType String The type of entries that you want listed. Specify Record, Composite, or
Field.
Output Variables
EntryNames String List The list of returned flat file dictionary entries.
pub.flatFile.generate:saveXMLAsFFDictionary
Creates a flat file dictionary in the Integration Server namespace by converting the specified flat file
dictionary that is in XML format into a namespace flat file dictionary. If a flat file dictionary with the same
name already exists in the Integration Server namespace, use the pub.flatFile.generate:deleteFFDictionary service
to delete the flat file dictionary before invoking this service. This service throws an exception is if a flat file
dictionary with the same name already exists when it is invoked.
Input Variables
FFDictionaryName String The fully–qualified name of the flat file dictionary that you want to create in
the Integration Server namespace.
PackageName String The name of the Integration Server package in which to save the flat file
dictionary.
FFXML String The flat file dictionary (as an XML string) that you want to create in the
Integration Server namespace. The XML string must conform to the
pub.flatFile.generate:FFDictionary IS document type.
Note: To see examples of how to supply the XML string in FFXML by mapping data
from another file, see the samples provided in the WmFlatFileSamples package
and that are described in “Flat File Generation Sample Services” on page 111. For
sample code that shows how to retrieve the data for FFXML from an XML file in
the local file system, see “Creating a Service that Retrieves the XML File” on
page 140 in Appendix C, “Programmatically Creating Flat File Schemas and
Dictionaries”.

maxNumOfErrors String (optional) The maximum number of errors that you want returned. The
default is 100.
The service ensures the flat file dictionary is valid before saving it in the
Integration Server namespace. The validation occurs in two stages.
1 Structural validation of the XML.
2 Logical validation of the XML contents.
If structural validation errors occur, the service reports the structural validation
errors, but does not proceed with logical validation. When the XML string contains
no structural validation errors, the service proceeds with logical validation and
reports any logical validation errors.
Output Variables
saved String Whether the flat file dictionary was saved successfully. It will have one of the
following values.
Value Description
true The flat file dictionary was successfully saved.
false The flat file dictionary was not successfully saved.
Errors String List (optional) Errors that occurred while attempting to save the flat file
dictionary to the Integration Server namespace.
Warnings String List (optional) Warnings about the flat file dictionary that was created.
Usage Note
Use this service to add a new flat file dictionary. Use the pub.flatFile.generate:updateFFDictionaryEntryFromXML if
you want to update one or more entries in a flat file dictionary rather than creating a new flat file
dictionary.
pub.flatFile.generate:saveXMLASFFSchema
Creates a flat file schema in the Integration Server namespace by converting the specified flat file schema
that is in XML format into a namespace flat file schema. If a flat file schema with the same name already
exists in the Integration Server namespace, use the pub.flatFile.generate:deleteFFSchema service to delete the flat
file schema before invoking this service. This service throws an exception is if a flat file schema with the
same name already exists when it is invoked.

Input Variables
FFSchemaName String The fully–qualified name of the flat file schema that you want to create in the
Integration Server namespace.
PackageName String The name of the Integration Server package in which to save the flat file
schema.
FFXML String The flat file schema (as an XML string) that you want to create in the
Integration Server namespace. The XML string must conform to the
pub.flatFile.generate:FFSchema IS document type.
Dictionaries”.
default is 100.
The service ensures the flat file schema is valid before saving it in the Integration
Server namespace. The validation occurs in two stages.
Output Variables
saved String Whether the flat file schema was saved successfully. It will have one of the
following values.
Value Description
true The flat file schema was successfully saved.
false The flat file schema was not successfully saved.
Errors String List (optional) Errors that occurred while attempting to save the flat file
schema to the Integration Server namespace.
Warnings String List (optional) Warnings about the flat file schema that was created.

pub.flatFile.generate:updateFFDictionaryEntryFromXML
Updates one or more entries in a flat file dictionary in the Integration Server namespace. This service goes
through all entries that you specify in the FFXML variable. If an entry with the same name and type
already exists in the flat file dictionary, this service overwrites the existing entry. If the entry does not
already exist, this service creates the entry in the specified flat file dictionary.
Input Variables
FFDictionaryName String The fully–qualified name of the flat file dictionary that contains the entries
that you are replacing and/or adding.
FFXML String The dictionary entries (as an XML string) that you want to use to replace an
existing entry or that you want to add to the flat file dictionary. The XML string in
FFXML must conform to the pub.flatFile.generate:FFDictionary IS document type.
Dictionaries”.
default is 100.
This service ensures the entries are valid before saving them in the flat file
dictionary. The validation occurs in two stages.
Output Variables
saved String Whether the dictionary entry was saved successfully. It will have one of the
following values.
Value Description
true The dictionary entry was successfully saved.
false The dictionary entry was not successfully saved.

Errors String List (optional) Errors that occurred while attempting to save the entry to the

flat file dictionary.
Warnings String List (optional) Warnings about the dictionary entry that was updated or
added.
Flat File Generation IS Document Types

The IS document types in the pub.flatFile.generate folder define the format of the information that you must
provide to add or update dictionary entries, entire flat file dictionaries, and flat file schemas. For more
information about how to use these IS document types, see Appendix C, “Programmatically Creating Flat
File Schemas and Dictionaries”.
pub.flatFile.generate:FFDictionary
This IS document type defines the format to use when supplying a flat file dictionary or dictionary entry
(in the FFXML variable) and the format that services return (in the FFXML variable) when you are
retrieving a flat file dictionary or dictionary entry. The structure for this IS document type is defined in the
following XML schema:
IntegrationServer_directory\packages\WmFlatFile\pub\FFGeneration.xsd
Variables
FFDictionary Document The dictionary entries that you want to add or update. FFDictionary has
the following structure:
RecordDictionary Document List (optional) The dictionary entries for records that
Entry you want to add or update in the flat file dictionary. Leave this
null if you do not want to add or update record entries.
EntryName String The name of the record.
RecordDefinition Document The definition of the record. The
information you specify in a record
definition is the same as the information that
you specify when creating a flat file
dictionary using the Flat File Schema Editor.
For descriptions of the fields, see Chapter 3,
“Creating and Editing Flat File Schemas and
Dictionaries”.

Composite Document List (optional) The dictionary entries for composites

DictionaryEntry that you want to add or update in the flat file dictionary. Leave
this null if you do not want to add or update composite entries.
EntryName String The name of the composite.
Composite Document The definition of the composite.
Definition The information you specify in a composite
definition is the same as the information that
you specify when creating a flat file
dictionary using the Flat File Schema Editor.
For descriptions of the fields, see Chapter 3,
Dictionaries”.
FieldDictionary Document List (optional) The dictionary entries for fields that you
Entry want to add or update in the flat file dictionary. Leave this null if
you do not want to add or update field entries.
EntryName String The name of the field.
FieldDefinition Document The definition of the field. The
information you specify in a field definition
is the same as the information that you
specify when creating a flat file dictionary
using the Flat File Schema Editor. For
descriptions of the fields, see Chapter 3,
Dictionaries”.
Usage Notes
If you are using this IS document type to supply a flat file dictionary as input to the
pub.flatFile.generate:saveXMLAsFFDictionary, be sure to supply all dictionary entries. If you are using this IS
document type to update an existing dictionary, provide only the entries that you want to add or update
and invoke the pub.flatFile.generate:updateFFDictionaryEntryFromXML to update the flat file dictionary.

pub.flatFile.generate:FFSchema
This IS document type defines the format to use when supplying a flat file schema (in the FFXML variable)
and the format that services return (in the FFXML variable) when you are retrieving a flat file schema. The
structure for this IS document type is defined in the following XML schema:
Variables
FFSchema Document The flat file schema that you want to add or update. FFSchema has the
following structure:
Delimiters Document The delimiters used in the flat files that adhere to this
flat file schema. The information that you specify for Delimiters
corresponds to the data you specify on the Flat File Definition tab
in the Flat File Schema Editor. For a description of the fields, see
“Step 1: Add and Configure the Flat File Schema” on page 36.
Document Document The structure of the flat files that adhere to this flat file
Structure schema. The information that you specify for DocumentStructure
corresponds to the data you specify on the Flat File Structure tab in
the Flat File Schema Editor. For a description of the fields, see
“Step 3: Define the Structure of the Flat File Schema” on page 48.
Ordered String Whether the child records appear in the
flat file in the order they are defined in the flat
file schema.
RecordStructure Document List Definitions of the records within
the flat file.
Ordered String Whether the child
records appear in the flat
file in the order they are
defined in the flat file
schema.
RecordUsage Document Information
about how the record is
used, including either the
dictionary reference for
this record or the
definition of the record.

RecordStructure Document List Child records

of this record. This is a
recursive reference to the
RecordStructure defined in
FFSchema/DocumentStructu
re.
RecordParser Document The type of record parser. In this IS document, specify
only the one variable that corresponds to the type of record
parser to use. That is, specify one of FixedLengthParser,
DelimitedParser, VariableLengthParser, or EDIParser. For
DelimitedParser, VariableLengthParser, and EDIParser, you do not
need to specify a value; just have the variable in the pipeline.
DefaultRecord Document (optional) The dictionary name and entry name that
Reference identifies the default record for the flat file schema. If you
specify a default record, when using the flat file schema to parse
a flat file schema, the default record is used for any record that
cannot be recognized.
Record Document Where to locate the identifier to use to correlate a
Identifier record in the flat file to a record definition in the flat file schema.
Specify either the NthFieldIdentifier variable or the
FixedPositionIdentifier variable:
Use NthFieldIdentifier to identify the field in the record
(counting from zero) that contains the identifier.
Use FixedPositionIdentifier to identify the character position
in the record (counting from zero) where the record
identifier is located.
Undefined String Whether you want the pub.flatFile:convertToValues service to
DataAllowed generate undefined data errors when you use this flat file
schema to convert a flat file to an IData object.
Specify true if you want to allow undefined data and do not
want the pub.flatFile:convertToValues service to flag undefined
data errors.
Specify false if you do not want to allow undefined data
and you do want the pub.flatFile:convertToValues service to flag
undefined data errors.
Document String List Areas for this flat file schema. An area is a way to
Areas associate an arbitrary string with a given record.

FloatingRecord String (optional) The name of the record that is defined in the
schema as a floating record.
Note: If the floating record has an alternate name, specify the
alternate name.
Description String Description of the flat file schema.
Flat File Generation Sample Services

The sample.flatFile.generateFFSchema folder, which is in the WmFlatFileSamples package, contains sample
services that illustrate how to use the services in the pub.flatFile.generate folder. The WmFlatFileSamples
package is located in the Knowledge Base on the Advantage Web site.
sample.flatFile.generateFFSchema:delimited
Creates a flat file schema for parsing a delimited flat file. The created flat file schema references a flat file
dictionary for record definitions, which this service also creates. The flat file schema will be created based
on data in a .txt definition file. Each record in the .txt definition file contains comma–separated data. There
are three types of definitions in the .txt file:
RecordDefinition, which defines a type of record that is contained in the flat file. A RecordDefinition
contains the following:
RecordDefinition keyword
Record identifier, which is the identifier for the record being defined
ChildRecord, which follows a RecordDefinition and defines the child records of the previous record that
was defined by a RecordDefinition. A ChildRecord contains the following:
ChildRecord keyword
Child record identifier, which is the identifier for the child record being defined.
Repeat count, which is the maximum number times this child record can appear after its parent
record
FieldDefinition, which follows either a RecordDefinition or a ChildRecord. It defines the fields that are in
the previous record that was defined by a RecordDefinition or a ChildRecord. A FieldDefinition
contains the following:
FieldDefinition keyword
Field name, which is a brief description of the contents of the field, for example, name, address
Field position, which is where the field occurs in the record

The sample.flatFile.generateFFSchema:delimited service parses the .txt definition file and loops over each
RecordDefinition creating a flat file dictionary entry for each. Each record will contain the fields defined
by the corresponding FieldDefinitions. Records will be nested as indicated by the ChildRecords. The first
RecordDefinition defined in the .txt definition file will be considered the root node of the flat file
document. All other records must be children of that record. If a record is listed in the .txt definition file as
a ChildRecord only (and not defined as a RecordDefinition), that record will appear in the flat file schema,
referencing a dictionary entry that does not exist.
The WmFlatFileSamples\pub\sample\flatFile\generateFFSchema\delimited directory contains the
following files to use with the sample:
FFDefinition.txt, which is a sample .txt definition file that you can use as input to this service
FFExample.txt, which is an example of a delimited flat file that can be parsed using the flat file
schema created from the data in the FFDefinition.txt file.
Input Variables
TXTFilename String The name of the .txt definition file to parse.
SchemaName String The fully–qualified name of the flat file schema to create. An error is
generated if this flat file schema already exists.
DictionaryName String The name of the flat file dictionary associated with the flat file schema you
are creating. The dictionary will be created if it does not already exist.
PackageName String The name of the Integration Server package in which to create the flat file
schema and dictionary.
RecordId String The field number that contains the record identifier.
Delimiters Document An IData object that contains the segment terminator and the field and
subfield separators used by the flat files that the flat file schema will be used to
parse.
record String Character used to terminate records.
field String Character used to separate fields.
subfield String Character used to separate subfields.
Output Variables
None

sample.flatFile.generateFFSchema:fixedLength
Creates a flat file schema for parsing a fixed length flat file. The created flat file schema contains record
definitions rather than referencing a flat file schema. The flat file schema will be created based on data in a
CSV definition file. Each record in the CSV definition file contains the following fields:
Record identifier, which is the identifier for the record being defined.
Field name, which is a brief description of the contents of the field, for example, name, address.
Field start position, which is the position of the first byte to extract from the record.
Field length, which is number of bytes in the field.
The CSV file must be sorted by the record identifier. The sample.flatFile.generateFFSchema:fixedLength service
reads the CSV file and creates the flat file schema to parse a fixed length flat file.
The WmFlatFileSamples\pub\sample\flatFile\generateFFSchema\fixedLength directory contains the
following files to use with the sample:
FFDefinition.csv, which is a sample CSV file that you can use as input to this service
FFExample.txt, which is an example of a fixed length flat file that can be parsed using the flat file
schema created from the data in the FFDefinition.csv file.
Input Variables
CSVFilename String The fully–qualified name of the CSV definition file to parse.
SchemaName String The fully–qualified name of the flat file schema to create. An error is
generated if this flat file schema already exists.
PackageName String The name of the Integration Server package in which to create the flat file
schema and dictionary.
RecordLength String The number indicating the length of each record defined by the flat file
schema.
RecordId String The number that indicates where in the record the record identifier starts.
Output Variables
None


Appendix A. Validation Errors
Validation Error Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Validation Error Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

A. Validation Errors
Validation Error Contents

When the validate variable of the pub.flatFile:convertToValues service is set to true and an
object within the flat file does not conform to the flat file schema, the service generates
errors when validating the flat file. If the service finds that an object is invalid, it returns
validation errors in the errors output of the convertToValues service.
Example of Validation Results
This errors array contains the following types of information about the errors that
occurred in a record during validation.
Each error in a record generates one entry in the errors array. For more information
about the fields in the errors array, see “General Error Entries in the errors Array” on
page 117.
Each error in a child record also generates one item within the parent record, which
appears nested in the error array in the variable childErrors.
Errors with conditional validators can generate detail information about the
conditions violated in a child errorDetails array, one entry in the array for each violated
condition. The pub.flatFile:convertToValues service only generates details about violated
conditions when the flag/detailedErrors input variable of the pub.flatFile:convertToValues

service is set to true. For more information, see “Entries for Conditional Validator
Errors in errorDetails Array” on page 117.
General Error Entries in the errors Array

An entry in the errors array contains the following information. For information about
errorDetails entries, see “Entries for Conditional Validator Errors in errorDetails Array”
on page 117.
Variable in errors Description

errorPosition Indicates where the error occurred within the parent element. For
a record, this indicates the position of the record within the flat file.
For a composite, this indicates the position of the field within the
composite. For a nested child record, this indicates the position of
the field within the record.
errorMessage A brief description of the error.
errorCode A number indicating the type of error. For a list, see “Validation
Error Codes” on page 120.
reference The name of the element in the flat file schema that contained the
error.
badData The value of the data that failed validation.
childErrors (Array). Indicates that a field or subfield within the record or
composite generated a validation error.
Entries for Conditional Validator Errors in errorDetails Array

The errorDetails array includes detail information about the conditions that were violated
when the following are true:
When you set the validate and flag/detailedErrors input variables of the
pub.flatFile:convertToValues service to true.
–AND–
The pub.flatFile:convertToValues service encounters errors with conditional validator
The following shows a sample of the errors array that includes the errorDetails array, which
contains details about violated conditions. See the table below the sample for more
information.

Example of Validation Results with Conditional Validation Errors
Portion of the array Description
1 errors For a description of this portion of the errors array, see “General
Error Entries in the errors Array” on page 117.
When the flag/detailedErrors input variable of the
pub.flatFile:convertToValues service is set to false, this is the only
information generated in the errors array about violated conditions.
When the flag/detailedErrors is true, the pub.flatFile:convertToValues
service generates the detail in errorDetails array described below.

Portion of the array Description
2 errorDetails To provide information about the conditions that were violated, the
convertToValues service generates an entry in the errorDetails array for
each violated condition.
Variable in
errorDetails Description
errorPosition Always zero; the errorPosition is not meaningful at
this level of the errorDetails array.
errorMessage A brief description of the condition that was
violated.
errorCode A number indicating the type of error. For a list,
see “Validation Error Codes” on page 120.
reference Always null; the reference is not meaningful at this
level of the errorDetails array.
badData Always null; the badData is not meaningful at this
level of the errorDetails array.
condition The condition that was violated.
errorDetails Details about how the condition was violated. See
the description below.
3 errorDetails The child errorDetails array contains detail about how the condition
was violated. The convertToValues service generates an entry for each
way the condition was violated.
Variable in
errorDetails Description
errorPosition Indicates the position in the record that contains
the field that caused the condition to be violated.
errorMessage A brief description of the condition that was
violated.
errorCode A number indicating the type of error. For a list,
see “Validation Error Codes” on page 120.
reference The name of the element in the flat file schema that
contained the error.
badData The value of the data that failed validation.

Validation Error Codes

The following table describes the validation error codes that you might receive when the
validate variable of the pub.flatFile:convertToValues service is set to true and you are validating
a flat file or testing a flat file schema.
Error Code Description

1 In the flat file schema, the Mandatory drop‐down menu was set to true for
this element, but the element does not occur in the flat file.
2 Reserved for future use.
3 Unexpected element. This field is not allowed in the record or composite
in which it appears.
4 In the flat file schema, a length validator was specified in the Validator
property for this field. The value in the flat file exceeded the maximum
length specified.
5 In the flat file schema, a length validator was specified in the Validator
property for this field. The value in the flat file did not meet the minimum
length specified.
7 In the flat file schema, a code list validator was specified in the Validator
property for this field. The value in the flat file was not listed in the flat file
schema as a valid value.
10 In the flat file schema, a conditional validator was specified in the Validator
property for this composite or record. The contents of this composite or
record did not meet the conditional requirements. The errorMessage
variable contains the number of the condition that failed. If you had a
validation string of C010203R0405 and both conditions failed, the error
message would state that rules 0 and 1 were violated. If only the second is
violated, it states that the rule 1 was violated.
If you require more detail about conditional validator errors, set the
flag/detailedErrors input variable of the pub.flatFile:convertToValues service to
true. For a description of the error details that convertToValues service
generates when you set flag/detailedErrors to true, see “Entries for
Conditional Validator Errors in errorDetails Array” on page 117.

Error Code Description

11 Indicates that this record is not defined anywhere in the flat file schema.
You will receive this error only when you have not specified a Default
Record or selected Allow Undefined Data where this record appears in the flat
file. If you have specified either of these properties, you will not receive
this error.
12 Indicates that this record is defined in this flat file schema, but it occurs in
the flat file in a location prohibited by the flat file schema.
14 In the flat file schema, you specified a maximum repeat value in the Max
Repeat property for a particular record. The number of record instances in
the flat file exceeded the specified maximum number of repeats.
16 Within a record, this indicates that the record contains a composite or field
error. For a composite, this indicates the that composite contains a subfield
error.
17 A string could not be formatted into the intended format. A format service
reported that the data could not be formatted properly. For information
about field format services, see “Format Services” on page 66 of Chapter 3,
“Creating and Editing Flat File Schemas and Dictionaries” of this guide.
18 Indicates that a conditional validation rule requires a field, but that field is
not present in the data.
19 Indicates that a field is excluded by a conditional validation rule, but the
field is present in the data.


Appendix B. Flat File Byte Count Parser
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Handling Partial Characters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Stateful Encodings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

B. Flat File Byte Count Parser
Overview
The flat file parser measures the lengths and positions of all records and fields in flat file
schemas in terms of bytes, not characters. Prior to Version 6.5, Service Pack 2, the parser
measured all lengths and positions in characters.
Note: This change does not affect users who currently parse flat files using single‐byte
encodings because one byte equals one character. Thus, there is no functional difference
between parsing bytes and characters.
This parser supports both single‐byte encodings (equivalent to character encodings) and
multi‐byte encodings. With multi‐byte encodings, a byte count can differ from a character
count since a single character can be encoded to multiple bytes. Both the convertToValues
and the convertToString services support byte‐count parsing for multi‐byte encodings. In
addition, convertToString now includes an optional setting that returns a document as a byte
array instead of a string (see “pub.flatFile:convertToString” on page 90).
Important! Multi‐byte encoded files must run on JVM version 1.4 or later.
There is no change in the way in which the parser handles the following entities:
Delimited fields and records.
A delimiter may be a single character, but may not be byte based. Delimiter and
release character extractors still extract the specified character from the input file
based on a character position.
The field length validator.
The parser still uses the String.length() method to validate field lengths (in
characters). However, the byte count validator validates the total number of bytes in a
field.
Important! The parser does not support the writing or reading of binary data (e.g., COMP‐3
COBOL fields), nor does it support multiple kinds of encodings within one file.
Handling Partial Characters

The parser reads and writes only complete characters. Partial characters are characters
that violate character boundary conditions, as described below.

Reading Partial Characters

The following table describes how the parser reads partial characters:
Character Boundary Condition How the Parser Handles the Condition

Reading a fixed position field that The field starts on the next complete character.
begins in the middle of a multi‐byte The partial character is ignored.
character.
Reading a fixed position field that The field ends on the previous complete
ends in the middle of a multi‐byte character. The partial character is not included
character. in the field.
Reading a fixed position record that If the bytes encode to an invalid character, an
begins or ends in the middle of a exception is thrown.
multi‐byte character.
Note: If the bytes that begin or end the record are
encoded to a valid character, it will be the wrong
character.
To illustrate a case where the parser reads fixed position records that begin and end in the
middle of a multi‐byte character, consider the following multi‐byte encoding:
12121212
These eight bytes represent four two‐byte characters. If we specified a fixed length file
with a record length of three bytes instead of two bytes, the parser would read this
encoding as follows, producing an undesirable result:
Record 1: 121
Record 2: 212
Record 3: 12
Note that:
Record 1 contains one character, formed by the first two bytes of the file. The last byte
is a partial character; the parser ignores it.
Record 2 contains one character, formed by the fifth and sixth bytes of the file. The
character formed by the third and fourth bytes is lost because these bytes span
Records 1 and 2, and cannot be properly decoded.
Record 3 contains one character, formed by the seventh and eighth bytes.

Writing Partial Character Encodings

Partial characters present a similar problem when writing to a fixed length file. The parser
considers all fixed position fields to be “self contained”. This means that all encoding
information for a fixed position field is contained in the byte range specified for the field.
Keep this in mind when writing multi‐byte encodings to fixed length fields because it is
possible to specify a field or record that does not end on a character boundary.
Consider a fixed length field that is 10 bytes, but the string for that field encodes to more
than 10 bytes. In this case, the parser will truncate the byte array to fit into 10 bytes. This
could result in the creation of invalid characters. Thus, the parser always truncates a string
on a character boundary; only complete characters are written to the output file.
The following table describes how the parser writes partial characters:
Character Boundary Condition How the Parser Handles the Condition

Writing a string to a fixed position Truncates the string to fit the field.
field where the string is longer than
the fixed position field (where it
breaks at a character boundary).
Writing to a fixed position field that The field ends on the previous complete
ends in the middle of a multi‐byte character. The partial character is not included
character. in the field, and is replaced by one or more pad
characters. For an example, see below.
Writing to a fixed position field in Does not generate an error during creation of the
the middle of a delimited field that file. Parsing the created file will likely result in
contains a stateful encoding. an encoding error.
To illustrate a case where the parser writes to a fixed position field that ends in the middle
of a multi‐byte character, consider the following multi‐byte encoding:
Number of
Field Bytes in Field Character 1 Character 2
Field1 4 12 12
Field2 4 12 345
The parser encodes this multi‐byte encoding as follows:
Field Value
Field1 1212
Field2 12PP

The parser encodes Field1 properly; it considers character 1 and character 2 to be complete
characters.
The parser encodes Field2 as follows:
It considers character 1 to be a complete character
Since byte 3 does not begin on a character boundary, the parser considers character 2
to be a partial character. It truncates bytes, 3, 4, and 5 because those three bytes would
extend beyond the end of the field. It replaces these three bytes with two pad
characters (represented by PP).
Stateful Encodings
The parser can read any well‐formed document that adheres to the rules of the
document’s particular kind of encoding. The parser can only write to files that have fixed
length fields or delimited records.
Note: Escape encodings behave similarly to stateful encodings. The parser truncates
characters preceding the escape sequence so that the integrity of the escape sequence is
maintained.
Stateful encodings have two specific bytes that indicate when the record’s state has
changed:
| is a “shift‐in” byte that indicates a change from the normal state to an alternate state
O (the character O, not zero) is a “shift‐out” byte that indicates a change from the
alternate state to the normal state
Writing Stateful Encodings

Consider a record definition with the following four‐byte fields:
Start End
Field Position Position Record Value
A 0 3 Character value “AB” encodes to 5 bytes: |123O.
The character “A” is represented by bytes 1 and 2.
The character “B” is represented by byte 3.
B 4 8 Character value “C” encodes to 4 btyes: |45O.
The parser encodes this record as follows:
|12O|45O

Notice that the parser truncated byte 3 (the character “B”) from the first field so that the
field would fit within the four‐byte limit. This truncation occurred on the character
boundary between the characters A and B, creating a properly encoded record.
Note: A different method of encoding, using a “padding” byte, would have produced this
result:
|12345OP
where P is an added padding byte, to extend the record length to the required eight bytes.
The parser does not use this method of encoding. Using this method, extracting the value
for the second field produces:
45OP
The parser cannot properly write this byte sequence. All fixed position fields must be
encoded as if they were a stand‐alone sequence of bytes. No stateful encoding information
may be carried from one fixed position field to any other fixed position field. However,
delimited fields may carry stateful information from one field to the next field. In contrast,
delimited records may not carry stateful information from one record to the next record.
Thus, delimited fields are not necessarily “self contained”, but delimited records are.

Appendix C. Programmatically Creating Flat File Schemas and
Dictionaries
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Creating Flat File Dictionary Entries, Dictionaries, and Schemas . . . . . . . . . . . . . . . . . . . . . . . 130
Modifying Flat File Dictionary Entries, Dictionaries, and Schemas . . . . . . . . . . . . . . . . . . . . . . 134
Deleting Flat File Dictionary Entries, Dictionaries, and Schemas . . . . . . . . . . . . . . . . . . . . . . . 140
Sample Flow Services for Working with XML Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

C. Programmatically Creating Flat File Schemas and Dictionaries
Overview
You can use the services in the pub.flatFile.generate folder of the WmFlatFile package to
programmatically create, modify, and delete:
flat file dictionary entries
entire flat file dictionaries
flat file schemas
To specify the data for the flat file dictionary entries, entire flat file dictionaries, or flat file
schemas, you can:
Create the information in an XML file and execute a service you create that retrieves the
XML data from the XML file, then invokes the appropriate services in the
pub.flatFile.generate folder to make the changes in the Integration Server namespace.
When creating the XML data, it should conform to the following XML schema:
Create a service that maps the data to an IS document (IData object), convert the IData object

to an XML string, and then invoke the appropriate services in the pub.flatFile.generate
folder to make the changes in the Integration Server namespace. The IS document
type to which the IData object must conform is provided in the WmFlatFile package:
For a flat file dictionary entry or dictionary, use the pub.flatFile.generate:FFDictionary IS
document type.
For a flat file schema, use the pub.flatFile.generate:FFSchema IS document type.
The sample.flatFile.generateFFSchema folder, which is in the WmFlatFileSamples package,
contains sample services that illustrate how to use the services in the pub.flatFile.generate
folder. The sample services are the sample.flatFile.generateFFSchema:delimited service and the
sample.flatFile.generateFFSchema:fixedLength service, which are described in “Flat File
Generation Sample Services” on page 111.
Creating Flat File Dictionary Entries, Dictionaries, and Schemas

To create a flat file dictionary entry, entire flat file dictionary, or flat file schema, you
supply the data for the item you want to create; then invoke the appropriate service to
create the item in the Integration Server namespace.

Creating Flat File Dictionary Entries

The following describes the basic steps to create flat file dictionary entries when using
either an XML file or mapping data:
Using this method Follow this procedure

XML File 1 Supply the data for the entries you want to create by creating
the XML file that contains the data and that conforms to the
XML schema:
IntegrationServer_directory\packages\WmFlatFile\
pub\FFGeneration.xsd
2 Retrieve the contents of the XML file as an XML string. For
more information, see “Creating a Service that Retrieves the
XML File” on page 140.
3 Create the entries in the Integration Server namespace by
invoking the pub.flatFile.generate:updateFFDictionaryEntryFromXML
service.
Mapping Data 1 Map the data for the entries you want to create to an IData
object that conforms to the pub.flatFile.generate:FFDictionary IS
document type.
2 Convert the IData object to an XML string by invoking the
pub.xml:documentToXMLString service. When you invoke this
service:
Set the encode input variable to true.
Set the documentTypeName input variable to
pub.flatFile.generate:FFDictionary.
For more information about the pub.xml:documentToXMLString
service, see the webMethods Integration Server Built‐In Services
Reference.
3 Create the entries in the Integration Server namespace by
invoking the pub.flatFile.generate:updateFFDictionaryEntryFromXML
service.

Creating an Entire Flat File Dictionary with Data

The following describes the basic steps to create an entire flat file dictionary when using
either an XML file or mapping data. This procedure describes how to create a flat file
dictionary that contains data. If you want to create an empty flat file dictionary, see
“Creating an Empty Flat File Dictionary” on page 133.
Important! The flat file dictionary you are creating must not already exist in the Integration
Server namespace. If the flat file dictionary already exists, the
pub.flatFile.generate:saveXMLAsFFDictionary service throws an exception.

XML File 1 Supply the data that includes all entries for the dictionary you
want to create by creating an XML file that contains the data
and that conforms to the XML schema:
3 Create the dictionary in the Integration Server namespace by
invoking the pub.flatFile.generate:saveXMLAsFFDictionary service.
Mapping Data 1 Map the data for all entries for the dictionary you want to
create to an IData object that conforms to the
service:
Reference.
invoking the pub.flatFile.generate:saveXMLAsFFDictionary service.

Creating an Empty Flat File Dictionary

The following describes how to create an empty flat file dictionary. If you want to create a
flat file dictionary that contains data, see “Creating an Entire Flat File Dictionary with
Data” on page 132.
Important! The flat file dictionary you are creating must not already exist in the Integration
Server namespace. If the flat file dictionary already exists, the
pub.flatFile.generate:createFFDictionary service throws an exception.
Follow this procedure

1 Create the empty dictionary in the Integration Server namespace by invoking the
pub.flatFile.generate:createFFDictionary service.
Creating a Flat File Schema

The following describes the basic steps to create a flat file schema when using either an
XML file or mapping data:
Important! The flat file schema you are creating must not already exist in the Integration
Server namespace. If the flat file schema already exists, the
pub.flatFile.generate:saveXMLASFFSchema service throws an exception.

XML File 1 Supply the data for the flat file schema you want to create by
creating an XML file that contains the data and that conforms
to the XML schema:
3 Create the flat file schema in the Integration Server namespace
by invoking the pub.flatFile.generate:saveXMLASFFSchema service.


Mapping Data 1 Map the data for the flat file schema you want to create to an
IData object that conforms to the pub.flatFile.generate:FFSchema IS
document type.
service:
pub.flatFile.generate:FFSchema.
Reference.
invoking the pub.flatFile.generate:saveXMLASFFSchema service.
Modifying Flat File Dictionary Entries, Dictionaries, and

Schemas
This section describes how to modify an existing flat file dictionary entry, entire flat file
dictionary, or flat file schema.
Modifying an Existing Flat File Dictionary Entry

To modify an existing flat file dictionary entry, you first retrieve from the Integration
Server namespace the dictionary entry that you want to modify. You make your
modifications to the data; then invoke the appropriate service to write the changes back to

the Integration Server namespace. The following describes the basic steps to modify a
dictionary entry either using an XML file or mapping data.

XML File 1 Retrieve the existing information for the dictionary entry from
the Integration Server namespace by invoking the
pub.flatFile.generate:getFFDictionaryEntryAsXML service and write it to
an XML file. For more information, see “Retrieving Namespace
Data to Write to an XML File” on page 141.
2 Update the data for the dictionary entry in the XML file. The
XML file must conform to the XML schema:
4 Update the dictionary entry in the Integration Server
namespace by invoking the
pub.flatFile.generate:updateFFDictionaryEntryFromXML service.
Mapping Data 1 Retrieve the existing information for the dictionary entry from
pub.flatFile.generate:getFFDictionaryEntryAsXML service. The data is
returned as an XML string in the FFXML variable.
2 To convert the XML string in the FFXML variable to an IData
object:
a Invoke the pub.xml:xmlStringToXMLNode service to convert the
XML string to an XML node.
b Invoke the pub.xml:XMLNodeToDocument service to convert the
XML node to an IData object. When you invoke this
service:
Set the makeArrays input variable to false.
This creates an IData object that conforms to the
For more information about the pub.xml:xmlStringToXMLNode and
pub.xml:XMLNodeToDocument services, see the webMethods
Integration Server Built‐In Services Reference.
3 Map data to the IData object to make your changes.


service:
Reference.
5 Update the dictionary entry in the Integration Server
namespace by invoking the
pub.flatFile.generate:updateFFDictionaryEntryFromXML service.
Modifying an Existing Flat File Dictionary
To modify an existing flat file dictionary, you first retrieve from the Integration Server
namespace the item that you want to modify. You make your modifications to the data.
Delete the dictionary from the namespace before invoking the appropriate service to write
the changes back to the Integration Server namespace. The following describes the basic
steps to modify a flat file dictionary either using an XML file or mapping data.

XML File 1 Retrieve the existing information for the flat file dictionary
from the Integration Server namespace by invoking the
pub.flatFile.generate:findDependants service and write it to an XML
file. For more information, see “Retrieving Namespace Data to
Write to an XML File” on page 141.
2 Update the data for the flat file dictionary in the XML file. The
4 Delete the existing flat file dictionary from the Integration
Server namespace by invoking the
pub.flatFile.generate:deleteFFDictionary service.
5 Create the flat file dictionary in the Integration Server
namespace again by invoking the
pub.flatFile.generate:saveXMLAsFFDictionary service.


Mapping Data 1 Retrieve the existing information for the flat file dictionary
from the Integration Server namespace by invoking the
pub.flatFile.generate:findDependants service. The data is returned as
an XML string in the FFXML variable.
object:
service:
service:
Reference.
5 Delete the existing flat file dictionary from the Integration
Server namespace by invoking the
pub.flatFile.generate:deleteFFDictionary service.
6 Create the flat file dictionary in the Integration Server
namespace again by invoking the
pub.flatFile.generate:saveXMLAsFFDictionary service.

Modifying an Existing Flat File Schema

To modify an existing flat file dictionary or flat file schema, you first retrieve from the
Integration Server namespace the item that you want to modify. You make your
modifications to the data. Delete the item from the namespace before invoking the
appropriate service to write the changes back to the Integration Server namespace. The
following describes the basic steps to modify a flat file schema either using an XML file or
mapping data.

XML File 1 Retrieve the existing information for the flat file schema from
pub.flatFile.generate:getFFSchemaAsXML service and write it to an
XML file. For more information, see “Retrieving Namespace
Data to Write to an XML File” on page 141.
2 Update the data for the flat file schema in the XML file. The
4 Delete the existing flat file schema from the Integration Server
namespace by invoking the pub.flatFile.generate:deleteFFSchema
service.
again by invoking the pub.flatFile.generate:saveXMLASFFSchema
service.


Mapping Data 1 Retrieve the existing information for the flat file schema from
pub.flatFile.generate:getFFSchemaAsXML service. The data is
returned as an XML string in the FFXML variable.
object:
service:
pub.flatFile.generate:FFSchema IS document type.
service:
Reference.
5 Delete the existing flat file schema from the Integration Server
namespace by invoking the pub.flatFile.generate:deleteFFSchema
service.
again by invoking the pub.flatFile.generate:saveXMLASFFSchema
service.

Deleting Flat File Dictionary Entries, Dictionaries, and Schemas

To delete a flat file dictionary entry, entire flat file dictionary, or flat file schema, you
invoke the appropriate service.
For this item... Use this service to delete the item from the namespace
flat file dictionary entry pub.flatFile.generate:deleteFFDictionaryEntry
entire flat file dictionary pub.flatFile.generate:deleteFFDictionary
flat file schema pub.flatFile.generate:deleteFFSchema
Sample Flow Services for Working with XML Files

This section shows sample flow services that show how to retrieve data from an XML file
in the local file system and how to retrieve data from the Integration Server namespace
that can be written to a file in the local file system.
Creating a Service that Retrieves the XML File

The following shows a sample flow service for retrieving data from an XML file.
Sample code for retrieving data from an XML file

1
Flow
Operation Description
1 Invoke the pub.file:getFile service to retrieve the XML file from the local file
system. The XML data must conform to the following XML schema:
IntegrationServer_directory\packages\WmFlatFile\pub\
FFGeneration.xsd
For more information about this service, see the webMethods Integration
Server Built‐In Services Reference.

Flow
2 Invoke the pub.string:bytesToString service to convert the file (in bytes format)
to a String. For more information about this service, see the webMethods
3 Invoke the appropriate service to make your changes to the Integration
Server namespace. In this sample, the flow service invokes the
pub.flatFile.generate:saveXMLASFFSchema to save the XML data as a flat file
schema in the Integration Server namespace.
With the INVOKE saveXMLAsFFSchema flow operation selected, in the
pipeline, map the output from the pub.string:bytesToString service to the
FFXML input of the pub.flatFile.generate:saveXMLASFFSchema service.
Retrieving Namespace Data to Write to an XML File

The following shows a sample flow service that retrieves data from the Integration Server
namespace that can be written to an XML file.
Sample code for retrieving namespace data that can be written to an XML file
1
Flow
1 Invoke the appropriate service to retrieve the Integration Server
namespace data for a flat file dictionary entry, entire flat file dictionary, or
flat file schema. In this sample, the flow service invokes the
pub.flatFile.generate:findDependants service to retrieve the data for an entire flat
file dictionary.

Flow
2 Invoke the pub.string:stringToBytes service to convert the namespace data (in
String format) to a byte[]. For more information about this service, see the
webMethods Integration Server Built‐In Services Reference.
With the INVOKE stringToBytes flow operation selected, in the pipeline, map
the FFXML output from the pub.flatFile.generate:findDependants service to the
input of the stringToBytes service.
3 Add code to or invoke a service that you create to write the byte[] to an
XML file.

Index
Index
A content type, flat file 78

access permissions, setting 73 conventions used in this document 7
alternate code sets, enabling use of 74 converting flat file to IS document (IData) 27
alwaysSortFields variable 74 converting IS document (IData) to flat file 31
convertToString service 90
convertToValues service 93
B
createDocumentType service 98
byte count validator, field 65 createFFDictionary service 99
creating flat file dictionaries 69
C adding entries programmatically 131
character position field extractor 11 programmatically 132
choosing services to receive flat files 79 programmatically creating empty dictionaries 133
code list validator 64 when to create 19
code sets, alternate creating flat file schemas 36
enabling use of 74 configuring the structure 48
composite adding element definitions 49
defined 10 adding element references 49
extractors 61 record definition properties 55
reusing definitions 18 programmatically 133
concepts setting up flat file schemas 36
flat file dictionaries 18 specifying properties 41
flat file schemas 12
flat files 10 D
flat files supported 11
default records 18
overview 10
deleteFFDictionary service 99
conditional validators 62
deleteFFDictionaryEntry service 100
configuration file
deleteFFSchema service 100
alwaysSortFields variable 74
deleting
customizing 74
flat file dictionaries entries programmatically 140
enableFormatting variable 66
flat file dictionaries programmatically 140
spacePadJustifies variable 76
flat file dictionaries, managing dependencies on 72
useAlternateCodeSets variable 74
flat file schemas programmatically 140
useReadableDelimiterReporting variable 75
fomat services, managing dependencies on 67
configuring flat file schema structure 48
delimited field extractors 11
adding element definitions 49
delimited records 11
adding element references 49
delimiter record parser
record definition properties 55
defined 36
constraint types in flat file schemas 12
properties 39
content handler, flat file 78
delimiters, specifying 37

Index
delimiters, using readable and non-readable delimiters 75 sorting 74

diagrams validators 64
flat file processing 24 findDependants service 101
inbound conversion and validation 26 findReferences service 101
outbound conversion 31 fixed length record parser
dictionaries, flat file defined 36
creating empty dictionaries programmatically 133 properties 40
creating programmatically 132 fixed length records 11
deleting programmatically 140 flat file dictionaries
managing dependencies on 72 creating 69
modifying programmatically 136 creating empty dictionaries programmatically 133
retrieving data for from XML file 140 creating entries programmatically 131
dictionary entries, flat file creating programmatically 132
creating programmatically 131 defined 18
deleting programmatically 140 deleting entries programmatically 140
modifying programmatically 134 deleting programmatically 140
retrieving data for from XML file 140 editing 72
documentation Flat File Schema Editor 19
additional 8 identifying 20
conventions used 7 modifying entries programmatically 134
feedback 8 modifying programmatically 136
retrieving data for from XML file 140
E setting access permissions 73
EDI record parser 37 flat file dictionary entries, retrieving data for from XML file 140
editing an existing flat file schema or dictionary 72 Flat File Schema Editor 19
elements 10 Flat File Defintion tab 20
reusing definitions 18 Flat File Structure tab 20
extractors 61 Properties panel 21
flat file schemas
constraints
F
content type 12
ff.cnf configuration file structural 12
alwaysSortFields variable 74
syntax rule 12
customizing 74 creating 36
enableFormatting variable 66 creating programmatically 133
defined 12
useAlternateCodeSets variable 74 deleting programmatically 140
useReadableDelimiterReporting variable 75 editing 72
FFDictionary IS document type 107
Flat File Schema Editor 19
FFSchema IS document type 109 identifying 20
field identifying records 14
defined 10
modifying programmatically 138
extractors 61 retrieving data for from XML file 140
reusing definitions 18

Index
setting access permissions 73 H

testing 68 handling large flat files 30
validation 12 iterator variable of convertToValues service 94
flat file user interface
See Flat File schema Editor
I
flat files
creating from IS documents (IData objects) 31 identifiers, record 15
identifying flat file schema records 14
defined 10
mapping 27 identifying flat file schemas and dictionaries 20
processing 24 inbound flat file processing 25
default records 18
sending and receiving 78
content handler 78 diagram 26
content type 78 how to 26
large files 30
services for receiving 79
via a flow service 79 mapping data 27
via a string variable 79 parsing 26
undefined data 16
via e-mail 84
via file polling 84 validation error codes 12
via FTP 81 validation error contents 12
IS documents (IData objects), creating from flat files 26
via HTTP 80
via Trading Networks 78
services 88 L
types supported 11 large flat file handling 30
floating records iterator variable of convertToValues service 94
behavior of 43 length validator, field 64
examples of, inbound parsing 44 listFFDictionaryEntries service 103
examples of, outbound parsing 47
specifying in flat file schema 41 M
flow services, requirements to receive flat files 79 mapping flat file data 27
format services 66 modifying
creating 66 flat file dictionaries entries programmatically 134
disabling 66 flat file dictionaries programmatically 136
error messages 66 flat file schemas programmatically 138
managing dependencies on 67
processing 25 O
FormatService service 88
outbound flat file processing 31, 32
diagram 31
G how to 31
getFFDictionaryEntryAsXML service 102 parsing 31
getFFSchemaAsXML service 102
getSupportedEncodings service 98

Index
P pub.flatFile.generate:FFDictionary IS document type 107

parsers, record 14 pub.flatFile.generate:FFSchema IS document type 109
parsing pub.flatFile.generate:findDependants service 101
inbound flat files 26 pub.flatFile.generate:findReferences service 101
outbound flat files 31 pub.flatFile.generate:getFFDictionaryEntryAsXML service 102
position field extractors 11 pub.flatFile.generate:getFFSchemaAsXML service 102
posting a flat file via HTTP 80 pub.flatFile.generate:listFFDictionaryEntries service 103
processing flat files 24 pub.flatFile.generate:saveXMLAsFFDictionary service 103
diagram 24 pub.flatFile.generate:saveXMLASFFSchema service 104
format services 25 pub.flatFile.generate:updateFFDictionaryEntryFromXML service
106
inbound 25
pub.flatFile:convertToString service 90
default records 18
pub.flatFile:convertToValues service 93
diagram 26
pub.flatFile:FormatService service 88
how to 26
pub.flatFile:getSupportedEncodings service 98
large files 30
mapping data 27
parsing 26 R
undefined data 16 record
validation error codes 120 default 18
validation error contents 116 defined 10
outbound 31, 32 identifying 14
diagram 31 reusing definitions 18
how to 31 record identifiers 15
parsing 31 records with no identifiers 18
processing IS documents (IData) 27 record parsers 14, 36
program code conventions in this document 7 delimiter
programmatically defined 36
create empty flat file dictionaries 133 properties 39
create flat file dictionaries 132 EDI 37
create flat file dictionary entries 131 fixed length
create flat file schemas 133 defined 36
delete flat file dictionaries 140 properties 40
delete flat file dictionary entries 140 general properties 37
delete flat file schemas 140 specifying delimiters 37
modify flat file dictionaries 136 variable length, defined 36
modify flat file dictionary entries 134 records, floating
modify flat file schemas 138 behavior of 43
properties of flat file schemas, specifying 41 examples of, inbound parsing 44
pub.flatFile.generate:createDocumentType service 98 examples of, outbound parsing 47
pub.flatFile.generate:createFFDictionary service 99 specifying in flat file schema 41
pub.flatFile.generate:deleteFFDictionary service 99 reusing element definitions 18
pub.flatFile.generate:deleteFFDictionaryEntry service 100
pub.flatFile.generate:deleteFFSchema service 100

Index
S U
sample.flatFile.generateFFSchema:delimited service 111 undefined data 16
sample.flatFile.generateFFSchema:fixedLength service 113 updateFFDictionaryEntryFromXML service 106
saveXMLAsFFDictionary service 103 updating
saveXMLASFFSchema service 104 flat file dictionaries entries programmatically 134
schemas, flat file flat file dictionaries programmatically 136
creating programmatically 133 flat file schemas programmatically 138
deleting programmatically 140 useAlternateCodeSets variable 74
modifying programmatically 138 useReadableDelimiterReporting variable 75
retrieving data for from XML file 140
sending and receiving flat files V
content handler 78 validation
content type 78 defined 12
overview 78 error codes 120
services for receiving 79 error contents 116
via a flow service 79 validators 62
via a string variable 79 conditional 62
via e-mail 84 field 64
via file polling 84 byte count 65
via FTP 81 code list 64
via Trading Networks 78 length 64
setting access permissions 73 variable length record parser, defined 36
setting up flat file schemas 36 variable length records 11
sorting order of fields in composites and records 74
W
submitting a flat file via a string variable 79
submitting a flat file via e-mail 84 WmFlatFile package contents 87
flat file services 88
submitting a flat file via file polling 84
submitting a flat file via FTP 81
submitting a flat file via HTTPsending and receiving flat files X
via HTTP 80 XML files, retrieving data from 140
T
testing flat file schemas 68
Trading Networks, sending flat files to 78
troubleshooting information 8
typographical conventions in this document 7

Index

Developer Flat File Schema Developer's Guide: Title Page

Uploaded by

Copyright:

Available Formats

Developer Flat File Schema Developer's Guide: Title Page

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Developer Flat File Schema Developer's Guide: Title Page

Uploaded by

Copyright:

Available Formats

Title Page

Developer Version 7.1 Flat File Schema Developer’s Guide

Document ID: DEV-FFS-DG-71-20070831

About This Guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Chapter 2. Processing Flat Files Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Flat File Schema Developer’s Guide Version 7.1 3

Chapter 3. Creating and Editing Flat File Schemas and Dictionaries . . . . . . . . . . . . . . . . 35

Chapter 4. Sending and Receiving Flat Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Flat File Schema Developer’s Guide Version 7.1 4

Submitting a Flat File to Integration Server via File Polling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Chapter 5. WmFlatFile Built–in Services Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Appendix A. Validation Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Flat File Schema Developer’s Guide Version 7.1 5

Appendix B. Flat File Byte Count Parser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Appendix C. Programmatically Creating Flat File Schemas and Dictionaries . . . . . . . . . 129

Flat File Schema Developer’s Guide Version 7.1 6

About This Guide

Flat File Schema Developer’s Guide Version 7.1 7

Flat File Schema Developer’s Guide Version 7.1 8

What Is a Flat File? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

What Is a Flat File Schema? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

What Type of Flat File Constraints Can You Define? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

How Is a Flat File Schema Used to Parse Records? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

What Is a Flat File Dictionary? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

When Should I Create a Flat File Dictionary? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

What Is the Flat File Schema Editor? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Flat File Schema Developer’s Guide Version 7.1 9

What Is a Flat File?

Field record delimiter = !

Rec1*C1Field1+C1Field2*FieldA*C2Field1+C2Field2! field delimiter = *

Flat File Schema Developer’s Guide Version 7.1 10

What Types of Flat Files Does webMethods Support?

Flat File Schema Developer’s Guide Version 7.1 11

What Is a Flat File Schema?

What Type of Flat File Constraints Can You Define?

Flat File Schema Developer’s Guide Version 7.1 12

Flat File Schema Developer’s Guide Version 7.1 13

How Is a Flat File Schema Used to Parse Records?

Flat File Schema Developer’s Guide Version 7.1 14

Fixed Length Record Parser. This parser splits a file into records of the same

Variable Length Record Parser. This parser expects each record to be preceded by two

Flat File Schema Developer’s Guide Version 7.1 15

Flat File Schema Developer’s Guide Version 7.1 16

The Allow Undefined Data check box for each record in the flat file schema on the Flat File

Flat File Schema Developer’s Guide Version 7.1 17

What Is a Flat File Dictionary?

Flat File Schema Developer’s Guide Version 7.1 18

When Should I Create a Flat File Dictionary?

What Is the Flat File Schema Editor?

Flat File Schema Developer’s Guide Version 7.1 19

Identifying Flat File Schemas and Dictionaries

Navigation Flat File

Flat File Dictionary. Identifies a flat file dictionary in the Navigation Panel.

Note: A green check mark next to a flat file schema or dictionary ( ) indicates that the

Flat File Schema Developer’s Guide Version 7.1 20

Flat File Structure Tab

Flat File Schema Developer’s Guide Version 7.1 21

Flat File Schema Developer’s Guide Version 7.1 22

Processing Inbound Flat Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Rec1C1Field1+C1Field2FieldAC2Field1+C2Field2! field delimiter =