RTI Data-Distribution Service (DDS) Master Class 2011

DDS: A Next-Generation Approach to Building Distributed Real-Time Systems 2011 Masterclass Gerardo Pardo-Castellote, Ph.D. Co-chair OMG DDS SIG CTO, Real-Time Innovations [email_address] http://www.rti.com

Outline Day 1 Part I Concepts Introduction to DDS and RTI Overview of Technology Run-time services (Persistence, Recording, Routing, …) Day 2 Part II Basic Hand’s on Examples Development tools Future directions and Standards: Part II Application development cycle Advanced Exercise

Beyond real-time: Systems that interact directly with the Real World Must adapt to changing environment Cannot stop processing the information Live within world-imposed timing Beyond traditional interpretation of real-time © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL

Challenge: More Data, More Speed, More Sources TRENDS : Growing Information Volume Lowering Decision Latency Increasing System Availability Accelerating technology insertion and deployment Next-generation systems needs : Performance Scalability Robustness & Availability Platform Integration & Evolution Safety-Critical Certification Security © 2009 Real-Time Innovations, Inc.

“ Real World” Systems are integrated using a Data-Centric Model Grounded on the “physics” of the problem domain Tied to the nature of the sensors and real objects in the system (vehicles, device types, …) Provides governance across disparate teams & organizations Central authority can define data model necessary for interoperability Increased decoupling from use-cases and components Avoids over constraining applications Open, Evolvable, Platform-Independent The use-cases, algorithms might change between missions or versions of the system Copyright © 2010 RTI - All rights Reserved. COMPANY CONFIDENTIAL. Realizing this data-model requires a middleware infrastructure App App App

Everyday Example: Calendaring Alternative Process #1: Email: “Meeting Monday at 10:00.” Email: “Meeting moved to Tuesday.” Email: “Here’s dial-in info for meeting…” Rick: “Where do I have to be? When?” Rick: ( sifting through email… )

Example: Calendaring Alternative Process #2: Calendar: ( add meeting Monday at 10:00 ) Calendar: ( move meeting to Tuesday ) Calendar: ( add dial-in info ) Rick: “Where do I have to be? When?” Rick: ( check calendar )

Example: FaceBook Alternative Process #1: Use the FaceBook web site. Alternative Process #2 : The web site doesn’t exist. Deduce your friends’ information based on the notifications FaceBook sends you.

What’s the Difference? State . Things have attributes and characteristics The meeting will run 1:00–2:00 in the conference room . My friend’s phone number is 555-1234 and he’s currently grooming his cat . The car is blue and is traveling north from Sunnyvale at 65 mph . … whether they exist in the real world, in the computer, or both … whether or not we observe or acknowledge them “ State” (“data”) is a snapshot of those attributes and characteristics. Best Practice: operate on state directly, not dialogs about state.

Data-Centricity = Describing the world as it is Implication : State of the world can be maintained by infrastructure , not each app the part of you care about at a certain point in time

Not Data-Centricity = Saying anything else… “ Hey you: go do this.” “ The thing changed in this way.” Implication : State must be inferred, reconstructed, managed by each app ( Sometimes called “ message -centricity ” : focus on what’s said vs. what is )

Why is it better to just describe the world? Reconstructing the state of the world is hard Must infer based on all previous messages Maintaining all these messages is expensive Each app makes these inferences => duplicate effort People make mistakes Many copies of state => may be different => bugs vs. Uniform operations on state => fewer bugs

Why is it better to just describe the world? Computers are stupid Integrate based on what to do => must know everything you might do => future extension hard Integrate based on what exists => do whatever you like; no one else cares => future extension easy Integration effort doesn’t scale Integrate at level of function => everything integrates with everything else => effort scales with square of number of apps Integrate at level of state => everything integrates with common model => effort scales with number of apps

So it’s “better.” Who cares? Faster to implement => Save time and money Easier to integrate and update => Protect your investment More reliable systems => Protect your business

Before We Forget: the Definition A data-centric architecture : … is based on a data model that is: Appropriately documented— i.e. understandable by humans Formally defined— i.e. understandable by machines Discoverable— i.e. can be found during execution The data model is independent of any domain-specific functionality / application . i.e. made of nouns , not verbs The (instantiated) data model is the only authoritative source of state in the system. For example, data structures in IDL file. Calendar Event = Start Time Duration Location Organizer

DDS Lets You Observe a Changing World Other data-centric technologies: Databases: SQL Web: HTTP (mostly) … assume the world changes slowly … use network resources inefficiently … are highly centralized App App App App App App Unreliable Failure here kills many apps State Server Slow A few updates/sec Not scalable 100 apps => 100x load

DDS Lets You Observe a Changing World DDS: … allows you to observe frequent changes … uses network resources efficiently … is decentralized State: Global Data Space App App App App App App Fast 100,000’s updates/sec Scalable Load indep. # apps Reliable No single pt. failure Managed with QoS

DDS Lets You Observe a Changing World JBC-P replaced home-brew messaging w/ DDS: Tracks 20x more objects with fewer failures … with 97% less code ( 1.5M lines  50K ) … with 99% less CPU resources ( 88 cores  0.8 ) State: Global Data Space App App App App App App Fast 100,000’s updates/sec Scalable Load indep. # apps Reliable No single pt. failure Managed with QoS

DDS Lets You Observe a Changing World Domain : world you’re talking about Topic : group of similar things Similar structure (“type”) what Similar way they change when over time (“QoS”) how Instance : individual thing DataWriter : source of observations about part of the world (topic) DataReader : observer of part of the world (topic) Domain (e.g. Yellowstone Park) Topic (e.g. bears in the park) Instance (e.g. Yogi the bear)

DDS: Standard Data-Centric middleware for Application Integration © 2009 Real-Time Innovations, Inc. Data Distribution Service Streaming Data Sensors Events Real-Time Applications Enterprise Applications Actuators

Open Architecture Vendor independent API for portability Wire protocol for interoperability Multiple implementations 7 of API 4 support RTPS (+1 non-DDS) Heterogeneous C, C++, Java, .NET (C#, C++/CLI) Linux, Windows, VxWorks, other embedded & real time Loosely coupled © 2009 Real-Time Innovations, Inc. Real-Time Publish-Subscribe Wire Protocol (RTPS) DDS Middleware Library DDS API Cross-vendor portability Cross-vendor interoperability

Family of Specifications © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL Network / TCP / UDP / IP Web-Enabled DDS 2011 DDS Implementation App DDS Implementation App DDS Implementation DDS Spec 2004 DDS Interoperablity 2006 UML Profile for DDS 2008 DDS for Lw CCM 2009 DDS X-Types 2010 DDS Security 2011 2010 DDS-STD-C++ DDS-JAVA5 App

DDS adopted by key programs US Navy Open Architecture Mandates DDS for Pub-Sub SPAWAR NESI Mandates DDS for Pub-Sub SOA DISR Mandates DDS for Pub-Sub API Mandates DDS-RTPS for Pub-Sub Interop European Air Traffic Control DDS used to interoperate ATC centers UK Generic Vehicle Architecture Mandates DDS for vehicle comm. Mandates DDS-RTPS for interop.

RTI DDS Application Examples Aegis Weapon System Lockheed Martin Radar, weapons, displays, C2 B-1B Bomber Boeing C2, communications, weapons Common Link Integration Processing (CLIP) Northrop Grumman Standards-compliant interface to legacy and new tactical data links Air Force, Navy, B-1B and B-52 ScanEagle UAV Boeing Sensors, ground station Advanced Cockpit Ground Control Station Predator and SkyWarrior UAS General Atomics Telemetry data, multiple workstations RoboScout Base10 Internal data bus and link to communications center © 2009 Real-Time Innovations, Inc.

RTI DDS Application Examples Multi-ship simulator FORCE Technology Controls, simulation display Mobile asset tracking Wi-Tronix GPS, operational status over wireless links Highway traffic monitoring City of Tokyo Roadway sensors, roadside kiosks, control center Driver safety Volkswagen vision systems, analysis, driver information systems Medical imaging NMR and MRI Sensors, RF generators, user interface, control computers Automated trading Automated Trading Desk (ATD, now Citigroup) Market data feed handlers, pricing engines, algorithmic trading applications © 2009 Real-Time Innovations, Inc.

RTI DDS Application Examples Full-immer sion simulation National Highway Transportation Safety Authority Migrated from CORBA, DCOM for performance Air-Traffic Management INDRA. Deployed in UK, Germany, Spain Standards, Performance, Scalability Indu strial Control Schneider Electric VxWorks-based PLCs communicate via RTI-DDS Signal Processing PLATH GMBH RTI supports modular programming across product line Large Telescopes European Southern Observatory Performance & Scalability 1000 mirrors, 1sec loop Radar Systems AWACS upgrade Evolvability, Mainteinability, and supportability © 2009 Real-Time Innovations, Inc.

Architecture for the next-generation systems Existing technologies are reaching robustness/performance/scalability limits RTI DDS brings a fundamental new architecture and approach Fully decentralized, peer-to-peer, “no bottlenecks” architecture Superior Wire Protocol Powerful data-centric model Built-in Robustness and High-Availability Standards-based, multi-platform Single-lane traffic No prioritization Brokers as choke-points RTI Approach

Benefits of the DDS approach Simple & Powerful Data-Centric Pub-Sub Model Reduces Risk and Development/Integration Time Enhances effective performance by delivering the right data to the right place with the right QoS Standards-based: API and Protocol Unsurpassed Performance and Scalability Priority-aware no choke-points architecture Builds higher quality systems and lowers TCO Built-in high-value capabilities Handles Availability & other “ hard problems” Easy to maintain and Evolve Leverage multicore © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL Messaging & Caching Event Processing Database Bridge Persistence & Durability Recording Redundancy & Failover SQL

DDS Global Data Space And Programming Model

Data-Centric Pub-Sub Model © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL Persistence Service Recording Service Essentially a virtual, decentralized global data space © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL Source (key) Latitude Longitude Altitude UAV1 37.4 -122.0 500.0 UAV2 40.7 -74.0 250.0 UAV3 50.2 -0.7 2000.0

Data-Centric Model “ Global Data Space ” generalizes Subject-Based Addressing Data objects addressed by DomainId, Topic and Key Domains provide a level of isolation Topic groups homogeneous subjects (same data-type & meaning) Key is a generalization of subject Key can be any set of fields, not limited to a “x.y.z …” formatted string Data Writer Data Writer Data Writer Data Writer Data Reader Data Reader Data Reader Data Reader Data Writer example Data Object

Data-Centric Model “ Global Data Space ” generalizes Subject-Based Addressing Data objects addressed by DomainId, Topic and Key Domains provide a level of isolation Topic groups homogeneous subjects (same data-type & meaning) Key is a generalization of subject Key can be any set of fields, not limited to a “x.y.z …” formatted string Data Writer Data Writer Data Writer Data Writer Data Reader Data Reader Data Reader Data Reader Data Writer example Topic

Data-Centric Model “ Global Data Space ” generalizes Subject-Based Addressing Data objects addressed by DomainId, Topic and Key Domains provide a level of isolation Topic groups homogeneous subjects (same data-type & meaning) Key is a generalization of subject Key can be any set of fields, not limited to a “x.y.z …” formatted string Data Writer Data Writer Data Writer Data Writer Data Reader Data Reader Data Reader Data Reader Data Writer example Key (subject)

DDS communications model Participants scope the global data space (domain) Topics define the data-objects (collections of subjects) Writers publish data on Topics Readers subscribe to data on Topics QoS Policies are used configure the system Listeners are used to notify the application of events Data Reader “ Alarm” Domain Participant Data Writer “ Alarm” Domain Participant New subscriber! example Listener Offered QoS Listener Got new data Requested QoS

Real-Time Quality of Service (QoS) © 2009 Real-Time Innovations, Inc. Volatility User QoS Delivery Presentation Redundancy Infrastructure Transport QoS Policy DURABILITY HISTORY READER DATA LIFECYCLE WRITER DATA LIFECYCLE LIFESPAN ENTITY FACTORY RESOURCE LIMITS RELIABILITY TIME BASED FILTER DEADLINE CONTENT FILTERS QoS Policy USER DATA TOPIC DATA GROUP DATA PARTITION PRESENTATION DESTINATION ORDER OWNERSHIP OWNERSHIP STRENGTH LIVELINESS LATENCY BUDGET TRANSPORT PRIORITY

Integrating components to generic middleware technology Copyright © 2010 RTI - All rights Reserved. . Data Model Custom Mapping Custom Integration Akin to implementing an OO design on a Procedural Language: Requires mapping inheritance, encapsulation, exceptions, … Comp Comp Comp Comp Middleware Artifacts

Integrating components to data-centric middleware technology Copyright © 2010 RTI - All rights Reserved. . Data Model Standard Mapping(*) Standard API No custom mappings / code necessary Direct support for data-centric actions: create, dispose, read/take Comp Comp Comp Comp DDS Global Data Space

Example: Message-Centric Legacy Define message-sets / handshakes Copyright © 2010 RTI - All rights Reserved. . Publish Subscribe Component or Use-case based Schema, Limited QoS) x=float(45.6) y=float(78.9) id=“AA123” 0x0000000641410102030042366666429DC “ My app knows this means dispose.” Nothing to base filters, xforms on Error checking dev  integration Self-describing data is verbose

Example: Modern Data-Centric Design Start with Data Model / Schemas / Meaning Copyright © 2010 RTI - All rights Reserved. . Publish Subscribe Data Schema x : float y : float id : string ( key ) New 45.6 78.9 “ AA123” Update 56.7 89.0 “ AA123” New 65.4 32.1 “ DL987” Dispose “ AA123” X Map this into XML; rows + cols Express content-based filters Propagate data efficiently

Example: Modern Data-Centric Design Attach QoS to Data Model Once infrastructure understands data items, can attach QoS contracts to them “ Keep only the latest value” or “I need updates at this rate” make no sense unless per-item Flight AA123 updates shouldn’t overwrite DL987, even if AA123 is updated more frequently Update rate for one track shouldn’t change just because another track appeared Copyright © 2010 RTI - All rights Reserved. . Publish Subscribe Data Schema x : float y : float id : string ( key ) Quality of Service Deadline Time-Based Filter History

Domain and Domain Participants Domain 1 Domain 2 Domain 3 Added Func. Multiple Domain System Using Multiple domains for Scalability, Modularity & Isolation demo_domain_0 demo_domain_1 Node 1 - App 1 Pub/Sub Node 2 - App 1 Subscribe Node 4 - App 1 Pub/Sub Node 4 - App 2 Publish Node 3 - App 1 Pub/Sub Node 5 - App 1 Subscribe Node 5 - App 2 Pub/Sub Node 6 - App 1 Pub/Sub

DDS Theoretical Foundation: Observer Pattern Directed State Pattern

Observer Pattern: Use Cases How do I model state that changes over time? How do I decouple data consumption from data production in time? What if consumers need to come and go while the system is operational? What if new functionality needs to be added to the system while it’s operational? Examples Vehicles move from place to place Bid/offer prices of an asset change Sensors take new measurements periodically

Observer Pattern: Introduction Roles Subject : Changes its state over time Has identity, abstract “address” Observer : Sees those changes when it cares to Known to infrastructure, anonymous to subjects Notified when subject changes; may examine new state or not Operations (CRUD) Create : create new subject exists with certain state Read : access state(s) of subject(s) Update : modify existing subject with new state Delete : delete existing subject

Observer Pattern: Introduction Observer ( Data Bus : Notify) Subject Update Delete Create Read

Observer Pattern: How To Premier pattern in DDS , central to data-centric design! Roles Subject address = domain  topic  instance Observer = DataReader Operations Create = write new instance Read = read or take samples Update = write existing instance Delete = dispose instance

Observer Pattern: How To Domain : world you’re talking about Best practice : use to isolate subsystems Best practice : connect subsystems with Routing Service Topic : group of similar objects Similar structure (type) Best practice : model complete state, not diffs Best practice : use DDS type system, not opaque bytes or custom encapsulations Similar way they change over time (QoS) Best practice : differentiate … how subject changes (writer QoS) … from what observer sees (reader QoS) Instance : individual object Best practice : declare key fields always Best practice : key fields establish identity e.g. Social Security Number, Track ID, etc. Best practice : key fields allow resource management per object over time e.g. “keep last 5” Topic (e.g. bears in the park) Instance (e.g. Yogi the bear) Domain (e.g. Yellowstone Park)

Observer Pattern: How To Subject Change Contract : History keep last 10 Deadline 1 sec Observer Requirements : History keep last 5 Deadline 5 sec Minimum separation 2 sec Example Type : typedef long BearID; struct Bear { @Key BearID which_bear; long pos_x; long pos_y; float speed; bool is_growling; bool is_charging; };

Observer Pattern: How To Two variants : Publish when it changes Uses network efficiently when changes are slow / sporadic Examples: e.g. DDS discovery e.g. Vehicles / devices that start and stop Best practice : use durability, reliability, keep-last history depth >= 1 Publish periodically Good for continuous feedback control algorithms e.g. Vehicles / devices while they’re moving e.g. Stuff that tends to be connected to Matlab / Simulink, LabView Smoothes resource use when changes are regular Examples: Sensors, actuators in a control loop Best practice : declare finite deadline Best practice : consider time-based filter for slow readers and best efforts

Observer Pattern: Benefits Supports dynamic system composition, evolution Subject states don’t depend on previous states: observers can come, go whenever without being out of sync Uses of data unspecified: functions can be added, removed Maximizes parallelism, minimizes blocking Subject changes on its own time scale Observer reads on its time scale Data bus can overwrite, filter, down-sample as necessary Allows flexible deployment Subjects don’t care about identities, locations, number of observers Observers don’t care about physical locations of subjects; just need abstract “address” decoupled

Observer Pattern: Challenges Challenge: Large state may change in small ways Communicating new state may be unnecessarily expensive If Observer cares what changed, may be expensive to determine Response: Don’t sweat it Are the data rates high enough that it matters? How much network/memory/CPU/complexity will you pay to “fix” it? Response: Send diffs and reassemble Tools at your disposal: DDS-XTypes: optional fields (also in RTI [dynamic] sparse types) Portable today: sequence<MyType, 1> Multiple topics (with same key) Topic 1 : complete object state Topic 2 : cumulative changes to last state from Topic 1 Use durability and explicit correlation IDs to merge 1 and 2 Cost : increased logic, state in your applications

Observer Pattern: See Also High-Volume patterns Low-Latency patterns [GoF] “Observer” [EIP] “Publish-Subscribe Channel” [EIP] “Datatype Channel” [EIP] “Quality-of-Service Channel”

Directed State Pattern: Use Cases How can one party request that another party do something? How can a producer know that observers(s) have acted on its data? How can a producer observe the “result” of a remote action? Examples : Defense: Fire control Financial Services: Order execution

Directed State Pattern: Introduction Best practice : keep interactions data-centric Insight : Objectives are state and can be distributed Insight : Desires and intentions are state and can be distributed Anti-pattern : point-to-point commands, replies Why not : Couple applications based on function and role Why not : Constrain future functional changes to system Why not : Integration effort doesn’t scale as number of apps increases

Directed State Pattern: Introduction 3 States : Current Objective (“goal”)— Optional Requested State 2+ Roles (special case of Observer pattern): Effector aka Subject wrt Current State and Objective State aka Observer wrt Requested Objective State Requester aka Subject wrt Requested Objective aka Observer wrt Current State and Objective State ( Observer —with respect to any state) Current State Objective State change Requested State Effector executes this Requester changes this

Directed State Pattern: How To Reliability QoS policy : Current : Reliable or Best Effort Requested : Reliable History QoS policy : Current, Objective : Keep Last n Requested : Keep Last 1 Requester Effector Data Bus if long-running if short-running request processed? feedback Current State write Objective State write Requested State read, filter Current State read, filter Objective State read, filter same type same or different types same key Requested State write

Directed State Pattern: How To Variant: Simple state, Fast transition Current + Requested States; no Objective State States use same type Flow: Current = x Requested  y Current  y Requested unregister Variant: Simple state, Slow transition Current, Objective, Requested States States use same type Flow: Current, Objective = x Requested  y Objective  y Current  x + 1  x + 2  … y Requested unregister

Directed State Pattern: How To Variant: Complex state, slow transition Current, Objective, Requested States Requested States use different type than Current Requests have identity and lifecycle: e.g. REQUESTED, PROCESSING, DONE Objective indicates which request is in process Flow: Effector : Current, Objective = x Requester : Requested  id = 1 , value = y , state = REQ Effector : Objective  id = 1 , value = y , state = PROC Requester : Requested unregister id = 1 Requester : Requested  id = 2 , value = z , state = REQ Effector : Current  x + 1  x + 2  … y Effector : Objective  id = 1 , value = y , state = DONE Effector : Objective  id = 2 , value = z , state = PROC Requester : Requested unregister id = 2 Effector : Current  y + 1  y + 2  … z Effector : Objective  id = 2 , value = z , state = DONE

Directed State Pattern: How To Example: Fire Missile Fire Control Fire Effector Launcher = 5 State = Idle Launcher #5 Firing Authorized Target Identified (Objectives  ) (Current  ) Request ID = 5 Launcher = 5 Action = Fire State = Requested 1 Request ID = 5 Launcher = 5 Action = Fire State = Processing 2 Request ID = 6 Launcher = 6 Action = Fire State = Requested 4 Request ID = 5 Launcher = 5 Action = Fire State = Done 7 Launcher = 5 State = Prep’ing 3 Launcher = 5 State = Firing 5 Launcher = 5 State = Idle 6

Directed State Pattern: Benefits Allows components to express intent and communicate about actions Allows additional observing components to leverage request, objective data Avoids tight application-to-application coupling of RPC-like (anti-)patterns More flexible asynchronous behavior Stronger type-based, QoS-based governance Easier insertion, removal of application components Easier insertion, removal of infrastructure components

Directed State Pattern: Challenges Beware degradation into RPC-style request-reply Code smell : blocking behavior, e.g. wait_for_acknowledgments Code smell : point-to-point comm. back channels, e.g. payloads in app ACKs Code smell : stateless interactions, e.g. many occurrences of keep-all reliability

Directed State Pattern: Challenges Manage state ownership One party established state, another needs to change it—why not “pass” ownership on single topic? Why not : Embedding workflow of current  objective state transition into multiple components => maintenance pain Why not : “Same” objects may be written by different parties but with different ownership, e.g. I own #1, you own #2, she owns #3, … Why not : Moving ownership => hard to manage fault tolerance, e.g. persistence Best practice : always distinguish Current, (Requested) and Objective states Each should have a single owner Fail-over should occur among redundant copies, not different functional units

Directed State Pattern: See Also Patterns Observer pattern Anti-Patterns [GoF] “Command” [EIP] “Command Message” [EIP] “Correlation Identifier” [EIP] “Point-to-Point Channel” [EIP] “Request-Reply” [EIP] “Return Address” [EIP] “Scatter-Gather”

Part I Summary Real-World Systems are facing information volume, velocity, and mgmt. challenges Common solution is integration around a Data Model DDS is a family of OMG specifications that directly supports data-centric publish-subscribe communications Use of RTI DDS results in reduced programming, decreased cost, and lowered risk Copyright © 2010 RTI - All rights Reserved. . Cost and Interoperability are the key drivers Resources: http://dds.omg.org http://community.rti.com http://www.rti.com/dds

Day 2: Exercises Gerardo Pardo-Castellote, Ph.D. Co-chair OMG DDS SIG CTO, Real-Time Innovations [email_address] http://www.rti.com

Part II Solving integration use-cases with RTI Recording, Persistence, Database Integration, Web Access Hello World with RTI DDS Configuring QoS via XML Development and Run-time Tools: Ping, Spy, Analyzer, Wireshark, Excel Discovery and Builtin-Topics Protocol, Performance & Scalability. Future directions and Standards

RTI DDS builds Higher quality, Lower TCO Systems DDS Global Data Space Messaging & Caching Event Processing Database Bridge Persistence & Durability Recording Redundancy & Failover SQL © 2010 Real-Time Innovations, Inc. Presence Discovery Historical Cache Availability Redundancy Failover Mgmt Durable Data DDS QoS Security Guard Hooks Pre-built components address many challenging use-cases Extended QoS (*) Durable Writers (*) Parallel Persistence (*) Excel Visualization (*) Monitoring (*) Recording (*) Database Connectivity (*) Web Accessibility (*) Transformation (*) Event Processing (*) Secure TLS Transport (*) WAN Routing (*) Configurability via XML (*) Dynamic Data & Types (*) Pluggable Transports, Pluggable Discovery(*) (*) Extended capabilities in RTI DDS Comp Comp Comp

Real-Time Recording Service Applications: Future analysis and debugging Post-mortem Compliance checking Replay for testing and simulation purposes Record high-rate data arriving in real-time Non-intrusive – multicast reception © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL Demo: Start RecordingService Start ShapesDemo See output files Convert to: HTML XML View Data: HTML XML sqlite stop_all start

Data Persistence A standalone service that persists data outside of the context of a DataWriter Permanent Storage Permanent Storage Can be configured for: Redundancy Load balancing © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL Demo: PersistenceService ShapesDemo Application failure Application re-start Persistence Svc failure Application re-start start stop_all Data Writer Global Data Space Data Reader Persistence Service Persistence Service Data Reader Data Writer

Ownership and High Availability Owner determined per Topic and Key Only writer with highest strength can publish a Key Automatic failover when highest strength writer: Loses liveliness Misses a deadline Stops writing the subject Shared Ownership allows any writer to update any object Producer / Writer strength=10 Topic T1 K1 K2 Producer / Writer strength=5 Producer / Writer strength=1 K1 Primary K1 Backup K2 Primary K2 Backup © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL ShapesDemo

Relational Database Integration Relational Actions Topic T1 I1 I2 I3 I1 I2 I3 Table T1 Publish-Subscribe Action Write() Read() & Take() Dispose() Wait() & Listener UPDATE [ 2 , 3 ] & INSERT SELECT DELETE Event driven – The fastest way to observe database changes! © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 1. start mysql 2. start RTC ShapesDemo sql_gui stop_all sql_shell RTI Real-Time Connect

RTI Routing Service Selective, real-time data forwarding and transformation Can Change Topic Name and Topic Schema Allows for custom transformations via “plugin” Can filter/guard data QoS managed, can cache last-known value for data Dynamically configured Location independent deployment DDS Router GUARD XFORM Topic A Topic B ShapesDemo_0 start ShapesDemo_1 stop_all

Global Scalability: LAN to WAN… …without sacrificing Performance and Security DDS Routing Site A Site B Site C Site D WAN / Internet TCP/TLS/SSL Topics: Site Status Alarms Health Logs Sensor Data Proc Sensor Data Topics: Site Status Result Data Topics: Site Status Proc Sensor Data Result Data Alarms Topics: Site Status Sensor Data © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL DDS Routing DDS Routing DDS Routing

Web Accessibility Direct access to real-time data from Web-Based Applications Tactical Real-Time Data Web Enabled DDS Web Enabled DDS Web Enabled DDS GUARD Recorded Data Recorded Data Files Recorded Track Files 1. start replay & router 2. start view maps simulated tracks stop_all DDS Recording Service

COTS tools: Excel – Interacting with your data Display live RTI DDS Data in Excel Perform real-time computations and charts Publish RTI DDS data from Excel blank saved demo ShapesDemo

Hands-on Example (C++) Type Definition MyType.idl rtiddsgen MyType.h MyTypeSupport.c MyTypePublisher.cpp MyTypeSubscriber.cpp MyType.sln Publisher.exe Subscriber.exe Three minutes to a running app!! Define your data Create your project Build Run: publisher subscriber Aux: File Browser Console Delete Files rtiddsspy compiler

Exercise #1 - Hello World Define you data type: Create a directory “HelloWorld” Create a file called HelloWorld.idl and open it in VisualStudio Add the following contents: const long MSG_LEN=256; struct Hello { string<MSG_LEN> user; //@key string<MSG_LEN> msg; };

Run rtiddsgen (for C++) rtiddsgen HelloWorld.idl -language C++ -example i86Win32VS2005 -replace -ppDisable rtiddsgen HelloWorld.idl -language Java -example i86Win32jdk -replace -ppDisable Look at the directory you should see: HelloWorld-vs2005.sln And Several other files… Open the Solution File (type hello-vs2005.sln on the console) Look at HelloMsgPublisher.cxx Look at HelloMsgSubscriber.cxx Build the Solution

Run rtiddsgen (for Java) rtiddsgen hello.idl -language Java -example i86Win32jdk -replace -ppDisable Look at the directory you should see: makefile_hello_i86Win32jdk And Several other files… Look at HelloMsgPublisher.java Look at HelloMsgSubscriber.java You can use the makefile to build and the Java programs: gmake –f makefile_hello_i86Win32jdk

Execute the program C++: On one window run: objs86Win32VS2005elloMsgPublisher.exe On another window run: objs86Win32VS2005elloMsgSubscriber.exe Java On one window run: gmake –f makefile_hello_i86Win32jdk HelloMsgPublisher On another window run: gmake –f makefile_hello_i86Win32jdk HelloMsgSubscriber You should see the subscribers getting an empty string…

Modify the program to produce something C++: Open HelloMsgPublisher.cxx in VisualStudio Java: Open HelloMsgPublisher.java in your preferred tool Look for the comment: /* Modify the data to be sent here * / Add the line: strcpy_s(instance->user, MSG_LEN, “ Your name here "); strcpy_s(instance->msg, MSG_LEN, "Hello world "); Kill the Publisher, Rebuild the publisher and run it again

Playing with rtiddsspy Run rtiddsspy while the other applications are running Start and stop applications. What do you see in rtiddsspy

Example: Publication // Entities creation DomainParticipant participant = TheParticipantFactory->create_participant( domain_id, participant_qos, participant_listener); Publisher publisher = domain->create_publisher( publisher_qos, publisher_listener); Topic topic = domain->create_topic( “ MyTopic”, “Text”, topic_qos, topic_listener); DataWriter writer = publisher->create_datawriter( topic, writer_qos, writer_listener); TextDataWriter twriter = TextDataWriter::narrow(writer); TextStruct my_text; twriter->write(&my_track);

Example: Subscription // Entities creation Subscriber subscriber = domain->create_subscriber( subscriber_qos, subscriber_listener); Topic topic = domain->create_topic( “ Track”, “TrackStruct”, topic_qos, topic_listener); DataReader reader = subscriber->create_datareader( topic, reader_qos, reader_listener); // Use listener-based or wait-based access

How to Get Data? (Listener-Based) // Listener creation and attachment Listener listener = new MyListener(); reader->set_listener(listener); // Listener code MyListener::on_data_available( DataReader reader ) { TextSeq received_data; SampleInfoSeq sample_info; TextDataReader reader = TextDataReader::narrow(reader); treader->take( &received_data, &sample_info, …) // Use received_data printf(“Got: %s”, received_data[0]->contents); }

How to Get Data? (WaitSet-Based) // Creation of condition and attachement Condition foo_condition = treader->create_readcondition(…); waitset->add_condition(foo_condition); // Wait ConditionSeq active_conditions; waitset->wait(&active_conditions, timeout); // Wait returns when there is data (or timeout) FooSeq received_data; SampleInfoSeq sample_info; treader->take_w_condition (&received_data, &sample_info, foo_condition); // Use received_data printf(“Got: %s”, received_data[0]->contents);

Listeners, Conditions & WaitSets Middleware must notify user application of relevant events: Arrival of data But also: QoS violations Discovery of relevant entities These events may be detected asynchronously by the middleware … Same issue arises with POSIX signals DDS allows the application to choice: Either to get notified asynchronously using a Listener Or to wait synchronously using a WaitSet Both approaches are unified using STATUS changes

Status Changes DDS defines A set of enumerated STATUS The statuses relevant to each kind of DDS Entity DDS entities maintain a value for each STATUS struct LivelinessChangedStatus { long active_count; long inactive_count; long active_count_change; long inactive_count_change; } STATUS Entity INCONSISTENT_TOPIC Topic DATA_ON_READERS Subscriber LIVELINESS_CHANGED DataReader REQUESTED_DEADLINE_MISSED DataReader RUQESTED_INCOMPATIBLE_QOS DataReader DATA_AVAILABLE DataReader SAMPLE_LOST DataReader SUBSCRIPTION_MATCH DataReader LIVELINESS_LOST DataWriter OFFERED_INCOMPATIBLE_QOS DataWriter PUBLICATION_MATCH DataWriter

Listeners, Conditions and Statuses A DDS Entity is associated with: A listener of the proper kind (if attached) A StatusCondition (if activated) The Listener for an Entity has a separate operation for each of the relevant statuses STATUS Entity Listener operation INCONSISTENT_TOPIC Topic on_inconsistent_topic DATA_ON_READERS Subscriber on_data_on_readers LIVELINESS_CHANGED DataReader on_liveliness_changed REQUESTED_DEADLINE_MISSED DataReader on_requested_deadline_missed RUQESTED_INCOMPATIBLE_QOS DataReader on_requested_incompatible_qos DATA_AVAILABLE DataReader on_data_available SAMPLE_LOST DataReader on_sample_lost SUBSCRIPTION_MATCH DataReader on_subscription_match LIVELINESS_LOST DataWriter on_liveliness_lost OFFERED_INCOMPATIBLE_QOS DataWriter on_offered_incompatible_qos PUBLICATION_MATCH DataWriter on_publication_match

Listeners & Condition duality A StatusCondition can be selectively activated to respond to any subset of the statuses An application can wait changes in sets of StatusConditions using a WaitSet Each time the value of a STATUS changes DDS Calls the corresponding Listener operation Wakes up any threads waiting on a related status change Asynchronous notification via Listener operation Synchronous notification via activation/wakeup of conditions/waitsets DDS Entity Status Change

Exercise #2 – Shapes Publisher Create a new directory Shapes In the Directory create a file called ShapeType.idl Edit the file to have the following content: const long COLOR_LEN=64; struct ShapeType { string<COLOR_LEN>color; //@key long x; long y; long shapesize; }; Run: rtiddsgen ShapeType.idl -language C++ -example i86Win32VS2005 –replace -ppNotRun

rtiddsgen Details rtiddsgen [-d <outdir>] [-language <C|C++|Java|C++/CLI|C#>] [-namespace] [-package <packagePrefix>] [-example <arch>] [-replace] [-debug] [-corba [client header file]] [-optimization <level of optimization>] [-stringSize <Unbounded strings size>] [-sequenceSize <Unbounded sequences size>] [-notypecode] [-ppDisable] [-ppPath <path to the preprocessor>] [-ppOption <option>] [-D <name>[=<value>]] [-U <name>] [-I <directory>] [-noCopyable] [-use42eAlignment] [-help] [-version] [-convertToIdl | -convertToXml | -convertToXsd | -convertToWsdl] [[-inputIdl] <IDLInputFile.idl> | [-inputXml] <XMLInputFile.xml> | [-inputXsd] <XSDInputFile.xsd> | [-inputWsdl] <WSDLInputFile.wsdl>] DefinitionFile can be IDL, XSD and XML file -example generates example pub/sub apps and makefiles for compilation. -replace replaces everything that’s generated. Use if the data type definition has changed. Always use with caution if you’ve made modifications.

IDL vs. XML: IDL Example struct MemberStruct{ short sData; } typedef MemberStructType; //@top-level false

IDL vs. XML: XML Example <? xml version ="1.0“ encoding ="UTF-8"?> < types xmlns:xsi ="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation ="../rti_dds_topic_types.xsd"> < struct name ="MemberStruct" topLevel ="false"> < member name ="sData“ type ="short"/> </ struct > < typedef name ="MemberStructType" type ="nonBasic“ nonBasicTypeName ="MemberStruct“ topLevel ="false"/> </ types >

IDL vs. XSD: XSD Example <? xml version ="1.0" encoding ="UTF-8"?> < xsd:schema xmlns:xsd ="http://www.w3.org/2001/XMLSchema" xmlns:dds ="http://www.omg.org/dds" xmlns:tns ="http://www.omg.org/IDL-Mapped/" targetNamespace ="http://www.omg.org/IDL-Mapped/"> < xsd:import namespace ="http://www.omg.org/dds" schemaLocation ="rti_dds_topic_types_common.xsd"/> < xsd:complexType name ="MemberStruct"> < xsd:sequence > < xsd:element name ="sData" minOccurs ="1" maxOccurs ="1" type ="xsd:short"/> </ xsd:sequence > </ xsd:complexType >  < xsd:complexType name ="MemberStructType"> < xsd:complexContent > < xsd:restriction base ="tns:MemberStruct"> < xsd:sequence > < xsd:element name ="sData" type ="xsd:short" minOccurs ="1" maxOccurs ="1"/> </ xsd:sequence > </ xsd:restriction > </ xsd:complexContent > </ xsd:complexType >  </ xsd:schema >

Part II Solving integration use-cases with RTI Recording, Persistence, Database Integration, Web Access Hello World with RTI DDS Development and Run-time Tools: Ping, Spy, Analyzer, Wireshark, Excel Discovery and Builtin-Topics Monitoring Configuring QoS via XML Protocol, Performance & Scalability. Future directions and Standards

Tools provide insight into a distributed system RTI Analyzer Understand connections and data flow Tune QoS properties without changing code RTI Excel Addin View Data Inject Data Compute / Analyze RT Data RTI Protocol Analyzer Sniff the wire and analyze traffic RTI Monitoring Visualize and monitor system operation Detect system problems, bottlenecks, etc.

XML-Defined Qos Profiles: Overview QoS can be specified in XML file(s) and is automatically loaded when the application starts XML – specified QoS will be automatically used when DDS entities are created Globally for all new entities Selectively, depending on the Topic associated with the entity XML –specified QoS can be accessed via new API calls: create_<ENTITY>_with_profile() get_< ENTITY >_qos_from_profile() set_default_< ENTITY >_qos_with_profile()

Benefits of using QoS profiles Simplifies application programming. Editing XML is simpler than editing/compiling source code Same settigs can be shared between applications/languages Accelerates development Changing QoS does not require re-compilation & re-deployment Increases flexibility and maintainability Can change QoS applications to meet changing system needs without requiring re-deployment Improves code reuse Separates configuration/deployment from application logic Same code can be re-deployed on different hardware/networks/systems just changing the QoS file Enhances portability Application code is not dependent on vendor QoS extensions

More on using QoS profiles Could increase brittleness QoS files can change between runs. What was working no longer works the same way… Portability XML profile format now standardized as part of “DDS for light-weight CCM” specification Will be folded in the next revision of the standard File-based use of QoS profiles is always portable Direct calls to the C/C++/Java are not part of the standard yet Makes deployment dependent on accessing a file-system… XML can be compiled into the code, but that negates some of the benefits

What is a QoS Profile? Set of related Entity-QoS, usually one per entity kind, identified by a name. Provides all the information needed to fully configure the QoS for any DDS Entity ( DomainParticipant, Publisher, DataWriter, …) QoS policies and attributes can be set explicitly, in the profile or otherwise are inherited from it’s “base profile” Profiles that do not specify a “base profile” automatically inherit from the DDS-Specified Default QoS. May contain multiple definitions of a EntityQos (e.g. DataWriterQos) Can be disambiguated by Topic related to the Entity If ambiguous the last definition prevails Defined inside Qos libraries

What is inside an XML QoS file? A collection of QoS profiles organized inside QoS Libraries Example Field names are the same as in C++/Java source code. Eg.: datawriter_qos.reliability.max_blocking_time.sec

How are XML Qos files specified? NDDSHOME directory (first to be loaded) $NDDSHOME/resource/qos_profiles_XXX/xml/NDDS_QOS_PROFILES.xml Automatically loaded if present unless: DDS_ProfileQosPolicy::ignore_resource_profile == DDS_BOOLEAN_TRUE NDDS_QOS_PROFILES Environment variable NDDS_QOS_PROFILES= [<url1> | <url2>]; … <file_url> ::= file://usr/local/default_dds.xml <string_url> ::= str://<dds><qos_library>…</qos_library></dds> Automatically loaded if defined unless: DDS_ProfileQosPolicy::ignore_environment_profile == DDS_BOOLEAN_TRUE Application directory USER_QOS_PROFILES.xml Automatically loaded if present unless: DDS_ProfileQosPolicy::ignore_user_profile == DDS_BOOLEAN_TRUE DDS_ProfileQosPolicy::url_profile Automatically loaded if defined DDS_ProfileQosPolicy::string_profile Automatically loaded if defined Duplicate Qos Libraries and/or Qos Profiles within a library are not allowed

Side note: URL Format URLs are required for either specifying QoS profile location, or for specifying a profile string File URL : Example: file:// usr/local/default_dds.xml String URL Can be set in code, or as part of the environment variable Must contain whole XML document Example: str:// <dds><qos_library> . . . </qos_library><dds> URL groups Redundant files in URL groups provide fault-tolerance Only the first one found will be loaded Example: [file://usr/local/default_dds.xml | file://usr/local/alternative_default_dds.xml]

API access to XML QoS Policies If XML QoS policies are used, they are automatically loaded before creating the first participant XML QoS policies can be explicitly reloaded and unloaded via API calls: DDSDomainParticipantFactory::reload_profiles( ) DDSDomainParticipantFactory::unload_profiles( ) XML QoS can be explicitly applied to an entity by calling <ENTITY>::set_qos_with_profile(), create_<ENTITY>_with_profile() XML QoS can be retrieved by calling DDSDomainParticipantFactory::get<ENTITY>_qos_with_profile()

QoS Profile Inheritance A QoS profile can inherit the values of another QoS profile Base profile set using the base_name tag Base profile can be in another file and/or library A Profile that does not specify a base_name implicitly inherits from the Default Profile Example: < qos_profile name = " ReliableKeepLast " base_name = " BasicQos " > < datawriter_qos > < reliability > < kind > RELIABLE_RELIABILITY_QOS </ kind > < max_blocking_time > < sec > 1 </ sec > < nanosec > 0 </ nanosec > </ max_blocking_time > </ reliability > < history > < kind > KEEP_LAST_HISTORY_QOS </ kind > < depth > 10 </ depth > </ history > </ datawriter_qos > </ qos_profile >

Modifying the default QoS using XML files The “default” QoS used by DDS can be modified via an explicit call to: set_default_<ENTITY>_qos_with_profile() Alternatively a QoS profile can be marked such that it is used as the default QoS In combination with the “topic filter” (next slide) this can be used to surgically modify the QoS of a deployed application < qos_library name = " MyQosLib " > < qos_profile name = " ReliableKeepLast " is_default_qos = " true " > < datawriter_qos > … </ datawriter_qos > < datareader_qos > … </ datareader_qos > </ qos_profile > </ qos_library >

Fine-tuning which QoS applies to an Entity via Topic filters Allows a QoS profile to affect only selected DataReaders or DataWriters Affected DataReaders or DataWriters are specified via an expression in their Topic name Single QoS file and profile can be used to configure the whole system QoS of selected entities can be changed without application code changes QoS can be inherited from another library. E.g.: < datawriter_qos topic_filter = " Alarm* " > … </ datawriter_qos >

XML QoS Notes If both XML and source code are used to configure an entity, the last one set will be used If the entity already exists, set_qos_with_profile() can be called after reload() only of no immutable QoS have changed DTD validation is done automatically. Specify the DTD in the XML file when you want to overwrite the DTD used by the middleware XSD is available. Makes editing very easy when using a XML/XSD aware editor More details on Chapter 15 of the Reference Manual

Configuration of QoS via XML-defined profiles Try it out using the shapes demo!

Part II Solving integration use-cases with RTI Recording, Persistence, Database Integration, Web Access Hello World with RTI DDS Development and Run-time Tools: Ping, Spy, Analyzer, Wireshark, Excel Discovery and Builtin-Topics Configuring QoS via XML Protocol, Performance & Scalability. Future directions and Standards

20X Faster than JMS / Broker-based solutions Platform: Linux 2.6 on AMD Athlon, Dual core, 2.2 GHz RTI DDS is about 20X faster than JMS RTI DDS reliable multicast exhibits near perfect scalability ( 2KB messages ) © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL

Extremely low latency and jitter © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL Reliable, ordered delivery over Gigabit Ethernet between 2.4 GHz Core 2 Quad processors running 32-bit Red Hat Enterprise Linux 5.0

Orders of magnitude more scalable than broker-based solutions Going from 1 to 888 subscribers of the same data has only a 10% impact on throughput New topics can be added to a system without impacting the latency and throughput on other topics Throughput with 8 topics is 8x the throughput with 1 topic http://www.rti.com/products/dds/benchmarks-cpp-linux.html © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL

Realizing Performance & Scalability RTI DDS operates peer-to-peer, without brokers RTI DDS uses RTPS, an Advanced Multi-Session protocol supporting Reliable Multicast RTI DDS Approach RTPS JMS AMQP ESBs … Others: Broker-based middleware © 2010 Real-Time Innovations, Inc.

Realizing Performance & Scalability RTI DDS completely isolates each process and communicates peer-to-peer Other DDS implementations communicate vis shared memory with local daemon RTI DDS Approach RTPS SharedMem RTPS Others: Broker-based middleware © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL

Advanced Scalability & Performance Techniques Latency and Priority Aware message batching Content-Aware multi-channel reliable multicast Enhanced Reliable Protocol Selective ACKs (SACKs) for Confirmed Reliability NACK-only Reliable Protocol for Massive Scalability Smart caching integrated with the message protocol Content-Filtering at the source © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL

Message Batching write() sender receiver write() sender Send queue Receive queue Send queue Receive queue Without batching each message is separately sent. For small messages protocol headers might be bigger than payload With batching messages are held a little and combined into larger batches maximizing throughout and minimizing CPU receiver Transparent: Receiver still sees individual messages © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL

Reliability with Batching Reliability must work even when messages are batched ACK or NACK of individual samples would negate some of the benefits of batching… => Protocol must be batch aware so that it can ACK/NACK complete batches! B3 B2 B1 B3 B2 B1 ACK(B3), NACK(B2) Repair B2 B3 B2 B1 write() sender receiver © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL

Classic (TCP Style) Reliable Protocol No packet loss situation Company Confidential 01 02 03 04 01 02 03 04, HB 01 02 03 04 ACK 1-4 05 06 07 08 05 06 07 08, HB 05 06 07 08 ACK 1-8 © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL ShapesDemo

Classic (TCP Style) Reliable Protocol with some packet loss 01 02 03 04 01 02 03 04, HB 01 02 X ACK 1-2, NACK 3 05 06 07 08 05 06 07 08, HB 06 07 08 ACK 1-8 03 04 05 X X Packets 04 and 05 are received but the protocol drops them because a prior packet 03 is missing. This wastes valuable bandwidth © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL

RTI DDS Reliability (Reader Cache + SACK) improves performance when packet loss occurs 01 02 03 04 01 02 03 04, HB 01 02 X 04 ACK 1-2, SACK 3 05 06 07 08 05 06 07 08, HB 05 06 07 08 ACK 1-8 03 Packets 04 and 05 are received and cached waiting for the repair of 03. No bandwidth is wasted. © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL

RTI DDS NACK-only reliability eliminates ACK traffic if there no packet loss 01 02 03 04 01 02 03 04, HB 01 02 03 04 05 06 07 08 05 06 07 08, HB 05 06 07 08 No ACK traffic under normal operating conditions © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL

RTI DDS NACK-only reliability greatly reduces traffic even with packet loss 01 02 03 04 01 02 03 04, HB 01 02 X 04 NACK 3 05 06 07 08 05 06 07 08, HB 05 06 07 08 03 Negative Acknowledgments sent only when some message is lost This approach is far more scalable when there are many subscribers © 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL

OMG Standards update DDS API - 2003 DDS Wire Interoperability - 2006 UML Profile for DDS - 2007 DDS for lwCCM – 2008 Extensible Topics for DDS – 2010 ISO C++ API – 2010 Java 5 API - 2010 Upcoming standards: + Web Enabled DDS (Mar 2011) + Secure DDS (Dec 2011)

Web-Protocol Access to DDS Global Data Space Stateless access of data via standard Web Technologies & Protocols Not a bridge, broker, or message router Web App DDS RTPS Global Data Space App App App App Web Integration Service HTTP

DDS Security RFP A PIM and language PSMs for set of “plugins” that provide Authentication, Authorization, Data Tagging, Signing, and Encryption A set of pre-defined plugins that can be used to define and enforce security policies. Extensions to the DDS-RTPS Wire Protocol to support secure interoperability between DDS Systems.

Pub/Sub middleware (DDS) Application Secure OS Secure Transport DDS Application Secure DDS Global Data Credential Mgmt Authentication Access Control Data Tagging Encryption DDS Application DDS Application ? ? ? ?

App. Other DDS System Secure DDS middleware Authentication Plugin Access Control Plugin Data Encryption Plugin Data Tagging Plugin Crypto Module (e.g. TPM ) Secure Transport (e.g. TLS) CS application component certificates Kernel Policies Network Driver Network Encrypted Data TAGS Other DDS System Other DDS System App. App. Secure Kernel (e.g. SE Linux, MILS) ? Data cache Protocol Engine DDS Entities ? 1 2 3 4 5 6 7 8 9

Part III Application development cycle Advanced Exercise

Exercise: How could “chat rooms” be implemented? Different Topics for each Chat room? Map to Partitions? Add field to the message and use content-filtered Topics? Same as before and also make room part of the Key? Others? Discuss pros and cons of each approach

Exercise: How could we implement Ground control stations that monitor UAVs Different Topics for each UAV? Or use Keys? Different Domains for each Ground Station? Or Partitions? How to control multiple UAVs from the same ground station? How to switch the ground station that controls the UAV? How to do failover between ground stations? How to direct a message to one or all UAVs? How to detect loss of connection to an UAV? Discuss pros and cons of each approach

Edit the chat_publisher.cxx Go to the line with comment: /* Main loop */ Add the line: strcpy_s(instance->name, NAME_LEN, "Gerardo Pardo"); (Use your own name) Go to the line with comment: /* Modify the data to be sent here */ Add the lines: instance->age = count; strcpy_s(instance-> msg , NAME_LEN, “ Como va todo? "); (Use your age and personalized message) Rebuild and execute

Exercise #3 – Chat Room Create a new directory Chat In the Directory create a file called chat.idl Edit the file to have the following content: const long NAME_LEN=64; const long MSG_LEN=256; struct ChatMsg { string<NAME_LEN>name; //@key long age; string<MSG_LEN> chatRoom; string<MSG_LEN> msg; }; Run: rtiddsgen chat.idl -language C++ -example i86Win32VS2005 –replace -ppNotRun

Exercise #3 (continued) Use Qos Set RELIABILITY Set HISTORY to KEEP_LAST or KEEP_ALL Test different depths Use Partitions Create several Partitions: E.g. by ChatRoomName Publish in your ChatRoom Subscribe to one or more ChatRooms

Exercise #4 (continued) Use content filters Edit the chat_subscriber.cxx Add the lines: DDSContentFilteredTopic *cftopic; DDS_StringSeq filter_params; filter_params.maximum(0); cf T opic = participant- > create_contentfilteredtopic( "Selected Chats", topic, "age > 4", filter_params); Look of the call to create_datareader Replace “topic” with “cfTopic” in the paramater list.

Exercise #3 (continued) Use Exclusive Ownership Set up in pairs edit the chat_publisher.cxx and use the same “name” for both of you Re-run the publisher application you will see mixed messages. Edit the chat_publisher.cxx Before creating the data writer add the lines publisher->get_default_datawriter_qos(dwq); d wq.ownership.kind = DDS_EXCLUSIVE_OWNERSHIP_QOS; dwq.ownership_strength .value = 10; Replace DDS_DATAWRITER_QOS_DEFAULT with dwq In the create_datawriter() call Edit the chat_subscriber.cxx Before creating the data reader add the lines DDS_DataReaderQos drq; subscriber->get_default_datareader_qos(drq); drq.ownership = DDS_EXCLUSIVE_OWNERSHIP_QOS; Replace DDS_DATAWRITER_QOS_DEFAULT with drq in the create_datareader() call

Exercise Use durable data with a persistence service

Exercise Isolate domains and transform data using the routing service

Summary Reduces software lifecycle costs Loose coupling Replaces need for custom middleware in high performance, real time applications Reduces risk Standards-compliant API and wire protocol Multiple implementations Widely adopted Most widely proven and mature implementation Highest performance Industry-leading expertise and services capability Free trial, research and IR&D licenses Comprehensive platform support

RTI Data-Distribution Service (DDS) Master Class 2011

More Related Content

RTI Data-Distribution Service (DDS) Master Class 2011

Editor's Notes