Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Accelerating big data with ioMemory and Cisco UCS and NOSQL

•
•
Sumeet Bansal
Sumeet Bansal

This document discusses accelerating big data solutions using Oracle NoSQL Database, Cisco UCS, and Fusion-io storage. It provides an overview of big data characteristics and business benefits. It then summarizes Oracle NoSQL Database architecture and how its collaboration with Cisco UCS and Fusion-io storage provides optimized, high-performance solutions. The document outlines performance testing using YCSB that showed the solution can achieve over 1 million operations per second. It also discusses how Fusion-io's software-defined storage architecture provides extreme throughput and low latency.

1 of 23
Download to read offline
ACCELERATING BIG DATA: IOMEMORY, CISCO UCS AND NOSQL Ashok Joshi, Senior Director – Oracle NoSQL development, Oracle Sumeet Bansal, Principal Solutions Architect, Fusion-io
AGENDA ▸ Big Data overview ▸ Oracle NoSQL Database overview ▸ Real-time big data management – a business perspective ▸ NoSQL testing with YCSB ▸ The Fusion-io value October 1, 2012 2
VOLUME, VELOCITY, VARIETY, VALUE Big Data characteristics ▸ Terabytes, Petabytes ▸ Multiple sources for data ▸ Text, images, XML, JSON, sensor readings… ▸ Not “master” data, but important for business ▸ “Real-time” needs October 1, 2012 3
WHO USES BIG DATA? Web Services Big Energy Telco • Clickstream Analysis • Granular rate plans • Churn reduction • Abuse Prevention • Grid management • Network optimization Storage Financial Services Government • Cost efficient • Customer loyalty • Regulatory compliance • Analytics-ready • Risk • Environmental monitoring • Data Store • Trading • Cyber security • Scalable • Fraud • Distributed • Compliance • Credit scoring Large-scale, E-commerce • Recommendation engines • Cross-channel analytics • Golden path to purchase October 1, 2012 4
BIG DATA BUSINESS BENEFITS GLOBAL PERSONAL EUROPE PUBLIC US HEALTH CARE US RETAIL MANUFACTURING LOCATION DATA SECTOR ADMIN $300 B 60+% -50% $100 B €250 B Increase industry Increase net Decrease dev., Increase service Increase industry value per year margin assembly costs provider revenue value per year “In a big data world, a competitor that fails to sufficiently develop its capabilities will be left behind.” October 1, 2012 5
AGENDA ▸ Big Data overview ▸ Oracle NoSQL Database overview ▸ Real-time big data management – a business perspective ▸ NoSQL testing with YCSB ▸ The Fusion-io value October 1, 2012 6
NOSQL DATABASE ARCHITECTURE Highlights Application Application ▸ Available; scalable; fast NoSQL Database NoSQL Database NoSQL Database ▸ Simple administration, key- Driver Driver Driver value data model; transaction support ▸ Transparent load balancing; elastic ▸ Commercial grade software and support ▸ Integrated with related Oracle technologies Storage Nodes Storage Nodes October 1, 2012 7
AGENDA ▸ Big Data overview ▸ Oracle NoSQL Database overview ▸ Real-time big data management – a business perspective ▸ NoSQL testing with YCSB ▸ The Fusion-io value October 1, 2012 8
NOSQL DB AND CISCO UCS COLLABORATION WHY DOES IT MATTER ▸ Many components: network, processors, memory, software, storage – tested, tuned and optimized ▸ Business can focus on core competency and leveraging benefits of big data October 1, 2012 9
NOSQL DB AND FUSION-IO COLLABORATION WHY DOES IT MATTER ▸ Speed (latency) is critical • Amazon study: Every 100 millisecond increase in latency costs 1% in sales http://highscalability.com/latency-everywhere-and-it-costs-you-sales-how-crush-it ▸ ioMemory enables consistent, extremely low latency and extreme throughput October 1, 2012 10
ORACLE NOSQL DB, CISCO UCS, FUSION-IO ▸ Commercial grade solution and support ▸ Tested, tuned, optimized for real-time data management ▸ For equivalent performance, much lower CapEx and OpEx compared to commodity (DIY) solutions October 1, 2012 11
AGENDA ▸ Big Data overview ▸ Oracle NoSQL Database overview ▸ Real-time big data management – a business perspective ▸ NoSQL testing with YCSB ▸ The Fusion-io value October 1, 2012 12
SYSTEM UNDER TEST ▸ YCSB (Yahoo! Cloud Serving Benchmark) • 10 client machines generate load • Mixed workload (5% updates/95% reads) ▸ 15 UCS C240 M3 Rack Servers ▸ 30 Fusion’s ioDrive2 ▸ 2TB of data October 1, 2012 13
CONFIGURATION DETAILS Client machine (YCSB driver) UCS 240 M3 server 1 2 1 2 1 2 NoSQL Database on ioDrive2 Note: only two shards are shown in the illustration October 1, 2012 14
PERFORMANCE TEST RESULTS Number of shards 2 4 8 10 Mixed workload (95 read/5 302,152 558,569 1,028,868 1,244,550 write) throughput (ops/sec) Read latency (milliseconds) 0.76 0.79 0.85 0.88 Mixed workload update 3.08 3.82 4.29 4.47 latency (milliseconds) October 1, 2012 15
PUTTING PERFORMANCE IN CONTEXT Highlights ▸ For Example, Twitter: ~150K API calls/sec ▸ We can achieve that performance on a single UCS c240 server using two ioDrive2s ▸ Plenty of capacity to handle fluctuating demand without compromising performance http://blog.programmableweb.com/2011/05/25/who-belongs-to-the-api-billionaires-club/ October 1, 2012 16
AGENDA ▸ Big Data overview ▸ Oracle NoSQL Database overview ▸ Real-time big data management – a business perspective ▸ NoSQL testing with YCSB ▸ The Fusion-io value October 1, 2012 17
CUT-THROUGH ARCHITECTURE AND VSL FOR EXTREME THROUGHPUT AND LOW LATENCY ▸ Sophisticated architecture Host • maximum performance Virtualization ioMemory DRAM / Tables Memory / CPU and cores Operating System and ▸ Intelligent software Application Memory • advanced features Virtual Storage Layer T RANSFERS (VSL) DAT A Commands PCIe Applications/Databases ioDrive ioMemory File System Data-Path Controller Kernel Banks Virtual Storage Layer (VSL) Channels Wide ioMemory October 1, 2012 18
SOFTWARE DEVELOPMENT KIT ADVANTAGES FOR GREATER PERFORMANCE OPTIMIZATION APPLICAT ION Application source code Conventional access Direct access I/O Memory access Transactional Native Key-Value Auto-Commit Logging Block File Pair Memory™ Simple Network Simple Block File Block Proprietary Storage OS Native Flash Translation Layer Storage Media Storage Media Traditional Storage Software Defined Storage October 1, 2012 19
TYPES OF IOMEMORY For Cisco UCS B-Series Blade Servers For Cisco UCS C-Series Rack Servers 365 GB, 785 GB, 1.2 TB Mezzanine Card 365 GB, 785 GB 2.4 TB October 1, 2012 20
BIG DATA ANALYTICS - HADOOP October 1, 2012 21
COMPLETE BIG DATA SOLUTION ▸ Right Partners – Cisco and Fusion-io ▸ Multiple technologies for a comprehensive big data solution – NoSQL, map-reduce, relational ▸ Tested, integrated, optimized, commercially supported solution delivered by leaders ▸ Cost-effective, reliable, ready for the enterprise October 1, 2012 22
T H AN K Y O U

More Related Content

Accelerating big data with ioMemory and Cisco UCS and NOSQL

  • 1. ACCELERATING BIG DATA: IOMEMORY, CISCO UCS AND NOSQL Ashok Joshi, Senior Director – Oracle NoSQL development, Oracle Sumeet Bansal, Principal Solutions Architect, Fusion-io
  • 2. AGENDA ▸ Big Data overview ▸ Oracle NoSQL Database overview ▸ Real-time big data management – a business perspective ▸ NoSQL testing with YCSB ▸ The Fusion-io value October 1, 2012 2
  • 3. VOLUME, VELOCITY, VARIETY, VALUE Big Data characteristics ▸ Terabytes, Petabytes ▸ Multiple sources for data ▸ Text, images, XML, JSON, sensor readings… ▸ Not “master” data, but important for business ▸ “Real-time” needs October 1, 2012 3
  • 4. WHO USES BIG DATA? Web Services Big Energy Telco • Clickstream Analysis • Granular rate plans • Churn reduction • Abuse Prevention • Grid management • Network optimization Storage Financial Services Government • Cost efficient • Customer loyalty • Regulatory compliance • Analytics-ready • Risk • Environmental monitoring • Data Store • Trading • Cyber security • Scalable • Fraud • Distributed • Compliance • Credit scoring Large-scale, E-commerce • Recommendation engines • Cross-channel analytics • Golden path to purchase October 1, 2012 4
  • 5. BIG DATA BUSINESS BENEFITS GLOBAL PERSONAL EUROPE PUBLIC US HEALTH CARE US RETAIL MANUFACTURING LOCATION DATA SECTOR ADMIN $300 B 60+% -50% $100 B €250 B Increase industry Increase net Decrease dev., Increase service Increase industry value per year margin assembly costs provider revenue value per year “In a big data world, a competitor that fails to sufficiently develop its capabilities will be left behind.” October 1, 2012 5
  • 6. AGENDA ▸ Big Data overview ▸ Oracle NoSQL Database overview ▸ Real-time big data management – a business perspective ▸ NoSQL testing with YCSB ▸ The Fusion-io value October 1, 2012 6
  • 7. NOSQL DATABASE ARCHITECTURE Highlights Application Application ▸ Available; scalable; fast NoSQL Database NoSQL Database NoSQL Database ▸ Simple administration, key- Driver Driver Driver value data model; transaction support ▸ Transparent load balancing; elastic ▸ Commercial grade software and support ▸ Integrated with related Oracle technologies Storage Nodes Storage Nodes October 1, 2012 7
  • 8. AGENDA ▸ Big Data overview ▸ Oracle NoSQL Database overview ▸ Real-time big data management – a business perspective ▸ NoSQL testing with YCSB ▸ The Fusion-io value October 1, 2012 8
  • 9. NOSQL DB AND CISCO UCS COLLABORATION WHY DOES IT MATTER ▸ Many components: network, processors, memory, software, storage – tested, tuned and optimized ▸ Business can focus on core competency and leveraging benefits of big data October 1, 2012 9
  • 10. NOSQL DB AND FUSION-IO COLLABORATION WHY DOES IT MATTER ▸ Speed (latency) is critical • Amazon study: Every 100 millisecond increase in latency costs 1% in sales http://highscalability.com/latency-everywhere-and-it-costs-you-sales-how-crush-it ▸ ioMemory enables consistent, extremely low latency and extreme throughput October 1, 2012 10
  • 11. ORACLE NOSQL DB, CISCO UCS, FUSION-IO ▸ Commercial grade solution and support ▸ Tested, tuned, optimized for real-time data management ▸ For equivalent performance, much lower CapEx and OpEx compared to commodity (DIY) solutions October 1, 2012 11
  • 12. AGENDA ▸ Big Data overview ▸ Oracle NoSQL Database overview ▸ Real-time big data management – a business perspective ▸ NoSQL testing with YCSB ▸ The Fusion-io value October 1, 2012 12
  • 13. SYSTEM UNDER TEST ▸ YCSB (Yahoo! Cloud Serving Benchmark) • 10 client machines generate load • Mixed workload (5% updates/95% reads) ▸ 15 UCS C240 M3 Rack Servers ▸ 30 Fusion’s ioDrive2 ▸ 2TB of data October 1, 2012 13
  • 14. CONFIGURATION DETAILS Client machine (YCSB driver) UCS 240 M3 server 1 2 1 2 1 2 NoSQL Database on ioDrive2 Note: only two shards are shown in the illustration October 1, 2012 14
  • 15. PERFORMANCE TEST RESULTS Number of shards 2 4 8 10 Mixed workload (95 read/5 302,152 558,569 1,028,868 1,244,550 write) throughput (ops/sec) Read latency (milliseconds) 0.76 0.79 0.85 0.88 Mixed workload update 3.08 3.82 4.29 4.47 latency (milliseconds) October 1, 2012 15
  • 16. PUTTING PERFORMANCE IN CONTEXT Highlights ▸ For Example, Twitter: ~150K API calls/sec ▸ We can achieve that performance on a single UCS c240 server using two ioDrive2s ▸ Plenty of capacity to handle fluctuating demand without compromising performance http://blog.programmableweb.com/2011/05/25/who-belongs-to-the-api-billionaires-club/ October 1, 2012 16
  • 17. AGENDA ▸ Big Data overview ▸ Oracle NoSQL Database overview ▸ Real-time big data management – a business perspective ▸ NoSQL testing with YCSB ▸ The Fusion-io value October 1, 2012 17
  • 18. CUT-THROUGH ARCHITECTURE AND VSL FOR EXTREME THROUGHPUT AND LOW LATENCY ▸ Sophisticated architecture Host • maximum performance Virtualization ioMemory DRAM / Tables Memory / CPU and cores Operating System and ▸ Intelligent software Application Memory • advanced features Virtual Storage Layer T RANSFERS (VSL) DAT A Commands PCIe Applications/Databases ioDrive ioMemory File System Data-Path Controller Kernel Banks Virtual Storage Layer (VSL) Channels Wide ioMemory October 1, 2012 18
  • 19. SOFTWARE DEVELOPMENT KIT ADVANTAGES FOR GREATER PERFORMANCE OPTIMIZATION APPLICAT ION Application source code Conventional access Direct access I/O Memory access Transactional Native Key-Value Auto-Commit Logging Block File Pair Memory™ Simple Network Simple Block File Block Proprietary Storage OS Native Flash Translation Layer Storage Media Storage Media Traditional Storage Software Defined Storage October 1, 2012 19
  • 20. TYPES OF IOMEMORY For Cisco UCS B-Series Blade Servers For Cisco UCS C-Series Rack Servers 365 GB, 785 GB, 1.2 TB Mezzanine Card 365 GB, 785 GB 2.4 TB October 1, 2012 20
  • 21. BIG DATA ANALYTICS - HADOOP October 1, 2012 21
  • 22. COMPLETE BIG DATA SOLUTION ▸ Right Partners – Cisco and Fusion-io ▸ Multiple technologies for a comprehensive big data solution – NoSQL, map-reduce, relational ▸ Tested, integrated, optimized, commercially supported solution delivered by leaders ▸ Cost-effective, reliable, ready for the enterprise October 1, 2012 22
  • 23. T H AN K Y O U