Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo
MongoDB Basic Concepts

                                    Norberto Leite

                            Senior Solutions Architect, EMEA
                                 norberto@10gen.com
                                         @nleite




Thursday, 25 October 12
Agenda

        •Overview
        •Replication
        •Scalability
        •Consistency & Durability
        •Flexibility, Developer Experienc




Thursday, 25 October 12
Your data needs started here...




                                   http://bit.ly/OT71M4
Thursday, 25 October 12
...but soon you had to be here




                                   http://bit.ly/Oxcsis

Thursday, 25 October 12
Basic Concepts
                            Application    Document
                                           Oriented
         High                              { author : “steve”,
                                             date : new Date(),
                                             text : “About MongoDB...”,
     Performance                             tags : [“tech”, “database”]}




                                             Fully
                                           Consistent

                   Horizontally Scalable

Thursday, 25 October 12
Tradeoff: Scale vs Functionality

                                    • memcached
        scalability & performance




                                       •key/value



                                                                     • RDBMS



                                            depth of functionality
Thursday, 25 October 12
Replication




Thursday, 25 October 12
Why do we need replication

        •Failover
        •Backups
        •Secondary batch jobs
        •High availability




Thursday, 25 October 12
Replica Sets
        Data Availability across nodes
        • Data Protection
          • Multiple copies of the data
          • Spread across Data Centers, AZs
        • High Availability
          • Automated Failover
          • Automated Recovery




Thursday, 25 October 12
Replica Sets


                 App      Write
                                   Primary
                                              Asynchronous
                          Read                 Replication

                                  Secondary
                          Read


                                  Secondary
                          Read




Thursday, 25 October 12
Replica Sets


                 App      Write
                                   Primary
                          Read

                                  Secondary
                          Read


                                  Secondary
                          Read




Thursday, 25 October 12
Replica Sets


                 App
                                   Primary

                          Write
                                   Primary    Automatic Election of
                                                  new Primary
                          Read

                                  Secondary
                          Read




Thursday, 25 October 12
Replica Sets


                 App
                                  Recovering

                          Write                New primary serves
                                   Primary            data
                          Read

                                  Secondary
                          Read




Thursday, 25 October 12
Replica Sets


                 App
                                  Secondary
                          Read

                          Write
                                   Primary
                          Read

                                  Secondary
                          Read




Thursday, 25 October 12
Scalability




Thursday, 25 October 12
Horizontal Scalability




Thursday, 25 October 12
Sharding
        Data Distribution across nodes
        • Data location transparent to your code
        • Data distribution is automatic
        • Data re-distribution is automatic
        • Aggregate system resources horizontally
        • No code changes




Thursday, 25 October 12
Sharding - Range distribution

                          sh.shardCollection("test.tweets", {_id: 1} , false)


                          shard01                     shard02                   shard03




Thursday, 25 October 12
Sharding - Range distribution


                          shard01   shard02   shard03

                           a-i        j-r      s-z




Thursday, 25 October 12
Sharding - Splits


                          shard01   shard02   shard03

                           a-i      ja-jz      s-z
                                     k-r




Thursday, 25 October 12
Sharding - Splits


                          shard01   shard02   shard03

                           a-i       ja-ji     s-z
                                     ji-js
                                    js-jw
                                     jz-r

Thursday, 25 October 12
Sharding - Auto Balancing


                          shard01   shard02   shard03

                           a-i       ja-ji     s-z
                                     ji-js
                          js-jw     js-jw
                                     jz-r      jz-r

Thursday, 25 October 12
Sharding - Auto Balancing


                          shard01   shard02   shard03

                           a-i       ja-ji     n-z
                                     ji-js
                          js-jw
                                               jz-r

Thursday, 25 October 12
Sharding - Routed Query
                                              find({_id: "norberto"})




                          shard01   shard02                  shard03

                           a-i       ja-ji                     n-z
                                     ji-js
                          js-jw
                                                               jz-r

Thursday, 25 October 12
Sharding - Routed Query
                                              find({_id: "norberto"})




                          shard01   shard02                  shard03

                           a-i       ja-ji                     n-z
                                     ji-js
                          js-jw
                                                               jz-r

Thursday, 25 October 12
Sharding - Scatter Gather
                                         find({email: "norberto@10gen.com"})




                          shard01   shard02                shard03

                           a-i       ja-ji                  n-z
                                     ji-js
                          js-jw
                                                            jz-r

Thursday, 25 October 12
Sharding - Scatter Gather
                                         find({email: "norberto@10gen.com"})




                          shard01   shard02                shard03

                           a-i       ja-ji                  n-z
                                     ji-js
                          js-jw
                                                            jz-r

Thursday, 25 October 12
Sharding - Caching
                       96 GB Mem
                      3:1 Data/Mem


                          shard01

                           a-i
     300 GB Data




                           j-r
                           n-z

                           300 GB



Thursday, 25 October 12
Aggregate Horizontal Resources
                       96 GB Mem      96 GB Mem      96 GB Mem
                      1:1 Data/Mem   1:1 Data/Mem   1:1 Data/Mem


                          shard01    shard02        shard03

                           a-i          j-r           n-z
     300 GB Data




                           100 GB      100 GB        100 GB


Thursday, 25 October 12
Consistency & Durability




Thursday, 25 October 12
Two choices for consistency

        •Eventual consistency
                •Allow updates when a system has been partitioned
                •Resolve conflicts later
                •Example: CouchDB, Cassandra

        •Immediate consistency
                •Limit the application of updates to a single master
                node for a given slice of data
                          •Another node can take over after a failure is detected
                •Avoids the possibility of conflicts
                •Example: MongoDB



Thursday, 25 October 12
Durability

        •For how long is my data available?
        •When do I now that my data is safe?
        •Where?
        •Mongodb style
                •Fire and Forget
                •Get Last Error
                •Journal Sync
                •Replica Safe



Thursday, 25 October 12
Data Durability




Thursday, 25 October 12
Flexibility




Thursday, 25 October 12
Data Model

        • Why JSON?
                • Provides a simple, well understood
                encapsulation of data
                • Maps simply to the object in your OO language
                • Linking & Embedding to describe relationships




Thursday, 25 October 12
Json




        place1 = {
        
 name : "10gen HQ",
        
 address : "578 Broadway 7th Floor",
        
 city : "New York",
        
   zip : "10011",
           tags : [ "business", "tech" ]
        }
Thursday, 25 October 12
Schema Design
        Relational Database




Thursday, 25 October 12
Schema Design
        MongoDB                     embedding




                          linking
Thursday, 25 October 12
Schemas in MongoDB

     Design documents that simply map to
     your application
     post = {author: "Hergé",
          date: new Date(),
          text: "Destination Moon",
          tags: ["comic", "adventure"]}

     > db.posts.save(post)


Thursday, 25 October 12
Embedding
       > db.blogs.find( { author: "Hergé"} )

         { _id : ObjectId("4c4ba5c0672c685e5e8aabf3"),
           author : "Hergé",
           date : ISODate("2011-09-18T09:56:06.298Z"),
           text : "Destination Moon",
           tags : [ "comic", "adventure" ],
           comments : [
   !         {
   !       !    author : "Kyle",
   !       !    date : ISODate("2011-09-19T09:56:06.298Z"),
   !       !    text : "great book"
   !         }
           ]
         }




Thursday, 25 October 12
JSON & Scaleout

        • Embedding removes need for
                • Distributed Joins
                • Two Phase commit
        • Enables data to be distributed across many nodes
        without penalty




Thursday, 25 October 12
http://bit.ly/UmUnsU
Thursday, 25 October 12
http://bit.ly/cnP77L
Thursday, 25 October 12
http://bit.ly/ODoMhh
Thursday, 25 October 12
http://bit.ly/uW2nk
Thursday, 25 October 12
download at mongodb.org!

                                     norberto@10gen.com

                          Support, Training, Consulting, Events, Meetups
                                     http://www.10gen.com



        Facebook!                            Twitter!                     LinkedIn!
  http://bit.ly/mongofb!              http://twitter.com/mongodb!   http://linkd.in/joinmongo!




Thursday, 25 October 12

More Related Content

MongoDB Fundamentals

  • 1. MongoDB Basic Concepts Norberto Leite Senior Solutions Architect, EMEA norberto@10gen.com @nleite Thursday, 25 October 12
  • 2. Agenda •Overview •Replication •Scalability •Consistency & Durability •Flexibility, Developer Experienc Thursday, 25 October 12
  • 3. Your data needs started here... http://bit.ly/OT71M4 Thursday, 25 October 12
  • 4. ...but soon you had to be here http://bit.ly/Oxcsis Thursday, 25 October 12
  • 5. Basic Concepts Application Document Oriented High { author : “steve”, date : new Date(), text : “About MongoDB...”, Performance tags : [“tech”, “database”]} Fully Consistent Horizontally Scalable Thursday, 25 October 12
  • 6. Tradeoff: Scale vs Functionality • memcached scalability & performance •key/value • RDBMS depth of functionality Thursday, 25 October 12
  • 8. Why do we need replication •Failover •Backups •Secondary batch jobs •High availability Thursday, 25 October 12
  • 9. Replica Sets Data Availability across nodes • Data Protection • Multiple copies of the data • Spread across Data Centers, AZs • High Availability • Automated Failover • Automated Recovery Thursday, 25 October 12
  • 10. Replica Sets App Write Primary Asynchronous Read Replication Secondary Read Secondary Read Thursday, 25 October 12
  • 11. Replica Sets App Write Primary Read Secondary Read Secondary Read Thursday, 25 October 12
  • 12. Replica Sets App Primary Write Primary Automatic Election of new Primary Read Secondary Read Thursday, 25 October 12
  • 13. Replica Sets App Recovering Write New primary serves Primary data Read Secondary Read Thursday, 25 October 12
  • 14. Replica Sets App Secondary Read Write Primary Read Secondary Read Thursday, 25 October 12
  • 17. Sharding Data Distribution across nodes • Data location transparent to your code • Data distribution is automatic • Data re-distribution is automatic • Aggregate system resources horizontally • No code changes Thursday, 25 October 12
  • 18. Sharding - Range distribution sh.shardCollection("test.tweets", {_id: 1} , false) shard01 shard02 shard03 Thursday, 25 October 12
  • 19. Sharding - Range distribution shard01 shard02 shard03 a-i j-r s-z Thursday, 25 October 12
  • 20. Sharding - Splits shard01 shard02 shard03 a-i ja-jz s-z k-r Thursday, 25 October 12
  • 21. Sharding - Splits shard01 shard02 shard03 a-i ja-ji s-z ji-js js-jw jz-r Thursday, 25 October 12
  • 22. Sharding - Auto Balancing shard01 shard02 shard03 a-i ja-ji s-z ji-js js-jw js-jw jz-r jz-r Thursday, 25 October 12
  • 23. Sharding - Auto Balancing shard01 shard02 shard03 a-i ja-ji n-z ji-js js-jw jz-r Thursday, 25 October 12
  • 24. Sharding - Routed Query find({_id: "norberto"}) shard01 shard02 shard03 a-i ja-ji n-z ji-js js-jw jz-r Thursday, 25 October 12
  • 25. Sharding - Routed Query find({_id: "norberto"}) shard01 shard02 shard03 a-i ja-ji n-z ji-js js-jw jz-r Thursday, 25 October 12
  • 26. Sharding - Scatter Gather find({email: "norberto@10gen.com"}) shard01 shard02 shard03 a-i ja-ji n-z ji-js js-jw jz-r Thursday, 25 October 12
  • 27. Sharding - Scatter Gather find({email: "norberto@10gen.com"}) shard01 shard02 shard03 a-i ja-ji n-z ji-js js-jw jz-r Thursday, 25 October 12
  • 28. Sharding - Caching 96 GB Mem 3:1 Data/Mem shard01 a-i 300 GB Data j-r n-z 300 GB Thursday, 25 October 12
  • 29. Aggregate Horizontal Resources 96 GB Mem 96 GB Mem 96 GB Mem 1:1 Data/Mem 1:1 Data/Mem 1:1 Data/Mem shard01 shard02 shard03 a-i j-r n-z 300 GB Data 100 GB 100 GB 100 GB Thursday, 25 October 12
  • 31. Two choices for consistency •Eventual consistency •Allow updates when a system has been partitioned •Resolve conflicts later •Example: CouchDB, Cassandra •Immediate consistency •Limit the application of updates to a single master node for a given slice of data •Another node can take over after a failure is detected •Avoids the possibility of conflicts •Example: MongoDB Thursday, 25 October 12
  • 32. Durability •For how long is my data available? •When do I now that my data is safe? •Where? •Mongodb style •Fire and Forget •Get Last Error •Journal Sync •Replica Safe Thursday, 25 October 12
  • 35. Data Model • Why JSON? • Provides a simple, well understood encapsulation of data • Maps simply to the object in your OO language • Linking & Embedding to describe relationships Thursday, 25 October 12
  • 36. Json place1 = { name : "10gen HQ", address : "578 Broadway 7th Floor", city : "New York", zip : "10011", tags : [ "business", "tech" ] } Thursday, 25 October 12
  • 37. Schema Design Relational Database Thursday, 25 October 12
  • 38. Schema Design MongoDB embedding linking Thursday, 25 October 12
  • 39. Schemas in MongoDB Design documents that simply map to your application post = {author: "Hergé", date: new Date(), text: "Destination Moon", tags: ["comic", "adventure"]} > db.posts.save(post) Thursday, 25 October 12
  • 40. Embedding > db.blogs.find( { author: "Hergé"} ) { _id : ObjectId("4c4ba5c0672c685e5e8aabf3"), author : "Hergé", date : ISODate("2011-09-18T09:56:06.298Z"), text : "Destination Moon", tags : [ "comic", "adventure" ], comments : [ ! { ! ! author : "Kyle", ! ! date : ISODate("2011-09-19T09:56:06.298Z"), ! ! text : "great book" ! } ] } Thursday, 25 October 12
  • 41. JSON & Scaleout • Embedding removes need for • Distributed Joins • Two Phase commit • Enables data to be distributed across many nodes without penalty Thursday, 25 October 12
  • 46. download at mongodb.org! norberto@10gen.com Support, Training, Consulting, Events, Meetups http://www.10gen.com Facebook! Twitter! LinkedIn! http://bit.ly/mongofb! http://twitter.com/mongodb! http://linkd.in/joinmongo! Thursday, 25 October 12