Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 53 views

    Big Data Defined: Andrew J. Brust

    Uploaded by

    Favio90
    This document defines big data and describes how it is analyzed. It explains that big data involves hundreds of terabytes to petabytes of information that is too large for traditional databases. Hadoop is used to perform distributed and parallel processing on big data using MapReduce. MapReduce splits and processes the data in the map step, aggregates results in the reduce step, and is commonly employed by Hadoop but also used by other systems. The document provides an example of how MapReduce could be used to count data by floor and platform in a building. It also discusses data scientists and their skills in statistics, subject matter expertise, and asking the right questions to analyze big data.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Big Data Defined: Andrew J. Brust

    Uploaded by

    Favio90
    53 views5 pages
    This document defines big data and describes how it is analyzed. It explains that big data involves hundreds of terabytes to petabytes of information that is too large for traditional databases. Hadoop is used to perform distributed and parallel processing on big data using MapReduce. MapReduce splits and processes the data in the map step, aggregates results in the reduce step, and is commonly employed by Hadoop but also used by other systems. The document provides an example of how MapReduce could be used to count data by floor and platform in a building. It also discusses data scientists and their skills in statistics, subject matter expertise, and asking the right questions to analyze big data.

    Original Description:

    Great presentation about big data :)

    Original Title

    Big Data Defined

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document defines big data and describes how it is analyzed. It explains that big data involves hundreds of terabytes to petabytes of information that is too large for traditional databases. Hadoop is used to perform distributed and parallel processing on big data using MapReduce. MapReduce splits and processes the data in the map step, aggregates results in the reduce step, and is commonly employed by Hadoop but also used by other systems. The document provides an example of how MapReduce could be used to count data by floor and platform in a building. It also discusses data scientists and their skills in statistics, subject matter expertise, and asking the right questions to analyze big data.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    53 views5 pages

    Big Data Defined: Andrew J. Brust

    Uploaded by

    Favio90
    This document defines big data and describes how it is analyzed. It explains that big data involves hundreds of terabytes to petabytes of information that is too large for traditional databases. Hadoop is used to perform distributed and parallel processing on big data using MapReduce. MapReduce splits and processes the data in the map step, aggregates results in the reduce step, and is commonly employed by Hadoop but also used by other systems. The document provides an example of how MapReduce could be used to count data by floor and platform in a building. It also discusses data scientists and their skills in statistics, subject matter expertise, and asking the right questions to analyze big data.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 5

    Big Data Defined

    Andrew J. Brust
    http://www.bluebadgeinsights.com
    andrew.brust@bluebadgeinsights.com
    Big Data Defined

    100s of TB – x PB

    Uses Hadoop

    Three Vs

    Too big for OLTP

    Uses distributed/parallel processing


    MapReduce

     Map step: split the data and pre-process it


     Reduce step: aggregate the results
     Most typical of Hadoop but employed by others, to various extents
    A MapReduce Example

    • Count by suite, on each floor

    • Send per-suite, per platform totals to lobby

    • Sort totals by platform

    • Send two platform packets to 10th, 20th, 30th floor

    • Tally up each platform

    • Collect the tallies

    • Merge tallies into one spreadsheet


    Data Scientists

    Near Abuse of
    But with:
    synonyms: term:
    • Statisticians • Subject • Hadoop
    • “Quants” matter experts
    expertise • “R”
    • Good at developers
    knowing (although
    the right that does
    questions to overlap)
    ask

    You might also like