The document outlines how to run an Elastic MapReduce job on AWS. It discusses uploading input data and a MapReduce jar file to S3 storage. It then shows how to create an EMR Hadoop cluster with a specified number of workers and hardware configuration. Finally, it demonstrates submitting a MapReduce job to the cluster, monitoring its status, and terminating the cluster once complete.
1 of 29
Download to read offline
More Related Content
EMR AWS Demo
1. ELASTIC MAP REDUCE OF AWS
Rim Moussa
ENI-Carthage
University of Carthage
2017/2018
1
2. DEMO OUTLINE
Simple Storage Service: S3
Job
Jar: Map Reduce code
Input: input data files
Output: output data files
All data must be on S3 including jar and input data
Create Hadoop Cluster
Size: number of workers
Hardware configuration
Stat the job
2
29. FIND OUT
29
S3
How to upload data from a terminal to S3
Scenario where data is some where in the net
Hadoop Master
Compile the job on the master
Submit the job from a terminal on the master
Performance Tuning
Hadoop cluster configuration
RAM allocated to each Mapper, Reducer
Data Compression
Code
Input Split Size
Adjust the number of reducers