EMR AWS Demo

1. ELASTIC MAP REDUCE OF AWS Rim Moussa ENI-Carthage University of Carthage 2017/2018 1

2. DEMO OUTLINE  Simple Storage Service: S3  Job  Jar: Map Reduce code  Input: input data files  Output: output data files  All data must be on S3 including jar and input data  Create Hadoop Cluster  Size: number of workers  Hardware configuration  Stat the job 2

3. S3 3

4. CREATE A BUCKET FOR INPUT DATA ON S3 4

5. CREATE A NEW S3 BUCKET 5

6. UPLOAD DATA INTO THE BUCKET 6

7. UPLOAD IN PROGRESS 7

8. UPLOAD JAR INTO S3 8

9. CREATE EMR HADOOP CLUSTER 9

10. 10

11. 11

12. 12

13. 13

14. CREATE A NEW KEY PAIR 14 Click here

15. 15

16. DOWNLOAD NEW KEY AND CHANGE ACCESS RIGHTS 16

17. CLUSTER PROVISIONING 17

18. HARDWARE TAB 18

19. BOOTSTRAPPING 19

20. CONNECT TO MASTER USING SSH 20

21. RESIZE A CLUSTER 21 Add or drop instances

22. INSTANCES ARE RUNNING 22

23. SUBMIT A MAPREDUCE JOB 23

24. JOB DETAILS 24

25. JOB IS RUNNING 25

26. 26

27. 27

28. /! TERMINATE CLUSTER 28

29. FIND OUT 29  S3  How to upload data from a terminal to S3  Scenario where data is some where in the net  Hadoop Master  Compile the job on the master  Submit the job from a terminal on the master  Performance Tuning  Hadoop cluster configuration  RAM allocated to each Mapper, Reducer  Data Compression  Code  Input Split Size  Adjust the number of reducers

EMR AWS Demo

More Related Content

EMR AWS Demo