Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 16 views

    Av Grating

    Uploaded by

    niharika sunkara
    The document describes a Java program that uses Hadoop MapReduce to analyze movie rating data from multiple input files to calculate the average rating for each movie. The program uses two mapper classes to parse the input files, a reducer class to calculate the averages, and outputs the results to a new file.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Av Grating

    Uploaded by

    niharika sunkara
    16 views5 pages
    The document describes a Java program that uses Hadoop MapReduce to analyze movie rating data from multiple input files to calculate the average rating for each movie. The program uses two mapper classes to parse the input files, a reducer class to calculate the averages, and outputs the results to a new file.

    Original Description:

    Hadoop movie rating

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document describes a Java program that uses Hadoop MapReduce to analyze movie rating data from multiple input files to calculate the average rating for each movie. The program uses two mapper classes to parse the input files, a reducer class to calculate the averages, and outputs the results to a new file.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    16 views5 pages

    Av Grating

    Uploaded by

    niharika sunkara
    The document describes a Java program that uses Hadoop MapReduce to analyze movie rating data from multiple input files to calculate the average rating for each movie. The program uses two mapper classes to parse the input files, a reducer class to calculate the averages, and outputs the results to a new file.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 5

    MOVIE RATING:

    DRIVER:

    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.fs.Path;
    import org.apache.hadoop.io.*;
    import org.apache.hadoop.mapreduce.Job;
    import org.apache.hadoop.mapreduce.lib.input.MultipleInputs;
    import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
    import org.apache.hadoop.mapreduce.lib.input.TextInputFormat;
    import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
    import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;

    public class avgrating


    {
    public static void main(String args[]) throws Exception
    {
    Configuration conf=new Configuration();
    Job job=Job.getInstance(conf,"abc");
    job.setJarByClass(avgrating.class);

    job.setOutputKeyClass(IntWritable.class);
    job.setOutputValueClass(Text.class);
    job.setReducerClass(ratingreducer.class);
    job.setInputFormatClass(TextInputFormat.class);
    job.setOutputFormatClass(TextOutputFormat.class);
    MultipleInputs.addInputPath(job,new
    Path(args[0]),TextInputFormat.class,ratingmapper1.class);
    MultipleInputs.addInputPath(job,new
    Path(args[1]),TextInputFormat.class,ratingmapper2.class);
    FileOutputFormat.setOutputPath(job,new Path(args[2]));
    job.waitForCompletion(true);
    }
    }

    MAPPER1:

    import java.io.IOException;
    import org.apache.hadoop.io.*;
    import org.apache.hadoop.mapreduce.Mapper;

    public class ratingmapper1 extends


    Mapper<LongWritable,Text,IntWritable,Text>
    {
    public void map(LongWritable key,Text value,Context
    context) throws IOException,InterruptedException
    {
    String s[]=value.toString().split("::");
    int p=Integer.parseInt(s[0]);
    context.write(new IntWritable(p),new Text(s[1]));
    }
    }

    MAPPER2:

    import java.io.IOException;
    import org.apache.hadoop.io.*;
    import org.apache.hadoop.mapreduce.Mapper;

    public class ratingmapper2 extends


    Mapper<LongWritable,Text,IntWritable,Text>
    {
    public void map(LongWritable key,Text value,Context
    context) throws IOException,InterruptedException
    {
    String s[]=value.toString().split("::");
    int p=Integer.parseInt(s[0]);
    //float t=Float.parseFloat(s[2]);
    context.write(new IntWritable(p),new Text(s[2]));
    }
    }

    REDUCER:

    import java.io.IOException;
    import java.util.Iterator;
    import org.apache.hadoop.io.*;
    import org.apache.hadoop.mapreduce.Reducer;

    public class ratingreducer extends


    Reducer<IntWritable,Text,Text,FloatWritable>
    {
    public void reduce(IntWritable key,Iterable<Text>
    vals,Context context) throws IOException,InterruptedException
    {
    float avg,sum=0,p;
    int c=0;
    //String n=" ";
    Text m=new Text();
    Iterator<Text> x=vals.iterator();
    while(x.hasNext())
    {
    String s=x.next().toString();
    if(s.length()>2)
    {
    m=new Text(s);
    }
    else
    {
    p=Float.parseFloat(s.toString());
    sum=sum+p;
    c++;
    }
    }
    avg=sum/c;
    context.write(m,new FloatWritable(avg));
    }
    }

    INPUT:

    FILE 1:

    1::Toy Story (1995)::Animation|Children's|Comedy


    2::Jumanji (1995)::Adventure|Children's|Fantasy
    3::Grumpier Old Men (1995)::Comedy|Romance
    4::Waiting to Exhale (1995)::Comedy|Drama
    5::Father of the Bride Part II (1995)::Comedy
    6::Heat (1995)::Action|Crime|Thriller
    7::Sabrina (1995)::Comedy|Romance
    8::Tom and Huck (1995)::Adventure|Children's
    9::Sudden Death (1995)::Action
    10::GoldenEye (1995)::Action|Adventure|Thriller
    11::American President, The (1995)::Comedy|Drama|Romance
    12::Dracula: Dead and Loving It (1995)::Comedy|Horror
    13::Balto (1995)::Animation|Children's
    14::Nixon (1995)::Drama
    15::Cutthroat Island (1995)::Action|Adventure|Romance
    16::Casino (1995)::Drama|Thriller

    FILE 2:
    1::1193::5::978300760

    1::661::3::978302109

    1::914::3::978301968

    1::3408::4::978300275

    1::2355::5::978824291

    1::1197::3::978302268

    1::1287::5::978302039

    1::2804::5::978300719

    1::594::4::978302268

    1::919::4::978301368

    1::595::5::978824268

    1::938::4::978301752

    1::2398::4::978302281

    1::2918::4::978302124

    1::1035::5::978301753

    1::2791::4::978302188

    1::2687::3::978824268

    1::2018::4::978301777

    1::3105::5::978301713

    1::2797::4::978302039

    Output:

    You might also like