Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 72 views

    Word Count

    Uploaded by

    niharika sunkara
    This document contains code for a MapReduce word count program in Java. It includes a Driver class that runs the job, a Mapper class that tokenizes the input text and emits each word along with a count of 1, and a Reducer class that sums the counts for each word. Sample input and output data is provided showing the word counts calculated from the given text.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Word Count

    Uploaded by

    niharika sunkara
    72 views3 pages
    This document contains code for a MapReduce word count program in Java. It includes a Driver class that runs the job, a Mapper class that tokenizes the input text and emits each word along with a count of 1, and a Reducer class that sums the counts for each word. Sample input and output data is provided showing the word counts calculated from the given text.

    Original Description:

    WordCount

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document contains code for a MapReduce word count program in Java. It includes a Driver class that runs the job, a Mapper class that tokenizes the input text and emits each word along with a count of 1, and a Reducer class that sums the counts for each word. Sample input and output data is provided showing the word counts calculated from the given text.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    72 views3 pages

    Word Count

    Uploaded by

    niharika sunkara
    This document contains code for a MapReduce word count program in Java. It includes a Driver class that runs the job, a Mapper class that tokenizes the input text and emits each word along with a count of 1, and a Reducer class that sums the counts for each word. Sample input and output data is provided showing the word counts calculated from the given text.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 3

    WORD COUNT:

    DRIVER:

    import org.apache.hadoop.mapreduce.*;
    import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
    import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
    import org.apache.hadoop.mapreduce.lib.input.TextInputFormat;
    import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
    import org.apache.hadoop.mapreduce.Job;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.io.LongWritable;
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.fs.Path;
    import java.io.*;
    import java.util.*;
    public class wc{
    public static void main(String args[]) throws Exception
    {
    Configuration conf=new Configuration();

    Job job=Job.getInstance(conf,"xyz");
    //job.setJarByClass(WordCount.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(LongWritable.class);
    //job.setNumReduceTasks(2);
    job.setMapperClass(wcmapper.class);
    job.setReducerClass(wcreducer.class);
    //job.setPartitionerClass(wcpartition.class);
    //job.setCombinerClass(wccombiner.class);
    FileInputFormat.setInputPaths(job,new Path(args[0]));
    FileOutputFormat.setOutputPath(job,new Path(args[1]));
    job.setJarByClass(wc.class);
    job.waitForCompletion(true);
    }
    }

    MAPPER:

    import org.apache.hadoop.mapreduce.*;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.io.LongWritable;
    import java.io.*;
    public class wcmapper extends
    Mapper<LongWritable,Text,Text,LongWritable>
    {
    LongWritable one=new LongWritable(1);

    public void map(LongWritable key,Text value,Context context)


    throws IOException,InterruptedException
    {
    String line=value.toString();
    String[] wordsinline=line.split(" ");
    for(int i=0;i<wordsinline.length;i++)
    context.write(new Text(wordsinline[i]),one);
    }
    }

    REDUCER:

    import org.apache.hadoop.mapreduce.*;
    import org.apache.hadoop.io.Text;
    import org.apache.hadoop.io.LongWritable;
    import java.io.*;
    public class wcreducer extends
    Reducer<Text,LongWritable,Text,LongWritable>
    {
    LongWritable totalWc=new LongWritable();
    public void reduce(Text
    key,Iterable<LongWritable>values,Context context)throws
    IOException,InterruptedException
    {
    long wordcount=0;
    for(LongWritable val:values)
    {
    wordcount=wordcount+val.get();
    }
    totalWc.set(wordcount);
    context.write(key,totalWc);
    }
    }

    INPUT:

    hadoop programs
    this is hadoop lab
    we are working on map reduce

    OUTPUT:

    You might also like