Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 18 views

    SplitPDFFile 1 To 7

    Uploaded by

    sivaani326
    The document describes how to implement matrix multiplication using MapReduce in Hadoop. It provides the code for the Map, Reduce, and driver classes to perform the parallel computation. The procedure explains the steps to compile the code into a JAR file and run the MapReduce job on Hadoop to multiply two matrices stored as text files and output the result.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    SplitPDFFile 1 To 7

    Uploaded by

    sivaani326
    18 views7 pages
    The document describes how to implement matrix multiplication using MapReduce in Hadoop. It provides the code for the Map, Reduce, and driver classes to perform the parallel computation. The procedure explains the steps to compile the code into a JAR file and run the MapReduce job on Hadoop to multiply two matrices stored as text files and output the result.

    Original Description:

    Bda

    Original Title

    SplitPDFFile_1_to_7

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document describes how to implement matrix multiplication using MapReduce in Hadoop. It provides the code for the Map, Reduce, and driver classes to perform the parallel computation. The procedure explains the steps to compile the code into a JAR file and run the MapReduce job on Hadoop to multiply two matrices stored as text files and output the result.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    18 views7 pages

    SplitPDFFile 1 To 7

    Uploaded by

    sivaani326
    The document describes how to implement matrix multiplication using MapReduce in Hadoop. It provides the code for the Map, Reduce, and driver classes to perform the parallel computation. The procedure explains the steps to compile the code into a JAR file and run the MapReduce job on Hadoop to multiply two matrices stored as text files and output the result.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 7

    EX NO:3 IMPLEMENTATION OF MATRIX MULTIPLICATION

    USING MAP REDUCE


    DATE:

    AIM:
    To implement matrix multiplication using mapreduce in Hadoop.

    PROGRAM:
    Map.java
    package matrix;
    import org.apache.hadoop.conf.*;
    import org.apache.hadoop.io.LongWritable;
    import org.apache.hadoop.io.Text;
    //import org.apache.hadoop.mapreduce.Mapper;
    import java.io.IOException;
    public class Map extends org.apache.hadoop.mapreduce.Mapper<LongWritable, Text, Text,
    Text>
    {
    @Override
    public void map(LongWritable key, Text value, Context context)
    throws IOException, InterruptedException {
    Configuration conf = context.getConfiguration();
    int m = Integer.parseInt(conf.get("m"));
    int p = Integer.parseInt(conf.get("p"));
    String line = value.toString();
    // (M, i, j, Mij);
    String[] indicesAndValue = line.split(",");
    Text outputKey = new Text();
    Text outputValue = new Text();
    if (indicesAndValue[0].equals("M")) {
    for (int k = 0; k < p; k++) {
    outputKey.set(indicesAndValue[1] + "," + k);
    // outputKey.set(i,k);
    outputValue.set(indicesAndValue[0] + "," + indicesAndValue[2]
    + "," + indicesAndValue[3]);
    // outputValue.set(M,j,Mij);
    context.write(outputKey, outputValue);
    }
    } else {
    // (N, j, k, Njk);
    for (int i = 0; i < m; i++) {
    outputKey.set(i + "," + indicesAndValue[2]); outputValue.set("N," +
    indicesAndValue[1] + ","
    + indicesAndValue[3]); context.write(outputKey, outputValue);
    }
    }
    }
    }
    MatrixMultiply.java
    package matrix;
    import org.apache.hadoop.conf.*;
    import org.apache.hadoop.fs.Path;
    import org.apache.hadoop.io.*;
    import org.apache.hadoop.mapreduce.*;
    import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
    import org.apache.hadoop.mapreduce.lib.input.TextInputFormat;
    import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
    import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat;
    public class MatrixMultiply {
    public static void main(String[] args) throws Exception { if (args.length != 2) {
    System.err.println("Usage: MatrixMultiply <in_dir> <out_dir>");
    System.exit(2);
    }
    Configuration conf = new Configuration();
    conf.set("m", "1000");
    conf.set("n", "100");
    conf.set("p", "1000");
    @SuppressWarnings("deprecation")
    Job job = new Job(conf, "MatrixMultiply");
    job.setJarByClass(MatrixMultiply.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(Text.class);
    job.setMapperClass(Map.class);
    job.setReducerClass(Reduce.class);
    job.setInputFormatClass(TextInputFormat.class);
    job.setOutputFormatClass(TextOutputFormat.class);
    FileInputFormat.addInputPath(job, new Path(args[0]));
    FileOutputFormat.setOutputPath(job, new Path(args[1]));
    job.waitForCompletion(true);
    }
    }

    Reduce.java
    package matrix;
    import org.apache.hadoop.io.Text;
    // import org.apache.hadoop.mapreduce.Reducer;
    import java.io.IOException;
    import java.util.HashMap;
    public class Reduce
    extends org.apache.hadoop.mapreduce.Reducer<Text, Text, Text, Text> { @Override
    public void reduce(Text key, Iterable<Text> values, Context context)
    throws IOException, InterruptedException {
    String[] value;
    //key=(i,k),
    //Values = [(M/N,j,V/W),..]
    HashMap<Integer, Float> hashA = new HashMap<Integer, Float>(); HashMap<Integer,
    Float> hashB = new HashMap<Integer, Float>(); for (Text val : values) {
    value = val.toString().split(",");
    if (value[0].equals("M")) {
    hashA.put(Integer.parseInt(value[1]), Float.parseFloat(value[2])); } else {
    hashB.put(Integer.parseInt(value[1]), Float.parseFloat(value[2]));
    }
    }
    int n = Integer.parseInt(context.getConfiguration().get("n"));
    float result = 0.0f;
    float m_ij;
    float n_jk;
    for (int j = 0; j < n; j++) {
    m_ij = hashA.containsKey(j) ? hashA.get(j) : 0.0f; n_jk = hashB.containsKey(j) ?
    hashB.get(j) : 0.0f; result += m_ij * n_jk;
    }
    if (result != 0.0f) {
    context.write(null,
    new Text(key.toString() + "," + Float.toString(result)));
    }
    }
    }
    PROCEDURE:
    Step 1:create a directory in hadoop
    Input the two text files with the matrix files int into the direcrory in Hadoop.
    Using
    Hadoop fs -put <file location>/<input directory>.
    Step 2:Open the netbeans application an open a new project.
    1.Choose categories as java and projects as java application.
    2.Set the project name as matrix and untick the create main class option.
    3.Add a new package a matrix and add the above three programs as three
    different class files

    4.Add libraries to the package by right clicking in the libraries tab an select add
    JAR/FOLDER.

    5.Add the folders like mapreduce and common.


    6.Right click on the matrix project and clean and build the project.

    7.The jar file will be added to the following location.

    Step 3 : Input the following jar file into the same directory where we inserted the matrix files.
    Using command hadoop jar <jar file name.jar jar name.package name.main class name > /
    input directory /output directory

    Step 4:Get the output matrix


    Using
    hadoop dfs -cat /<output directory>/*

    RESULT:
    Thus the implementation of matrix multiplication using map reduce was executed
    successfully.

    You might also like