Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 70 views

    Installing Spark

    Uploaded by

    Charles W Gitahi
    The document provides steps for installing Spark on Windows: 1. Download and install Hadoop and Java from provided links 2. Download Spark from the Apache website 3. Unzip Spark and set the SPARK_HOME environment variable to point to the Spark folder 4. Add SPARK_HOME/bin to the system Path variable and grant permissions to the temp folder 5. Test the Spark installation by running the spark-shell command

    Copyright:

    © All Rights Reserved
    Download as PPT, PDF, TXT or read online from Scribd

    Installing Spark

    Uploaded by

    Charles W Gitahi
    70 views11 pages
    The document provides steps for installing Spark on Windows: 1. Download and install Hadoop and Java from provided links 2. Download Spark from the Apache website 3. Unzip Spark and set the SPARK_HOME environment variable to point to the Spark folder 4. Add SPARK_HOME/bin to the system Path variable and grant permissions to the temp folder 5. Test the Spark installation by running the spark-shell command

    Original Description:

    Installing Spark

    Original Title

    Installing Spark.ppt

    Copyright

    © © All Rights Reserved

    Available Formats

    PPT, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document provides steps for installing Spark on Windows: 1. Download and install Hadoop and Java from provided links 2. Download Spark from the Apache website 3. Unzip Spark and set the SPARK_HOME environment variable to point to the Spark folder 4. Add SPARK_HOME/bin to the system Path variable and grant permissions to the temp folder 5. Test the Spark installation by running the spark-shell command

    Copyright:

    © All Rights Reserved
    Download as PPT, PDF, TXT or read online from Scribd
    Download as ppt, pdf, or txt
    70 views11 pages

    Installing Spark

    Uploaded by

    Charles W Gitahi
    The document provides steps for installing Spark on Windows: 1. Download and install Hadoop and Java from provided links 2. Download Spark from the Apache website 3. Unzip Spark and set the SPARK_HOME environment variable to point to the Spark folder 4. Add SPARK_HOME/bin to the system Path variable and grant permissions to the temp folder 5. Test the Spark installation by running the spark-shell command

    Copyright:

    © All Rights Reserved
    Download as PPT, PDF, TXT or read online from Scribd
    Download as ppt, pdf, or txt
    of 11

    FACULTY OF COMPUTING AND INFORMATION

    MANAGEMENT

    Installing Spark
    Steps for Installing spark
     1.Download and Install Hadoop
     Hadoop can be downloaded from the link below:
     http://hadoop.apache.org/releases.html 
     Because this version does not include windows 10 binary, refer the
    to tutorial for installing hadoop in Windows environment:
    Steps for Installing spark
    2. Download and Install Java virtual machine.
    The latest version can be downloaded from
    http://www.oracle.com/technetwork/java/javase/downloads/inde

    x.html

    3. Download Spark from http://spark.apache.org/downloads.html 

    4. Unzip and extract your download into a local folder.


    Winzip or winrar can be used to unpack spark zip file.

    Spark folder within drive and be used to storing the unzipped file.
    Steps for Installing spark
     5. Create SPARK HOME variable

     From windows "Search advanced system settings" program-> click

    the button of "Environment Variables"


    Add new user variable called SPARK_HOME for storing a path that
    points the location Spark binary files
    SPARK_HOME --> C:\Spark\spark-2.3.2-bin-hadoop2.7

    This is an example.
    Steps for Installing spark
     6. edit the Path and add SPARK_HOME variable as follows
     %SPARK_HOME%\bin

     Example
    Steps for Installing spark
     5. Grant permission to temp folder
     Create a temp folder under c:\tmp
     Execute below command as administrator.
     winutils.exe chmod 777 C:\tmp
    Steps for Installing spark
     6. Test spark by typing "spark-shell“ command on the prompt
     Example
    Closing Spark Shell
     To close Spark shell, press Ctrl+D or type in :q 
     Example
    Connecting spark with python
     Pyspark command is used to connect spark with python
     Example
    References for installing Spark

     https://hernandezpaul.wordpress.com/2016/01/24/apache-spark-
    installation-on-windows-10/

     https://github.com/Cheng-Lin-Li/Spark/wiki/How-to-install-Spark-
    2.1.0-in-Windows-10-environment

    You might also like