Jul 11, 2020 The 2.6.3 version of Spark for Mac is provided as a free download on our software library. Our built-in antivirus scanned this Mac download and rated it as 100% safe. The following versions: 2.6, 2.5 and 2.1 are the most frequently downloaded ones by the program users. The default filename for the application's installer is spark263.dmg. Jan 11, 2015 How to Install Apache Spark on Mac OS X Yosemite January 11, 2015 genomegeek Apache Spark, BIg Data Leave a comment Apache Spark™ is a fast and general engine for large-scale data processing.
ContentsFollowing is a detailed step by step process to install latest Apache Spark on Mac OS. We shall first install the dependencies : Java and Scala. To install these programming languages and framework, we take help of Homebrew and xcode-select.
Open Terminal. Run the following command in Terminal :
/usr/bin/ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)” |
Enter the password if asked and continue.
>Cleaning up/Library/Caches/Homebrew... >Migrating/Library/Caches/Homebrew to/Users/apple/Library/Caches/Homebrew.. Already up-to-date. >Homebrew has enabled anonymous aggregate user behaviour analytics. Read the analytics documentation (andhow toopt-out) here: -Run `brew help` toget started http://docs.brew.sh |
To install Java, Scala and Apache Spark through command line interface in Terminal, we shall install xcode-select. Enter and run the following command in Terminal :
xcode-select –install |
To install Java through command line, enter and run the following command in the Terminal :
brew cask install java |
Ifyour Java application still asks forJRE installation, you might need Installing thisCask means you have AGREED tothe Oracle Binary Code https://www.oracle.com/technetwork/java/javase/terms/license/index.html >Satisfying dependencies >Downloading http://download.oracle.com/otn-pub/java/jdk/8u144-b01/090f390dda ######################################################################## 100.0% >Installing Cask java >Running installer forjava; your password may be necessary. >Packageinstallers may write toany location; options such as--appdir arei >installer:Packagename isJDK8Update144 >installer:The upgrade was successful. |
To install Scala through command line, enter and run the following command in Terminal :
brew install scala |
apples-MBP:~Prasanth$ brew install scala >Auto-updated Homebrew! >Updated Formulae >Downloading https://downloads.lightbend.com/scala/2.12.3/scala-2.12.3.tgz ######################################################################## 100.0% >Downloading https://raw.githubusercontent.com/scala/scala-tool-support/0a217 ######################################################################## 100.0% Tousewith IntelliJ, set the Scala home to: /usr/local/etc/bash_completion.d ? /usr/local/Cellar/scala/2.12.3:44files, 20.1MB, built in1minute1second |
To install Apache Spark through command line, enter and run the following command in the Terminal :
brew install apache-spark |
apples-MBP:~Prasanth$ brew install apache-spark >Downloading https://www.apache.org/dyn/closer.lua?path=spark/spark-2.2.0/spa >Best Mirror http://www-eu.apache.org/dist/spark/spark-2.2.0/spark-2.2.0-bin- ######################################################################## 100.0% ? /usr/local/Cellar/apache-spark/2.2.0:1,318files, 221.5MB, built in12minutes8seconds |
To verify if the installation is successful, run the spark using the following command in Terminal :
spark-shell |
apples-MBP:~Prasanth$ spark-shell Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). 17/08/01 21:52:51 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 17/08/01 21:52:58 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0 17/08/01 21:52:58 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException 17/08/01 21:52:59 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException Spark context Web UI available at http://192.168.1.101:4040 Spark context available as 'sc' (master = local[*], app id = local-1501604572582). Welcome to / _/_ _ __/ /_ /_/ ._/,// // version 2.2.0 Using Scala version2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java1.8.0_144) Type:help formore information. scala> |
We have successfully installed Apache Spark on Mac OS.
The installation directory would be /usr/local/Cellar/apache-spark/.
In this Apache Spark Tutorial, we have learnt to install latest Apache Spark on Mac OS .