SQL & Datasets

Brief Investigations: via Apple Historical Stock Prices

• Sources
• Development Notes
  • Logging
  • Software
  • Running Apache Spark Packages

Sources

The data set used herein is courtesy of Macrotrends

• Apple - 41 Year Stock Price History | AAPL

Development Notes

Logging

• scala-logging
  

  <groupId>com.typesafe.scala-logging</groupId>
  <artifactId>scala-logging_2.11</artifactId>
  <version>3.9.2</version>
  
  <groupId>ch.qos.logback</groupId>
  <artifactId>logback-classic</artifactId>
  <version>1.2.3</version>
  

  import com.typesafe.scalalogging.Logger

• ScalaLogging
  

  <groupId>com.typesafe.play</groupId>
  <artifactId>play_2.11</artifactId>
  <version>2.7.7</version>
  

  import play.api.Logger

• Log4j

  • Scala API
  • Tutorials
  • Console Appender

Software

• Java
  

  $ java -version
  
    java version "1.8.0_181"
    Java(TM) SE Runtime Environment (build 1.8.0_181-b13) <br/> 
    Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
  

• Scala
  

  $ scala -version
  
    Scala code runner version 2.11.12 -- Copyright 2002-2017, LAMP/EPFL

• Spark
  

  $ spark-submit.cmd --version # **if** Operating System **is** Windows
  
    Spark version 2.4.7

In terms of packaging, Maven, instead of Scala Build Tool (SBT), is now being used for all projects

• Maven
  

  $ mvn -version
  
    Apache Maven 3.6.3 
  
  # Packaging: either
  $ mvn clean package 
  
  # or 
  $ mvn clean install

Running Apache Spark Packages

• Either
  

  $ spark-submit 
    --class com.grey.StocksApp 
    --master local[*] 
    target/stocks-...-jar-with-dependencies.jar 
      https://raw.githubusercontent.com/briefings/stocks/develop/arguments.yaml
  

• or
  

  $ spark-submit
    --class com.grey.StocksApp 
    --name "stocks" 
    --master spark://10.0.0.6:7077 
    --num-executors 2 
    target/stocks-...-jar-with-dependencies.jar 
      https://raw.githubusercontent.com/briefings/stocks/develop/arguments.yaml

  The latter allows for computation analysis via the Spark Web UI.