AMI NAME : CS643FinalProjMovies AMI ID: ami-70496115 Region: US East (Ohio)

#Cloud Project# We tested 7 different test cases for Seasonal Movies, AvgRatings,Identifying Movie Genres, Top User influencers, Age groups based on gender, Zip codes for age groups

#Data Migration to HDFS#

After downloading the input file (states file), I followed 2 steps to transfer the contents from local machine to HDFS.

a.Copy this data from local to Namenode. b.Copy this data from Namenode to hdfs.

Use the following commands:

Create a directory called states in Namenode and transfer the states content from local machine to Namenode/Akshay

Namenode$ mkdir ~/Akshay local$ scp -i ~/.ssh/CS643proj-ami.pem /Downloads/Akshay/* Namenode:/Akshay/ Namenode$ hdfs dfs -mkdir /Akshay Namenode$ hdfs dfs -copyFromLocal ~/Akshay/* /Akshay/ Namenode$ hdfs dfs -mkdir /FinalResults

#AvgRating# Change directory to AvgRating Namenode$ cd AvgRating Namenode$ source ~/.bash_profile

Compile the program

Namenode$ javac AvgRatingSorted.java -cp $(hadoop classpath)

Create a jar file called WordCounter.jar

Namenode$ jar cf AvgratingSorted.jar AvgRatingSorted*.class

Run the application program

Namenode$ hadoop jar AvgRatingSorted.jar AvgRatingSorted /Akshay/input/ratings.dat/ /FinalResults/out_avgratingsorted

Namenode$ hdfs dfs -cat /FinalResults/out_avgratingsorted/part-r-00000

#MovieGenreIdentifier# Come to the directory Akshay Namenode$ cd ..

Change directory to usermovierating Namenode$ cd usermovierating Namenode$ source ~/.bash_profile

Compile the program

Namenode$ javac MovieGenreIdentifier.java -cp $(hadoop classpath)

Create a jar file called MovieGenreIdentifier.jar

Namenode$ jar cf MovieGenreIdentifier.jar MovieGenreIdentifier*.class

Run the application program

Namenode$ hadoop jar MovieGenreIdentifier.jar MovieGenreIdentifier /Akshay/input/movies.dat /FinalResults/out_moviegenreidentifer "Father of the Bride Part II (1995)"

Namenode$ hdfs dfs -cat /FinalResults/out_moviegenreidentifer/part-r-00000

#UsersRatedNMovies# Come to the directory Akshay Namenode$ cd ..

Change directory to usermovierating Namenode$ cd usermovierating Namenode$ source ~/.bash_profile Compile the program

Namenode$ javac UsersRatedNMovies.java -cp $(hadoop classpath)

Create a jar file called UsersRatedNMovies.jar

Namenode$ jar cf UsersRatedNMovies.jar UsersRatedNMovies*.class

Run the application program

Namenode$ hadoop jar UsersRatedNMovies.jar UsersRatedNMovies /Akshay/input/ratings.dat /Result/out_userratednmovies 1000

Namenode$ hdfs dfs -cat /FinalResults/out_userratednmovies/part-r-00000

#ZipCodeAvgAge# Come to the directory Akshay Namenode$ cd ..

Change directory to zipCodeAvgAge Namenode$ cd zipCodeAvgAge

Compile the program

Namenode$ javac ZipCodeAvgAge.java -cp $(hadoop classpath)

Create a jar file called ZipCodeAvgAge.jar

Namenode$ jar cf ZipCodeAvgAge.jar ZipCodeAvgAge*.class

Run the application program

Namenode$ hadoop jar ZipCodeAvgAge.jar ZipCodeAvgAge /Akshay/input/users.dat/ /FinalResults/temp_ZipCodeAvgAge/ /FinalResults/out_ZipCodeAvg/

Namenode$ hdfs dfs -cat /FinalResults/out_ZipCodeAvg/part-r-00000

#GroupUserBasedOnGenderAndAge# Come to the directory Akshay Namenode$ cd ..

Change directory to group Namenode$ cd group

Compile the program

Namenode$ javac GroupUserBasedOnGenderAndAge.java -cp $(hadoop classpath)

Create a jar file called GroupUserBasedOnGenderAndAge.jar

Namenode$ jar cf GroupUserBasedOnGenderAndAge.jar GroupUserBasedOnGenderAndAge*.class

Run the application program

Namenode$ hadoop jar GroupUserBasedOnGenderAndAge.jar GroupUserBasedOnGenderAndAge /Akshay/input/users.dat /FinalResults/out_group

Namenode$ hdfs dfs -cat /FinalResults/out_group/part-r-00000

#MovieIdentifierForGenre# Come to the directory Akshay Namenode$ cd ..

Change directory to movieByGenre Namenode$ cd movieByGenre

Compile the program

Namenode$ javac MovieIdentifierForGenre.java -cp $(hadoop classpath)

Create a jar file called MovieIdentifierForGenre.jar

Namenode$ jar cf MovieIdentifierForGenre.jar MovieIdentifierForGenre*.class

Run the application program

Namenode$ hadoop jar MovieIdentifierForGenre.jar MovieIdentifierForGenre /Akshay/input/movies.dat /FinalResults/out_movieByGenre_thriller "Thriller"

Namenode$ hdfs dfs -cat /FinalResults/out_movieByGenre_thriller/part-r-00000

#SeasonalMovies# Come to the directory Akshay Namenode$ cd ..

Change directory to SeasonalMovies Namenode$ cd SeasonalMovies

Compile the program

Namenode$ javac SeasonalMovies.java -cp $(hadoop classpath)

Create a jar file called SeasonalMovies.jar

Namenode$ jar cf SeasonalMovies.jar SeasonalMovies*.class

Run the application program

Namenode$ hadoop jar SeasonalMovies.jar SeasonalMovies /Akshay/input/movies_metadata.dat /FinalResults/out_SeasonalMovies

Namenode$ hdfs dfs -cat /FinalResults/out_SeasonalMovies/part-r-00000

Note : Pre Prep for any prior codes which are on HDFS which may prevent output #WORDCOUNTER# Namenode$ source ~/.profile Namenode$ cd Filename Namenode$ rm Filename*.class Namenode$ rm Fielname.jar Namenode$ hdfs dfs -rm -r path