Basic Hadoop Operations

Create a directory in hadoop:

• hadoop fs -mkdir hdfs://localhost:9000/user/hadoop/mytestdir2
• hadoop fs -mkdir /user/hadoop/mytestdir3

LISTING:

• hadoop fs -ls  hdfs://localhost:9000/user/hadoop/
• hadoop fs -ls /user/hadoop/

Create a local file and upload to hadoop directory: touch helloworld.txt or nano helloworld.txt

• hadoop fs -put helloworld.txt /shiraz/hadoop/ 
•  hadoop fs -cat /shiraz/hadoop/helloworld.txt

compiling:

•  hadoop com.sun.tools.javac.Main WordCount.java

you may need to export the HADOOP_CLASSPATH variable:

export HADOOP_CLASSPATH= /usr/lib/jvm/java-7-openjdk-amd64/lib/tools.jar

running a hadoop job:

• hadoop jar wc.jar WordCount /shiraz/hadoop/input /shiraz/hadoop/output/paracount

hadoop executable location:

• /usr/local/hadoop/bin/hadoop

conf/*-site.xml

• /usr/local/hadoop/etc

mapred-site.xml

yarn-site.xml

core-site.xml
hdfs-site.xml

hdfs location:

• /usr/local/hadoop_store/hdfs

start-all.sh, start-dfs.sh,  start-yarn.sh

• /usr/local/hadoop/sbin