This is a note when I installed Hadoop on a single machine (standalone).
I followed Hadoop Quick Start available at:
http://hadoop.apache.org/common/docs/r0.20.0/quickstart.html
Check Java version — Java 1.6.x is required, and Sun version is preferred. See:
http://hadoop.apache.org/common/docs/r0.20.0/quickstart.html
Download hadoop-0.20.2.tar.gz from:
http://hadoop.apache.org/mapreduce/releases.html#Download
$ tar xvzf hadoop-0.20.2.tar.gz
Skip rsync and ssh installation, as they are already available on the machine.
// Check the command usage:
$ cd hadoop-0.20.2
$ bin/hadoop
Usage: hadoop [--config confdir] COMMAND
where COMMAND is one of:
namenode -format format the DFS filesystem
secondarynamenode run the DFS secondary namenode
namenode run the DFS namenode
datanode run a DFS datanode
dfsadmin run a DFS admin client
mradmin run a Map-Reduce admin client
fsck run a DFS filesystem checking utility
fs run a generic filesystem user client
balancer run a cluster balancing utility
jobtracker run the MapReduce job Tracker node
pipes run a Pipes job
tasktracker run a MapReduce task Tracker node
job manipulate MapReduce jobs
queue get information regarding JobQueues
version print the version
jar run a jar file
distcp copy file or directories recursively
archive -archiveName NAME * create a hadoop archive
daemonlog get/set the log level for each daemon
or
CLASSNAME run the class named CLASSNAME
Most commands print help when invoked w/o parameters.
$ mkdir input
$ cp conf/*.xml input
$ bin/hadoop jar hadoop-*-examples.jar grep input output ‘dfs[a-z.]+’
Error: JAVA_HOME is not set.
$ /usr/java/latest/bin/java -version
java version “1.6.0_16″
$ export JAVA_HOME=/usr/java/latest
[added this line in the .bashrc]
Reference: http://www.atmarkit.co.jp/fjava/special/distributed03/distributed03_1.html
Try again:
$ bin/hadoop jar hadoop-*-examples.jar grep input output ‘dfs[a-z.]+’
No error message is shown this time.
$ cat output/*
1 dfsadmin