June 4, 2010

Hadoop Installation Note (Standalone)

Filed under: Java,Linux,step-by-step — mariojump @ 10:30 pm
Tags: ,

This is a note when I installed Hadoop on a single machine (standalone).

I followed Hadoop Quick Start available at:

Check Java version — Java 1.6.x is required, and Sun version is preferred. See:

Download hadoop-0.20.2.tar.gz from:

$ tar xvzf hadoop-0.20.2.tar.gz

Skip rsync and ssh installation, as they are already available on the machine.

// Check the command usage:
$ cd hadoop-0.20.2
$ bin/hadoop
Usage: hadoop [–config confdir] COMMAND
where COMMAND is one of:
namenode -format format the DFS filesystem
secondarynamenode run the DFS secondary namenode
namenode run the DFS namenode
datanode run a DFS datanode
dfsadmin run a DFS admin client
mradmin run a Map-Reduce admin client
fsck run a DFS filesystem checking utility
fs run a generic filesystem user client
balancer run a cluster balancing utility
jobtracker run the MapReduce job Tracker node
pipes run a Pipes job
tasktracker run a MapReduce task Tracker node
job manipulate MapReduce jobs
queue get information regarding JobQueues
version print the version
jar run a jar file
distcp copy file or directories recursively
archive -archiveName NAME * create a hadoop archive
daemonlog get/set the log level for each daemon
CLASSNAME run the class named CLASSNAME
Most commands print help when invoked w/o parameters.

$ mkdir input
$ cp conf/*.xml input
$ bin/hadoop jar hadoop-*-examples.jar grep input output ‘dfs[a-z.]+’
Error: JAVA_HOME is not set.

$ /usr/java/latest/bin/java -version
java version “1.6.0_16”

$ export JAVA_HOME=/usr/java/latest
[added this line in the .bashrc]

Try again:
$ bin/hadoop jar hadoop-*-examples.jar grep input output ‘dfs[a-z.]+’

No error message is shown this time.
$ cat output/*
1 dfsadmin

Create a free website or blog at