MarioNote

June 4, 2010

Hadoop Installation Note (Standalone)

Filed under: Java,Linux,step-by-step — mariojump @ 10:30 pm
Tags: ,

This is a note when I installed Hadoop on a single machine (standalone).

I followed Hadoop Quick Start available at:
http://hadoop.apache.org/common/docs/r0.20.0/quickstart.html

Check Java version — Java 1.6.x is required, and Sun version is preferred. See:
http://hadoop.apache.org/common/docs/r0.20.0/quickstart.html

Download hadoop-0.20.2.tar.gz from:
http://hadoop.apache.org/mapreduce/releases.html#Download

$ tar xvzf hadoop-0.20.2.tar.gz

Skip rsync and ssh installation, as they are already available on the machine.

// Check the command usage:
$ cd hadoop-0.20.2
$ bin/hadoop
Usage: hadoop [–config confdir] COMMAND
where COMMAND is one of:
namenode -format format the DFS filesystem
secondarynamenode run the DFS secondary namenode
namenode run the DFS namenode
datanode run a DFS datanode
dfsadmin run a DFS admin client
mradmin run a Map-Reduce admin client
fsck run a DFS filesystem checking utility
fs run a generic filesystem user client
balancer run a cluster balancing utility
jobtracker run the MapReduce job Tracker node
pipes run a Pipes job
tasktracker run a MapReduce task Tracker node
job manipulate MapReduce jobs
queue get information regarding JobQueues
version print the version
jar run a jar file
distcp copy file or directories recursively
archive -archiveName NAME * create a hadoop archive
daemonlog get/set the log level for each daemon
or
CLASSNAME run the class named CLASSNAME
Most commands print help when invoked w/o parameters.

$ mkdir input
$ cp conf/*.xml input
$ bin/hadoop jar hadoop-*-examples.jar grep input output ‘dfs[a-z.]+’
Error: JAVA_HOME is not set.

$ /usr/java/latest/bin/java -version
java version “1.6.0_16”

$ export JAVA_HOME=/usr/java/latest
[added this line in the .bashrc]
Reference: http://www.atmarkit.co.jp/fjava/special/distributed03/distributed03_1.html

Try again:
$ bin/hadoop jar hadoop-*-examples.jar grep input output ‘dfs[a-z.]+’

No error message is shown this time.
$ cat output/*
1 dfsadmin

January 17, 2010

Installing Subversive Plugin on Eclipse 3.5

Filed under: Android,Software Development,step-by-step — mariojump @ 10:49 am

Subversive is an Eclipse plugin to integrate with Subversion (SVN).

Subversive can be downloaded and installed from the Galileo Update Site.

Help > Install New Software… > select Galileo – http://download.eclipse.org/releases/galileo > Collaboration Tools.

Select:
Subversive SVN Team Provider (Incubation) 0.7.8.I20090904-1300
(This is the latest one as of Jan 17, 2010.)

Then, when I tried to use it with ProjectLocker, I got the following error.
“Selected SVN connector library is not available or cannot be loaded.”

To cope with this,
Add
http://www.polarion.org/projects/subversive/download/eclipse/2.0/update-site/
to the software update site in Eclipse.

And select:
Subversive SVN Connectors and the connector for your Subversion version.
Since ProjectLocker offers Subversion 1.6.6, I chose:

* Subversive SVN Connectors 2.2.1.I20091009-1900
* Native JavaHL 1.6 Implementation (Optional) 2.2.1.I20091009-1900
* JavaHL 1.6.0 Win32 Binaries (Optional) 2.2.1.I20091009-1900

Then, I could finally synchronize with ProjectLocker’s subversion.

[References]
http://www.eclipse.org/subversive/downloads.php
http://www.projectlocker.com/site/scm

Blog at WordPress.com.