Scribd
Upload
28 views3 pages

Hadoop Installation and Data Management Guide

The document outlines a series of experiments focused on installing and using Hadoop and its related technologies, including HDFS, MapReduce, Hive, HBase, and Sqoop. Each experiment includes aims, procedures, and sample outputs demonstrating various functionalities such as file management, matrix multiplication, word counting, and data import/export. The document serves as a practical guide for understanding and implementing Hadoop operations in a pseudo-distributed environment.

Uploaded by

aarthie2043
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
28 views3 pages

Hadoop Installation and Data Management Guide

The document outlines a series of experiments focused on installing and using Hadoop and its related technologies, including HDFS, MapReduce, Hive, HBase, and Sqoop. Each experiment includes aims, procedures, and sample outputs demonstrating various functionalities such as file management, matrix multiplication, word counting, and data import/export. The document serves as a practical guide for understanding and implementing Hadoop operations in a pseudo-distributed environment.

Uploaded by

aarthie2043
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

EXPERIMENT 1: Downloading and Installing Hadoop;

Understanding Different Hadoop Modes

Aim:
To install and configure Hadoop in pseudo-distributed mode and understand various Hadoop
operational modes.

Procedure/Steps:
1. Install Java (JDK 8 or higher).
2. Download Hadoop, extract it, and move to /usr/local/hadoop.
3. Configure environment variables and XML files ([Link], [Link], [Link]).
4. Format NameNode, start Hadoop services, and verify daemons using jps.

Sample Output:
NameNode
DataNode
SecondaryNameNode
ResourceManager
NodeManager

EXPERIMENT 2: HDFS File Management (Add, Retrieve, Delete


Files)

Aim:
To perform file management operations (add, retrieve, delete) in Hadoop Distributed File System
(HDFS).

Procedure/Steps:
1. Create HDFS directory: hdfs dfs -mkdir /user/hadoop
2. Copy file: hdfs dfs -put [Link] /user/hadoop/
3. List files: hdfs dfs -ls /user/hadoop/
4. Retrieve: hdfs dfs -get /user/hadoop/[Link] /home/user/
5. Delete: hdfs dfs -rm /user/hadoop/[Link]

Sample Output:
Found 1 items
-rw-r--r-- 1 hadoop supergroup 128 /user/hadoop/[Link]
Deleted /user/hadoop/[Link]

EXPERIMENT 3: Matrix Multiplication Using Hadoop MapReduce

Aim:
To implement matrix multiplication using the Hadoop MapReduce framework.

Procedure/Steps:
1. Create matrices A and B.
2. Upload both to HDFS.
3. Run MatrixMultiply MapReduce Java program.

Sample Output:
Matrix C:
C[0][0] = 36
C[0][1] = 41
C[1][0] = 64
C[1][1] = 73

EXPERIMENT 4: Word Count Using MapReduce

Aim:
To execute a basic word count program to understand the MapReduce paradigm.

Procedure/Steps:
1. Upload text file to HDFS input directory.
2. Run WordCount MapReduce program.
3. View output from HDFS.

Sample Output:
hadoop 3
mapreduce 2
bigdata 5

EXPERIMENT 5: Installation of Hive

Aim:
To install Apache Hive and perform sample operations.

Procedure/Steps:
1. Download and extract Hive.
2. Configure environment variables.
3. Initialize Hive metastore.
4. Run Hive shell and execute SQL queries.

Sample Output:
1 Aarthie 50000.0

EXPERIMENT 6: Installation of HBase

Aim:
To install HBase and perform basic shell operations.

Procedure/Steps:
1. Download and extract HBase.
2. Configure [Link] with HDFS root directory.
3. Start HBase and use shell to create and manipulate tables.

Sample Output:
ROW COLUMN+CELL
1 column=info:name, value=Aarthie
EXPERIMENT 7: Installing Thrift and Connecting HBase

Aim:
To configure the HBase Thrift server and access HBase using Python.

Procedure/Steps:
1. Start Thrift server: hbase thrift start
2. Install happybase in Python.
3. Connect and perform put and get operations.

Sample Output:
[(b'1', {b'info:name': b'Aarthie'})]

EXPERIMENT 8: Importing and Exporting Data Between


Databases

Aim:
To import and export data between Hadoop and RDBMS using Sqoop.

Procedure/Steps:
1. Install Sqoop.
2. Import data from MySQL to HDFS.
3. Export data from HDFS to MySQL.

Sample Output:
Imported 5 records from MySQL to HDFS successfully.
Exported 5 records back to MySQL successfully.

You might also like