Big Data using Apache Hadoop

About Course

In this course you will learn current industry requirements with Big Data Hadoop and Spark modules. This course is a combination of the Hadoop developer, Hadoop administrator, Hadoop testing and analytics with Apache Spark. This Big Data program will prepare you to clear Cloudera CCA175 Big Data certification.

Fundamentals of Hadoop and YARN and write applications using them
HDFS, MapReduce, Hive, Pig, Sqoop, Flume, and ZooKeeper
Spark, Spark SQL, Streaming, Data Frame, RDD, GraphX and MLlib writing Spark applications
Working with Avro data formats
Practicing real-life projects using Hadoop and Apache Spark
Be equipped to clear Big Data Hadoop Certification

Course Content

Module 1: Understanding Big Data

Different types of data

05:51
Different V’s of Big Data

02:01
Importance of Big Data

00:45
Understand RDBMS and its limitations for Big Data

01:39
Casestudy: Netflix

03:51
Casestudy: Big data in banking domain

05:19
Casestudy: Big data in e-commerce domain

07:31

Module 2: Introduction to Hadoop

Module 3: Storage Unit

Module 4: MapReduce and YARN

Module 5: Real time Big Data Processing

Module 6: Apache Pig

Module 7: Apache Hive

Features of Apache Hive

01:00:00
Command Line Interface, History of Apache Hive

01:00:00
Hive Data Types & Files Formats

01:00:00
Creating Managed Table, External Table, Partitioned Table

01:00:00
Dropping Tables, Alter Table, Inserting Data into Tables from Queries

02:00:00
SELECT from clauses, WHERE Clauses

01:00:00
GROUP BY Clauses, JOIN Statements, ORDER BY

02:00:00
SORT BY, DISTRIBUTE BY, CLUSTER BY Bucketing

02:00:00
UNION ALL, View, Hive Meta store

02:00:00

Module 8: Data Extraction Tool

Apache Sqoop, Sqoop Architecture, Need for Apache Sqoop

01:00:00
Sqoop Features Sqoop Connectors, Import Function, Incremental Import

02:00:00
Direct Mode Import, Performing Export Function

02:00:00
Exports, and Transnationality

02:00:00
Apache Flume, Flume Architecture, Features of Apache Flume

02:00:00
Need for Apache Flume, Transactions & Reliability

02:30:00
Source, Sink, Channel, HDFS Sink

01:00:00
Partitioning & Interceptors, File Formats

01:00:00
Integrating Flume with Applications

01:00:00

About Course

What Will You Learn?

Course Content

Module 1: Understanding Big Data

Different types of data

Different V’s of Big Data

Importance of Big Data

Understand RDBMS and its limitations for Big Data

Casestudy: Netflix

Casestudy: Big data in banking domain

Casestudy: Big data in e-commerce domain

Module 2: Introduction to Hadoop

Introduction to Apache Hadoop

Apache Hadoop Architecture

Apache Hadoop Daemons

Hadoop Distributed File System

Apache Hadoop Shell Commands

Apache Hadoop Configuration

Module 3: Storage Unit

Hadoop Distributed File System

Read Operation in HDFS

Write Operation in HDFS

Fault Tolerance in HDFS

Accessing HDFS using Java API

HDFS Daemons

Loading data on HDFS

Module 4: MapReduce and YARN

Yet Another Resource Negotiator (YARN)

YARN Architecture

Job Submission in YARN

MapReduce Architecture

MapReduce Programming in Eclipse

Module 5: Real time Big Data Processing

Real time big data streaming

Real time big data streaming tools

Industry use cases

Module 6: Apache Pig

Scalar, Complex, Load, Dump

Store, Foreach, Filter

Join, group, Order by

Distinct, Limit, Sample

Parallel, User Defined Function

Advanced Relational Operations

Co-group, Union, Cross

Program for Word Count Job

Module 7: Apache Hive

Features of Apache Hive

Command Line Interface, History of Apache Hive

Hive Data Types & Files Formats

Creating Managed Table, External Table, Partitioned Table

Dropping Tables, Alter Table, Inserting Data into Tables from Queries

SELECT from clauses, WHERE Clauses

GROUP BY Clauses, JOIN Statements, ORDER BY

SORT BY, DISTRIBUTE BY, CLUSTER BY Bucketing

UNION ALL, View, Hive Meta store

Module 8: Data Extraction Tool

Apache Sqoop, Sqoop Architecture, Need for Apache Sqoop

Sqoop Features Sqoop Connectors, Import Function, Incremental Import

Direct Mode Import, Performing Export Function

Exports, and Transnationality

Apache Flume, Flume Architecture, Features of Apache Flume

Need for Apache Flume, Transactions & Reliability

Source, Sink, Channel, HDFS Sink

Partitioning & Interceptors, File Formats

Integrating Flume with Applications

Module 9: Capstone Project Work

A Bigdata Project based on a case study