BigData Engineer Complete Syllabus
HADOOP | AWS | AZURE | DATABRICKS | MACHINE LEARNING
Presented By
Contact us @ +91 9715 010 010
Our Policy
▪ No Fast Track (30% Theory + 70% Hands-on)
▪ 100% Refund if you are not satisfied
▪ Interactive and Concept basis
▪ Latest Version upgrades (Hadoop 3x & Spark 3x)
▪ Private WhatsApp Group for your queries
Session Details
▪ Course Duration : 60 Hours (~approx.)
▪ Mode of Training : Online
▪ Programming : Python 3x
▪ Session Timing : Weekend (Saturday and Sunday)
▪ Per Class : 3:00 Hours
Course Materials
▪ Practice Hadoop VM Cluster (with Latest Version)
▪ Recorded Videos
▪ Practice Materials
▪ Online eLibraries
▪ Sample Interview Questions with Answer
Course Overview
Topic 1 – Python3 Fundamentals Topic 8 – Airflow Scheduler
Topic 2 – Hadoop 3x Topic 9 – AWS (Amazon Web Services)
Topic 3 – Hive Query Language Topic 10 – Azure Services
Topic 4 – SQOOP (Recording) Topic 11 – Databricks Cluster
Topic 5 – Spark (RDD, DF, SQL & ML) Topic 12 – Statistics Fundamentals
Topic 6 – Kafka Streaming Topic 13 – Machine Learning
Topic 7 – HBase (NoSQL) Topic 14 – Certification & Interview Tips
Topic - 1
Python Fundamentals Hands-on
Data Types Collections
Python 3
Variables •Standard, Int, String, •List, Tuple,
Installation Float, Char, Boolean Dictionary, Set
Classes Functions Control
•Objects, Method, •Lambda, Built in Statements String Slicing
Inheritance Functions, UDF •if else, elif, for, while
NumPy Pandas
Topic - 2
Hadoop Introduction
Brief Hadoop VS
Why Bigdata Google History of
Introduction 4V’s of Bigdata Google
needed now.? Concepts Databases
to Big Data Architecture
History of
Hadoop 1x vs Hadoop Secondary
Hadoop Layers Hadoop & Name Node
2x vs 3x Daemons: - Name Node
Ecosystems
Resource High
Node Manager
Data Node Manager / Job Heart Beat Block Report Availability
/ Task Tracker
Tracker (HA)
Replication
Special File
versus Erasure Block size Input Split
format
Encoding
Topic - 2
BigData Hadoop & YARN (cont..)
MR1 VS MR2 Mapper Reducer Combiner
Hadoop
Application
Commands Container YARN
Master
Hands-on
Hadoop Job
Opportunities
Topic - 3
Hive Introduction
Introduction Hive Hive Meta Hive Server 1 Beeline VS
on Hive Architecture Store vs 2 Hive CLI
Create Table Create Table Create Table Managed VS Connect hive
Thrift Server
with ORC with Parquet using Avro External Table via Beeline
Dynamic
Partitioning Static Partition Bucketing SerDe Hive Joins
Partition
Sample Top 10 Hive
Complex Data
Performance
Project 1 Types (JSON)
Tuning
Topic - 5
Apache Spark Introduction
Introduction of Spark Spark RDD
RDD Actions
Spark Architecture Components Transformation
RDD to Spark DataFrame
DataFrameReader DataFrameWriter
DataFrame DataFrames Transformation
Create a Create a Create a Create a
DataFrame
DataFrame via DataFrame using DataFrame using DataFrame using
Actions
JDBC Parquet file ORC Avro
Create a Create a
Sample
DataFrame using DataFrame using
Project 2
Hive Tables JSON
Topic - 5
Apache Spark Introduction (cont..)
Storage Level Spark-SQL Spark-SQL Spark-Streaming
Spark Cache()
Persist introduction Transformations (DStreams)
Spark-Structured Stateful Stateless Spark – Kafka Spark ML
Streaming Transformation Transformation Integration Libraries
Spark Job Spark Job Spark Job Top 10 Sample
Submission via Submission via Submission via Performance
Local Client Mode Cluster Mode Tuning in Spark Project 3
Topic - 6
Kafka Introduction
What is Kafka API Kafka VS Kafka
Producer
Kafka.? Connector Flume Architecture
In Sync Kafka
Offset Consumer Broker
Replica Serialization
Kafka Topic Kafka - Spark Sample
Creation Integration Project 3
Topic - 7
HBase Introduction
Introduction
CAP Theorem HMaster HRegionServer
of HBase
Row Key Column Family WAL HQuarumpeer
Data Model Sample
Operations Project 4
Topic - 8
Airflow Introduction
What is How DAG
WebServer Scheduler Task Instances
Airflow.? working
Bit shift
upstream and Sensors Executors Data Profiling Adhoc Queries
Downstream
Sqoop Spark Submit Sample
Operators Hive Operator
Operator Operator Project 5
Topic - 9
Amazon Web Services
Simple Storage Serverless
Introduction of Data Analytics
Service (S3) computing
AWS Services
Creation (Lambda)
Elastic Elastic Cloud
AWS Redshift MapReduce AWS RDS Computing
(EMR) (EC2)
Sample
Project 6
Topic - 10
Azure Services
Introduction of Data Analytics Virtual Machine
Blob Storage
Azure services Services (VM)
Azure Data Lake Azure Data Lake SQL Databases HD Insight
Gen 2 Gen 1
Sample
Project 7
Topic - 11
Databricks
Create a
Introduction Integrate with
Automated
of Databricks Azure
Cluster
Integrate with Schedule a
DBFS Storages
AWS Spark Job
Create a
DBFS Magic Sample
Interactive
functions Project 8
Cluster
Topic - 12
Statistics Fundamentals
Why Statistics Uni-variate Bi-variate Descriptive
Types of Data
needed.? Analysis Analysis Statistics
Mean,
VAR, Std Dev Inferential
Median, Skewness Kurtosis
and IQR Statistics
Mode
Central Limit Probability Hypothesis Summarize
Correlation
Theorem Distribution Testing data
Topic - 13
Machine Learning
Introduction of Types of Machine
ML Jargons Data Preprocessing
Machine Learning Learning
Missing Values EDA Handling Uniform Scaling Overfitting
Project 9 Project 10
Underfitting Confusion Matrix Build a Model using Build a Model Using
Python Libraries Spark ML Libraries
❑ Tips and Tricks for Cloudera Certification for Spark and Hadoop
Developer (CCA 175).
❑ Bigdata Interview related tips and tricks and Bigdata Interview
question with answers.
Please feel free to reach us If you have any
queries…
+91 9715 010 010
