UNIT – II Introduction to Artificial Intelligence and Data Science
UNIT -II
Data Science
Data Science is a multidisciplinary field that combines various techniques, algorithms, processes, and
systems to extract insights and knowledge from structured and unstructured data. It involves using
scientific methods, algorithms, and systems to analyze large volumes of data and to uncover hidden
patterns, correlations, and trends that are valuable for decision-making.
Data science has gained significant importance in recent years due to the increasing volume of data
generated by businesses, governments, and individuals, as well as the advances in computing power
and storage technologies.
Tools and Technologies in Data Science
1. Programming Languages:
o Python: One of the most popular languages for data science due to its simplicity and
a vast array of libraries such as Pandas, NumPy, SciPy, and Scikit-learn.
o R: A language and environment specifically designed for statistics and data analysis,
with numerous packages for statistical computing and visualization.
o SQL: A language for managing and querying relational databases, which is essential
for retrieving and manipulating data.
o Java and Scala: Commonly used for big data processing frameworks like Apache
Hadoop and Apache Spark.
2. Data Science Libraries:
o Pandas: A Python library for data manipulation and analysis, providing data
structures like DataFrames.
o NumPy: A library for numerical computations in Python, used for handling arrays
and performing mathematical operations.
o Matplotlib and Seaborn: Libraries for creating static, animated, and interactive
visualizations in Python.
o TensorFlow and Keras: Frameworks for building and training deep learning models.
3. Data Visualization Tools:
o Tableau: A powerful tool for creating interactive and shareable dashboards.
o Power BI: A Microsoft tool for business analytics, enabling users to visualize data
and share insights.
o [Link]: A JavaScript library for creating interactive data visualizations on the web.
pg. 1 drajaydutta13@[Link]
UNIT – II Introduction to Artificial Intelligence and Data Science
4. Cloud Platforms:
o Google Cloud, Amazon Web Services (AWS), and Microsoft Azure provide scalable
infrastructure for storing and processing big data, as well as offering machine
learning tools and services.
Types of Data
Data can be categorized based on its structure, nature, and usage. Understanding the types of data
is essential for data analysis, processing, and deriving insights. Below are the primary classifications
of data:
1. Structured Data
• Definition: Structured data refers to data that is highly organized and formatted in a way
that is easy to process using traditional tools such as databases or spreadsheets. It is
typically stored in a tabular format (rows and columns).
• Characteristics: It has a fixed schema with clearly defined fields and types. Structured data is
often stored in relational databases or data warehouses and can be easily queried and
analyzed.
• Examples:
o Customer information (name, address, phone number) stored in a database.
o Sales data (transaction amount, date, product details) in a table.
o Employee data (employee ID, department, salary) in an HR system.
2. Unstructured Data
• Definition: Unstructured data refers to data that does not have a predefined structure or
format. This data type is often textual or multimedia and does not fit neatly into rows and
columns.
• Characteristics: Unstructured data is more difficult to analyze because it lacks a clear format
or organization. Advanced tools like Natural Language Processing (NLP), image recognition,
and machine learning are often used to extract meaning from unstructured data.
• Examples:
o Text data such as emails, documents, and social media posts.
o Multimedia content like images, audio files, and videos.
pg. 2 drajaydutta13@[Link]
UNIT – II Introduction to Artificial Intelligence and Data Science
o Web content like blogs, forums, and reviews.
3. Semi-Structured Data
• Definition: Semi-structured data is a mix between structured and unstructured data. It does
not have a fixed schema, but it contains tags or markers that make it easier to organize and
analyze than completely unstructured data.
• Characteristics: It often uses formats like XML, JSON, or YAML, where data elements are
stored with labels or keys that make it more interpretable than unstructured data but not as
rigid as structured data.
• Examples:
o XML files or JSON documents used for data exchange between applications.
o Log files generated by web servers or applications.
o Emails with subject lines, dates, and content but no strict structure.
4. Time-Series Data
• Definition: Time-series data is data that is collected and indexed in chronological order. This
type of data typically involves observations recorded at regular intervals over time.
• Characteristics: Time-series data is used for trend analysis, forecasting, and anomaly
detection. It allows for tracking changes over time and making predictions based on
historical patterns.
• Examples:
o Stock market prices recorded every minute, hour, or day.
o Temperature readings taken every hour.
o Website traffic or user engagement data collected over days or months.
5. Categorical Data
• Definition: Categorical data refers to data that can be divided into specific groups or
categories. Each category represents a distinct label or value, and the values cannot be
mathematically quantified.
• Characteristics: Categorical data is often used in classification tasks where different groups
need to be identified and analyzed. Categorical data can be further classified into nominal
(no inherent order) and ordinal (ordered categories) types.
pg. 3 drajaydutta13@[Link]
UNIT – II Introduction to Artificial Intelligence and Data Science
• Examples:
o Gender (Male/Female/Other) – Nominal.
o Product categories (Electronics, Clothing, Home goods) – Nominal.
o Education level (High School, Bachelor’s, Master’s, Ph.D.) – Ordinal.
Key Components of Data Science
1. Data Collection
o Data science begins with data collection, which can come from a variety of sources,
such as sensors, databases, online transactions, social media, and IoT devices. The
data can be structured (e.g., databases) or unstructured (e.g., text, images, videos).
2. Data Cleaning
o Raw data is often incomplete, inconsistent, or erroneous, so cleaning the data is a
crucial step. This process involves handling missing values, removing duplicates, and
correcting errors to ensure the data is accurate and reliable.
3. Exploratory Data Analysis (EDA)
o EDA is the process of analyzing the data visually and statistically to understand its
structure and patterns. Common techniques include plotting histograms, scatter
plots, and box plots, as well as calculating summary statistics like mean, median, and
standard deviation.
4. Data Modeling
o Once the data is cleaned and explored, data scientists use statistical models and
machine learning algorithms to create models that can predict outcomes or identify
patterns in the data. Common techniques include regression, classification,
clustering, and time series forecasting.
5. Data Visualization
o Data visualization involves creating graphical representations of the data to help
communicate findings clearly and effectively. Visualizations can include bar charts,
line graphs, pie charts, heat maps, and interactive dashboards.
6. Interpretation and Decision-Making
o The ultimate goal of data science is to use the insights gained from the data to make
informed decisions. Data scientists work with stakeholders to translate data insights
pg. 4 drajaydutta13@[Link]
UNIT – II Introduction to Artificial Intelligence and Data Science
into actionable recommendations, helping organizations make data-driven
decisions.
Applications of Data Science
1. Healthcare
o Data science is used to analyze patient data, predict disease outbreaks, personalize
treatment plans, and optimize hospital operations. Machine learning models can
help in early diagnosis (e.g., cancer detection from medical imaging).
2. Finance
o In the financial sector, data science is applied to fraud detection, algorithmic trading,
credit scoring, and risk management. Predictive models can help assess stock market
trends and predict future asset values.
3. Retail
o Retailers use data science for demand forecasting, inventory management,
customer segmentation, and recommendation systems. E-commerce platforms like
Amazon and Netflix use recommendation algorithms to suggest products based on
user behavior.
4. Marketing
o Data science helps in customer segmentation, sentiment analysis, and targeted
advertising. It is used to analyze customer behavior, optimize marketing campaigns,
and improve customer experience.
5. Transportation
o Data science optimizes route planning, traffic management, and vehicle
maintenance. Companies like Uber and Lyft use data science for dynamic pricing,
route optimization, and demand prediction.
6. Sports
o Data science in sports involves performance analysis, player scouting, and injury
prediction. Machine learning algorithms are used to analyze player statistics and
optimize team strategies.
pg. 5 drajaydutta13@[Link]
UNIT – II Introduction to Artificial Intelligence and Data Science
Careers in Data Science
Data Science is a rapidly growing field that combines statistics, computer science, and domain
knowledge to extract insights and make data-driven decisions. As organizations increasingly rely on
data to drive decisions and innovation, a variety of career opportunities in Data Science have
emerged. Below are some of the key career paths within the field:
1. Data Scientist
• Role: Data scientists are responsible for analyzing complex data to uncover trends, patterns,
and insights that can be used for decision-making. They use statistical methods, machine
learning algorithms, and programming skills to analyze large datasets and create predictive
models.
• Skills Required:
o Proficiency in programming languages like Python, R, and SQL.
o Strong statistical and mathematical knowledge.
o Expertise in machine learning, data visualization, and big data technologies.
o Experience with tools like Hadoop, Spark, and TensorFlow.
• Typical Responsibilities:
o Developing data models and algorithms.
o Cleaning, processing, and analyzing large datasets.
o Visualizing data insights for business stakeholders.
o Conducting research to enhance data science techniques.
2. Data Analyst
• Role: Data analysts focus on interpreting data and turning it into actionable insights. They
are typically involved in data cleaning, data visualization, and generating reports that help
businesses make informed decisions.
• Skills Required:
pg. 6 drajaydutta13@[Link]
UNIT – II Introduction to Artificial Intelligence and Data Science
o Strong command over Excel, SQL, and other data analysis tools (e.g., Tableau, Power
BI).
o Good understanding of statistics and data manipulation.
o Ability to create compelling data visualizations and reports.
• Typical Responsibilities:
o Analyzing data sets to identify trends and patterns.
o Preparing reports and dashboards for business stakeholders.
o Performing exploratory data analysis (EDA).
o Assisting with decision-making through data insights.
3. Machine Learning Engineer
• Role: Machine learning engineers design, build, and deploy machine learning models. They
work closely with data scientists to put predictive models into production and ensure that
they scale effectively.
• Skills Required:
o Proficiency in machine learning algorithms and frameworks (e.g., Scikit-learn,
TensorFlow, Keras, PyTorch).
o Expertise in programming languages like Python, Java, and C++.
o Knowledge of cloud platforms and big data tools (e.g., AWS, Azure, Hadoop).
• Typical Responsibilities:
o Building and optimizing machine learning models for deployment.
o Ensuring models are scalable and efficient.
o Collaborating with data scientists to implement algorithms in production.
o Monitoring model performance and retraining when necessary.
4. Data Engineer
• Role: Data engineers are responsible for designing, building, and maintaining the
infrastructure that allows for the collection, storage, and processing of large datasets. They
focus on building data pipelines that ensure clean, reliable, and accessible data for analysis.
• Skills Required:
pg. 7 drajaydutta13@[Link]
UNIT – II Introduction to Artificial Intelligence and Data Science
o Expertise in SQL and NoSQL databases (e.g., MySQL, MongoDB, Cassandra).
o Proficiency in programming languages such as Python, Java, and Scala.
o Experience with cloud platforms (AWS, Google Cloud, Azure).
o Familiarity with tools like Apache Hadoop, Kafka, and Spark.
• Typical Responsibilities:
o Building and maintaining scalable data architectures.
o Ensuring efficient data processing and ETL pipelines.
o Collaborating with data scientists and analysts to ensure data accessibility.
o Optimizing data storage and retrieval systems.
5. Business Intelligence (BI) Analyst
• Role: BI analysts focus on interpreting complex business data to provide actionable insights
for decision-making. They often work with data visualization tools and reporting platforms
to create reports and dashboards that track key business metrics.
• Skills Required:
o Strong skills in data visualization tools (e.g., Tableau, Power BI).
o Proficiency in SQL and database management.
o Knowledge of business processes and KPIs.
o Ability to communicate insights to non-technical stakeholders.
• Typical Responsibilities:
o Analyzing business data to identify opportunities for improvement.
o Developing and maintaining dashboards to track business performance.
o Providing recommendations based on data analysis to improve business strategies.
o Conducting regular reporting on key performance indicators (KPIs).
6. Data Architect
• Role: Data architects are responsible for designing the structure of data systems. They
create data models and define how data will be stored, accessed, and integrated across an
pg. 8 drajaydutta13@[Link]
UNIT – II Introduction to Artificial Intelligence and Data Science
organization. Their goal is to ensure the architecture supports both current and future data
needs.
• Skills Required:
o Expertise in database design, data modeling, and data management.
o Experience with cloud platforms and big data technologies.
o Knowledge of ETL processes and data warehousing.
o Strong programming and SQL skills.
• Typical Responsibilities:
o Designing and implementing data systems and architectures.
o Ensuring data quality, scalability, and security.
o Defining data governance policies and best practices.
o Collaborating with other teams to optimize data storage and access.
7. Data Visualization Specialist
• Role: Data visualization specialists focus on representing complex data in visually appealing
and easily understandable ways. They use charts, graphs, and interactive dashboards to
communicate insights to non-technical stakeholders.
• Skills Required:
o Proficiency in data visualization tools like Tableau, Power BI, or [Link].
o Strong graphic design skills and attention to detail.
o Ability to translate complex data into clear, understandable visuals.
• Typical Responsibilities:
o Creating visually engaging reports and dashboards.
o Ensuring that visualizations are aligned with business goals and user needs.
o Working with business stakeholders to understand the best way to present data.
o Maintaining and updating dashboards with new data.
pg. 9 drajaydutta13@[Link]
