Cloud-based

Data Analytics
Lecture 8

1
 Azure SQL

Azure NoSQL Stores and databases

Objectives
Use cases

Business Benefits
Azure SQL
Azure Data platform
Hybrid Cloud Off premises
On premises
Shared
Lower cost

Dedicated
Higher cost
Higher administration Lower administration
5
SQL Server on Azure virtual machines

6
Hybrid Deployment

7
Business Benefits of
Azure SQL VM
• Running SQL Server on virtual
machines allows you to meet unique
and diverse business needs through
a combination of on-premises and
cloud-hosted deployments, while
using the same set of server
products, development tools, and
expertise across these
environments.

8
Azure SQL Databases
Azure SQL DB
• Azure SQL Database is a PaaS
offering from Microsoft. You
create a managed database server
in the cloud, and then deploy your
databases on this server.
• A SQL Database server is a logical
construct that acts as a central
administrative point for multiple
single or pooled databases, logins,
firewall rules, auditing rules,
threat detection policies, and
failover groups.
10
Azure SQL DB
Elastic Pool
• In Elastic pool, multiple
databases can share the
same resources, such as
memory, data storage space,
and processing power
through multiple-tenancy.
• Elastic Pool enables you to
use the resources available in
the pool, and then release
the resources once
processing has completed.
11
Use cases
• Azure SQL Database is often used for:
• Modern cloud applications that need
to use the latest stable SQL Server
features.
• Applications that require high
availability.
• Systems with a variable load, that need
the database server to scale up and
down quickly.
12
Business Benefits of Azure SQL Database

Provides scalability
Azure SQL Database
features so that you can
automatically updates Provides high
increase the resources
and patches the SQL availability guarantees
available to store and
Server software
process data

Auditing tracks
Offers advanced threat
database events and Helps secure your data
protection provides
writes them to an audit by providing encryption
advanced security
log in your Azure
capabilities
storage account

13
 Azure SQL Database
The developer’s intelligent cloud-database
service

•Built for application developers

•Let’s you focus on your business application
•Accelerates your time to market
•Built-in advisors learn your app’s unique
characteristics; adapts to maximize performance,
reliability, and data protection
•Helps you build secure apps and connect
to your database by supporting the languages and
platforms that you prefer
Learns and Intelligent capabilities

adapts • Suggests actions for how to optimize your

database performance
• Automatically implements solutions that
adapt to the app’s needs
• Ultimately gives you time back to focus on
your business
 Intelligent capabilities
• Built-in performance monitoring with
intelligent advisors helps reduce
troubleshooting time
• The Azure management portal
exposes real-time metrics
• Azure SQL Database includes Intelligent
advisors:
o Database Advisor
o Query Performance Insight
o Query Store
Database Advisor
Improve the database with recommendations

17
 Query Performance Insight

See how database

resources are being
consumed
Query Store
Comprehensive query-performance information when you need it most

Workload-data recorder for

Query Store
your database:
• Queries, plans, and compilation and runtime

?
statistics available at your fingertips
• Allows you to easily identify and fix
performance issues in the minutes
Queries SQL Database
Enables the following scenarios:
• Finding regressed queries
• Identifying top resource consuming queries
• Ad-hoc workload optimization
• Smooth application upgrades
Deeper Insight
Protects and secures your app data

• Built-in protection and security

• Meets stringent regulatory-
compliance requirements
• Minimal custom coding
• Advanced encryption technologies
• Powerful business-continuity features
Most secure database

Most secure database

Surrounded by layers of protection
• Windows authentication
NEWë
• Azure Active Directory auth.

Identity

NEWë
• SQL threat analytics
• SQL auditing
Monitor activity

NEWë
• Row-level security
NEWë
• Dynamic data masking
Control access
NEWë
• Always encrypted
• Transparent data encryption
Protect data
• Secure development lifecycle
• Least vulnerable last 6 years
Secure Code
Azure Active Directory authentication
Manage user identities in one location.
Use Azure Active Directory user identities
and groups to enable access to Azure SQL Customer 1

Database and other Microsoft services. Customer 2

Customer 3

Azure SQL Database

Benefits include:
•Limit proliferation of user identities
•Allow password rotation in one place
•Eliminate password storing
Row-level security
Protect data privacy by ensuring the right access across rows

Give users access only the rows

applicable to their role
Customer 1
Simplify the design and coding of Customer 2
security in your apps Customer 3

SQL Database
Administer with SQL Server Management
Studio or SQL Server Data Tools
 Dynamic data masking
Limit the exposure of sensitive data by hiding it from users
Table.PhoneNo

• Auto-discovery of potentially +1-313-555-5796

+972-4-777-1978
sensitive data to mask +1-248-666-6550

• Configurable masking policy

from the Azure portal or via DDL On-the-fly masking of sensitive
in the server Azure SQL
Database data in query results

• On-the-fly obfuscation of data in PhoneNum

query results XXX-XXX-5796

• Flexibility to define a set of

XXX-XXX-1978

privileged users for un-masked

data access
 Azure SQL Database encryption: overview
Encryption type Type Customer value

Encryption-in-transit Transport Layer Security (TLS) from the Protects data between the client and the server against snooping and
client to the server man-in-the-middle attacks. Azure SQL Database is phasing out Secure Sockets Layer
(SSL) 3.0 and TLS 1.0 in favor of TLS 1.2.

Encryption-at-rest Transparent Data Encryption (TDE) for Protects data on the disk. Key management is done by Azure, which makes it easier to
Azure SQL Database obtain compliance.

Encryption-end-to-end Always Encrypted for client-side Data is protected end-to-end, but the application is aware of encrypted columns. This is
column encryption used in the absence of data masking and TDE for compliance-related scenarios.

In-transit At-rest
End-to-end
Customer data Database files,
backups, Tx log,
TempDB
 Overall Azure SQL Database Benefits

Increased productivity Faster time to market Lower TCO Reduced risks

47% staff hours 75% faster 53% less 71% fewer cases
reclaimed for other app deployment cycles expensive than on- of unplanned
tasks DB management hours prem/hosted downtime
Other

Azure SQL
Database

“Now, those people can do “We can get things out faster with “To be able to do what we’re “The last time we had downtime, a
development and create more Azure SQL Database” doing in Azure, we’d need an half a day probably lost us $100k”
revenue opportunities for us.” investment of millions.”
27
Use cases for
Managed Instance
• Consider Azure SQL Database managed
instance if you want to lift-and-shift an
on-premises SQL Server instance and all
its databases to the cloud, without
incurring the management overhead of
running SQL Server on a virtual
machine.
28
 Managed instance deployment enables a system
administrator to spend less time on administrative tasks.

Business Automated tasks include operating system and database

benefits of management system software installation and patching,
dynamic instance resizing and configuration, backups,

Managed database replication (including system databases), high

availability configuration, and configuration of health and
performance monitoring data streams.
Instance
Managed instance has near 100% compatibility with SQL
Server Enterprise Edition, running on-premises.

29
PostgreSQL, MariaDB, and MySQL

30
 • You get the following features with Azure
Database for MySQL:
• High availability features built-in.
Benefits of • Predictable performance.
• Easy scaling that responds quickly to
Azure demand.

Database for • Secure data, both at rest and in

motion.
MySQL • Automatic backups and point-in-time
restore for the last 35 days.
• Enterprise-level security and
compliance with legislation.

31
 • Azure Database for MariaDB delivers:
• Built-in high availability with no
additional cost.
• Predictable performance, using
Benefits of Azure inclusive pay-as-you-go pricing.
• Scaling as needed within seconds.
Database for • Secured protection of sensitive data
MariaDB at rest and in motion.
• Automatic backups and point-in-
time-restore for up to 35 days.
• Enterprise-grade security and
compliance.

32
 • Azure Database for PostgreSQL has two
deployment options: Single-server and
Hyperscale.
• The single-server deployment option for

Benefits of PostgreSQL provides similar benefits as

Azure Database for MySQL.

Azure
• Hyperscale (Citus) is a deployment
option that scales queries across
multiple server nodes to support large
33
Database for database loads.
• Azure Database for PostgreSQL is a highly

PostgreSQL
available service.
• It contains built-in failure detection and
failover mechanisms.
• Users of PostgreSQL will be familiar with
the pgAdmin tool, which you can use to
manage and monitor a PostgreSQL database.
You can continue to use this tool to connect
to Azure Database for PostgreSQL.
Migrate data
to Azure

34
 NoSQL Azure Data
Stores

35
Big Data File Formats
Introducing Not Only SQL: No

Key/Value Column

Document Graph
Key-value store

• A query specifies the keys to identify the items to be retrieved. You can't search
on values. An application that retrieves data from a key-value store is responsible
for parsing the contents of the values returned.
Column family database

The most widely used column family database management

system is Apache Cassandra. Azure Cosmos DB supports the
column-family approach through the Cassandra API.
Document database
• Document databases typically store data in JSON
format, or they could be encoded using other formats
such as XML, YAML, JSON, BSON.

• Typically, a document contains the entire data for an

entity.

• A document store does not require that all

documents have the same structure. This free-form
approach provides a great deal of flexibility.

• Applications can store different data in documents as

business requirements change.
Graph
Advantages

Flexible Indexing
Independent
Schema Support

Easy
JSON Ready Fast
Manipulation

Large
Version Proof ORM Free
Datasets
Azure Storage
Azure Table storage

44
Partitioning

45
Partitioning

46
 • The primary advantages of using Azure Table Storage
tables over other ways of storing data include:
Use cases • It's simpler to scale. It takes the same time to
insert data in an empty table, or a table with
and Benefits billions of entries. An Azure storage account can
hold up to 500 TB of data.
of using • A table can hold semi-structured data
• There's no need to map and maintain the
Azure Table complex relationships typically required by a
normalized relational database.
Storage • Row insertion is fast
• Data retrieval is fast, if you specify the partition
and row keys as query criteria

47
Azure Blob storage
• Azure currently supports three different types of blob:
• Block blobs. A block blob is handled as a set of blocks.
Each block can vary in size, up to 100 MB. A block
blob can contain up to 50,000 blocks, giving a
maximum size of over 4.7 TB.
• Page blobs. A page blob is organized as a collection of
fixed size 512-byte pages. A page blob is optimized to
support random read and write operations; you can
fetch and store data for a single page if necessary. A
page blob can hold up to 8 TB of data.
• Append blobs. An append blob is a block blob
optimized to support append operations. You can
only add blocks to the end of an append blob;
updating or deleting existing blocks isn't supported.48
Azure Blob storage

49
 Common uses of Azure Blob Storage include:
• Serving images or documents directly to a
Use cases browser, in the form of a static website.
and Benefits Visit Static website hosting in Azure
storage for detailed information.
of using • Storing files for distributed access
Azure Blob • Streaming video and audio
• Storing data for backup and restore,
Storage disaster recovery, and archiving
• Storing data for analysis by an on-
premises or Azure-hosted service

50
 Azure Cosmos DB

51
 Azure Cosmos DB
A globally distributed, massively scalable, multi-model database service

Cosmos DB’s API for

MongoDB
Table API

Column-family
Document

Key-value Graph

Guaranteed low latency at the 99th percentile

Elastic scale out
of storage & throughput Five well-defined consistency models

Turnkey global distribution Comprehensive SLAs

Azure Cosmos DB

53
Azure
Cosmos DB

54
Use cases and Benefits of using Azure Cosmos
DB

Retail and marketing. Microsoft

IoT and telematics. These systems uses CosmosDB for its own e-
typically ingest large amounts of commerce platforms that run as
data in frequent bursts of activity. part of Windows Store and Xbox
Live.

Web and mobile applications.

Azure Cosmos DB is commonly
used within web and mobile
Gaming. The database tier is a
applications, and is well suited for
crucial component of gaming
modeling social interactions,
applications.
integrating with third-party
services, and for building rich
personalized experiences.
55
Resource Model
Account

Database

Container

Item
 ********.azure.com
Account

Database IGeAvVUp …

Container

Item

What we have here is a database account, has a URI and a set

of keys which you can then connect to an application to your
cosmos db
Account

Database

Container

Item

You can create one or multiple Azure Cosmos databases

under your account.
Account

Database

Container

Item
Account

Database

Container

Item
Account

Database

Container

Item
Account

Database

Container

Item
Account

Database

Container

Item
Account

Database

Container

Item
Account

Database

Container

Item
Account

Database

Container

Item
Account

Database

Container

Item
Account

Database

Container User

Item Permission
Account

Database

Container = Collection Graph Table

Item
Account

Database

Container

Item
Account

Database

Container

Item
Account

Database

Container

Item
Account

Database

Container

Item
Account

Database

Container
Collection Graph Table

Item
Document Vertices/Edges Row
 Account

Database

Container

Item Sproc Trigger UDF

You can operate underlying items; these are the records stored in your database or rather in the container.
 Account

Database

Container

Item Sproc Trigger UDF Conflict

The container also exposes a construct known as conflict-conflicts are used to resolve conflicts. These are going to be
exceptionally rare, however they are means for recovering records.
Well-defined consistency models
• Intuitive programming model
• 5 Well-defined, consistency models
• Overridable on a per-request basis

• Clear tradeoffs
• Latency
• Availability
• Throughput
Consistency Level Guarantees

Strong Linearizability (once operation is complete, it will be visible to all)

Bounded Staleness Consistent Prefix.

Reads lag behind writes by at most k prefixes or t interval
Similar properties to strong consistency (except within staleness window), while
preserving 99.99% availability and low latency.

Session Consistent Prefix.

Within a session: monotonic reads, monotonic writes, read-your-writes, write-follows-
reads
Predictable consistency for a session, high read throughput + low latency

Consistent Prefix Reads will never see out of order writes (no gaps).

Eventual Potential for out of order reads. Lowest cost for reads of all consistency levels.
Bounded-Staleness: Bounds are set server-side via the Azure Portal
Setting the Consistency Level in your
Cosmos DB account

81
References

• https://docs.microsoft.com/en-
us/azure/azure-sql/database/sql-database-
paas-overview
• https://docs.microsoft.com/en-
us/azure/cosmos-db/introduction
• https://docs.microsoft.com/en-
us/azure/storage/blobs/storage-blobs-
overview

82