Part I

What are Databases?

 What are Databases?

What are Databases?

1 Overview & Motivation

2 Architectures

3 Areas of Application

4 History

Saake Database Concepts Last Edited: April 2019 1–1

 What are Databases?

Educational Objective for Today . . .

Motivation for using database systems

Knowledge of basic architectures

Saake Database Concepts Last Edited: April 2019 1–2

 What are Databases? Overview & Motivation

What are Databases?

Data = logically grouped pieces of information
Base1
: the bottom or lowest part of something : the part on which
something rests or is supported
: something (such as a group of people or things) that provides
support for a place, business, etc.

Base of a column in architecture

1
http://www.merriam-webster.com/dictionary/base
Saake Database Concepts Last Edited: April 2019 1–3
 What are Databases? Overview & Motivation

Areas of Application

Saake Database Concepts Last Edited: April 2019 1–4

 What are Databases? Overview & Motivation

How is Data Managed?

Without databases
Each application manages its own data
Data is stored multiple times redundancy
Problems
I Waste of storage space
I “Forgetting” of changes
I No centralized, “standardized” data management

Saake Database Concepts Last Edited: April 2019 1–5

 What are Databases? Overview & Motivation

Problems of Data Redundancy

Other software systems cannot process large amounts of data
efficiently
Many users or applications cannot access the same data in
parallel without interfering with each other
Application developers / users cannot develop / use applications
without knowing
I internal representation of data
I storage media or computers
(no data independence)
No data security; potential loss of data

Saake Database Concepts Last Edited: April 2019 1–6

 What are Databases? Overview & Motivation

Idea: Data Integration Using Database Systems

Application ... Application

Database Management System =

DBMS Software for Managing Databases

DBS =
Database System

structured data, which is

managed by DBMS
Database

Saake Database Concepts Last Edited: April 2019 1–7

 What are Databases? Overview & Motivation

Motivation
Database systems
are center piece of
modern IT
systems

. . . ubiquitous

Database
specialists are in
high demand

Saake Database Concepts Last Edited: April 2019 1–8

 What are Databases? Overview & Motivation

Questions
1 How to organize (model and use) data?
2 How to store data safely and persistently?
3 How to process huge amounts of data ( terabytes) efficiently?
4 How can many users ( 10,000) access data concurrently?

Saake Database Concepts Last Edited: April 2019 1–9

 What are Databases? Architectures

Principles: Codd’s Nine Rules

1 Integration: uniform, non-redundant data management
2 Operations: insert, query, update, delete
3 Catalog: access to the database description in a data dictionary
4 User views: different users/applications must be able to have a
different perception of the data
5 Integrity: ensure conformity of database contents with real world
6 Security: prevention of unauthorized access
7 Transactions: multiple DB operations handled as an atomic unit
8 Synchronisation: coordination of concurrent transactions
9 Recovery: data backup and recovery after system errors

Saake Database Concepts Last Edited: April 2019 1–10

 What are Databases? Architectures

Data Independence and Schemata

Based on coarse DBMS architecture
Decouple user and implementation view
Goals include:
I Separate modeling view from internal storage
I Portability
I Simplify tuning
I Standardized interfaces

Saake Database Concepts Last Edited: April 2019 1–11

 What are Databases? Architectures

Schema Architecture

External External
Schema 1
... Schema N

Query Processing

Data Representation
Conceptual
Schema

Internal
Schema

Saake Database Concepts Last Edited: April 2019 1–12

 What are Databases? Architectures

Schema Architecture /2
Connection between
I Conceptual schema (result of data definition)
I Internal schema (definition of file structure and access paths)
I External schemata (result of view definition)
I Application programs (result of application programming)

Saake Database Concepts Last Edited: April 2019 1–13

 What are Databases? Architectures

Schema Architecture /3
Separation schema — instance
I Schema (metadata, data description)
I Instance (user data, database state or shape)
Database schema consists of
I Internal, conceptual, external schemata and application programs
Conceptual schema contains, e.g.:
I Structure descriptions
I Integrity descriptions
I Authorization rules (DB accesses that a user may perform)

Saake Database Concepts Last Edited: April 2019 1–14

 What are Databases? Architectures

Data Independence /2
Stability of user interface with respect to changes
Physical: Changes to file structure or access paths do not
influence the conceptual schema
Logical: Changes to conceptual schema and certain external
schemata do not influence other external schemata or application
programs

Saake Database Concepts Last Edited: April 2019 1–15

 What are Databases? Architectures

Data Independence /3
Potential impact of changes to the conceptual schema:
I external schemata may be affected (changing attributes)
I application programs may be affected (recompilation of application
programs, adaptions may be necessary)
But: necessary changes are recognized and monitored by the
DBMS

Saake Database Concepts Last Edited: April 2019 1–16

 What are Databases? Architectures

Application Example: Music Store

Title

Musician
Year

Price
Critique(s)

Tracks

Saake Database Concepts Last Edited: April 2019 1–17

 What are Databases? Architectures

Layer Architecture by Example

Conceptual view: presentation in tables (relations)
Artist MNr Name Country
103 Apocalyptica Finland
104 Subway To Sally Germany
105 Rammstein Germany

Album ANr Title Year Genre MNr ! Artist

1014 Amplified 2006 Rock 103
1015 Nord Nord Ost 2005 Rock 104
1016 Rosenrot 2005 Rock 105
1021 Engelskrieger 2003 Rock 104
1025 Reflections 2006 Rock 103

Saake Database Concepts Last Edited: April 2019 1–18

 What are Databases? Architectures

Layer Architecture by Example /2

External view: data in a flat relation

ANr Title Year Genre Artist

1014 Amplified 2006 Rock Apocalyptica
1015 Nord Nord Ost 2005 Rock Subway To Sally
1016 Rosenrot 2005 Rock Rammstein
1021 Engelskrieger 2003 Rock Subway To Sally
1025 Reflections 2006 Rock Apocalyptica

Saake Database Concepts Last Edited: April 2019 1–19

 What are Databases? Architectures

Layer Architecture by Example /3

External view: data in a hierarchically structured relation

Artist Album
Title Year Genre
Apocalyptica Amplified 2006 Rock
Reflections 2003 Rock
Subway To Sally Nord Nord Ost 2005 Metal
Engelskrieger 2003 Rock
Rammstein Rosenrot 2005 Rock

Saake Database Concepts Last Edited: April 2019 1–20

 What are Databases? Architectures

Layer Architecture by Example /4

Internal presentation
Tree-like Access
on
Album Number
1000 1500 2000

Partial
Storage
of Data Sets
1014 Amplified 2006
in a Tree

1015 Nord Nord Ost 2005

Space for
Data Sets
Overflow
19.99 Rock 103 ....
15.99 Rock 104 ....

Saake Database Concepts Last Edited: April 2019 1–21

 What are Databases? Architectures

System Architectures
Description of components of a database system
Standardized interfaces between components

Architecture proposals
I ANSI-SPARC architecture
three layer architecture
I Five layer architecture
describes transformation components in detail
Lecture “Database Implementation Techniques”

Saake Database Concepts Last Edited: April 2019 1–22

 What are Databases? Architectures

ANSI-SPARC Architecture
ANSI: American National Standards Institute
SPARC: Standards Planning and Requirement Committee
Proposal from 1978
Basically refines coarse architecture
I Internal layer / operation system refined
I Multiple interactive and programming components
I Interfaces named and standardized

Saake Database Concepts Last Edited: April 2019 1–23

 What are Databases? Architectures

ANSI-SPARC Architecture /2
External Layer Conceptual Layer Internal Layer

Query
Optimizer Mapping Disk Access
Updates

DB- Data
P1
Operations Dictionary

Embedding
...

View File
Pn GUIs
Definition Organisation
Data Definition

Saake Database Concepts Last Edited: April 2019 1–24

 What are Databases? Architectures

Classification of Components
Definition components: data definition, file organization, view
definition
Programming components: DB programming with embedded DB
operations
User components: interactive application programs, query and
update
Transformation components: optimizer, analysis, disk access
Data dictionary: contains data from definition components,
provides other components with this information

Saake Database Concepts Last Edited: April 2019 1–25

 What are Databases? Architectures

Five Layer Architecture

Refinement of transformation steps
Set-Oriented
SOI
Interface
Translation, Access Path Selection,
Data System
Access Control, Integrity Control
Record-Oriented
ROI
Interface Data Dictionary, Currency Pointer,
Access System Sorting, Concurrency control
Internal-Record
IRI Interface
Record Manager, Access Path Management,
Storage System Lock Management, Logging, Recovery
System Buffer
SBI Interface
System Buffer Management,
Buffer Manager
Page Replacement
File
FI
Interface
External Storage Management
Operating System

DI Device
Interface
External Storage
Saake Database Concepts Last Edited: April 2019 1–26
 What are Databases? Architectures

Application Architectures
Architecture of database applications typically based on
client-server model: server ⌘ database system
Client Server
(Customer) (Service Provider)

1. Request

2. Processing

3. Answer

Saake Database Concepts Last Edited: April 2019 1–27

 What are Databases? Architectures

Application Architectures /2
Separation of functionality of an application
I Presentation and user interaction
I Application logic (“business logic”)
I Data management functionality (store, query, . . . ).

User Interface Client

User Interface

Application Logic Client

Application Logic Application-
DB-Interface
Server
DB-Interface

DB-Server
DB-Server

Two Layer Architecture

Three Layer Architecture
Saake Database Concepts Last Edited: April 2019 1–28
 What are Databases? Areas of Application

Some concrete systems

(Object-)relational DBMS
I Oracle11g, IBM DB2 V.10, Microsoft SQL Server 2012, SAP HANA
I MySQL (www.mysql.org), PostgreSQL (www.postgresql.org)
Pseudo DBMS
I MS Access
NoSQL systems
I Graph database systems (InfiniteGraph, neo4j), document
databases (MongoDB), key-value stores, ....

Saake Database Concepts Last Edited: April 2019 1–29

 What are Databases? Areas of Application

Areas of Application
Classical areas of application:
I Many objects (15,000 books, 300 users, 100 books borrowed per
week, . . . )
I Few object types (BOOK, USER, BORROWING)
I For instance, book keeping systems, order tracking systems, library
systems, . . .
Current applications:
I E-Commerce, decision supporting systems (data warehouses,
OLAP), NASA’s Earth Observation System (petabyte databases),
data mining

Saake Database Concepts Last Edited: April 2019 1–30

 What are Databases? Areas of Application

Database Sizes

eBay Data Warehouse 10 PB (⇡ 10 · 1015 bytes)

Teradata DBMS, 72 nodes, 10,000 users,
millions of queries/day
WalMart Data Warehouse 2,5 PB
Teradata DBMS, NCR MPP hardware;
product information (sales etc.) of 2,900 stores;
50,000 queries/week
Facebook 400 TB
x.000 MySQL server
Hadoop/Hive, 610 nodes, 15 TB/day
US Library of Congress 10-20 TB
not digitized

PB for Petabyte is in the order of 1015

Saake Database Concepts Last Edited: April 2019 1–31

 What are Databases? History

Historical Developments: 60s

Start of 60s: elementary files, application specific file structure
(device-dependent, redundant, inconsistent)
End of 60s: file management systems (SAM, ISAM) with service
programs (sorting) (device-independent, but redundant and
inconsistent)
DBS based on hierarchical model, network model
I Pointer structures between data
I Weak separation of internal / conceptual layer
I Navigational DML
I Separation DML / programming language

Saake Database Concepts Last Edited: April 2019 1–32

 What are Databases? History

Historical Developments: 70s and 80s

70s: database systems (device and data independence,
redundancy free, consistent)
Relational database systems
I Data in table structures
I 3 layer concept
I Declarative DML
I Separation DML / programming language

Saake Database Concepts Last Edited: April 2019 1–33

 What are Databases? History

History of RDBMS
1970: Ted Codd (IBM) ! relational model as conceptual basis of
relational DBS
1974: System R (IBM) ! first prototype of an RDBMS
I Two modules: RDS, RSS; ca. 80,000 LOC (PL/1, PL/S, assembler),
ca. 1.2 MB of code
I Query language SEQUEL
I First deployment 1977
1975: University of California at Berkeley (UCB) ! Ingres
I Query language QUEL
I Predecessor of Postgres, Sybase, . . .
1979: Oracle Version 2

Saake Database Concepts Last Edited: April 2019 1–34

 What are Databases? History

Historical Developments: (80s and) 90s

Knowledge base systems
I Data in table structures
I Strongly declarative DML, integrated database programming
language
Object-oriented database systems
I Data in more complex object structures (separation of object and its
data)
I Declarative or navigational DML
I Often integrated database programming language
I Often incomplete separation of layers

Saake Database Concepts Last Edited: April 2019 1–35

 What are Databases? History

Historical Developments: Today

New hardware architectures
I Multicore processors, terabytes of main memory: in-memory
database systems (e.g., SAP HANA)
Support for specific applications
I Cloud databases: Hosting of databases, scalable data
management solutions (Amazon RDS, Microsoft Azure)
I Data stream processing: online processing of live data, e.g., stock
exchange data, sensor data, RFID data, . . . (StreamBase, MS
StreamInsight, IBM Infosphere Streams)
I Big Data: Handle petabytes of data through highly scalable, parallel
processing, data analysis (Hadoop, Hive, Google Spanner & F1,
...)
I NoSQL databases (“Not only SQL”): non-relational databases,
flexible schema (document-centered), “light-weight” because SQL
functionality like transactions is omitted, powerful declarative query
languages with joins, etc. (CouchDB, MongoDB, Cassandra, . . . )

Saake Database Concepts Last Edited: April 2019 1–36

 What are Databases? History

Trends
User-generated content, e.g., Google:
I Daily processing of 20 PB
I 15 hours of video uploaded to YouTube every minute
I Reading 20 PB would take 12 years with a 50 MB/s hard disk drive
Linked data and data web
I Provision, exchange and linking of structured data on the Web
I Enables querying (with query languages like SPARQL) and further
processing
I Examples: DBpedia, GeoNames

Saake Database Concepts Last Edited: April 2019 1–37

 What are Databases? History

Summary
Motivation for using database systems
Codd’s rules
3 layer schema architecture & data independence
Areas of application

Saake Database Concepts Last Edited: April 2019 1–38

 What are Databases? History

Control Questions
What is the advantage of using database
systems compared to application-specific
data management?
What does data independence mean and
how is it achieved?
Which are areas of application of
database systems?

Saake Database Concepts Last Edited: April 2019 1–39