Database Management Systems

Reliability Management

DB
MG
1
 DBMS Architecture

SQL INSTRUCTION

OPTIMIZER

CONCURRENCY CONTROL
MANAGEMENT OF ACCESS
METHODS

BUFFER MANAGER RELIABILITY MANAGEMENT

Index Files
System DATABASE
Catalog
Data Files

2
DB
MG
 Reliability Manager

It is responsible of the atomicity and durability

ACID properties
It implements the following transactional
commands
begin transaction (B, usually implicit)
commit work (C)
rollback work (A, for abort)
It provides the recovery primitives
warm restart
for main memory failures
cold restart
3
DB
MG
 Reliability Manager

It manages the reliability of read/write requests

by interacting with the buffer manager
It may generate new read/write requests for
reliability purposes
It exploits the log file
a persistent archive recording DBMS activity
stored on stable memory
It prepares data for performing recovery by
means of the operations
checkpoint
dump
4
DB
MG
 Stable memory

Memory that is resistant to failure

it is an abstraction
it is approximated by means of
redundancy
robust write protocols
Failures in stable memory are considered
catastrophic

5
DB
MG
 Log file

Sequential file written in stable memory

It records transaction activities in chronological
order
Log record types
Transaction records
System records
Writing the log
Records are written in the current block in
sequential order
Records belonging to different transactions are
interleaved
6
DB
MG
 Transaction records

Describe the activities performed by each

transaction in execution order
Transaction delimiters
Begin B(T)
Commit C(T)
Abort/Rollback A(T)
where T is the Transaction Identifier

7
DB
MG
 Transaction records

Data modifications
Insert I(T, O, AS)
Delete D(T, O, BS)
Update U(T, O, BS, AS)
where
O is the written object (RID)
AS is the After State (state of object O after the
modification)
BS is the Before State (state of object O before the
modification)

8
DB
MG
 System records

Record system operations saving data on disk or

other tertiary (or off-line) storage
Dump
Checkpoint CK(L)
where L = T1, T2, …, Tn
is the set of TIDs of active transactions

9
DB
MG
 Log example

Top of the log

B(T1) U(T2,O2,BS2,AS2) C(T1)

time
DUMP B(T2) CHECK I(T1,O1,AS1) D(T2,O3,BS3)
POINT

10
DB
MG
 Undoing and redoing actions

Undo of an action on an object O

Action Undo action
insert O delete O
update O write the before state (BS) of O
delete O write the before state (BS) of O

Redo of an action on an object O

Action Redo action
insert O write the after state (AS) of O
update O write the after state (AS) of O
delete O delete O
11
DB
MG
 Undoing and redoing actions

Idempotency property
Undo or Redo can be repeated an arbitary number of
times without changing the final outcome
UNDO (UNDO(action)) = UNDO(action)
Useful for managing crashes during the recovery
process

12
DB
MG
 Checkpoint

Operation periodically requested by the Reliability

Manager to the Buffer Manager
It allows a faster recovery process
During the checkpoint, the DBMS
writes data on disk for all completed transactions
synchronous write
records the active transactions

13
DB
MG
 Execution of a checkpoint

1. The TIDs of all active transactions are recorded

after the checkpoint is started, no transaction can
commit until the checkpoint ends
2. The pages of concluded transactions (committed
or aborted) are synchronously written on disk
by means of the force primitive
3. At the end of step 2, a checkpoint record is
synchronously written on the log
it contains the set of active transactions
it is written by means of the force primitive

14
DB
MG
 Checkpoint

After a checkpoint
The effect of all committed transactions is
permanently stored on disk
The state of data pages written by active
transactions is unknown

15
DB
MG
 Dump

It creates a complete copy of the database

typically performed when the system is offline
the database copy is stored in stable memory
tertiary storage or off-line storage
the copy may be incremental
At the end, a dump record is written in the log
file
Date and time of the dump
Dump device used

16
DB
MG
 Rules for writing the log

Designed to allow recovery in presence of failure

WAL
Commit precedence

17
DB
MG
 Write Ahead Log

The before state (BS) of data in a log record is

written in stable memory before database data is
written on disk
During recovery, it allows the execution of undo
operations on data already written on disk

18
DB
MG
 Commit precedence

The after state (AS) of data in a log record is

written in stable memory before commit
During recovery, it allows the execution of redo
operations for transactions that already
committed, but were not written on disk

19
DB
MG
 Practical rules for writing the log

BS and AS are written together

WAL
The log must be written before the record in the
database
Commit precedence
The log must be written before commit

20
DB
MG
 Practical rules for writing the log

The log is written synchronously (force)

for data modifications written on disk
on commit
The log is written asynchronously
for abort/rollback

21
DB
MG
 Commit record

The commit record on the log is a border line

If it is not written in the log, the transaction should
be undone upon failure
If it is written, the transaction should be redone
upon failure

22
DB
MG
 Protocols for writing the log and the database

Log

B(T) U(T, x, BSx, ASx) U(T, y, BSy, ASy) C(T)

Time

W(x) W(y)
DB

All database disk writes are performed before

commit
It does not require redo of committed transactions

23
DB
MG
 Protocols for writing the log and the database

Log

B(T) U(T, x, BSx, ASx) U(T, y, BSy, ASy) C(T)

Time

W(x) W(y)
DB

All database disk writes are performed after

commit
It does not require undo of uncommitted
transactions
24
DB
MG
 Protocols for writing the log and the database

Log

B(T) U(T, x, BSx, ASx) U(T, y, BSy, ASy) C(T)

Time

W(x) W(y)
DB

Disk writes for the database take place both

before and after commit
It requires both the undo and redo operations
Mixed approach adopted in real systems
25
DB
MG
 Writing the log

The usage of robust protocols to guarantee

reliability is costly
Comparable with database update cost
It is required to guarantee the ACID properties
Log writing is optimized
Compact format
Parallelism
Commit of groups of transactions

26
DB
MG
 Database Management Systems

Recovery Management

DB
MG
27
 Types of failures

System failure
Caused by software problems or power supply
interruptions
It causes losing the main memory content (DBMS
buffer) but not the disk (both database and log)
Media failure
Caused by failure of devices managing secondary
memory
It causes losing the database content on disk, but
not the log content (stored in stable storage)

28
DB
MG
 Fail-stop model

STOP

Normal
Operation

RECOVERY

29
DB
MG
 Recovery

When a failure occurs

The system is stopped
Recovery depends on the failure type
Warm restart
performed for system failures
Cold restart
performed for media failures
When recovery ends
the system becomes again available to
transactions

30
DB
MG
 Database Management Systems

Warm Restart

DB
MG
31
 Transaction categories

tC tF time

T1

T2

T3

T4

T5

tC = time of the last checkpoint crash!

tF = time of failure 32
DB
MG
 Transaction categories

Transactions completed before the checkpoint

(T1)
No recovery action is needed
Transactions which committed, but for which
some writes on disk may not have been done yet
(T2 and T4)
redo is needed
Active transactions at the time of failure
(T3 and T5)
they did not commit
undo is needed
33
DB
MG
 Checkpoint record

The checkpoint record is not needed to enable

recovery
It provides a faster warm restart
Without checkpoint record
The entire log needs to be read until the last dump

34
DB
MG
 Warm restart algorithm

1. Read backwards the log until the last checkpoint

record
2. Detect transactions which should be
undone/redone
a) At the last checkpoint
UNDO = { Active transactions at checkpoint }
REDO = { } (empty)

35
DB
MG
 Warm restart algorithm

b) Read forward the log

UNDO = Add all transactions for which the begin
record is found
REDO = Move transactions from UNDO to REDO list
when the commit record is found
Transactions ending with rollback remain in the UNDO
list
At the end of step 2
UNDO = list of transactions to be undone
REDO = list of transactions to be redone

36
DB
MG
 Warm restart algorithm

tC tF time

T1

T2

T3

T4

T5

UNDO = {T2,T3}
tC = time of last checkpoint
REDO = { }
DB tF = time of failure 37
MG
 Warm restart algorithm

tC t1 tF time

T1

T2

T3

T4

T5

UNDO = {T2,T3,T4}
tC = time of last checkpoint
REDO = { }
DB tF = time of failure 38
MG
 Warm restart algorithm

tC t1 t2 tF time

T1

T2

T3

T4

T5

UNDO = {T3,T4}
tC = time of last checkpoint
REDO = {T2}
DB tF = time of failure 39
MG
 Warm restart algorithm

tC t1 t2 t3 tF time

T1

T2

T3

T4

T5

UNDO = {T3,T4,T5}
tC = time of last checkpoint
REDO = {T2}
DB tF = time of failure 40
MG
 Warm restart algorithm

tC t1 t2 t3 t4 tF time

T1

T2

T3

T4

T5

Final lists
UNDO = {T3,T5}
tC = time of last checkpoint
REDO = {T2,T4}
DB tF = time of failure 41
MG
 Warm restart algorithm
3. Data Recovery
a) The log is read backwards from the time of
failure until the beginning of the oldest
transaction in the UNDO list
Actions performed by all transactions in the UNDO
list are undone
For each transaction the begin record should be
reached
even if it is earlier than the last checkpoint

42
DB
MG
 Warm restart algorithm
b) The log is read forward from the beginning of the
oldest transaction in the REDO list
Actions of transactions in the REDO list are applied
to the database
For each transaction, the starting point is its begin
record

43
DB
MG
 Warm restart example

Log snippet
B(T1) B(T2) U(T2, O1, B1 , A1) I(T1, O2, A2) B(T3)
C(T1) B(T4) U(T3, O2, B3, A3) U(T4, O3, B4, A4)
CK(T2, T3, T4) C(T4) B(T5) U(T3, O3, B5, A5)
U(T5, O4, B6, A6) D(T3, O3, B7) A(T3) C(T5)
I(T2, O6, A8) failure

44
DB
MG
 Warm restart example

Log snippet
B(T1) B(T2) U(T2, O1, B1 , A1) I(T1, O2, A2) B(T3)
C(T1) B(T4) U(T3, O2, B3, A3) U(T4, O3, B4, A4)
CK(T2, T3, T4) C(T4) B(T5) U(T3, O3, B5, A5)
U(T5, O4, B6, A6) D(T3, O3, B7) A(T3) C(T5)
I(T2, O6, A8)
1. At the checkpoint
UNDO = {T2,T3,T4}
REDO = { }

45
DB
MG
 Warm restart example
B(T1) B(T2) U(T2, O1, B1 , A1) I(T1, O2, A2) B(T3)
C(T1) B(T4) U(T3, O2, B3, A3) U(T4, O3, B4, A4)
CK(T2, T3, T4) C(T4) B(T5) U(T3, O3, B5, A5)
U(T5, O4, B6, A6) D(T3, O3, B7) A(T3) C(T5)
I(T2, O6, A8)
2. Read the log forward
Operation UNDO REDO
CK {T2,T3,T4} {}

46
DB
MG
 Warm restart example
B(T1) B(T2) U(T2, O1, B1 , A1) I(T1, O2, A2) B(T3)
C(T1) B(T4) U(T3, O2, B3, A3) U(T4, O3, B4, A4)
CK(T2, T3, T4) C(T4) B(T5) U(T3, O3, B5, A5)
U(T5, O4, B6, A6) D(T3, O3, B7) A(T3) C(T5)
I(T2, O6, A8)
2. Read the log forward
Operation UNDO REDO
CK {T2,T3,T4} {}
C(T4) {T2,T3} {T4}
B(T5) {T2,T3,T5} {T4}
A(T3) {T2,T3,T5} {T4}
Final
C(T5) {T2,T3} {T4,T5} lists 47
DB
MG
 Warm restart example
B(T1) B(T2) U(T2, O1, B1 , A1) I(T1, O2, A2) B(T3)
C(T1) B(T4) U(T3, O2, B3, A3) U(T4, O3, B4, A4)
CK(T2, T3, T4) C(T4) B(T5) U(T3, O3, B5, A5)
U(T5, O4, B6, A6) D(T3, O3, B7) A(T3) C(T5)
I(T2, O6, A8)
3. Undo transactions in UNDO = {T2,T3}
a) DELETE O6
b) INSERT O3 = B7
c) O3 = B5
d) O2 = B3
e) O1 = B1
48
DB
MG
 Warm restart example
B(T1) B(T2) U(T2, O1, B1 , A1) I(T1, O2, A2) B(T3)
C(T1) B(T4) U(T3, O2, B3, A3) U(T4, O3, B4, A4)
CK(T2, T3, T4) C(T4) B(T5) U(T3, O3, B5, A5)
U(T5, O4, B6, A6) D(T3, O3, B7) A(T3) C(T5)
I(T2, O6, A8)
4. Redo transactions in REDO = {T4,T5}
a) O3 = A4
b) O4 = A6

49
DB
MG
 Warm restart example

tC tF time

T1

T2

T3 Abort

T4

T5

REDO = {T4,T5}
tF = time of failure
UNDO = {T2,T3}
50
DB
MG
 Database Management Systems

Cold Restart

DB
MG
51
 Cold restart

It manages failures damaging (a portion of) the

database on disk
Main steps
1. Access the last dump to restore the damaged
portion of the database on disk
2. Starting from the last dump record, read the log
forward and redo all actions on the database and
transaction commit/rollback
3. Perform a warm restart

52
DB
MG
 Cold restart

Alternative to steps 2 and 3

Perform only actions of committed transactions
It requires two log reads
Detect committed transactions
build a REDO list
Redo actions of transactions in REDO list

53
DB
MG