Synchronization
Synchronization
02/08/22 1
Synchronization and Coordination
Distributed Algorithms
Time and Clocks
Global State
Concurrency Control
Distributed Transactions
02/08/22 2
Synchronization
How processes cooperate and synchronize
with each other?
Simultaneously access a shared resources ,
such as printer,
Agree on the ordering of events, such as
whether message m1 from process P was
sent before or after message m2 from process
Q.
02/08/22 3
DISTRIBUTED ALGORITHMS
Algorithms that are intended to work in a distributed
environment.
Used to accomplish tasks such as:
Communication
Accessing resources
Allocating resources
etc.
Synchronization and coordination linked to
distributed algorithms
Achieved using distributed algorithms
02/08/22 4
SYNCHRONOUS Vs ASYNCHRONOUS
DISTRIBUTED SYSTEMS
02/08/22 5
Synchronous Distributed System:
Time variance is bounded
Execution: bounded execution speed and time
Communication: bounded transmission delay
Clocks: bounded clock drift (and differences in clocks)
Effect:
Easier to design distributed algorithms
Very restrictive requirements
• Limit concurrent use of network
• Require precise clocks and synchronization
02/08/22 6
Asynchronous Distributed System
Time variance is not bounded
Execution: different steps can have varying duration
Communication: transmission delays vary widely
Clocks: arbitrary clock drift
Effect:
Allows no assumption about time intervals
• Most asynch DS problems hard to solve
• Solution for asynch DS is also a solution for synch DS
Most real distributed systems are hybrid synch and asynch.
02/08/22 7
EVALUATING DISTRIBUTEDA LGORITHMS
General Properties:
Performance
• number of messages exchanged
• response/wait time
• delay
• throughput
• complexity:
Efficiency
• resource usage: memory, CPU, etc.
Scalability
Reliability
• number of points of failure (low is good)
02/08/22 8
SYNCHRONIZATION AND COORDINATION
02/08/22 9
COORDINATION
Coordination refers to coordinating the actions of separate processes
relative to each other and allowing them to agree on global state (such as
values of a shared variable).
“Coordinate actions and agree on values.”
Coordinate Actions:
What actions will occur
Who will perform actions
Agree on Values:
Agree on global value
Agree on environment
Agree on state
02/08/22 10
Cont …
Examples of coordination include ensuring
that processes agree on –
what actions will be performed?
who will be performing actions?
02/08/22 11
SYNCHRONIZATION
Synchronization is coordination with respect
to time, and refers to the ordering of events
and execution of instructions in time.
“Ordering of all actions”
02/08/22 12
Cont …
Examples of synchronization include,
Ordering distributed events and ensuring that
a process performs an action at a particular
time.
02/08/22 13
MAIN ISSUES
Time and Clocks: synchronizing clocks and using time
in distributed algorithms
Global State: how to acquire knowledge of the system’s
global state
Concurrency Control: coordinating concurrent access
to resources
Coordination: when do processes need to coordinate
and how do they do it
02/08/22 14
TIME AND CLOCKS
02/08/22 15
TIME AND CLOCKS
It is often important to know when events
occurred and in what order they occurred.
In a non-distributed system dealing with time
is trivial as there is a single shared clock,
where all processes see the same time.
02/08/22 16
Cont …
In a distributed system, on the other hand,
each computer has its own clock.
Because no clock is perfect each of these
clocks has its own skew which causes clocks
on different computers to drift and eventually
become out of sync.
02/08/22 17
CLOCKS
Computer Clocks:
oscillates at known frequency
Clock Skew:
different computers run at slightly different rates
Clocks get out of sync
Maximum drift rate
Timestamps:
Used to denote at which time an event occurred
02/08/22 18
PHYSICAL CLOCKS
Based on actual time:
• Physical clocks keep track of physical time.
02/08/22 19
PHYSICAL CLOCKS
Examples:
Performing events at an exact time (turn lights on/off,
lock/unlock gates)
sorting of events (for security, for profiling)
Tracking (tracking a moving object with separate cameras)
02/08/22 20
SYNCHRONIZING PHYSICAL CLOCKS
External Synchronization:
Clocks synchronize to an external time source
Synchronize with UTC every δ seconds
Internal Synchronization:
Clocks synchronize locally
Only synchronized with each other
Time Server:
Server that has the correct time
Server that calculates the correct time
02/08/22 21
What can go wrong?
02/08/22 22
Computer Clock
We need to measure time accurately:
to know the time an event occurred at a computer
Algorithms for clock synchronization useful
for
concurrency control based on timestamp ordering
authenticity of requests e.g. in Kerberos etc
Each computer in a DS has its own internal
clock
02/08/22 23
Clock Synchronization
There exists a time server receiving signals
from a UTC source
Cristian’s algorithm
There is no UTC source available
Berkley’s algorithm
Exact time does not matter!
Lamport’s algorithm
02/08/22 24
Clock Sync. Algorithm: Cristian's
02/08/22 25
Berkeley Algorithm
An algorithm for internal synchronization of a group
of computers
A master polls to collect clock values from the
others (slaves)
The master uses round trip times to estimate the
slaves’ clock values
It takes an average
It sends the required adjustment to the slaves
If master fails, can elect a new master to take over
02/08/22 26
The Berkeley Algorithm
a) The time daemon asks all the other machines for their clock
values
b) The machines answer
c) Takes an average & tells everyone how to adjust their clock
02/08/22 27
Berkeley Algorithm
02/08/22 28
Berkeley Algorithm
Average time: 3:05
02/08/22 29
LOGICAL CLOCKS
The relative ordering of Events is more important than actual
physical time in many applications
In a single process the ordering of events (e.g., state changes) is
trivial
In a distributed system, however, besides local ordering of events,
all processes must also agree on ordering of causally related events
(e.g., sending and receiving of a single message)
Local ordering:
System consists of N processes pi
If pi observes e before e′ , we have e →e′
Global ordering:
Lamport’s happened before relation →
Smallest relation, such that
1. e →e ′
2. For every message m , send (m ) → receive (m )
3. Transitivity: e→ e′ and e′ →e ′′ implies e →e′′
02/08/22 30
ORDERING EVENTS
Event ordering linked with concept of
causality:
Saying that event a happened before event b is
same as saying that event a casually affects
event b
If events a and b happen on processes that do not
exchange any data, their exact ordering is not
important
02/08/22 31
Relation “has happened before”
Smallest relation satisfying the three
conditions:
If a and b are events in the same process and a
comes before b, then a b
If a is the sending of a message by a process and
b its receipt by another process then
ab
If a b and b c then a c.
02/08/22 32
Relation “has happened before”
We cannot always order events: relation “has
happened” before is only a preorder
If a did not happen before b, it cannot
causally affect b.
02/08/22 33
LOGICAL CLOCKS
02/08/22 34
Lamport Algorithm
02/08/22 35
Lamport Algorithm
02/08/22 36
VECTOR CLOCKS
02/08/22 37
VECTOR CLOCKS
02/08/22 38
Tutorial- Lamport and Vector Clock
For the figure below, write down the lamport clock and
vector clock value at each event.
02/08/22 39
Tutorial- Lamport and Vector Clock
02/08/22 40
GLOBAL STATE
02/08/22 41
Determining Global States
Global State
02/08/22 42
Determining Global States
Knowing the global state of distributed system may be useful
for many reasons.
For example:
How to know that system is dealing with deadlock or
distributed computation has correctly terminated?
02/08/22 43
GLOBAL STATE
Determining global properties: often difficult, but essential for
some applications.
Distributed garbage collection:
Do any references exist to a given object?: Collects remote server
objects that are no longer referenced by any client in the network
.
Distributed deadlock detection:
Do processes wait in a cycle for each other?
Distributed algorithm termination detection:
Did a set of processes finish all activity? (Consider messages
in transit)
All these properties are stable: once they occur, they do
not disappear without outside intervention.
02/08/22 44
Analysis of GLOBAL STATE (Distributed
Snapshot)
A distributed snapshot reflects a state in which the
distributed system might have been.
A snapshot reflects a consistent global state
Example: if we have recorded that a process Q has
received a message from another process P,
then we should also recorded that process P has
actually sent that message
Otherwise, a snapshot will contain the recording of
messages that have been received but never sent,
Which indicate the inconsistent global state
The reverse condition (P has sent a message that Q
has not yet received) is allowed
02/08/22 45
More on States
process state
memory state + register state + open files + kernel buffers
+…
Or
application specific info like transactions completed,
functions executed etc,.
channel state
“Messages in transit” i.e. those messages that have been
sent but not yet received
02/08/22 46
Why global state determination is difficult in
Distributed Systems?
Distributed State :
Have to collect information that is spread
across several machines!!
02/08/22 47
Difficulties due to Non Determinism
Deterministic Computation
At any point in computation there is at most one event
that can happen next.
Non-Deterministic Computation
At any point in computation there can be more than one
event that can happen next.
02/08/22 48
Deterministic Computation Example
A Variant of producer-consumer example
02/08/22 49
Example: Initial State
02/08/22 50
Example
02/08/22 51
Example
02/08/22 52
Example
02/08/22 53
Example
02/08/22 54
Example
02/08/22 55
Deterministic state diagram
02/08/22 56
Non-deterministic computation
3 processes
p
m1
q
m2
r m3
02/08/22 57
Election Algorithms
In distributed computing, leader election is the
process of designating a single process as the
organizer of some task distributed among several
computers (nodes).
Before the task is begun, all network nodes are
unaware which node will serve as the "leader," or
coordinator, of the task.
After a leader election algorithm has been run,
however, each node throughout the network
recognizes a particular, unique node as the task
leader.
02/08/22 58
Election Algorithms
Many distributed algorithms require one process to
act as coordinator, initiator, or otherwise perform
special role.
02/08/22 59
Election Algorithms
Election
Goal: ensure that the election achieves an
agreement among all the processes
the P with the highest id is elected
02/08/22 60
The Bully Algorithm
When a process notices that the coordinator in no
longer responding to requests, it initiates an
election.
A process ,P, holds an election as follows
1. P sends an ELECTION message to all process with higher
numbers.
2. If no one responds, P wins the election and becomes
coordinator.
3. If one of the higher-ups answers, it takes over. P’s job is
done
4. Finally only a P (the new coordinator) will remain and it
will inform the other by sending a msg
5. If a process is restarted, the first action is to trigger an
election
02/08/22 61
The Bully Algorithm (1)
02/08/22 64
A Ring Algorithm
When a msg return to a P who sent it (verify
by inspecting the list), the msg is turned into
COORDINATOR and circulated, to report:
New coordinator: P of the list with highest id
Multiple messages can circulate over the network
02/08/22 65
A Ring Algorithm
Once around, change to COORDINATOR (biggest)
02/08/22 67
Concurrency in a Distributed System:
02/08/22 68
DISTRIBUTED MUTUAL EXCLUSION
Concurrent access to distributed resources
Must prevent race conditions during critical
regions
Requirements:
1. Safety: At most one process may execute the
critical section at a time
2. Liveness: Requests to enter and exit the critical
section ultimately succeed
3. Ordering: Requests are processed in happened-
before ordering
02/08/22 69
METHOD 1: CENTRAL SERVER
Simplest approach: use a central server that
controls the entering and exiting of critical
sections.
Requests to enter and exit a critical section are
sent to a lock server (or coordinator)
Permission to enter is granted by receiving a
token
When critical section left, token is returned to
the server
02/08/22 70
A Centralized Algorithm
a) Process 1 asks the coordinator for permission to enter a critical region. Permission is
granted
b) Process 2 then asks permission to enter the same critical region. The coordinator
does not reply. (Or, can say “denied”)
c) When process 1 exits the critical region, it tells the coordinator, when then replies to
2.
02/08/22 71
Cont...
This scheme is easy to implement, but it does
not scale well due to the central authority.
Moreover, it is vulnerable to failure of the
central server.
02/08/22 72
A Distributed Algorithm
02/08/22 73
M ETHOD 2: TOKEN RING
Implementation:
All processes are organised in a logical ring structure
A token message is forwarded along the ring
Before entering the critical section, a process has to wait until the
token comes
Must retain the token until the critical section is left
02/08/22 74
Properties:
Ring limits scalability
Token messages consume bandwidth
Failing nodes or channels can break the ring (token
might be lost)
02/08/22 75
METHOD 3: USING MULTICASTS AND LOGICAL
CLOCKS
02/08/22 76
Process behaviour:
1. If a process wants to enter a critical section, it
•multicasts a message( Li, pi) and
•waits until it has received a reply from every process
2. If a process is in Released , it immediately replies to any
request to enter the critical section
3. If a process is in Held , it delays replying until it is finished
with the critical section
4. If a process is in Wanted , it replies to a request immediately
only if the requesting timestamp is smaller than the one in
its own request
02/08/22 77
Properties:
02/08/22 78
MUTUAL EXCLUSION: A COMPARISON
Messages Exchanged : Messages per entry/exit of critical section
o Centralised:3 (two to enter and one to leave)
o Ring:1→ ∞
o Multicast:2(n−1)
Delay: Delay before entering critical section
o Centralised: 2
o Ring: 0→n−1
o Multicast:2(n−1)
Reliability:
o Centralised: coordinator crashes
o Ring: lost token, process crashes
o Multicast: any process crashes
02/08/22 79
DISTRIBUTED TRANSACTIONS
02/08/22 80
TRANSACTIONS
Transaction:
Comes from database world: the concept of a transaction originates from the
database community as a mechanism to maintain the consistency of
databases.
Defines a sequence of operations
Atomic in presence of multiple clients and failures
Mutual Exclusion :
Protect shared data against simultaneous access
Allow multiple data items to be modified in single atomic action
Transaction Model:
Operations:
• Begin Transaction
• End Transaction
• Read
• Write
End of Transaction:
• Commit
• Abort
02/08/22 81
ACID PROPERTIES
atomic: must follow an "all or nothing" rule. Each
transaction is said to be atomic if when one part of
the transaction fails, the entire transaction fails and
database state is left unchanged.
consistent: concurrent transactions will not produce
inconsistent results;
isolated: transactions do not interfere with each
other i.e. no intermediate state of a transaction is
visible outside , it refers to the requirement that other
operations cannot access data that has been modified
during a transaction that has not yet completed.
durable: after a commit, results are permanent (even
if server or hardware fails)
02/08/22 82
CLASSIFICATION OF
TRANSACTIONS
Flat: sequence of operations that satisfies ACID
Nested: hierarchy of transactions
Distributed: (flat) transaction that is executed on distributed
data
Flat Transactions:
Simple
Failure all changes undone
BeginTransaction
accountA -= 100;
accountB += 50;
accountC += 25;
accountD += 25;
EndTransaction
02/08/22 83
NESTED TRANSACTION
Subdivide a complex transaction into many smaller sub-
transactions
As a result, if a single sub-transaction fails, the work that
went into others is not necessary wasted
Example:
Booking a flight
• Sydney Manila
• Manila Amsterdam
• Amsterdam Toronto
What to do?
Abort whole transaction
Partially commit transaction and try alternative for aborted part
Commit nonaborted parts of transaction
02/08/22 84
Cont…
02/08/22 86
TRANSACTION IMPLEMENTATION
Private Workspace:
Perform all tentative operations on a shadow copy of the server state
Atomically swap with main copy on Commit
Discard shadow on Abort.
02/08/22 87
Cont…
Writeahead Log:
In-place update with writeahead logging
Log is reverted when a transaction Aborts
02/08/22 88
CONCURRENCY CONTROL
It is often necessary to allow transactions to occur
simultaneously (for example, to allow multiple travel
agents to simultaneously reserve seats on the same
flight).
Due to the consistency and isolation properties of
transactions concurrent transaction must not be
allowed to interfere with each other.
Concurrency control algorithms for transactions
guarantee that multiple transactions can be executed
simultaneously while providing a result that is the
same as if they were executed one after another.
02/08/22 89
CONCURRENCY CONTROL
Simultaneous Transactions:
Clients accessing bank accounts
Travel agents booking flights
Problems:
Simultaneous transactions may interfere
• Lost update
• Inconsistent retrieval
Consistency and Isolation require that there is no interference
Concurrency Control Algorithms:
Guarantee that multiple transactions can be executed simultaneously
while still being isolated.
As though transactions executed one after another
02/08/22 90
CONFLICTS AND SERIALISABILITY
conflict: operations (from the same, or different
transactions) that operate on same data
read-write conflict: one of the operations is a write
write-write conflict: more than one operation is a
write
Define a Schedule of operations:
Total ordering (interleaving) of operations
Legal schedules provide results as though the transactions
were serialised (i.e., performed one after another) (serial
equivalence)
02/08/22 91
MANAGING CONCURRENCY
Transaction Managers:
02/08/22 92
Dealing with Concurrency:
Locking
Timestamp Ordering
Optimistic Control
02/08/22 93
LOCKING
Pessimistic approach: prevent illegal
schedules
The locking algorithms require that each
transaction obtains a lock from a scheduler
process before performing a read or a write
operation.
The scheduler is responsible for granting and
releasing locks in such a way that legal schedules
are produced.
Ensures that only valid schedules result
02/08/22 94
TWO PHASE LOCKING (2PL)
02/08/22 95
PROBLEMS WITH LOCKING
Deadlock
Cascaded Aborts: If a transaction (T1) reads
the results of a write of another transaction
(T2) that is subsequently aborted, then the
first transaction (T1) will also have to be
aborted.
02/08/22 96
TIMESTAMP ORDERING
02/08/22 98
Cont….
In the validation phase the scheduler attempts
to detect conflicts with other transactions that
were in progress during the working phase.
If conflicts are detected then one of the
conflicting transactions are aborted.
02/08/22 99
Cont…
In the update phase, assuming that the
transaction was not aborted, all the updates
made on the shadow copy are made
permanent.
02/08/22 100
DISTRIBUTED TRANSACTIONS
In distributed system, a single transaction
will, in general, involve several servers:
transaction may require several services,
transaction involves files stored on different
servers
All servers must agree to Commit or Abort,
and do this atomically.
02/08/22 101
Distributed Flat Transaction:
02/08/22 102
Distributed Nested Transaction:
02/08/22 103
DISTRIBUTED CONCURRENCY
CONTROL
02/08/22 104
DISTRIBUTED LOCKING
Distributed 2PL:
Data can be replicated
Scheduler on each machine responsible for locking own
data
Read lock: contact any replica
Write lock: contact all replicas
02/08/22 105
ATOMICITY AND DISTRIBUTED
TRANSACTIONS
Distributed Transaction Organisation:
Each distributed transaction has a coordinator, the
server handling the initial BeginTransaction call
Coordinator maintains a list of workers, i.e. other
servers involved in the transaction
Each worker needs to know coordinator
02/08/22 106
DISTRIBUTED ATOMIC COMMIT
Transaction may only be able to commit when all
workers are ready to commit (e.g. validation in
optimistic concurrency)
Distributed commit requires at least two phases:
1. Voting phase: all workers vote on commit, coordinator
then decides whether to commit or abort.
2. Completion phase: all workers commit or abort
according to decision.
Basic protocol is called two-phase commit (2PC)
02/08/22 107
Two-phase commit: Coordinator
02/08/22 108
Two-phase commit: Worker
02/08/22 109
Failures can be due to:
server failures/hosts:
• If a host fails in the 2PC protocol, then, after being
restarted, it aborts all transactions.
restarting worker aborts all transactions
Failure of communication channels/network:
• coordinator aborts after timeout.
02/08/22 110
Two-phase commit with timeouts: Worker
02/08/22 111
Two-phase commit with timeouts:
Coordinator
On timeout re-sends CanCommit, On GetDecision repeats
decision
02/08/22 112
Limitations(2PC)
Once node voted “yes”, cannot change its
mind, even if crashes.
Atomic state update to ensure “yes” vote is stable.
If coordinator crashes, all workers may be
blocked.
Can use different protocols (e.g. three-phase
commit),
in some circumstances workers can obtain result
from other workers.
02/08/22 113
SUMMARY
Distributed Algorithms:
Timing models
Time:
Clock synchronisation
Logical clocks
Vector clocks
Concurrency Control:
Distributed mutual exclusion
Distributed transactions
2PC
02/08/22 114