14/07/2009

Radio networks QoS assessment and evaluation tools

ITU/BDT Arab Regional Workshop on ICT Applications for Rural Communications

Sami Tabbane
Damascus - Syria, 27-29 July 2009

Summary

QoS definition QoS assessment tools

2

14/07/2009

Quality of Service definitions and objectives

3

End-toEnd-to-end Quality of Service

14/07/2009

Quality of Service definitions (1) QoS (ITU-T): The collective effect of service performance which determines the degree of satisfaction of a user of the service. Network Performance, NP (ITU-T): The ability of a network or network portion to provide the functions related to communication between users.
5

Quality of Service definitions (2)

User domain: throughput, accuracy, dependability (reliability, availability), Provider domain: delay, loss, utilisation,
User Domain User QoS Requirements Provider Domain QoS offered by Provider

QoS experienced By User

QoS achieved by User

6

14/07/2009

Quality of Service definitions (3)

QoS and NP, Performance network (ITU Rec. E800)
Quality of Service
Servic support (ability to provide a service and help its usage) Service operability (capacity of a service to be used) Service accessibility Service sustainability Service integrity Service security

Serveability performance QoS

Traffic performance

NP
Transmission

Billing (probability that the network accounts the communications by type, destination,

Resources and environment

Dependability (availability, reliability, maintenance, support)

Quality of Service and user satisfaction

Commercial offer Competition Trends

Users expectations in terms of QoS Users satisfaction

Technical QoS
Network performance Terminal performance

Non-technical QoS
Customer care 8

Sales points

14/07/2009

QoS interactions
ETSI Technical Report Network Aspects (NA); General aspects of Quality of Service (QoS) and Network Performance (NP)

SLA (Service Level (Service Agreement) Agreement)

ITU-T Rec. E.860 Framework of a service level agreement (2002). formal definition of a relation between 2 entities at least: for example the service provider and his customer. Consists in 2 parts: - Legal and financial specifications, - Service level specifications (technical).
10

14/07/2009

SLA (2)
ITU-T: QoS is the degree of conformance of the service delivered to a user by a provider with an agreement between them. Negotiation Phase Documentation Phase Implementation Phase Maintenance and Reporting Phase
Yes

Satisfaction (user/provider)
No

Termination Phase

11

SLA parameters example

Generally measured on a monthly basis: - RTT (Round Trip Time), - Packet loss ratio (ping ICMP), - Jitter. Sampling can be achieved on a 5 minutes basis typically.
12

14/07/2009

Speech Processing, Transmission and Quality Aspects (STQ); QoS aspects for popular services in GSM and 3G networks; Part 2: Definition of Quality of Service parameters and their computation

ETSI TS 102 250-2 V1.5.1 (2007-10)

13

Quality of Service indicators classes

Indicators classes: Network access, Service access, Service integrity, Service sustainability.
14

14/07/2009

Usage phases from the user point of view

Network Availability

Network Accessibility

Service Accessibility

Service Integrity

Service Retainability

15

Quality of Service criteria

16

14/07/2009

Quality of Service users criteria

4 main criteria allow users to evaluate the network Quality of Service: Coverage, Call establishment, Voice quality, Call dropping.
17

Quality of Service operator evaluation

Coverage Successful call rate
Cannot be assessed through the system. Radio measurements and subscribers.

System measurements

Communication Radio measurements quality during the System measurements call Voice quality analysers Call drop rate System measurements
18

14/07/2009

Quality of Service Coverage

ITU Rec. 800: probability that the user receives a signal in specific conditions. Reasons/Actions: - Number of sites - Position of the sites - Antennas type, direction and height (RF design) - Output power and cable losses (installation or maintenance).

19

Quality of Service Successful call rates

Example: the subscriber cannot establish calls. ITU Rec. 800: probability that a service can be obtained by the user in certain operational conditions. Reasons/Actions: - Minimum access level in the cell - Difference between the dimensioning of the resources and the offered traffic - Interferences 20 - Radio coverage.

10

14/07/2009

Accessibility evaluation from a regulator point of view

Method: conduct several call attempts spaced with 30 seconds minimum. Accessibility rate = (number successful of communications in 10 seconds maximum)/(total call attempts number).
21

Quality of Service voice quality (1)

Example: bad communication quality. Reasons/Actions: - external interferences - co-channel or adjacent channel interferences - out-of-coverage - bad installation - transmission network (ex.: echo cancelor or trunks).
22

11

14/07/2009

Main factors having an impact on the voice quality (2)

Hardware (TRX, ) Handover Coverage Congestion Interference Codec (HR/FR/EFR/AMR) Transmission errors (A, Abis) Synchronization 23 Advanced features (TFO, PC, DTX, FH, )

Quality of Service Call dropping

Reasons/Actions: - coverage - interferences - handover problems - radio parameters local adjustment.
24

12

14/07/2009

Quality of Service in GSM/GPRS

25

GSM GoS (typical values)

Interfaces Um/TCH Um/SDCCH GSM-RTC A Abis Ater Blocking rate 2% (1 to 5%) 0.1% (0 to 1%) 0.5% (0.1 to 1%) 0.1% (0 to 1%) 0% 0.1% (0 to 1%)
26

13

14/07/2009

QoS indicators (1)

Excellent
MOS 4-5

Good
3.8-4.2

Average Best Effort

2.9-3.8 2-2.9 > 3 sec.

Call < 1.5 sec. 1.5 3 > 3 sec. establishment sec. delay Round-trip delay < 150 150 250 250 (ms) 450 Jitter (ms) <1 < 18 > 18 Packet loss (%) <1 <5 < 10

> 450 > 80 < 10

27

Other QoS indicators (2)

Transmission delay: - fixed network (PSTN): 35 msec end-to-end, - Node B RNC [3GPP]: < 10 ms, - UTRAN (real time services): < 5 ms, - UTRAN (non real time services): < 10 ms, - Radio interface : 70-80 msec in each direction. Loss rate: - UTRAN: < 10-4 (real time traffic), 10-7 (non real time traffic).
28

14

14/07/2009

Other QoS indicators (3)

Round-trip transmission delays for IP packets: - GPRS: 650 msec, - EDGE (R99): 300-500 ms, - UMTS: 200 ms.
29

Quality of Service in UMTS

30

15

14/07/2009

UMTS QoS classes

Traffic classes Conversational Streaming Interactive Background
Question/an - Minimum swer mode, jitter - Minimises the errors Video Web browsing - No delay constraints, - Minimises the errors Emails

-Minimum Main characteristi jitter, cs - Low delay Example Voice

31

Multimedia services
real time communications voice text video non real time communications audio streaming video streaming data files text messaging (ex. SMS) Email Web browsing multimedia messaging
32

16

14/07/2009

Some values of QoS UMTS parameters classes

Traffic class maximum throughput (kb/s) scheduling max. SDU size (octets) Corrupted SDU delivery residual BER SDU error rate transfer delay (ms)
5*10-2, 10-2, 5*10-3, 10-4, 10-6 10-2, 7*10-3, 10-3, 10-4, 10-5 100 maximum value

Conversation.

Streaming

Interactive

Background

< 2048 Yes / No 1 500 or 1 502 Yes/No

5*10-2, 10-2, 5*10-3, 10-3, 10-4, 10-5, 10-6 10-1, 10-2, 7*10-3, 10-3, 10-4, 10-5 250 maximum value 4*10-3, 10-5, 6*10-8 10-3, 10-4, 10-6 4*10-3, 10-5, 6*10-8 10-3, 10-4, 10-6

33

QoS negociation
UE End-to-end service
CM (Communication Mgt) - throughput max, - throughput garantit, - delay de transfert, - QoS negociable (O/N)

UTRAN

CN

UMTS Bearer Service: Request for UMTS QoS Class RAB Assignment Request RRM Admission control

RAB and radio link establishment

QoS negociation RAB Assignment Response

34

UMTS service bearer with the negociated QoS

17

14/07/2009

QoS parameters measurements tools

35

Measurement types

BTS

BSC
Counter A B C

MSC
Value 3.15 1.05 0.95

Call X Y Z

SpQ Bad Excellent Good

NE counters

OMC counters
Subs. A
+33 6 XXXX XXXX +33 6 XXXX XXXX

Measurement field

Subs. B
+33 1 XXXX XXXX +33 4 XXXX XXXX

Dropped Call
No Yes

Drive tests

Capture tool

CDR: Call Data Record

36

18

14/07/2009

Measurement tools typology

Field measurements Measurement tools System measurements OMC counters
Generic Specific

Passive capture tools

Calls generators
37

A. Field measurements
38

19

14/07/2009

Drive test equipment

measures and softwares

GPS

External antennas

Controler

Energy Processing

Mobile QoS test equipment

Man to machine interface
39

Drive tests (1) TEMS example

Measurement tool based on the TEMS includes: PC software with a serial port for the data, TEMS mobile, GPS receiver.
40

20

14/07/2009

Drive tests (2)

TEMS information
- service cell and neighbouring cells. - radio parameters: RXLev, RXQual, TXPower, DTX, Timing Advance, FER, SQI (voice quality), C1, C2, - current channel : CGI (MCC, MNC, LAC, CI), BSIC, BCCH ARFCN, TCH ARFCN, Time slot, Channel type, Channel mode (FR, EFR, HR), Hopping Channel, Hopping Frequencies, HSN (Hopping Sequence Number). - Map to display the measurement itinerary. - level 2 messages (RR-RSP, DISC-CMD, UA-RSP, SABM-CMD, ) et 3 (Synch Channel Information, System Information Type 6, Measurement Report, Synch Channel Information, Paging Request, Assignement Complete, Handover Complete, ). - Frequency scanning.
41

Benchmark

42

21

14/07/2009

Drive tests

43

QVOICE

3 parts: QVM (QV Mobile), QVS (QV Stationary) et QVP (QV Post processing).
44

22

14/07/2009

Voice quality measurement principle

45

B. System measurements
46

23

14/07/2009

B.1. OMC measurements

47

Some tools
Specific Alcatel: RNO Siemens: SPOTS Ericsson: TEMS Analyzer
Generic
APIC from Metrica MyCom from MyCom AirCom NetAct SQM: Nokia OVPI: HP (for IP equipments)

48

24

14/07/2009

KPI processing tools

Commercial tools:

BiVision, ADC/Metrica, NetAct (Nokia, for3G),

UTRAN Network and Service Analyzer (Tektronix), Actix.
49

Analysis based on OMC-R counters OMCAnalysis tools using these counters (generally they are specific). example: RNO or NPA of Alcatel, SPOTS from Siemens, etc.

50

25

14/07/2009

B.2. Passive capture tools

51

Examples (1)

Tektronics: K12/15XX Network General: Sniffer Pro + NPO MyCom: NIMS-PrOptima Tekelec: Steleus 2.5 G (GPRS) and Steleus 3G (Iu interfaces).
52

26

14/07/2009

Examples (2)

HP: Ovis (data services tests, produces KPIs). RadCom: Network Consultant (A, Gb, Gi, Gn, Iub, Iur, Iu, Gi and Gn interfaces) Trafica (NetAct from Nokia) Ipanema: Ipanema (2,5 G and 3G data traffic). Cigale (Astellia): 2 and 3G traffic.
53

B.3. Protocol analysers

54

27

14/07/2009

Network interfaces analysis

Company Nethawk Agilent Tektronix Radcom Acterna Product 3G Analyser Signaling analyser K15 Performer analyser Telecom Protocol Analyzer
55

Network interface analysis

K12/15XX From TEKTRONIX

56

28

14/07/2009

B.4. Calls generators

57

Some tools

P3: Vodafone specific WQM (Agilent).

58

29

14/07/2009

B.4. Measurements combination

59

GIS display
Mapping of call drop using system information

Sites and sectors with call drop rates higher than the average
60

30

14/07/2009

C. QoS assessment examples for network optimization

61

Optimization throughout Life Cycle

Subs. Number
Pre-launch optimization Single site verification RF plan review

Drop call Congestion Poor quality Access failure

RF adjustment Para. adjustment Maintenance and upgrade Network expansion .

Customized and differentiated services Billing strategies

Investment DT & CQT KPI analysis Complaints

Return

Construction phase Troubleshooting

Commercial launch

Booming phase Potential Seeking

Mature phase Balancing

Time

31

14/07/2009

Pilot channel pollution

Pilot channel pollution: when receiving multiple pilots with high power but without a predominant signal do not allow to synchronize on one of them. Solutions: sectorization, antennas tilting, power adjustment.
63

Soft HO tuning
SHO parameters: - AS_Th: macrodiversity threshold (report margin), - AS_Th_Hyst: AS-Th hysteresis, - AS_Rep_Hyst: AS-Rep hysterisis, - T: triggering duration, - AS_Max_Size: AS maximum size.
64

32

14/07/2009

Optimization process
1 2 Drive tests Post processing UETR CTR GPEH OSS- RC counters 3 Change proposal Network KPI 1 3 2

UETR: user equipment traffic recording CTR: Cell traffic recording GPEH: General performance event handling

65

Network KPI
Accessibility RRC establishment SR (LA/RA update, SMS) CSSR (CS + PS) (RRC+RAB) Paging SR Retainability Soft/Softer HO SR Inter-frequency HO SR IRAT HO SR IRAT CC SR LA/RA update Cell selection/reselection SR Integrity Drop rate (CS + PS) Data throughput (PS) Congestion

66

33

14/07/2009

Dropped call classification

Dropped call classification: 1.Missing neighbors 2.Pour coverage 3.Pilot pollution 4.Congestion 5.Not radio 6.Equipment fault
67

Dropped call classification

1. Missing neighbors

68

34

14/07/2009

Dropped call classification

2. Poor coverage

69

Dropped call classification

3. Pilot pollution

70

35

14/07/2009

Dropped call classification

4. Congestion

71

Dropped call classification

5. Not radio

72

36

14/07/2009

Dropped call classification

6. Equipment fault

73

Blocked call classification

Blocked call classification: 1. Security & authentication mode failure 2. UE freeze 3. Disconnect on RAB setup 4. Unavailable resources 5. UE sensitivity fault 6. Unanswered RRC requests 7. Barred network
74

37

14/07/2009

Blocked call classification

1. Security & authentication failure

75

Blocked call classification

2. UE freeze

76

38

14/07/2009

Blocked call classification

3. Disconnect on RAB setup

77

Blocked call classification

4. Unavailable resources

78

39

14/07/2009

Blocked call classification

5. UE sensitivity fault

79

Blocked call classification

6. Unanswered RRC request

80

40

14/07/2009

Blocked call classification

7. Barred network

81

Poor Coverage example

41

14/07/2009

Poor Cell Dominance example

Pilot Pollution example

42

14/07/2009

Example of missing neighbor

(China Mobile, Dongguan City) Mobile,

85

Example of neighbor missing (Ec/Io) (Ec/Io)

86

43

14/07/2009

Example of neighbor missing (RSCP)

87

Example of neighbor missing

88

44

14/07/2009

Corner effect (Phone Event)

89

Corner effect (Ec/Io) (Ec/Io)

90

45

14/07/2009

Corner effect (RSCP)

91

Conclusions
Network QoS allows assess the planning and operating efficiency. Many tools are used to evaluate the network performance from many the sides.
92

46