Performance Analysis with the Oracle 10g Active Session History

Graham Wood
[email protected]

Sifting through the ASHes

Oracle Corporation

Agenda
Introduction What is ASH Querying ASH data Comparison of ASH and Statspack/AWR Comparison of ASH and SQL trace/tkprof EM use of ASH data Conclusions

Oracle Statistics
Instance level statistics ( AWR, Statspack)

Too little detail Stop short of complete diagnosis Can be collected automatically Too much detail Intrusive. Hard to see big-picture Must be enabled manually Need prior knowledge that problem exists

Trace level statistics (sql_trace)

Oracle Statistics
Solution: Active Session History

Sample session activity in the system including: Session id Wait event SQL id Object Always on for first fault analysis Just right!

Active Session History

Sampled, detailed,non-intrusive activity data Part of Oracle 10g On by default Licensed as part of the Diagnostic pack

Active Session History (ASH)

Samples Active sessions every second

Like doing select * from v$session_wait w/o SQL 2MB per CPU, 5% shared_pool, 2% sga_target Waiting on non-idle event or on CPU 10,000 sessions => 200 active sessions Design goal: one hour activity held in memory

Writes into ASH buffer in SGA memory

Active == Non-idle sessions

Data volume based on activity

Active Session History (ASH)

SQL> select * from v$sgastat where name like 'ASH buffers'; POOL NAME BYTES ------------ -------------------------- ---------shared pool ASH buffers 65011712 SQL> select min(sample_time), max(sample_time) from v$active_session_history; MIN(SAMPLE_TIME) -------------------------------------------------------------------MAX(SAMPLE_TIME) -------------------------------------------------------------------20-FEB-05 10.31.38.615 PM 21-FEB-05 02.39.28.950 AM

Active Session History (ASH)

Query for Melanie Craft Novels Browse and Read Reviews Add item to cart Checkout using one-click

DB Time

Active Session History (ASH)

Query for Melanie Craft Novels Browse and Read Reviews Add item to cart Checkout using one-click

DB Time

Time
7:38:26 7:42:35 7:50:59 7:52:33

SID
213 213 213 213

Module
Book by author Get review id Add to cart One click

SQL ID
qa324jffritcf aferv5desfzs5 hk32pekfcbdfr abngldf95f4de

State
WAITING CPU WAITING WAITING

Event
db file sequential read

buffer busy wait log file sync

ASH: On disk
Captured as part of AWR snapshots

DBA_HIST_ACTIVE_SESS_HISTORY 10 second samples Whenever circular buffer is 66% full No missed data Table is partitioned for easy purging

Takes samples from in-memory ASH

On-demand flush if required

Seven days history by default

Active Session History

V$SESSION V$SESSION_WAIT V$ACTIVE_SESSION_HISTORY DBA_HIST_ACTIVE_SESS_HISTORY

Session state objects

Circular buffer in SGA
(2MB per CPU)

MMON Lite (MMNL)

Every 1 hour or out-of-space

AWR

WRH$_ACTIVE_SESSION_HISTORY

ASH: Challenges Space

Memory Usage

Module, Action, Client_id (~50%) Variable length rows Write 1 out of every 10 samples Direct-path INSERTS

Disk Usage

Log generation

Active Session History

V$SESSION V$SESSION_WAIT V$ACTIVE_SESSION_HISTORY DBA_HIST_ACTIVE_SESS_HISTORY Write 1 out of 10 samples

Session state objects

Circular buffer in SGA
(2MB per CPU)

MMON Lite (MMNL)

Every 1 hour or out-of-space

AWR

Direct-path INSERTS Variable length rows

ASH: Challenges Time

Reader-Writer Concurrency

No Consistent-Read requirement 1 Writer Multiple Readers Readers go unlatched Both V$ view and DBA_HIST view

Indexed on time

Active Session History

V$SESSION V$SESSION_WAIT V$ACTIVE_SESSION_HISTORY DBA_HIST_ACTIVE_SESS_HISTORY Readers go unlatched Indexed on time Indexed on time

Session state objects

MMON Lite (MMNL)

Circular buffer in SGA

(2MB per CPU)

Every 1 hour or out-of-space

Writer goes one direction Readers go the opposite way

AWR

What you can do with it

STATISTICAL analysis of where time was being spent by many different dimension.

What events were taking most time? What was a session doing? What does a SQL statement wait for?

Can decide on dimension after the event!

ASH: Dimensions
Session Waits

Event, P1, P2, P3 Sql_id, Opcode,Plan_hash Object#, File#, Block# Program, Module, Action, Client_id, Service

SQL

Objects

Application

Combinations of the above, CUBEs, ROLLUPs,

Accessing ASH data

Dump to trace file V$ACTIVE_SESSION_HISTORY DBA_HIST_ACTIVE_SESS_HISTORY ASH report EM Diagnostic Pack

Dumping ASH to file

>oradebug setmypid >oradebug dump ashdump 10 >alter session set events 'immediate trace name ashdump level 10';

10 ==> minutes of history you want to dump Generated file can be loaded into database using supplied control file rdbms/demo/ashldr.ctl

V$ACTIVE_SESSION_HISTORY
Gives most recent data first Control C or set pause on is your friend Simpleash.sql

ASH: desc v$active_session_history

Session
Name Null? ----------------------------------------- -------SAMPLE_ID SAMPLE_TIME SESSION_ID SESSION_SERIAL# USER_ID SESSION_TYPE SESSION_STATE QC_SESSION_ID QC_INSTANCE_ID EVENT EVENT_ID EVENT# SEQ# P1 P2 P3 SQL_ID SQL_CHILD_NUMBER SQL_PLAN_HASH_VALUE SQL_OPCODE CURRENT_OBJ# CURRENT_FILE# CURRENT_BLOCK# PROGRAM MODULE ACTION CLIENT_ID SERVICE_HASH WAIT_TIME TIME_WAITED Type ---------------------------NUMBER TIMESTAMP(3) NUMBER NUMBER NUMBER VARCHAR2(10) VARCHAR2(7) NUMBER NUMBER VARCHAR2(64) NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER VARCHAR2(13) NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER VARCHAR2(48) VARCHAR2(48) VARCHAR2(32) VARCHAR2(64) NUMBER NUMBER NUMBER

Wait

SQL Object Application

How to Sift the ASHes

group bys and count(*)s

Proxy for non-idle elapsed time Proportions of actual time spent

Can analyze any time slice More samples More accurate results

ASH: Top SQL

select sql_id, count(*), round(count(*) /sum(count(*)) over (), 2) pctload from v$active_session_history where sample_time > sysdate - 1/24/60 and session_type <> BACKGROUND group by sql_id order by count(*) desc;

Returns most active SQL in the past minute

ASH: Top SQL

SQL_ID 25wtt4ycbtkyz 7umwqvcy7tusf 01vunx6d35khz bdyq2uph07cmp 9y4f9n5hr23yr 0bnc9a5kkf4wn bv1gns48hgxpk gq82c5361nxbq djzkbxr7cm122 b2bakhq4w7rbv 8jydryyvdwcqp 69x6zf5myht7s 2ccawhzy8b7ua 4z5z7xb2g04m6 COUNT(*) 456 123 119 102 73 57 57 57 57 57 57 57 57 55 PCTLOAD 32.95 8.89 8.6 7.37 5.27 4.12 4.12 4.12 4.12 4.12 4.12 4.12 4.12 3.97 ------------- ---------- ----------

ASH: Top IO SQL

select ash.sql_id, count(*) from v$active_session_history ash, v$event_name evt where ash.sample_time > sysdate 1/24/60 and ash.session_state = WAITING and ash.event_id = evt.event_id and evt.wait_class = User I/O group by sql_id order by count(*) desc;

Returns SQL spending most time doing I/Os Similarly, can do Top Sessions, Top Files, Top Objects

DBA_HIST_ACTIVE_SESS_HISTORY
Similar to in-memory ASH but adds

DB_ID INSTANCE_NUMBER SNAP_ID

One sample every 10 seconds

ASH data gotchas

Samples are a proxy for time not for counts Times are sampled times, not statistically valid for avg, min, max Beware of Obj#, File#, Block# (not cleared) Temp file numbers Wait time vs Time waited SQL*Forms RPC bug# 4137362 Time period of data available in V$ACTIVE_SESSION_HISTORY is variable

ASH: Bad SQL

select e.event, e.total_waits - nvl(b.total_waits,0) total_waits, e.time_waited - nvl(b.time_waited,0) time_waited from v$active_session_history b, v$active_session_history e, stats$snapshot sn Where snap_time > sysdate-&1 And e.event not like '%timer' And e.event not like '%message%' And e.event not like '%slave wait%' And e.snap_id = sn.snap_id And b.snap_id = e.snap_id-1 And b.event = e.event And e.total_timeouts > 100 And (e.total_waits - b.total_waits > 100 or e.time_waited - b.time_waited > 100) ;

ASH: Bad SQL

select sum(a.time_waited) total_time from v$active_session_history a, v$event_name b where a.event# = b.event# and sample_time > '21-NOV-04 12:00:00 AM' and sample_time < '21-NOV-04 05:00:00 AM' and b.wait_class = 'User I/O'

ASH: Bad SQL

select sum(a.time_waited) total_time from v$active_session_history a, v$event_name b where a.event# = b.event# and sample_time > '21-NOV-04 12:00:00 AM' and sample_time < '21-NOV-04 05:00:00 AM' and b.wait_class = 'User I/O'

Total time spent waiting on IO?

ASH: Bad SQL

select sum(a.time_waited) total_time from v$active_session_history a, v$event_name b where a.event# = b.event# and sample_time > '21-NOV-04 12:00:00 AM' and sample_time < '21-NOV-04 05:00:00 AM' and b.wait_class = 'User I/O'

Total time spent waiting on IO? Totals sampled IO times

ASH: Bad SQL

select sum(a.time_waited) total_time from v$active_session_history a, v$event_name b where a.event# = b.event# and sample_time > '21-NOV-04 12:00:00 AM' and sample_time < '21-NOV-04 05:00:00 AM' and b.wait_class = 'User I/O'

Total time spent waiting on IO? Totals sampled IO times Assumes that 5 hours history in memory

ASH: Bad SQL

select sess_id,username,program,wait_event,sess_time, round(100*(sess_time/total_time),2) pct_time_waited from (select a.session_id sess_id, decode(session_type,'background',session_type,c.username) username, a.program program, b.name wait_event, sum(a.time_waited) sess_time from sys.v_$active_session_history a, sys.v_$event_name b, sys.dba_users c where a.event# = b.event# and a.user_id = c.user_id and sample_time > '21-NOV-04 12:00:00 AM' and sample_time < '21-NOV-04 05:00:00 AM' and b.wait_class = 'User I/O' group by a.session_id, decode(session_type,'background',session_type,c.username), a.program, b.name), (select sum(a.time_waited) total_time

ASH: WAIT_TIME vs TIME_WAITED

WAIT_TIME

Same as V$SESSION_WAIT 0 WAITING any other value ON CPU

TIME_WAITED Actual time waited for that event Updated later upon event completion

ASH: TIME_WAITED
Remember actual time waited User Session (SID = 3) After Sample 1 Sample 1 Sample 2 Time

Sample
1

Session

State

Event

Wait_time Time_waited
0 0

3 WAITING db file scattered read

After Sample 2, Sample 1 is updated

Sample
1

Session

State

Event

Wait_time Time_waited
0 5ms

3 WAITING db file scattered read

ASH vs AWR/Statpack
Instance wide data Time based data Counts/occurrence data Analyze any time period Detailed session level data Individual Wait event data Sampled data Time based analysis ASH Yes Yes No Yes Yes Yes Yes Yes AWR Yes Yes Yes No No No No Yes

ASH vs AWR
Top 5 Timed Events ~~~~~~~~~~~~~~~~~~ % Total Event Waits Time (s) DB Time Wait Class ------------------------------ ------------ ----------- --------- -------------log file sync 990,495 233,649 43.79 Commit latch: library cache 642,247 157,188 29.46 Concurrency latch: cache buffers chains 133,136 39,747 7.45 Concurrency latch: library cache pin 84,638 22,998 4.31 Concurrency latch free 61,709 20,079 3.76 Other Top Foreground Events ~~~~~~~~~~~~~~~~~~~~~ log file sync latch: library cache latch: cache buffers chains latch free latch: library cache pin

46.01% 23.13% 6.50% 4.63% 2.99%

Commit Concurrency Concurrency Other Concurrency

D E M O N S T R A T I O N

ASH Report

ASH vs SQLtrace/tkprof
ASH Parse/Exec/Fetch breakdown Time based data Counts/occurrence data Detailed session level data Individual Wait event data Complete trace of operations Always on Bind variables available No Yes No Yes Yes No Yes No SQLtrace Yes Yes Yes Yes Yes Yes No Yes

D E M O N S T R A T I O N

ASH Session Report

D E M O N S T R A T I O N

EM Diagnostic Pack

ASH: Top SQL from EM

Wait Class + ASH: I/O Drill-down from EM

jmillistake3.png

ASH: What new in 10gR2

Blocking sid (maybe in 10.1.0.5) XID

Conclusion
ASH data always available Allows instance wide performance analysis Allows detailed session level performance analysis But it is sampled data, so use statistical analysis techniques

QUESTIONS ANSWERS