Welcome
Janis Griffin Oracle 19c Automatic
Senior Sales Engineer
Indexing - Inside & Out
Who Am I
Senior Sales Engineer / DBA
[email protected] Twitter® - @DoBoutAnything
– Current – 30+ Years in Oracle®,
DB2®, ASE, SQL Server®,
MySQL®
– DBA and Developer
Specialize in Performance Tuning
Review Database Performance for
Customers Common Question – How do I
tune it?
Agenda
• 19c Automatic Indexing – What is it?
• High Level Steps
– Capture
– Verify
– Decide
– Monitor
– Report
• 19c Automatic Indexing – How It Works
– DBMS_AUTO_INDEX Package
– How to drop Automatic Indexes
• Several Case Studies
3 quest.com | confidential
19c Automatic Indexing – What is it?
• Implements indexes based expert index tuning knowledge
– Identifies ‘candidate indexes’ based on table column usage
– Without DBA involvement
o Except for DBA can set preferences
> View report of indexes and their impact on the application
• Works incrementally
– So needs to be iterative and continuous
– Created as invisible
o Uses ‘SYS_AI’ as the name prefix
– Automatic indexes are tested
o If improved performance – indexes made visible
o If no improvement – indexes are marked unusable
> Later removed
4 quest.com | confidential
19c Automatic Indexing High Level Steps
• No DBA interaction
• All tuning activities
– Auditable via reporting
Capture
Monitor Identify
Online
Verify
Validation
Decide
5 quest.com | confidential
19c Automatic Indexing – How It Works
• Capture
– Captures the application SQL history into a SQL repository
– Includes SQL, plans, bind values, execution statistics, etc. Capture
• Identify Candidate Indexes Monitor Identify
– That may help the newly captured SQL statements
Online
– Creates indexes as unusable invisible indexes Validation
Verify
Decide
o Metadata only
– Drops indexes obsoleted by newly created indexes (logical merge)
• Verify
– Ask optimizer if index candidates will be used for captured SQL statements
– Materialize indexes and run SQL to validate that performance improved
– All verification is done outside application workflow
6 quest.com | confidential
19c Automatic Indexing – How It Works
• Decide
– If performance is better for all statements, indexes are marked visible
– If performance worse for all statements, indexes remain invisible
– If performance worse for some statements
o Indexes are marked visible except for SQL statements that regressed
• Online Validation
– Validation of new indexes continues for other statements online
– Only one of the sessions executing a SQL statement
Capture
o is allowed to use the new indexes
Monitor Identify
• Monitor Online
Validation
Verify
Decide
– Index usage is continuously monitored
– Automatically created indexes will be dropped if not used in a long time
7 quest.com | confidential
19c Automatic Indexing Benefits
• Great for OLTP, OLAP, mixed workloads but critical for OLTP
• Applies to tuned and un-tuned applications
– If tuned
o Existing secondary indexes may be outdated
o Important indexes are missing
o Some secondary indexes can be dropped and auto indexes can be added
– If un-tuned
o Existing indexes support primary and unique key constraints Capture
• Can be used in all stages of application lifecycle Monitor Identify
• Support single and concatenated indexes Online
Validation
Verify
– Function-based indexes Decide
– Compression advanced low
8 quest.com | confidential
19c Automatic Indexing
• Automatic indexing defaults to run in same database as application
• Indexing task consumes CPU, memory and storage
– Resource manager plan limits task to 1 CPU
– DBA can control
o Which temp tablespace is used to build indexes
o Which tablespace and how much space can be used by auto indexing
Production Database
CPU Task
Memory Build
Storage Store
9 quest.com | confidential
Automatic Indexing Requirements
• Feature is only available to Enterprise Edition on Engineered Systems
– Exadata only
– Workaround for testing
o In CDB as sysdba
Alter system set “_Exadata_feature_on”=true scope=spfile;
Shutdown immediate;
Startup
– Unfortunately this is not supported
o Don’t use on production system
10 quest.com | confidential
19c DBMS_AUTO_INDEX Controls Auto Indexing
• Automatic indexing procedures
– CONFIGURE
o AUTO_INDEX_MODE – Turns on, off or report only
> IMPLEMENT - Turns on automatic indexing
> New indexes that improve performance are made visible & used by optimizer
> REPORT ONLY -Turns on automatic indexing
> New indexes remain invisible
> OFF - Turns off automatic indexing (DEFAULT)
o AUTO_INDEX_SCHEMA
> Can include/exclude schemas using ALLOW parameter
> Is case sensitive & can use wildcards
> If NULL, all schemas can use auto index
11 quest.com | confidential
Configure.Auto_Index_Mode Example
12 quest.com | confidential
Configure.Auto_Index_Schema Example
COLUMN parameter_name FORMAT A40
COLUMN parameter_value FORMAT A15
SELECT parameter_name, parameter_value
FROM cdb_auto_index_config
ORDER BY 1, 2;
13 quest.com | confidential
DBMS_AUTO_INDEX.CONFIGURE - Cont.
– CONFIGURE
o AUTO_INDEX_RETENTION_FOR_AUTO
> Number of days (default 373) auto indexes retained after last used date
o AUTO_INDEX_RETENTION_FOR_MANUAL
> Number of days (default NULL) manual indexes retained after last used date
o AUTO_INDEX_REPORT_RETENTION
> Number of days automatic indexing logs are retained before deletion
> Automatic indexing report is based of the logs (Default is 31 days)
o AUTO_INDEX_DEFAULT_TABLESPACE
> Tablespace to use to store auto indexes (Default is NULL)
o AUTO_INDEX_SPACE_BUDGET
> Percentage of tablespace size to use for auto indexes
> Can only be used when using default tablespace is used
14 quest.com | confidential
DBMS_AUTO_INDEX.CONFIGURE - Cont.
– CONFIGURE additional commands
o PARAMETER_VALUE is specific to parameter
> If NULL, setting is assigned a default value
o AUTO_INDEX_COMPRESSION enables/disables advanced compression
> ON for Advanced Low Compression
> OFF for no compression (Default)
o ALLOW for AUTO_INDEX_SCHEMA parameter
> TRUE adds the specified schema to the inclusion list
> FALSE adds the schema to the exclusion list
> NULL remove the schema from the list that it is currently added
15 quest.com | confidential
DBMS_AUTO_INDEX.CONFIGURE Examples
16 quest.com | confidential
DBMS_AUTO_INDEX.CONFIGURE Examples
17 quest.com | confidential
New (CDB/DBA) Views for Auto Indexes
• DBA_AUTO_INDEX_CONFIG *
– Display the current configuration of the automation index
• DBA_AUTO_INDEX_EXECUTIONS
– History of Automatic Indexing task executions
• DBA_AUTO_INDEX_IND_ACTIONS
– Actions performed on automatic indexes (e.g create, rebuild, etc…)
• DBA_AUTO_INDEX_SQL_ACTIONS
– Actions performed on SQL to verify automatic indexes
• DBA_AUTO_INDEX_STATISTICS
– Shows statistics related to automatic indexes
• DBA_AUTO_INDEX_VERIFICATIONS
– Shows statistics about PLAN_HASH_VALUE (original buffer gets, etc…)
18 quest.com | confidential
Additional Views for Auto Indexes
• DBA_ADVISOR_TASKS – new tasks
• DBA_INDEXES – new column (AUTO)
SELECT owner,
index_name,
auto,
index_type,
table_owner,
table_type
FROM dba_indexes
WHERE table_owner = 'TEST’
-- WHERE auto='YES'
ORDER BY owner, index_name;
19 quest.com | confidential
SMB$CONFIG Table
• Shows both documented & undocumented settings for Auto Indexes
20 quest.com | confidential
Other DBMS_AUTO_INDEX Functions
• DROP_SECONDARY_INDEXES
– Deletes all the indexes, except the ones used for constraints
o From a schema or a table
o Example - begin dbms_auto_index.drop_secondary_indexes('SH'); end;
• REPORT_ACTIVITY SELECT DBMS_AUTO_INDEX.report_activity(
activity_start => SYSTIMESTAMP-1,
– Returns a report of automatic indexing operations activity_end => SYSTIMESTAMP,
o Executed during a specific period type => 'TEXT’,
section => 'ALL’)
FROM dual;
declare
report clob :=null;
begin
report :=DBMS_AUTO_INDEX.REPORT_ACTIVITY (
activity_start => TO_TIMESTAMP('2020-01-01', 'YYYY-MM-DD'),
activity_end => TO_TIMESTAMP('2020-01-31', 'YYYY-MM-DD'),
type => 'TEXT',
section => 'SUMMARY',
level => 'BASIC');
dbms_output.put_line(report);
end;
• REPORT_LAST_ACTIVITY - Returns a report of the latest operation
21 quest.com | confidential
19c Automatic Indexing Reporting & Hints
• Each auto index task generates a report
– Reports can be generated via
o DBMS_AUTO_INDEX.REPORT_ACTIVITY function
> Date/Time range
> Format (XML, HTML, Text)
> Level (basic, typical, all)
> Section
> Summary, Index Details,
• Verification Details, Errors, All
• Use hints to control auto indexes
– /*+ USE_AUTO_INDEXES */
– /*+ NO_USE_AUTO_INDEXES */
22 quest.com | confidential
Report_Activity Example
23 quest.com | confidential
Report_Activity Cont.
24 quest.com | confidential
Report_Activity Cont.
25 quest.com | confidential
Other DBMS_AUTO_INDEX Procedures
• REPORT_LAST_ACTIVITY - Returns a report of the latest operation
declare
report clob := null; select dbms_auto_index.report_last_activity() from dual;
begin
report := DBMS_AUTO_INDEX.REPORT_LAST_ACTIVITY (
type => 'TEXT',
section=> 'ALL',
level=> 'TYPICAL');
dbms_output.put_line(report);
end;
– Type can be TEXT (default), HTML or XML
– Section can be SUMMERY, INDEX_DETAILS,VERIFICATION_DETAILS, ERROR or ALL
o Can combined
> SUMMARY + INDEX_DETAILS – shows summary and index_details
> ALL – ERRORS – shows every section except errors
– Level = Basic, Typical or All
26 quest.com | confidential
3 Case Studies
• Tuning Examples
– Used throughout the years with many Oracle versions – 10 & up
– Manual results were compared with the Tuning Advisor suggestions
o Consistent in previous releases
> Advisor usually missed the mark or got close but required additional DBA intervention
• Oracle 19C – Test Automatic Indexing
– Billing Query for a University
– Sale Order Query
– Popular Airline Flights in USA
27 quest.com | confidential
Billing Query for a University
• Slow performance was reported by a customer
– Having trouble with their billing system
– The following query was identified as performing poorly
SELECT s.fname, s.lname, r.signup_date
FROM student s
INNER JOIN registration r ON s.student_id = r.student_id
INNER JOIN class c ON r.class_id = c.class_id
WHERE c.name = 'SQL TUNING'
AND r.signup_date BETWEEN
to_date(:beg_date,'DD-MON-YY') and to_date(:beg_date,'DD-MON-YY') +1
AND r.cancelled = 'N’;
– Table sizes
o Registration – 80,000 rows
o Student – 18,000 rows
o Class – 1000 rows
28 quest.com | confidential
Original Plan
29 quest.com | confidential
Original Performance
30 quest.com | confidential
Auto Indexes Enabled for Schema ‘Test’
31 quest.com | confidential
Review Process of Registration (class_id, canceled)
SELECT a.execution_name, a.table_name,
a.index_name, b.stat_name, a.start_time
FROM dba_auto_index_ind_actions a, dba_auto_index_statistics b
WHERE a.execution_name = b.execution_name
ORDER BY 5,3;
DBA_AUTO_INDEX_VERIFICATIONS
32 quest.com | confidential
Auto Indexes Created
• Shows status of indexes
– 2 indexes are taking up space
33 quest.com | confidential
With Compression
EXEC DBMS_AUTO_INDEX.CONFIGURE('AUTO_INDEX_COMPRESSION','ON');
34 quest.com | confidential
Report_Activity
35 quest.com | confidential
Report_Activity Cont.
36 quest.com | confidential
Auto Index Plan
37 quest.com | confidential
Auto Index on Registration
38 quest.com | confidential
With Auto Index
39 quest.com | confidential
DBA Tuned Query
Create index cl_name
on class (name);
Create index cov_reg
on registration
(class_id,signup_date,
cancelled);
40 quest.com | confidential
Best Performance
41 quest.com | confidential
Did it measure up?
Auto Cost
107 (class)
76 (reg)
create index cov_reg on registration(class_id, signup_date, cancelled)
DBA Tuned Cost
5
42 quest.com | confidential
Sale Order Query
• HammerDB load utility – Slow running query
SELECT c_last, c_first, c_street_1, c_city, c_state, c_zip, Order_line 60,461,709
c_phone, o_entry_d, d_name, ol_delivery_d, ol_quantity, ol_amount
FROM order_line, orders, district, customer, stock
WHERE o_id = ol_o_id Orders 6,046,215
AND o_c_id=c_id
AND s_i_id = ol_i_id District 50
AND d_id = ol_d_id
AND ol_w_id = :B2
AND ol_d_id = :B4
Customer 150,000
AND (ol_o_id < :B3 )
AND ol_o_id >= (:B3 - 20) Stock 500,000
AND s_w_id = :B2
AND s_quantity < :B1
AND d_id = :B4
AND c_last like :B5 ;
43 quest.com | confidential
Existing Indexes
44 quest.com | confidential
Original Execution Plan
45 quest.com | confidential
Original Performance
46 quest.com | confidential
Include SOE Schema for Auto Indexing
SELECT index_name,table_name,
auto,visibility, compression,
segment_created, status
FROM user_indexes
WHERE auto='YES';
47 quest.com | confidential
Automatic Indexes
Total Space: 225m
Visible: 9m
48 quest.com | confidential
New Execution Plan
49 quest.com | confidential
Performance
50 quest.com | confidential
DBA Fine Tunes the Query
• create index orders_i2 on orders(o_id,o_c_id, o_entry_d);
51 quest.com | confidential
Popular Airline Flights in USA
SELECT
o.carrier, uc.description AS carrier_name
,ao.description AS origin_airport,co.Description AS origin_city
,o.fl_date,o.fl_num,o.tail_num
,ad.description AS destination_airport
,cd.Description AS destination_city ,w.Description Day_of_Week
FROM t_ontime o
INNER JOIN L_UNIQUE_CARRIERS uc ON uc.Code = o.UNIQUE_CARRIER
INNER JOIN L_AIRPORT_ID ao ON ao.Code = o.ORIGIN_AIRPORT_ID
INNER JOIN L_AIRPORT_ID ad ON ad.Code = o.DEST_AIRPORT_ID
INNER JOIN L_CITY_MARKET_ID co ON co.Code = o.ORIGIN_CITY_MARKET_ID
INNER JOIN L_CITY_MARKET_ID cd ON cd.Code = o.DEST_CITY_MARKET_ID
INNER JOIN L_WEEKDAYS w ON w.Code = o.DAY_OF_WEEK
WHERE to_date(fl_date,'YYYY-MM-DD') BETWEEN &beg_date AND &end_date
AND co.Description = &city
AND w.Description = &day_of_week;
L_UNIQUE_CARRIERS: 1620
L_AIRPORT_ID: 6438
L_CITY_MARKET_ID: 5823 US DOT - On-time Performance
L_WEEKDAYS: 8
T_ONTIME: 6784044
52 quest.com | confidential
Only Access Path is Full Table Scans
• No Original Indexes
53 quest.com | confidential
Automatic Indexes
• Visible Indexes
54 quest.com | confidential
Automatic Index Report
select DBMS_AUTO_INDEX.REPORT_LAST_ACTIVITY('TEXT','ALL','ALL') from dual
55 quest.com | confidential
New Plan
56 quest.com | confidential
Original Performance
57 quest.com | confidential
Auto Index Performance
58 quest.com | confidential
Summary
• Automatic Indexing can speed up performance
– 19c Optimizer has come along way
• Beware of just turning it on blindly
– Especially in production
– Watch out for baselines
o Created even when you have ‘capture’ turned off
– Auto Indexes can be Invisible / Visible
• Consider using in development / test
– Be cautious using in production
• Control at schema level
• Turn on compression for space savings
59 quest.com | confidential
