Table Controlled Partitioning The rules of the game
Kurt Struyf, Competence Partners
Runion du Guide DB2A pour z/OS France Jeudi 31 janvier 2008 Htel Mercure, Paris-La Dfense
Bio Kurt Struyf
Kurt Struyf started his career at a major Belgian bank. Where he was part of the system DBA team. He worked for an outsourcing company where he installed, tuned and migrated DB2 systems for multiple customers. He was also directly involved in the design and tuning of DB2 related applications and structures. Currently Kurt is working as a consultant/contractor for Competence Partners. He has over ten years experience as a (system) DBA and has been installing, migrating, troubleshooting and tuning DB2 systems throughout Europe. Besides his consultancy missions he has been teaching a broad spectrum of DB2 courses through IBM education services, both in Europe and the USA. These courses range from basic courses like DB2 fundamentals, SQL workshops, application programming, database administration workshops, over DB2 application data recovery and application Performance and Tuning, towards more advanced classes like DB2 System Administration, System Performance Analysis, System/ Disaster Recovery Workshop etc He was a speaker at several IDUG European and North America conferences . Hes been a speaker at numerous DB2 regional user groups in Europe.
Agenda
Partitioning Pre-V8 Partitioning during V8 Consequences of table controlled partitioning Partition Management
Add partition Rotate partition Reorg rebalance
Partitioning during V9
3
Before Version 8
Partitioned Partitioned table tablespace Non-Partitioning index1 (NPI1)
Partitioning index
100 200 300
Non-Partitioning index2 (NPI2)
Syntax Before Version 8
CREATE TABLESPACE Z9PARTTS IN ZTRUYKDB NUMPARTS 3 ;
CREATE TABLE TRUYK.Z9PARTTB (COL1 SMALLINT, COL2 CHAR(2)) IN ZTRUYKDB.Z9PARTTS ;
INCOMPLETE SYSIBM.SYSTABLES Column STATUS =I
Syntax Before Version 8
CREATE INDEX TRUYK.Z9PARTI1 ON TRUYK.Z9PARTTB ( COL1 CLUSTER ( PART 1 VALUES ( 100) PART 2 VALUES ( 200) PART 3 VALUES ( 300)) ASC )
SYSIBM.SYSINDEXES Column TYPE = 2
SYSIBM.SYSINDEXPART Column LIMITKEY = 100 Column LIMITKEY = 200 Column LIMITKEY = 300
6
Problem Before Version 8
Partitioned Partitioned table tablespace Non-Partitioning index1 (NPI1)
Partitioning index
100 200
Only one PARTITIONED index 300 Non-Partitioning index2 (NPI2) Has to be the CLUSTERING index Build2 phase
7
Agenda
Partitioning Pre-V8 Partitioning during V8 Consequences of table controlled partitioning Partition Management
Add partition Rotate partition Reorg rebalance
Partitioning during V9
8
You have a choice
INDEX CONTROLLED
CREATE TABLE TRUYK.Z9PARTTB ( EMPNO SMALLINT, FNAME CHAR(10) LASTNAME CHAR(10) ADDRESS CHAR(20) CITY CHAR(10) STATE CHAR(2)) IN ZTRUYKDB.Z9PARTTS ; - - - - - - - - - - - - - CREATE INDEX TRUYK.Z9PARTI1 ON TRUYK.Z9PARTTB ( EMPNO ASC ) CLUSTER PARTITION BY RANGE ( PARTITION 1 ENDING AT ( 100) PARTITION 2 ENDING AT ( 200) PARTITION 3 ENDING AT ( 300))
TABLE CONTROLLED
CREATE TABLE TRUYK.Z9PARTTB ( EMPNO SMALLINT, FNAME CHAR(10) LASTNAME CHAR(10) ADDRESS CHAR(20) CITY CHAR(10) STATE CHAR(2)) PARTITION BY (EMPNO ASC) (PARTITION 1 ENDING AT ( 100), PARTITION 2 ENDING AT ( 200), PARTITION 3 ENDING AT ( 300)) IN ZTRUYKDB.Z9PARTTS ;
COMPLETE
What has changed in Version 8
Table controlled partitioning means : NO INDEX needed to control partitioned table
Any index can become the clustering index. Any index can be PARTIONED. An index is qualified as partioning or secondary.
10
Clustering index
CREATE TABLE TRUYK.Z9PARTTB ( EMPNO SMALLINT, FNAME CHAR(10) LASTNAME CHAR(10) ADDRESS CHAR(20) CITY CHAR(10) STATE CHAR(2)) PARTITION BY (EMPNO ASC) (PARTITION 1 ENDING AT ( 100), PARTITION 2 ENDING AT ( 200), PARTITION 3 ENDING AT ( 300)) IN ZTRUYKDB.Z9PARTTS ; CREATE INDEX TRUYK.Z9PARTI1 ON TRUYK.Z9PARTTB (EMPNO ASC ) PARTITIONED ; CREATE INDEX TRUYK.Z9PARTI2 ON TRUYK.Z9PARTTB (STATE ASC ) CLUSTER
Z9PARTI1
Z9PARTTB
005AL 002CT 050NY 001OH 120AZ 187CA 150NY 111OH 205AL 202AZ 250NY 201TX
Z9PARTI2 AL AZ CA CT IL MD NY OH TX STATE
100
200
300
EMPNO
11
Change Clustering index
Active immediately Reorg recommended
Insert values (301, AAA) Old clustering index (empno) ALTER CLUSTER Insert values (302, AAA)
New clustering index (Name)
Insert values (302, AAA)
12
Partitioning or Secondary
NEW definition of partitioning index
PARTITIONING index = FIRST columns of the index, match the columns of the PARTITION BY clause
EVERY other index is a SECONDARY index
13
Partitioning or Secondary
CREATE TABLE TRUYK.Z9PARTTB ( EMPNO SMALLINT, FNAME CHAR(10) LASTNAME CHAR(10) ADDRESS CHAR(20) CITY CHAR(10) STATE CHAR(2)) PARTITION BY (EMPNO ASC) (PARTITION 1 ENDING AT ( 100), PARTITION 2 ENDING AT ( 200), PARTITION 3 ENDING AT ( 300)) IN ZTRUYKDB.Z9PARTTS ; CREATE INDEX TRUYK.Z9PARTI1 ON TRUYK.Z9PARTTB (EMPNO ASC ) PARTITIONED ; CREATE INDEX TRUYK.Z9PARTI3 ON TRUYK.Z9PARTTB (EMPNO ASC , LASTNAME DESC );
PARTITIONING
CREATE INDEX TRUYK.Z9PARTI4 ON TRUYK.Z9PARTTB (LASTNAME ASC , CITY ASC ) PARTITIONED
SECONDARY
PARTITIONING
CREATE INDEX TRUYK.Z9PARTI2 ON TRUYK.Z9PARTTB (STATE ASC ) CLUSTER SECONDARY
14
Partitioned or Non-partitioned
CREATE TABLE TRUYK.Z9PARTTB ( EMPNO SMALLINT, FNAME CHAR(10) LASTNAME CHAR(10) ADDRESS CHAR(20) CITY CHAR(10) STATE CHAR(2)) PARTITION BY (EMPNO ASC) (PARTITION 1 ENDING AT ( 100), PARTITION 2 ENDING AT ( 200), PARTITION 3 ENDING AT ( 300)) IN ZTRUYKDB.Z9PARTTS ; CREATE INDEX TRUYK.Z9PARTI1 ON TRUYK.Z9PARTTB (EMPNO ASC ) PARTITIONED ; CREATE INDEX TRUYK.Z9PARTI2 ON TRUYK.Z9PARTTB (STATE ASC ) CLUSTER CREATE INDEX TRUYK.Z9PARTI3 ON TRUYK.Z9PARTTB (EMPNO ASC , LASTNAME DESC );
CREATE INDEX TRUYK.Z9PARTI4 ON TRUYK.Z9PARTTB (LASTNAME ASC , CITY ASC ) PARTITIONED
15
Partitioned or Non-partitioned
Z9PARTI1
Z9PARTTB
005AL 002IL 050NY 001OH 120AZ 187CA 150NY 111OH 205AL 202AZ 250NY 201TX
Z9PARTI2 AL AZ CA CT IL MD NY OH TX STATE
Z9PARTI3
001, Presley 002, Sinatra 003, Berry 004, Lewis 121, Cole 122, Martin 299, Cash 300, Holly
Z9PARTI4
Presley, Memphis Sinatra, Chicago Berry, St.-Louis Lewis, Ferriday Cole, Montgomery Martin, Steubenville Cash, Nashville Holly, Lubbock
100
200
300
EMPNO
EMPNO, Lastname
Lastname, City
PARTITIONED
PARTITIONED
16
DPSI
Z9PARTI4
Presley, Memphis Sinatra, Chicago Berry, St.-Louis Lewis, Ferriday Cole, Montgomery Martin, Steubenville Cash, Nashville Holly, Lubbock
Data Partitioned Secondary Index Advantages : - No Build2 phase - No contention during LOAD part - Performance, if partition is known
Lastname, City
SECONDARY
Disadvantages : - Cannot be UNIQUE - Performance, if partition not known - Increased DSMAX - Larger EDMPOOL
PARTITIONED
17
Display statement
18
Agenda
Partitioning Pre-V8 Partitioning during V8 Consequences of table controlled partitioning Partition Management
Add partition Rotate partition Reorg rebalance
Partitioning during V9
19
V8 syntax table controlled
CREATE INDEX PARTITIONED
CREATE TABLE TRUYK.Z9PARTTB ( EMPNO SMALLINT, FNAME CHAR(10) LASTNAME CHAR(10) ADDRESS CHAR(20) CITY CHAR(10) STATE CHAR(2)) PARTITION BY (EMPNO ASC) (PARTITION 1 ENDING AT ( 100), PARTITION 2 ENDING AT ( 200), PARTITION 3 ENDING AT ( 300)) IN ZTRUYKDB.Z9PARTTS ;
ALTER INDEX NOT CLUSTER on the partitioning index ALTER INDEX CLUSTER on the partitioning index DROP PARTITIONING INDEX ALTER TABLE ADD PARTITION ALTER TABLE ROTATE PARTITION ALTER TABLE ALTER PARTITION n CREATE INDEX ENDING AT ...
BE CAREFUL !!
20
Consequence of alter
ALTER INDEX TRUYK.Z9PARTI1 NOT CLUSTER; ---------+---------+---------+---------+---------+---------+---------+---DSNT404I SQLCODE = 20272, WARNING: TABLE SPACE Z9PARTTS HAS BEEN CONVERTED TO USE TABLE-CONTROLLED PARTITIONING INSTEAD OF INDEX-CONTROLLED PARTITIONING, ADDITIONAL INFORMATION: 300 SYSIBM.SYSTABLES
PARTKEYCOLNUM
SYSIBM.SYSINDEXES
0 TYPE P
PARTKEYCOLNUM
TYPE PART 1 PART 2 PART 3
2 LIMITKEY LIMITKEY LIMITKEY 100 200 300
SYSIBM.SYSINDEXPART
PART 1 PART 2 PART 3
LIMITKEY LIMITKEY LIMITKEY
SYSIBM.SYSTABLEPART
PART 1 PART 2 PART 3
LIMITKEY LIMITKEY LIMITKEY
PART 1 PART 2 PART 3
LIMITKEY LIMITKEY LIMITKEY
100 200 XFF
21
Consequence of alter
PART 1 PART 2 PART 3 LIMITKEY LIMITKEY LIMITKEY 100 200 XFF
INSERT INTO Z9PARTTB VALUES (305,'Richie','Vallens', 'some street', 'LosAngeles', 'CA')
Index controlled partitioning
Table controlled partitioning
HIGH LIMIT KEY IS ENFORCED
22
Agenda
Partitioning Pre-V8 Partitioning during V8 Consequences of table controlled partitioning Partition Management
Add partition Rotate partition Reorg rebalance
Partitioning during V9
23
Add Partition
Before V8
Unload all partitions Drop tablespace Create tablespace with more parts Load partitions Full Image Copy
In V8
Alter table
Table Controlled partitioning needed
24
Add Partition
ALTER TABLE ADDPART_TEST ADD PARTITION ENDING AT (500) INCLUSIVE NO partition number
Z9PARTTB
005AL 002CT 050NY 001OH 120AZ 187CA 150NY 111OH 205AL 202AZ 250NY 201TX
A001
100
200
A002 A003
300
CAN NOT specify size
A004
500
25
Recover add Partition
100
005AL 002CT 050NY 001OH 120AZ 187CA 150NY 111OH 205AL 202AZ 250NY 201TX 305AL 302AZ 350NY 301TX 005AL 002CT 050NY 001OH 120AZ 187CA 150NY 111OH 205AL 202AZ 250NY 201TX
100
005AL 002CT 050NY 001OH 120AZ 187CA 150NY 111OH 205AL 202AZ 250NY 201TX FIC
100
200
100
200
200
300
005AL 002CT 050NY 001OH 120AZ 187CA 150NY 111OH 205AL 202AZ 250NY 201TX
200
300
300
500
300
500
500
600
T1
T2 Alter add partition
T3
T4 recover to T2
Alter add partition
26
Index controlled partitioning and add partition (1/3)
100
200
300
005AL 002CT 050NY 001OH 120AZ 187CA 150NY 111OH 305AL 202AZ 250NY 201TX
DSNT408I SQLCODE = -636, ERROR: THE PARTITIONING KEYS FOR PARTITION 4 ARE NOT SPECIFIED IN ASCENDING OR DESCENDING ORDER
PART 1 LIMITKEY PART 2 LIMITKEY PART 3 LIMITKEY
100 200 XFF
T1
T2 Alter add partition ending at 500
T3 Why ?
27
Index controlled partitioning and add partition (2/3)
DSNT404I SQLCODE = 610, WARNING: A CREATE/ALTER ON OBJECT TRUYK.Z9PARTTB HAS PLACED OBJECT IN REORG PENDING
PART 1 LIMITKEY PART 2 LIMITKEY PART 3 LIMITKEY
100 200 XFF
REORP REORP
T5 Alter add partition ALTER TABLE Z9PARTTB ALTER PARTITION 3 ENDING AT ( 300) ending at 500 T3
T4
T6 Reorg tablespace Z9PARTS
28
Index controlled partitioning and add partition (3/3)
BUT !!
DSNT404I SQLCODE = 610, WARNING: A CREATE/ALTER ON OBJECT TRUYK.Z9PARTTB HAS PLACED OBJECT IN REORG PENDING
PART 1 LIMITKEY PART 2 LIMITKEY PART 3 LIMITKEY
100 200 XFF
PART 1 LIMITKEY PART 2 LIMITKEY PART 3 LIMITKEY
100 200 300
Child tables in CHKP
1 row discarded T3 T4 T5 Reorg tablespace Z9PARTS
29
ALTER TABLE Z9PARTTB ALTER PARTITION 3 ENDING AT ( 300)
Agenda
Partitioning Pre-V8 Partitioning during V8 Consequences of table controlled partitioning Partition Management
Add partition Rotate partition Reorg rebalance
Partitioning during V9
30
Rotate Partition
CREATE TABLESPACE Z9ROTATS IN ZTRUYKDB NUMPARTS 3 ;
CREATE TABLE Z9ROTATB ( YEAR SMALLINT, SALES CHAR(10), REGION CHAR(10)) PARTITION BY (YEAR ASC) (PARTITION 1 ENDING AT (2004), PARTITION 2 ENDING AT (2005), PARTITION 3 ENDING AT (2006)) IN ZTRUYKDB.Z9ROTATS
31
Rotate Partition
32
Rotate Partition
ALTER TABLE ROT_PART_TEST ROTATE PARTITION FIRST TO LAST ENDING AT (2007 ) INCLUSIVE RESET
1 1 2005 2 2 2006 3 3 2007
2004
2006
2005
2007
2006
2008
ALTER TABLE ROT_PART_TEST ROTATE PARTITION FIRST TO LAST ENDING AT (2008) INCLUSIVE RESET
33
Rotate Partition
34
Rotate Partition
SELECT DSNUM, LOGICAL_PART, ICTYPE, STYPE FROM SYSIBM.SYSCOPY WHERE DBNAME = 'ZTRUYKDB' AND TSNAME = 'Z9ROTATS' ; ---------+---------+---------+---------+--DSNUM LOGICAL_PART ICTYPE STYPE ---------+---------+---------+---------+--1 1 Q 2 2 Q 3 3 Q 1 1 F 2 2 F ---------+---------+---------+---------+ 3 3 F DSNUM LOGICAL_PART ICTYPE STYPE ---------+---------+---------+---------+ 2 3 A R 1 3 A R 1 1 Q 2 2 Q 3 3 Q 1 1 F 2 2 F 3 3 F 35
Rotate Partition
DB2 issues individual deletes - logging - performance DB2 holds DBD-lock DB2 invalidate plans, packages, statement cache
GOOD IDEA : LOAD PART REPLACE
36
Houston We Have A Problem
NO Point-in-time RECOVERY Possible
37
Agenda
Partitioning Pre-V8 Partitioning during V8 Consequences of table controlled partitioning Partition Management
Add partition Rotate partition Reorg rebalance
Partitioning during V9
38
Reorg Rebalance
ALTER TABLE Z9ROTATB ADD PARTITION ENDING AT (2009 ) ;
ALTER TABLE Z9ROTATB ROTATE PARTITION FIRST TO LAST ENDING AT (2010 ) INCLUSIVE RESET ;
39
Reorg Rebalance
REORG TABLESPACE ZTRUYKDB.Z9ROTATS PART(2:3) SHRLEVEL REFERENCE ;
Expected result reorg part 2, 4, 3
---------+---------+---------+---------+-DSNUM LOGICAL_PART ICTYPE STYPE ---------+---------+---------+---------+-3 4 W 2 2 W
ACTUAL result : reorg of DSNUM 2 and 3
40
Add and Rotate Partition Combined
When combined with add partition keep track of physical dataset
Reorg doesnt solve logical/physical problem
ALTER TABLESPACE PART N -orREORG TABLESPACE PART N
41
Reorg Rebalance
Reorg rebalance allows an automatic more even distribution of data over the partitions New limit keys are determined at reorg time Shrlevel reference is possible
42
Reorg Rebalance
SELECT CARD, DSNUM, LIMITKEY, LOGICAL_PART FROM SYSIBM.SYSTABLEPART WHERE DBNAME = 'ZTRUYKDB' AND TSNAME = 'Z9PARTTS'; ---------+---------+---------+--------+---------+ CARD DSNUM LIMITKEY LOGICAL_PART ---------+---------+---------+--------+---------+ 25 1 100 25 1 200 350 1 300
1 2 3
43
Reorg Rebalance
SELECT CARD, DSNUM, LIMITKEY, LOGICAL_PART FROM SYSIBM.SYSTABLEPART WHERE DBNAME = 'ZTRUYKDB' AND TSNAME = 'Z9PARTTS';
---------+---------+---------+--------+---------+ CARD DSNUM LIMITKEY LOGICAL_PART ---------+---------+---------+--------+---------+
152 130 118 1 1 1 210 216 300 1 2 3
REORG TABLESPACE ZTRUYKDB.Z9PARTTS PART(N:M) REBALANCE SHRLEVEL REFERENCE
44
Reorg Rebalance
ALTER TABLE Z9ROTATB ADD PARTITION ENDING AT (2009 ) ;
ALTER TABLE Z9ROTATB ROTATE PARTITION FIRST TO LAST ENDING AT (2010 ) INCLUSIVE RESET ;
45
Reorg Rebalance
REORG TABLESPACE ZTRUYKDB.Z9ROTATS PART(2:3) REBALANCE SHRLEVEL REFERENCE ;
DSNUGUTC - REORG TABLESPACE ZTRUYKDB.Z9ROTATS PART(2:3) REBALANCE SHRLEVEL REFEERENCE DSNURFIT - PARTITION RANGE NOT CONTIGUOUS - REBALANCE IGNORED DSNUGBAC - UTILITY EXECUTION TERMINATED, HIGHEST RETURN CODE=8
46
Reorg rebalance
Be careful : repartition based on amount of keys. Varchar rows can still cause uneven VSAM size Uneven distribution of keys can still cause uneven VSAM size (e.g. many dupilcate keys)
47
Agenda
Partitioning Pre-V8 Partitioning during V8 Consequences of table controlled partitioning Partition Management
Add partition Rotate partition Reorg rebalance
Partitioning during V9
48
Partitioning during V9
Introduction of Universal Table spaces
Advantages : partition by growth better space management improved mass delete performance immediate of segments after drop table Restrictions : Cannot be used as workfile More space map pages Partition by growth
CREATE TABLESPACE TEST01TS IN TEST01DB USING STOGROUP SG1 DSSIZE 2G MAXPARTITIONS 24 LOCKSIZE ANY SEGSIZE 4; COMMIT;
Questions ?
[email protected] www.cp.be
