Statistics on Partitioned Tables - Part 1

If you've ever worked on large databases that use partitioned and subpartitioned tables, you'll be aware that there are significant challenges in maintaining up-to-date/appropriate statistics. We've encountered a few problems at work recently and I decided it would be an idea to put together a series of posts covering the basics of what can become quite an involved topic because it's not difficult to find yourself going round in circles reading the documentation, Oracle Support Notes, blog posts, forum threads and the rest until you don't know whether you're coming or going!

I'll steer clear of any remotely advanced angle and try to take some time to show simple, practical examples that might be useful to the great unwashed masses (like me). I'm pretty certain that everything I'm going to post has already been written about by the likes of Jonathan Lewis, Christian Antognini, Randolf Geist, Martin Widlake and others, but I want to write it in my own way that I can understand Sometimes I have a feeling when I write certain blog posts that I'm going to be discussing things which are apparently obvious to experienced people but I'm not convinced most people quite understand. I've no idea how many parts there might be because there's no plan here, but I know it's going to end up being too much for one post.

Added later - whilst digging out a link to Martin's blog, I noticed that he's planning a whole DBMS_STATS series soon. Sigh. Keep an eye out for that, because it will be as in-depth as always. I'll stick to the simple stuff here!

This is all on Oracle 10.2.0.4 running on Linux although we have several stats-related patches applied (probably more on those later) and I'll probably run the same tests on my own 11.2.0.1 installation later to identify any differences.

All of the examples will be based on the following table definition

SQL> CREATE TABLE TEST_TAB1
(
  REPORTING_DATE            NUMBER              NOT NULL,
  SOURCE_SYSTEM             VARCHAR2(30 CHAR)   NOT NULL,
  SEQ_ID                    NUMBER              NOT NULL,
  STATUS                    VARCHAR2(1 CHAR)    NOT NULL
)
PARTITION BY RANGE (REPORTING_DATE)
SUBPARTITION BY LIST (SOURCE_SYSTEM)
SUBPARTITION TEMPLATE
  (SUBPARTITION GROT VALUES ('GROT') TABLESPACE TEST_DAT01,
   SUBPARTITION JUNE VALUES ('JUNE') TABLESPACE TEST_DAT01,
   SUBPARTITION HALO VALUES ('HALO')  TABLESPACE TEST_DAT01,
   SUBPARTITION OTHERS  VALUES (DEFAULT)   TABLESPACE TEST_DAT01)
(  
  PARTITION P_20100131 VALUES LESS THAN (20100201) NOLOGGING NOCOMPRESS,  
  PARTITION P_20100201 VALUES LESS THAN (20100202) NOLOGGING NOCOMPRESS,  
  PARTITION P_20100202 VALUES LESS THAN (20100203) NOLOGGING NOCOMPRESS,  
  PARTITION P_20100203 VALUES LESS THAN (20100204) NOLOGGING NOCOMPRESS,  
  PARTITION P_20100204 VALUES LESS THAN (20100205) NOLOGGING NOCOMPRESS,  
  PARTITION P_20100205 VALUES LESS THAN (20100206) NOLOGGING NOCOMPRESS,  
  PARTITION P_20100206 VALUES LESS THAN (20100207) NOLOGGING NOCOMPRESS,  
  PARTITION P_20100207 VALUES LESS THAN (20100208) NOLOGGING NOCOMPRESS  
)
NOCOMPRESS 
NOCACHE
NOPARALLEL
MONITORING;

Table created.

SQL> CREATE UNIQUE INDEX TEST_TAB1_IX1 ON TEST_TAB1
(REPORTING_DATE, SOURCE_SYSTEM, SEQ_ID)
  LOCAL NOPARALLEL COMPRESS 1;

Index created.

So there is a partition per REPORTING_DATE which is sub-partitioned depending on the SOURCE_SYSTEM that sent the data. It's probably worth pointing out at this stage that the table definition and test data does not match that used in the system I'm working on, but is similar enough to illustrate the issues and is pretty similar to several other systems I've seen or worked on in the past. Speaking of test data, I'd better insert some.

SQL> INSERT INTO TEST_TAB1 VALUES (20100201, 'GROT', 1000, 'P');

1 row created.

SQL> INSERT INTO TEST_TAB1 VALUES (20100202, 'GROT', 30000, 'P');

1 row created.

SQL> INSERT INTO TEST_TAB1 VALUES (20100203, 'GROT', 2000, 'P');

1 row created.

SQL> INSERT INTO TEST_TAB1 VALUES (20100204, 'GROT', 1000, 'P');

1 row created.

SQL> INSERT INTO TEST_TAB1 VALUES (20100205, 'GROT', 2400, 'P');

1 row created.

SQL> INSERT INTO TEST_TAB1 VALUES (20100201, 'JUNE', 500, 'P');

1 row created.

SQL> INSERT INTO TEST_TAB1 VALUES (20100201, 'HALO', 700, 'P');

1 row created.

SQL> INSERT INTO TEST_TAB1 VALUES (20100202, 'HALO', 1200, 'P');

1 row created.

SQL> INSERT INTO TEST_TAB1 VALUES (20100201, 'WINE', 400, 'P');

1 row created.

SQL> INSERT INTO TEST_TAB1 VALUES (20100206, 'WINE', 600, 'P');

1 row created.

SQL> INSERT INTO TEST_TAB1 VALUES (20100204, 'WINE', 700, 'P');

1 row created.

SQL> COMMIT;

Commit complete.

With table and data created, I'll gather some statistics using default options and it's probably worth pointing out at this stage that everyone I've spoken to at Oracle is very keen that people should start off with the default options for reasons that will hopefully become apparent.

SQL> exec dbms_stats.gather_table_stats('TESTUSER', 'TEST_TAB1', GRANULARITY => 'DEFAULT');

PL/SQL procedure successfully completed.

So let's see what statistics have been gathered and focus on the simple NUM_ROWS for now.

SQL> select  table_name, global_stats, last_analyzed, num_rows
  2  from user_tables
  3  where table_name='TEST_TAB1'
  4  order by 1, 2, 4 desc nulls last;

TABLE_NAME                     GLO LAST_ANALYZED          NUM_ROWS
------------------------------ --- -------------------- ----------
TEST_TAB1                      YES 10-FEB-2010 16:31:17         11

SQL> select  table_name, partition_name, global_stats, last_analyzed, num_rows
  2  from user_tab_partitions
  3  where table_name='TEST_TAB1'
  4  order by 1, 2, 4 desc nulls last;

TABLE_NAME                     PARTITION_NAME                 GLO LAST_ANALYZED          NUM_ROWS
------------------------------ ------------------------------ --- -------------------- ----------
TEST_TAB1                      P_20100131                     YES 10-FEB-2010 16:31:17          0
TEST_TAB1                      P_20100201                     YES 10-FEB-2010 16:31:17          4
TEST_TAB1                      P_20100202                     YES 10-FEB-2010 16:31:17          2
TEST_TAB1                      P_20100203                     YES 10-FEB-2010 16:31:17          1
TEST_TAB1                      P_20100204                     YES 10-FEB-2010 16:31:17          2
TEST_TAB1                      P_20100205                     YES 10-FEB-2010 16:31:17          1
TEST_TAB1                      P_20100206                     YES 10-FEB-2010 16:31:17          1
TEST_TAB1                      P_20100207                     YES 10-FEB-2010 16:31:17          0  

8 rows selected.

SQL> select  table_name, subpartition_name, global_stats, last_analyzed, num_rows
  2  from user_tab_subpartitions
  3  where table_name='TEST_TAB1'
  4  order by 1, 2, 4 desc nulls last;

TABLE_NAME                     SUBPARTITION_NAME              GLO LAST_ANALYZED          NUM_ROWS
------------------------------ ------------------------------ --- -------------------- ----------
TEST_TAB1                      P_20100131_GROT                NO   
TEST_TAB1                      P_20100131_HALO                NO   
TEST_TAB1                      P_20100131_JUNE                NO   
TEST_TAB1                      P_20100131_OTHERS              NO   
TEST_TAB1                      P_20100201_GROT                NO   
TEST_TAB1                      P_20100201_HALO                NO   

<output snipped ....there are a lot of subpartitions, all missing stats!>

So at the moment the row counts look spot-on and there are Global Statistics on both the Table and the Partitions of the table and no statistics at all on the Subpartitions. First, let's talk about global statistics. There are several good resources kicking around describing global stats so I'll list just a couple here. I always like a documentation reference and although this is the 11.2 documentation and so some of it isn't correct for 10g, I like the very simple mention of global stats given in the first two paragraphs on 13.3.1.3 - global stats are statistics on the table that describe the table as a whole, in addition to the stats on the underlying partitions. The important point is that sometimes the optimiser will use the global stats, sometimes the partition stats and sometimes both, depending on the query.  For those of you with Support access, Note 236935.1 goes into more detail.

However, our example is complicated by the fact that we have subpartitions too. So at this stage we have global stats that describe the table as a whole (including all of the underlying partitions) and global stats on each partition that describe that partition (and all of its underlying subpartitions). At this stage, let's just assume that having global stats is 'a good thing' which is why Oracle's default option is to gather them at the Table and Partition levels. In the next post I'll look at why they're important.

Why no Subpartition stats, then? Well, the optimiser is only going to use stats on subpartitions when it can guarantee that it's going to use a single subpartition and as that's probably less likely than you think, Oracle doesn't collect those stats by default, but is able to use higher level partition stats to guess what's going on at the subpartition level too. However, if you do think your queries are going to be able to drill down to a specific subpartition effectively, you can choose to gather subpartition statistics too. Beware though that, as far as I'm aware, the optimiser won't use subpartition stats at all, prior to 10.2.0.4 so there's no benefit to the additional overhead if you're running an earlier version.

In the next post I'll look at why global stats are both a good and bad thing ....