Posted on

Finding unused indexes

Oracle has a built-in facility to monitor the usage of some objects, including indexes. This is useful for identifying indexes that are not being used, in many cases you can improve DML performance by dropping those indexes. This method is not 100% accurate, some usages of indexes are not caught with this, so you should still be careful with the drops.
The DDL to turn on monitoring is “ALTER INDEX .. MONITORING USAGE” (and NOMONITORING USAGE to turn it off). This command is well documented elsewhere, what I’m able to contribute is a shell script to automate and report on this infrastructure. The script is below, but first a few notes about this process.
  1. The Oracle-supplied view to report about what is being monitored is worse than useless to a DBA. It only works on your current schema, and that’s such a big drawback that I wrote my own view to see all of the schemas.
  2. The script avoids monitoring certain indexes that you will not want to delete anyway, even if you find they are not being used. Indexes owned by default schemas are ignored. Unique indexes are ignored (since they are enforcing uniqueness you will still want them). Only normal and bitmap types are considered.
  3. Please examine the usage instructions, you will want to run this first with -v to create the needed view (mentioned above) in each DB.
  4. The default behavior is to find any indexes that have never been monitored and turn on monitoring, then find any monitored indexes that have been used and turn off monitoring for those, finally it creates a report file of the unused indexes.
EXEC_CMD=”sqlplus -s / as sysdba”
CMD_FILE=index_monitor-${ORACLE_SID}.sql
:> $CMD_FILE

usage() {
echo “Usage: $0 [-r] [-n] [-v]”
echo ”  -r – only report on unused indexes, no changes are made”
echo ”  -n – new indexes, monitor any indexes that have never been monitored”
echo ”  -v – create the view (only needed on the first run)”
exit 1
}

install_view() {
sqlplus -s / as sysdba <<!
CREATE VIEW sys.v\$all_object_usage (owner, index_name, table_name, monitoring, used, start_monitoring, end_monitoring )
AS
SELECT do.owner, io.name, t.name, DECODE (BITAND (i.flags, 65536), 0, ‘NO’, ‘YES’),
DECODE (BITAND (ou.flags, 1), 0, ‘NO’, ‘YES’), ou.start_monitoring, ou.end_monitoring
FROM sys.obj\$ io, sys.obj\$ t, sys.ind\$ i, sys.object_usage ou, dba_objects do
WHERE i.obj# = ou.obj# AND io.obj# = ou.obj# AND t.obj# = i.bo# AND ou.obj# = do.object_id;
EXIT
!
}

# Start monitoring any new indexes
turn_on_monitoring() {
(sqlplus -s / as sysdba <<!
set lines 100
set pages 0
set feed off
PROMPT set pages 0
PROMPT set echo on
SELECT ‘alter index ‘||owner||’.”‘||index_name||'” monitoring usage;’ “Monitoring changes” FROM dba_indexes
WHERE index_type IN (‘NORMAL’, ‘BITMAP’) AND uniqueness = ‘NONUNIQUE’
AND owner NOT LIKE ‘%SYS%’ AND owner NOT LIKE ‘APEX%’ AND owner NOT IN (‘OUTLN’, ‘PUBLIC’, ‘DBSNMP’, ‘XDB’, ‘ORDDATA’)
AND (owner, index_name) NOT IN (SELECT index_owner, index_name FROM dba_ind_columns c
WHERE (c.table_owner, c.table_name, c.column_name, c.column_position) IN
(SELECT c.owner, c.table_name, cc.column_name, cc.position
FROM dba_constraints  c, dba_constraints  r, dba_cons_columns cc, dba_cons_columns rc
WHERE c.constraint_type = ‘R’ AND c.r_owner = r.owner AND c.r_constraint_name = r.constraint_name
AND c.constraint_name = cc.constraint_name AND c.owner = cc.owner AND r.constraint_name = rc.constraint_name
AND r.owner = rc.owner AND cc.position = rc.position
AND c.owner NOT LIKE ‘%SYS%’ AND c.owner NOT LIKE ‘APEX%’ AND c.owner NOT IN (‘OUTLN’, ‘PUBLIC’, ‘DBSNMP’, ‘XDB’, ‘ORDDATA’)))
AND (owner, index_name) NOT IN (SELECT owner, index_name FROM sys.v\$all_object_usage);
prompt exit
EXIT
!
)
}

# Turn off monitoring of used indexes
turn_off_monitoring() {
(sqlplus -s / as sysdba <<!
set lines 100
set pages 0
set feed off
PROMPT set pages 0
PROMPT set echo on
select ‘alter index ‘ || owner || ‘.”‘ || index_name || ‘” nomonitoring usage;’ from sys.v\$all_object_usage where used = ‘YES’ and monitoring = ‘YES’;
prompt exit
EXIT
!
)
}

# Unused Index Report
report() {
export REP_NAME=unused_indexes.lst
tty -s && echo “Unused index report: ${REP_NAME}”
(sqlplus -s / as sysdba <<!
set lines 200
set pages 0
select owner || ‘.’ || index_name || ‘ on ‘ || table_name || ‘ not used since ‘ || to_char(trunc(to_date(start_monitoring,’MM-DD-YYYY HH24:MI:SS’)),’MM/DD/YY’)
from sys.v\$all_object_usage
where used = ‘NO’ and monitoring = ‘YES’ order by trunc(to_date(start_monitoring,’MM-DD-YYYY HH24:MI:SS’)), owner, table_name, index_name;
EXIT
!
) > ${REP_NAME}
}

# Handle parameters
while getopts “:rnv” opt; do
case $opt in
r)
EXEC_CMD=”true”
;;
n)
# only run this if requested, it is slow
turn_on_monitoring > $CMD_FILE
;;
v)
install_view
;;
\?)
echo “Invalid option: -$OPTARG” >&2
usage
;;
: )
echo “Option -$OPTARG requires an argument.” >&2
usage
;;
esac
done

# always run this
turn_off_monitoring >> $CMD_FILE

LNCNT=`wc -l $CMD_FILE|cut -f1 -d” “`
if [ “$LNCNT” -gt 3 ] ; then
more $CMD_FILE
$EXEC_CMD @$CMD_FILE
echo ” ”
fi

# always run the report
report

Posted on

Managing empty tables

Does your database have a lot of empty tables of (potentially) unknown origin? While the returned space is usually insignificant, there are some simple things you can do to help manage these.
An empty table may or may not have a segment in a tablespace. It does not need one, now that we have defered segment creation, so let’s get rid of those. This SQL is helpful with that:
— empty table management
SELECT ‘exec DBMS_STATS.GATHER_TABLE_STATS(”’||owner||”’,”’||table_name||”’);’ stats_cmd,
‘select count(*) from ‘||owner||’.’||table_name||’;’ cnt_cmd, ‘truncate table ‘||owner||’.’||table_name||’ drop all storage;’ trunc_cmd
FROM dba_tables WHERE num_rows = 0 AND temporary = ‘N’ AND nested = ‘NO’ AND segment_created = ‘YES’
AND owner NOT LIKE ‘%SYS%’ AND owner NOT LIKE ‘APEX%’ AND owner NOT IN (‘OUTLN’, ‘PUBLIC’, ‘DBSNMP’, ‘XDB’, ‘ORDDATA’);
There are 3 columns here, the first gathers new stats for the tables. This is very safe and very fast and will eliminate from consideration any tables that actually have rows.
The second column is a set of count commands, you can run these to verify that all the tables are empty, because the third column is DDL to truncate the tables and remove the segments. You want to be careful with this of course, hence the first 2 columns. I would visually check the listing before running it, and of course a backup of the tables is always a good idea. In fact, exporting the tables is an other way to confirm they are truly empty since the expdp logs show the records exported.
Note that if there are foreign key dependencies on any table, the truncate will fail. It’s up to you if it is worth disabling and re-enabling the dependent FK around the truncate.
If these tables were created by your own development process, you might want to give a list of tables to development for possible removal. You can check that the tables are not used by any objects inside the db with the following SQL. This does not prove that no outside code (or dynamic SQL inside PL/SQL) uses the tables, so you still must be cautious and remove these first in an environment where thorough testing takes place.
— empty tables with no “in db” dependencies
— empty tables
SELECT owner, table_name FROM dba_tables
WHERE num_rows = 0 AND temporary = ‘N’ AND NESTED = ‘NO’
AND owner NOT LIKE ‘%SYS%’ AND owner NOT LIKE ‘APEX%’ AND owner NOT IN (‘OUTLN’, ‘PUBLIC’, ‘DBSNMP’, ‘XDB’, ‘ORDDATA’)
MINUS
— empty tables with “in db” dependencies
SELECT referenced_owner, referenced_name FROM dba_dependencies d
WHERE (referenced_owner, referenced_name) IN
(SELECT owner, table_name FROM dba_tables
WHERE num_rows = 0 AND temporary = ‘N’ AND NESTED = ‘NO’ AND owner NOT LIKE ‘%SYS%’ AND owner NOT LIKE ‘APEX%’
AND owner NOT IN (‘OUTLN’, ‘PUBLIC’, ‘DBSNMP’, ‘XDB’, ‘ORDDATA’));
Posted on

Data Uniqueness in Oracle

It is very common for a data model to define a set of columns that must be unique in the table. This is enforced with a primary key, a unique key, or a unique index. These are all slightly different, but obviously related concepts. Any constraint that declares uniqueness (either a PK or UK) needs a unique index to help enforce that. A unique index maintains this by inserting a row in it’s b-tree for each row in the table that has at least one column that is not null. It is a fast operation for an insert or update to check for uniqueness if this index is in place, in fact it’s is simply the result of inserting the new leaf in the b-tree, if it succeeds then the uniqueness constraint has been satisfied.
This implies that the columns in the constraint and the index are an exact match.But what if that is not true? You might think that is an impossible case, but it can happen, and it’s worth monitoring for because of the potentially horrible effects of that index being missing (worst case – a full table scan to verify the uniqueness before insert or update of each record).
Fortunately, the check for this is very simple:
SELECT cc.owner, cc.table_name, c.index_name, cc.column_name, cc.position
FROM dba_cons_columns cc
JOIN dba_constraints c ON (c.owner = cc.owner AND c.constraint_name = cc.constraint_name)
WHERE cc.owner NOT LIKE ‘%SYS%’ AND c.constraint_type IN (‘P’, ‘U’)
MINUS
SELECT table_owner, table_name, index_name, column_name, column_position
FROM sys.dba_ind_columns;

This query finds all the columns that are involved in a unique constraint (a primary key also implies uniqueness, hence the P in the constraint_type filter), then it removes from that set all the columns that are indexed. The results are the index columns that do not match the constraints in column number and order. It may not be necessary to include the position of the columns in this check, but I like to find any discrepancies in order also because it implies a logical mismatch in the design.