Register Login

Investigating Unix File System Full

Updated May 18, 2018

Overview

Managing file systems [Unix or Linux] is one of the important jobs for the SAP Administrators. Some of the organizations may have a separate System Administrator to manage OS. Still there are some SAP related file system which require frequent attention by SAP Administrator (e.g., /usr/sap/trans, /usr/sap, /oracle/<sid> file systems). File system full may cause serious consequences to the system even SAP system may come to halt.

Procedure

Login to UNIX for the affected server:

If the file system /usr/sap/PR2 is full

% cd /usr/sap/PR2

Check that this is the file system and that it is really full:

% df –k .

Filesystem            kbytes    used   avail capacity  Mounted on
/dev/vx/dsk/PrdDb2SAPdg/sapPR2
11534336 4614400 867568    99%    /usr/sap/PR2

This shows that the directory is 99% full.

Find which directories are large to find what is taking up the space:

% du –k ./

1895248 ./DVEBMGS01/data
656     ./DVEBMGS01/log/sapccm4x
906824  ./DVEBMGS01/log
233840  ./DVEBMGS01/work/jim030702
210656  ./DVEBMGS01/work/trace_030702121651
15616   ./DVEBMGS01/work/trace_030702133030
15616   ./DVEBMGS01/work/trace_030702133133
20440   ./DVEBMGS01/work/trace_041402074711
20440   ./DVEBMGS01/work/trace_041402074853
12760   ./DVEBMGS01/work/trace_040902105703
1796824 ./DVEBMGS01/work
72      ./DVEBMGS01/sec
4598968 ./DVEBMGS01
4598984 .

 

The above display shows that the /DVEBMGS01/work (1796824) and /DVEBMGS01/data (1895248) directory are using the most space.  These are the directories that you should start with.

Do the following command ‘ls –alt | head’ to see new files in the directories listed above. 
This will show which files have changed recently.  

PRDDB2 > cd DVEBMGS01/work
PRDDB2 > ls -alt |head

total 3812960
-rw-r--r--   1 pr2adm   sapsys       867 Sep 24 16:31 ENQSTAT
drwxr-xr-x   2 pr2adm   sapsys    253952 Sep 24 16:31 .
-rw-rw----   1 pr2adm   sapsys   80260630 Sep 24 16:31 stat
-rw-r--r--   1 pr2adm   sapsys        19 Sep 24 16:23 rslgssta
-rw-rw----   1 pr2adm   sapsys   1830871040 Sep 17 11:10 PAGFIL01
-rw-r--r--   1 pr2adm   sapsys         6 Sep 13 20:34 rslgcpid
-rw-r--r--   1 pr2adm   sapsys         6 Sep 13 20:34 rslgspid
-rw-rw----   1 pr2adm   sapsys     31157 Feb 11  2003 AT010003
-rw-rw----   1 pr2adm   sapsys      4526 Feb 11  2003 AT010002

This shows that the stat and PAGEFILE have been modified most recently.  These files can not be deleted so check the other large directories.

In order to look for any new(less than 24 hours) large files use the following command:

PRDDB2 >  find /usr/sap/PR2/DVEBMGS01 -mtime 0 –size +1000000c -ls

This will show any files that have been modified in the last 24 hours and are larger than 1,00,000 bytes.  Examine this list for files that can potentially be deleted. 

Now look for older(more than 30 days old) files that are larger than 1 megabyte.

PRDDB2 >  find /usr/sap/PR2/DVEBMGS01 -mtime +30 –size +1000000c -ls

11745 16000 -rw-rw----   1 pr2adm   sapsys   16384000 Apr 16 18:37 /usr/sap/PR2/DVEBMGS01/log/ZRPLBATCH_20030416
2841 16000 -rw-rw----   1 pr2adm   sapsys   16384000 May 15 18:31 /usr/sap/PR2/DVEBMGS01/log/RP_ZRPLBATCH_PRDDB2_20030515
2840 16000 -rw-rw----   1 pr2adm   sapsys   16384000 May 30 19:41 /usr/sap/PR2/DVEBMGS01/log/RP_ZRPLBATCH_PRDDB2_20030530
4001 16008 -rw-rw----   1 pr2adm   sapsys   16384000 Jun  8 19:43 /usr/sap/PR2/DVEBMGS01/log/RP_ZRPLBATCH_PRDDB2_200300608.trc
4152 4520 -rw-rw----   1 pr2adm   sapsys   16384000 Jun 10 23:29 /usr/sap/PR2/DVEBMGS01/log/LOG_ZRIO_MTM_P_DEL_0791ZL1TUE.trc
4153 16008 -rw-rw----   1 pr2adm   sapsys   16384000 Jun 10 23:49 /usr/sap/PR2/DVEBMGS01/log/LOG_ZRIO_MTM_P_DEL_0791.trc
5182 131072 -rw-rw----   1 pr2adm   sapsys   134217728 Jun 19 23:20 /usr/sap/PR2/DVEBMGS01/log/POS_PARA_PRDDB2_01_20030619
253 1480 -rw-rw-rw-   1 pr2adm   sapsys    1511552 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/jim030702/dev_ms
271 3120 -rw-r--r--   1 pr2adm   sapsys    3188235 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/jim030702/dev_rfc0
272 7424 -rw-r--r--   1 pr2adm   sapsys    7594622 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/jim030702/dev_rfc1
283 1464 -rw-r--r--   1 pr2adm   sapsys    1498627 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/jim030702/dev_rfc2
337 189568 -rw-r--r--   1 pr2adm   sapsys   194111387 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/jim030702/dev_w1
390 2600 -rw-rw-r--   1 pr2adm   sapsys    2655750 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/jim030702/dev_w58
400 18960 -rwxrwxr-x   1 pr2adm   sapsys   19407144 Apr  7  2001 /usr/sap/PR2/DVEBMGS01/work/jim030702/dw.sapPR2_DVEBMGS01
857 1480 -rw-rw-rw-   1 pr2adm   sapsys    1511552 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/jim030702/dev_ms_jim
149 1480 -rw-rw-rw-   1 pr2adm   sapsys    1511552 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/trace_030702121651/dev_ms
175 3120 -rw-r--r--   1 pr2adm   sapsys    3188235 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/trace_030702121651/dev_rfc0
180 7424 -rw-r--r--   1 pr2adm   sapsys    7594622 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/trace_030702121651/dev_rfc1
206 1464 -rw-r--r--   1 pr2adm   sapsys    1498627 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/trace_030702121651/dev_rfc2
470 189568 -rw-r--r--   1 pr2adm   sapsys   194111387 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/trace_030702121651/dev_w1
523 2600 -rw-rw-r--   1 pr2adm   sapsys    2655750 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/trace_030702121651/dev_w58
548 3120 -rw-r--r--   1 pr2adm   sapsys    3188235 Mar  7  2002 /usr/sap/PR2/DVEBMGS01/work/trace_030702133030/dev_rfc0

The above command shows files that have not been modified in more that 30 days and are larger then 1 megabyte.  As you can see this found files that can be deleted.

 

Delete-able files (this list should be added to as we further define):

*.old files in the /usr/sap/<sid>/<instance>/work directory

*.trg files older then 30 days

data files that are older then 30 days that are in the SAP work directories.

Spool files usinf spad->admin->delete old spool requests

Trace files using sm50 , process->trace->reset files

.dmp files older than 7 days

tempse with sp12

 

Useful UNIX commands:

List files in a directory in order by size:

% ls -l | sort -b +4n |more  

Display the size in kilobytes of the file system that your current default directory is in:

% df –k ./

Display the size in kilobytes of all subdirectories below your current default directory

% du –k ./

Display all file systems in order (smallest to largest) sorted by % full.

% df -k | sed 's/%//g' | sort –bn -k5 

Find files in my current mount point more than 30 days old and larger than 100,000 bytes:

% find ./ -mtime +30 –size +100000c  

Find files in my current mount point that are less than 1 day old and larger than 100,000 bytes:

% find ./ -mtime -0 –size +100000c  

Find list the files and, remove all files that named “*.trg”, and are more than 30 days old:

% find ./ -name “*.trg” –mtime +30 –ls –exec rm {} ;


×