1 General administration task

1.1 Start and stop of an SAP R/3 instance

1.1.1

Starting the R/3 System

1. Start the Central Instance

Operating system UNIX

Logon UNIX shell .

1
<host>> startsap
<sid>adm ( startsap_<host>_<instance no>)
2
3 4 5
startdb script

Central
saposcol
Database Instance
If not started If not started

2. Start additional R/3 instances

 SAP AG 1999

 The operating system user logs on to the UNIX operating system as user <sid>adm.
 To start R/3, run the shell script startsap_<host>_<instance_no> from the home directory of user
<sid>adm. The script startsap_<host>_<instance no> has the alias startsap.
 startsap starts the saposcol process, which is the statistics collector for operating system resource data, if
it is not yet running.
 startsap calls the script startdb, which starts the database if it is not already started.
 startsap then starts the central instance.
 The R/3 System administrator can start additional instances and application servers. To start the instances
independently of the database, use the script startsap.
 startsap has the following options:
 startsap r3: Checks if the database is running; if it is, only the instance is started
 startsap db: Starts only the database
 startsap all: Default entry; starts both the database and the R/3 instance
1.1.2

Starting an R/3 Instance

UNIX shell . Start profile

Instance startup > startsap_<host>_<nr>
sequence

sapstart

Default and instance profile

Processes SE CO MS Disp+work

Services WP … WP gwrd

Connection

Database
 SAP AG 1999

 This graphic displays the R/3 start procedure in more detail.

 Script startsap calls program SAPSTART.
 Program SAPSTART reads the START PROFILE and starts the R/3 components and/or services listed in
/usr/sap/<SID>/SYS/profile/START_<instance>_<hostname>.
 On a central instance, SAPSTART starts the message server, dispatcher, collector, and the sender.
 On a dialog instance, only the sender and the dispatcher are started. The collector and sender are used
to implement the central R/3 System log.
 The dispatcher forks and creates child processes:
 The work processes (dialog, background, spool, update, . . .) are created according to the information
in the profiles /usr/sap/<SID>/SYS/profile/<SID>_<instance>_<hostname> and
/usr/sap/<SID>/SYS/profile/DEFAULT.PFL.
 The gateway reader. This does not depend on the profiles, and it is always started.
 All the work processes except the gateway reader connect to the database.
 To provide a stable startup procedure, the parameter read sequence (also known as the parameter replace
sequence) is defined during startup as follows:
 R/3 processes read the appropriate parameters from a C source in the R/3 kernel
 The default profile /usr/sap/<SID>/SYS/profile/DEFAULT.PFL is read; profile values already defined
in the C source are replaced with the values in the default profile
 The instance profile /usr/sap/<SID>/SYS/profile/<SID>_<instance>_<hostname> is read; profile
values already defined in the default profile or in the C source are replaced with the values defined in
the instance profile
 This procedure ensures that system parameter values reflect the instance profile and the values in the
default profile and the C source.

1.2 R/3 Startup Logs and Traces

1.2.1

R/3 Startup Logs and Traces

$HOME/<sid>adm/startsap_<host>_<instance no.>

$HOME/<sid>adm/startsap_<host>_<instance no.>.log

/sapmnt/<SID>/<Instance><No>/work/ ...

stderr1 … m Standard error files of program SAPSTART

sapstart<m>.trc Trace files of program SAPSTART

Startup log of program SAPSTART

sapstart.log

dev_ms Trace file of the message server

dev_disp Trace file of the dispatcher

Time dev_w0 … n
Trace files of the work processes

 SAP AG 1999

 The R/3 startup scripts log their actions to log files in the home directory of the user <sid>adm.
 R/3 work directories contains trace files and error files for messages relating to the startup of work
processes. There is a work directory for each R/3 instance. The work directory contains information that
may not be found in the R/3 System log.
 The work directory files are initialized in chronological order.
 To define the level of information written to the trace files, set the profile parameter rdisp/TRACE in the
instance profile. The values for this parameter are:
0: Write only errors (no traces)
1: Write error messages and warnings (default)
2: Write error messages and a short trace
3: Write error messages and the complete trace
 These files can be viewed at OS level or in R/3:
At OS level, you can use UNIX command ‘page’, ‘more’, or ‘cat’.
In R/3, you can use transaction AL11.
In R/3, you can use transaction SM50 to see the developer trace for a particular work process: choose
Process Trace Display file (or click on Display file).
1.3 Processes after starting SAP Instance

The process IDs of the various R/3 processes clearly show the R/3 startup procedure.
 sapstart creates the dispatcher, collector, and sender.
 saposcol is started directly from the script startsap.
 The UNIX init process has the process ID 1.

1.3.1 Troubleshooting during startup phase

In case of trouble, please check that all process from SAP and ORACLE are started.

SAP R/3 processes: dw.sap... work-process and R/3 dispatcherprocess

co.sap... global syslog collector process
se.sap... global syslog sender process
ms.sap... message server process
gw.sap... Gateway
...sapstart... R/3 Startup Process
...saposcol SAP OS Collector-Process

ORACLE prozesse: ora_reco<SID> Recover-Prozess

ora_dbwr<SID> Database Writer
ora_db0{1-N]<SID> Globaler Syslog Sender Prozess
ora_lgwr<SID> Log Writer
ora_arch<SID> Archiver
ora_ckpt<SID> Checkpoint
ora_pmon<SID> Prozess Monitor
orasrv Interprozess Communication Server
oracle<SID> Shadow Prozess (mehrere)

You can also use SAP transactions to find the error:

Transaktion: Beschreibung:
SM21 Dient der Untersuchung und der Anzeige der System Logs

SM50 Prozess-Übersicht. Folgende Prozesse müssen da sein:

D Dialog-Prozess
V Update Prozess
E Enqueue Prozess
B Batch Prozess
S Spool Prozess

SM51 Server-Übersicht. Enthält jeweils einen Eintrag pro Instanz

SICK Installation Consistency Check

In case that a startup is not possible you have to analyse TRACE-Files located under:

 /usr/sap/<SID>/DVEBMGS00/work
 Check that /etc/services has read-permission for <SID>adm and ora<SID>

 If startup is not possible due to a database error, try to start the database manualy as followed:

logon as ora<sid>
sqldba lmode=y
SQLDBA> connect internal
SQLDBA> startup

 If the database inconsistent you can try the following steps

SQLDBA> startup mount

SQLDBA> recover database;
SQLDBA> alter database open;

1.3.2 Start SAP R/3 minimal system

You can create an miniamal profile for diagnistic.

Login as <SID>adm. Backup the active instance profile, e.g instance P01

$ cd /usr/sap/P01/SYS/profile
$ cp P01_DVEBMGS00 P01_DVEBMGS00.orig

2. Create the new instance profile. The new profile must minimum contain the following parameter:

Parameter: Wert:

SAPSYSTEMNAME = P01
SAPSYSTEM = 00
rdisp/TRACE =2
rdisp/wp_no_dia =2

3. start SAP R/3 system and check the trace files under

/usr/sap/<SID>/DVEBMGS00/work/

ATTENTION: Please don’t forget to reset the trace-level rdisp/TRACE. Normaly this
parameter is not set and so the default is 1.

With trace level 2 and higher you got a lot of MB’s and can running out of space.

Alternatative you can use the following trace levels:

Das setzen des Parameters rdisp/TRACE auf 2 erhöht den Trace-Level und sorgt dafür, das die Trace-Files
unter /usr/sap/<SID>/DVEBMGS00/work/ signifikant mehr Informationen enthalten. Der Parameter
rdisp/wp_no_dia sorgt dafür, das nur 2 Workprozesse hochgefahren werden und damit das System nicht
so hoch belastet wird.
ACHTUNG: Das Heruntersetzen des TRACE-Levels sollte NIE vergessen werden.
Voreingestellt ist der Parameter rdisp/TRACE in dem Instanz-Profil gar nicht
eingetragen. Das System nimmt dann automatisch den TRACE-Level 1 an. Mit
dem TRACE-Level 2 werden sehr schnell TRACE-Files von mehreren hundert
MB geschrieben.
1.3.3 SAP Entwickler Traces

Für jede Instanz werden im Verzeichnis /usr/sap/<SID>/<INSTANZNAME>/work TRACE-Files

geschrieben. Folgende Files sind vorhanden:

File: Beschreibung:

sapstart.log Enthält Informationen über den Start des R/3 Systems

dev_disp TRACE-File des Dispatcher-Prozess
dev_w<NR.> TRACE-File des jeweiligen Work-Prozess
dev_ms TRACE-File des Message Server-Prozess
dev_st<USER> TRACE-File des saptemu

Der Trace-Level für die dev-Files kann mit dem Profile-Parameter

rdisp/TRACE =1 ( Grundeinstellung)

im Instanz-Profile /usr/sap/<SID>/SYS/profile/<SID>_DVEBMGS00 eingestellt werden. Folgende Level

können angegeben werden:

Wert: Bedeutung:

0 Kein TRACE
1 Nur Fehlermeldungen
2 Voller TRACE

Zur Auswertung der TRACE-Dateien bei einem Fehler, kann folgendes Vorgehen helfen:

1. TRACE-Datei, die zuletzt beschrieben wurde herausfinden mit:

# ll -t

2. Die größte TRACE-Datei herausfinden

3. Herausfinden, welche der TRACE-Dateien den String ERROR enthält mit:

# find . -name "dev*" -exec grep -l "ERROR" {} \;

1.4 Nützliche Transaktionscodes

TCode: Beschreibung:
/nxxxx Terminates current transaction and starts transaction "xxxx"
/n Terminates current transaction /nend Terminates all separate sessions and logs
off
/nex Terminates all separate sessions and logs off immediately
/oxxxx Opens a new sessions and starts transaction "xxxx" inn new session
/o Lists existing sessions and allows deletion or opening of a new session
/h Switches into debugging mode
/hs Switches into debugging mode and activates the debugging of system functions
/$SYNC Buffer Command: Resets all buffers in the system
/$CUA Buffer Command: Resets the CUA buffer of the application server
 /$TAB Buffer Command: Resets the table buffer of the application server
/$NAM Buffer Command: Resets the nametab buffer of the application server
/$DYNP Buffer Command: Resets the screen buffer if the application server
/bdel Deletes the current batch input transaction
/bend Terminates batch unput processing and sets the sessions to "Failed"

Einige zusätzliche Kommandos:

TCode: Beschreibung:
%sc FIND if find is not an option in an SAP screen
%pc SAVE it to a file on your pc
%pri/%prin Will print the current screen
%bc1 Takes you back one screen (Green Arrow)
%bc2 Cancel (Red X)
%bck Yellow Arrow

Ein Screendump vom einem SAP Bildschirm kann mit der Tastenkombination:

AltGr/Druck

erzeugt werden. Innerhalb eines Microsoft Dokumentes wie Word oder Power Point kann es dann wie
gewohnt mit der Kombination:

STRG-V

eingefügt werden.
1.4.1 R/3 Start-Profile

Beim Starten des SAP R/3 Systems werden eine Reihe von Profilen eingelesen. Nachfolgend eine
Übersicht über die Verteilung der verschiedenen Profile im Filesystem:

... usr ...

Jede Instanz hat ein Verzeichnis, in
Hier stehen Daten, auf die sap dem Instanz-Spezifische Laufzeit-
alle Instanzen zugreifen. Daten gespeichert werden
<SID>

SYS <Instanzname> <Instanzname>

gui exe profile global Informationen über alle, von log data work
Prozessen dieser Instanz er-
zeugten syslog-Einträge

Roll u. Paging Daten

SLOGJ Zentrales Syslog eines Appl. Servers

DEFAULT.PFL Systemweiter Profile

Trace u. Error Informationen,
TEMU.PFL Sapgui Profile
Startup-Meldungen und
<SID>_DVEBMGS00 Speicherabzüge der Instanz
R/3 Instanz-Profil. Startet die Instanz,
Definiert die Anzahl der WP´s, Phys. Mem usw.
START_DVEBMGS00
R/3 Start-Profile. Startet die Workprozesse,
Message Server, Application Server und Syslog Collector Daemon

Die Priorisierung dieser Profile ist wie folgt:

1. Höchste Priorität haben Profile in der Kommandozeile. Beispielsweise in

saptemu pf=TEMU.PFL

2. Die nächste Priorität haben die Instanz-Profile mit ihren Parametern, Beispielsweise in
C11_DVEBMGS00_hpcc024

3. Das nächste Profile, das beim Starten immer gelesen wird, ist das zentrale Systemprofile
DEFAULT.PFL

4. Die niedrigste Priorität haben Parameter, die in den C-Sourcen fest vorbelegt wurden
 Oracle - Oracle Directory Structure in R/3

Directory Contains File name examples

dbs SAP and Oracle profiles, init<SID>.ora, init<SID>.dba, init<SID>.sap

bin Oracle executables

saptrace Background (Oracle alert file)

usertrace
sapdata1 Datafiles /btabd1/btabd.data1, system.data1,
. ctrl<SID>.dbf, /btabi1/btabi.data1
. ...
sapdata<n>

sapbackup BRBACKUP, BRRESTORE logs

Profile init<SID>.ora
saparch BRARCHIVE logs, Oracle archive dir ctrl<SID>.dbf log_archive_format = %t_%s
sapcheck SAPDBA logs (-next, -check, -analyze)
sapreorg SAPDBA logs(default),
default compression directory
origlogA Online redo log files log_g101m1.dbf, log_g103m1.dbf
origlogB Online redo log files log_g102m1.dbf, log_g104m1.dbf
mirrlogA Online redo log files log_g101m2.dbf, log_g103m2.dbf
mirrlogB Online redo log files log_g102m2.dbf, log_g104m2.dbf

 SAP AG 1999

 Directory and file names are standardized in the R/3 environment. We recommend that you use the
following standards:
 Tablespace files reside in the sapdata<n> directories
 The online redo log files reside in the origlog and mirrlog directories
 The offline redo log files are written to the saparch directory
 There should be at least 3 copies of the Oracle control file on different disks
 The profile init<SID>.ora configures the Oracle instance, and resides in directory dbs
(NT: database)
 The profile init<SID>.sap configures the backup tools brbackup and brarchive, and resides in
directory dbs (NT: database)
 The profile init<SID>.dba configures the SAPDBA tool, and resides in directory dbs
(NT: database)
 The Oracle alert file is written to directory saptrace/background
 Trace files of the Oracle shadow processes are written to the directory saptrace/usertrace
 During reorganization, export datasets are written to directory sapreorg
 The directories saparch, sapcheck, sapreorg, and sapbackup are used by the SAP database tools.
1.4.2

Workload Statistics (1)

CPU time
Network

Network
Wait Roll Load Processing time Database time
time in time

Response time
Presentation
Server Application Server Database Server

 SAP AG 1999

 Workload time statistics include:

 Response time in milliseconds: Starts when a user request enters the dispatcher queue; ends when the
next screen is returned to the user. The response time does not include the time to transfer from the
screen to the front end.
 Wait time in milliseconds: This is the time a user request sits in the dispatcher queue. It starts when
user request is entered in the dispatcher queue; and ends when the request starts being processed.
 Roll-in time in milliseconds: The amount of time needed to roll user context information into the work
process.
 Load time in milliseconds: The time needed to load from the database and generate objects like ABAP
source code, CUA, and screen information.
 Processing time: This is equivalent to response time minus the sum of wait time, database request
time, load time, roll time, and enqueue time.
 Database request time: Starts when a database request is put through to the database interface; ends
when the database interface has delivered the result.
 CPU time in milliseconds: This is the CPU time used by the R/3 work process
 Roll-wait time and GUI time are covered in the unit Interface Monitoring.
1.4.3

Workload Statistics (2)

SAP Transaction ST03N,

the Workload Monitor,
summarizes statistics
 SAP AG 1999

 To access the Workload Monitor, use Transaction code ST03N. or, from the R/3 initial screen, choose
ToolsAdministration MonitorPerformanceWorkload Analysis. Choose Performance
database and make the selections required by the dialog boxes. The Performance: Workload Overview
for Server is displayed.
 In the “Administrator’s Mode” that is chosen by default the time frame of the workload analysis is the
current day.
 Important for this training:
 Change to “Expert Mode”. Choose under “Detailed Analysis” -> “Last minutes load”
 During a workload exercise, restrict the time period to the time when the exercise is running.
 Under Analysis views, you can access, for example:
 Workload overview - Workload statistics according to work process type
 Transaction Profile - Workload statistics according to transaction.
 Time Profile - Workload statistics according to hour
1.4.4

Initial Analysis Roadmap (1)

Problem symptoms:

Large roll-wait time

Performance
problem?
Large load time

Check:
Yes Large database
All Workload request times
transactions on all servers
affected? (Cont.)

Large CPU times

(Average)
No

Program Processing time

analysis much larger than
CPU time

 SAP AG 1999

 If a problem is detected, the data in the Workload Monitor (Transaction ST03N) can be used as follows to
identify the area of the system where the problem is located.
 First check for general performance problems affecting all transactions. Good general performance is
normally indicated by:
 Wait time < 10% response time
 Main menu (choose Transaction Profile) < 100 ms
 In the Workload Monitor, the following values normally indicate good performance:
 Average roll-in time < 20 ms
 Average roll wait time < 200 ms
 Average load (and generation) time < 10 % of response time (<50 ms)
 Average database request time < 40 % of (response time - wait time)
 Average CPU time < 40 % of (response time - wait time)
 Average CPU time Not much less than processing time
 Average response time - Depends on customer requirements – there is no general rule
1.4.5

Initial Analysis Roadmap (2)

Large roll-wait time Communication problem with GUI

or external system

Large load time Program buffer, CUA buffer, or screen buffer too small

Large database CPU/memory bottleneck on database server, network

request times problems, expensive SQL statements, database locks,
missing indexes, missing statistics, small buffers

Large CPU times

(Average) Expensive ABAP processing, for example, processing
large tables, frequent accessing of R/3 buffers

Processing time CPU bottlenecks, network problems,

much larger than
CPU time communication problems

 SAP AG 1999

 These are some of the more common problems indicated by Workload Monitor statistics.