Vadim Klimov

November 6, 2012 8 minute read

Message Staging and Logging Options in

Advanced Adapter Engine of PI/PO 7.3x/7.4
Follow RSS feed Like
51 Likes 83,435 Views 24 Comments

Link: https://blogs.sap.com/2012/11/06/message-
staging-and-logging-options-in-advanced-
adapter-engine-of-pi-73x/
Intro
One of features that were available in ABAP stack of PI 7.1x systems and that its Java
stack was lacking is customizable behavior of message persistence for various
processing steps. Transition towards Java-only processing in PI brought special
attention to this topic due to major requirements focusing on more precise
monitoring of messages processed in Advanced Adapter Engine. One of very
common use cases is the requirement to check payload of the processed message
before and after mapping execution (for example, in order to identify correctness of
the mapping execution for the particular message executed at runtime). As a result,
several significant and very useful enhancements in this area – namely,
customizable message staging and logging in Advanced Adapter Engine – were
provided in PI 7.3x and extended in PI 7.31.

In this blog, I would like to summarize various message staging and logging options
that are available in Advanced Adapter Engine of PI 7.3x.

Overview
Following message staging and logging options are available in Advanced Adapter
Engine of PI 7.3x:

 Global staging/logging;
 Scenario-specific staging/logging using advanced settings of Integrated
Configuration;
 Scenario-specific logging using adapter module in communication channel.
In PI 7.30, message staging customization is only possible globally for the entire
Advanced Adapter Engine – meaning that all scenarios running in the corresponding
Advanced Adapter Engine are affected by respective staging configuration.
Customizable global message logging and scenario-specific staging/logging options
are not available in PI 7.30 yet.

In PI 7.31, customizable global logging (originally for synchronous messages)

mechanism as well as scenario-specific staging and logging options were introduced
making it possible to customize staging and logging of messages in Advanced
Adapter Engine differently for specific scenarios using advanced settings of
Integrated Configuration or a dedicated adapter module configured in the respective
communication channel.

In SP04 for PI 7.31, functionality of message logging is extended making it possible

to use logging of asynchronous messages.

Before proceeding with discussion of various staging and logging options, let’s firstly
highlight major differences between these two mechanisms of the message
persistence in Advanced Adapter Engine.

Purpose of message staging is persistence of an asynchronous message at a

particular runtime processing step (stage) for later editing/restart of the message
from the persisted version as well as for displaying persisted (staged) message.

Purpose of message logging is persistence of any kind of an

asynchronous/synchronous message at a particular runtime processing step for later
displaying of the persisted (logged) message. Please note that editing or restart of
the logged message is not possible.

Message logging is a more lightweight mechanism in comparison to message staging

since there is less information persisted during logging (only information required for
message displaying) than during staging (all information necessary for message
displaying and restarting) as well as because during staging the persisted message
is put to the queue and retrieved from it for processing steps where staging applies.
Thus, in case of pure monitoring purposes, staging will bring more overhead to
message processing, so logging is recommended in this case.

Redundant staging and logging configured for several (many) processing steps
should be avoided since it brings negative impact to message processing time in
Advanced Adapter Engine as well as additional load on database resulting in
increased execution time of the scenario in PI and increased database growth rate.
 Global staging/logging
Message staging and logging behavior is configured adjusting corresponding
properties of the J2EE service XPI Adapter: XI:

 Property xiadapter.stage.conf – for staging configuration;

 Property xiadapter.logger.conf – for logging configuration.

For both mentioned properties, value syntax is composed by specifying processing

step(s) and staging/logging mode (condition) for each of specified processing step. If
messages should be staged/logged at several processing steps, respective
step/mode pairs are delimited with comma ( , ).

Possible processing steps and modes that are relevant for staging are provided in
tables below:

Processing Description
Step

BI After sender adapter processing, but before inbound schema

validation

VI Before scenario look-up step (check if scenario is local Advanced

Adapter Engine based or dual-stack based involving Integration
Engine)

MS Before mapping step

AM After mapping step, but before outbound schema validation

VO After outbound schema validation

Mode Description

0 MODE_NO_STORE: No message persistence (staging)

1 MODE_STORE_ON_ERROR: Message is persisted only if its next

processing step (stage) throws an exception

2 MODE_STORE_AND_CONTINUE: Message is persisted and processing is

 Mode Description

continued with the next step. If its next processing step throws an
exception, the processor reschedules the message from this storage
and returns

Note: the mode is deprecated in PI 7.31 due to performance

inefficiency during message persistence

3 MODE_STORE_AND_RETURN: Message is persisted and

rescheduled, processing is continued with the next step

For logging, only processing steps BI, MS and AM are relevant.

Mode values that are relevant for logging are derived from ‘basic’ 4-bit binary values
by summarization of respective ‘basic’ binary values and its further conversion to a
decimal value. Relevant ‘basic’ values are enumerated in the table below:

Binary Description
Value

0000 No logging

0001 Asynchronous messages are logged

0010 Synchronous messages are logged

0100 Onlу еrrοr mеѕѕаgеѕ аrе lοggеd

1000 Logging with cοntеnt hіdіng (payload is hidden)

Sample custom configuration of global message staging is provided below:

In this example, asynchronous messages are persisted (staged) in two versions –
before mapping and after mapping.

In Message Monitor, the persisted versions of the message can be observed

(highligted on the screenshot below):

Please also note that during staging, the message was put into the queue and
retrieved it multiple times (corresponding to amount of processing steps for which
staging was configured and should be conducted for the processed message). This
can be noticed when examining audit log of the message:
Similarly, sample custom configuration of global message logging is provided below:

In this example, asynchronous messages are persisted (logged) in two versions, at

the same processing steps as in the global staging example – before mapping and
after mapping.
In Message Monitor, the persisted versions of the message can be observed
(highligted on the screenshot below):

In logging, there is no multiple operations of queue write (put) / read (retrieve) due
to difference in nature of the logging mechanism in comparison to the staging
mechanism:

Absence of additional queue write/read operations results in less overhead caused

by logging than by staging – the effect that was already mentioned earlier in this
blog.
 Scenario-specific staging/logging using advanced
settings of Integrated Configuration
Sometimes it is necessary to apply specific (more or less precise) staging or logging
for the particular scenario – meaning that staging/logging configuration for the
corresponding scenario should differ from the globally defined one (described in the
previous section of the blog). In order to fulfill such a requirement, configuration of
scenario-specific staging and logging can be performed using the tab ‘Advanced
Settings’ of the Integrated Configuration object in PI 7.31. On this tab, the one can
choose at which processing steps staging or logging should be activated for which
modes.

Valid processing steps that can be chosen for staging and logging using advanced
settings of Integrated Configuration are the same as those applicable for global
staging and logging configuration and described earlier.

For staging, following modes are valid:

 None (no staging);

 Store (stage always);
 Store on error (stage only in case of error at subsequent processing step).

For logging, following modes are valid:

 None (no logging);

 Log (log always);
 Log on error (log only in case of error at subsequent processing step);
 Log without payload (log always, payload is hidden);
 Log on error without payload (log only in case of error at subsequent processing
step, payload is hidden).

Sample configuration of combined options of staging and logging of Integrated

Configuration is provided below:
In this example, messages (including their payloads) are persisted (staged) right
after sender adapter processing and are further persisted (logged) before mapping
and after mapping.

In Message Monitor, the persisted (both staged and logged) versions of the message
can be observed (highligted on the screenshot below):
Scenario-specific logging using adapter module in
communication channel
In PI 7.31, a SAP standard adapter module MessageLoggerBean is provided – this
module can be inserted in the adapter module sequence of the corresponding
communication channel.

Fully qualified adapter module name is

localejbs/AF_Modules/MessageLoggerBean .

Adapter module parameters and their valid values are enumerated in the table
below:

Parameter Description Parameter Values

Name

log.conditi Condition (mode) at Integer value derived from following

on which logging should ‘basic’ values:
be activated
 1 – LOG_MODE_ASYNC, asynchronous
messages are logged;
 2 – LOG_MODE_SYNC, synchronous
messages are logged;
 4 – LOG_MODE_ERROR, only error
messages are logged;
 8 – LOG_MODE_HIDE, business content
is hidden

log.locatio Optional parameter Name of the processing step within the

n that specifies adapter module processing sequence. If
processing step within not specified, location default is used.
the adapter module
processing sequence
where the message
should be logged
Sample configuration of the adapter module processing sequence of the
communication channel that utilizes the described adapter module is provided
below:

In this example, asynchronous messages (including their payloads) that are

processed by the receiver communication channel’s SOAP adapter bean are
persisted (logged).

Below is an extract of the audit log of the processed message were log entries
reflecting execution of MessageLoggerBean are highlighted – as seen, logging mode
(controlled by log condition adapter module parameter) is indicated:
In Message Monitor, the persisted version of the message can be observed, version
of the message is controlled by log location adapter module parameter (highligted
on the screenshot below):

Outro
Described staging and logging options can be employed in combination making
staging and logging in Advanced Adapter Engine even more flexible.
 For example, the below persisted versions of the processed message are outcome of
joint usage of:

 Global staging before mapping;

 Scenario-specific logging after mapping for an utilized Integrated Configuration
object;
 Adapter module MessageLoggerBean in a receiver communication channel for
SOAP adapter bean log location.

References
There are also other very nice materials on SCN that cover message staging and
logging functionality available in Advanced Adapter Engine of PI 7.3x – namely, it is
highly recommended to go through following blogs to get additional information on
the topic:

 PI/XI: PI 7.3 message version persistence for ICO integration flows – teaser, the
blog written by Michal Krawczyk: http://scn.sap.com/community/pi-and-soa-
middleware/blog/2010/10/18/pixi-pi-73-message-version-persistence-for-ico-
integration-flows–teaser
 Scenario specific message staging and logging in PI 7.31 AAE, the blog written by
Nageshwar Reddy:
http://scn.sap.com/community/pi-and-soa-middleware/blog/2012/07/26/scenario-
specific-message-staging-and-logging-in-pi-731
 Message Versioning in the Advance Adapter Engine of PI 7.30, the blog written by
William Li:
http://scn.sap.com/community/pi-and-soa-middleware/blog/2012/01/03/message-
versioning-in-the-advance-adapter-engine-of-pi-730





 Alert Moderator
Assigned tags
 SAP Process Integration
 advanced adapter engine
 message logging
 message staging

Related Blog Posts

 Advanced Adapter Engine Extended (AEX) – Installation and Configuration – I

By Former Member, Nov 01, 2012

 Advanced Adapter Engine Extended (AEX) – Installation and Configuration – III

By Former Member, Nov 01, 2012

 SFTP Adapter in PI 7.3x

By Anand Shankar, Jan 29, 2013

Related Questions
 Message is not visible in adapter engine - SAP PI 7.4 (Dual Stack)

By Swanand Joshi, Oct 03, 2017

 Communication channel data provider not registered for selected Adapter Engine PO 7.5

By Sergey Polyuhov, May 08, 2017

 File Content Conversion for SFTP Adapter in PO 7.4

By Sam Maverik, Aug 17, 2016

24 Comments
You must be Logged on to comment or reply to a post.

Former Member

November 6, 2012 at 3:45 pm

Hi Vadim,

Excellent Blog. Difference between staging and logging clearly outlined.

We are on PI 7.30 I can only see xiadapter.stage.conf in NWA and not the
log property.
 Is there any SP/Release restriction.

Many Thanks,

Sudharshan N A

o Like(0)

Vadim Klimov Post author

November 6, 2012 at 4:14 pm

Hi Sudharshan,

Customizable logging isn’t possible in 7.3. In 7.3, only customizable

global staging is available, global customizable logging was introduced
in PI 7.31. Thank you for this comment – I updated the blog with this
remark so that now it shouldn’t be confusing. Strictly speaking,
customizable message persistence was introduced in PI 7.3, but many
significant improvements and nice features in this area were actually
implemented in PI 7.31.

Regards,

Vadim

 Like(0)

Stefan Drechßler

November 9, 2012 at 9:08 am

Hi Vadim,

Good blog!

We have an problem with a async/sync bridge using an adapter module

(AF_Modules/RequestResponseBean) in 7.31. We don’t find the error message
in the log viewer in the nwa, only in the message monitoring of pi we got this
error:
exception caught with cause

com.sap.engine.interfaces.messaging.api.exception.MessagingException:

while trying to invoke the method

com.sap.aii.af.sdk.xi.mo.xmb.DynamicConfiguration.get(java.lang.String,

java.lang.String) of an object loaded from local variable ‘dynConfig’

We are setup your log module in our adapter, but we don’t see the error
message.

Have an idea for us?

By the way check your picture, same are not display correct..

Regards,

Stefan

o Like(0)

Vadim Klimov Post author

November 9, 2012 at 10:11 am

Hi Stefan,

Thank you for pointing to the issue with pictures – I re-published the
blog and now they should be back again.

As for the error you get… MessageLoggerBean is intended to log the

version of the message for the particular step (module) executed in the
adapter module processing sequence – thus, if some error happened
after logged version of the message, this will not be caught by the
logger module. Instead of it, you can use XPI Inspector tool and use
communication channel based inspection to collect details of the faced
error – namely, full callstack that caused this error and also error
details and context. For the error itself, I would propose you to create a
 thread on SCN for it so that we (and other SCN community members)
could try to sort it out and help you in solving the problem.

Regards,

Vadim

 Like(0)

Nageshwar Reddy

November 23, 2012 at 4:27 am

Hi Vadim Klimov,
Very detailed blog. Thanks.

I also had a blog written on staging and logging (PI 731): Here is the link

http://scn.sap.com/community/pi-and-soa-middleware/blog/2012/07/26/
scenario-specific-message-staging-and-logging-in-pi-731

o Like(0)

Vadim Klimov Post author

November 23, 2012 at 6:03 am

Hi Reddy,

This is surely a fair comment, thank you. I updated the blog with
references and included links to SCN materials on message
logging/staging in AAE 7.3x topic.

Regards,

Vadim

 Like(0)

Former Member

May 7, 2013 at 3:43 pm

 Hi Vadim,

Great blog. Very detailed. I just wanted to share something I noticed…

In the case of multiple Receivers or multiple Interfaces after interface determination, the
MS stage will always get staged for each new message id, regardless of global or specific
settings.

In global I had MS=1 and for specific I had MS=None.

In the case of single receiver and single target interface it works as expected.

But when conditions match for multiple receivers or multiple interfaces, the
message gets staged at MS for each split message.

Jonathan

o Like(0)

Saurabh Kumbhare

February 4, 2014 at 10:34 am

Nice Blog..Cheers!!

o Like(0)

Former Member

March 26, 2014 at 1:14 pm

Well explained.

o Like(0)

Pavan kumar

December 1, 2014 at 12:34 pm

Very well Explained.Fantastic blog

Cheers
 Pavan

o Like(0)

Former Member

January 14, 2015 at 5:04 pm

Thanks, excellent blog!

Is there any chance you know me where to find the “Advanced Settings” Tab
of the ICO in an IFlow?

I could edit the generated ICO, but editing generated objects is a very bad
idea.

o Like(0)

Vadim Klimov Post author

January 15, 2015 at 6:19 am

Hello Elias,

You don’t need to change generated iFlow properties in Integration

Directory. In order to access iFlow-specific logging and staging
parameterization, in NWDS, please open the corresponding iFlow, go to
view ‘Properties’ and then to tab ‘Runtime Configuration’:

If you don’t see the the view ‘Properties’ in the SAP Process Integration
Designer perspective, please go to Window > Show View > Other…
and select General > Properties there.

If Properties view looks different than the one provided on the

screenshot above, please ensure that you didn’t select any of sub-
objects used in the iFlow (like business component, communication
channel, mapping, etc.), but the iFlow object is selected (e.g. click on
empty area in canvas of the iFlow representation in Model
Configurator).
 Regards,

Vadim

 Like(0)

Former Member

January 15, 2015 at 8:24 am

Hi Vadim,

you were right, the “Properties” view was not active. Displaying
it solved the problem.

Thank you!

Regards

Elias

 Like(0)

Former Member

April 6, 2015 at 12:55 pm

Excellent Blog…

o Like(0)

Xavier San Sebastián

April 23, 2015 at 2:18 pm

Great blog. Just a question. How we can define the persistence time of a
message as usually defined in SXMB_ADM?

o Like(0)

o
 Xavier San Sebastián

April 23, 2015 at 2:20 pm

Auto responding from How to persist the messages on Adapter

engine?
Asynchronous messages. Retention period is adjusted using
propertiesxiadapter.inbound.persistDuration.default (for inbound messages)
andxiadapter.outbound.persistDuration.default (for outbound messages) of
J2EE service XPI Adapter: XI (com.sap.aii.adapter.xi.svc). Values of both of
them are in milliseconds, the default value is86400000 which is exactly 24 hours.
Please note that in addition to setting these properties, respective archiving and
deletion jobs periodicity should be checked in Adapter Engine.

Synchronous messages. Synchronous messages are not persisted in database and

are only stored in-memory of the Message Store – here total amount of stored
messages matters (only last X messages are stored). Amount of synchronous
messages that are kept in Message Store is regulated using property
messaging.syncMessageRemover.messageCount of J2EE service XPI
Service: Messaging System (com.sap.aii.af.ms.svc). Each given period of
time (regulated by propertymessaging.syncMessageRemover.checkInterval
(in minutes) of the same J2EE service), amount of syncronous messages kept in
Message Store is counted and corresponding messages are cleaned out of it by the
respective thread (so, speaking differently, this property specifies a sleep time for
this thread). Please note here that in the time period between neighbouring
checks, there can be a case when amount of synchronous messages is more
thanmessaging.syncMessageRemover.messageCount: e.g. in case of high
load or large value ofmessaging.syncMessageRemover.checkInterval.

 Like(0)

Martin Babayan

April 23, 2015 at 3:03 pm

Just one additional remark to your message, Xavi.

With 7.31 SP12 (7.40 SP07) configuration of individual/scenario-

specific archiving & deletion retention periods feature is available.
1960448 – Configure message retention period per interface

 Like(0)

S Kumar

April 29, 2015 at 10:11 am

Excellent Blog
 o Like(0)

Former Member

May 22, 2015 at 3:11 pm

Hi Vadim,

thanks for the very detailed and interesting blog.

I just tested the Module MessageLoggerBean in one of our communication

channels (SOAP Adapter, Transport Protocol HTTP Axis).

It works fine when I put it at the beginning of the module order and I see the
entry in the message editor versions.

But when I put the MessagerLoggerBean after the

AF_Adapters/axis/HandlerBean (after module key wssec or trp, were the logon
Information for the called web service is set) the additional logging entry is
not being displayed in the message editor anymore. The message protocol
says that the LoggerBean is processed.

What am I doing wrong?

Best regards,

Stefan

o Like(0)

Vadim Klimov Post author

May 25, 2015 at 8:37 am

Hi Stefan,

Generally speaking, MessageLoggerBean does not depend on specific

adapter modules before/after which it is called. I would suggest to
perform a trace of the affected Axis communication channel (for
example, using XPI Inspector) and get more details regarding execution
of MessageLoggerBean. In the trace, please pay special attention to
entries corresponding to invocation of the method
MessageLoggerBean.performLogging() since actual logging logic is
called (or not called) from there and the method writes important trace
 entries indicating if message logging takes place and if not, then what
the reason for this is.

Regards,

Vadim

 Like(0)

Former Member

May 12, 2017 at 5:28 pm

I am new to PI/PO, I would like to know if we can use log4j type logging to add
customized logging in PO.

o Like(0)

Former Member

May 12, 2017 at 5:30 pm

to finish off my comment, what Iam looking at is add customized logging at

sender channel, receiver channel using modules and use those log info to
external system like splunk where I can view for different purposes.

o Like(0)

Vadim Klimov Post author

May 12, 2017 at 8:37 pm

Hi Ashok,

The entire message staging and logging functionality is based on usage

of specifically dsigned tables as a persistence layer – hence, usage of
alternative logging frameworks is not of much use here.

Generally speaking, application server Java comes with SAP proprietary

implementation of the logger framework (SAP Logging API). There are
certain possibilities to customize its behaviour and utilize it in
conjunction with alternative log destinations – as a result, log records
can be shipped not only to default log and trace files, but to some
 other arbitrary targets (remote servers, messaging systems,
databases, log processing systems, etc.). It shall be noted that
fulfilment of such requirement will require custom development. As an
example for one of such use cases, you can have a look into the
following blog.
In case you need to utilize alternative logging frameworks and their
implementations, it is also possible, but will again require custom
development – you can find examples of usage of alternative logging
frameworks in custom Java applications deployed to SAP application
server Java, but that is not related to staging/logging functionality
described here.

Regards,

Vadim

 Like(0)

Former Member

November 10, 2017 at 3:08 am

I am new to PI/PO,I want to keep synchronous message more longer.I think I

can configue this parameter,but it didn’t work.What’s “active Message
Logging”?

Thanks.