JBOSS Administration and Configuration Guide
JBOSS Administration and Configuration Guide
JBoss Community
Administration And Configuration Guide
Copyright © 2009 Red Hat, Inc. This material may only be distributed subject to the terms and
conditions set forth in the Open Publication License, V1.0, (the latest version is presently available at
http://www.opencontent.org/openpub/).
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix, Fedora, the Infinity
Logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
This book is a guide to the administration and configuration of JBoss Enterprise Application Platform
5.0.
What this Book Covers xi
1. Introduction 1
1.1. JBoss Enterprise Application Platform Use Cases .......................................................... 2
iii
Administration And Configuration Guide
6. JBoss AOP 61
6.1. Some key terms ................................................................................................. 61
6.2. Creating Aspects in JBoss AOP .......................................................................... 63
6.3. Applying Aspects in JBoss AOP .......................................................................... 63
6.4. Packaging AOP Applications ............................................................................... 64
6.5. The JBoss AspectManager Service ..................................................................... 66
6.6. Loadtime transformation in the JBoss Enterprise Application Platform Using Sun
JDK .......................................................................................................................... 67
6.7. JRockit ............................................................................................................... 67
6.8. Improving Loadtime Performance in tje JBoss Enterprise Application Platform
Environment .............................................................................................................. 68
6.9. Scoping the AOP to the classloader .................................................................... 68
6.9.1. Deploying as part of a scoped classloader ................................................ 68
6.9.2. Attaching to a scoped deployment ............................................................ 69
7. Transaction Management 71
7.1. Overview ............................................................................................................ 71
7.2. Configuration Essentials ..................................................................................... 71
7.3. Transactional Resources ..................................................................................... 73
7.4. Last Resource Commit Optimization (LRCO) ........................................................ 74
7.5. Transaction Timeout Handling ............................................................................. 74
7.6. Recovery Configuration ....................................................................................... 75
7.7. Troubleshooting .................................................................................................. 75
7.8. Installing JBossTS JTS ....................................................................................... 76
7.9. Installing JBossTS XTS ...................................................................................... 76
7.10. Transaction Management Console ..................................................................... 77
7.11. Experimental Components ................................................................................. 77
7.12. Source code and upgrading .............................................................................. 77
8. Remoting 79
8.1. Background ........................................................................................................ 79
8.2. JBoss Remoting Configuration ............................................................................ 79
8.2.1. MBeans ................................................................................................... 79
8.2.2. POJOs .................................................................................................... 80
8.3. Multihomed servers ............................................................................................ 82
8.4. Address translation ............................................................................................. 83
iv
8.5. Where are they now? ......................................................................................... 83
8.6. Further information. ............................................................................................ 83
9. JBoss Messaging 85
9.1. Configuring JBoss Messaging ............................................................................. 85
9.1.1. Configuring the SecurityStore ................................................................... 85
9.1.2. SecurityStore Attributes ............................................................................ 86
9.2. Configuring the ServerPeer ................................................................................. 86
9.3. Server Attributes ................................................................................................ 89
9.3.1. ServerPeerID ........................................................................................... 89
9.3.2. DefaultQueueJNDIContext ........................................................................ 89
9.3.3. DefaultTopicJNDIContext .......................................................................... 89
9.3.4. PostOffice ............................................................................................... 89
9.3.5. DefaultDLQ ............................................................................................. 89
9.3.6. DefaultMaxDeliveryAttempts ..................................................................... 89
9.3.7. DefaultExpiryQueue ................................................................................. 90
9.3.8. DefaultRedeliveryDelay ............................................................................ 90
9.3.9. MessageCounterSamplePeriod ................................................................. 90
9.3.10. FailoverStartTimeout .............................................................................. 90
9.3.11. FailoverCompleteTimeout ....................................................................... 90
9.3.12. DefaultMessageCounterHistoryDayLimit .................................................. 90
9.3.13. ClusterPullConnectionFactory ................................................................. 90
9.3.14. DefaultPreserveOrdering ........................................................................ 90
9.3.15. RecoverDeliveriesTimeout ...................................................................... 91
9.3.16. SuckerPassword .................................................................................... 91
9.3.17. StrictTCK ............................................................................................... 91
9.3.18. Destinations ........................................................................................... 91
9.3.19. MessageCounters .................................................................................. 91
9.3.20. MessageStatistics .................................................................................. 91
9.3.21. SupportsFailover .................................................................................... 91
9.3.22. PersistenceManager ............................................................................... 91
9.3.23. JMSUserManager .................................................................................. 91
9.3.24. SecurityStore ......................................................................................... 92
9.4. MBean operations of the ServerPeer MBean ........................................................ 92
9.4.1. DeployQueue .......................................................................................... 92
9.4.2. UndeployQueue ....................................................................................... 92
9.4.3. DestroyQueue ......................................................................................... 92
9.4.4. DeployTopic ............................................................................................. 92
9.4.5. UndeployTopic ......................................................................................... 93
9.4.6. DestroyTopic ............................................................................................ 93
9.4.7. ListMessageCountersAsHTML .................................................................. 93
9.4.8. ResetAllMesageCounters ......................................................................... 93
9.4.9. EnableMessageCounters .......................................................................... 93
9.4.10. DisableMessageCounters ....................................................................... 93
9.4.11. RetrievePreparedTransactions ................................................................ 93
9.4.12. ShowPreparedTransactionsAsHTML ....................................................... 93
v
Administration And Configuration Guide
vi
14.2.2. External Load Balancer Architecture ...................................................... 139
14.3. Load-Balancing Policies .................................................................................. 140
14.3.1. Client-side interceptor architecture ........................................................ 140
14.3.2. External load balancer architecture ....................................................... 141
15. Clustering Building Blocks 143
15.1. Group Communication with JGroups ................................................................ 143
15.1.1. The Channel Factory Service ................................................................ 144
15.1.2. The JGroups Shared Transport ............................................................. 145
15.2. Distributed Caching with JBoss Cache ............................................................. 147
15.2.1. The JBoss Enterprise Application Platform CacheManager Service .......... 147
15.3. The HAPartition Service .................................................................................. 150
15.3.1. DistributedReplicantManager Service .................................................... 153
15.3.2. DistributedState Service ....................................................................... 153
15.3.3. Custom Use of HAPartition ................................................................... 154
16. Clustered JNDI Services 155
16.1. How it works .................................................................................................. 155
16.2. Client configuration ......................................................................................... 157
16.2.1. For clients running inside the Enterprise Application Platform .................. 157
16.2.2. For clients running outside the Enterprise Application Platform ................ 159
16.3. JBoss configuration ......................................................................................... 161
16.3.1. Adding a Second HA-JNDI Service ....................................................... 164
17. Clustered Session EJBs 167
17.1. Stateless Session Bean in EJB 3.0 .................................................................. 167
17.2. Stateful Session Beans in EJB 3.0 ................................................................... 168
17.2.1. The EJB application configuration ......................................................... 168
17.2.2. Optimize state replication ..................................................................... 170
17.2.3. CacheManager service configuration ..................................................... 170
17.3. Stateless Session Bean in EJB 2.x .................................................................. 173
17.4. Stateful Session Bean in EJB 2.x .................................................................... 174
17.4.1. The EJB application configuration ......................................................... 174
17.4.2. Optimize state replication ..................................................................... 175
17.4.3. The HASessionStateService configuration ............................................. 175
17.4.4. Handling Cluster Restart ....................................................................... 176
17.4.5. JNDI Lookup Process ........................................................................... 177
17.4.6. SingleRetryInterceptor .......................................................................... 178
18. Clustered Entity EJBs 179
18.1. Entity Bean in EJB 3.0 .................................................................................... 179
18.1.1. Configure the distributed cache ............................................................. 179
18.1.2. Configure the entity beans for cache ..................................................... 182
18.1.3. Query result caching ............................................................................ 184
18.2. Entity Bean in EJB 2.x .................................................................................... 188
19. HTTP Services 191
19.1. Configuring load balancing using Apache and mod_jk ....................................... 191
19.1.1. Download the software ......................................................................... 191
19.1.2. Configure Apache to load mod_jk ......................................................... 192
19.1.3. Configure worker nodes in mod_jk ........................................................ 193
19.1.4. Configuring JBoss to work with mod_jk ................................................. 195
19.2. Configuring HTTP session state replication ...................................................... 195
19.2.1. Enabling session replication in your application ...................................... 196
vii
Administration And Configuration Guide
viii
23.1.8. Buddy Replication ................................................................................ 254
23.2. Deploying Your Own JBoss Cache Instance ..................................................... 255
23.2.1. Deployment Via the CacheManager Service .......................................... 255
23.2.2. Deployment Via a -service.xml File ................................................. 258
23.2.3. Deployment Via a -jboss-beans.xml File .......................................... 259
V. Index 279
Index 281
ix
x
What this Book Covers
The primary focus of this book is the presentation of the standard JBoss Enterprise Application
Platform 5.0 architecture components from both the perspective of their configuration and architecture.
As a user of a standard JBoss distribution you will be given an understanding of how to configure
the standard components. This book is not an introduction to JavaEE or how to use JavaEE
in applications. It focuses on the internal details of the JBoss server architecture and how our
implementation of a given JavaEE container can be configured and extended.
As a JBoss developer, you will be given a good understanding of the architecture and integration
of the standard components to enable you to extend or replace the standard components for your
infrastructure needs. We also show you how to obtain the JBoss source code, along with how to build
and debug the JBoss server.
xi
xii
Chapter 1.
Introduction
JBoss Enterprise Application Platform 5 is built on top of the new JBoss Microcontainer. The JBoss
Microcontainer is a lightweight container that supports direct deployment, configuration and lifecycle of
plain old Java objects (POJOs).
The JBoss Microcontainer project is standalone and replaces the JBoss JMX Microkernel used in the
4.x JBoss Enterprise Application Platforms.
The JBoss Microcontainer integrates nicely with the JBoss Aspect Oriented Programming framework
(JBoss AOP). JBoss AOP is discussed in Chapter 6, JBoss AOP Support for JMX in JBoss Enterprise
Application Platform 5 remains strong and MBean services written against the old Microkernel are
expected to work.
A sample Java EE 5 application that can be run on top of JBoss Enterprise Application Platform
5.0.0.GA and above which demonstrates many interesting technologies is the Seam Booking
Application available with this distribution. This example application makes use of the following
technologies running on JBoss Enterprise Application Platform 5:
• EJB3
• JSF
• Facelets
• Ajax4JSF
• Seam
Many key features of JBoss Enterprise Application Platform 5 are provided by integrating standalone
JBoss projects which include:
• JBoss EJB3 included with JBoss Enterprise Application Platform 5 provides the implementation
of the latest revision of the Enterprise Java Beans (EJB) specification. EJB 3.0 is a deep overhaul
and simplification of the EJB specification. EJB 3.0's goals are to simplify development, facilitate a
test driven approach, and focus more on writing plain old java objects (POJOs) rather than coding
against complex EJB APIs.
• JBoss Messaging is a high performance JMS provider in the JBoss Enterprise Middleware Stack
(JEMS), included with JBoss Enterprise Application Platform 5 as the default messaging provider.
It is also the backbone of the JBoss ESB infrastructure. JBoss Messaging is a complete rewrite of
JBossMQ, which is the default JMS provider for JBoss Enterprise Application Platform 4.2.
• JBoss Cache comes in two flavors: a traditional tree-structured node-based cache, and a
PojoCache, an in-memory, transactional, and replicated cache system that allows users to operate
on simple POJOs transparently without active user management of either replication or persistency
aspects.
• JBossWS 2 is the web services stack for JBoss Enterprise Application Platform 5 providing Java EE
compatible web services, JAXWS-2.x.
1
Chapter 1. Introduction
• JBoss Transactions is the default transaction manager for JBoss Enterprise Application Platform
5. JBoss Transactions is founded on industry proven technology and 18 year history as a leader in
distributed transactions, and is one of the most interoperable implementations available.
• JBoss Web is the Web container in JBoss Enterprise Application Platform 5, an implementation
based on Apache Tomcat that includes the Apache Portable Runtime (APR) and Tomcat native
technologies to achieve scalability and performance characteristics that match and exceed the
Apache Http server.
• Simple web applications with JSPs/Servlets upgrades to JBoss Enterprise Application Platform with
Tomcat Embedded.
• Intermediate web applications with JSPs/Servlets using a web framework such as Struts, Java
Server Faces, Cocoon, Tapestry, Spring, Expresso, Avalon, Turbine.
• Complex web applications with JSPs/Servlets, SEAM, Enterprise Java Beans (EJB), Java
Messaging (JMS), caching etc.
2
Part I. JBoss Enterprise
Application Platform Infrastructure
Chapter 2.
The directory structure of JBoss Enterprise Application Platform 5 resembles that of the 4.x series with
some notable differences:
5
Chapter 2. JBoss Enterprise Application Platform 5 architecture
The org.jboss.system.server.profileservice.ProfileServiceBootstrap is
an implementation of the org.jboss.bootstrap.spi.Bootstrap interface that loads the
deployments associated with the current profile. The {profile-name} is the name of the profile being
loaded and corresponds to the server -c command line argument. The default {profile-name} is
default. The deployers, deploy
6
Part II. JBoss Enterprise Application
Platform 5 Configuration
Chapter 3.
Deployment
Deploying applications on JBoss Enterprise Application Platform is achieved by copy the application
into the JBOSS_HOME/server/default/deploy directory. You can replace default with different
server profiles such as all or minimal. We will cover those later in this chapter. The JBoss Enterprise
Application Platform constantly scans the deploy directory to pick up new applications or any changes
to existing applications. This enables the hot deployment of applications on the fly, while JBoss
Enterprise Application Platform is still running.
• The WAR application archive (e.g., myapp.war) packages a Java EE web application in a JAR file.
It contains servlet classes, view pages, libraries, and deployment descriptors in WEB-INF such as
web.xml, faces-config.xml, and jboss-web.xml etc..
• The EAR application archive (e.g., myapp.ear) packages a Java EE enterprise application in a JAR
file. It typically contains a WAR file for the web module, JAR files for EJB modules, as well as META-
INF deployment descriptors such as application.xml and jboss-app.xml etc.
• The JBoss Microcontainer (MC) beans archive (typical suffixes include, .beans, .deployer) packages
a POJO deployment in a JAR file with a META-INF/jboss-beans.xml descriptor. This format is
commonly used by the JBoss Enterprise Application Platform component deployers.
• The SAR application archive (e.g., myservice.sar) packages a JBoss service in a JAR file. It is
mostly used by JBoss Enterprise Application Platform internal services that have not been updated
to support MC beans style deployments.
• The *-ds.xml file defines connections to external databases. The data source can then be reused
by all applications and services in JBoss Enterprise Application Platform via the internal JNDI.
• You can deploy *-jboss-beans.xml files with MC beans definitions. If you have the approriate
JAR files available in the deploy or lib directories, the MC beans can be deployed using such a
standalone XML file.
• You can deploy *-service.xml files with MBean service definitions. If you have the appropriate
JAR files available in the deploy or lib directories, the MBeans specified in the XML files will be
started. This is the way you deploy many JBoss Enterprise Application Platform internal services
that have not been updated to support POJO style deployment, such as the JMS queues.
• You can also deploy JAR files containing EJBs or other service objects directly in JBoss Enterprise
Application Platform. The list of suffixes that are recognized as JAR files is specified in the conf/
bootstrap/deployers.xml JARStructure bean constructor set.
9
Chapter 3. Deployment
Exploded Deployment
The WAR, EAR, MC beans and SAR deployment packages are really just JAR files
with special XML deployment descriptors in directories like META-INF and WEB-INF.
JBoss Enterprise Application Platform allows you to deploy those archives as expanded
directories instead of JAR files. That allows you to make changes to web pages etc on the
fly without re-deploying the entire application. If you do need to re-deploy the exploded
directory without re-start the server, you can just touch the deployment descriptors (e.g.,
the WEB-INF/web.xml in a WAR and the META-INF/application.xml in an EAR) to
update their timestamps.
Note
The exact contents of the server/[profile name] directory depends on the profile
service implementation and is subject to change as the management layer and embedded
server evolve.
• The minimal profile starts the core server container without any of the enterprise services. It is
a good starting point if you want to build a customized version of JBoss Enterprise Application
Platform that only contains the services you need.
• The default profile is the mostly common used profile for application developers. It supports the
standard Java EE 5.0 programming APIs (e.g., Annotations, JPA, and EJB3).
• The standard profile is the profile that has been tested for JavaEE compliance. The major
differences with the existing configurations is that call-by-value and deployment isolation are
enabled by default, along with support for rmiiiop and juddi (taken from the all config).
• The all profile is the default profile with clustering support and other enterprise extensions.
• The production profile is based on the all profile but optimized for production environments.
• The web profile is a new experimental lightweight configuration created around JBoss Web that
will follow the developments of the JavaEE 6 web profile. Except for the servlet/jsp container
it provides support for JTA/JCA and JPA. It also limits itself to allowing access to the server only
through the http port. Please note that this configuration is not JavaEE certified and will most likely
change in the following releases.
The detailed services and APIs supported in each of those profiles will be discussed throughout.
10
Chapter 4.
Microcontainer
JBoss Enterprise Application Platform 5.0 uses the Microcontainer to integrate enterprise services
together with a Servlet/JSP container, EJB container, deployers and management utilities in order to
provide a standard Java EE environment. If you need additional services then you can simply deploy
these on top of Java EE to provide the functionality you need. Likewise any services that you do not
need can be removed by changing the configuration. You can even use the Microcontainer to do this in
other environments such as Tomcat and GlassFish by plugging in different classloading models during
the service deployment phase.
Since JBoss Microcontainer is very lightweight and deals with POJOs, it can also be used to deploy
services into a Java ME runtime environment. This opens up new possibilities for mobile applications
that can now take advantage of enterprise services without requiring a full JEE application server. As
with other lightweight containers, JBoss Microcontainer uses dependency injection to wire individual
POJOs together to create services. Configuration is performed using either annotations or XML
depending on where the information is best located. Unit testing is made extremely simple thanks to
a helper class that extends JUnit to setup the test environment, allowing you to access POJOs and
services from your test methods using just a few lines of code.
11
Chapter 4. Microcontainer
• Tools represents a variety of tools to assist users with use and development with JBoss
Microcontainer.
• Integr. represents the aop-mc-int module, which handles integration between the JBoss AOP
and JBoss Microcontainer.
• OSGi represents several integration classes that adapt the OSGi module for the Microcontainer.
• Reliance represents two modules: Drools-int and jBPM-int. These modules define Drools
dependencies.
• Deployers load components from from various modules (such as POJOs, JMX, Spring, Java EE)
into the Microcontainer at runtime.
• CL represents the Classloader, a new peer classloader module that handles the OSGi bundle
module.
• VFS represents the Virtual File System. This is an abstract layer used to identify known file system
issues within a single module.
• Kernel defines the core kernel SPI, including bootstrap, configuration, POJO deployments,
dependency, events, bean metadata and bean registry. It contains the following modules:
• Dependency
• Kernel
• AOP-MC-int
• Spring-int
• Guice-int
• Managed represents two modules: managed and metatype. These modules define the base
objects that define the management view of a component.
4.2. Configuration
To configure the Microcontainer bootstrap you can use the JBOSS_HOME/server/
<server_configuration>/conf/bootstrap.xml and JBOSS_HOME/server/
<server_configuration>/conf/bootstrap/*.xml files where <server_configuration>
represents the name of the server profile, for example, all, default, standard, web or minimal. The
bootstrap.xml simply references Microcontainer deployment descriptors that should be loaded in
the indicated order. The current default profile bootstrap.xml references are:
• classloader.xml - the root class loading beans for the peer class loading model
12
Configuration
• MainDeployer : An update of the JMX based MainDeployer from earlier versions to one based
on the Microcontainer and the Virtual File System. Deployer aspects are registered with the
MainDeployer as an ordered list via inject of the deployers property.
• JARDeployer : This bean is a structural deployment aspect which handles the legacy nested
deployment behavior of adding non-deployable jars to the current deployment classpath.
• FileStructure : this bean is a structural deployment aspect which recognizes well know deployment
file types specified by FileManager and suffix.
• SARDeployer : this bean is a port of the legacy JMX SARDeployer. It handles the legacy *-
service.xml style of mbean deployment descriptors and maps this into a ServiceDeployment
POJO.
• JMXKernel : Manages the instantiation of a JMX kernel and MBeanServer in the jboss domain. It is
used by the SARDeployer. It will be used by other management deployment aspects in the future
to expose kernel beans via JMX.
• HDScanner : A bean that queries the profile service for changes in deploy directory contents and
redeploys updated content, undeploys removed content, and add new deployment content to the
profile service.
13
Chapter 4. Microcontainer
4.3. References
More information on the JBoss Microcontainer project can be obtained from http://www.jboss.org/
1
jbossmc/" .
1
http://www.jboss.org/jbossmc/
14
Chapter 5.
Web Services
Web services are a key contributing factor in the way Web commerce is conducted today. Web
services enable applications to communicate by sending small and large chunks of data to each other.
A web service is essentially a software application that supports interaction of applications over a
computer network or the world wide web. Web services usually interact through XML documents
that map to an object, computer program, business process or database. To communicate, an
application sends a message in XML document format to a web service which sends this message
to the respective programs. Responses may be received based on requirements, the web service
receives and then sends them in XML document format to the required program or applications. Web
services can be used in many ways, examples include supply chain information management and
business integration.
JBossWS is a web service framework included as part of the JBoss Enterprise Application Platform.
It implements the JAX-WS specification that defines a programming model and run-time architecture
for implementing web services in Java, targeted at the Java Platform, Enterprise Edition 5 (Java EE 5).
Even though JAX-RPC is still supported (the web service specification for J2EE 1.4), JBossWS does
put a clear focus on JAX-WS.
Other benefits exist for enterprise systems that incorporate web service technologies for internal
heterogenous subsystems communication as web service interoperability boosts service reuse and
composition. Web services eliminates the need to rewrite whole functionalities because they were
developed by another enterprise department using a different software language.
Nowadays they are meant to be used for loosely-coupled coarse-grained communication, message
(document) exchange. Recent times has seen many specifications (WS-*) discussed and finally
approved to establish standardized ws-related advanced aspects, including reliable messaging,
message-level security and cross-service transactions. Web service specifications also include the
notion of registries to collect service contract references, to easily discover service implementations.
This all means that the web services technology platform suits complex enterprise communication and
is not simply the latest way of doing remote procedure calls.
5.3. Document/Literal
With document style web services two business partners agree on the exchange of complex
business documents that are well defined in XML schema. For example, one party sends a document
15
Chapter 5. Web Services
describing a purchase order, the other responds (immediately or later) with a document that describes
the status of the purchase order. The payload of the SOAP message is an XML document that can be
validated against XML schema. The document is defined by the style attribute on the SOAP binding.
With document style web services the payload of every message is defined by a complex type in XML
schema.
<complexType name='concatType'>
<sequence>
<element name='String_1' nillable='true' type='string'/>
<element name='long_1' type='long'/>
</sequence>
</complexType>
<element name='concat' type='tns:concatType'/>
<message name='EndpointInterface_concat'>
<part name='parameters' element='tns:concat'/>
</message>
<message name='EndpointInterface_concat'>
<part name='parameters' type='tns:concatType'/>
</message>
@WebService
@SOAPBinding(parameterStyle = SOAPBinding.ParameterStyle.BARE)
public class DocBareServiceImpl
16
Document/Literal (Wrapped)
{
@WebMethod
public SubmitBareResponse submitPO(SubmitBareRequest poRequest)
{
...
}
}
The trick is that the Java beans representing the payload contain JAXB annotations that define how
the payload is represented on the wire.
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "SubmitBareRequest", namespace="http://
soapbinding.samples.jaxws.ws.test.jboss.org/", propOrder = { "product" })
@XmlRootElement(namespace="http://
soapbinding.samples.jaxws.ws.test.jboss.org/", name = "SubmitPO")
public class SubmitBareRequest
{
@XmlElement(namespace="http://
soapbinding.samples.jaxws.ws.test.jboss.org/", required = true)
private String product;
...
}
@WebService
public class DocWrappedServiceImpl
{
@WebMethod
@RequestWrapper (className="org.somepackage.SubmitPO")
@ResponseWrapper (className="org.somepackage.SubmitPOResponse")
public String submitPO(String product, int quantity)
{
...
}
}
Note
With JBossWS the request and response wrapper annotations are not required, they will
be generated on demand using sensible defaults.
17
Chapter 5. Web Services
5.6. RPC/Literal
With RPC there is a wrapper element that names the endpoint operation. Child elements of the RPC
parent are the individual parameters. The SOAP body is constructed based on some simple rules:
• The port type operation name defines the endpoint method name
With RPC style web services the portType names the operation (i.e. the java method on the endpoint)
<portType name='EndpointInterface'>
<operation name='echo' parameterOrder='String_1'>
<input message='tns:EndpointInterface_echo'/>
<output message='tns:EndpointInterface_echoResponse'/>
</operation>
</portType>
<message name='EndpointInterface_echo'>
<part name='String_1' type='xsd:string'/>
</message>
<message name='EndpointInterface_echoResponse'>
<part name='result' type='xsd:string'/>
</message>
18
RPC/Encoded
Note
There is no complex type in XML schema that could validate the entire SOAP message
payload.
@WebService
@SOAPBinding(style = SOAPBinding.Style.RPC)
public class JSEBean01
{
@WebMethod
@WebResult(name="result")
public String echo(@WebParam(name="String_1") String input)
{
...
}
}
The element names of RPC parameters/return values may be defined using the JAX-WS
Annotations#javax.jws.WebParam and JAX-WS Annotations#javax.jws.WebResult respectively.
5.7. RPC/Encoded
1 2
SOAP encodeding style is defined by the infamous chapter 5 of the SOAP-1.1 specification. It has
3
inherent interoperability issues that cannot be fixed. The Basic Profile-1.0 prohibits this encoding
4
style in 4.1.7 SOAP encodingStyle Attribute . JBossWS has basic support for RPC/Encoded that is
provided as is for simple interop scenarios with SOAP stacks that do not support literal encoding.
Specifically, JBossWS does not support:-
• element references
@WebService
1
http://www.w3.org/TR/2000/NOTE-SOAP-20000508/#_Toc478383512
2
http://www.w3.org/TR/2000/NOTE-SOAP-20000508/
3
http://www.ws-i.org/Profiles/BasicProfile-1.0-2004-04-16.html
4
http://www.ws-i.org/Profiles/BasicProfile-1.0-2004-04-16.html#refinement16448072
19
Chapter 5. Web Services
@SOAPBinding(style = SOAPBinding.Style.RPC)
public class JSEBean01
{
@WebMethod
public String echo(String input)
{
...
}
}
<web-app ...>
<servlet>
<servlet-name>TestService</servlet-name>
<servlet-class>org.jboss.test.ws.jaxws.samples.jsr181pojo.JSEBean01</
servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>TestService</servlet-name>
<url-pattern>/*</url-pattern>
</servlet-mapping>
</web-app>
<war warfile="${build.dir}/libs/jbossws-samples-jsr181pojo.war"
webxml="${build.resources.dir}/samples/jsr181pojo/WEB-INF/web.xml">
<classes dir="${build.dir}/classes">
<include name="org/jboss/test/ws/samples/jsr181pojo/JSEBean01.class"/
>
</classes>
</war>
Note
Only the endpoint implementation bean and web.xml file are required.
20
EJB3 Stateless Session Bean (SLSB)
http://yourhost:8080/jbossws/services
It is also possible to generate the abstract contract off line using jboss tools. For details of that see
5
#Top Down (Using wsconsume)
@Stateless
@Remote(EJB3RemoteInterface.class)
@RemoteBinding(jndiBinding = "/ejb3/EJB3EndpointInterface")
@WebService
@SOAPBinding(style = SOAPBinding.Style.RPC)
public class EJB3Bean01 implements EJB3RemoteInterface
{
@WebMethod
public String echo(String input)
{
...
}
}
Above you see an EJB-3.0 stateless session bean that exposes one method both on the remote
interface and as an endpoint operation.
<jar jarfile="${build.dir}/libs/jbossws-samples-jsr181ejb.jar">
<fileset dir="${build.dir}/classes">
<include name="org/jboss/test/ws/samples/jsr181ejb/EJB3Bean01.class"/
>
<include name="org/jboss/test/ws/samples/jsr181ejb/
EJB3RemoteInterface.class"/>
</fileset>
</jar>
21
Chapter 5. Web Services
http://yourhost:8080/jbossws/services
It is also possible to generate the abstract contract offline using JbossWS tools. For details of that
7
please see #Top Down (Using wsconsume)
Java SEIs provide a high level Java-centric abstraction that hides the details of converting between
Java objects and their XML representations for use in XML-based messages. However, in some
cases it is desirable for services to be able to operate at the XML message level. The Provider
interface offers an alternative to SEIs and may be implemented by services wishing to work at the
XML message level.
A Provider based service instance’s invoke method is called for each message received for the
service.
@WebServiceProvider
@ServiceMode(value = Service.Mode.PAYLOAD)
public class ProviderBeanPayload implements Provider<Source>
{
public Source invoke(Source req)
{
// Access the entire request PAYLOAD and return the response PAYLOAD
}
}
Service.Mode.PAYLOAD is the default and does not have to be declared explicitly. You can also use
Service.Mode.MESSAGE to access the entire SOAP message (for example, with MESSAGE the
Provider can also see SOAP Headers)
5.15. WebServiceContext
The WebServiceContext is treated as an injectable resource that can be set at the time an endpoint
is initialized. The WebServiceContext object will then use thread-local information to return the
correct information regardless of how many threads are concurrently being used to serve requests
addressed to the same endpoint object.
@WebService
public class EndpointJSE
{
@Resource
WebServiceContext wsCtx;
7
http://www.jboss.org/community/wiki/JBossWS-JAX-WSTools#TopDown_Using_wsconsume
22
Web Service Clients
@WebMethod
public String testGetMessageContext()
{
SOAPMessageContext jaxwsContext =
(SOAPMessageContext)wsCtx.getMessageContext();
return jaxwsContext != null ? "pass" : "fail";
}
...
@WebMethod
public String testGetUserPrincipal()
{
Principal principal = wsCtx.getUserPrincipal();
return principal.getName();
}
@WebMethod
public boolean testIsUserInRole(String role)
{
return wsCtx.isUserInRole(role);
}
}
5.16.1. Service
Service is an abstraction that represents a WSDL service. A WSDL service is a collection of related
ports, each of which consists of a port type bound to a particular protocol and available at a particular
endpoint address.
For most clients, you will start with a set of stubs generated from the WSDL. One of these will be the
service, and you will create objects of that class in order to work with the service (see "static case"
below).
Static case
Most clients will start with a WSDL file, and generate some stubs using jbossws tools like
wsconsume. This usually gives a mass of files, one of which is the top of the tree. This is the service
implementation class.
The generated implementation class can be recognised as it will have two public constructors, one
with no arguments and one with two arguments, representing the wsdl location (a java.net.URL) and
the service name (a javax.xml.namespace.QName) respectively.
Usually you will use the no-argument constructor. In this case the WSDL location and service name
are those found in the WSDL. These are set implicitly from the WebServiceClient annotation that
decorates the generated class.
The following code snippet shows the generated constructors from the generated class:
23
Chapter 5. Web Services
@WebServiceClient(name="StockQuoteService", targetNamespace="http://
example.com/stocks", wsdlLocation="http://example.com/stocks.wsdl")
public class StockQuoteService extends javax.xml.ws.Service
{
public StockQuoteService()
{
super(new URL("http://example.com/stocks.wsdl"), new QName("http://
example.com/stocks", "StockQuoteService"));
}
...
}
8
Section #Dynamic Proxy explains how to obtain a port from the service and how to invoke
an operation on the port. If you need to work with the XML payload directly or with the XML
9
representation of the entire SOAP message, have a look at #Dispatch .
Dynamic case
In the dynamic case, when nothing is generated, a web service client uses Service.create to
create Service instances, the following code illustrates this process.
When a Service instance is used to create a proxy or a Dispatch instance then the handler
resolver currently registered with the service is used to create the required handler chain. Subsequent
8
http://www.jboss.org/community/wiki/JBossWS-UserGuide#Dynamic_Proxy
9
http://www.jboss.org/community/wiki/JBossWS-UserGuide#Dispatch
10
http://www.jboss.org/community/wiki/JBossWS-UserGuide#Handler_Framework
24
Dynamic Proxy
changes to the handler resolver configured for a Service instance do not affect the handlers on
previously created proxies, or Dispatch instances.
5.16.1.3. Executor
Service instances can be configured with a java.util.concurrent.Executor. The
executor will then be used to invoke any asynchronous callbacks requested by the application. The
setExecutor and getExecutor methods of Service can be used to modify and retrieve the
executor configured for a service.
/**
* The getPort method returns a proxy. A service client
* uses this proxy to invoke operations on the target
* service endpoint. The <code>serviceEndpointInterface</code>
* specifies the service endpoint interface that is supported by
* the created dynamic proxy instance.
*/
public <T> T getPort(QName portName, Class<T> serviceEndpointInterface)
{
...
}
/**
* The getPort method returns a proxy. The parameter
* <code>serviceEndpointInterface</code> specifies the service
* endpoint interface that is supported by the returned proxy.
* In the implementation of this method, the JAX-WS
* runtime system takes the responsibility of selecting a protocol
* binding (and a port) and configuring the proxy accordingly.
* The returned proxy should not be reconfigured by the client.
*
*/
public <T> T getPort(Class<T> serviceEndpointInterface)
{
...
}
The Service Endpoint Interface (SEI) is usually generated using tools. For details see Top Down
12
(Using wsconsume) .
13
A generated static Service usually also offers typed methods to get ports. These methods also
return dynamic proxies that implement the SEI.
11
http://java.sun.com/javaee/5/docs/api/javax/xml/ws/Service.html
12
http://www.jboss.org/community/wiki/JBossWS-JAX-WSTools#TopDown_Using_wsconsume
13
http://www.jboss.org/community/wiki/JBossWS-UserGuide#Service
25
Chapter 5. Web Services
@WebEndpoint(name = "TestEndpointPort")
public TestEndpoint getTestEndpointPort()
{
return (TestEndpoint)super.getPort(TESTENDPOINTPORT,
TestEndpoint.class);
}
}
5.16.3. WebServiceRef
The WebServiceRef annotation is used to declare a reference to a Web service. It follows the
resource pattern exemplified by the javax.annotation.Resource annotation in JSR-250 [5]
1. To define a reference whose type is a generated service class. In this case, the type and value
element will both refer to the generated service class type. Moreover, if the reference type can be
inferred by the field or method declaration then the annotation is applied to the type, and value
elements may have the default value (Object.class, that is). If the type cannot be inferred, then
at least the type element must be present with a non-default value.
2. To define a reference whose type is a SEI. In this case, the type element may be present with
its default value if the type of the reference can be inferred from the annotated field and method
declaration, but the value element must always be present and refer to a generated service class
type (a subtype of javax.xml.ws.Service). The wsdlLocation element, if present, overrides
theWSDL location information specified in the WebService annotation of the referenced
generated service class.
@WebServiceRef
public TestEndpoint port3;
}
26
WebServiceRef
WebServiceRef Customization
In JBoss Enterprise Application Platform 5.0 we offer a number of overrides and extensions to the
WebServiceRef annotation. These include:
Example:
<service-ref>
<service-ref-name>OrganizationService</service-ref-name>
<wsdl-override>file:/wsdlRepository/organization-service.wsdl</wsdl-
override>
</service-ref>
..
<service-ref>
<service-ref-name>OrganizationService</service-ref-name>
<config-name>Secure Client Config</config-name>
<config-file>META-INF/jbossws-client-config.xml</config-file>
<handler-chain>META-INF/jbossws-client-handlers.xml</handler-chain>
</service-ref>
<service-ref>
<service-ref-name>SecureService</service-ref-name>
<service-class-
name>org.jboss.tests.ws.jaxws.webserviceref.SecureEndpointService</service-
class-name>
<service-qname>{http://org.jboss.ws/wsref}SecureEndpointService</
service-qname>
<port-info>
<service-endpoint-
interface>org.jboss.tests.ws.jaxws.webserviceref.SecureEndpoint</service-
endpoint-interface>
<port-qname>{http://org.jboss.ws/wsref}SecureEndpointPort</port-
qname>
<stub-property>
<name>javax.xml.ws.security.auth.username</name>
<value>kermit</value>
</stub-property>
<stub-property>
<name>javax.xml.ws.security.auth.password</name>
<value>thefrog</value>
</stub-property>
</port-info>
</service-ref>
27
Chapter 5. Web Services
5.16.4. Dispatch
XML Web Services use XML messages for communication between services and service clients.
The higher level JAX-WS APIs are designed to hide the details of converting between Java method
invocations and the corresponding XML messages, but in some cases operating at the XML message
level is desirable. The Dispatch interface provides support for this mode of interaction.
Message
In this mode, client applications work directly with protocol-specific message structures. For example,
when used with a SOAP protocol binding, a client application would work directly with a SOAP
message.
Message Payload
In this mode, client applications work with the payload of messages rather than the messages
themselves. For example, when used with a SOAP protocol binding, a client application would work
with the contents of the SOAP Body rather than the SOAP message as a whole.
Dispatch is a low level API that requires clients to construct messages or message payloads as
XML and requires an intimate knowledge of the desired message or payload structure. Dispatch is a
generic class that supports input and output of messages or message payloads of any type.
28
Oneway Invocations
// access future
String retStr = (String) response.get();
assertEquals("Async", retStr);
}
@WebService (name="PingEndpoint")
@SOAPBinding(style = SOAPBinding.Style.RPC)
public class PingEndpointImpl
{
private static String feedback;
...
@WebMethod
@Oneway
public void ping()
{
log.info("ping");
feedback = "ok";
}
...
@WebMethod
public String feedback()
{
log.info("feedback");
return feedback;
}
}
14
http://www.jboss.org/community/wiki/JBossWS-UserGuide#Web_Service_Endpoints
29
Chapter 5. Web Services
Client and server-side handlers are organized into an ordered list known as a handler chain. The
handlers within a handler chain are invoked each time a message is sent or received. Inbound
messages are processed by handlers prior to binding provider processing. Outbound messages are
processed by handlers after any binding provider processing.
Handlers are invoked with a message context that provides methods to access and modify inbound
and outbound messages and to manage a set of properties. Message context properties may be used
to facilitate communication between individual handlers and between handlers and client and service
implementations. Different types of handlers are invoked with different types of message context.
@WebService
@HandlerChain(file = "jaxws-server-source-handlers.xml")
public class SOAPEndpointSourceImpl
{
...
}
2. A relative path from the source file or class file. (ex: bar/handlerfile1.xml)
30
Message Context
Properties are scoped as either APPLICATION or HANDLER. All properties are available to all
handlers associated with particular endpoint. E.g., if a logical handler puts a property in the message
context, that property will also be available to any protocol handlers in the chain during the execution.
APPLICATION scoped properties are also made available to client applications and service endpoint
implementations. The default scope for a property is HANDLER.
31
Chapter 5. Web Services
Note
In case of the latter JBossWS generates the required fault wrapper beans at runtime if
they are not part of the deployment
5.18. DataBinding
In order to support this, we built on a JAXB RI feature whereby it allows you to specify a
RuntimeInlineAnnotationReader implementation during JAXBContext creation (see JAXBRIContext).
We call this feature "JAXB Annotation Introduction" and we've made it available for general
consumption i.e. it can be checked out, built and used from SVN:
• http://anonsvn.jboss.org/repos/jbossws/projects/jaxbintros/
5.19. Attachments
JBoss-WS4EE relied on a deprecated attachments technology called SwA (SOAP with Attachments).
SwA required soap/encoding which is disallowed by the WS-I Basic Profile. JBossWS provides
support for WS-I AP 1.0, and MTOM instead.
5.19.1. MTOM/XOP
This section describes Message Transmission Optimization Mechanism (MTOM) and XML-binary
Optimized Packaging (XOP), a means of more efficiently serializing XML Infosets that have certain
types of content. The related specifications are
32
MTOM/XOP
17
• SOAP Message Transmission Optimization Mechanism (MTOM)
18
• XML-binary Optimized Packaging (XOP)
The above table shows a list of supported endpoint parameter types. The recommended approach
19
is to use the javax.activation.DataHandler classes to represent binary data as service endpoint
parameters.
Note
Microsoft endpoints tend to send any data as application/octet-stream. The only Java type
that can easily cope with this ambiguity is javax.activation.DataHandler
package org.jboss.test.ws.jaxws.samples.xop.doclit;
import javax.ejb.Remote;
import javax.jws.WebService;
import javax.jws.soap.SOAPBinding;
import javax.xml.ws.BindingType;
@Remote
@WebService(targetNamespace = "http://org.jboss.ws/xop/doclit")
@SOAPBinding(style = SOAPBinding.Style.DOCUMENT, parameterStyle =
SOAPBinding.ParameterStyle.BARE)
@BindingType(value="http://schemas.xmlsoap.org/wsdl/soap/http?mtom=true")
(1)
public interface MTOMEndpoint
{
...
}
33
Chapter 5. Web Services
Web service clients can use the same approach described above or
rely on the Binding API to enable MTOM (Excerpt taken from the
org.jboss.test.ws.jaxws.samples.xop.doclit.XOPTestCase):
...
Service service = Service.create(wsdlURL, serviceName);
port = service.getPort(MTOMEndpoint.class);
// enable MTOM
binding = (SOAPBinding)((BindingProvider)port).getBinding();
binding.setMTOMEnabled(true);
Note
You might as well use the JBossWS configuration templates to setup deployment defaults.
5.19.2. SwaRef
20
WS-I Attachment Profile 1.0 defines mechanism to reference MIME attachment parts using
21
swaRef . In this mechanism the content of XML element of type wsi:swaRef is sent as MIME
attachment and the element inside SOAP Body holds the reference to this attachment in the CID URI
22
scheme as defined by RFC 2111 .
/**
* Payload bean that will use SwaRef encoding
*/
@XmlRootElement
public class DocumentPayload
{
private DataHandler data;
public DocumentPayload()
{
}
20
http://www.ws-i.org/Profiles/AttachmentsProfile-1.0-2004-08-24.html
21
http://www.ws-i.org/Profiles/AttachmentsProfile-1.0-2004-08-24.html#Referencing_Attachments_from_the_SOAP_Envelope
22
http://www.ietf.org/rfc/rfc2111.txt
34
SwaRef
@XmlElement
@XmlAttachmentRef
public DataHandler getData()
{
return data;
}
With document wrapped endpoints you may even specify the @XmlAttachmentRef annotation on
the service endpoint interface:
@WebService
public interface DocWrappedEndpoint
{
@WebMethod
DocumentPayload beanAnnotation(DocumentPayload dhw, String test);
@WebMethod
@XmlAttachmentRef
DataHandler parameterAnnotation(@XmlAttachmentRef DataHandler data,
String test);
<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'>
<env:Header/>
<env:Body>
<ns2:parameterAnnotation xmlns:ns2='http://
swaref.samples.jaxws.ws.test.jboss.org/'>
<arg0>cid:[email protected]</arg0>
<arg1>Wrapped test</arg1>
</ns2:parameterAnnotation>
</env:Body>
</env:Envelope>
35
Chapter 5. Web Services
5.20. Tools
The JAX-WS tools provided by JBossWS can be used in a variety of ways. First we will look at
server-side development strategies, and then proceed to the client. When developing a Web Service
Endpoint (the server-side) you have the option of starting from Java (bottom-up development), or
from the abstact contract (WSDL) that defines your service (top-down development). If this is a new
service (no existing contract), the bottom-up approach is the fastest route; you only need to add a few
annotations to your classes to get a service up and running. However, if you are developing a service
with an already defined contract, it is far simpler to use the top-down approach, since the provided tool
will generate the annotated code for you.
• Providing a new service, and you want the contract to be generated for you
• Replacing the implementation of an existing Web Service without breaking compatibility with older
clients
• Exposing a service that conforms to a contract specified by a third party (e.g. a vender that calls you
back using an already defined protocol).
• Creating a service that adheres to the XML Schema and WSDL you developed by hand up front
Command Description
23
wsprovide Generates JAX-WS portable artifacts, and
provides the abstract contract. Used for bottom-
up development.
24
wsconsume Consumes the abstract contract (WSDL and
Schema files), and produces artifacts for both a
server and client. Used for top-down and client
development
25
wsrunclient Executes a Java client (that has a main method)
using the JBossWS classpath.
36
Bottom-Up (Using wsprovide)
like. However, all of the annotations have sensible defaults, so only the @WebService annotation is
required.
package echo;
@javax.jws.WebService
public class Echo
{
public String echo(String input)
{
return input;
}
}
A JSE or EJB3 deployment can be built using this class, and it is the only Java code needed to deploy
on JBossWS. The WSDL, and all other Java artifacts called "wrapper classes" will be generated
for you at deploy time. This actually goes beyond the JAX-WS specification, which requires that
wrapper classes be generated using an offline tool. The reason for this requirement is purely a vender
implementation problem, and since we do not believe in burdening a developer with a bunch of
additional steps, we generate these as well. However, if you want your deployment to be portable
to other application servers, you will need to use a tool and add the generated classes to your
deployment.
26
This is the primary purpose of the wsprovide tool, to generate portable JAX-WS artifacts.
Additionally, it can be used to "provide" the abstract contract (WSDL file) for your service. This can be
27
obtained by invoking wsprovide using the "-w" option:
<service name='EchoService'>
<port binding='tns:EchoBinding' name='EchoPort'>
<soap:address location='REPLACE_WITH_ACTUAL_URL'/>
</port>
</service>
37
Chapter 5. Web Services
<portType name='Echo'>
<operation name='echo' parameterOrder='echo'>
<input message='tns:Echo_echo'/>
<output message='tns:Echo_echoResponse'/>
</operation>
</portType>
Note
Remember that when deploying on JBossWS you do not need to run this tool. You
only need it for generating portable artifacts and/or the abstract contract for your service.
Let us create a POJO endpoint for deployment on JBoss Enterprise Application Platform. A simple
web.xml needs to be created:
<web-app xmlns="http://java.sun.com/xml/ns/j2ee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/
xml/ns/j2ee/web-app_2_4.xsd"
version="2.4">
<servlet>
<servlet-name>Echo</servlet-name>
<servlet-class>echo.Echo</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>Echo</servlet-name>
<url-pattern>/Echo</url-pattern>
</servlet-mapping>
</web-app>
The web.xml and the single class can now be used to create a WAR:
$ mkdir -p WEB-INF/classes
$ cp -rp echo WEB-INF/classes/
$ cp web.xml WEB-INF
$ jar cvf echo.war WEB-INF
added manifest
adding: WEB-INF/(in = 0) (out= 0)(stored 0%)
adding: WEB-INF/classes/(in = 0) (out= 0)(stored 0%)
adding: WEB-INF/classes/echo/(in = 0) (out= 0)(stored 0%)
adding: WEB-INF/classes/echo/Echo.class(in = 340) (out= 247)(deflated 27%)
adding: WEB-INF/web.xml(in = 576) (out= 271)(deflated 52%)
38
Top-Down (Using wsconsume)
cp echo.war $JBOSS_HOME/server/default/deploy
28
At deploy time JBossWS will internally invoke wsprovide , which will generate the WSDL. If
deployment was successful, and you are using the default settings, it should be available here: http://
localhost:8080/echo/Echo?wsdl
For a portable JAX-WS deployment, the wrapper classes generated earlier could be added to the
deployment.
Note
wsconsume seems to have a problem with symlinks on unix systems
Using the WSDL file from the bottom-up example, a new Java implementation that adheres to this
30
service can be generated. The "-k" option is passed to wsconsume to preserve the Java source files
that are generated, instead of providing just classes:
$ wsconsume -k EchoService.wsdl
echo/Echo.java
echo/EchoResponse.java
echo/EchoService.java
echo/Echo_Type.java
echo/ObjectFactory.java
echo/package-info.java
echo/Echo.java
echo/EchoResponse.java
echo/EchoService.java
echo/Echo_Type.java
echo/ObjectFactory.java
echo/package-info.java
File Purpose
Echo.java Service Endpoint Interface
Echo_Type.java Wrapper bean for request message
28
http://www.jboss.org/community/wiki/JBossWS-wsprovide
29
http://www.jboss.org/community/wiki/JBossWS-wsconsume
30
http://www.jboss.org/community/wiki/JBossWS-wsconsume
39
Chapter 5. Web Services
Examining the Service Endpoint Interface reveals annotations that are more explicit than in the class
written by hand in the bottom-up example, however, these evaluate to the same contract:
The only missing piece (besides the packaging) is the implementation class, which can now be written
using the above interface.
package echo;
@javax.jws.WebService(endpointInterface="echo.Echo")
public class EchoImpl implements Echo
{
public String echo(String arg0)
{
return arg0;
}
}
40
Client Side
Let's repeat the process of the top-down section, although using the deployed WSDL, instead of
31
the one generated offline by wsprovide . The reason why we do this is just to get the right value
for soap:address. This value must be computed at deploy time, since it is based on container
configuration specifics. You could of course edit the WSDL file yourself, although you need to ensure
that the path is correct.
Offline version:
<service name='EchoService'>
<port binding='tns:EchoBinding' name='EchoPort'>
<soap:address location='REPLACE_WITH_ACTUAL_URL'/>
</port>
</service>
Online version:
<service name="EchoService">
<port binding="tns:EchoBinding" name="EchoPort">
<soap:address location="http://localhost.localdomain:8080/echo/Echo"/
>
</port>
</service>
32
Using the online deployed version with wsconsume :
$ wsconsume -k http://localhost:8080/echo/Echo?wsdl
echo/Echo.java
echo/EchoResponse.java
echo/EchoService.java
echo/Echo_Type.java
echo/ObjectFactory.java
echo/package-info.java
echo/Echo.java
echo/EchoResponse.java
echo/EchoService.java
echo/Echo_Type.java
echo/ObjectFactory.java
echo/package-info.java
The one class that was not examined in the top-down section, was EchoService.java. Notice how
it stores the location the WSDL was obtained from.
31
http://www.jboss.org/community/wiki/JBossWS-wsprovide
32
http://www.jboss.org/community/wiki/JBossWS-wsconsume
41
Chapter 5. Web Services
static
{
URL url = null;
try
{
url = new URL("http://localhost:8080/echo/Echo?wsdl");
}
catch (MalformedURLException e)
{
e.printStackTrace();
}
ECHOSERVICE_WSDL_LOCATION = url;
}
public EchoService()
{
super(ECHOSERVICE_WSDL_LOCATION, new QName("http://echo/",
"EchoService"));
}
@WebEndpoint(name = "EchoPort")
public Echo getEchoPort()
{
return (Echo)super.getPort(new QName("http://echo/", "EchoPort"),
Echo.class);
}
}
As you can see, this generated class extends the main client entry point in JAX-WS,
javax.xml.ws.Service. While you can use Service directly, this is far simpler since it provides
the configuration info for you. The only method we really care about is the getEchoPort() method,
which returns an instance of our Service Endpoint Interface. Any Web Services operation can
then be called by just invoking a method on the returned interface.
Note
It is not recommended to refer to a remote WSDL URL in a production application. This
causes network I/O every time you instantiate the Service Object. Instead, use the tool
on a saved local copy, or use the URL version of the constructor to provide a new WSDL
location.
42
Command-line & Ant Task Reference
import echo.*;
..
public class EchoClient
{
public static void main(String args[])
{
if (args.length != 1)
{
System.err.println("usage: EchoClient <message>");
System.exit(1);
}
33
It can then be easily executed using the wsrunclient tool. This is just a convenience tool that invokes
java with the needed classpath:
...
EchoService service = new EchoService();
Echo echo = service.getEchoPort();
43
Chapter 5. Web Services
36
• wsrunclient reference page
• http://java.sun.com/webservices/docs/2.0/jaxws/customizations.html
The schema for the binding customization files can be found here:
37
• binding customization
5.21.1. WS-Addressing
38
This section describes how WS-Addressing can be used to provide a staful service endpoint.
5.21.1.1. Specifications
WS-Addressing is defined by a combination of the following specifications from the W3C
Recommendation. The WS-Addressing API is standardized by JSR-224 - Java API for XML-Based
39
Web Services (JAX-WS)
40
• Web Services Addressing 1.0 - Core
41
• Web Services Addressing 1.0 - SOAP Binding
@WebMethod
public void checkout()
{ ... }
@WebMethod
38
http://www.w3.org/TR/ws-addr-core
39
http://www.jcp.org/en/jsr/detail?id=224
44
WS-Addressing
It uses the JAX-WS 2.1 defined javax.xml.ws.soap.Addressing annotation to enable the server
side addressing handler.
port2.addItem("Mars Bar");
port2.addItem("Porsche");
}
Below you see the SOAP messages that are beeing exchanged.
<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'>
<env:Header xmlns:wsa='http://schemas.xmlsoap.org/ws/2004/08/addressing'>
<wsa:To>uri:jbossws-samples-wsaddr/TestService</wsa:To>
<wsa:Action>http://org.jboss.ws/addressing/stateful/action</wsa:Action>
45
Chapter 5. Web Services
<wsa:ReferenceParameters>
<ns1:clientid xmlns:ns1='http://somens'>clientid-1</ns1:clientid>
</wsa:ReferenceParameters>
</env:Header>
<env:Body>
<ns1:addItem xmlns:ns1='http://org.jboss.ws/samples/wsaddr'>
<String_1>Ice Cream</String_1>
</ns1:addItem>
</env:Body>
</env:Envelope>
<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'>
<env:Header xmlns:wsa='http://schemas.xmlsoap.org/ws/2004/08/addressing'>
<wsa:To>http://www.w3.org/2005/08/addressing/anonymous</wsa:To>
<wsa:Action>http://org.jboss.ws/addressing/stateful/actionReply</
wsa:Action>
<ns1:clientid xmlns:ns1='http://somens'>clientid-1</ns1:clientid>
</env:Header>
<env:Body>
<ns1:addItemResponse xmlns:ns1='http://org.jboss.ws/samples/wsaddr'/>
</env:Body>
</env:Envelope>
...
<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'>
<env:Header xmlns:wsa='http://schemas.xmlsoap.org/ws/2004/08/addressing'>
<wsa:To>uri:jbossws-samples-wsaddr/TestService</wsa:To>
<wsa:Action>http://org.jboss.ws/addressing/stateful/action</wsa:Action>
<wsa:ReferenceParameters>
<ns1:clientid xmlns:ns1='http://somens'>clientid-1</ns1:clientid>
</wsa:ReferenceParameters>
</env:Header>
<env:Body>
<ns1:getItems xmlns:ns1='http://org.jboss.ws/samples/wsaddr'/>
</env:Body>
</env:Envelope>
<env:Envelope xmlns:env='http://schemas.xmlsoap.org/soap/envelope/'>
<env:Header xmlns:wsa='http://schemas.xmlsoap.org/ws/2004/08/addressing'>
<wsa:To>http://www.w3.org/2005/08/addressing/anonymous</wsa:To>
<wsa:Action>http://org.jboss.ws/addressing/stateful/actionReply</
wsa:Action>
<ns1:clientid xmlns:ns1='http://somens'>clientid-1</ns1:clientid>
</env:Header>
<env:Body>
<ns1:getItemsResponse xmlns:ns1='http://org.jboss.ws/samples/wsaddr'>
<result>[Ice Cream, Ferrari]</result>
</ns1:getItemsResponse>
</env:Body>
</env:Envelope>
46
WS-BPEL
5.21.2. WS-BPEL
42
WS-BPEL is not supported with JAX-WS, please refer to JAX-RPC User Guide#WS-BPEL .
5.21.3. WS-Security
WS-Security addresses message level security. It standardizes authorization, encryption, and digital
signature processing of web services. Unlike transport security models, such as SSL, WS-Security
applies security directly to the elements of the web service message. This increases the flexibility of
your web services, by allowing any message model to be used (point to point, multi-hop relay, etc).
This chapter describes how to use WS-Security to sign and encrypt a simple SOAP message.
Specifications
Note
You need to setup both the endpoint configuration and the WSSE declarations i. e. two
separate steps.
42
http://www.jboss.org/community/wiki/JBossWS-JAX-RPCUserGuide#WSBPEL
49
http://www.jboss.org/community/wiki/JBossWS-JAX-WSEndpointConfiguration
50
http://www.jboss.org/community/wiki/JBossWS-JAX-WSClientConfiguration
47
Chapter 5. Web Services
<jboss-ws-security xmlns="http://www.jboss.com/ws-security/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.jboss.com/ws-security/config
http://www.jboss.com/ws-security/schema/jboss-ws-security_1_0.xsd">
(1) <key-store-file>WEB-INF/wsse.keystore</key-store-file>
(2) <key-store-password>jbossws</key-store-password>
(3) <trust-store-file>WEB-INF/wsse.truststore</trust-store-file>
(4) <trust-store-password>jbossws</trust-store-password>
(5) <config>
(6) <sign type="x509v3" alias="wsse"/>
(7) <requires>
(8) <signature/>
</requires>
</config>
</jboss-ws-security>
1. This specifies that the key store we wish to use is WEB-INF/wsse.keystore, which is located in
our war file.
2. This specifies that the store password is "jbossws". Password can be encypted using the {EXT}
and {CLASS} commands. Please see samples for their usage.
3. This specifies that the trust store we wish to use is WEB-INF/wsse.truststore, which is
located in our war file.
4. This specifies that the trust store password is also "jbossws". Password can be encrypted using
the {EXT} and {CLASS} commands. Please see samples for their usage.
5. Here we start our root config block. The root config block is the default configuration for all
services in this war file.
6. This means that the server must sign the message body of all responses. Type means that we are
using X.509v3 certificate (a standard certificate). The alias option says that the certificate and key
pair to use for signing is in the key store under the "wsse" alias
7. Here we start our optional requires block. This block specifies all security requirements that must
be met when the server receives a message.
8. This means that all web services in this war file require the message body to be signed.
By default an endpoint does not use the WS-Security configuration. Users can use proprietary
51
@EndpointConfig annotation to set the config name. See JAX-WS_Endpoint_Configuration for the
list of available config names.
@WebService
@EndpointConfig(configName = "Standard WSSecurity Endpoint")
public class HelloJavaBean
{
...
}
51
http://www.jboss.org/community/wiki/JBossWS-JAX-WSEndpointConfiguration
48
WS-Security
<jboss-ws-security xmlns="http://www.jboss.com/ws-security/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.jboss.com/ws-security/config
http://www.jboss.com/ws-security/schema/jboss-ws-security_1_0.xsd">
(1) <config>
(2) <sign type="x509v3" alias="wsse"/>
(3) <requires>
(4) <signature/>
</requires>
</config>
</jboss-ws-security>
1. Here we start our root config block. The root config block is the default configuration for all web
service clients (Call, Proxy objects).
2. This means that the client must sign the message body of all requests it sends. Type means
that we are to use a X.509v3 certificate (a standard certificate). The alias option says that the
certificate/key pair to use for signing is in the key store under the "wsse" alias
3. Here we start our optional requires block. This block specifies all security requirements that must
be met when the client receives a response.
4. This means that all web service clients must receive signed response messages.
<sysproperty key="org.jboss.ws.wsse.keyStore"
value="${tests.output.dir}/resources/jaxrpc/samples/wssecurity/
wsse.keystore"/>
<sysproperty key="org.jboss.ws.wsse.trustStore"
value="${tests.output.dir}/resources/jaxrpc/samples/wssecurity/
wsse.truststore"/>
<sysproperty key="org.jboss.ws.wsse.keyStorePassword" value="jbossws"/>
<sysproperty key="org.jboss.ws.wsse.trustStorePassword" value="jbossws"/>
<sysproperty key="org.jboss.ws.wsse.keyStoreType" value="jks"/>
<sysproperty key="org.jboss.ws.wsse.trustStoreType" value="jks"/>
Below you see the incomming SOAP message with the details of the security headers ommited. The
idea is, that the SOAP body is still plain text, but it is signed in the security header and therefore can
not be manipulated in transit.
49
Chapter 5. Web Services
Incomming SOAPMessage
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
<env:Header>
<wsse:Security env:mustUnderstand="1" ...>
<wsu:Timestamp wsu:Id="timestamp">...</wsu:Timestamp>
<wsse:BinarySecurityToken ...>
...
</wsse:BinarySecurityToken>
<ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
...
</ds:Signature>
</wsse:Security>
</env:Header>
<env:Body wsu:Id="element-1-1140197309843-12388840" ...>
<ns1:echoUserType xmlns:ns1="http://org.jboss.ws/samples/wssecurity">
<UserType_1 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<msg>Kermit</msg>
</UserType_1>
</ns1:echoUserType>
</env:Body>
</env:Envelope>
The provider can be configured as part of your environment via static registration by adding an entry to
the java.security properties file (found in $JAVA_HOME/jre/lib/security/java.security,
where $JAVA_HOME is the location of your JDK and JRE distribution). You will find detailed instructions
in the file but basically it comes down to adding a line:
security.provider.<n>=org.bouncycastle.jce.provider.BouncyCastleProvider
Note
Issues may arise if the Sun provided providers are not first.
Where users will put the provider jar is mostly up to them, although with jdk5 the best (and in some
cases only) place to have it is in $JAVA_HOME/jre/lib/ext. Under Windows there will normally be
a JRE and a JDK install of Java. If user think he have installed it correctly and it still doesn't work then
with high probability the provider installation is not used.
52
http://www.bouncycastle.org/specifications.html#install
50
XML Registries
This chapter describes how to configure the jUDDI registry in JBoss and some sample code outlines
for using JAXR API to publish and query the jUDDI registry.
DataSources configuration
Database Tables (Should they be created on start, Should they be dropped on stop, Should they be
dropped on start etc)
JAXR Connection Factory to be bound in JNDI. (Should it be bound? and under what name?)
Add authorized users to access the jUDDI registry. (Add a sql insert statement in a single line)
51
Chapter 5. Web Services
javax.xml.registry.ConnectionFactoryClass=org.apache.ws.scout.registry.
ConnectionFactoryImpl
jaxr.query.url=http://localhost:8080/juddi/inquiry
jaxr.publish.url=http://localhost:8080/juddi/publish
scout.proxy.transportClass=org.jboss.jaxr.scout.transport.SaajTransport
Please remember to change the hostname from "localhost" to the hostname of the UDDI service/
JBoss Server.
You can pass the System Properties to the JVM in the following ways:
• When the client code is running inside JBoss (maybe a servlet or an EJB). Then you will need to
pass the System properties in the run.sh or run.bat scripts to the java process via the "-D"
option.
• When the client code is running in an external JVM. Then you can pass the properties either as "-D"
options to the java process or explicitly set them in the client code(not recommended).
System.setProperty(propertyname, propertyvalue);
52
XML Registries
Let us now look at some of the common programming tasks performed while using the JAXR API:
/**
* Does authentication with the uddi registry
*/
protected void login() throws JAXRException
{
PasswordAuthentication passwdAuth = new PasswordAuthentication(userid,
passwd.toCharArray());
Set creds = new HashSet();
creds.add(passwdAuth);
connection.setCredentials(creds);
}
Save a Business
/**
* Creates a Jaxr Organization with 1 or more services
*/
protected Organization createOrganization(String orgname) throws
JAXRException
{
Organization org = blm.createOrganization(getIString(orgname));
org.setDescription(getIString("JBoss Inc"));
Service service = blm.createService(getIString("JBOSS JAXR Service"));
53
Chapter 5. Web Services
return org;
}
54
XML Registries
Query a Business
/**
* Locale aware Search a business in the registry
*/
public void searchBusiness(String bizname) throws JAXRException
{
try
{
// Get registry service and business query manager
this.getJAXREssentials();
55
Chapter 5. Web Services
}
}
For more examples of code using the JAXR API, please refer to the resources in the Resources
Section.
5.21.4.4. Troubleshooting
• I cannot connect to the registry from JAXR. Please check the inquiry and publish url passed to
the JAXR ConnectionFactory.
• I cannot connect to the jUDDI registry. Please check the jUDDI configuration and see if there are
any errors in the server.log. And also remember that the jUDDI registry is available only in the "all"
configuration.
• I cannot authenticate to the jUDDI registry. Have you added an authorized user to the jUDDI
database, as described earlier in the chapter?
• I would like to view the SOAP messages in transit between the client and the UDDI Registry.
56
Please use the tcpmon tool to view the messages in transit. TCPMon
5.21.4.5. Resources
57
• JAXR Tutorial and Code Camps
58
• J2EE 1.4 Tutorial
59
• J2EE Web Services by Richard Monson-Haefel
5.22.1.1. EndpointConfig
/**
* Defines an endpoint or client configuration.
* This annotation is valid on an endpoint implementaion bean or a SEI.
*/
@Retention(value = RetentionPolicy.RUNTIME)
@Target(value = { ElementType.TYPE })
public @interface EndpointConfig
{
...
/**
60
http://www.jboss.org/community/wiki/JBossWS-JAX-WSAnnotations
56
Proprietary Annotations
5.22.1.2. WebContext
/**
* Provides web context specific meta data to EJB based web service
endpoints.
*
* @author [email protected]
* @since 26-Apr-2005
*/
@Retention(value = RetentionPolicy.RUNTIME)
@Target(value = { ElementType.TYPE })
public @interface WebContext
{
...
/**
* The contextRoot element specifies the context root that the web
service endpoint is deployed to.
* If it is not specified it will be derived from the deployment short
name.
*
* Applies to server side port components only.
*/
String contextRoot() default "";
...
/**
* The virtual hosts that the web service endpoint is deployed to.
*
* Applies to server side port components only.
*/
String[] virtualHosts() default {};
57
Chapter 5. Web Services
/**
* Relative path that is appended to the contextRoot to form fully
qualified
* endpoint address for the web service endpoint.
*
* Applies to server side port components only.
*/
String urlPattern() default "";
/**
* The authMethod is used to configure the authentication mechanism for
the web service.
* As a prerequisite to gaining access to any web service which are
protected by an authorization
* constraint, a user must have authenticated using the configured
mechanism.
*
* Legal values for this element are "BASIC", or "CLIENT-CERT".
*/
String authMethod() default "";
/**
* The transportGuarantee specifies that the communication
* between client and server should be NONE, INTEGRAL, or
* CONFIDENTIAL. NONE means that the application does not require any
* transport guarantees. A value of INTEGRAL means that the application
* requires that the data sent between the client and server be sent in
* such a way that it can't be changed in transit. CONFIDENTIAL means
* that the application requires that the data be transmitted in a
* fashion that prevents other entities from observing the contents of
* the transmission. In most cases, the presence of the INTEGRAL or
* CONFIDENTIAL flag will indicate that the use of SSL is required.
*/
String transportGuarantee() default "";
/**
* A secure endpoint does not by default publish it's wsdl on an
unsecure transport.
* You can override this behaviour by explicitly setting the
secureWSDLAccess flag to false.
*
* Protect access to WSDL. See http://jira.jboss.org/jira/browse/
JBWS-723
*/
boolean secureWSDLAccess() default true;
}
5.22.1.3. SecurityDomain
/**
58
Web Services Appendix
/**
* The name for the unauthenticated pricipal
*/
String unauthenticatedPrincipal() default "";
}
5.24. References
64
1. JSR-224 - Java API for XML-Based Web Services (JAX-WS) 2.0
65
2. JSR 222 - Java Architecture for XML Binding (JAXB) 2.0
66
3. JSR-250 - Common Annotations for the Java Platform
67
4. JSR 181 - Web Services Metadata for the Java Platform
61
http://www.jboss.org/community/wiki/JBossWS-JAX-WSEndpointConfiguration
62
http://www.jboss.org/community/wiki/JBossWS-JAX-WSClientConfiguration
63
http://www.jboss.org/community/wiki/JBossWS-JAX-WSAnnotations
59
60
Chapter 6.
JBoss AOP
JBoss AOP is a 100% Pure Java Aspected Oriented Framework usable in any programming
environment or tightly integrated with our application server. Aspects allow you to more easily
modularize your code base when regular object oriented programming just doesn't fit the bill. It can
provide a cleaner separation from application logic and system code. It provides a great way to expose
integration points into your software. Combined with JDK 1.5 Annotations, it also is a great way to
expand the Java language in a clean pluggable way rather than using annotations solely for code
generation.
JBoss AOP is not only a framework, but also a prepackaged set of aspects that are applied via
annotations, pointcut expressions, or dynamically at runtime. Some of these include caching,
asynchronous communication, transactions, security, remoting, and many many more.
An aspect is a common feature that's typically scattered across methods, classes, object hierarchies,
or even entire object models. It is behavior that looks and smells like it should have structure, but you
can't find a way to express this structure in code with traditional object-oriented techniques.
For example, metrics is one common aspect. To generate useful logs from your application, you have
to (often liberally) sprinkle informative messages throughout your code. However, metrics is something
that your class or object model really shouldn't be concerned about. After all, metrics is irrelevant to
your actual application: it doesn't represent a customer or an account, and it doesn't realize a business
rule. It's simply orthogonal.
Joinpoint
A joinpoint is any point in your java program. The call of a method. The execution of a constructor the
access of a field. All these are joinpoints. You could also think of a joinpoint as a particular Java event.
Where an event is a method call, constructor call, field access etc...
Invocation
An Invocation is a JBoss AOP class that encapsulates what a joinpiont is at runtime. It could contain
information like which method is being called, the arguments of the method, etc...
Advice
An advice is a method that is called when a particular joinpoint is executed, i.e., the behavior that is
triggered when a method is called. It could also be thought of as the code that does the interception.
Another analogy is that an advice is an "event handler".
Pointcut
Pointcuts are AOP's expression language. Just as a regular expression matches strings, a pointcut
expression matches a particular joinpoint.
Introductions
An introduction modifies the type and structure of a Java class. It can be used to force an existing
class to implement an interface or to add an annotation to anything.
61
Chapter 6. JBoss AOP
Aspect
An Aspect is a plain Java class that encapsulates any number of advices, pointcut definitions, mixins,
or any other JBoss AOP construct.
Interceptor
An interceptor is an Aspect with only one advice named "invoke". It is a specific interface that you can
implement if you want your code to be checked by forcing your class to implement an interface. It also
will be portable and can be reused in other JBoss environments like EJBs and JMX MBeans.
In AOP, a feature like metrics is called a crosscutting concern, as it's a behavior that "cuts" across
multiple points in your object models, yet is distinctly different. As a development methodology, AOP
recommends that you abstract and encapsulate crosscutting concerns.
For example, let's say you wanted to add code to an application to measure the amount of time
it would take to invoke a particular method. In plain Java, the code would look something like the
following.
While this code works, there are a few problems with this approach:
1. It's extremely difficult to turn metrics on and off, as you have to manually add the code in the try>/
finally block to each and every method or constructor you want to benchmark.
2. The profiling code really doesn't belong sprinkled throughout your application code. It makes your
code bloated and harder to read, as you have to enclose the timings within a try/finally block.
3. If you wanted to expand this functionality to include a method or failure count, or even to register
these statistics to a more sophisticated reporting mechanism, you'd have to modify a lot of
different files (again).
This approach to metrics is very difficult to maintain, expand, and extend, because it's dispersed
throughout your entire code base. And this is just a tiny example! In many cases, OOP may not always
be the best way to add metrics to a class.
Aspect-oriented programming gives you a way to encapsulate this type of behavior functionality. It
allows you to add behavior such as metrics "around" your code. For example, AOP provides you
62
Creating Aspects in JBoss AOP
with programmatic control to specify that you want calls to BankAccountDAO to go through a metrics
aspect before executing the actual body of that code.
The first step in creating a metrics aspect in JBoss AOP is to encapsulate the metrics feature in its
own Java class. Listing Two extracts the try/finally block in Listing One's BankAccountDAO.withdraw()
method into Metrics, an implementation of a JBoss AOP Interceptor class.
Under JBoss AOP, the Metrics class wraps withdraw(): when calling code invokes withdraw(), the AOP
framework breaks the method call into its parts and encapsulates those parts into an Invocation object.
The framework then calls any aspects that sit between the calling code and the actual method body.
When the AOP framework is done dissecting the method call, it calls Metric's invoke method at line 3.
Line 8 wraps and delegates to the actual method and uses an enclosing try/finally block to perform the
timings. Line 13 obtains contextual information about the method call from the Invocation object, while
line 14 displays the method name and the calculated metrics.
Having the metrics code within its own object allows us to easily expand and capture additional
measurements later on. Now that metrics are encapsulated into an aspect, let's see how to apply it.
63
Chapter 6. JBoss AOP
strings, a pointcut expression matches events/points within your application. For example, a valid
pointcut definition would be "for all calls to the JDBC method executeQuery(), call the aspect that
verifies SQL syntax."
An entry point could be a field access, or a method or constructor call. An event could be an exception
being thrown. Some AOP implementations use languages akin to queries to specify pointcuts. Others
use tags. JBoss AOP uses both. Listing Three shows how to define a pointcut for the metrics example.
Lines 1-3 define a pointcut that applies the metrics aspect to the specific method
BankAccountDAO.withdraw(). Lines 4-6 define a general pointcut that applies the metrics aspect to all
methods in all classes in the com.mc.billing package. There is also an optional annotation mapping if
you do not like XML. See our Reference Guide for more information.
JBoss AOP has a rich set of pointcut expressions that you can use to define various points/events in
your Java application so that you can apply your aspects. You can attach your aspects to a specific
Java class in your application or you can use more complex compositional pointcuts to specify a wide
range of classes within one expression.
With AOP, as this example shows, you're able to pull together crosscutting behavior into one object
and apply it easily and simply, without polluting and bloating your code with features that ultimately
don't belong mingled with business logic. Instead, common crosscutting concerns can be maintained
and extended in one place.
Notice too that the code within the BankAccountDAO class has no idea that it's being profiled. This
is what aspect-oriented programmers deem orthogonal concerns. Profiling is an orthogonal concern.
In the OOP code snippet in Listing One, profiling was part of the application code. With AOP, you
can remove that code. A modern promise of middleware is transparency, and AOP (pardon the pun)
clearly delivers.
Just as important, orthogonal behavior could be bolted on after development. In Listing One,
monitoring and profiling must be added at development time. With AOP, a developer or an
administrator can (easily) add monitoring and metrics as needed without touching the code. This is
a very subtle but significant part of AOP, as this separation (obliviousness, some may say) allows
aspects to be layered on top of or below the code that they cut across. A layered design allows
features to be added or removed at will. For instance, perhaps you snap on metrics only when you're
doing some benchmarks, but remove it for production. With AOP, this can be done without editing,
recompiling, or repackaging the code.
64
Packaging AOP Applications
*-aop.xml along with your package (this is how the base-aop.xml, included in the jboss-
aop.deployer file works) or you can include it in the jar file containing your classes. If you include
your xml file in your jar, it must have the file extension .aop and a jboss-aop.xml file must be contained
in a META-INF directory, for instanace: META-INF/jboss-aop.xml.
In the JBoss Enterprise Application Platform 5, you must specify the schema used, otherwise your
information will not be parsed correctly. You do this by adding the xmlns="urn:jboss:aop-
beans:1:0 attribute to the root aop element, as shown here:
<aop xmlns="urn:jboss:aop-beans:1.0">
</aop>
If you want to create anything more than a non-trivial example, using the .aop jar files, you can make
any top-level deployment contain a .aop file containing the xml binding configuration. For instance you
can have a .aop file in an .ear file, or a .aop file in a war file. The bindings specified in the META-INF/
jboss-aop.xml file contained in the .aop file will affect all the classes in the whole war file
<application>
<display-name>AOP in JBoss example</display-name>
<module>
<java>example.aop</java>
</module>
<module>
<ejb>aopexampleejb.jar</ejb>
</module>
<module>
<web>
<web-uri>aopexample.war</web-uri>
<context-root>/aopexample</context-root>
</web>
</module>
</application>
Important
In the JBoss Enterprise Application Platform 5, the contents of the .ear file are deployed
in the order they are listed in the application.xml. When using loadtime weaving
the bindings listed in the example.aop file must be deployed before the classes being
advised are deployed, so that the bindings exist in the system before (for example) the ejb
and servlet classes are loaded. This is acheived by listing the .aop file at the start of the
65
Chapter 6. JBoss AOP
application.xml. Other types of archives are deployed before anything else and so do
not require special consideration, such as .sar and .war files.
In JBoss Enterprise Application Platform 5 the AspectManager Service is configured using a JBoss
Microcontainer bean. The configuration file is jboss-5.x.x.GA/server/xxx/conf/aop.xml. The
AspectManager Service is deployed with the following xml:
<bean name="AspectManager"
class="org.jboss.aop.deployers.AspectManagerJDK5">
<property name="enableLoadtimeWeaving">false</property>
<!-- only relevant when EnableLoadtimeWeaving is true.
When transformer is on, every loaded class gets transformed.
If AOP can't find the class, then it throws an exception.
Sometimes, classes may not have all the classes they reference.
So, the Suppressing is needed. (For instance, JBoss cache in the default
configuration) -->
<property name="suppressTransformationErrors">true</property>
<property name="prune">true</property>
<property name="exclude">org.jboss.</property>
<!-- This avoids instrumentation of hibernate cglib enhanced proxies
<property name="optimized">true</property>
<property name="verbose">false</property>
<!-- Available choices for this attribute are:
org.jboss.aop.instrument.ClassicInstrumentor (default)
org.jboss.aop.instrument.GeneratedAdvisorInstrumentor -->
<!-- <property
name="instrumentor">org.jboss.aop.instrument.ClassicInstrumentor</
property>-->
66
Loadtime transformation in the JBoss Enterprise Application Platform Using Sun JDK
Later we will talk about changing the class of the AspectManager Service. T,o do this replace the
contents of the class attribute of the bean element.
If you want to do load-time transformations with JBoss Enterprise Application Platform 5 and Sun JDK,
these are the steps you must take.
• Copy the pluggable-instrumentor.jar from the lib/ directory of your JBoss AOP
distribution to the bin/ directory of your JBoss Enterprise Application Platform.
• Next edit run.sh or run.bat (depending on what OS you're on) and add the following to the
JAVA_OPTS environment variable:
Important
The class of the AspectManager Service must be
org.jboss.aop.deployers.AspectManagerJDK5 or
org.jboss.aop.deployment.AspectManagerServiceJDK5 as these are what work
with the -javaagent option.
6.7. JRockit
JRockit also supports the -javaagent switch mentioned in Section 6.6, “Loadtime transformation
in the JBoss Enterprise Application Platform Using Sun JDK”. If you wish to use that, then the steps
in Section 6.6, “Loadtime transformation in the JBoss Enterprise Application Platform Using Sun
JDK” are sufficient. However, JRockit also comes with its own framework for intercepting when
classes are loaded, which might be faster than the -javaagent switch. If you want to do load-time
transformations using the special JRockit hooks, these are the steps you must take.
67
Chapter 6. JBoss AOP
• Copy the jrockit-pluggable-instrumentor.jar from the lib/ directory of your JBoss AOP
distribution to the bin/ directory of your the JBoss Enterprise Application Platform installation.
• Next edit run.sh or run.bat (depending on what OS you're on) and add the following to the
JAVA_OPTS and JBOSS_CLASSPATH environment variables:
JAVA_OPTS="$JAVA_OPTS -Dprogram.name=$PROGNAME \
-Xmanagement:class=org.jboss.aop.hook.JRockitPluggableClassPreProcessor"
JBOSS_CLASSPATH="$JBOSS_CLASSPATH:jrockit-pluggable-instrumentor.jar"
68
Attaching to a scoped deployment
<boss-app>
<loader-repository>
jboss.test:service=scoped
</loader-repository>
</jboss-app>
We can later deploy a AOP file containing aspects and configuration to attach that deployment to the
scoped EAR. This is done using the loader-repository tag in the AOP files META-INF/jboss-
aop.xml file.
This has the same effect as deploying the AOP file as part of the EAR as we saw previously, but
allows you to hot deploy aspects into your scoped application.
69
70
Chapter 7.
Transaction Management
7.1. Overview
Transaction support in JBoss Enterprise Application Platform is provided by JBossTS, a mature,
modular, standards based, highly configurable transaction manager. By default the server runs with
the local-only JTA module of JBossTS installed. This module provides an implementation of the
standard JTA API for use by other internal components, such as the EJB container, as well as direct
use by application code. It is suitable for coordinating ACID transactions that involve one or more XA
Resource managers, such as databases or message queues.
Two additional, optional, JBossTS transaction modules are also shipped with JBoss Enterprise
Application Platform and may be deployed to provide additional functionality if required. These are:
• JBossTS JTS, a transaction manager capable of distributing transaction context on remote IIOP
method calls, thus creating a single distributed transaction spanning multiple JVMs. This is useful for
e.g. large scale applications that span multiple servers, or for standards based interoperability with
transactional business logic running in CORBA based systems. The functionality of this module can
be accessed through the standard JTA API, making it a drop-in replacement that does not require
changes to transactional business logic. It is necessary only to change the server configuration.
Details of this process are given below.
• JBossTS XTS, an XML based transaction service implementing the WS-AtomicTransaction (WS-
AT) and WS-BusinessActivity (WS-BA) version 1.2 specifications. This additional module utilizes
core transaction support provided by the JTA or JTS, as well as web services functionality provided
by JBossWS Native. It is deployed into the server as an application, a process detailed below.
Applications may use WS-AT to provide standards based, distributed ACID transactions in a manner
broadly similar to JTS but using a web services transport rather than a CORBA one. The WS-BA
implementation compliments this by providing an alternative, compensation based transaction
model, well suited to coordinating long running, loosely coupled business processes. XTS also
implements a WS-Coordination service which, usually, is only accessed internally by the local WS-
AT and WS-BA implementations. However, this WS-C service can also be used to provide remote
coordination for WS-AT and WS-BA transactions created in other JBoss server instances or non-
JBoss containers.
71
Chapter 7. Transaction Management
• transactionTimeout – the default time, in seconds, after which a transaction will be considered to be
stuck and may be rolled back by the transaction manager.
The transaction timeout functionality is useful to prevent poorly written code, such as large SQL
queries, from blocking the system indefinitely. The default value is 300 seconds. This should be
adjusted to suit your environment and workload.
Note that transaction timeouts are processed asynchronously, a design decision which can take
some applications by surprise. See the transaction timeout handling section below for more details.
• objectStoreDir - the directory to which transaction data will be logged. This bean property overrides
the jbossts-properties.xml config file value for com.arjuna.ats,arjuna,objectstore.objectStoreDir
This transaction log is required to complete transactions in the case of system failures. The
performance and reliability of the storage device used for it are critical. In general, local RAID disk
should be preferred. Remote storage may be used provided it correctly implements file locking.
However, requiring network I/O for this storage can be a significant bottleneck on performance.
The ObjectStore will normally contain one file per in-flight transaction, each a few kbytes in size.
These are distributed over a directory tree for optimal performance. The small file size and rapid
creation/deleting of files lends itself well to SSD based storage devices, although traditional hard
drives may of course still be used. If a RAID controller is used, it should be configured for write
through cache, in much the same manner as database storage devices. Writing of the transaction
log is automatically skipped in the case of transactions that are rolling back or contain only a single
resource.
The following additional configuration properties present in the jbossts-properties.xml may also be of
interest:
• com.arjuna.common.util.logging.DebugLevel
This setting determines the internal log threshold for the transaction manager codebase. It is
independent of the server's wider log4j logging configuration and represents an additional hurdle
that log messages must pass before being printed. The default value is “0x00000000” i.e. no
debug logging. INFO and WARN messages will still be printed by default. This provides optimal
performance. The value “0xffffffff” should be used when full debug logging is required. This is very
verbose and will result in large log files. Log messages that pass the internal DebugLevel check will
be passed to the server's logging system for further processing. Thus it may also be necessary to
set appropriate configuration for “com.arjuna” code in the server/[name]/conf/jboss-log4j.xml file.
Note that whilst a value of “0xffffffff” may be left in place permanently and the log4j settings used to
turn logging on or off, this is less performant than using the internal DebugLevel checking.
• com.arjuna.ats.arjuna.coordinator.commitOnePhase
This setting determines if the transaction manager will automatically apply the one-phase commit
optimization to the transaction completion protocol in cases where only a single resource is
registered with the transaction. It is enabled (“YES”) by default to provide optimal performance,
since no transaction log write is necessary in such cases. Some resource managers may not be
compatible with this optimization and it is occasionally necessary to disable it. This can be done by
changing the value to “NO”.
• com.arjuna.ats.arjuna.objectstore.transactionSync
72
Transactional Resources
This setting controls the flushing of transaction logs to disk during the transaction termination.
It is enabled (“ON”) by default, which results in a FileDescriptor.sync call for each committing
transaction. This is required to provide recovery guarantees and hence ACID properties. If the
applications running in the server can tolerate data inconsistency or loss, greater performance
may be achieved by disabling this behavior by setting the property value to “OFF”. This is not
recommended – it is usually preferable to recraft such applications to avoid using the transaction
manager entirely.
These properties determine the behavior of the transaction recovery system. Correct configuration
is essential to ensure transactions are resolved correctly in the event of a server crash and restart.
See the crash recovery section below for details.
Additional properties that may be added to the jbossts-properties.xml include the following. Care
should be taken to place these in the appropriate section of the file, or they may not be correctly
processed.
• com.arjuna.ats.arjuna.coordinator.enableStatistics
This property enables the gathering of transaction statistics, which may be viewed via methods
on the TransactionManagerService bean or, more commonly, its corresponding JMX MBean. This
option is disabled by default, as the additional locking needed to record statistics accurately may
cause a slight performance impact. Thus the statistics getter methods will thus normally return zero
values. To enable the option, set its value to “YES” in the properties file.
Database connection pools configured via the application server's -ds.xml files using <xa-datasource>
(see chapter 11) will automatically interact with the transaction manager. i.e. Connections obtained
by looking up such datasource in JNDI and calling getConnection will automatically participate
correctly in an ongoing transaction. This is the dominant use case and should be preferred where
transactional guarantees for data access are required. For cases where the database cannot support
XA transactions, it is also feasible to deploy a connection pool using <local-xa-datasource> Such
datasources participate in the managed transaction using the last resource commit optimization (see
below) and as such provide more limited transactional guarantees. Applications using this approach
should be aware of the limitations and implemented accordingly. Connections obtained from a <no-
tx-datasource> will not interact with the transaction manager and any work done on such connections
must be explicitly committed or rolled back by the application via the JDBC API.
Many databases require additional configuration if they are to be used as XA resource managers.
For example, MS SQL Server requires configuration of the DTC service and installation of a server
side component of the JDBC drivers. Some versions of Oracle similarly require a server side package
73
Chapter 7. Transaction Management
to be installed in the database instance. PostgreSQL installations may require an alteration to the
number of outstanding transactions they permit – the default is normally too low for production usage.
MySQL has significant limitations on its XA implementation and is not recommended for use in an XA
transaction. If it is used, the InnoDB storage engine must be configured. Please consult your DBA or
database documentation for further product specific information. In addition, it is important to take any
further database configuration steps needed to support XA recovery, see the recovery section below.
JBoss Messaging provides an XA aware driver and can participate in XA transactions. This is also the
case for many 3rd party message queuing (JMS) products, subject to the same caveats on product
specific configuration as with databases.
It is not transactionally safe (or rather, it is even more unsafe) to use more than a single one-phase
resource in the same transaction. For this reason JBossTS treats an attempt to enlist a second such
resource as an error and will terminate the transaction. This use case is most commonly found in
applications migrating from JBossAS 4.0.x servers, where this usage was not considered an error.
Whenever possible the <local-tx-datasource> should be changed to <xa-datasource> to resolve the
difficulty. Where this is not possible, the transaction manager may be configured to allow multiple last
resources. Although this is not recommended, details of the configuration steps may be found on the
jboss.org wiki.
74
Recovery Configuration
After the prepare phase of the transaction commit, the transaction manager writes state information to
disk in order that, should a failure occur, it can still complete the transaction and thereby assure ACID
properties. The transaction log storage, which JBossTS terms the ObjectStore, should be configured
in accordance with the guidance provided above. In addition, it is necessary to configure the recovery
system to process the logs.
The information that is written into the logs includes the transaction identity and the xid values
associated to the XAResources enlisted with the transaction. In case where the XAResource itself
implements Serializable, it is also written to the log. Such cases simplify recovery considerably,
but unfortunately are a minority. Most XAResources, including those presented to the transaction
manager by the application server's <xa-datasource>, are not Serializable. Therefore, it is necessary
to explicitly provide the transaction manager with sufficient information to instantiate a new
XAResource instance connected to the same resource manager upon server restart. For resource
managers configured by <xa-datasource> elements in -ds.xml files, this is best achieved by using
the AppServerJDBCXARecovery plugin. One instance is required for each resource manager. JBoss
Messaging includes its own recovery plugin, which must be configured if messaging operations
are required within a transaction. Refer to the JBoss Messaging documentation for further details.
Users may develop their own plugins for resource managers not covered by the previous cases, by
implementing the XAResourceRecovery interface. See the JBossTS documentation for further details.
In addition to configuring the recovery plugins, it is necessary to set a unique identifier for each
JBossTS (i.e. application server) instance and to configure which node identifiers each server
will attempt to recover. For most situations it is recommended that each server instance operate
independently, using its own ObjectStore and recovering its own transactions. In such cases the
com.arjuna.ats.arjuna.xa.nodeIdentifier and com.arjuna.ats.jta.xaRecoveryNode properties in each
server instance should share the same value and this value should be unique to the instance.
7.7. Troubleshooting
This section presents guidance on the possible causes are resolutions for common transaction related
problems.
JBossTS sends log statements though two levels of filters. Firstly, its own internal logging
abstraction layer, then the application server's log4j. A log statement must pass both filters to be
printed. The most likely case is that's you have enabled only one or the other. See information on
the DebugLevel property above.
• My server logs show WARN Adding multiple last resources is disallowed. and my
transactions are aborted. Why?
You are probably using <local-xa-datasource> rather than <xa-datasource> See the Last Resource
Commit Optimization section above and refer to http://www.jboss.org/community/wiki/Multiple1pc
• My server crashed (or was killed). Now it's running again, but my logs are
filling with WARN [com.arjuna.ats.jta.logging.loggerI18N]
[com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] Could not
find new XAResource to use for recovering non-serializable XAResource
75
Chapter 7. Transaction Management
You probably forgot to configure recovery for one or more resource managers used in a
transaction. See the recovery section above and http://www.jboss.org/community/wiki/
TxNonSerializableXAResource
• My transactions take a long time and sometimes strange things happen. The server log contains
WARN [arjLoggerI18N] [BasicAction_58] - Abort of action id ... invoked
while multiple threads active within it.
Transactions which exceed their timeout may be rolled back. This is done by a background thread,
which can confuse some application code that may be expecting an interrupt. See the transaction
timeout handling section above and http://www.jboss.org/community/wiki/TxMultipleThreads
Additional information on these and other issues may be found in the JBossTS documentation and on
the wiki at http://www.jboss.org/community/wiki/JBossTransactions
76
Transaction Management Console
from the application server configuration. Manual configuration changes are necessary only for
deployments where applications require use of a transaction coordinator on a separate host, for which
the XTS documentation should be consulted. The directory tree created by unpacking the jbossxts.sar
file will contain a jbossxts-api.jar Developers may link against this .jar at build time, but should not
package it with their applications in order to avoid classloading problems. The remaining .jar files
contain internal implementation classes and should not be used directly by application code.
• txbridge
For certain use cases it is desirable to be able to invoke traditional transaction components such
as EJBs, within the scope of a Web Services transaction. Likewise, such components may wish to
invoke transactional web services. This involves two distinct transaction types: JTA i.e. XA based
transactions for the JEE components and WS-AT transactions for the web services. It is necessary
to integrate these transactions into a single entity spanning all the involved components. It is this
problem that the transaction bridge addresses.
• BA Framework
The techniques required for writing extended, compensation based transactions for WS-BA are
still in their infancy. The API provided by XTS is low level, requiring the business programmer
to undertake much of the transaction plumbing work. The BA Framework provides high level
annotations that move responsibility for much of the transaction handling into the application server
middleware. Using the framework, programmers can focus on writing business logic and thus be
considerably more productive.
77
Chapter 7. Transaction Management
The source code for JBossTS can be downloaded direct from the project's svn repository at http://
anonsvn.jboss.org/repos/labs/labs/jbosstm/ To find the version matching the binaries in JBoss
Enterprise Application Platform, consult your server logs. At startup the server prints a string similar to:
The value given for the tag corresponds to a tree under /tags/ in the svn repository. Note that the
version refers to the JBossTS releases consumed by Enterprise Application Platform, not the
Enterprise Application Platform release numbering. Users building the Enterprise Application Platform
from source may also consult the version.jboss.jbossts value in component-matix/pom.xml
Please note that installing any version of JBossTS other than those provided with the Enterprise
Application Platform distribution or its CP releases is not supported. Also note that, whilst some
JBossTS components are packaged individually, it is not supported to mix and match versions. i.e.
do not run the JTA from one tag with the XTS from another. API and functionality changes between
releases may make this unstable.
78
Chapter 8.
Remoting
The main objective of JBoss Remoting is to provide a single API for most network based invocations
and related services that use pluggable transports and data marshallers. The JBoss Remoting API
provides the ability for making synchronous and asynchronous remote calls, push and pull callbacks,
and automatic discovery of remoting servers. The intention is to allow for the addition of different
transports to fit different needs, yet still maintain the same API for making the remote invocations and
only requiring configuration changes, not code changes, to fit these different needs.
Out of the box, Remoting supplies multiple transports (bisocket, http, rmi, socket, servlet, and their
ssl enabled counterparts), standard and compressing data marshallers, and a configurable facility
for switching between standard jdk serialization and JBoss Serializabion. It is also capable of remote
classloading, has extensive facilities for connection failure notification, performs call by reference
optimization for client/server invocations collocated in a single JVM, and implements multihomed
servers.
In the Enterprise Application Platform, Remoting supplies the transport layer for the EJB2, EJB3, and
Messaging subsystems. In each case, the configuration of Remoting is largely predetermined and
fixed, but there are times when it is useful to know how to alter a Remoting configuration.
8.1. Background
A Remoting server consists of a Connector, which wraps and configures a transport specific server
invoker. A connector is represented by an InvokerLocator string, such as
socket://bluemonkeydiamond.com:8888/?timeout=10000&serialization=jboss
which indicates that a server using the socket transport is accessible at port 8888 of host
bluemonkeydiamond.com, and that the server is configured to use a socket timeout of 10000 and to
use JBoss Serialization. A Remoting client can use an InvokerLocator to connect to a given server.
In the Enterprise Application Platform, Remoting servers and clients are created far below the
surface and are accessible only through configuration files. Moreover, when a SLSB, for example, is
downloaded from the JNDI directory, it comes with a copy of the InvokerLocator, so that it knows how
to contact the appropriate Remoting server. The important fact to note is that, since the server and
its clients share the InvokerLocator, the parameters in the InvokerLocator serve to configure
both clients and servers.
8.2.1. MBeans
In the HornetQ JMS subsystem, a Remoting server is configured in the file remoting-bisocket-
service.xml, which, in abbreviated form, looks like
<mbean code="org.jboss.remoting.transport.Connector"
79
Chapter 8. Remoting
name="jboss.messaging:service=Connector,transport=bisocket"
display-name="Bisocket Transport Connector">
<attribute name="Configuration">
<config>
<invoker transport="bisocket">
<attribute name="marshaller"
isParam="true">org.jboss.jms.wireformat.JMSWireFormat</attribute>
<attribute name="unmarshaller"
isParam="true">org.jboss.jms.wireformat.JMSWireFormat</attribute>
<attribute name="serverBindAddress">${jboss.bind.address}</
attribute>
<attribute name="serverBindPort">4457</attribute>
<attribute name="callbackTimeout">10000</attribute>
...
</invoker>
...
</config>
</attribute>
</mbean>
The InvokerLocator is derived from this file. The important fact to note is that the attribute
"isParam" determines if a parameter is to be included in the InvokerLocator. If "isParam" is
omitted (or if it is set to "false"), then the parameter applies only to the server and is not communicated
to the client. It follows that the InvokerLocator for this Remoting server is (assuming the value of
${jboss.bind.address} is bluemonkeydiamond.com)
bisocket://bluemonkeydiamond.com:4457/?marshaller=
org.jboss.jms.wireformat.JMSWireFormat&
unmarshaller=org.jboss.jms.wireformat.JMSWireFormat
8.2.2. POJOs
The same Connector could be configured by way of the
org.jboss.remoting.ServerConfiguration POJO:
<bean name="HornetQConnector"
class="org.jboss.remoting.transport.Connector">
<annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX
(name="jboss.messaging:service=Connector,transport=bisocket",
80
POJOs
exposedInterface=org.jboss.remoting.transport.ConnectorMBean.class,
registerDirectly=true)</annotation>
<property name="serverConfiguration"><inject
bean="HornetQConfiguration"/></property>
</bean>
81
Chapter 8. Remoting
...
</bean>
which results in a StringBuffer with a value something like (according to the ServiceBindingManager
configuration values for HornetQConnector:bindingHome1 and HornetQConnector:bindingHome2)
"external.acme.com:5555!internal.acme.com:4444", and (2) replacing the "serverBindAddress" and
"serverBindPort" parameters with
<entry>
82
Address translation
<key>homes</key>
<value><value-factory bean="homes2" method="toString"/></value>
</entry>
bisocket://multihome/?homes=external.acme.com:5555!internal.acme.com:
4444&marshaller=org.jboss.jms.wireformat.JMSWireFormat&
unmarshaller=org.jboss.jms.wireformat.JMSWireFormat
EJB2: ${JBOSS_HOME}/server/${CONFIG}/deploy/remoting-jboss-beans.xml
EJB3: ${JBOSS_HOME}/server/${CONFIG}/deploy/ejb3-connectors-jboss-beans.xml
HornetQ: ${JBOSS_HOME}/server/${CONFIG}/deploy/messaging/remoting-bisocket-service.xml
1
http://www.jboss.org/jbossremoting/docs/guide/2.5/html/index.html
83
84
Chapter 9.
JBoss Messaging
JBoss Messaging is the new enterprise messaging system from JBoss. It is a complete rewrite
of JBossMQ, the legacy JBoss JMS provider. It is the default JMS provider on JBoss Enterprise
Application Platform 5.
JBoss Messaging is a high Performance JMS 1.1 compliant implementation integrated with JBoss
Transactions. It also offers:
• Transparent Failover
JBoss Messaging provides an open source and standards-based messaging platform that brings
enterprise-class messaging to the mass market. It also implements a high performance, robust
messaging core that is designed to support the largest and most heavily utilized SOA, enterprise
service buses (ESBs) and other integration needs ranging from the simplest to the highest network
demands.
It allows you to smoothly distribute your application load across your cluster, intelligently balancing and
utilizing each nodes CPU cycles, with no single point of failure. This provides a highly scalable and
performance implementation for clustering.
JBoss Messaging includes a JMS front-end to deliver messaging in a standards-based format as well
as being designed to be able to support other messaging protocols in the future.
<depends optional-attribute-name="SecurityStore"
proxy-type="org.jboss.jms.server.SecurityStore">
jboss.messaging:service=SecurityStore
</depends>
85
Chapter 9. JBoss Messaging
DefaultSecurityConfig
Default security configuration is used when the security configuration for a specific queue or topic has
not been overridden in the destination's deployment descriptor. It has exactly the same syntax and
semantics as in JBossMQ.
The DefaultSecurityConfig attribute element should contain one <security> element. The <security>
element can contain multiple <role> elements. Each <role> element defines the default access for that
particular role.
If the read attribute is true then that role will be able to read (create consumers, receive messaages or
browse) destinations by default. If the write attribute is true then that role will be able to write (create
producers or send messages) to destinations by default. If the create attribute is true then that role will
be able to create durable subscriptions on topics by default.
SecurityDomain
The JAAS security domain to be used by this server peer.
SuckerPassword
This defines how the SecurityStore will authenticate the sucker user (JBM.SUCKER).
<mbean code="org.jboss.jms.server.ServerPeer"
name="jboss.messaging:service=ServerPeer"
xmbean-dd="xmdesc/ServerPeer-xmbean.xml">
<!-- The unique id of the server peer - in a cluster each node MUST have a
unique value - must be an integer -->
<attribute name="ServerPeerID">${jboss.messaging.ServerPeerID:0}</
attribute>
<!-- The default JNDI context to use for queues when they are deployed
without specifying one -->
86
Configuring the ServerPeer
<attribute name="DefaultQueueJNDIContext">/queue</attribute>
<!-- The default JNDI context to use for topics when they are deployed
without specifying one -->
<attribute name="DefaultTopicJNDIContext">/topic</attribute>
<attribute name="PostOffice">jboss.messaging:service=PostOffice</
attribute>
<!-- The default Dead Letter Queue (DLQ) to use for destinations.
This can be overridden on a per destinatin basis -->
<attribute
name="DefaultDLQ">jboss.messaging.destination:service=Queue,name=DLQ</
attribute>
<attribute name="DefaultMaxDeliveryAttempts">10</attribute>
<!-- The default Expiry Queue to use for destinations. This can be
overridden on a per destinatin basis -->
<attribute name="DefaultExpiryQueue">
jboss.messaging.destination:service=Queue,name=ExpiryQueue
</attribute>
<attribute name="DefaultRedeliveryDelay">0</attribute>
<attribute name="MessageCounterSamplePeriod">5000</attribute>
<!-- The maximum amount of time for a client to wait for failover to start
on the server side after
it has detected failure -->
<attribute name="FailoverStartTimeout">60000</attribute>
<!-- The maximum amount of time for a client to wait for failover to
complete on the server side after
it has detected failure -->
<attribute name="FailoverCompleteTimeout">300000</attribute>
87
Chapter 9. JBoss Messaging
<attribute name="StrictTck">false</attribute>
<!-- The maximum number of days results to maintain in the message counter
history -->
<attribute name="DefaultMessageCounterHistoryDayLimit">-1</attribute>
<!-- The name of the connection factory to use for creating connections
between nodes to pull messages -->
<attribute name="ClusterPullConnectionFactoryName">
jboss.messaging.connectionfactory:service=ClusterPullConnectionFactory
</attribute>
<attribute name="DefaultPreserveOrdering">false</attribute>
<!-- Max. time to hold previously delivered messages back waiting for
clients to reconnect after failover -->
<attribute name="RecoverDeliveriesTimeout">300000</attribute>
<depends optional-attribute-
name="PersistenceManager">jboss.messaging:service=PersistenceManager</
depends>
<depends optional-attribute-
name="JMSUserManager">jboss.messaging:service=JMSUserManager</depends>
<depends>
jboss.messaging:service=Connector,transport=bisocket
88
Server Attributes
</depends>
<depends optional-attribute-name="SecurityStore"
proxy-type="org.jboss.jms.server.SecurityStore">
jboss.messaging:service=SecurityStore
</depends>
</mbean>
...
9.3.1. ServerPeerID
The ServerPeerID is the unique ID of the server peer that every node you deploy must have. This
applies whether the different nodes form a cluster, or are only linked via a message bridge. The ID
must be a valid integer.
Note
The scope of ServerPeerID is currently from 0 to 255.
9.3.2. DefaultQueueJNDIContext
The default JNDI context to use when binding queues. Defaults to /queue.
9.3.3. DefaultTopicJNDIContext
The default JNDI context to use when binding topics.wa Defaults to /topic.
9.3.4. PostOffice
This is the post office that the ServerPeer uses. You will not normally need to change this attribute.
The post office is responsible for routing messages to queues and maintaining the mapping between
addresses and queues.
9.3.5. DefaultDLQ
This is the name of the default DLQ (Dead Letter Queue) the server peer will use for destinations.
The DLQ can be overridden on a per destination basis - see the destination MBean configuration for
more details. A DLQ is a special destination where messages are sent when the server has attempted
to deliver them unsuccessfully more than a certain number of times. If the DLQ is not specified at all
then the message will be removed after the maximum number of delivery attempts. The maximum
number of delivery attempts can be specified using the attribute DefaultMaxDeliveryAttempts for a
global default or individually on a per destination basis.
9.3.6. DefaultMaxDeliveryAttempts
The default for the maximum number of times delivery of a message will be attempted before sending
the message to the DLQ, if configured.
89
Chapter 9. JBoss Messaging
The default value is 10.This value can also be overridden on a per destination basis.
9.3.7. DefaultExpiryQueue
This is the name of the default expiry queue the server peer will use for destinations. The expiry can
be overridden on a per destination basis - see the destination MBean configuration for more details.
An expiry queue is a special destination where messages are sent when they have expired. Message
expiry is determined by the value of Message::getJMSExpiration() If the expiry queue is not specified
at all then the message will be removed after it is expired.
9.3.8. DefaultRedeliveryDelay
When redelivering a message after failure of previous delivery it is often beneficial to introduce a delay
perform redelivery in order to prevent thrashing of delivery-failure, delivery-failure etc.
Change this if your application could benefit with a delay before redelivery. This value can also be
overridden on a per destination basis.
9.3.9. MessageCounterSamplePeriod
Periodically the server will query each queue to gets its statistics. This is the period.
9.3.10. FailoverStartTimeout
The maximum number of milliseconds the client will wait for failover to start on the server side when a
problem is detected.
9.3.11. FailoverCompleteTimeout
The maximum number of milliseconds the client will wait for failover to complete on the server side
after it has started. The default value is 300000 (five minutes).
9.3.12. DefaultMessageCounterHistoryDayLimit
JBoss Messaging provides a message counter history which shows the number of messages arriving
on each queue of a certain number of days. This attribute represents the maxiumum number of days
for which to store message counter history. It can be overridden on a per destination basis.
9.3.13. ClusterPullConnectionFactory
The name of the connection factory to use for pulling messages between nodes. If you wish to turn
off message sucking between queues altogether, but retain failover, then you can ommit this attribute
altogether.
9.3.14. DefaultPreserveOrdering
If true, then strict JMS ordering is preserved in the cluster. See the cluster configurations section for
more details. Default is false.
90
RecoverDeliveriesTimeout
9.3.15. RecoverDeliveriesTimeout
When failover occurs, already delivered messages will be kept aside, waiting for clients to reconnect.
In the case that clients never reconnect (e.g. the client is dead) then eventually these messages will
timeout and be added back to the queue. The value is in ms. The default is 5 mins.
9.3.16. SuckerPassword
JBoss Messaging internally makes connections between nodes in order to redistribute messages
between clustered destinations. These connections are made with the user name of a special
reserved user. On this parameter you define the password used as these connections are made.
You will need to configure the Sucker password in the SecurityStore mbean configuration file
(messaging-jboss-beans.xml), the ServerPeer Sucker password will be ignored.
Warning
This must be specified at install time, or the default password will be used. Any one who
then knows the default password will be able to gain access to any destinations on the
server. This value must be changed at install time.
9.3.17. StrictTCK
Set to true if you want strict JMS TCK semantiocs
9.3.18. Destinations
Returns a list of the destinations (queues and topics) currently deployed.
9.3.19. MessageCounters
JBoss Messaging provides a message counter for each queue.
9.3.20. MessageStatistics
JBoss Messaging provides statistics for each message counter for each queue.
9.3.21. SupportsFailover
Set to false to prevent server side failover occurring in a cluster when a node crashes.
9.3.22. PersistenceManager
This is the persistence manager that the ServerPeer uses. You will not normally need to change this
attribute.
9.3.23. JMSUserManager
This is the JMS user manager that the ServerPeer uses. You will not normally need to change this
attribute.
91
Chapter 9. JBoss Messaging
9.3.24. SecurityStore
This is the pluggable SecurityStore. If you redefine this SecurityStore, notice it will need to
authenticate the MessageSucker user ("JBM.SUCKER") with all the special permissions required by
clustering.
9.4.1. DeployQueue
This operation lets you programmatically deploy a queue. There are two overloaded versions of this
operation. If the queue already exists but is undeployed it is deployed. Otherwise it is created and
deployed. The name parameter represents the name of the destination to deploy. The jndiName
parameter (optional) represents the full jndi name where to bind the destination. If this is not specified
then the destination will be bound in <DefaultQueueJNDIContext>/<name>.
The first version of this operation deploys the destination with the default paging parameters. The
second overloaded version deploys the destination with the specified paging parameters. See the
section on configuring destinations for a discussion of what the paging parameters mean.
9.4.2. UndeployQueue
This operation lets you programmatically undeploy a queue. The queue is undeployed but is NOT
removed from persistent storage. This operation returns true if the queue was successfull undeployed.
otherwise it returns false.
9.4.3. DestroyQueue
This operation lets you programmatically destroy a queue. The queue is undeployed and then all its
data is destroyed from the database.
Warning
Be cautious when using this method since it will delete all data for the queue.
This operation returns true if the queue was successfully destroyed. otherwise it returns false.
9.4.4. DeployTopic
This operation lets you programmatically deploy a topic.
If the topic already exists but is undeployed it is deployed. Otherwise it is created and deployed.
The jndiName parameter (optional) represents the full jndi name where to bind the destination. If this is
not specified then the destination will be bound in <DefaultTopicJNDIContext>/<name>.
92
UndeployTopic
The first version of this operation deploys the destination with the default paging parameters. The
second overloaded version deploys the destination with the specified paging parameters. See the
section on configuring destinations for a discussion of what the paging parameters mean.
9.4.5. UndeployTopic
This operation lets you programmatically undeploy a topic. The queue is undeployed but is NOT
removed from persistent storage. This operation returns true if the topic was successfully undeployed.
otherwise it returns false.
9.4.6. DestroyTopic
This operation lets you programmatically destroy a topic.
The topic is undeployed and then all its data is destroyed from the database.
Warning
Be careful when using this method since it will delete all data for the topic.
This operation returns true if the topic was successfully destroyed. otherwise it returns false.
9.4.7. ListMessageCountersAsHTML
This operation returns message counters in an easy to display HTML format.
9.4.8. ResetAllMesageCounters
This operation resets all message counters to zero.
9.4.9. EnableMessageCounters
This operation enables all message counters for all destinations. Message counters are disabled by
default.
9.4.10. DisableMessageCounters
This operation disables all message counters for all destinations. Message counters are disabled by
default.
9.4.11. RetrievePreparedTransactions
Retrieves a list of the Xids for all transactions currently in a prepared state on the node.
9.4.12. ShowPreparedTransactionsAsHTML
Retrieves a list of the Xids for all transactions currently in a prepared state on the node in an easy to
display HTML format.
93
94
Chapter 10.
Please note that in this chapter, we explain how to use alternative databases to support all services
in JBoss Enterprise Application Platform. This includes all the system level services such as EJB
and JMS. For individual applications (e.g., WAR or EAR) deployed in JBoss Enterprise Application
Platform, you can still use any backend database by setting up the appropriate data source
connection.
We assume that you have already installed the external database server, and have it running. You
should create an empty database named jboss, accessible via the username / password pair
jbossuser / jbosspass. The jboss database is used to store JBoss Enterprise Application
Platform internal data -- JBoss Enterprise Application Platform will automatically create tables and
data in it.
• MySQL JDBC drivers can be downloaded from the MySQL web site http://www.mysql.com/products/
connector/.
• Postgres JDBC drivers can be downloaded from the Postgres web site http://jdbc.postgresql.org/.
• Oracle JDBC drivers can be downloaded from the Oracle web site http://www.oracle.com/
1
technology/software/tech/java/sqlj_jdbc/index.html .
• IBM DB2 JDBC drivers can be downloaded from the IBM web site http://www-306.ibm.com/
software/data/db2/java/.
• Sybase JDBC drivers can be downloaded from the Sybase jConnect product page http://
www.sybase.com/products/allproductsa-z/softwaredeveloperkit/jconnect
• MS SQL Server JDBC drivers can be downloaded from the MSDN web site http://
msdn.microsoft.com/data/jdbc/.
95
Chapter 10. Use Alternative Databases with JBoss Enterprise Application Platform
If java is not enabled you might see this exception being thrown when you try to use any of the above
services.
CMP Configuration
To use Container Managed Persistence for user defined Java objects with Sybase Adaptive Server
Enterprise the java classes should be installed in the database. The system table 'sysxtypes' contains
one row for each extended, Java-SQL datatype. This table is only used for Adaptive Servers enabled
for Java. Install java classes using the installjava program.
2. The jar file you are trying to install should be created without compression.
3. Java classes that you install and use in the server must be compiled with JDK 1.2.2.
If you compile a class with a later JDK, you will be able to install it in the server using
the installjava utility, but you will get a java.lang.ClassFormatError exception when you
attempt to use the class. This is because Sybase Adaptive Server uses an older JVM
internally, and hence requires the java classes to be compiled with the same.
96
Configuring JDBC DataSources
Figure 10.1. The simplified JCA DataSource configuration descriptor top-level schema elements
Multiple datasource configurations may be specified in a configuration deployment file. The child
elements of the datasources root are:
• mbean: Any number mbean elements may be specified to define MBean services that should be
included in the jboss-service.xml descriptor that results from the transformation. This may be
used to configure services used by the datasources.
97
Chapter 10. Use Alternative Databases with JBoss Enterprise Application Platform
adaptor ManagedConnection. To ensure that all work in a local transaction occurs over the same
ManagedConnection, it includes a xid to ManagedConnection map. When a Connection
is requested or a transaction started with a connection handle in use, it checks to see if a
ManagedConnection already exists enrolled in the global transaction and uses it if found.
Otherwise, a free ManagedConnection has its LocalTransaction started and is used. The
xa-datasource child element schema is given in Figure 10.4, “The XA DataSource configuration
schema”.
98
Configuring JDBC DataSources
99
Chapter 10. Use Alternative Databases with JBoss Enterprise Application Platform
100
Configuring JDBC DataSources
101
Chapter 10. Use Alternative Databases with JBoss Enterprise Application Platform
Figure 10.5. The schema for the experimental non-XA DataSource with failover
102
Configuring JDBC DataSources
Figure 10.6. The schema for the experimental XA Datasource with failover
• jndi-name: The JNDI name under which the DataSource wrapper will be bound. Note that this
name is relative to the java:/ context, unless use-java-context is set to false. DataSource
wrappers are not usable outside of the server VM, so they are normally bound under the java:/,
which isn't shared outside the local VM.
• use-java-context: If this is set to false the the datasource will be bound in the global JNDI context
rather than the java: context.
103
Chapter 10. Use Alternative Databases with JBoss Enterprise Application Platform
• user-name: This element specifies the default username used when creating a new connection.
The actual username may be overridden by the application code getConnection parameters or
the connection creation context JAAS Subject.
• password: This element specifies the default password used when creating a new connection. The
actual password may be overridden by the application code getConnection parameters or the
connection creation context JAAS Subject.
• security-domain: Specifying this element indicates that connections in the pool should be
distinguished by JAAS Subject based information. The content of the security-domain is the
name of the JAAS security manager that will handle authentication. This name correlates to the
JAAS login-config.xml descriptor application-policy/name attribute.
• use-strict-min: This element specifies whether idle connections that are below the minimum pool
size should be closed.
• min-pool-size: This element specifies the minimum number of connections a pool should hold.
These pool instances are not created until an initial request for a connection is made. This defaults
to 0.
• max-pool-size: This element specifies the maximum number of connections for a pool. No more
than the max-pool-size number of connections will be created in a pool. This defaults to 20.
• blocking-timeout-millis: This element specifies the maximum time in milliseconds to block while
waiting for a connection before throwing an exception. Note that this blocks only while waiting for
a permit for a connection, and will never throw an exception if creating a new connection takes an
inordinately long time. The default is 5000.
• idle-timeout-minutes: This element specifies the maximum time in minutes a connection may be
idle before being closed. The actual maximum time depends also on the IdleRemover scan time,
which is 1/2 the smallest idle-timeout-minutes of any pool.
• new-connection-sql: This is a SQL statement that should be executed when a new connection is
created. This can be used to configure a connection with database specific settings not configurable
via connection properties.
104
Configuring JDBC DataSources
• org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter
• org.jboss.resource.adapter.jdbc.vendor.MySQLExceptionSorter
• org.jboss.resource.adapter.jdbc.vendor.SybaseExceptionSorter
• org.jboss.resource.adapter.jdbc.vendor.InformixExceptionSorte
• track-statements: This boolean element specifies whether to check for unclosed statements
when a connection is returned to the pool. If true, a warning message is issued for each unclosed
statement. If the log4j category org.jboss.resource.adapter.jdbc.WrappedConnection
has trace level enabled, a stack trace of the connection close call is logged as well. This is a debug
feature that can be turned off in production.
• depends: The depends element specifies the JMX ObjectName string of a service that the
connection manager services depend on. The connection manager service will not be started until
the dependent services have been started.
• type-mapping: This element declares a default type mapping for this datasource. The type mapping
should match a type-mapping/name element from standardjbosscmp-jdbc.xml.
• connection-url: This is the JDBC driver connection URL string, for example,
jdbc:hsqldb:hsql://localhost:1701.
• driver-class: This is the fully qualified name of the JDBC driver class, for example,
org.hsqldb.jdbcDriver.
• TRANSACTION_READ_UNCOMMITTED
• TRANSACTION_READ_COMMITTED
• TRANSACTION_REPEATABLE_READ
• TRANSACTION_SERIALIZABLE
105
Chapter 10. Use Alternative Databases with JBoss Enterprise Application Platform
• TRANSACTION_NONE
• no-tx-separate-pools: The presence of this element indicates that two connection pools are
required to isolate connections used with JTA transaction from those used without a JTA transaction.
The pools are lazily constructed on first use. Its use case is for Oracle (and possibly other vendors)
XA implementations that don't like using an XA connection with and without a JTA transaction.
• track-connection-by-tx: Specifying a true value for this element makes the connection
manager keep an xid to connection map and only put the connection back in the pool when the
transaction completes and all the connection handles are closed or disassociated (by the method
calls returning). As a side effect, we never suspend and resume the xid on the connection's
XAResource. This is the same connection tracking behavior used for local transactions.
The XA spec implies that any connection may be enrolled in any transaction using any xid for
that transaction at any time from any thread (suspending other transactions if necessary). The
original JCA implementation assumed this and aggressively delisted connections and put them
back in the pool as soon as control left the EJB they were used in or handles were closed. Since
some other transaction could be using the connection the next time work needed to be done on
the original transaction, there is no way to get the original connection back. It turns out that most
XADataSource driver vendors do not support this, and require that all work done under a particular
xid go through the same connection.
• isSameRM-override-value: A boolean flag that allows one to override the behavior of the
javax.transaction.xa.XAResource.isSameRM(XAResource xaRes) method
behavior on the XA managed connection. If specified, this value is used unconditionally as the
isSameRM(xaRes) return value regardless of the xaRes parameter.
• url-delimeter: This element specifies a character used to separate multiple JDBC URLs.
• url-property: In the case of XA datasources, this property specifies the name of the xa-
datasource-property that contains the list of JDBC URLs to use.
106
Creating a DataSource for the External Database
• MySQL: mysql-ds.xml
• PostgreSQL: postgres-ds.xml
• Oracle: oracle-ds.xml
• DB2: db2-ds.xml
• Sybase: sybase-ds.xml
The following code snippet shows the mysql-ds.xml file as an example. All the other *-ds.xml
files are very similiar. You will need to change the connection-url, as well as the user-name /
password, to fit your own database server installation.
<datasources>
<local-tx-datasource>
<jndi-name>MySqlDS</jndi-name>
<connection-url>jdbc:mysql://localhost:3306/jboss</connection-url>
<driver-class>com.mysql.jdbc.Driver</driver-class>
<user-name>jbossuser</user-name>
<password>jbosspass</password>
<exception-sorter-class-name>
org.jboss.resource.adapter.jdbc.vendor.MySQLExceptionSorter
</exception-sorter-class-name>
<!-- should only be used on drivers after 3.22.1 with "ping" support
<valid-connection-checker-class-name>
org.jboss.resource.adapter.jdbc.vendor.MySQLValidConnectionChecker
</valid-connection-checker-class-name>
-->
<!-- sql to call when connection is created
<new-connection-sql>some arbitrary sql</new-connection-sql>
-->
<!-- sql to call on an existing pooled connection when it is obtained from
pool -
MySQLValidConnectionChecker is preferred for newer drivers
<check-valid-connection-sql>some arbitrary sql</check-valid-connection-sql>
-->
107
Chapter 10. Use Alternative Databases with JBoss Enterprise Application Platform
</datasources>
Once you customized the *-ds.xml file to connect to your external database, you need to copy it
to the <JBoss_Home>/server/all/deploy directory. The database connection is now available
through the JNDI name specified in the *-ds.xml file.
10.4.1. General
• <mbean> - a standard jboss mbean deployment
• <jndi-name> - the jndi name where it is bound. This is prefixed with java by default:
• <use-java-context> - set this to false to drop the java: context from the jndi name
10.4.2. XA
<xa-resource-timeout> - the number of seconds passed to
XAResource.setTranasctionTimeout()
when not zero. This feature is available on JBoss Enterprise Application Platform 4.0.3 and above.
• nothing - uses the user/password specified in -ds.xml for DataSources or the getConnection/
createConnection method without a user/password (the default).
108
Change Database for the JMS Services
• <blocking-timeout-millis> - the length of time to wait for a connection to become available when all
the connections are checked out (default 5000 == 5 seconds, from 3.2.4 it is 30000 == 30 seconds)
• <idle-timeout-minutes> - the number of minutes after which unused connections are closed (default
15 minutes)
• <interleaving/> - enables interleaving for XA connection factories (this feature was added in
JBoss-5.x)
• <prefill> - whether to attempt to prefill the connection pool to the minimum number of connections.
NOTE: only supporting pools (OnePool) support this feature. A warning can be found in the logs if
the pool does not support this. This feature is available in JBoss 4.0.5 and above.
• <use-fast-fail> - Whether or not to continue to attempt to acquire a connection from the pool even
if the nth attempt has failed. False by default. This is to address performance issues where SQL
validation may take significant time and resources to execute.
Note
The min and max pool size are per subpool so be careful with these parameters if you
have lots of identities.
• MySQL: mysql-persistence-service.xml
• PostgreSQL: postgresql-persistence-service.xml
• Oracle: oracle-persistence-service.xml
109
Chapter 10. Use Alternative Databases with JBoss Enterprise Application Platform
• DB2: db2-persistence-service.xml
• Sybase: sybase-persistence-service.xml
<fk-constraint>true</fk-constraint>
110
The Easy Way
To change the JNDI name, just open the *-ds.xml file for your external database, and change the
value of the jndi-name property to DefaultDS. For instance, in mysql-ds.xml, you'd change
MySqlDS to DefaultDS and so on. You will need to remove the <JBoss_Home>/server/all/
deploy/hsqldb-ds.xml file after you are done to avoid duplicated DefaultDS definition.
.. ...
<mbean code="org.jboss.messaging.core.jmx.JDBCPersistenceManagerService"
name="jboss.messaging:service=PersistenceManager" xmbean-dd="xmdesc/
JDBCPersistenceManager-xmbean.xml">
<depends>jboss.jca:service=DataSourceBinding,name=DefaultDS</depends>
A safer and more flexible way to hook up JBoss Enterprise Application Platform services with the
external datasource is to manually change the DefaultDS in all standard JBoss services to the
datasource JNDI name defined in your *-ds.xml file (e.g., the MySqlDS in mysql-ds.xml etc.).
Below is a complete list of files that contain DefaultDS. You can update them all to use the external
database on all JBoss services or update some of them to use different combination of datasources
for different services.
• ${jboss.dist}/server/${server}/deploy/juddi-service.sar/META-INF/jboss-
service.xml: This file configures the UUDI service.
• ${jboss.dist}/server/${server}/deploy/juddi-service.sar/juddi.war/WEB-
INF/jboss-web.xml: This file configures the UUDI service.
• <JBoss_Home>/server/all/deploy/juddi-service.sar/juddi.war/WEB-INF/
juddi.properties: This file configures the UUDI service.
111
Chapter 10. Use Alternative Databases with JBoss Enterprise Application Platform
• ${jboss.dist}/server/${server}/deploy/uuid-key-generator.sar/META-INF/
jboss-service.xml: This file configures the UUDI service.
• ${jboss.dist}/server/${server}/deploy/messaging/messaging-jboss-
beans.xml and ${jboss.dist}/server/${server}/deploy/messaging/persistence-
service.xml: Those files configure the JMS persistence service as we discussed earlier.
The Oracle database creates tables of the form schemaname.tablename. The TIMERS and
HILOSEQUENCES tables needed by JBoss Enterprise Application Platform would not get created on a
schema if the table already exists on a different schema. To work around this issue, you need to edit
the ${jboss.dist}/server/${server}/deploy/ejb2-timer-service.xml file to change
the table name from TIMERS to something like schemaname2.tablename.
<mbean code="org.jboss.ejb.txtimer.DatabasePersistencePolicy"
name="jboss.ejb:service=EJBTimerService,persistencePolicy=database">
<!-- DataSourceBinding ObjectName -->
<depends optional-attribute-name="DataSource">
jboss.jca:service=DataSourceBinding,name=DefaultDS
</depends>
<!-- The plugin that handles database persistence -->
<attribute name="DatabasePersistencePlugin">
org.jboss.ejb.txtimer.GeneralPurposeDatabasePersistencePlugin
</attribute>
<!-- The timers table name -->
<attribute name="TimersTable">TIMERS</attribute>
</mbean>
<depends>jboss:service=TransactionManager</depends>
112
DataSource configuration
• <no-tx-datasource> - a DataSource that does not take part in JTA transactions using a
java.sql.Driver
• <local-tx-datasource> - a DataSource that does not support two phase commit using a
java.sql.Driver
• <connection-property> - used to configure the connections retrieved from the java.sql.Driver. For
example:
<connection-property name="char.encoding">UTF-8</connection-property>
• <xa-datasource-property> - This contains that properties that are used to configure the
XADataSource. For example:
<xa-datasource-property name="IfxWAITTIME">10</xa-datasource-property>
<xa-datasource-property name="IfxIFXHOST">myhost.mydomain.com</xa-
datasource-property>
<xa-datasource-property name="PortNumber">1557</xa-datasource-property>
<xa-datasource-property name="DatabaseName">mydb</xa-datasource-property>
<xa-datasource-property name="ServerName">myserver</xa-datasource-property>
• <isSameRM-override-value> - In order to fix issues with Oracle this property should be set to false
113
Chapter 10. Use Alternative Databases with JBoss Enterprise Application Platform
• <use-java-context> - A boolean indicating if the jndi-name should be prefixed with java: which
causes the DataSource to only be accessible from within the jboss server vm. The default is true.
• <user-name> - the user name used when creating the connection (not used when security is
configured)
• <password> - the password used when creating the connection (not used when security is
configured)
• <transaction-isolation> - the default transaction isolation of the connection (unspecified means use
the default provided by the database):
• TRANSACTION_READ_UNCOMMITTED
• TRANSACTION_READ_COMMITTED
• TRANSACTION_REPEATABLE_READ
• TRANSACTION_SERIALIZABLE
• TRANSACTION_NONE
• <new-connection-sql> - an sql statement that is executed against each new connection. This can be
used to set the connection schema, etc.
• <check-valid-connection-sql> - an sql statement that is executed before it is checked out from the
pool to make sure it is still valid. If the sql fails, the connection is closed and new ones created.
• <track-statements> - (a) whether to monitor for unclosed Statements and ResultSets and issue
warnings when the user forgets to close them (default nowarn)
• <share-prepared-statements> - (b) with prepared statement cache enabled whether two requests in
the same transaction should return the same statement (from jboss-4.0.2 - default false).
• <set-tx-query-timeout> - whether to enable query timeout based on the length of time remaining until
the transaction times out (default false - NOTE: This was NOT ported to 4.0.x until 4.0.3)
• <query-timeout> - a static configuration of the maximum of seconds before a query times out (since
4.0.3)
114
Common DataSource parameters
• <validate-on-match> - Prior to JBoss 4.0.5, connection validation occurred when the JCA layer
attempted to match a managed connection. With the addition of <background-validation> this is no
longer required. Specifying <validate-on-match> forces the old behavior. NOTE: this is typically NOT
used in conjunction with <background-validation>
• <prefill> - whether to attempt to prefill the connection pool to the minimum number of connections.
NOTE: only supporting pools (OnePool) support this feature. A warning can be found in the logs if
the pool does not support this. This feature will appear in JBoss 4.0.5.
• <idle-timeout-minutes> - indicates the maximum time a connection may be idle before being closed.
Default is 15 minutes.
• <url-delimiter> - From JBoss5 database failover is part of the main datasource config
• <url-property> - From JBoss5 database failover is part of the main datasource config
• <stale-connection-checker-class-name> - An implementation of
org.jboss.resource.adapter.jdbc.StateConnectionChecker that will decide whether SQLExceptions
that notify of bad connections throw org.jboss.resource.adapter.jdbc.StateConnectionException
(from JBoss5)
From JBoss Enterprise Application Platform 3.2.6 and above, track-statements has a new option:
<track-statements>nowarn</track-statements
This option closes Statements and ResultSets without a warning. It is also the new default value.
The purpose is to workaround questionable driver behavior where the driver applies auto-commit
semantics to local transactions.
Assuming the prepared statements are the same. For some drivers, ps2.executeQuery() will
automatically close rs1 so we actually need two real prepared statements behind the scenes. This
*should* only be for the auto-commit semantic where re-running the query starts a new transaction
automatically. For drivers that follow the spec, you can set it to true to share the same real prepared
statement.
115
Chapter 10. Use Alternative Databases with JBoss Enterprise Application Platform
<!--pooling parameters-->
<min-pool-size>5</min-pool-size>
<max-pool-size>100</max-pool-size>
<blocking-timeout-millis>5000</blocking-timeout-millis>
<idle-timeout-minutes>15</idle-timeout-minutes>
<!-- sql to call when connection is created
<new-connection-sql>some arbitrary sql</new-connection-sql>
-->
<set-tx-query-timeout/>
<query-timeout>300</query-timeout> <!-- maximum of 5 minutes for queries --
>
<!-- If you supply the usr/pw from a JAAS login module -->
<security-domain>MyRealm</security-domain>
<!-- if your app supplies the usr/pw explicitly getConnection(usr, pw) -->
<application-managed-security/>
</local-tx-datasource>
116
Generic Datasource Sample
<!-- you can include regular mbean configurations like this one -->
<mbean code="org.jboss.tm.XidFactory"
name="jboss:service=XidFactory">
<attribute name="Pad">true</attribute>
</mbean>
<user-name>x</user-name>
<password>y</password>
<transaction-isolation>TRANSACTION_SERIALIZABLE</transaction-isolation>
<!--pooling parameters-->
<min-pool-size>5</min-pool-size>
<max-pool-size>100</max-pool-size>
<blocking-timeout-millis>5000</blocking-timeout-millis>
<idle-timeout-minutes>15</idle-timeout-minutes>
<!-- sql to call when connection is created
<new-connection-sql>some arbitrary sql</new-connection-sql>
-->
<!-- If you supply the usr/pw from a JAAS login module -->
<security-domain/>
<!-- if your app supplies the usr/pw explicitly getConnection(usr, pw) -->
<application-managed-security/>
</xa-datasource>
</datasources>
117
Chapter 10. Use Alternative Databases with JBoss Enterprise Application Platform
<datasources>
<local-tx-datasource>
<jndi-name>DefaultDS</jndi-name>
<connection-url>jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)
(HOST=YOUR_HOST)(PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=YOUR_SERVICE_NAME)))</connection-url>
<driver-class>oracle.jdbc.driver.OracleDriver</driver-class>
<user-name>jboss</user-name>
<password>password</password>
<exception-sorter-class-
name>org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter</
exception-sorter-class-name>
<metadata>
<type-mapping>Oracle9i</type-mapping>
</metadata>
<transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-
isolation>
</local-tx-datasource>
</datasources>
<datasources>
<xa-datasource>
<jndi-name>YourJndiName</jndi-name>
<track-connection-by-tx>true</track-connection-by-tx>
<isSameRM-override-value>false</isSameRM-override-value>
<xa-datasource-class>oracle.jdbc.xa.client.OracleXADataSource</xa-
datasource-class>
<xa-datasource-property
name="URL">jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)
(HOST=YOUR_HOST)(PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=YOUR_SERVICE_NAME)))</xa-datasource-property>
<xa-datasource-property name="User">your_user</xa-datasource-property>
<xa-datasource-property name="Password">your_password</xa-datasource-
property>
<!-- Checks the Oracle error codes and messages for fatal errors -->
<exception-sorter-class-
name>org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter</
exception-sorter-class-name>
<!-- Oracles XA datasource cannot reuse a connection outside a
transaction once enlisted in a global transaction and vice-versa -->
<no-tx-separate-pools/>
<!-- corresponding type-mapping in the standardjbosscmp-jdbc.xml
(optional) -->
118
Configuring a DataSource for remote usage
<metadata>
<type-mapping>Oracle9i</type-mapping>
</metadata>
</xa-datasource>
</datasources>
You can also use the following URL structure, which supports load-balancing:
jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=ON)
(ADDRESS=(PROTOCOL=TCP)(HOST=YOUR_NODE_1)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)
(HOST=YOUR_NODE_2)(PORT=1521)))
(CONNECT_DATA=(SERVICE_NAME=YOUR_SERVICE_NAME)))
<datasources>
<local-tx-datasource>
<jndi-name>GenericDS</jndi-name>
<use-java-context>false</use-java-context>
<connection-url>...</connection-url>
This results in the DataSource being bound under the JNDI name "GenericDS" instead of the default
of "java:/GenericDS" which restricts the lookup to the same VM as the jboss server.
Note
JBoss does not recommend using this feature on a production environment. It requires
accessing a connection pool remotely and this is an anti-pattern as connections are
not serializable. Besides, transaction propagation is not supported and it could lead to
connection leaks if the remote clients are unreliable (i.e crashes, network failure). If you
do need to access a datasource remotely, JBoss recommends accessing it via a remote
session bean facade.
<datasources>
<local-tx-datasource>
...
<security-domain>MyDomain</security-domain>
...
</local-tx-datasource>
119
Chapter 10. Use Alternative Databases with JBoss Enterprise Application Platform
</datasources>
Add an application-policy to the login-config.xml file. The authentication section should include the
configuration for your login-module. For example, if you want to encrypt the database password, use
the SecureIdentityLoginModule login module.
<application-policy name="MyDomain">
<authentication>
<login-module code="org.jboss.resource.security.SecureIdentityLoginModule"
flag="required">
<module-option name="username">scott</module-option>
<module-option name="password">-170dd0fbd8c13748</module-option>
<module-option name="managedConnectionFactoryName">
jboss.jca:service=LocalTxCM,name=OracleDSJAAS
</module-option>
</login-module>
</authentication>
</application-policy>
In case you plan to fetch the data source connection from a web application, make sure authentication
is turned on for the web application. This is in order for the Subject to be populated. If you wish
for users to be able to connect anonymously, an additional login module needs to be added to the
application-policy, in order to populate the security credentials. Add the UsersRolesLoginModule
as the first login module in the chain. The usersProperties and rolesProperties parameters can be
directed to dummy files.
<login-module code="org.jboss.security.auth.spi.UsersRolesLoginModule"
flag="required">
<module-option name="unauthenticatedIdentity">nobody</module-option>
<module-option name="usersProperties">props/users.properties</module-
option>
<module-option name="rolesProperties">props/roles.properties</module-
option>
</login-module>
120
Chapter 11.
Pooling
11.1. Strategy
1
JBossJCA uses a ManagedConnectionPool to perform the pooling. The
ManagedConnectionPool is made up of subpools depending upon the strategy chosen and other
pooling parameters.
Note
The xml names imply this is just about security. This is misleading.
(
ConnectionRequestInfo
)
Note
This is the only supported behaviour for "local" transactions. This element is deprecated
in JBoss Enterprise Application Platform 5 where transaction stickiness is enabled by
default. XA users can explicitly enable interleaving with <interleaving/> element.
1
http://www.jboss.org/community/wiki/JBossJCA
121
Chapter 11. Pooling
Upto <max-pool-size/> threads can be inside the pool at the same time (or using connections from a
pool).
Once this limit is reached, threads wait for the <blocking-timeout-seconds/> to use the pool before
2
throwing a No Managed Connections Available
You may want to use the <allocation-retry/> and <allocation-retry-wait-millis/> elements to have the
pool retry to obtain a connection before throwing the exception.
• <min-pool-size/> - When the number of connections falls below this size, new connections are
created
• <prefill/> - Feature Request has been implemented for 4.0.5. Note: the only pooling strategy that
supports this feature is OnePool?, or ByNothing? pooling criteria.
The pool filling is done by a separate "Pool Filler" thread rather than blocking application threads.
Idle checking is done on a separate "Idle Remover" thread on an LRU (least recently used) basis. The
check is done every idle-timeout-minutes divided by 2 for connections unused for idle-timeout-minutes.
The pool itself operates on an MRU (most recently used) basis. This allows the excess connections to
be easily identified.
Should closing idle connections cause the pool to fall below the min-pool-size, new/fresh connections
are created.
Note
If you have long running transactions and you use interleaving (i.e. don't track-connection-
by-tx) make sure the idle timeout is greater than the transaction timeout. When
2
http://www.jboss.org/community/wiki/WhatDoesTheMessageNoManagedConnectionsAvailableMean
122
Dead connections
interleaving the connection is returned to the pool for others to use. If however nobody
does use it, it would be a candidate for removal before the transaction is committed.
before handing the connection to the application. If this fails, another connection is selected until there
are no more connections at which point new connections are constructed.
The potentially more performant check is to use vendor specific features, e.g. Oracle's or MySQL's
pingDatabase() via the
<valid-connection-checker-class-name/>
<exception-sorter-class-
name>org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter</
exception-sorter-class-name>
For
FATAL
4
http://www.jboss.org/community/docs/DOC-10020?uniqueTitle=false
123
124
Chapter 12.
2. You have <track-connection-by-tx/> in your oracle-xa-ds.xml (not necessarily for JBoss Enterprise
Application Platform 5.x where it is enabled by default and the element is deprecated).
5. That your jbosscmp-jdbc.xml is specifying the same version of oracle as the one you use.
Configuring Oracle Database for XA Support You can configure Oracle database to support XA
resources. This enables you to use JDBC 2.0-compliant Oracle driver. To XA-initialize Oracle
database, complete the following steps:
Make sure that Oracle JServer is installed with your database. If it is not installed, you must add it
using Oracle Database Configuration Assistant. Choose "Change an Existing DB" and then select the
database to which you want to add Oracle JServer. Choose "Next", then "Oracle JServer" and then
"Finish". If the settings you have made to your database previously, are not suitable or insufficient
for the Oracle JServer installation, the system prompts you to enter additional parameters. The
database configuration file ( init.ora ) is located in \oracle\admin\<your_db_name>\pfile
directory. Execute initxa.sql over your database. By default, this script file is located in \oracle
\ora81\javavm\install. If errors occur during the execution of the file, you must execute the SQL
statements from the file manually. Use DBA Studio to create a package and package body named
JAVA_XA in SYS schema, and a synonym of this package (also named JAVA_XA) in PUBLIC schema.
A slightly more detailed set of instructions can be found at Configuring and using XA distributed
1
transactions in WebSphere Studio - Oracle Exception section .
1
http://www.ibm.com/developerworks/websphere/library/techarticles/0407_woolf/0407_woolf.html?
ca=dnp-327#oracle_exception
125
126
Part III. Clustering Guide
Chapter 13.
The JBoss Enterprise Application Platform comes with clustering support out of the box, as part of the
all configuration. The all configuration includes support for the following:
• A scalable, fault-tolerant JNDI implementation (HA-JNDI).
• Ability to integrate with hardware and software load balancers, including special integration with
mod_jk and other JK-based software load balancers.
• EJB session bean clustering, for both stateful and stateless beans, and for both EJB3 and EJB2.
• A framework for keeping local EJB2 entity caches consistent across a cluster by invalidating cache
entries across the cluster when a bean is changed on any node.
• Deploying a service or application on multiple nodes in the cluster but having it active on only one
(but at least one) node is called a HA Singleton.
• Keeping deployed content in sync on all nodes in the cluster via the Farm service.
In this Clustering Guide we aim to provide you with an in depth understanding of how to use JBoss
Enterprise Application Platform's clustering features. In this first part of the guide, the goal is to
provide some basic "Quick Start" steps to encourage you to start experimenting with JBoss Enterprise
Application Platform Clustering, and then to provide some background information that will allow you
to understand how JBoss Enterprise Application Platform Clustering works. The next part of the guide
then explains in detail how to use these features to cluster your JEE services. Finally, we provide
some more details about advanced configuration of JGroups and JBoss Cache, the core technologies
that underlie JBoss Enterprise Application Platform Clustering.
129
Chapter 13. Introduction and Quick Start
• Install JBoss Enterprise Application Platform on all your servers. In its simplest form, this is
just a matter of unzipping the JBoss download onto the filesystem on each server.
If you want to run multiple JBoss Enterprise Application Platform instances on a single server,
you can either install the full JBoss distribution onto multiple locations on your filesystem, or you
can simply make copies of the all configuration. For example, assuming the root of the JBoss
distribution was unzipped to /var/jboss, you would:
$ cd /var/jboss/server
$ cp -r all node1
$ cp -r all node2
• For each node, determine the address to bind sockets to. When you start JBoss, whether
clustered or not, you need to tell JBoss on what address its sockets should listen for traffic. (The
default is localhost which is secure but isn't very useful, particularly in a cluster.) So, you need to
decide what those addresses will be.
• Ensure multicast is working. By default JBoss Enterprise Application Platform uses UDP multicast
for most intra-cluster communications. Make sure each server's networking configuration supports
multicast and that multicast support is enabled for any switches or routers between your servers.
If you are planning to run more than one node on a server, make sure the server's routing table
includes a multicast route. See the JGroups documentation at http://www.jgroups.org for more on
this general area, including information on how to use JGroups' diagnostic tools to confirm that
multicast is working.
Note
JBoss Enterprise Application Platform clustering does not require the use of UDP
multicast; the Enterprise Application Platform can also be reconfigured to use TCP
unicast for intra-cluster communication.
• Determine a unique integer "ServerPeerID" for each node. This is needed for JBoss Messaging
clustering, and can be skipped if you will not be running JBoss Messaging (i.e. you will remove JBM
from your server configuration's deploy directory). JBM requires that each node in a cluster has a
unique integer id, known as a "ServerPeerID", that should remain consistent across server restarts.
A simple 1, 2, 3, ..., x naming scheme is fine. We'll cover how to use these integer ids in the next
section.
Beyond the above required steps, the following two optional steps are recommended to help ensure
that your cluster is properly isolated from other JBoss Enterprise Application Platform clusters that
may be running on your network:
• Pick a unique name for your cluster. The default name for a JBoss Enterprise Application
Platform cluster is "DefaultPartition". Come up with a different name for each cluster in your
environment, e.g. "QAPartition" or "BobsDevPartition". The use of "Partition" is not required; it's just
a semi-convention. As a small aid to performance try to keep the name short, as it gets included
130
Launching a JBoss Enterprise Application Platform Cluster
in every message sent around the cluster. We'll cover how to use the name you pick in the next
section.
• Pick a unique multicast address for your cluster. By default JBoss Enterprise Application
Platform uses UDP multicast for most intra-cluster communication. Pick a different multicast address
for each cluster you run. Generally a good multicast address is of the form 239.255.x.y. See
1
http://www.29west.com/docs/THPM/multicast-address-assignment.html for a good discussion on
multicast address assignment. We'll cover how to use the address you pick in the next section.
See Section 22.6.2, “Isolating JGroups Channels” for more on isolating clusters.
Let's look at a few different scenarios for doing this. In each scenario we'll be creating a two node
cluster, where the ServerPeerID for the first node is 1 and for the second node is 2 . We've decided
to call our cluster "DocsPartition" and to use 239.255.100.100 as our multicast address. These
scenarios are meant to be illustrative; the use of a two node cluster shouldn't be taken to mean that is
the best size for a cluster; it's just that's the simplest way to do the examples.
This is the most common production scenario. Assume the machines are named "node1" and
"node2", while node1 has an IP address of 192.168.0.101 and node2 has an address of
192.168.0.102. Assume the "ServerPeerID" for node1 is 1 and for node2 it's 2. Assume on each
machine JBoss is installed in /var/jboss.
$ cd /var/jboss/bin
$ ./run.sh -c all -g DocsPartition -u 239.255.100.100 \
-b 192.168.0.101 -Djboss.messaging.ServerPeerID=1
On node2, it's the same except for a different -b value and ServerPeerID:
$ cd /var/jboss/bin
$ ./run.sh -c all -g DocsPartition -u 239.255.100.100 \
-b 192.168.0.102 -Djboss.messaging.ServerPeerID=2
The -c switch says to use the all config, which includes clustering support. The -g switch sets
the cluster name. The -u switch sets the multicast address that will be used for intra-cluster
communication. The -b switch sets the address on which sockets will be bound. The -D switch
sets system property jboss.messaging.ServerPeerID, from which JBoss Messaging gets its
unique id.
131
Chapter 13. Introduction and Quick Start
Running multiple nodes on the same machine is a common scenario in a development environment,
and is also used in production in combination with Scenario 1. (Running all the nodes in a
production cluster on a single machine is generally not recommended, since the machine itself
becomes a single point of failure.) In this version of the scenario, the machine is multihomed,
i.e. has more than one IP address. This allows the binding of each JBoss instance to a different
address, preventing port conflicts when the nodes open sockets.
Assume the single machine has the 192.168.0.101 and 192.168.0.102 addresses assigned,
and that the two JBoss instances use the same addresses and ServerPeerIDs as in Scenario 1. The
difference from Scenario 1 is we need to be sure each Enterprise Application Platform instance has
its own work area. So, instead of using the all config, we are going to use the node1 and node2
configs we copied from all earlier in the previous section.
$ cd /var/jboss/bin
$ ./run.sh -c node1 -g DocsPartition -u 239.255.100.100 \
-b 192.168.0.101 -Djboss.messaging.ServerPeerID=1
For the second instance, it's the same except for different -b and -c values and a different
ServerPeerID:
$ cd /var/jboss/bin
$ ./run.sh -c node2 -g DocsPartition -u 239.255.100.100 \
-b 192.168.0.102 -Djboss.messaging.ServerPeerID=2
This is similar to Scenario 2, but here the machine only has one IP address available. Two
processes can't bind sockets to the same address and port, so we'll have to tell JBoss to use
different ports for the two instances. This can be done by configuring the ServiceBindingManager
service by setting the jboss.service.binding.set system property.
$ cd /var/jboss/bin
$ ./run.sh -c node1 -g DocsPartition -u 239.255.100.100 \
-b 192.168.0.101 -Djboss.messaging.ServerPeerID=1 \
-Djboss.service.binding.set=ports-default
$ cd /var/jboss/bin
$ ./run.sh -c node2 -g DocsPartition -u 239.255.100.100 \
-b 192.168.0.101 -Djboss.messaging.ServerPeerID=2 \
132
Web Application Clustering Quick Start
-Djboss.service.binding.set=ports-01
This tells the ServiceBindingManager on the first node to use the standard set of ports (e.g.
JNDI on 1099). The second node uses the "ports-01" binding set, which by default for each
port has an offset of 100 from the standard port number (e.g. JNDI on 1199). See the conf/
bindingservice.beans/META-INF/bindings-jboss-beans.xml file for the full
ServiceBindingManager configuration.
Note that this setup is not advised for production use, due to the increased management
complexity that comes with using different ports. But it is a fairly common scenario in development
environments where developers want to use clustering but cannot multihome their workstations.
Note
Including -Djboss.service.binding.set=ports-default on the command line
for node1 isn't technically necessary, since ports-default is the ... default. But using
a consistent set of command line arguments across all servers is helpful to people less
familiar with all the details.
That's it; that's all it takes to get a cluster of JBoss Enterprise Application Platform servers up and
running.
• Configuring Your Web Application for Clustering. This aspect involves telling JBoss you
want clustering behavior for a particular web app, and it couldn't be simpler. Just add an empty
distributable element to your application's web.xml file:
<?xml version="1.0"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-
app_2_5.xsd"
version="2.5">
133
Chapter 13. Introduction and Quick Start
<distributable/>
</web-app>
Simply doing that is enough to get the default JBoss Enterprise Application Platform web session
clustering behavior, which is appropriate for most applications. See Section 19.2, “Configuring HTTP
session state replication” for more advanced configuration options.
@javax.ejb.Stateless
@org.jboss.ejb3.annotation.Clustered
public class MyBean implements MySessionInt {
For EJB2 session beans, or for EJB3 beans where you prefer XML configuration over annotations,
simply add a clustered element to the bean's section in the JBoss-specific deployment descriptor,
jboss.xml:
<jboss>
<enterprise-beans>
<session>
<ejb-name>example.StatelessSession</ejb-name>
<jndi-name>example.StatelessSession</jndi-name>
<clustered>true</clustered>
</session>
</enterprise-beans>
</jboss>
See Chapter 17, Clustered Session EJBs for more advanced configuration options.
134
Entity Clustering Quick Start
in Hibernate 3.3. In the JPA/Hibernate context, a second level cache refers to a cache whose contents
are retained beyond the scope of a transaction. A second level cache may improve performance by
reducing the number of database reads. You should always load test your application with second
level caching enabled and disabled to see whether it has a beneficial impact on your particular
application.
If you use more than one JBoss Enterprise Application Platform instance to run your JPA/Hibernate
application and you use second level caching, you must use a cluster-aware cache. Otherwise a
cache on server A will still hold out-of-date data after activity on server B updates some entities.
JBoss Enterprise Application Platform provides a cluster-aware second level cache based on JBoss
Cache. To tell JBoss Enterprise Application Platform's standard Hibernate-based JPA provider to
enable second level caching with JBoss Cache, configure your persistence.xml as follows:
value="org.hibernate.cache.jbc2.JndiMultiplexedJBossCacheRegionFactory"/>
<property name="hibernate.cache.region.jbc2.cachefactory"
value="java:CacheManager"/>
<!-- Other configuration options ... -->
</properties>
</persistence-unit>
</persistence>
That tells Hibernate to use the JBoss Cache-based second level cache, but it doesn't tell it what
entities to cache. That can be done by adding the org.hibernate.annotations.Cache
annotation to your entity class:
package org.example.entities;
import java.io.Serializable;
import javax.persistence.Entity;
import org.hibernate.annotations.Cache;
import org.hibernate.annotations.CacheConcurrencyStrategy;
@Entity
@Cache (usage=CacheConcurrencyStrategy.TRANSACTIONAL)
public class Account implements Serializable
135
Chapter 13. Introduction and Quick Start
See Chapter 18, Clustered Entity EJBs for more advanced configuration options and details on how to
configure the same thing for a non-JPA Hibernate application.
Note
Clustering can add significant overhead to a JPA/Hibernate second level cache, so
don't assume that just because second level caching adds a benefit to a non-clustered
application that it will be beneficial to a clustered application. Even if clustered second
level caching is beneficial overall, caching of more frequently modified entity types may
be beneficial in a non-clustered scenario but not in a clustered one. Always load test your
application.
136
Chapter 14.
Clustering Concepts
In the next section, we discuss basic concepts behind JBoss' clustering services. It is helpful that you
understand these concepts before reading the rest of the Clustering Guide.
On the same Enterprise Application Platform instance, different services can create their own
Channel. In a standard startup of the Enterprise Application Platform 5 all configuration, two different
services create a total of four different channels – JBoss Messaging creates two and a core general
purpose clustering service known as HAPartition creates two more. If you deploy clustered web
applications, clustered EJB3 SFSBs or a clustered JPA/Hibernate entity cache, additional channels will
be created. The channels the Enterprise Application Platform connects can be divided into three broad
categories: a general purpose channel used by the HAPartition service, channels created by JBoss
Cache for special purpose caching and cluster wide state replication, and two channels used by JBoss
Messaging.
So, if you go to two Enterprise Application Platform 5.0.x instances and execute run -c all, the
channels will discover each other and you'll have a conceptual cluster. It's easy to think of this as
a two node cluster, but it's important to understand that you really have multiple channels, and hence
multiple two node clusters.
On the same network, you may have different sets of servers whose services wish to cluster.
Figure 14.1, “Clusters and server nodes” shows an example network of JBoss server instances
divided into three sets, with the third set only having one node. This sort of topology can be set up
simply by configuring the Enterprise Application Platform instances such that within a set of nodes
meant to form a cluster the Channel configurations and names match while they differ from any other
channel configurations and names match while they differ from any other channels on the same
network. The Enterprise Application Platform tries to make this is easy as possible, such that servers
that are meant to cluster only need to have the same values passed on the command line to the -g
(partition name) and -u (multicast address) startup switches. For each set of servers, different values
should be chosen. The sections on “JGroups Configuration” and “Isolating JGroups Channels” cover in
detail how to configure the Enterprise Application Platform such that desired peers find each other and
unwanted peers do not.
137
Chapter 14. Clustering Concepts
The proxy's clustering logic maintains up-to-date knowledge about the cluster. For instance, it knows
the IP addresses of all available server nodes, the algorithm to distribute load across nodes (see next
section), and how to failover the request if the target node not available. As part of handling each
service request, if the cluster topology has changed the server node updates the proxy with the latest
changes in the cluster. For instance, if a node drops out of the cluster, each proxy is updated with the
new topology the next time it connects to any active node in the cluster. All the manipulations done
by the proxy's clustering logic are transparent to the client application. The client-side interceptor
clustering architecture is illustrated in Figure 14.2, “The client-side interceptor (proxy) architecture for
clustering”.
138
External Load Balancer Architecture
139
Chapter 14. Clustering Concepts
A potential problem with an external load balancer architecture is that the load balancer itself may be a
single point of failure. It needs to be monitored closely to ensure high availability of the entire cluster's
services.
140
External load balancer architecture
• Random-Robin: for each call the target node is randomly selected from the list.
Implemented by org.jboss.ha.framework.interfaces.RoundRobin (legacy) and
org.jboss.ha.client.loadbalance.RoundRobin (EJB3).
• First Available: one of the available target nodes is elected as the main target and is thereafter
used for every call; this elected member is randomly chosen from the list of members in
the cluster. When the list of target nodes changes (because a node starts or dies), the
policy will choose a new target node unless the currently elected node is still available.
Each client-side proxy elects its own target node independently of the other proxies, so if a
particular client downloads two proxies for the same target service (for example, an EJB),
each proxy will independently pick its target. This is an example of a policy that provides
“session affinity” or “sticky sessions”, since the target node does not change once established.
Implemented by org.jboss.ha.framework.interfaces.FirstAvailable (legacy) and
org.jboss.ha.client.loadbalance.aop.FirstAvailable (EJB3).
• First Available Identical All Proxies: has the same behaviour as the "First Available" policy
but the elected target node is shared by all proxies in the same client-side VM that are
associated with the same target service. So if a particular client downloads two proxies for
the same target service (e.g. an EJB), each proxy will use the same target. Implemented by
org.jboss.ha.framework.interfaces.FirstAvailableIdenticalAllProxies (legacy)
and org.jboss.ha.client.loadbalance.aop.FirstAvailableIdenticalAllProxies
(EJB3).
Each of the above is an implementation of a simple interface; users are free to write their own
implementations if they need some special behavior. In later sections we'll see how to configure the
load balance policies used by different services.
141
142
Chapter 15.
JGroups is a toolkit for reliable point-to-point and point-to-multipoint communication. JGroups is used
for all clustering-related communications between nodes in a JBoss Enterprise Application Platform
cluster.
JBoss Cache is a highly flexible clustered transactional caching library. Many Enterprise Application
Platform clustering services need to cache some state in memory while 1) ensuring for high availability
purposes that a backup copy of that state is available on another node if it can't otherwise be
recreated (e.g. the contents of a web session) and 2) ensuring that the data cached on each node in
the cluster is consistent. JBoss Cache handles these concerns for most JBoss Enterprise Application
Platform clustered services. JBoss Cache uses JGroups to handle its group communication
requirements. POJO Cache is an extension of the core JBoss Cache that JBoss Enterprise
Application Platform uses to support fine-grained replication of clustered web session state. See
Section 15.2, “Distributed Caching with JBoss Cache” for more on how JBoss Enterprise Application
Platform uses JBoss Cache and POJO Cache.
HAPartition is an adapter on top of a JGroups channel that allows multiple services to use the
channel. HAPartition also supports a distributed registry of which HAPartition-based services are
running on which cluster members. It provides notifications to interested listeners when the cluster
membership changes or the clustered service registry changes. See Section 15.3, “The HAPartition
Service” for more details on HAPartition.
The other higher level clustering services make use of JBoss Cache or HAPartition, or, in the case
of HA-JNDI, both. The exception to this is JBoss Messaging's clustering features, which interact with
JGroups directly.
The characteristics of a JGroups Channel are determined by the set of protocols that compose it.
Each protocol handles a single aspect of the overall group communication task; for example the UDP
protocol handles the details of sending and receiving UDP datagrams. A Channel that uses the UDP
protocol is capable of communicating with UDP unicast and multicast; alternatively one that uses
the TCP protocol uses TCP unicast for all messages. JGroups supports a wide variety of different
protocols (see Section 22.1, “Configuring a JGroups Channel's Protocol Stack” for details), but the
Enterprise Application Platform ships with a default set of channel configurations that should meet
most needs.
143
Chapter 15. Clustering Building Blocks
By default, UDP multicast is used by all JGroups channels used by the Enterprise Application Platform
(except for one TCP-based channel used by JBoss Messaging).
Note
If several services request a channel with the same configuration name from the
ChannelFactory, they are not handed a reference to the same underlying Channel.
Each receives its own Channel, but the channels will have an identical configuration.
A logical question is how those channels avoid forming a group with each other if
each, for example, is using the same multicast address and port. The answer is that
when a consuming service connects its Channel, it passes a unique-to-that-service
cluster_name argument to the Channel.connect(String cluster_name)
method. The Channel uses that cluster_name as one of the factors that determine
whether a particular message received over the network is intended for it.
You can add a new stack configuration by adding a new stack element to the server/all/
deploy/cluster/jgroups-channelfactory.sar/META-INF/jgroups-channelfactory-
stacks.xml file. You can alter the behavior of an existing configuration by editing this file. Before
doing this though, have a look at the other standard configurations the Enterprise Application Platform
ships; perhaps one of those meets your needs. Also, please note that before editing a configuration
you should understand what services are using that configuration; make sure the change you
are making is appropriate for all affected services. If the change isn't appropriate for a particular
service, perhaps its better to create a new configuration and change some services to use that new
configuration.
• udp
144
The JGroups Shared Transport
UDP multicast based stack meant to be shared between different channels.Message bundling is
disabled, as it can add latency to synchronous group RPCs. Services that only make asynchronous
RPCs (for example, JBoss Cache configured for REPL_Enterprise Application PlatformYNC) and
do so in high volume may be able to improve performance by configuring their cache to use the
udp-async stack below. Services that only make synchronous RPCs (for example JBoss Cache
configured for REPL_SYNC or INVALIDATION_SYNC) may be able to improve performance by
using the udp-sync stack below, which does not include flow control.
• udp-async
Same as the default udp stack above, except message bundling is enabled in the transport protocol
(enable_bundling=true). Useful for services that make high-volume asynchronous RPCs (e.g.
high volume JBoss Cache instances configured for REPL_Enterprise Application PlatformYNC)
where message bundling may improve performance.
• udp-sync
UDP multicast based stack, without flow control and without message bundling. This can be used
instead of udp if (1) synchronous calls are used and (2) the message volume (rate and size) is not
that large. Don't use this configuration if you send messages at a high sustained rate, or you might
run out of memory.
• tcp
TCP based stack, with flow control and message bundling. TCP stacks are usually used when IP
multicasting cannot be used in a network (e.g. routers discard multicast).
• tcp-sync
TCP based stack, without flow control and without message bundling. TCP stacks are usually used
when IP multicasting cannot be used in a network (e.g.routers discard multicast). This configuration
should be used instead of tcp above when (1) synchronous calls are used and (2) the message
volume (rate and size) is not that large. Don't use this configuration if you send messages at a high
sustained rate, or you might run out of memory.
• jbm-control
Stack optimized for the JBoss Messaging Control Channel. By default uses the same UDP
transport protocol config as is used for the default 'udp' stack defined above. This allows the JBoss
Messaging Control Channel to use the same sockets, network buffers and thread pools as are used
by the other standard JBoss Enterprise Application Platform clustered services (see Section 15.1.2,
“The JGroups Shared Transport”
• jbm-data
145
Chapter 15. Clustering Building Blocks
apps, clustered EJB3 SFSBs and a clustered JPA/Hibernate second level cache are all used. So many
channels can consume a lot of resources, and can be a real configuration nightmare if the network
environment requires configuration to ensure cluster isolation.
Beginning with Enterprise Application Platform 5, JGroups supports sharing of transport protocol
instances between channels. A JGroups channel is composed of a stack of individual protocols, each
of which is responsible for one aspect of the channel's behavior. A transport protocol is a protocol that
is responsible for actually sending messages on the network and receiving them from the network.
The resources that are most desirable for sharing (sockets and thread pools) are managed by the
transport protocol, so sharing a transport protocol between channels efficiently accomplishes JGroups
resource sharing.
The protocol stack configurations used by the Enterprise Application Platform 5 ChannelFactory all
have a singleton_name configured. In fact, if you add a stack to the ChannelFactory that doesn't
include a singleton_name, before creating any channels for that stack, the ChannelFactory will
synthetically create a singleton_name by concatenating the stack name to the string "unnamed_",
e.g. unnamed_customStack.
146
Distributed Caching with JBoss Cache
• DistributedStateService
Users can also create their own JBoss Cache and POJO Cache instances for custom use by their
applications, see Chapter 23, JBoss Cache Configuration and Deployment for more on this.
• Caches are internal details of the services that use them. They shouldn't be first-class deployments.
• Services would find their cache via JMX lookups. Using JMX for purposes other exposing
management interfaces is just not the JBoss 5 way.
In JBoss 5, the scattered cache deployments have been replaced with a new CacheManager service,
deployed via the JBOSS_HOME/server/all/deploy/cluster/jboss-cache-manager.sar.
The CacheManager is a factory and registry for JBoss Cache instances. It is configured with a set of
named JBoss Cache configurations. Services that need a cache ask the cache manager for the cache
by name; the cache manager creates the cache (if not already created) and returns it. The cache
manager keeps a reference to each cache it has created, so all services that request the same cache
configuration name will share the same cache. When a service is done with the cache, it releases it to
the cache manager. The cache manager keeps track of how many services are using each cache, and
will stop and destroy the cache when all services have released it.
147
Chapter 15. Clustering Building Blocks
• standard-session-cache
• field-granularity-session-cache
• sfsb-cache
• ha-partition
• mvcc-entity
A config appropriate for JPA/Hibernate entity/collection caching that uses JBC's MVCC locking (see
notes below).
• optimistic-entity
A config appropriate for JPA/Hibernate entity/collection caching that uses JBC's optimistic locking
(see notes below).
• pessimistic-entity
A config appropriate for JPA/Hibernate entity/collection caching that uses JBC's pessimistic locking
(see notes below).
• mvcc-entity-repeatable
• pessimistic-entity-repeatable
• local-query
A config appropriate for JPA/Hibernate query result caching. Does not replicate query results. DO
NOT store the timestamp data Hibernate uses to verify validity of query results in this cache.
• replicated-query
A config appropriate for JPA/Hibernate query result caching. Replicates query results. DO NOT
store the timestamp data Hibernate uses to verify validity of query result in this cache.
• timestamps-cache
148
The JBoss Enterprise Application Platform CacheManager Service
A config appropriate for the timestamp data cached as part of JPA/Hibernate query result caching.
A replicated timestamp cache is required if query result caching is used, even if the query results
themselves use a non-replicating cache like local-query.
• mvcc-shared
A config appropriate for a cache that's shared for JPA/Hibernate entity, collection, query result and
timestamp caching. Not an advised configuration, since it requires cache mode REPL_SYNC, which
is the least efficient mode. Also requires a full state transfer at startup, which can be expensive.
Maintained for backwards compatibility reasons, as a shared cache was the only option in JBoss 4.
Uses JBC's MVCC locking.
• optimistic-shared
A config appropriate for a cache that's shared for JPA/Hibernate entity, collection, query result and
timestamp caching. Not an advised configuration, since it requires cache mode REPL_SYNC, which
is the least efficient mode. Also requires a full state transfer at startup, which can be expensive.
Maintained for backwards compatibility reasons, as a shared cache was the only option in JBoss 4.
Uses JBC's optimistic locking.
• pessimistic-shared
A config appropriate for a cache that's shared for JPA/Hibernate entity, collection, query result and
timestamp caching. Not an advised configuration, since it requires cache mode REPL_SYNC, which
is the least efficient mode. Also requires a full state transfer at startup, which can be expensive.
Maintained for backwards compatibility reasons, as a shared cache was the only option in JBoss 4.
Uses JBC's pessimistic locking.
• mvcc-shared-repeatable
• pessimistic-shared-repeatable
Note
For more on JBoss Cache's locking schemes, see Section 23.1.4, “Concurrent Access”)
Note
For JPA/Hibernate second level caching, REPEATABLE_READ is only useful if the
application evicts/clears entities from the EntityManager/Hibernate Session and then
expects to repeatably re-read them in the same transaction. Otherwise, the Session's
internal cache provides a repeatable-read semantic.
149
Chapter 15. Clustering Building Blocks
. . .
. . .
</bean>
150
The HAPartition Service
we'll be discussing in the rest of this guide, including smart client-side clustered proxies, EJB 2 SFSB
replication and entity cache management, farming, HA-JNDI and HA singletons. Custom services can
also make use of HAPartition.
The following snippet shows the HAPartition service definition packaged with the standard JBoss
Enterprise Application Platform distribution. This configuration can be found in the server/all/
deploy/cluster/hapartition-jboss-beans.xml file.
<bean name="HAPartitionCacheHandler"
class="org.jboss.ha.framework.server.HAPartitionCacheHandlerImpl">
<property name="cacheManager"><inject bean="CacheManager"/></property>
<property name="cacheConfigName">ha-partition</property>
</bean>
<bean name="HAPartition"
class="org.jboss.ha.framework.server.ClusterPartition">
<depends>jboss:service=Naming</depends>
<annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX
(name="jboss:service=HAPartition,partition=
${jboss.partition.name:DefaultPartition}",
exposedInterface=org.jboss.ha.framework.server.ClusterPartitionMBean.class,
registerDirectly=true)</annotation>
<property name="partitionName">${jboss.partition.name:DefaultPartition}</
property>
<property name="nodeAddress">${jboss.bind.address}</property>
<!-- Max time (in ms) to wait for state transfer to complete. Increase
for large states -->
<property name="stateTransferTimeout">30000</property>
<!-- Max time (in ms) to wait for RPC calls to complete. -->
<property name="methodCallTimeout">60000</property>
<property name="threadPool"><inject
bean="jboss.system:service=ThreadPool"/></property>
151
Chapter 15. Clustering Building Blocks
<property name="distributedStateImpl">
<bean name="DistributedState"
class="org.jboss.ha.framework.server.DistributedStateImpl">
<annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX
(name="jboss:service=DistributedState,partitionName=
${jboss.partition.name:DefaultPartition}",
exposedInterface=org.jboss.ha.framework.server.DistributedStateImplMBean.class,
registerDirectly=true)</annotation>
Much of the above is boilerplate; below we'll touch on the key points relevant to end users. There are
two beans defined above, the HAPartitionCacheHandler and the HAPartition itself.
• partitionName is an optional attribute to specify the name of the cluster. Its default value is
DefaultPartition. Use the -g (a.k.a. --partition) command line switch to set this value at
JBoss startup.
• stateTransferTimeout specifies the timeout (in milliseconds) for initial application state transfer.
State transfer refers to the process of obtaining a serialized copy of initial application state from
other already-running cluster members at service startup. Its default value is 30000.
• methodCallTimeout specifies the timeout (in milliseconds) for obtaining responses to group RPCs
from the other cluster members. Its default value is 60000.
The HAPartitionCacheHandler is a small utility service that helps the HAPartition integrate
with JBoss Cache (see Section 15.2.1, “The JBoss Enterprise Application Platform CacheManager
Service”). HAPartition exposes a child service called DistributedState (see Section 15.3.2,
“DistributedState Service”) that uses JBoss Cache; the HAPartitionCacheHandler helps ensure
consistent configuration between the JGroups Channel used by Distributed State's cache and the
one used directly by HAPartition.
• cacheConfigName the name of the JBoss Cache configuration to use for the HAPartition-
related cache. Indirectly, this also specifies the name of the JGroups protocol stack configuration
HAPartition should use. See Section 23.1.5, “JGroups Integration” for more on how the JGroups
protocol stack is configured.
In order for nodes to form a cluster, they must have the exact same partitionName and the
HAPartitionCacheHandler's cacheConfigName must specify an identical JBoss Cache
configuration. Changes in either element on some but not all nodes would prevent proper clustering
behavior.
You can view the current cluster information by pointing your browser to the JMX console of any JBoss
instance in the cluster (i.e., http://hostname:8080/jmx-console/) and then clicking on the
jboss:service=HAPartition,partition=DefaultPartition MBean (change the MBean
152
DistributedReplicantManager Service
name to reflect your partitionr name if you use the -g startup switch). A list of IP addresses for the
current cluster members is shown in the CurrentView field.
Note
While it is technically possible to put a JBoss server instance into multiple HAPartitions at
the same time, this practice is generally not recommended, as it increases management
complexity.
The DRM is a distributed registry that allows HAPartition users to register objects under a given key,
making available to callersthe set of objects registered under that key by the various members of t he
cluster. The DRM also provides a notification mechanism so interested listeners can be notified when
the contents of the registry changes.
There are two main usages for the DRM in JBoss Enterprise Application Platform:
Here the keys are the names of the various services that need a clustered smart proxy (see
Section 14.2.1, “Client-side interceptor architecture”, e.g. the name of a clustered EJB. The
value object each node stores in the DRM is known as a "target". It's something a smart proxy's
transport layer can use to contact the node (e.g. an RMI stub, an HTTP URL or a JBoss Remoting
InvokerLocator). The factory that builds clustered smart proxies accesses the DRM to get the
set of "targets" that should be injected into the proxy to allow it to communicate with all the nodes in
a cluster.
Here the keys are the names of the various services that need to function as High Availablity
Singletons (see the HEnterprise Application Platformingleton chapter). The value object each node
stores in the DRM is simply a String that acts as a token to indicate that the node has the service
deployed, and thus is a candidate to become the "master" node for the HA singleton service.
In both cases, the key under which objects are registered identifies a particular clustered service. It is
useful to understand that every node in a cluster doesn't have to register an object under every key.
Only services that are deployed on a particular node will register something under that service's key,
and services don't have to be deployed homogeneously across the cluster. The DRM is thus useful
as a mechanism for understanding a service's "topology" around the cluster -- which nodes have the
service deployed.
153
Chapter 15. Clustering Building Blocks
provides coordinated management of arbitary application state around the cluster. It is supported for
backwards compatibility reasons, but new applications should not use it; they should use the much
more sophisticated JBoss Cache instead.
154
Chapter 16.
• Transparent failover of naming operations. If an HA-JNDI naming Context is connected to the HA-
JNDI service on a particular JBoss Enterprise Application Platform instance, and that service fails
or is shut down, the HA-JNDI client can transparently fail over to another Enterprise Application
Platform instance.
• Load balancing of naming operations. A HA-JNDI naming Context will automatically load balance its
requests across all the HA-JNDI servers in the cluster.
• Unified view of JNDI trees cluster-wide. A client can connect to the HA-JNDI service running on any
node in the cluster and find objects bound in JNDI on any other node. This is accomplished via two
mechanisms:
• Cross-cluster lookups. A client can perform a lookup and the server side HA-JNDI service has the
ability to find things bound in regular JNDI on any node in the cluster.
• A replicated cluster-wide context tree. An object bound into the HA-JNDI service will be replicated
around the cluster, and a copy of that object will be available in-VM on each node in the cluster.
JNDI is a key component for many other interceptor-based clustering services: those services register
themselves with JNDI so the client can look up their proxies and make use of their services. HA-JNDI
completes the picture by ensuring that clients have a highly-available means to look up those proxies.
However, it is important to understand that using HA-JNDI (or not) has no effect whatsoever on the
clustering behavior of the objects that are looked up. To illustrate:
• If an EJB is not configured as clustered, looking up the EJB via HA-JNDI does not somehow result
in the addition of clustering capabilities (load balancing of EJB calls, transparent failover, state
replication) to the EJB.
• If an EJB is configured as clustered, looking up the EJB via regular JNDI instead of HA-JNDI does
not somehow result in the removal of the bean proxy's clustering capabilities.
On the server side, the HA-JNDI service maintains a cluster-wide context tree. The cluster wide
tree is always available as long as there is one node left in the cluster. Each node in the cluster also
maintains its own local JNDI context tree. The HA-JNDI service on each node is able to find objects
bound into the local JNDI context tree, and is also able to make a cluster-wide RPC to find objects
bound in the local tree on any other node. An application can bind its objects to either tree, although
155
Chapter 16. Clustered JNDI Services
in practice most objects are bound into the local JNDI context tree. The design rationale for this
architecture is as follows:
• It avoids migration issues with applications that assume that their JNDI implementation is local. This
allows clustering to work out-of-the-box with just a few tweaks of configuration files.
• In a homogeneous cluster, this configuration actually cuts down on the amount of network traffic. A
homogenous cluster is one where the same types of objects are bound under the same names on
each node.
• Designing it in this way makes the HA-JNDI service an optional service since all underlying cluster
code uses a straight new InitialContext to lookup or create bindings.
On the server side, a naming Context obtained via a call to new InitialContext() will be bound
to the local-only, non-cluster-wide JNDI Context. So, all EJB homes and such will not be bound to the
cluster-wide JNDI Context, but rather, each home will be bound into the local JNDI.
When a remote client does a lookup through HA-JNDI, HA-JNDI will delegate to the local JNDI service
when it cannot find the object within the global cluster-wide Context. The detailed lookup rule is as
follows.
• If the binding is not in the cluster-wide tree, delegate the lookup query to the local JNDI service and
return the received answer if available.
• If not available, the HA-JNDI service asks all other nodes in the cluster if their local JNDI service
owns such a binding and returns the answer from the set it receives.
In practice, objects are rarely bound in the cluster-wide JNDI tree; rather they are bound in the local
JNDI tree. For example, when EJBs are deployed, their proxies are always bound in local JNDI, not
HA-JNDI. So, an EJB home lookup done through HA-JNDI will always be delegated to the local JNDI
instance.
Note
If different beans (even of the same type, but participating in different clusters) use the
same JNDI name, this means that each JNDI server will have a logically different "target"
bound under the same name. (JNDI on node 1 will have a binding for bean A and JNDI
on node 2 will have a binding, under the same name, for bean B). Consequently, if a client
performs a HA-JNDI query for this name, the query will be invoked on any JNDI server of
the cluster and will return the locally bound stub. Nevertheless, it may not be the correct
stub that the client is expecting to receive! So, it is always best practice to ensure that
across the cluster different names are used for logically different bindings.
Note
If a binding is only made available on a few nodes in the cluster (for example because a
bean is only deployed on a small subset of nodes in the cluster), the probability is higher
that a lookup will hit a HA-JNDI server that does not own this binding and thus the lookup
will need to be forwarded to all nodes in the cluster. Consequently, the query time will be
156
Client configuration
longer than if the binding would have been available locally. Moral of the story: as much as
possible, cache the result of your JNDI queries in your client.
Note
You cannot currently use a non-JNP JNDI implementation (i.e. LDAP) for your local
JNDI implementation if you want to use HA-JNDI. However, you can use JNDI federation
using the ExternalContext MBean to bind non-JBoss JNDI trees into the JBoss JNDI
namespace. Furthermore, nothing prevents you using one centralized JNDI server for
your whole cluster and scrapping HA-JNDI and JNP.
The Context.PROVIDER_URL property points to the HA-JNDI service configured in the deploy/
cluster/hajndi-jboss-beans.xml file (see Section 16.3, “JBoss configuration”). By default this
service listens on the interface named via the jboss.bind.address system property, which itself
is set to whatever value you assign to the -b command line option when you start JBoss Enterprise
Application Platform (or localhost if not specified). The above code shows an example of accessing
this property.
However, this does not work in all cases, especially when running several JBoss Enterprise
Application Platform instances on the same machine and bound to the same IP address, but
configured to use different ports. A safer method is to not specify the Context.PROVIDER_URL
but instead allow the InitialContext to statically find the in-VM HA-JNDI by specifying the
jnp.partitionName property:
157
Chapter 16. Clustered JNDI Services
p.put(Context.INITIAL_CONTEXT_FACTORY,
"org.jnp.interfaces.NamingContextFactory");
p.put(Context.URL_PKG_PREFIXES, "jboss.naming:org.jnp.interfaces");
// HA-JNDI is registered under the partition name passed to JBoss via -g
String partitionName = System.getProperty("jboss.partition.name",
"DefaultPartition");
p.put("jnp.partitionName", partitionName);
return new InitialContext(p);
This example uses the jboss.partition.name system property to identify the partition with which
the HA-JNDI service works. This system property is set to whatever value you assign to the -g
command line option when you start JBoss Enterprise Application Platform (or DefaultPartition if
not specified).
Do not attempt to simplify things by placing a jndi.properties file in your deployment or by editing
the Enterprise Application Platform's conf/jndi.properties file. Doing either will almost certainly
break things for your application and quite possibly across the server. If you want to externalize your
client configuration, one approach is to deploy a properties file not named jndi.properties, and
then programatically create a Properties object that loads that file's contents.
Within the bean definition in the ejb-jar.xml or in the war's web.xml you will need to define two
resource-ref mappings, one for the connection factory and one for the destination.
<resource-ref>
<res-ref-name>jms/ConnectionFactory</res-ref-name>
<res-type>javax.jms.QueueConnectionFactory</res-type>
<res-auth>Container</res-auth>
</resource-ref>
<resource-ref>
<res-ref-name>jms/Queue</res-ref-name>
<res-type>javax.jms.Queue</res-type>
<res-auth>Container</res-auth>
</resource-ref>
Using these examples the bean performing the lookup can obtain the connection factory by looking up
'java:comp/env/jms/ConnectionFactory' and can obtain the queue by looking up 'java:comp/env/jms/
Queue'.
Within the JBoss-specific deployment descriptor (jboss.xml for EJBs, jboss-web.xml for a WAR) these
references need to be mapped to a URL that makes use of HA-JNDI.
<resource-ref>
158
For clients running outside the Enterprise Application Platform
<res-ref-name>jms/ConnectionFactory</res-ref-name>
<jndi-name>jnp://${jboss.bind.address}:1100/ConnectionFactory</jndi-name>
</resource-ref>
<resource-ref>
<res-ref-name>jms/Queue</res-ref-name>
<jndi-name>jnp://${jboss.bind.address}:1100/queue/A</jndi-name>
</resource-ref>
The URL should be the URL to the HA-JNDI server running on the same node as the bean; if the
bean is available the local HA-JNDI server should also be available. The lookup will then automatically
query all of the nodes in the cluster to identify which node has the JMS resources available.
The ${jboss.bind.address} syntax used above tells JBoss to use the value of the
jboss.bind.address system property when determining the URL. That system property is itself
set to whatever value you assign to the -b command line option when you start JBoss Enterprise
Application Platform.
No other jndi.properties file should be deployed inside the Enterprise Application Platform because
of the possibility of its being found on the classpath when it shouldn't and thus disrupting the internal
operation of the server. For example, if an EJB deployment included a jndi.properties configured for
HA-JNDI, when the server binds the EJB proxies into JNDI it will likely bind them into the replicated
HA-JNDI tree and not into the local JNDI tree where they belong.
16.2.1.3. How can I tell if things are being bound into HA-JNDI that
shouldn't be?
Go into the the jmx-console and execute the list operation on the jboss:service=JNDIView
mbean. Towards the bottom of the results, the contents of the "HA-JNDI Namespace" are listed.
Typically this will be empty; if any of your own deployments are shown there and you didn't explicitly
bind them there, there's probably an improper jndi.properties file on the classpath. Please visit the
1
following link for an example: Problem with removing a Node from Cluster
java.naming.provider.url=server1:1100,server2:1100,server3:1100,server4:1100
1
http://www.jboss.com/index.html?module=bb&op=viewtopic&t=104715
159
Chapter 16. Clustered JNDI Services
When initialising, the JNP client code will try to get in touch with each server node from the list, one
after the other, stopping as soon as one server has been reached. It will then download the HA-JNDI
stub from this node.
Note
There is no load balancing behavior in the JNP client lookup process itself. It just goes
through the provider lists and uses the first available server to obtain the stub. The HA-
JNDI provider list only needs to contain a subset of HA-JNDI nodes in the cluster; once
the HA-JNDI stub is downloaded, the stub will include information on all the available
servers. A good practice is to include a set of servers such that you are certain that at
least one of those in the list will be available.
The downloaded smart proxy contains the list of currently running nodes and the logic to load balance
naming requests and to fail-over to another node if necessary. Furthermore, each time a JNDI
invocation is made to the server, the list of targets in the proxy interceptor is updated (only if the list
has changed since the last call).
If the property string java.naming.provider.url is empty or if all servers it mentions are not
reachable, the JNP client will try to discover a HA-JNDI server through a multicast call on the network
(auto-discovery). See Section 16.3, “JBoss configuration” for how to configure auto-discovery on
the JNDI server nodes. Through auto-discovery, the client might be able to get a valid HA-JNDI
server node without any configuration. Of course, for auto-discovery to work, the network segment(s)
between the client and the server cluster must be configured to propagate such multicast datagrams.
Note
By default the auto-discovery feature uses multicast group address 230.0.0.4 and port
1102.
In addition to the java.naming.provider.url property, you can specify a set of other properties.
The following list shows all clustering-related client side properties you can specify when creating a
new InitialContext. (All of the standard, non-clustering-related environment properties used with
regular JNDI are also available.)
• jnp.disableDiscovery: When set to true, this property disables the automatic discovery
feature. Default is false.
• jnp.discoveryTimeout: Determines how many milliseconds the context will wait for a response
to its automatic discovery packet. Default is 5000 ms.
160
JBoss configuration
• jnp.discoveryGroup: Determines which multicast group address is used for the automatic
discovery. Default is 230.0.0.4. Must match the value of the AutoDiscoveryAddress configured on
the server side HA-JNDI service. Note that the server side HA-JNDI service by default listens on the
address specified via the -u startup switch, so if -u is used on the server side (as is recommended),
jnp.discoveryGroup will need to be configured on the client side.
• jnp.discoveryPort: Determines which multicast port is used for the automatic discovery. Default
is 1102. Must match the value of the AutoDiscoveryPort configured on the server side HA-JNDI
service.
<annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX
(name="jboss:service=HAJNDI",
exposedInterface=org.jboss.ha.jndi.HANamingServiceMBean.class)</
annotation>
<!-- The partition used for group RPCs to find locally bound objects
on other nodes -->
<property name="HAPartition"><inject bean="HAPartition"/></property>
<property name="localNamingInstance">
<inject bean="jboss:service=NamingBeanImpl"
property="namingInstance"/>
</property>
<!-- The thread pool used to control the bootstrap and auto discovery
lookups -->
<property name="lookupPool"><inject
bean="jboss.system:service=ThreadPool"/></property>
161
Chapter 16. Clustered JNDI Services
<property name="port">
<!-- Get the port from the ServiceBindingManager -->
<value-factory bean="ServiceBindingManager"
method="getIntBinding">
<parameter>jboss:service=HAJNDI</parameter>
<parameter>Port</parameter>
</value-factory>
</property>
162
JBoss configuration
You can see that this bean has a number of other services injected into different properties:
• HAPartition accepts the core clustering service used manage HA-JNDI's clustered proxies and
to make the group RPCs that find locally bound objects on other nodes. See Section 15.3, “The
HAPartition Service” for more.
• distributedTreeManager accepts a handler for the replicated tree. The standard handler uses
JBoss Cache to manage the replicated tree. The JBoss Cache instance is retrieved using the
injected HAPartitionCacheHandler bean. See Section 15.3, “The HAPartition Service” for more
details.
• lookupPool accepts the thread pool used to provide threads to handle the bootstrap and auto
discovery lookups.
Besides the above dependency injected services, the available configuration attributes for the HAJNDI
bean are as follows:
• bindAddress specifies the address to which the HA-JNDI server will bind to listen for naming proxy
download requests from JNP clients. The default value is the value of the jboss.bind.address
system property, or localhost if that property is not set. The jboss.bind.address system
property is set if the -b command line switch is used when JBoss is started.
• port specifies the port to which the HA-JNDI server will bind to listen for naming proxy download
requests from JNP clients. The value is obtained from the ServiceBindingManager bean configured
in conf/bootstrap/bindings.xml. The default value is 1100.
• Backlog specifies the maximum queue length for incoming connection indications for the TCP
server socket on which the service listens for naming proxy download requests from JNP clients.
The default value is 50.
• rmiBindAddress specifies the address to which the HA-JNDI server will bind to listen for
RMI requests (e.g. for JNDI lookups) from naming proxies. The default value is the value of
the jboss.bind.address system property, or localhost if that property is not set. The
jboss.bind.address system property is set if the -b command line switch is used when JBoss
is started.
• rmiPort specifies the port to which the server will bind to communicate with the downloaded stub.
The value is obtained from the ServiceBindingManager bean configured in conf/bootstrap/
bindings.xml. The default value is 1101. If no value is set, the operating system automatically
assigns a port.
• discoveryDisabled is a boolean flag that disables configuration of the auto discovery multicast
listener. The default is false.
• autoDiscoveryAddress specifies the multicast address to listen to for JNDI automatic discovery.
The default value is the value of the jboss.partition.udpGroup system property, or 230.0.0.4
if that is not set. The jboss.partition.udpGroup system property is set if the -u command line
switch is used when JBoss is started.
163
Chapter 16. Clustered JNDI Services
• autoDiscoveryGroup specifies the port to listen on for multicast JNDI automatic discovery packets.
The default value is 1102.
• autoDiscoveryBindAddress sets the interface on which HA-JNDI should listen for auto-
discovery request packets. If this attribute is not specified and a bindAddress is specified, the
bindAddress will be used.
• autoDiscoveryTTL specifies the TTL (time-to-live) for autodiscovery IP multicast packets. This
value represents the number of network hops a multicast packet can be allowed to propagate before
networking equipment should drop the packet. Despite its name, it does not represent a unit of time.
• clientSocketFactory is an optional attribute that specifies the fully qualified classname of the
java.rmi.server.RMIClientSocketFactory that should be used to create client sockets.
The default is null.
• serverSocketFactory is an optional attribute that specifies the fully qualified classname of the
java.rmi.server.RMIServerSocketFactory that should be used to create server sockets.
The default is null.
<depends>jboss:service=Naming</depends>
<annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX
(name="jboss:service=HAPartition,partition=SecondaryPartition",
exposedInterface=org.jboss.ha.framework.server.ClusterPartitionMBean.class,
registerDirectly=true)</annotation>
164
Adding a Second HA-JNDI Service
<property name="cacheHandler"><inject
bean="SecondaryHAPartitionCacheHandler"/></property>
<property name="partitionName">SecondaryPartition</property>
....
</bean>
<bean name="MySpecialPartitionHAJNDI"
class="org.jboss.ha.jndi.HANamingService">
<annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX
(name="jboss:service=HAJNDI,partitionName=SecondaryPartition",
exposedInterface=org.jboss.ha.jndi.HANamingServiceMBean.class)</
annotation>
<property name="distributedTreeManager">
<bean
class="org.jboss.ha.jndi.impl.jbc.JBossCacheDistributedTreeManager">
<property name="cacheHandler"><inject
bean="SecondaryHAPartitionPartitionCacheHandler"/></property>
</bean>
</property>
<property name="port">56789</property>
<property name="rmiPort">56790</property>
<property name="autoDiscoveryGroup">56791</property>
.....
</bean>
165
166
Chapter 17.
To cluster a stateless session bean in EJB 3.0, simply annotate the bean class with the @Clustered
annotation. This annotation contains optional parameters for overriding both the load balance policy
and partition to use.
• partition specifies the name of the cluster the bean participates in. While the @Clustered
annotation lets you override the default partition, DefaultPartition, for an individual bean, you
can override this for all beans using the jboss.partition.name system property.
org.jboss.ha.client.loadbalance.RoundRobin
Starting with a random target, always favors the next available target in the list, ensuring
maximum load balancing always occurs.
org.jboss.ha.client.loadbalance.RandomRobin
Randomly selects its target without any consideration to previously selected targets.
org.jboss.ha.client.loadbalance.aop.FirstAvailable
Once a target is chosen, always favors that same target; i.e. no further load balancing occurs.
Useful in cases where "sticky session" behavior is desired, e.g. stateful session beans.
org.jboss.ha.client.loadbalance.aop.FirstAvailableIdenticalAllProxies
Similar to FirstAvailable, except that the favored target is shared across all proxies.
167
Chapter 17. Clustered Session EJBs
@Stateless
@Clustered
public class MyBean implements MySessionInt
{
public void test()
{
// Do something cool
}
}
Rather than using the @Clustered annotation, you can also enable clustering for a session bean in
jboss.xml:
<jboss>
<enterprise-beans>
<session>
<ejb-name>NonAnnotationStateful</ejb-name>
<clustered>true</clustered>
<cluster-config>
<partition-name>FooPartition</partition-name>
<load-balance-
policy>org.jboss.ha.framework.interfaces.RandomRobin</load-balance-policy>
</cluster-config>
</session>
</enterprise-beans>
</jboss>
Note
The <clustered>true</clustered> element is really just an alias for the
<container-name>Clustered Stateless SessionBean</container-name>
element in the conf/standardjboss.xml file.
In the bean configuration, only the <clustered> element is necessary to indicate that the bean needs to
support clustering features. The default values for the optional <cluster-config> elements match those
of the corresponding properties from the @Clustered annotation.
168
The EJB application configuration
• name specifies the name of a cache configuration registered with the CacheManager service
discussed in Section 17.2.3, “CacheManager service configuration”. By default, the sfsb-cache
configuration will be used.
• maxSize specifies the maximum number of beans that can cached before the cache should start
passivating beans, using an LRU algorithm.
• idleTimeoutSeconds specifies the max period of time a bean can go unused before the cache
should passivate it (irregardless of whether maxSize beans are cached.)
• removalTimeoutSeconds specifies the max period of time a bean can go unused before the
cache should remove it altogether.
@Stateful
@Clustered
@CacheConfig(maxSize=5000, removalTimeoutSeconds=18000)
public class MyBean implements MySessionInt
{
private int state = 0;
As with stateless beans, the @Clustered annotation can alternatively be omitted and the clustering
configuration instead applied to jboss.xml:
169
Chapter 17. Clustered Session EJBs
<jboss>
<enterprise-beans>
<session>
<ejb-name>NonAnnotationStateful</ejb-name>
<clustered>true</clustered>
<cache-config>
<cache-max-size>5000</cache-max-size>
<remove-timeout-seconds>18000</remove-timeout-seconds>
</cache-config>
</session>
</enterprise-beans>
</jboss>
Before replicating your bean, the container will check if your bean implements the Optimized
interface. If this is the case, the container calls the isModified() method and will only replicate
the bean when the method returns true. If the bean has not been modified (or not enough to require
replication, depending on your own preferences), you can return false and the replication would not
occur.
<bean name="StandardSFSBCacheConfig"
class="org.jboss.cache.config.Configuration">
<!-- Name of cluster. Needs to be the same for all members -->
<property name="clusterName">${jboss.partition.name:DefaultPartition}-
SFSBCache</property>
<!--
Use a UDP (multicast) based stack. Need JGroups flow control (FC)
because we are using asynchronous replication.
170
CacheManager service configuration
-->
<property name="multiplexerStack">${jboss.default.jgroups.stack:udp}</
property>
<property name="fetchInMemoryState">true</property>
<property name="nodeLockingScheme">PESSIMISTIC</property>
<property name="isolationLevel">REPEATABLE_READ</property>
<property name="useLockStriping">false</property>
<property name="cacheMode">REPL_ASYNC</property>
<!--
Number of milliseconds to wait until all responses for a
synchronous call have been received. Make this longer
than lockAcquisitionTimeout.
-->
<property name="syncReplTimeout">17500</property>
<!-- Max number of milliseconds to wait for a lock acquisition -->
<property name="lockAcquisitionTimeout">15000</property>
<!-- The max amount of time (in milliseconds) we wait until the
state (ie. the contents of the cache) are retrieved from
existing members at startup. -->
<property name="stateRetrievalTimeout">60000</property>
<!--
SFSBs use region-based marshalling to provide for partial state
transfer during deployment/undeployment.
-->
<property name="useRegionBasedMarshalling">false</property>
<!-- Must match the value of "useRegionBasedMarshalling" -->
<property name="inactiveOnStartup">false</property>
<property name="exposeManagementStatistics">true</property>
<property name="buddyReplicationConfig">
<bean class="org.jboss.cache.config.BuddyReplicationConfig">
<!--
A way to specify a preferred replication group. We try
and pick a buddy who shares the same pool name (falling
back to other buddies if not available).
-->
<property name="buddyPoolName">default</property>
171
Chapter 17. Clustered Session EJBs
<property name="buddyCommunicationTimeout">17500</property>
<property name="buddyLocatorConfig">
<bean
class="org.jboss.cache.buddyreplication.NextMemberBuddyLocatorConfig">
<!-- The number of backup nodes we maintain -->
<property name="numBuddies">1</property>
<!-- Means that each node will *try* to select a buddy on
a different physical host. If not able to do so
though, it will fall back to colocated nodes. -->
<property name="ignoreColocatedBuddies">true</property>
</bean>
</property>
</bean>
</property>
<property name="cacheLoaderConfig">
<bean class="org.jboss.cache.config.CacheLoaderConfig">
<!-- Do not change these -->
<property name="passivation">true</property>
<property name="shared">false</property>
<property name="individualCacheLoaderConfigs">
<list>
<bean class="org.jboss.cache.loader.FileCacheLoaderConfig">
<!-- Where passivated sessions are stored -->
<property name="location">${jboss.server.data.dir}${/}sfsb</
property>
<!-- Do not change these -->
<property name="async">false</property>
<property name="fetchPersistentState">true</property>
<property name="purgeOnStartup">true</property>
<property name="ignoreModifications">false</property>
<property name="checkCharacterPortability">false</property>
</bean>
</list>
</property>
</bean>
</property>
172
Stateless Session Bean in EJB 2.x
<property name="regionName">/</property>
<property name="evictionAlgorithmConfig">
<bean
class="org.jboss.cache.eviction.NullEvictionAlgorithmConfig"/>
</property>
</bean>
</property>
<!-- EJB3 integration code will programatically create other regions
as beans are deployed -->
</bean>
</property>
</bean>
Eviction
The default SFSB cache is configured to support eviction. The EJB3 SFSB container uses the
JBoss Cache eviction mechanism to manage SFSB passivation. When beans are deployed, the EJB
container will programatically add eviction regions to the cache, one region per bean type.
CacheLoader
A JBoss Cache CacheLoader is also configured; again to support SFSB passivation. When beans
are evicted from the cache, the cache loader passivates them to a persistent store; in this case
to the filesystem in the $JBOSS_HOME/server/all/data/sfsb directory. JBoss Cache supports
a variety of different CacheLoader implementations that know how to store data to different
persistent store types; see the JBoss Cache documentation for details. However, if you change the
CacheLoaderConfiguration, be sure that you do not use a shared store, e.g. a single schema in a
shared database. Each node in the cluster must have its own persistent store, otherwise as nodes
independently passivate and activate clustered beans, they will corrupt each other's data.
Buddy Replication
Using buddy replication, state is replicated to a configurable number of backup servers in the cluster
(aka buddies), rather than to all servers in the cluster. To enable buddy replication, adjust the following
properties in the buddyReplicationConfig property bean:
• Set enabled to true.
• Use the buddyPoolName to form logical subgroups of nodes within the cluster. If possible, buddies
will be chosen from nodes in the same buddy pool.
<jboss>
<enterprise-beans>
<session>
173
Chapter 17. Clustered Session EJBs
<ejb-name>nextgen.StatelessSession</ejb-name>
<jndi-name>nextgen.StatelessSession</jndi-name>
<clustered>true</clustered>
<cluster-config>
<partition-name>DefaultPartition</partition-name>
<home-load-balance-
policy>org.jboss.ha.framework.interfaces.RoundRobin</home-load-balance-
policy>
<bean-load-balance-
policy>org.jboss.ha.framework.interfaces.RoundRobin</bean-load-balance-
policy>
</cluster-config>
</session>
</enterprise-beans>
</jboss>
• partition-name specifies the name of the cluster the bean participates in. The default value
is DefaultPartition. The default partition name can also be set system-wide using the
jboss.partition.name system property.
• home-load-balance-policy indicates the class to be used by the home stub to balance calls made
on the nodes of the cluster. By default, the proxy will load-balance calls in a RoundRobin fashion.
• bean-load-balance-policy Indicates the class to be used by the bean stub to balance calls made
on the nodes of the cluster. By default, the proxy will load-balance calls in a RoundRobin fashion.
<jboss>
<enterprise-beans>
<session>
<ejb-name>nextgen.StatefulSession</ejb-name>
<jndi-name>nextgen.StatefulSession</jndi-name>
<clustered>True</clustered>
<cluster-config>
<partition-name>DefaultPartition</partition-nam>
<home-load-balance-
policy>org.jboss.ha.framework.interfaces.RoundRobin</home-load-balance-
policy>
174
Optimize state replication
<bean-load-balance-
policy>org.jboss.ha.framework.interfaces.FirstAvailable</bean-load-balance-
policy>
<session-state-manager-jndi-name>/HASessionState/Default</session-
state-manager-jndi-name>
</cluster-config>
</session>
</enterprise-beans>
</jboss>
In the bean configuration, only the <clustered> tag is mandatory to indicate that the bean works in
a cluster. The <cluster-config> element is optional and its default attribute values are indicated in
the sample configuration above.
The description of the remaining tags is identical to the one for stateless session bean. Actions on the
clustered stateful session bean's home interface are by default load-balanced, round-robin. Once the
bean's remote stub is available to the client, calls will not be load-balanced round-robin any more and
will stay "sticky" to the first node in the list.
Before replicating your bean, the container will detect if your bean implements this method. If your
bean does, the container calls the isModified() method and it only replicates the bean when
the method returns true. If the bean has not been modified (or not enough to require replication,
depending on your own preferences), you can return false and the replication would not occur. This
feature is available on JBoss Enterprise Application Platform 3.0.1+ only.
<bean name="HASessionStateService"
class="org.jboss.ha.hasessionstate.server.HASessionStateService">
<annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX
(name="jboss:service=HASessionState",
exposedInterface=org.jboss.ha.hasessionstate.server.
HASessionStateServiceMBean.class,
registerDirectly=true)</annotation>
175
Chapter 17. Clustered Session EJBs
</bean>
• HAPartition is a required attribute to inject the HAPartition service that HA-JNDI uses for intra-
cluster communication.
• jndiName is an optional attribute to specify the JNDI name under which this
HASessionStateService bean is bound. The default value is /HAPartition/Default.
• beanCleaningDelay is an optional attribute to specify the number of miliseconds after which the
HASessionStateService can clean a state that has not been modified. If a node, owning a bean,
crashes, its brother node will take ownership of this bean. Nevertheless, the container cache of
the brother node will not know about it (because it has never seen it before) and will never delete
according to the cleaning settings of the bean. That is why the HASessionStateService needs
to do this cleanup sometimes. The default value is 30*60*1000 milliseconds (i.e., 30 minutes).
The 3.2.7+/4.0.2+ releases contain a RetryInterceptor that can be added to the proxy client side
interceptor stack to allow for a transparent recovery from such a restart failure. To enable it for an
EJB, setup an invoker-proxy-binding that includes the RetryInterceptor. Below is an example jboss.xml
configuration.
<jboss>
<session>
<ejb-name>nextgen_RetryInterceptorStatelessSession</ejb-name>
<invoker-bindings>
<invoker>
<invoker-proxy-binding-name>clustered-retry-stateless-rmi-invoker</
invoker-proxy-binding-name>
<jndi-name>nextgen_RetryInterceptorStatelessSession</jndi-name>
</invoker>
</invoker-bindings>
<clustered>true</clustered>
</session>
<invoker-proxy-binding>
176
JNDI Lookup Process
<name>clustered-retry-stateless-rmi-invoker</name>
<invoker-mbean>jboss:service=invoker,type=jrmpha</invoker-mbean>
<proxy-factory>org.jboss.proxy.ejb.ProxyFactoryHA</proxy-factory>
<proxy-factory-config>
<client-interceptors>
<home>
<interceptor>org.jboss.proxy.ejb.HomeInterceptor</interceptor>
<interceptor>org.jboss.proxy.SecurityInterceptor</interceptor>
<interceptor>org.jboss.proxy.TransactionInterceptor</interceptor>
<interceptor>org.jboss.proxy.ejb.RetryInterceptor</interceptor>
<interceptor>org.jboss.invocation.InvokerInterceptor</
interceptor>
</home>
<bean>
<interceptor>org.jboss.proxy.ejb.StatelessSessionInterceptor</
interceptor>
<interceptor>org.jboss.proxy.SecurityInterceptor</interceptor>
<interceptor>org.jboss.proxy.TransactionInterceptor</interceptor>
<interceptor>org.jboss.proxy.ejb.RetryInterceptor</interceptor>
<interceptor>org.jboss.invocation.InvokerInterceptor</
interceptor>
</bean>
</client-interceptors>
</proxy-factory-config>
</invoker-proxy-binding>
</jboss>
1. It will check its own static retryEnv field. This field can be set by client code via a call to
RetryInterceptor.setRetryEnv(Properties). This approach to configuration has two downsides: first,
it reduces portability by introducing JBoss-specific calls to the client code; and second, since a
static field is used only a single configuration per JVM is possible.
2. If the retryEnv field is null, it will check for any environment properties bound to
a ThreadLocal by the org.jboss.naming.NamingContextFactory class. To use
this class as your naming context factory, in your jndi.properties set property
java.naming.factory.initial=org.jboss.naming.NamingContextFactory. The advantage of this
approach is use of org.jboss.naming.NamingContextFactory is simply a configuration option
in your jndi.properties file, and thus your java code is unaffected. The downside is the naming
properties are stored in a ThreadLocal and thus are only visible to the thread that originally
created an InitialContext.
3. If neither of the above approaches yield a set of naming environment properties, a default
InitialContext is used. If the attempt to contact a naming server is unsuccessful, by default the
InitialContext will attempt to fall back on multicast discovery to find an HA-JNDI naming server.
See the section on “ClusteredJNDI Services” for more on multicast discovery of HA-JNDI.
177
Chapter 17. Clustered Session EJBs
17.4.6. SingleRetryInterceptor
The RetryInterceptor is useful in many use cases, but a disadvantage it has is that it will continue
attempting to re-lookup the HA proxy in JNDI until it succeeds. If for some reason it cannot succeed,
this process could go on forever, and thus the EJB call that triggered the RetryInterceptor will never
return. For many client applications, this possibility is unacceptable. As a result, JBoss doesn't make
the RetryInterceptor part of its default client interceptor stacks for clustered EJBs.
In the 4.0.4.RC1 release, a new flavor of retry interceptor was introduced, the
org.jboss.proxy.ejb.SingleRetryInterceptor. This version works like the RetryInterceptor, but only
makes a single attempt to re-lookup the HA proxy in JNDI. If this attempt fails, the EJB call will fail just
as if no retry interceptor was used. Beginning with 4.0.4.CR2, the SingleRetryInterceptor is part of the
default client interceptor stacks for clustered EJBs.
The downside of the SingleRetryInterceptor is that if the retry attempt is made during a portion of a
cluster restart where no servers are available, the retry will fail and no further attempts will be made.
178
Chapter 18.
• If you persist a cache-enabled entity bean instance to the database via the entity manager, the entity
will be inserted into the cache.
• If you update an entity bean instance, and save the changes to the database via the entity manager,
the entity will be updated in the cache.
• If you remove an entity bean instance from the database via the entity manager, the entity will be
removed from the cache.
• If loading a cached entity from the database via the entity manager, and that entity does not exist in
the database, it will be inserted into the cache.
As well as a region for caching entities, the second-level cache also contains regions for caching
collections, queries, and timestamps. The Hibernate setup used for the JBoss EJB 3.0 implementation
uses JBoss Cache as its underlying second-level cache implementation.
Configuration of a the second-level cache is done via your EJB3 deployment's persistence.xml.
e.g.
179
Chapter 18. Clustered Entity EJBs
hibernate.cache.use_second_level_cache
Enables second-level caching of entities and collections.
hibernate.cache.use_query_cache
Enables second-level caching of queries.
hibernate.cache.region.factory_class
Defines the RegionFactory implementation that dictates region-specific caching behavior.
Hibernate ships with 2 types of JBoss Cache-based second-level caches: shared and multiplexed.
A shared region factory uses the same Cache for all cache regions - much like the legacy
CacheProvider implementation in older Hibernate versions.
org.hibernate.cache.jbc2.SharedJBossCacheRegionFactory
Uses a single JBoss Cache configuration, from a newly instantiated CacheManager, for all
cache regions.
org.hibernate.cache.jbc2.JndiSharedJBossCacheRegionFactory
Uses a single JBoss Cache configuration, from an existing CacheManager bound to JNDI, for
all cache regions.
180
Configure the distributed cache
A multiplexed region factory uses separate Cache instances, using optimized configurations for
each cache region.
org.hibernate.cache.jbc2.MultiplexedJBossCacheRegionFactory
Uses separate JBoss Cache configurations, from a newly instantiated CacheManager, per
cache region.
181
Chapter 18. Clustered Entity EJBs
org.hibernate.cache.jbc2.JndiMultiplexedJBossCacheRegionFactory
Uses separate JBoss Cache configurations, from a JNDI-bound CacheManager, see
Section 15.2.1, “The JBoss Enterprise Application Platform CacheManager Service”, per
cache region.
Now, we have JBoss Cache configured to support distributed caching of EJB 3.0 entity beans. We still
have to configure individual entity beans to use the cache service.
@Entity
@Cache(usage = CacheConcurrencyStrategy.TRANSACTIONAL)
public class Account implements Serializable
{
// ... ...
}
A very simplified rule of thumb is that you will typically want to do caching for objects that rarely
change, and which are frequently read. You can fine tune the cache for each entity bean in the
appropriate JBoss Cache configuration file, e.g. jboss-cache-manager-jboss-beans.xml. For instance,
you can specify the size of the cache. If there are too many objects in the cache, the cache can evict
the oldest or least used objects, depending on configuration, to make room for new objects. Assuming
the region_prefix specified in persistence.xml was myprefix, the default name of the cache
region for the com.mycompany.entities.Account entity bean would be /myprefix/com/
mycompany/entities/Account.
182
Configure the entity beans for cache
<bean class="org.jboss.cache.config.EvictionRegionConfig">
<property name="regionName">/</property>
<property name="evictionAlgorithmConfig">
<bean class="org.jboss.cache.eviction.LRUAlgorithmConfig">
<!-- Evict LRU node once we have more than this number of
nodes -->
<property name="maxNodes">10000</property>
<!-- And, evict any node that hasn't been accessed in this
many seconds -->
<property name="timeToLiveSeconds">1000</property>
<!-- Don't evict a node that's been accessed within this many
seconds.
Set this to a value greater than your max expected
transaction length. -->
<property name="minTimeToLiveSeconds">120</property>
</bean>
</property>
</bean>
</property>
<property name="evictionRegionConfigs">
<list>
<bean class="org.jboss.cache.config.EvictionRegionConfig">
<property name="regionName">/myprefix/com/mycompany/entities/
Account</property>
<property name="evictionAlgorithmConfig">
<bean class="org.jboss.cache.eviction.LRUAlgorithmConfig">
<property name="maxNodes">10000</property>
<property name="timeToLiveSeconds">5000</property>
<property name="minTimeToLiveSeconds">120</property>
</bean>
</property>
</bean>
... ...
</list>
</property>
</bean>
</property>
</bean>
If you do not specify a cache region for an entity bean class, all instances of this class will be cached
using the defaultEvictionRegionConfig as defined above. The @Cache annotation exposes an
optional attribute “region” that lets you specify the cache region where an entity is to be stored, rather
than having it be automatically be created from the fully-qualified class name of the entity class.
@Entity
@Cache(usage = CacheConcurrencyStrategy.TRANSACTIONAL, region = ”Account”)
public class Account implements Serializable
{
// ... ...
}
183
Chapter 18. Clustered Entity EJBs
184
Query result caching
Next, you create a named query associated with an entity, and tell Hibernate you want to cache the
results of that query:
@Entity
@Cache(usage = CacheConcurrencyStrategy.TRANSACTIONAL, region = ”Account”)
@NamedQueries(
{
@NamedQuery(
name = "account.bybranch",
query = "select acct from Account as acct where acct.branch = ?1",
hints = { @QueryHint(name = "org.hibernate.cacheable", value =
"true") }
)
})
public class Account implements Serializable
{
// ... ...
}
The @NamedQueries, @NamedQuery and @QueryHint annotations are all in the javax.persistence
package. See the Hibernate and EJB3 documentation for more on how to use EJB3 queries and on
how to instruct EJB3 to cache queries.
By default, Hibernate stores query results in JBoss Cache in a region named {region_prefix}/org/
hibernate/cache/StandardQueryCache. Based on this, you can set up separate eviction handling for
your query results. So, if the region prefix were set to myprefix in persistence.xml, you could, for
example, create this sort of eviction handling:
185
Chapter 18. Clustered Entity EJBs
<bean class="org.jboss.cache.config.EvictionRegionConfig">
<property name="regionName">/myprefix/Account</property>
<property name="evictionAlgorithmConfig">
<bean
class="org.jboss.cache.eviction.LRUAlgorithmConfig">
<property name="maxNodes">10000</property>
<property name="timeToLiveSeconds">5000</property>
<property name="minTimeToLiveSeconds">120</
property>
</bean>
</property>
</bean>
<bean class="org.jboss.cache.config.EvictionRegionConfig">
<property name="regionName">/myprefix/org/hibernate/
cache/StandardQueryCache</property>
<property name="evictionAlgorithmConfig">
<bean
class="org.jboss.cache.eviction.LRUAlgorithmConfig">
<property name="maxNodes">100</property>
<property name="timeToLiveSeconds">600</property>
<property name="minTimeToLiveSeconds">120</
property>
</bean>
</property>
</bean>
</list>
</property>
</bean>
</property>
</bean>
@Entity
@Cache(usage = CacheConcurrencyStrategy.TRANSACTIONAL, region = ”Account”)
@NamedQueries(
{
@NamedQuery(
name = "account.bybranch",
query = "select acct from Account as acct where acct.branch = ?1",
hints =
{
@QueryHint(name = "org.hibernate.cacheable", value = "true"),
@QueryHint(name = ”org.hibernate.cacheRegion, value = ”Queries”)
}
)
})
public class Account implements Serializable
186
Query result caching
{
// ... ...
}
187
Chapter 18. Clustered Entity EJBs
</bean>
... ...
</list>
</property>
</bean>
</property>
</bean>
To use a clustered entity bean, the application does not need to do anything special, except for looking
up EJB 2.x remote bean references from the clustered HA-JNDI.
To cluster EJB 2.x entity beans, you need to add the <clustered> element to the application's
jboss.xml descriptor file. Below is a typical jboss.xml file.
<jboss>
<enterprise-beans>
<entity>
<ejb-name>nextgen.EnterpriseEntity</ejb-name>
<jndi-name>nextgen.EnterpriseEntity</jndi-name>
<clustered>True</clustered>
<cluster-config>
<partition-name>DefaultPartition</partition-name>
<home-load-balance-
policy>org.jboss.ha.framework.interfaces.RoundRobin</home-load-balance-
policy>
<bean-load-balance-
policy>org.jboss.ha.framework.interfaces.FirstAvailable</bean-load-balance-
policy>
</cluster-config>
</entity>
</enterprise-beans>
</jboss>
The EJB 2.x entity beans are clustered for load balanced remote invocations. All the bean instances
are synchronized to have the same contents on all nodes.
However, clustered EJB 2.x Entity Beans do not have a distributed locking mechanism or a distributed
cache. They can only be synchronized by using row-level locking at the database level (see <row-
lock> in the CMP specification) or by setting the Transaction Isolation Level of your JDBC driver to
be TRANSACTION_SERIALIZABLE. Because there is no supported distributed locking mechanism
or distributed cache Entity Beans use Commit Option "B" by default (See standardjboss.xml
and the container configurations Clustered CMP 2.x EntityBean, Clustered CMP EntityBean, or
188
Entity Bean in EJB 2.x
Clustered BMP EntityBean). It is not recommended that you use Commit Option "A" unless your
Entity Bean is read-only. (There are some design patterns that allow you to use Commit Option "A"
with read-mostly beans. You can also take a look at the Seppuku pattern http://dima.dhs.org/misc/
readOnlyUpdates.html. JBoss may incorporate this pattern into later versions.)
Note
If you are using Bean Managed Persistence (BMP), you are going to have to implement
synchronization on your own. The MVCSoft CMP 2.0 persistence engine (see http://
www.jboss.org/jbossgroup/partners.jsp) provides different kinds of optimistic locking
strategies that can work in a JBoss cluster.
189
190
Chapter 19.
HTTP Services
HTTP session replication is used to replicate the state associated with web client sessions to other
nodes in a cluster. Thus, in the event one of your nodes crashes, another node in the cluster will be
able to recover. Two distinct functions must be performed:
State replication is directly handled by JBoss. When you run JBoss in the all configuration, session
state replication is enabled by default. Just configure your web application as <distributable>
in its web.xml (see Section 19.2, “Configuring HTTP session state replication”), deploy it, and its
session state is automtically replicated across all JBoss instances in the cluster.
However, load-balancing is a different story; it is not handled by JBoss itself and requires an external
load balancer. This function could be provided by specialized hardware switches or routers (Cisco
LoadDirector for example) or by specialized software running on commodity hardware. As a very
common scenario, we will demonstrate how to set up a software load balancer using Apache httpd
and mod_jk.
Note
A load-balancer tracks HTTP requests and, depending on the session to which the
request is linked, it dispatches the request to the appropriate node. This is called load-
balancing with sticky-sessions or session affinity: once a session is created on a node,
every future request will also be processed by that same node. Using a load-balancer that
supports sticky-sessions but not configuring your web application for session replication
allows you to scale very well by avoiding the cost of session state replication: each
request for a session will always be handled by the same node. But in case a node dies,
the state of all client sessions hosted by this node (the shopping carts, for example) will
be lost and the clients will most probably need to login on another node and restart with a
new session. In many situations, it is acceptable not to replicate HTTP sessions because
all critical state is stored in a database or on the client. In other situations, losing a client
session is not acceptable and, in this case, session state replication is the price one has to
pay.
191
Chapter 19. HTTP Services
Next, download mod_jk binaries. Several versions of mod_jk exist as well. We strongly advise you
to use mod_jk 1.2.x, as both mod_jk and mod_jk2 are deprecated, unsupported and no further
development is going on in the community. The mod_jk 1.2.x binary can be downloaded from
http://www.apache.org/dist/jakarta/tomcat-connectors/jk/binaries/. Rename the
downloaded file to mod_jk.so and copy it under APACHE_HOME/modules/.
# JkRequestLogFormat
JkRequestLogFormat "%w %V %T"
192
Configure worker nodes in mod_jk
• The LoadModule directive must reference the mod_jk library you have downloaded in the previous
section. You must indicate the exact same name with the "modules" file path prefix.
• The JkMount directive tells Apache which URLs it should forward to the mod_jk module (and, in
turn, to the Servlet containers). In the above file, all requests with URL path /application/* are
sent to the mod_jk load-balancer. This way, you can configure Apache to serve static contents (or
PHP contents) directly and only use the loadbalancer for Java applications. If you only use mod_jk
as a loadbalancer, you can also forward all URLs (i.e., /*) to mod_jk.
In addition to the JkMount directive, you can also use the JkMountFile directive to specify a mount
points configuration file, which contains multiple Tomcat forwarding URL mappings. You just need
to create a uriworkermap.properties file in the APACHE_HOME/conf directory. The format of
the file is /url=worker_name. To get things started, paste the following example into the file you
created:
This will configure mod_jk to forward requests to /jmx-console and /web-console to Tomcat.
You will most probably not change the other settings in mod_jk.conf. They are used to tell mod_jk
where to put its logging file, which logging level to use and so on.
193
Chapter 19. HTTP Services
worker.list=loadbalancer,status
# Define Node1
# modify the host as your host IP or DNS name.
worker.node1.port=8009
worker.node1.host=node1.mydomain.com
worker.node1.type=ajp13
worker.node1.lbfactor=1
worker.node1.cachesize=10
# Define Node2
# modify the host as your host IP or DNS name.
worker.node2.port=8009
worker.node2.host= node2.mydomain.com
worker.node2.type=ajp13
worker.node2.lbfactor=1
worker.node2.cachesize=10
# Load-balancing behaviour
worker.loadbalancer.type=lb
worker.loadbalancer.balance_workers=node1,node2
worker.loadbalancer.sticky_session=1
#worker.list=loadbalancer
Basically, the above file configures mod_jk to perform weighted round-robin load balancing with sticky
sessions between two servlet containers (that is, JBoss AS instances) node1 and node2 listening on
port 8009.
In the workers.properties file, each node is defined using the worker.XXX naming convention
where XXX represents an arbitrary name you choose for each of the target Servlet containers. For
each worker, you must specify the host name (or IP address) and the port number of the AJP13
connector running in the Servlet container.
The lbfactor attribute is the load-balancing factor for this specific worker. It is used to define the
priority (or weight) a node should have over other nodes. The higher this number is for a given worker
relative to the other workers, the more HTTP requests the worker will receive. This setting can be used
to differentiate servers with different processing power.
The cachesize attribute defines the size of the thread pools associated to the Servlet container
(i.e. the number of concurrent requests it will forward to the Servlet container). Make sure this
number does not outnumber the number of threads configured on the AJP13 connector of the Servlet
container. Please review http://jakarta.apache.org/tomcat/connectors-doc/config/
workers.html for comments on cachesize for Apache 1.3.x.
The last part of the conf/workers.properties file defines the loadbalancer worker. The only thing
you must change is the worker.loadbalancer.balanced_workers line: it must list all workers
previously defined in the same file: load-balancing will happen over these workers.
The sticky_session property specifies the cluster behavior for HTTP sessions. If you specify
worker.loadbalancer.sticky_session=0, each request will be load balanced between node1
194
Configuring JBoss to work with mod_jk
and node2; i.e., different requests for the same session will go to different servers. But when a user
opens a session on one server, it is always necessary to always forward this user's requests to the
same server, as long as that server is available. This is called a "sticky session", as the client is always
using the same server he reached on his first request. To enable session stickiness, you need to set
worker.loadbalancer.sticky_session to 1.
Note
A non-loadbalanced setup with a single node requires a worker.list=node1 entry.
On each clustered JBoss node, we have to name the node according to the name specified in
workers.properties. For instance, on JBoss instance node1, edit the JBOSS_HOME/server/
all/deploy/jbossweb.sar/server.xml file (replace /all with your own server name if
necessary). Locate the <Engine> element and add an attribute jvmRoute:
You also need to be sure the AJP connector in server.xml is enabled (i.e., uncommented). It is enabled
by default.
At this point, you have a fully working Apache+mod_jk load-balancer setup that will balance call to the
Servlet containers of your cluster while taking care of session stickiness (clients will always use the
same Servlet container).
Note
For more updated information on using mod_jk 1.2 with JBoss AS, please refer
to the JBoss wiki page at http://www.jboss.org/community/wiki/
UsingModjk12WithJBoss.
195
Chapter 19. HTTP Services
In Section 19.1.3, “Configure worker nodes in mod_jk”, we covered how to use sticky sessions to
make sure that a client in a session always hits the same server node in order to maintain the session
state. However, sticky sessions by themselves are not an ideal solution. If a node goes down, all its
session data is lost. A better and more reliable solution is to replicate session data across the nodes
in the cluster. This way, if a server node fails or is shut down, the load balancer can fail over the next
client request to any server node and obtain the same session state.
<?xml version="1.0"?>
<web-app xmlns="http://java.sun.com/xml/ns/j2ee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee
http://java.sun.com/xml/ns/j2ee/web-
app_2_4.xsd"
version="2.4">
<distributable/>
</web-app>
You can futher configure session replication using the replication-config element in the jboss-
web.xml file. However, the replication-config element only needs to be set if one or more of
the default values described below is unacceptable. Here is an example:
<jboss-web>
<replication-config>
<cache-name>custom-session-cache</cache-name>
<replication-trigger>SET</replication-trigger>
<replication-granularity>ATTRIBUTE</replication-granularity>
<replication-field-batch-mode>true</replication-field-batch-mode>
<use-jk>false</use-jk>
<max-unreplicated-interval>30</max-unreplicated-interval>
<snapshot-mode>instant</snapshot-mode>
<snapshot-interval>1000</snapshot-interval>
<session-notification-
policy>com.example.CustomSessionNotificationPolicy</session-notification-
policy>
</replication-config>
</jboss-web>]]>
196
Enabling session replication in your application
All of the above configuration elements are optional and can be ommitted if the default value is
acceptable. A couple are commonly used; the rest are very infrequently changed from the defaults.
We'll cover the commonly used ones first.
The replication-trigger element determines when the container should consider that session
data must be replicated across the cluster. The rationale for this setting is that after a mutable object
stored as a session attribute is accessed from the session, in the absence of a setAttribute call
the container has no clear way to know if the object (and hence the session state) has been modified
and needs to be replicated. This element has 3 valid values:
• SET_AND_GET is conservative but not optimal (performance-wise): it will always replicate session
data even if its content has not been modified but simply accessed. This setting made (a little) sense
in AS 4 since using it was a way to ensure that every request triggered replication of the session's
timestamp. Since setting max_unreplicated_interval to 0 accomplishes the same thing at
much lower cost, using SET_AND_GET makes no sense with AS 5.
• SET assumes that the developer will explicitly call setAttribute on the session if the data needs
to be replicated. This setting prevents unnecessary replication and can have a major beneficial
impact on performance, but requires very good coding practices to ensure setAttribute is
always called whenever a mutable object stored in the session is modified.
The replication-granularity element determines the granularity of what gets replicated if the
container determines session replication is needed. The supported values are:
• SESSION indicates that the entire session attribute map should be replicated when any attribute
is considered modified. Replication occurs at request end. This option replicates the most data
and thus incurs the highest replication cost, but since all attributes values are always replicated
together it ensures that any references between attribute values will not be broken when the session
is deserialized. For this reason it is the default setting.
• ATTRIBUTE indicates that only attributes that the session considers to be potentially modified are
replicated. Replication occurs at request end. For sessions carrying large amounts of data, parts
of which are infrequently updated, this option can significantly increase replication performance.
However, it is not suitable for applications that store objects in different attributes that share
references with each other (e.g. a Person object in the "husband" attribute sharing with another
Person in the "wife" attribute a reference to an Address object). This is because if the attributes
are separately replicated, when the session is deserialized on remote nodes the shared references
will be broken.
• FIELD is useful if the classes stored in the session have been bytecode enhanced for use by POJO
Cache. If they have been, the session management layer will detect field level changes within
objects stored to the session, and will replicate only those changes. This is the most performant
setting. Replication is only for individual changed data fields inside session attribute objects. Shared
object references will be preserved across the cluster. Potentially most performant, but requires
changes to your application (this will be discussed later).
The other elements under the replication-config element are much less frequently used.
197
Chapter 19. HTTP Services
The cacheName element indicates the name of the JBoss Cache configuration that should be used
for storing distributable sessions and replicating them around the cluster. This element lets web
applications that require different caching characteristics specify the use of separate, differently
configured, JBoss Cache instances. In JBoss AS 4 the cache to use was a server-wide configuration
that could not be changed per web application. The default value is standard-session-cache if
the replication-granularity is not FIELD, field-granularity-session-cache if it is.
See Section 19.2.3, “Configuring the JBoss Cache instance used for session state replication” for
more details on JBoss Cache configuration for web tier clustering.
The useJK element indicates whether the container should assume that a JK-based software
load balancer (e.g. mod_jk, mod_proxy, mod_cluster) is being used for load balancing for this web
application. If set to true, the container will examine the session ID associated with every request and
replace the jvmRoute portion of the session ID if it detects a failover.
The default value is null (i.e. unspecified). In this case the session manager will use the presence
or absence of a jvmRoute configuration on its enclosing JBoss Web Engine (see Section 19.1.4,
“Configuring JBoss to work with mod_jk”) to determine whether JK is used.
You need only set this to false for web applications whose URL cannot be handled by the JK load
balancer.
A value of 0 means the timestamp will be replicated whenever the session is accessed. A value of -1
means the timestamp will be replicated only if some other activity during the request (e.g. modifying an
attribute) has resulted in other replication work involving the session. A positive value greater than the
HttpSession.getMaxInactiveInterval() value will be treated as probable misconfiguration
and converted to 0; i.e. replicate the metadata on every request. Default value is 60.
The snapshot-mode element configures when sessions are replicated to the other nodes. Possible
values are instant (the default) and interval.
The typical value, instant, replicates changes to the other nodes at the end of requests, using the
request processing thread to perform the replication. In this case, the snapshot-interval property
is ignored.
With interval mode, a background task is created that runs every snapshot-interval
milliseconds, checking for modified sessions and replicating them.
Note that this property has no effect if replication-granularity is set to FIELD. If it is FIELD,
instant mode will be used.
198
HttpSession Passivation and Activation
The snapshot-interval element defines how often (in milliseconds) the background task that
replicates modified sessions should be started for this web application. Only meaningful if snapshot-
mode is set to interval.
Event notifications that may make sense in a non-clustered environment may or may not
make sense in a clustered environment; see https://jira.jboss.org/jira/browse/JBAS-5778
for an example of why a notification may not be desired. Configuring an appropriate
ClusteredSessionNotificationPolicy gives the application author fine-grained control over
what notifications are issued.
• When the container requests the creation of a new session. If the number of currently active
sessions exceeds a configurable limit, an attempt is made to passivate sessions to make room in
memory.
• Periodically (by default every ten seconds) as the JBoss Web background task thread runs.
• When the web application is deployed and a backup copy of sessions active on other servers is
acquired by the newly deploying web application's session manager.
• The session has not been in use for longer than a configurable maximum idle time.
• The number of active sessions exceeds a configurable maximum and the session has not been in
use for longer than a configurable minimum idle time.
In both cases, sessions are passivated on a Least Recently Used (LRU) basis.
199
Chapter 19. HTTP Services
<jboss-web>
<max-active-sessions>20</max-active-sessions>
<passivation-config>
<use-session-passivation>true</use-session-passivation>
<passivation-min-idle-time>60</passivation-min-idle-time>
<passivation-max-idle-time>600</passivation-max-idle-time>
</passivation-config>
</jboss-web>
• max-active-session
Determines the maximum number of active sessions allowed. If the number of sessions managed
by the the session manager exceeds this value and passivation is enabled, the excess will be
passivated based on the configured passivation-min-idle-time. If after passivation is
completed (or if passivation is disabled), the number of active sessions still exceeds this limit,
attempts to create new sessions will be rejected. If set to -1 (the default), there is no limit.
• use-session-passivation
Determines whether session passivation will be enabled for the web application. Default is false.
• passivation-min-idle-time
Determines the minimum time (in seconds) that a session must have been inactive before the
container will consider passivating it in order to reduce the active session count to obey the value
defined by max-active-sessions. A value of -1 (the default) disables passivating sessions
before passivation-max-idle-time. Neither a value of -1 nor a high value are recommended
if max-active-sessions is set.
• passivation-max-idle-time
Determines the maximum time (in seconds) that a session can be inactive before the container
should attempt to passivate it to save memory. Passivation of such sessions will take place
regardless of whether the active session count exceeds max-active-sessions. Should be less
than the web.xml session-timeout setting. A value of -1 (the default) disables passivation
based on maximum inactivity.
The total number of sessions in memory includes sessions replicated from other cluster nodes that
are not being accessed on this node. Take this into account when setting max-active-sessions.
The number of sessions replicated from other nodes will also depend on whether buddy replication is
enabled.
Say, for example, that you have an eight node cluster, and each node handles requests from 100
users. With total replication, each node would store 800 sessions in memory. With buddy replication
enabled, and the default numBuddies setting (1), each node will store 200 sessions in memory.
200
Configuring the JBoss Cache instance used for session state replication
The name of the JBoss Cache configuration to use is controlled by the cacheName element in the
application's jboss-web.xml (see Section 19.2.1, “Enabling session replication in your application”).
In most cases, though, this does not need to be set as the default values of standard-session-
cache and field-granularity-session-cache (for applications configured for FIELD
granularity) are appropriate.
The JBoss Cache configurations in the CacheManager service expose a number of options. See
Chapter 23, JBoss Cache Configuration and Deployment and the JBoss Cache documentation for a
more complete discussion. The standard-session-cache and field-granularity-session-
cache configurations are already optimized for the web session replication use case, and most of the
settings should not be altered. Administrators may be interested in altering the following settings:
• cacheMode
The default is REPL_ASYNC, which specifies that a session replication message sent to the cluster
does not wait for responses from other cluster nodes confirming that the message has been
received and processed. The alternative mode, REPL_SYNC, offers a greater degree of confirmation
that session state has been received, but reduces performance significantly. See Section 23.1.2,
“Cache Mode” for further details.
Set to true to enable buddy replication. See Section 23.1.8, “Buddy Replication”. Default is false.
Set to a value greater than the default (1) to increase the number of backup nodes onto which
sessions are replicated. Only relevant if buddy replication is enabled. See Section 23.1.8, “Buddy
Replication”.
A way to specify a preferred replication group when buddy replication is enabled. JBoss Cache tries
to pick a buddy who shares the same pool name (falling back to other buddies if not available). Only
relevant if buddy replication is enabled. See Section 23.1.8, “Buddy Replication”.
• multiplexerStack
Name of the JGroups protocol stack the cache should use. See Section 15.1.1, “The Channel
Factory Service”.
• clusterName
Identifying name JGroups will use for this cache's channel. Only change this if you create a new
cache configuration, in which case this property should have a different value from all other cache
configurations.
201
Chapter 19. HTTP Services
If you wish to use a completely new JBoss Cache configuration rather than editing one of the existing
ones, please see Section 23.2.1, “Deployment Via the CacheManager Service”.
First, you need to identify the classes that you need to prepare. You can identify these classes by
using annotations, like so:
@@org.jboss.cache.pojo.annotation.Replicable
public class Address
{
...
}
If you annotate a class with @Replicable, then all of its subclasses will be automatically annotated
as well. Similarly, you can annotate an interface with @Replicable and all of its implementing
classes will be annotated. For example:
@org.jboss.cache.aop.InstanceOfAopMarker
public class Person
{
...
}
There is no need to annotate Student. POJO Cache will recognize it as @Replicable because it is a
sub-class of Person.
JBoss AS 5 requires JDK 5 at runtime, but some users may still need to build their projects using
JDK 1.4. In this case, annotating classes can be done via JDK 1.4 style annotations embedded in
JavaDocs. For example:
/**
* Represents a street address.
* @@org.jboss.cache.pojo.annotation.Replicable
*/
public class Address
{
202
Using FIELD level replication
...
}
Once you have annotated your classes, you will need to perform a pre-processing step to bytecode
enhance your classes for use by POJO Cache. You need to use the JBoss AOP pre-compiler
annotationc and post-compiler aopc to process the above source code before and after they are
compiled by the Java compiler. The annotationc step is only need if the JDK 1.4 style annotations
are used; if JDK 5 annotations are used it is not necessary. Here is an example of how to invoke those
commands from command line.
Please see the JBoss AOP documentation for the usage of the pre- and post-compiler. The JBoss
AOP project also provides easy to use ANT tasks to help integrate those steps into your application
build process.
Note
You can see a complete example of how to build, deploy, and validate a FIELD-
level replicated web application from this page: http://www.jboss.org/community/wiki/
httpsessionfieldlevelexample. The example bundles the pre- and post-compile tools so
you do not need to download JBoss AOP separately.
Finally, let's see an example on how to use FIELD-level replication on those data classes. First, we
see some servlet code that reads some data from the request parameters, creates a couple of objects
and stores them in the session:
husband.setAddress(addr);
wife.setAddress(addr); // husband and wife share the same address!
203
Chapter 19. HTTP Services
Notice that in there is no need to call session.setAttribute() after you make changes to the
data object, and all changes to the fields are automatically replicated across the cluster.
Besides plain objects, you can also use regular Java collections of those objects as session attributes.
POJO Cache automatically figures out how to handle those collections and replicate field changes in
their member objects.
19.4.1. Configuration
To enable clustered single sign-on, you must add the ClusteredSingleSignOn valve to the
appropriate Host elements of the JBOSS_HOME/server/all/deploy/jbossweb.sar/
server.xml file. The valve element is already included in the standard file; you just need to
uncomment it. The valve configuration is shown here:
<Valve className="org.jboss.web.tomcat.service.sso.ClusteredSingleSignOn" /
>
• className is a required attribute to set the Java class name of the valve implementation to use.
This must be set to org.jboss.web.tomcat.service.sso.ClusteredSingleSign.
• cacheConfig is the name of the cache configuration (see Section 15.2.1, “The JBoss Enterprise
Application Platform CacheManager Service”) to use for the clustered SSO cache. Default is
clustered-sso.
• cookieDomain is used to set the host domain to be used for sso cookies. See Section 19.4.4,
“Configuring the Cookie Domain” for more. Default is "/".
• maxEmptyLife is the maximum number of seconds an SSO with no active sessions will be usable
by a request. The clustered SSO valve tracks what cluster nodes are managing sessions related to
an SSO. A positive value for this attribute allows proper handling of shutdown of a node that is the
only one that had handled any of the sessions associated with an SSO. The shutdown invalidates
the local copy of the sessions, eliminating all sessions from the SSO. If maxEmptyLife were zero,
204
SSO Behavior
the SSO would terminate along with the local session copies. But, backup copies of the sessions
(if they are from clustered webapps) are available on other cluster nodes. Allowing the SSO to live
beyond the life of its managed sessions gives the user time to make another request which can fail
over to a different cluster node, where it activates the the backup copy of the session. Default is
1800, i.e. 30 minutes.
• processExpiresInterval is the minimum number of seconds between efforts by the valve to find
and invalidate SSO's that have exceeded their 'maxEmptyLife'. Does not imply effort will be spent
on such cleanup every 'processExpiresInterval', just that it won't occur more frequently than that.
Default is 60.
Upon access to a protected resource in any web app, the user will be challenged to authenticate,
using the login method defined for the web app.
Once authenticated, the roles associated with this user will be utilized for access control decisions
across all of the associated web applications, without challenging the user to authenticate themselves
to each application individually.
A session timeout does not invalidate the SSO if other sessions are still valid.
19.4.3. Limitations
There are a number of known limitations to this Tomcat valve-based SSO implementation:
• Only useful within a cluster of JBoss servers; SSO does not propagate to other resources.
• Requires cookies. SSO is maintained via a cookie and URL rewriting is not supported.
• Unless requireReauthentication is set to true, all web applications configured for the same
SSO valve must share the same JBoss Web Realm and JBoss Security security-domain. This
means:
• In server.xml you can nest the Realm element inside the Host element (or the surrounding
Engine element), but not inside a context.xml packaged with one of the involved web
applications.
205
Chapter 19. HTTP Services
For example, suppose we have a case where two apps, with URLs http://app1.xyz.com and
http://app2.xyz.com, that wish to share an SSO context. These apps could be running on
different servers in a cluster or the virtual host with which they are associated could have multiple
aliases. This can be supported with the following configuration:
<Valve className="org.jboss.web.tomcat.service.sso.ClusteredSingleSignOn"
cookieDomain="xyz.com" />
206
Chapter 20.
Every node deployed must have a unique id, including those in a particular LAN cluster, and also
those only linked by message bridges.
JBoss Messaging balances messages between nodes, catering for faster or slower consumers to
efficiently balance processing load across the cluster.
If you do not want message redistribution between nodes, but still want to retain
the other characteristics of clustered destinations, you can specify the attribute
ClusterPullConnectionFactoryName on the server peer.
Note
The side effect of setting this to true is that messages cannot be distributed as freely
around the cluster.
207
Chapter 20. JBoss Messaging Clustering Notes
If the supportsFailover attribute of the connection factory is set to true then automatic failover is
enabled. This will automatically failover from one server to another, transparently to the user, in case
of failure.
If automatic failover is not required or you wish to do manual failover (JBoss MQ style) this can be set
to false, and you can supply a standard JMS ExceptionListener on the connection which will be called
in case of connection failure. You would then need to manually close the connection, lookup a new
connection factory from HA JNDI and recreate the connection.
208
Chapter 21.
When the master fails or is shut down, another master is selected from the remaining nodes and the
service is restarted on the new master. Thus, other than a brief interval when one master has stopped
and another has yet to take over, the service is always being provided by one but only one node.
209
Chapter 21. Clustered Deployment Options
So, by placing your deployments in deploy-hasingleton you know that they will be deployed
only on the master node in the cluster. If the master node cleanly shuts down, they will be cleanly
undeployed as part of shutdown. If the master node fails or is shut down, they will be deployed on
whatever node takes over as master.
210
HASingleton Deployment Options
• If the master node fails and another node takes over as master, your singleton service needs to
go through the entire deployment process before it will be providing services. Depending on the
complexity of your service's deployment, and the extent of startup activity in which it engages, this
could take a while, during which time the service is not being provided.
First, we have a POJO that we want to make an HA singleton. The only thing special about it is it
needs to expose a public method that can be called when it should begin providing service, and
another that can be called when it should stop providing service:
We used startSingleton and stopSingleton in the above example, but you could name the
methods anything.
Next, we deploy our service, along with an HASingletonController to control it, most likely packaged in
a .sar file, with the following META-INF/jboss-beans.xml:
211
Chapter 21. Clustered Deployment Options
<deployment xmlns="urn:jboss:bean-deployer:2.0">
<!-- This bean is an example of a clustered singleton -->
<bean name="HASingletonExample"
class="org.jboss.ha.examples.HASingletonExample">
<annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX
(name="jboss:service=HASingletonExample",
exposedInterface=org.jboss.ha.examples.HASingletonExampleMBean.class)</
annotation>
</bean>
<bean name="ExampleHASingletonController"
class="org.jboss.ha.singleton.HASingletonController">
<annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX
(name="jboss:service=ExampleHASingletonController",
exposedInterface=org.jboss.ha.singleton.HASingletonControllerMBean.class,
registerDirectly=true)</annotation>
<property name="HAPartition"><inject bean="HAPartition"/></property>
<property name="target"><inject bean="HASingletonExample"/></property>
<property name="targetStartMethod">startSingleton</property>
<property name="targetStopMethod">stopSingleton</property>
</bean>
</deployment>
The primary advantage of this approach over deploy-ha-singleton. is that the above example can be
placed in deploy or farm and thus can be hot deployed and farmed deployed. Also, if our example
service had complex, time-consuming startup requirements, those could potentially be implemented in
create() or start() methods. JBoss will invoke create() and start() as soon as the service is deployed;
it doesn't wait until the node becomes the master node. So, the service could be primed and ready
to go, just waiting for the controller to implement startSingleton() at which point it can immediately
provide service.
Although not demonstrated in the example above, the HASingletonController can support an
optional argument for either or both of the target start and stop methods. These are specified using
the targetStartMethodArgument and TargetStopMethodArgument properties, respectively.
Currently, only string values are supported.
<depends>HASingletonDeployerBarrierController</depends>
The way it works is that a BarrierController is deployed along with the HASingletonDeployer and
listens for JMX notifications from it. A BarrierController is a relatively simple Mbean that can subscribe
to receive any JMX notification in the system. It uses the received notifications to control the lifecycle
212
Determining the master node
of a dynamically created Mbean called the Barrier. The Barrier is instantiated, registered and brought
to the CREATE state when the BarrierController is deployed. After that, the BarrierController starts
and stops the Barrier when matching JMX notifications are received. Thus, other services need only
depend on the Barrier bean using the usual <depends> tag, and they will be started and stopped in
tandem with the Barrier. When the BarrierController is undeployed the Barrier is also destroyed.
This provides an alternative to the deploy-hasingleton approach in that we can use farming to
distribute the service, while content in deploy-hasingleton must be copied manually on all nodes.
On the other hand, the barrier-dependent service will be instantiated/created (i.e., any create()
method invoked) on all nodes, but only started on the master node. This is different with the deploy-
hasingleton approach that will only deploy (instantiate/create/start) the contents of the deploy-
hasingleton directory on one of the nodes.
So services depending on the barrier will need to make sure they do minimal or no work inside their
create() step, rather they should use start() to do the work.
Note
The Barrier controls the start/stop of dependent services, but not their destruction, which
happens only when the BarrierController is itself destroyed/undeployed. Thus
using the Barrier to control services that need to be "destroyed" as part of their normal
“undeploy” operation (like, for example, an EJBContainer) will not have the desired
effect.
For each member of the cluster, the HAPartition service maintains an attribute called the CurrentView,
which is basically an ordered list of the current members of the cluster. As nodes join and leave the
cluster, JGroups ensures that each surviving member of the cluster gets an updated view. You can
see the current view by going into the JMX console, and looking at the CurrentView attribute in the
jboss:service=DefaultPartition mbean. Every member of the cluster will have the same
view, with the members in the same order.
Let's say, for example, that we have a 4 node cluster, nodes A through D, and the current view can be
expressed as {A, B, C, D}. Generally speaking, the order of nodes in the view will reflect the order in
which they joined the cluster (although this is not always the case, and should not be assumed to be
the case).
To further our example, let's say there is a singleton service (i.e. an HASingletonController)
named Foo that's deployed around the cluster, except, for whatever reason, on B. The HAPartition
service maintains across the cluster a registry of what services are deployed where, in view order. So,
on every node in the cluster, the HAPartition service knows that the view with respect to the Foo
service is {A, C, D} (no B).
Whenever there is a change in the cluster topology of the Foo service, the HAPartition service
invokes a callback on Foo notifying it of the new topology. So, for example, when Foo started on D, the
Foo service running on A, C and D all got callbacks telling them the new view for Foo was {A, C, D}.
213
Chapter 21. Clustered Deployment Options
That callback gives each node enough information to independently decide if it is now the master. The
Foo service on each node uses the HAPartition's HASingletonElectionPolicy to determine if
they are the master, as explained in the Section 21.1.2.1, “HA singleton election policy”.
If A were to fail or shutdown, Foo on C and D would get a callback with a new view for Foo of {C, D}. C
would then become the master. If A restarted, A, C and D would get a callback with a new view for Foo
of {C, D, A}. C would remain the master – there's nothing magic about A that would cause it to become
the master again just because it was before.
HASingletonElectionPolicySimple
This policy selects a master node based relative age. The desired age is configured via the
position property, which corresponds to the index in the list of available nodes. position =
0, the default, refers to the oldest node; position = 1, refers to the 2nd oldest; etc. position
can also be negative to indicate youngness; imagine the list of available nodes as a circular linked
list. position = -1, refers to the youngest node; position = -2, refers to the 2nd youngest
node; etc.
<bean class="org.jboss.ha.singleton.HASingletonElectionPolicySimple">
<property name="position">-1</property>
</bean>
PreferredMasterElectionPolicy
This policy extends HASingletonElectionPolicySimple, allowing the configuration of a
preferred node. The preferredMaster property, specified as host:port or address:port, identifies
a specific node that should become master, if available. If the preferred node is not available, the
election policy will behave as described above.
<bean class="org.jboss.ha.singleton.PreferredMasterElectionPolicy">
<property name="preferredMaster">server1:12345</property>
</bean>
214
Farming Deployment
in exploded form) to the all/farm/ directory of any cluster member and the application will be
automatically duplicate across all nodes in the same cluster. If a node joins the cluster later, it will
pull in all farm deployed applications in the cluster and deploy them locally at start-up time. If you
delete the application from a running clustered server node's farm/ directory, the application will
be undeployed locally and then removed from all other clustered server nodes' farm/ directories
(triggering undeployment).
Farming is enabled by default in the all configuration in JBoss AS and thus requires no manual
setup. The required farm-deployment-jboss-beans.xml and timestamps-jboss-
beans.xml configuration files are located in the deploy/cluster directory. If you want to enable
farming in a custom configuration, simply copy these files to the corresponding JBoss deploy directory
$JBOSS_HOME/server/your_own_config/deploy/cluster. Make sure that your custom
configuration has clustering enabled.
While there is little need to customize the farming service, it can be customized via the
FarmProfileRepositoryClusteringHandler bean, whose properties and default values are
listed below:
<bean name="FarmProfileRepositoryClusteringHandler"
class="org.jboss.profileservice.cluster.repository.
DefaultRepositoryClusteringHandler">
• partition is a required attribute to inject the HAPartition service that the farm service uses for intra-
cluster communication.
• profile[Domain|Server|Name] are all used to identify the profile for which this handler is intended.
• immutable indicates whether or not this handler allows a node to push content changes
to the cluster. A value of true is equivalent to setting synchronizationPolicy to
org.jboss.system.server.profileservice.repository.clustered.sync.
ImmutableSynchronizationPolicy.
• lockTimeout defines the number of milliseconds to wait for cluster-wide lock acquisition.
• methodCallTimeout defines the number of milliseconds to wait for invocations on remote cluster
nodes.
215
Chapter 21. Clustered Deployment Options
was previously removed by the farming service but might still exist on the starting node if it was not
running when the removal took place. The default synchronization policy is defined as follows:
<bean name="FarmProfileSynchronizationPolicy"
class="org.jboss.profileservice.cluster.repository.
DefaultSynchronizationPolicy">
<property name="allowJoinAdditions"><null/></property>
<property name="allowJoinReincarnations"><null/></property>
<property name="allowJoinUpdates"><null/></property>
<property name="allowJoinRemovals"><null/></property>
<property name="allowMergeAdditions"><null/></property>
<property name="allowMergeReincarnations"><null/></property>
<property name="allowMergeUpdates"><null/></property>
<property name="allowMergeRemovals"><null/></property>
<property name="developerMode">false</property>
<property name="removalTrackingTime">2592000000</property><!-- 30 days
-->
<property name="timestampService"><inject
bean="TimestampDiscrepancyService"/></property>
</bean>
• developerMode enables a lenient synchronization policy that allows all changes. Enabling
developer mode is equivalent to setting each of the above properties to true and is intended for
development environments.
• removalTrackingTime defines the number of milliseconds for which this policy should
remembered removed items, for use in detecting reincarnations.
• timestampService estimates and tracks discrepancies in system clocks for current and past
members of the cluster. Default implementation is defined in timestamps-jboss-beans.xml.
216
Chapter 22.
JGroups Services
JGroups provides the underlying group communication support for JBoss Enterprise Application
Platform clusters. The interaction of clustered services with JGroups was covered in Section 15.1,
“Group Communication with JGroups”. This chapter focuses on the details of this interaction, with
particular attention to configuration details and troubleshooting tips.
This chapter is not intended as complete JGroups documentation. If you want to know more about
JGroups, you can consult:
The first section of this chapter covers the many JGroups configuration options in detail. JBoss
Enterprise Application Platform ships with a set of default JGroups configurations. Most applications
will work with the default configurations out of the box. You will only need to edit these configurations
when you deploy an application with special network or performance requirements.
Figure 22.1, “Protocol stack in JGroups” shows a conceptual cluster with each member's channel
composed of a stack of JGroups protocols.
217
Chapter 22. JGroups Services
This section of the chapter covers some of the most commonly used protocols, according to the
type of behaviour they add to the channel. We discuss a few key configuration attributes exposed by
each protocol, but since these attributes should be altered only by experts, this chapter focuses on
familiarizing users with the purpose of various protocols.
The JGroups configurations used in JBoss Enterprise Application Platform appear as nested elements
in the $JBOSS_HOME/server/all/cluster/jgroups-channelfactory.sar/META-INF/
jgroups-channelfactory-stacks.xml file. This file is parsed by the ChannelFactory service,
which uses the contents to provide correctly configured channels to the clustered services that require
them. See Section 15.1.1, “The Channel Factory Service” for more on the ChannelFactory service.
<![CDATA[<stack name="udp-async"
description="Same as the default 'udp' stack above, except
message bundling
is enabled in the transport protocol
(enable_bundling=true).
Useful for services that make high-volume
asynchronous
RPCs (e.g. high volume JBoss Cache instances
configured
for REPL_ASYNC) where message bundling may improve
performance.">
<config>
<UDP
218
Configuring a JGroups Channel's Protocol Stack
singleton_name="udp-async"
mcast_port="${jboss.jgroups.udp_async.mcast_port:45689}"
mcast_addr="${jboss.partition.udpGroup:228.11.11.11}"
tos="8"
ucast_recv_buf_size="20000000"
ucast_send_buf_size="640000"
mcast_recv_buf_size="25000000"
mcast_send_buf_size="640000"
loopback="true"
discard_incompatible_packets="true"
enable_bundling="true"
max_bundle_size="64000"
max_bundle_timeout="30"
ip_ttl="${jgroups.udp.ip_ttl:2}"
thread_naming_pattern="cl"
timer.num_threads="12"
enable_diagnostics="${jboss.jgroups.enable_diagnostics:true}"
diagnostics_addr="${jboss.jgroups.diagnostics_addr:224.0.0.75}"
diagnostics_port="${jboss.jgroups.diagnostics_port:7500}"
thread_pool.enabled="true"
thread_pool.min_threads="8"
thread_pool.max_threads="200"
thread_pool.keep_alive_time="5000"
thread_pool.queue_enabled="true"
thread_pool.queue_max_size="1000"
thread_pool.rejection_policy="discard"
oob_thread_pool.enabled="true"
oob_thread_pool.min_threads="8"
oob_thread_pool.max_threads="200"
oob_thread_pool.keep_alive_time="1000"
oob_thread_pool.queue_enabled="false"
oob_thread_pool.rejection_policy="discard"/>
<PING timeout="2000" num_initial_members="3"/>
<MERGE2 max_interval="100000" min_interval="20000"/>
<FD_SOCK/>
<FD timeout="6000" max_tries="5" shun="true"/>
<VERIFY_SUSPECT timeout="1500"/>
<BARRIER/>
<pbcast.NAKACK use_mcast_xmit="true" gc_lag="0"
retransmit_timeout="300,600,1200,2400,4800"
discard_delivered_msgs="true"/>
<UNICAST timeout="300,600,1200,2400,3600"/>
<pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"
max_bytes="400000"/>
<VIEW_SYNC avg_send_interval="10000"/>
<pbcast.GMS print_local_addr="true" join_timeout="3000"
shun="true"
view_bundling="true"
219
Chapter 22. JGroups Services
view_ack_collection_timeout="5000"
resume_task_timeout="7500"/>
<FC max_credits="2000000" min_threshold="0.10"
ignore_synchronous_response="true"/>
<FRAG2 frag_size="60000"/>
<!-- pbcast.STREAMING_STATE_TRANSFER/ -->
<pbcast.STATE_TRANSFER/>
<pbcast.FLUSH timeout="0" start_flush_timeout="10000"/>
</config>
</stack> ]]>
The <config> element contains all the configuration data for JGroups. This information is used to
configure a JGroups channel, which is conceptually similar to a socket, and manages communication
between peers in a cluster. Each element within the <config> element defines a particular JGroups
protocol. Each protocol performs one function. The combination of these functions defines the
characteristics of the channel as a whole. The next few sections describe common protocols and
explain the options available to each.
• stats whether the protocol should gather runtime statistics on its operations that can be exposed
via tools like the AS's administration console or the JGroups Probe utility. What, if any, statistics are
gathered depends on the protocol. Default is true.
Note
All of the protocols in the versions of JGroups used in JBoss Application Server 3.x and
4.x exposed down_thread and up_thread attributes. The JGroups version included
in JBoss Application Server 5 and later no longer uses those attributes, and a WARN
message will be written to the server log if they are configured for any protocol.
Note
The UDP, TCP, and TUNNEL protocols are mutually exclusive. You can only have one
transport protocol in each JGroups Config element
220
Transport Protocols
<![CDATA[ <UDP
singleton_name="udp-async"
mcast_port="${jboss.jgroups.udp_async.mcast_port:45689}"
mcast_addr="${jboss.partition.udpGroup:228.11.11.11}"
tos="8"
ucast_recv_buf_size="20000000"
ucast_send_buf_size="640000"
mcast_recv_buf_size="25000000"
mcast_send_buf_size="640000"
loopback="true"
discard_incompatible_packets="true"
enable_bundling="true"
max_bundle_size="64000"
max_bundle_timeout="30"
ip_ttl="${jgroups.udp.ip_ttl:2}"
thread_naming_pattern="cl"
timer.num_threads="12"
enable_diagnostics="${jboss.jgroups.enable_diagnostics:true}"
diagnostics_addr="${jboss.jgroups.diagnostics_addr:224.0.0.75}"
diagnostics_port="${jboss.jgroups.diagnostics_port:7500}"
thread_pool.enabled="true"
thread_pool.min_threads="8"
thread_pool.max_threads="200"
thread_pool.keep_alive_time="5000"
thread_pool.queue_enabled="true"
thread_pool.queue_max_size="1000"
thread_pool.rejection_policy="discard"
oob_thread_pool.enabled="true"
oob_thread_pool.min_threads="8"
oob_thread_pool.max_threads="200"
oob_thread_pool.keep_alive_time="1000"
oob_thread_pool.queue_enabled="false"
oob_thread_pool.rejection_policy="discard"/>]]>
This JGroups configuration has a number of attributes available. First we look at the attributes
available to the UDP protocol, followed by the attributes that are also used by the TCP and TUNNEL
transport protocols.
• ip_mcast specifies whether or not to use IP multicasting. The default is true. If set to false,
multiple unicast packets will be sent instead of one multicast packet. Any packet sent via UDP
protocol are UDP datagrams.
• mcast_addr specifies the multicast address (class D) for communicating with the group (i.e., the
cluster). The standard protocol stack configurations in JBoss AS use the value of system property
jboss.partition.udpGroup, if set, as the value for this attribute. Using the -u command
line switch when starting JBoss Application Server sets that value. See Section 22.6.2, “Isolating
JGroups Channels” for information about using this configuration attribute to ensure that JGroups
221
Chapter 22. JGroups Services
channels are properly isolated from one another. If this attribute is omitted, the default value is
228.8.8.8.
• mcast_port specifies the port to use for multicast communication with the group. See
Section 22.6.2, “Isolating JGroups Channels” for how to use this configuration attribute to ensure
JGroups channels are properly isolated from one another. If this attribute is omitted, the default is
45566.
• bind_port specifies the port to which the unicast receive socket should be bound. The default is 0;
i.e. use an ephemeral port.
• port_range specifies the number of ports to try if the port identified by bind_port is not available.
The default is 1, which specifies that only bind_port will be tried.
• ip_ttl specifies time-to-live (TTL) for IP Multicast packets. TTL is the commonly used term in
multicast networking, but is actually something of a misnomer, since the value here refers to how
many network hops a packet will be allowed to travel before networking equipment will drop it.
• tos specifies the traffic class for sending unicast and multicast datagrams.
The attributes that are common to all transport protocols, and thus have the same meanings when
used with TCP or TUNNEL, are:
• singleton_name provides a unique name for this transport protocol configuration. Used by the
application server's ChannelFactory to support sharing of a transport protocol instance by
different channels that use the same transport protocol configuration. See Section 15.1.2, “The
JGroups Shared Transport”.
• bind_addr specifies the interface on which to receive and send messages. By default, JGroups
uses the value of system property jgroups.bind_addr. This can also be set with the -b
command line switch. See Section 22.6, “Other Configuration Issues” for more on binding JGroups
sockets.
• receive_on_all_interfaces specifies whether this node should listen on all interfaces for multicasts.
The default is false. It overrides the bind_addr property for receiving multicasts. However,
bind_addr (if set) is still used to send multicasts.
• send_on_all_interfaces specifies whether this node sends UDP packets via all available network
interface controllers, if your machine has multiple network interface controllers available. This
means that the same multicast message is sent N times, so use with care.
• send_interfaces specifies a list of of interfaces via which to send multicasts. The multicast sender
socket will send on all of these interfaces. This is a comma-separated list of IP addresses or
222
Transport Protocols
Message bundling can have significant performance benefits for channels that are used for
high volume sending of messages where the sender does not block waiting for a response
from recipients (for example, a JBoss Cache instance configured for REPL_ASYNC.) It can add
considerable latency to applications where senders need to block waiting for responses, so it is
not recommended for certain situations, such as where a JBoss Cache instance is configured for
REPL_SYNC.
• loopback specifies whether the thread sending a message to the group should itself carry the
message back up the stack for delivery. (Messages sent to the group are always delivered to the
sending node as well.) If false, the sending thread does not carry the message; the transport
protocol waits to read the message off the network and uses one of the message delivery pool
threads for delivery. The default is false, but true is recommended to ensure that the channel
receives its own messages, in case the network interface goes down.
• enable_diagnostics specifies that the transport should open a multicast socket on address
diagnostics_addr and port diagnostics_port to listen for diagnostic requests sent by the
1
JGroups Probe utility .
• The various thread_pool attributes configure the behavior of the pool of threads JGroups uses
to carry ordinary incoming messages up the stack. The various attributes provide the constructor
arguments for an instance of java.util.concurrent.ThreadPoolExecutorService. In the
example above, the pool will have a minimum or core size of 8 threads, and a maximum size of 200.
If more than 8 pool threads have been created, a thread returning from carrying a message will wait
for up to 5000 milliseconds to be assigned a new message to carry, after which it will terminate. If no
threads are available to carry a message, the (separate) thread reading messages off the socket will
place messages in a queue; the queue will hold up to 1000 messages. If the queue is full, the thread
reading messages off the socket will discard the message.
• The various oob_thread_pool attributes are similar to the thread_pool attributes in that they
configure a java.util.concurrent.ThreadPoolExecutorService used to carry incoming
messages up the protocol stack. In this case, the pool is used to carry a special type of message
known as an Out-Of-Band (OOB) message. OOB messages are exempt from the ordered-delivery
requirements of protocols like NAKACK and UNICAST and thus can be delivered up the stack even
if NAKACK or UNICAST are queueing up messages from a particular sender. OOB messages are
often used internally by JGroups protocols and can be used by applications as well. For example,
when JBoss Cache is in REPL_SYNC mode, it uses OOB messages for the second phase of its two-
phase-commit protocol.
223
Chapter 22. JGroups Services
<TCP singleton_name="tcp"
start_port="7800" end_port="7800"/>
• start_port and end_port define the range of TCP ports to which the server should bind. The
server socket is bound to the first available port beginning with start_port. If no available port
is found (for example, because the ports are in use by other sockets) before the end_port, the
server throws an exception. If no end_port is provided, or end_port is lower than start_port,
no upper limit is applied to the port range. If start_port is equal to end_port, JGroups is forced
to use the specified port, since start_port fails if the specified port in not available. The default
value is 7800. If set to 0, the operating system will select a port. (This will only work for MPING or
TCPGOSSIP discovery protocols. TCCPING requires that nodes and their required ports are listed.)
• bind_port in TCP acts as an alias for start_port. If configured internally, it sets start_port.
• recv_buf_size, send_buf_size define receive and send buffer sizes. It is good to have a large
receiver buffer size, so packets are less likely to get dropped due to buffer overflow.
• conn_expire_time specifies the time (in milliseconds) after which a connection can be closed by
the reaper if no traffic has been received.
• reaper_interval specifies interval (in milliseconds) to run the reaper. If both values are 0, no reaping
will be done. If either value is > 0, reaping will be enabled. By default, reaper_interval is 0, which
means no reaper.
• sock_conn_timeout specifies max time in millis for a socket creation. When doing the initial
discovery, and a peer hangs, don't wait forever but go on after the timeout to ping other members.
Reduces chances of *not* finding any members at all. The default is 2000.
• use_send_queues specifies whether to use separate send queues for each connection. This
prevents blocking on write if the peer hangs. The default is true.
• external_addr specifies external IP address to broadcast to other group members (if different to
local address). This is useful when you have use (Network Address Translation) NAT, e.g. a node
on a private network, behind a firewall, but you can only route to it via an externally visible address,
which is different from the local address it is bound to. Therefore, the node can be configured to
broadcast its external address, while still able to bind to the local one. This avoids having to use the
TUNNEL protocol, (and hence a requirement for a central gossip router) because nodes outside the
firewall can still route to the node inside the firewall, but only on its external address. Without setting
the external_addr, the node behind the firewall will broadcast its private address to the other nodes
which will not be able to route to it.
224
Discovery Protocols
• tcp_nodelay specifies TCP_NODELAY. TCP by default nagles messages, that is, conceptually,
smaller messages are bundled into larger ones. If we want to invoke synchronous cluster method
calls, then we need to disable nagling in addition to disabling message bundling (by setting
enable_bundling to false). Nagling is disabled by setting tcp_nodelay to true. The default is
false.
Note
All of the attributes common to all protocols discussed in the UDP protocol section also
apply to TCP.
<TUNNEL singleton_name="tunnel"
router_port="12001"
router_host="192.168.5.1"/>
• reconnect_interval specifies the interval of time (in milliseconds) for which TUNNEL will attempt to
connect to the GossipRouter if the connection is not established. The default value is 5000.
Note
All of the attributes common to all protocols discussed in the UDP protocol section also
apply to TUNNEL.
225
Chapter 22. JGroups Services
Discovery protocols also assist merge protocols (see Section 22.5, “Merging (MERGE2)”) to detect
cluster-split situations.
The discovery protocols sit on top of the transport protocol, so you can choose to use different
discovery protocols depending on your transport protocol. These are also configured as sub-elements
in the JGroups <config> element.
22.1.3.1. PING
PING is a discovery protocol that works by either multicasting PING requests to an IP multicast
address or connecting to a gossip router. As such, PING normally sits on top of the UDP or TUNNEL
transport protocols. Each node responds with a packet {C, A}, where C=coordinator's address and
A=own address. After timeout milliseconds or num_initial_members replies, the joiner determines the
coordinator from the responses, and sends a JOIN request to it (handled by). If nobody responds, we
assume we are the first member of a group.
<PING timeout="2000"
num_initial_members="3"/>
<PING gossip_host="localhost"
gossip_port="1234"
timeout="2000"
num_initial_members="3"/>
• timeout specifies the maximum number of milliseconds to wait for any responses. The default is
3000.
• num_initial_members specifies the maximum number of responses to wait for unless timeout has
expired. The default is 2.
• gossip_refresh specifies the interval (in milliseconds) for the lease from the GossipRouter. The
default is 20000.
If both gossip_host and gossip_port are defined, the cluster uses the GossipRouter for the
initial discovery. If the initial_hosts is specified, the cluster pings that static list of addresses for
discovery. Otherwise, the cluster uses IP multicasting for discovery.
226
Discovery Protocols
Note
The discovery phase returns when the timeout ms have elapsed or the
num_initial_members responses have been received.
22.1.3.2. TCPGOSSIP
The TCPGOSSIP protocol only works with a GossipRouter. It works essentially the same way as the
PING protocol configuration with valid gossip_host and gossip_port attributes. It works on top of
both UDP and TCP transport protocols. Here is an example.
<TCPGOSSIP timeout="2000"
num_initial_members="3"
initial_hosts="192.168.5.1[12000],192.168.0.2[12000]"/>
• timeout specifies the maximum number of milliseconds to wait for any responses. The default is
3000.
• num_initial_members specifies the maximum number of responses to wait for unless timeout has
expired. The default is 2.
22.1.3.3. TCPPING
The TCPPING protocol takes a set of known members and pings them for discovery. This is
essentially a static configuration. It works on top of TCP. Here is an example of the TCPPING
configuration element in the JGroups config element.
<TCPPING timeout="2000"
num_initial_members="3"/
initial_hosts="hosta[2300],hostb[3400],hostc[4500]"
port_range="3">
• timeout specifies the maximum number of milliseconds to wait for any responses. The default is
3000.
• num_initial_members specifies the maximum number of responses to wait for unless timeout has
expired. The default is 2.
• port_range specifies the number of consecutive ports to be probed when getting the initial
membership, starting with the port specified in the initial_hosts parameter. Given the current
values of port_range and initial_hosts above, the TCPPING layer will try to connect to
227
Chapter 22. JGroups Services
22.1.3.4. MPING
MPING uses IP multicast to discover the initial membership. Unlike the other discovery protocols,
which delegate the sending and receiving of discovery messages on the network to the transport
protocol, MPING opens its own sockets to send and receive multicast discovery messages. As a result
it can be used with all transports, but it is most often used with TCP. TCP usually requires TCPPING,
which must explicitly list all possible group members. MPING does not have this requirement, and is
typically used where TCP is required for regular message transport, and UDP multicasting is allowed
for discovery.
<MPING timeout="2000"
num_initial_members="3"
bind_to_all_interfaces="true"
mcast_addr="228.8.8.8"
mcast_port="7500"
ip_ttl="8"/>
• timeout specifies the maximum number of milliseconds to wait for any responses. The default is
3000.
• num_initial_members specifies the maximum number of responses to wait for unless timeout has
expired. The default is 2..
• bind_addr specifies the interface on which to send and receive multicast packets. By default
JGroups uses the value of the system property jgroups.bind_addr, which can be set with the -b
command line switch. See Section 22.6, “Other Configuration Issues” for more on binding JGroups
sockets.
• bind_to_all_interfaces overrides the bind_addr and uses all interfaces in multihome nodes.
• mcast_addr, mcast_port, ip_ttl attributes are the same as related attributes in the UDP protocol
configuration.
228
Failure Detection Protocols
22.1.4.1. FD
FD is a failure detection protocol based on 'heartbeat' messages. This protocol requires that eat node
periodically ping its neighbour. If the neighbour fails to respond, the calling node sends a SUSPECT
message to the cluster. The current group coordinator can optionally verify that the suspected node
is dead (VERIFY_SUSPECT). If the node is still considered dead after this verification step, the
coordinator updates the cluster's membership view. The following is an example of FD configuration:
<FD timeout="6000"
max_tries="5"
shun="true"/>
• timeout specifies the maximum number of milliseconds to wait for the responses to the are-you-
alive messages. The default is 3000.
• max_tries specifies the number of missed are-you-alive messages from a node before the node is
suspected. The default is 2.
• shun specifies whether a failed node will be forbidden from sending messages to the group without
formally rejoining. A shunned node would need to rejoin the cluster via the discovery process.
JGroups allows applications to configure a channel such that, when a channel is shunned, the
process of rejoining the cluster and transferring state. (This is default behavior for JBoss Application
Server.)
Note
Regular traffic from a node is proof of life, so heartbeat messages are only sent when no
regular traffic is detected on the node for a long period of time.
22.1.4.2. FD_SOCK
FD_SOCK is a failure detection protocol based on a ring of TCP sockets created between group
members. Each member in a group connects to its neighbor, with the final member connecting to the
first, forming a ring. Node B becomes suspected when its neighbour, Node A, detects an abnormally
closed TCP socket, presumably due to a crash in Node B. (When nodes intend to leave the group,
they inform their neighbours so that they do not become suspected.)
The simplest FD_SOCK configuration does not take any attribute. You can declare an empty FD_SOCK
element in the JGroups <config> element.
<FD_SOCK/>
• bind_addr specifies the interface to which the server socket should be bound. By default, JGroups
uses the value of the system property jgroups.bind_addr. This system property can be set
with the -b command line switch. For more information about binding JGroups sockets, see
Section 22.6, “Other Configuration Issues”.
229
Chapter 22. JGroups Services
22.1.4.3. VERIFY_SUSPECT
This protocol verifies whether a suspected member is really dead by pinging that member once again.
This verification is performed by the coordinator of the cluster. The suspected member is dropped
from the cluster group if confirmed to be dead. The aim of this protocol is to minimize false suspicions.
Here's an example.
<VERIFY_SUSPECT timeout="1500"/>
• timeout specifies how long to wait for a response from the suspected member before considering it
dead.
• FD
• Low timeouts lead to higher probability of false suspicions and higher network traffic.
• High timeouts will not detect and remove crashed members for some time.
• FD_SOCK:
• Also, a crashed switch will not be detected until the connection runs into the TCP timeout
(between 2-20 minutes, depending on TCP/IP stack implementation).
A failure detection layer is intended to report real failures promptly, while avoiding false suspicions.
There are two solutions:
1. By default, JGroups configures the FD_SOCK socket with KEEP_ALIVE, which means that TCP
sends a heartbeat on socket on which no traffic has been received in 2 hours. If a host crashed
(or an intermediate switch or router crashed) without closing the TCP connection properly, we
would detect this after 2 hours (plus a few minutes). This is of course better than never closing the
connection (if KEEP_ALIVE is off), but may not be of much help. So, the first solution would be
to lower the timeout value for KEEP_ALIVE. This can only be done for the entire kernel in most
operating systems, so if this is lowered to 15 minutes, this will affect all TCP sockets.
230
Reliable Delivery Protocols
2. The second solution is to combine FD_SOCK and FD; the timeout in FD can be set such that it is
much lower than the TCP timeout, and this can be configured individually per process. FD_SOCK
will already generate a suspect message if the socket was closed abnormally. However, in the
case of a crashed switch or host, FD will make sure the socket is eventually closed and the
suspect message generated. Example:
<FD_SOCK/>
<FD timeout="6000" max_tries="5" shun="true"/>
<VERIFY_SUSPECT timeout="1500"/>
In this example, a member becomes suspected when the neighbouring socket has been closed
abnormally, in a process crash, for instance, since the operating system closes all sockets. However,
if a host or switch crashes, the sockets will not be closed. FD will suspect the neighbour after thirty
seconds (6000 milliseconds). Note that if this example system were stopped in a breakpoint in the
debugger, the node being debugged will be suspected once the timeout has elapsed.
A combination of FD and FD_SOCK provides a solid failure detection layer, which is why this technique
is used across the JGroups configurations included with JBoss Application Server.
22.1.5.1. UNICAST
The UNICAST protocol is used for unicast messages. It uses positive acknowlegements (ACK). It is
configured as a sub-element under the JGroups config element. If the JGroups stack is configured
with the TCP transport protocol, UNICAST is not necessary because TCP itself guarantees FIFO
delivery of unicast messages. Here is an example configuration for the UNICAST protocol:
<UNICAST timeout="300,600,1200,2400,3600"/>
• timeout specifies the retransmission timeout (in milliseconds). For instance, if the timeout is
100,200,400,800, the sender resends the message if it has not received an ACK after 100
milliseconds the first time, and the second time it waits for 200 milliseconds before resending, and
so on. A low value for the first timeout allows for prompt retransmission of dropped messages, but
means that messages may be transmitted more than once if they have not actually been lost (that
is, the message has been sent, but the ACK has not been received before the timeout). High values
(1000,2000,3000) can improve performance if the network is tuned such that UDP datagram loss
is infrequent. High values on networks with frequent losses will be harmful to performance, since
later messages will not be delivered until lost messages have been retransmitted.
231
Chapter 22. JGroups Services
22.1.5.2. NAKACK
The NAKACK protocol is used for multicast messages. It uses negative acknowlegements (NAK).
Under this protocol, each message is tagged with a sequence number. The receiver keeps track of
the received sequence numbers and delivers the messages in order. When a gap in the series of
received sequence numbers is detected, the receiver schedules a task to periodically ask the sender
to retransmit the missing message. The task is cancelled if the missing message is received. NAKACK
protocol is configured as the pbcast.NAKACK sub-element under the JGroups <config> element.
Here is an example configuration:
• retransmit_timeout specifies the series of timeouts (in milliseconds) after which retransmission is
requested if a missing message has not yet been received.
• use_mcast_xmit determines whether the sender should send the retransmission to the entire
cluster rather than just to the node requesting it. This is useful when the sender's network layer
tends to drop packets, avoiding the need to individually retransmit to each node.
• max_xmit_size specifies the maximum size (in bytes) for a bundled retransmission, if multiple
messages are reported missing.
• gc_lag specifies the number of messages to keep in memory for retransmission, even after the
periodic cleanup protocol (see Section 22.4, “Distributed Garbage Collection (STABLE)”) indicates
all peers have received the message. The default value is 20.
<pbcast.GMS print_local_addr="true"
join_timeout="3000"
join_retry_timeout="2000"
shun="true"
view_bundling="true"/>
232
Flow Control (FC)
• join_timeout specifies the maximum number of milliseconds to wait for a new node JOIN request to
succeed. Retry afterwards.
• join_retry_timeout specifies the number of milliseconds to wait after a failed JOIN before trying
again.
• print_local_addr specifies whether to dump the node's own address to the standard output when
started.
• shun specifies whether a node should shun (that is, disconnect) itself if it receives a cluster view in
which it is not a member node.
• disable_initial_coord specifies whether to prevent this node from becoming the cluster coordinator
during the initial connection of the channel. This flag does not prevent a node becoming the
coordinator after the initial channel connection, if the current coordinator leaves the group.
• view_bundling specifies whether multiple JOIN or LEAVE requests arriving at the same time are
bundled and handled together at the same time, resulting in only one new view that incorporates all
changes. This is is more efficient than handling each request separately.
<FC max_credits="2000000"
min_threshold="0.10"
ignore_synchronous_response="true"/>
• max_credits specifies the maximum number of credits (in bytes). This value should be smaller than
the JVM heap size.
• min_credits specifies the minimum number of bytes that must be received before the receiver will
send more credits to the sender.
• min_threshold specifies the percentage of the max_credits that should be used to calculate
min_credits. Setting this overrides the min_credits attribute.
233
Chapter 22. JGroups Services
Why is FC needed on top of TCP ? TCP has its own flow control!
FC is required for group communication where group messages must be sent at the
highest speed that the slowest receiver can handle. For example, say we have a cluster
comprised of nodes A, B, C and D. D is slow (perhaps overloaded), while the rest are fast.
When A sends a group message, it does so via TCP connections: A-A (theoretically), A-B,
A-C and A-D.
Say A sends 100 million messages to the cluster. TCP's flow control applies to A-B, A-
C and A-D individually, but not to A-BCD as a group. Therefore, A, B and C will receive
the 100 million messages, but D will receive only 1 million. (This is also why NAKACK is
required, even though TCP handles its own retransmission.)
JGroups must buffer all messages in memory in case an original sender S dies and
a node requests retransmission of a message sent by S. Since all members buffer all
messages that they receive, stable messages (messages seen by every node) must
sometimes be purged. (The purging process is managed by the STABLE protocol. For
more information, see Section 22.4, “Distributed Garbage Collection (STABLE)”.)
In the above case, the slow node D will prevent the group from purging messages above
1M, so every member will buffer 99M messages ! This in most cases leads to OOM
exceptions. Note that - although the sliding window protocol in TCP will cause writes to
block if the window is full - we assume in the above case that this is still much faster for A-
B and A-C than for A-D.
So, in summary, even with TCP we need to FC to ensure we send messages at a rate the
slowest receiver (D) can handle.
A good example of such an application is one that uses JGroups to make synchronous
group RPC calls. By synchronous, we mean the thread that makes the call blocks waiting
for responses from all the members of the group. In that kind of application, the threads on
A that are making calls would block waiting for responses from D, thus naturally slowing
the overall rate of calls.
And, of course, if your cluster only consists of two nodes, including FC in a TCP-
based protocol stack is unnecessary. There is no group beyond the single peer-to-peer
relationship, and TCP's internal flow control will handle that just fine.
Another case where FC may not be needed is for a channel used by a JBoss Cache
configured for buddy replication and a single buddy. Such a channel will in many respects
act like a two node cluster, where messages are only exchanged with one other node, the
buddy. (There may be other messages related to data gravitation that go to all members,
234
Fragmentation (FRAG2)
but in a properly engineered buddy replication use case these should be infrequent. But if
you remove FC be sure to load test your application.)
<FRAG2 frag_size="60000"/>
• frag_size specifies the maximum message size (in bytes) before fragmentation occurs. Messages
larger than this size are fragmented. For stacks that use the UDP transport, this value must be lower
than 64 kilobytes (the maximum UDP datagram size). For TCP-based stacks, it must be lower than
the value of max_credits in the FC protocol.
Note
TCP protocol already provides fragmentation, but a JGroups fragmentation protocol is
still required if FC is used. The reason for this is that if you send a message larger than
FC.max_credits, the FC protocol will block forever. So, frag_size within FRAG2
must always be set to a value lower than that of FC.max_credits.
<pbcast.STABLE stability_delay="1000"
desired_avg_gossip="5000"
max_bytes="400000"/>
235
Chapter 22. JGroups Services
• desired_avg_gossip specifies intervals (in milliseconds) of garbage collection runs. Set this to 0 to
disable interval-based garbage collection.
• max_bytes specifies the maximum number of bytes received before the cluster triggers a garbage
collection run. Set to 0 to disable garbage collection based on the bytes received.
• stability_delay specifies the maximum time period (in milliseconds) of a random delay introduced
before a node sends its STABILITY message at the end of a garbage collection run. The delay
gives other nodes concurrently running a STABLE task a change to send first. If used together with
max_bytes, this attribute should be set to a small number.
Note
Set the max_bytes attribute when you have a high traffic cluster.
<MERGE2 max_interval="10000"
min_interval="2000"
down_thread="false" up_thread="false"/>
• max_interval specifies the maximum number of milliseconds to wait before sending a MERGE
message.
• min_interval specifies the minimum number of milliseconds to wait before sending a MERGE
message.
JGroups chooses a random value between min_interval and max_interval to periodically send
the MERGE message.
Note
The application state maintained by the application using a channel is not merged by
JGroups during a merge. This must be done by the application.
Note
If MERGE2 is used in conjunction with TCPPING, the initial_hosts attribute must
contain all the nodes that could potentially be merged back, in order for the merge process
236
Other Configuration Issues
to work properly. Otherwise, the merge process may not detect all sub-groups, and may
miss those comprised solely of unlisted members.
First, it is important to understand that the value set in any bind_addr element in an XML
configuration file will be ignored by JGroups if it finds that the system property jgroups.bind_addr
(or a deprecated earlier name for the same thing, bind.address) has been set. The system property
has a higher priority level than the XML property. If JBoss Application Server is started with the -b (or
--host) switch, the application server will set jgroups.bind_addr to the specified value. If -b is
not set, the application server will bind most services to localhost by default.
So, what are best practices for managing how JGroups binds to interfaces?
• Binding JGroups to the same interface as other services. Simple, just use -b:
• Binding services (e.g., JBoss Web) to one interface, but use a different one for JGroups:
Specifically setting the system property overrides the -b value. This is a common usage pattern; put
client traffic on one network, with intra-cluster traffic on another.
• Binding services (e.g., JBoss Web) to all interfaces. This can be done like this:
However, doing this will not cause JGroups to bind to all interfaces! Instead , JGroups will bind
to the machine's default interface. See the Transport Protocols section for how to tell JGroups to
receive or send on all interfaces, if that is what you really want.
• Binding services (e.g., JBoss Web) to all interfaces, but specify the JGroups interface:
237
Chapter 22. JGroups Services
This setting tells JGroups to ignore the jgroups.bind_addr system property, and instead use
whatever is specfied in XML. You would need to edit the various XML configuration files to set the
various bind_addr attributes to the desired interfaces.
It is critical that these channels only communicate with their intended peers; not with the channels
used by other services and not with channels for the same service opened on machines not meant
to be part of the group. Nodes improperly communicating with each other is one of the most common
issues users have with JBoss Enterprise Application Platform clustering.
Whom a JGroups channel will communicate with is defined by its group name and, for UDP-based
channels, its multicast address and port. Isolating a JGroups channel means ensuring that different
channels use different values for the group name, the multicast address and, in some cases, the
multicast port.
To isolate JGroups clusters from other clusters on the network, you must:
• Make sure the channels in the various clusters use different group names. This can be controlled
with the command line arguments used to start JBoss; see Section 22.6.2.2.1, “Changing the Group
Name” for more information.
• Make sure the channels in the various clusters use different multicast addresses. This is also easy
to control with the command line arguments used to start JBoss.
• If you are not running on Linux, Windows, Solaris or HP-UX, you may also need to ensure that
the channels in each cluster use different multicast ports. This is more difficult than using different
group names, although it can still be controlled from the command line. See Section 22.6.2.2.3,
“Changing the Multicast Port”. Note that using different ports should not be necessary if your servers
are running on Linux, Windows, Solaris or HP-UX.
To isolate channels for different services from each other on the same set of application server
instances, each channel must have its own group name. The configurations that ship with JBoss
238
Isolating JGroups Channels
Application Server ensure that this is the case. However, if you create a custom service that
uses JGroups directly, you must use a unique group name. If you create a custom JBoss Cache
configuration, ensure that you provide a unique value in the clusterName configuration property.
In releases prior to JBoss Application Server 5, different channels running in the same application
server also had to use unique multicast ports. With the JGroups shared transport introduced in JBoss
AS 5 (see Section 15.1.2, “The JGroups Shared Transport”), it is now common for multiple channels
to use the same tranpsort protocol and its sockets. This makes configuration easier, which is one of
the main benefits of the shared transport. However, if you decide to create your own custom JGroups
protocol stack configuration, be sure to configure its transport protocols with a multicast port that is
different from the ports used in other protocol stacks.
This switch sets the jboss.partition.name system property, which is used as a component in the
configuration of the group name in all the standard clustering configuration files. For example,
<property name="clusterName">${jboss.partition.name:DefaultPartition}-
SFSBCache</property>
This switch sets the jboss.partition.udpGroup system property, which is referenced in all of the
standard protocol stack configurations in JBoss AS:
239
Chapter 22. JGroups Services
AS instance running the all configuration will use up to two different instances of the JGroups UDP
transport protocol, and will therefore open two multicast sockets. You can control the ports those
sockets use by using system properties on the command line. For example,
One of the most common causes of lost UDP datagrams is an undersized receive buffer on the
socket. The UDP protocol's mcast_recv_buf_size and ucast_recv_buf_size configuration
240
JGroups Troubleshooting
attributes are used to specify the amount of receive buffer JGroups requests from the operating
system, but the actual size of the buffer the operating system provides is limited by operating system-
level maximums. These maximums are often very low:
The command used to increase the above limits is operating system-specific. The table below shows
the command required to increase the maximum buffer to 25 megabytes. In all cases, root privileges
are required:
If you want to bind to a specific network interface card (NIC), use -bind_addr 192.168.0.2, where
192.168.0.2 is the IP address of the NIC to which you want to bind. Use this parameter in both the
sender and the receiver.
You should be able to type in the McastSenderTest window and see the output in the
McastReceiverTest window. If not, try to use -ttl 32 in the sender. If this still fails, consult a system
241
Chapter 22. JGroups Services
administrator to help you setup IP multicast correctly, and ask the admin to make sure that multicast
will work on the interface you have chosen or, if the machines have multiple interfaces, ask to be told
the correct interface. Once you know multicast is working properly on each machine in your cluster,
you can repeat the above test to test the network, putting the sender on one machine and the receiver
on another.
• B or C are running at 100% CPU for more than T seconds. So even if C sends a heartbeat ack to B,
B may not be able to process it because it is at 100%
• The network loses packets. This usually happens when there is a lot of traffic on the network, and
the switch starts dropping packets (usually broadcasts first, then IP multicasts, TCP packets last).
• B or C are processing a callback. Let's say C received a remote method call over its channel and
takes T+1 seconds to process it. During this time, C will not process any other messages, including
heartbeats, and therefore B will not receive the heartbeat ack and will suspect C.
242
Chapter 23.
Users considering deploying JBoss Cache for direct use by their own application are strongly
encouraged to read the JBoss Cache documentation available at http://www.jboss.org/jbosscache.
See also Section 15.2, “Distributed Caching with JBoss Cache” for information on how the standard
JBoss Enterprise Application Platform clustered services use JBoss Cache.
Most JBoss Cache configuration examples in this section use the JBoss Microcontainer schema
for building up an org.jboss.cache.config.Configuration object graph from XML. JBoss
Cache has its own custom XML schema, but the standard JBoss Enterprise Application Platform
CacheManager service uses the JBoss Microcontainer schema to be consistent with most other
internal Enterprise Application Platform services.
Before getting into the key configuration options, let's have a look at the most likely place that a user
would encounter them.
Note
Users can also use the CacheManager as a factory for custom caches used by directly by
their own applications; see Section 23.2.1, “Deployment Via the CacheManager Service”.
243
Chapter 23. JBoss Cache Configuration and Deployment
<bean name="CacheConfigurationRegistry"
class="org.jboss.ha.cachemanager.DependencyInjectedConfigurationRegistry">
<!-- If users wish to add configs using a more familiar JBC config
format
they can add them to a cache-configs.xml file specified by this
property.
However, use of the microcontainer format used below is
recommended.
<property name="configResource">META-INF/jboss-cache-configs.xml</
property>
-->
<!-- The standard configurations follow. You can add your own and/or
edit these. -->
<!-- Name of cluster. Needs to be the same for all members -->
<property name="clusterName">
${jboss.partition.name:DefaultPartition}-SessionCache</property>
<!-- Use a UDP (multicast) based stack. Need JGroups flow control
(FC)
because we are using asynchronous replication. -->
244
Editing the CacheManager Configuration
<property name="multiplexerStack">
${jboss.default.jgroups.stack:udp}</property>
<property name="fetchInMemoryState">true</property>
<property name="nodeLockingScheme">PESSIMISTIC</property>
<property name="isolationLevel">REPEATABLE_READ</property>
<property name="cacheMode">REPL_ASYNC</property>
<bean name="FieldSessionCacheConfig"
class="org.jboss.cache.config.Configuration">
.... details of the field-granularity-standard-session-cache
configuration
</bean>
</value>
</entry>
</map>
</property>
</bean>
The actual JBoss Cache configurations are specified using the JBoss Microcontainer's
schema rather than one of the standard JBoss Cache configuration formats. When JBoss
Cache parses one of its standard configuration formats, it creates a Java Bean of type
org.jboss.cache.config.Configuration with a tree of child Java Beans for some of the more
complex sub-configurations (i.e. cache loading, eviction, buddy replication). Rather than delegating
this task of XML parsing/Java Bean creation to JBC, we let the Enterprise Application Platform's
microcontainer do it directly. This has the advantage of making the microcontainer aware of the
configuration beans, which in later Enterprise Application Platform 5.x releases will be helpful in
allowing external management tools to manage the JBC configurations.
The configuration format should be fairly self-explanatory if you look at the standard configurations the
Enterprise Application Platform ships; they include all the major elements. The types and properties
of the various java beans that make up a JBoss Cache configuration can be seen in the JBoss Cache
javadocs. Here is a fairly complete example:
<bean name="StandardSFSBCacheConfig"
class="org.jboss.cache.config.Configuration">
245
Chapter 23. JBoss Cache Configuration and Deployment
<!-- Name of cluster. Needs to be the same for all members -->
<property name="clusterName">${jboss.partition.name:DefaultPartition}-
SFSBCache</property>
<!-- Use a UDP (multicast) based stack. Need JGroups flow control (FC)
because we are using asynchronous replication. -->
<property name="multiplexerStack">${jboss.default.jgroups.stack:udp}</
property>
<property name="fetchInMemoryState">true</property>
<property name="nodeLockingScheme">PESSIMISTIC</property>
<property name="isolationLevel">REPEATABLE_READ</property>
<property name="cacheMode">REPL_ASYNC</property>
<property name="useLockStriping">false</property>
<!--
SFSBs use region-based marshalling to provide for partial state
transfer during deployment/undeployment.
-->
<property name="useRegionBasedMarshalling">false</property>
<!-- Must match the value of "useRegionBasedMarshalling" -->
<property name="inactiveOnStartup">false</property>
<property name="exposeManagementStatistics">true</property>
<property name="buddyReplicationConfig">
<bean class="org.jboss.cache.config.BuddyReplicationConfig">
246
Editing the CacheManager Configuration
and pick a buddy who shares the same pool name (falling
back to other buddies if not available). -->
<property name="buddyPoolName">default</property>
<property name="buddyCommunicationTimeout">17500</property>
<property name="buddyLocatorConfig">
<bean
class="org.jboss.cache.buddyreplication.NextMemberBuddyLocatorConfig">
<!-- The number of backup nodes we maintain -->
<property name="numBuddies">1</property>
<!-- Means that each node will *try* to select a buddy on
a different physical host. If not able to do so
though, it will fall back to colocated nodes. -->
<property name="ignoreColocatedBuddies">true</property>
</bean>
</property>
</bean>
</property>
<property name="cacheLoaderConfig">
<bean class="org.jboss.cache.config.CacheLoaderConfig">
<!-- Do not change these -->
<property name="passivation">true</property>
<property name="shared">false</property>
<property name="individualCacheLoaderConfigs">
<list>
<bean
class="org.jboss.cache.loader.FileCacheLoaderConfig">
<!-- Where passivated sessions are stored -->
<property name="location">
${jboss.server.data.dir}${/}sfsb</property>
<!-- Do not change these -->
<property name="async">false</property>
<property name="fetchPersistentState">true</property>
<property name="purgeOnStartup">true</property>
<property name="ignoreModifications">false</property>
<property name="checkCharacterPortability">false</
property>
</bean>
</list>
</property>
</bean>
</property>
247
Chapter 23. JBoss Cache Configuration and Deployment
<property name="evictionConfig">
<bean class="org.jboss.cache.config.EvictionConfig">
<property name="wakeupInterval">5000</property>
<!-- Overall default -->
<property name="defaultEvictionRegionConfig">
<bean class="org.jboss.cache.config.EvictionRegionConfig">
<property name="regionName">/</property>
<property name="evictionAlgorithmConfig">
<bean
class="org.jboss.cache.eviction.NullEvictionAlgorithmConfig"/>
</property>
</bean>
</property>
<!-- EJB3 integration code will programatically create
other regions as beans are deployed -->
</bean>
</property>
</bean>
This controls how a cache instance on one node should notify the rest of the cluster when it makes
changes in its local state. There are three options:
• Synchronous means the cache instance sends a message to its peers notifying them of the
change(s) and before returning waits for them to acknowledge that they have applied the same
changes. If the changes are made as part of a JTA transaction, this is done as part of a 2 phase-
commit process during transaction commit. Any locks are held until this acknowledgment is
received. Waiting for acknowledgement from all nodes adds delays, but it ensures consistency
around the cluster. Synchronous mode is needed when all the nodes in the cluster may access the
cached data resulting in a high need for consistency.
• Asynchronous means the cache instance sends a message to its peers notifying them of the
change(s) and then immediately returns, without any acknowledgement that they have applied the
same changes. It does not mean sending the message is handled by some other thread besides
the one that changed the cache content; the thread that makes the change still spends some time
dealing with sending messages to the cluster, just not as much as with synchronous communication.
Asynchronous mode is most useful for cases like session replication, where the cache doing the
sending expects to be the only one that accesses the data and the cluster messages are used to
provide backup copies in case of failure of the sending node. Asynchronous messaging adds a
small risk that a later user request that fails over to another node may see out-of-date state, but
248
Transaction Handling
for many session-type applications this risk is acceptable given the major performance benefits
asynchronous mode has over synchronous mode.
• Local means the cache instance doesn't send a message at all. A JGroups channel isn't even used
by the cache. JBoss Cache has many useful features besides its clustering capabilities and is a
very useful caching library even when not used in a cluster. Also, even in a cluster, some cached
data does not need to be kept consistent around the cluster, in which case Local mode will improve
performance. Caching of JPA/Hibernate query result sets is an example of this; Hibernate's second
level caching logic uses a separate mechanism to invalidate stale query result sets from the second
level cache, so JBoss Cache doesn't need to send messages around the cluster for a query result
set cache.
This aspect deals with the content of messages sent around the cluster when a cache changes its
local state, i.e. what should the other caches in the cluster do to reflect the change:
• Replication means the other nodes should update their state to reflect the new state on the sending
node. This means the sending node needs to include the changed state, increasing the cost of the
message. Replication is necessary if the other nodes have no other way to obtain the state.
• Invalidation means the other nodes should remove the changed state from their local state.
Invalidation reduces the cost of the cluster update messages, since only the cache key of the
changed state needs to be transmitted, not the state itself. However, it is only an option if the
removed state can be retrieved from another source. It is an excellent option for a clustered JPA/
Hibernate entity cache, since the cached state can be re-read from the database.
These two aspects combine to form 5 valid values for the cacheMode configuration attribute:
• LOCAL means no cluster messages are needed.
• org.jboss.cache.transaction.JBossTransactionManagerLookup
This finds the standard transaction manager running in the application server. Use this for any
custom caches you deploy where you want caching to participate in any JTA transactions.
249
Chapter 23. JBoss Cache Configuration and Deployment
• org.jboss.cache.transaction.BatchModeTransactionManagerLookup
This is used in the cache configurations used for web session and EJB SFSB caching. It
specifies a simple mock TransactionManager that ships with JBoss Cache called the
BatchModeTransactionManager. This transaction manager is not a true JTA transaction
manager and should not be used for anything other than JBoss Cache. Its usage in JBoss
Enterprise Application Platform is to get most of the benefits of JBoss Cache's transactional
behavior for the session replication use cases, but without getting tangled up with end user
transactions that may run during a request.
Note
For caches used for JPA/Hibernate caching, the transactionManagerLookupClass
should not be configured. Hibernate internally configures the cache to use the same
transaction manager it is using for database access.
It achieves this by using data versioning and copying for concurrent writers. The theory is that
readers continue reading shared state, while writers copy the shared state, increment a version id,
and write that shared state back after verifying that the version is still valid (i.e., another concurrent
writer has not changed this state first).
Generally MVCC is a better choice than PESSIMISTIC, which is deprecated as of JBoss Cache 3.0.
But, for the session caching usage in JBoss Enterprise Application Platform 5.0.0, PESSIMISTIC is
still the default. This is largely because for the session use case there are generally not concurrent
threads accessing the same cache location, so the benefits of MVCC are not as great.
250
JGroups Integration
• OPTIMISTIC locking seeks to improve upon the concurrency available with PESSIMISTIC by
creating a "workspace" for each request/transaction that accesses the cache. Data accessed by the
request/transaction (even reads) is copied into the workspace, which is adds overhead. All data is
versioned; on completion of non-transactional requests or commits of transactions the version of
data in the workspace is compared to the main cache, and an exception is raised if there are are
inconsistencies. Otherwise changes to the workspace are applied to the main cache.
OPTIMISTIC locking is deprecated but is still provided to support backward compatibility. Users are
encouraged to use MVCC instead, which provides the same benefits at lower cost.
REPEATABLE_READ is the default isolation level, to maintain compatibility with previous versions
of JBoss Cache. READ_COMMITTED, while providing a slightly weaker isolation, has a significant
performance benefit over REPEATABLE_READ.
This is the simplest approach. The CacheManager service already has a reference to the Channel
Factory service, so the only configuration task is to configure the name of the JGroups protocol stack
configuration to use.
If you are configuring your cache via the CacheManager service's jboss-cache-manager-jboss-
beans.xml file (see Section 23.2.1, “Deployment Via the CacheManager Service”), add the following
to your cache configuration, where the value is the name of the protocol stack configuration.:
<property name="multiplexerStack">udp</property>
If you are deploying a cache via a JBoss Microcontainer -jboss-beans.xml file (see Section 23.2.3,
“Deployment Via a -jboss-beans.xml File”), you need inject a reference to the Channel Factory
service as well as specifying the protocol stack configuration:
<property name="runtimeConfig">
<bean class="org.jboss.cache.config.RuntimeConfig">
<property name="muxChannelFactory"><inject bean="JChannelFactory"/></
property>
</bean>
</property>
<property name="multiplexerStack">udp</property>
251
Chapter 23. JBoss Cache Configuration and Deployment
If you are deploying a cache MBean via -service.xml file (see Section 23.2.2, “Deployment Via
a -service.xml File”), CacheJmxWrapper is the class of your MBean; that class exposes a
MuxChannelFactory MBean attribute. You dependency inject the Channel Factory service into this
attribute, and set the protocol stack name via the MultiplexerStack attribute:
23.1.6. Eviction
Eviction allows the cache to control memory by removing data (typically the least frequently used
data). If you wish to configure eviction for a custom cache, see the JBoss Cache documentation
for all of the available options. For details on configuring it for JPA/Hibernate caching, see the
Eviction chapter in the "Using JBoss Cache as a Hibernate Second Level Cache" guide at http://
www.jboss.org/jbossclustering/docs/hibernate-jbosscache-guide-3.pdf. For web session caches,
eviction should not be configured; the distributable session manager handles eviction itself. For
EJB 3 SFSB caches, stick with the eviction configuration in the Enterprise Application Platform's
standard sfsb-cache configuration (see Section 15.2.1, “The JBoss Enterprise Application Platform
CacheManager Service”). The EJB container will configure eviction itself using the values included in
each bean's configuration.
If you wish to configure cache loading for a custom cache, see the JBoss Cache documentation for all
of the available options. Do not configure cache loading for a JPA/Hibernate cache, as the database
itself serves as a persistent store; adding a cache loader is just redundant.
The caches used for web session and EJB3 SFSB caching use passivation. Next we'll discuss the
cache loader configuration for those caches in some detail.
In most cases you don't need to do anything to alter the cache loader configurations for the standard
web session and SFSB caches; the standard JBoss Enterprise Application Platform configurations
should suit your needs. The following is a bit more detail in case you're interested or want to change
from the defaults.
252
Cache Loaders
The Cache Loader configuration for the standard-session-cache config serves as a good
example:
<property name="cacheLoaderConfig">
<bean class="org.jboss.cache.config.CacheLoaderConfig">
<!-- Do not change these -->
<property name="passivation">true</property>
<property name="shared">false</property>
<property name="individualCacheLoaderConfigs">
<list>
<bean class="org.jboss.cache.loader.FileCacheLoaderConfig">
<!-- Where passivated sessions are stored -->
<property name="location">
${jboss.server.data.dir}${/}session</property>
<!-- Do not change these -->
<property name="async">false</property>
<property name="fetchPersistentState">true</property>
<property name="purgeOnStartup">true</property>
<property name="ignoreModifications">false</property>
<property name="checkCharacterPortability">false</
property>
</bean>
</list>
</property>
</bean>
</property>
Some explanation:
• shared property MUST be false. Do not passivate sessions to a shared persistent store,
otherwise if another node activates the session, it will be gone from the persistent store and also
gone from memory on other nodes that have passivated it. Backup copies will be lost.
• class attribute on a cache loader config bean must refer to the configuration class for a cache
loader implementation (e.g. org.jboss.cache.loader.FileCacheLoaderConfig or
org.jboss.cache.loader.JDBCCacheLoaderConfig). See the JBoss Cache documentation
for more on the available CacheLoader implementations. If you wish to use JDBCCacheLoader
(to persist to a database rather than the filesystem used by FileCacheLoader) note the comment
above about the shared property. Don't use a shared database, or at least not a shared table in the
database. Each node in the cluster must have its own storage location.
• location property for FileCacheLoaderConfig defines the root node of the filesystem tree where
passivated sessions should be stored. The default is to store them in your JBoss Enterprise
Application Platform configuration's data directory.
253
Chapter 23. JBoss Cache Configuration and Deployment
• async MUST be false to ensure passivated sessions are promptly written to the persistent store.
• fetchPersistentState property MUST be true to ensure passivated sessions are included in the
set of session backup copies transferred over from other nodes when the cache starts.
• purgeOnStartup should be true to ensure out-of-date session data left over from a previous
shutdown of a server doesn't pollute the current data set.
If the cache on another node needs data that it doesn't have locally, it can ask the other nodes in the
cluster to provide it; nodes that have a copy will provide it as part of a process called "data gravitation".
The new node will become the owner of the data, placing a backup copy of the data on its buddies.
The ability to gravitate data means there is no need for all requests for data to occur on a node that
has a copy of it; any node can handle a request for any data. However, data gravitation is expensive
and should not be a frequent occurence; ideally it should only occur if the node that is using some
data fails or is shut down, forcing interested clients to fail over to a different node. This makes buddy
replication primarily useful for session-type applications with session affinity (a.k.a. "sticky sessions")
where all requests for a particular session are normally handled by a single server.
Buddy replication can be enabled for the web session and EJB3 SFSB caches. Do not add buddy
replication to the cache configurations used for other standard clustering services (e.g. JPA/Hibernate
caching). Services not specifically engineered for buddy replication are highly unlikely to work correctly
if it is introduced.
Configuring buddy replication is fairly straightforward. As an example we'll look at the buddy replication
configuration section from the CacheManager service's standard-session-cache config:
<property name="buddyReplicationConfig">
<bean class="org.jboss.cache.config.BuddyReplicationConfig">
<property name="buddyCommunicationTimeout">17500</property>
254
Deploying Your Own JBoss Cache Instance
<property name="dataGravitationRemoveOnFind">true</property>
<property name="dataGravitationSearchBackupTrees">true</property>
<property name="buddyLocatorConfig">
<bean
class="org.jboss.cache.buddyreplication.NextMemberBuddyLocatorConfig">
<!-- The number of backup copies we maintain -->
<property name="numBuddies">1</property>
<!-- Means that each node will *try* to select a buddy on
a different physical host. If not able to do so
though, it will fall back to colocated nodes. -->
<property name="ignoreColocatedBuddies">true</property>
</bean>
</property>
</bean>
</property>
• numBuddies -- to how many backup nodes should each node replicate its state.
• buddyPoolName -- allows logical subgrouping of nodes within the cluster; if possible, buddies will
be chosen from nodes in the same buddy pool.
The ignoreColocatedBuddies switch means that when the cache is trying to find a buddy, it will if
possible not choose a buddy on the same physical host as itself. If the only server it can find is running
on its own machine, it will use that server as a buddy.
Section 23.1.1, “Editing the CacheManager Configuration” shows the configuration of the
CacheManager's "CacheConfigurationRegistry" bean. To add a new configuration, you would add an
additional element inside that bean's newConfigurations <map>:
255
Chapter 23. JBoss Cache Configuration and Deployment
<bean name="CacheConfigurationRegistry"
class="org.jboss.ha.cachemanager.DependencyInjectedConfigurationRegistry">
.....
<property name="newConfigurations">
<map keyClass="java.lang.String"
valueClass="org.jboss.cache.config.Configuration">
<entry><key>my-custom-cache</key>
<value>
<bean name="MyCustomCacheConfig"
class="org.jboss.cache.config.Configuration">
.... details of the my-custom-cache configuration
</bean>
</value>
</entry>
.....
See Section 23.1.1, “Editing the CacheManager Configuration” for an example configuration.
• Dependency Injection
If your application uses the JBoss Microcontainer for configuration, the simplest mechanism is to
have it inject the CacheManager into your service.
• JNDI Lookup
Alternatively, you can find look up the CacheManger is JNDI. It is bound under
java:CacheManager.
import org.jboss.ha.cachemanager.CacheManager;
256
Deployment Via the CacheManager Service
• CacheManagerLocator
JBoss Enterprise Application Platform also provides a service locator object that can be used to
access the CacheManager.
import org.jboss.ha.cachemanager.CacheManager;
import org.jboss.ha.framework.server.CacheManagerLocator;
Once a reference to the CacheManager is obtained; usage is simple. Access a cache by passing
in the name of the desired configuration. The CacheManager will not start the cache; this is the
responsibility of the application. The cache may, however, have been started by another application
running in the cache server; the cache may be shared. When the application is done using the cache,
it should not stop. Just inform the CacheManager that the cache is no longer being used; the manager
will stop the cache when all callers that have asked for the cache have released it.
import org.jboss.cache.Cache;
import org.jboss.ha.cachemanager.CacheManager;
import org.jboss.ha.framework.server.CacheManagerLocator;
cache.start();
}
257
Chapter 23. JBoss Cache Configuration and Deployment
import org.jboss.cache.pojo.PojoCache;
import org.jboss.ha.cachemanager.CacheManager;
import org.jboss.ha.framework.server.CacheManagerLocator;
pojoCache.start();
}
<server>
<mbean code="org.jboss.cache.jmx.CacheJmxWrapper"
name="foo:service=ExampleCacheJmxWrapper">
<attribute name="TransactionManagerLookupClass">
org.jboss.cache.transaction.JBossTransactionManagerLookup
</attribute>
258
Deployment Via a -jboss-beans.xml File
<attribute name="MultiplexerStack">udp</attribute>
<attribute name="ClusterName">Example-EntityCache</attribute>
<attribute name="IsolationLevel">REPEATABLE_READ</attribute>
<attribute name="CacheMode">REPL_SYNC</attribute>
<attribute name="InitialStateRetrievalTimeout">15000</attribute>
<attribute name="SyncReplTimeout">20000</attribute>
<attribute name="LockAcquisitionTimeout">15000</attribute>
<attribute name="ExposeManagementStatistics">true</attribute>
</mbean>
</server>
The CacheJmxWrapper is not the cache itself (i.e. you can't store stuff in it). Rather, as it's name
implies, it's a wrapper around an org.jboss.cache.Cache that handles integration with JMX.
CacheJmxWrapper exposes the org.jboss.cache.Cache via its CacheJmxWrapperMBean
MBean interfaces Cache attribute; services that need the cache can obtain a reference to it via that
attribute.
<deployment xmlns="urn:jboss:bean-deployer:2.0">
259
Chapter 23. JBoss Cache Configuration and Deployment
<property name="multiplexerStack">udp</property>
<property name="clusterName">Example-EntityCache</property>
<property name="isolationLevel">REPEATABLE_READ</property>
<property name="cacheMode">REPL_SYNC</property>
<property name="initialStateRetrievalTimeout">15000</property>
<property name="syncReplTimeout">20000</property>
<property name="lockAcquisitionTimeout">15000</property>
<property name="exposeManagementStatistics">true</property>
</bean>
</deployment>
The bulk of the above is the creation of a JBoss Cache Configuration object; this is the same
as what we saw in the configuration of the CacheManager service (see Section 23.1.1, “Editing
the CacheManager Configuration”). In this case we're not using the CacheManager service as a
cache factory, so instead we create our own factory bean and then use it to create the cache (the
"ExampleCache" bean). The "ExampleCache" is then injected into a (fictitious) service that needs it.
An interesting thing to note in the above example is the use of the RuntimeConfig object. External
resources like a TransactionManager and a JGroups ChannelFactory that are visible to the
microcontainer are dependency injected into the RuntimeConfig. The assumption here is that in
some other deployment descriptor in the Enterprise Application Platform, the referenced beans have
already been described.
Using the configuration above, the "ExampleCache" cache will not be visible in JMX. Here's an
alternate approach that results in the cache being bound into JMX:
<deployment xmlns="urn:jboss:bean-deployer:2.0">
260
Deployment Via a -jboss-beans.xml File
</bean>
<bean name="ExampleCacheJmxWrapper"
class="org.jboss.cache.jmx.CacheJmxWrapper">
<annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX
(name="foo:service=ExampleCacheJmxWrapper",
exposedInterface=org.jboss.cache.jmx.CacheJmxWrapperMBean.class,
registerDirectly=true)
</annotation>
</bean>
</deployment>
Here the "ExampleCacheJmxWrapper" bean handles the task of creating the cache from the
configuration. CacheJmxWrapper is a JBoss Cache class that provides an MBean interface for a
cache. Adding an <annotation> element binds the JBoss Microcontainer @JMX annotation to the bean;
that in turn results in JBoss Enterprise Application Platform registering the bean in JXM as part of the
deployment process.
261
262
Part IV. Performance Tuning
Chapter 24.
Improvement of application design and undertaking performance review of your applications before
implementation is vital to avoiding bottlenecks after implementation. To undertake a performance
review you need to setup a test environment undertake and analyze the test results. To effectively
undertake a review, you also need to identify peak application workload times and the difference from
normal workload periods. Peak workload times could be during the day, week, certain periods of the
month, quarter or year. In understanding peaks workloads it is advisable not to go by averages as the
peaks may be much more than the averages calculated over a period. The system requirements are
bound by the peaks in the workload not the averages. On undertaking tuning it is recommended to
carry out a few more tests and tuning of your system until a satisfactory performance is achieved.
To understand hardware performance tuning issues, it is also very critical to understand the hardware
architecture of your system.
• a program counter which holds the location of the succeeding executable tasks,
• CPU cache which is a limited memory that holds data currently being processed by the CPU.
Understanding your CPU architecture can be helpful in identifying your CPU specifications
and how it works. For AMD CPU's please refer to http://www.amd.com/us-en/Processors/
265
Chapter 24. JBoss Enterprise Application Platform 5 Performance Tuning
ProductInformation/0,,30_118,00.html for more information. For Intel CPU's please refer to http://
www.intel.com/products/processor/index.htm?iid=subhdr+prod_proc for more information.
RAM is crucial for example when tuning your database management system to manage buffer cache.
This would involve storing frequently used database information in RAM for quick application access
while taking caution not to affect overall performance of other applications and operating system.
However retrieval and storage of information from disk drives takes much longer as they use
mechanical heads to read and write information to the cylinders of the disk. Storage areas in RAM and
in the CPU can be accessed with equal speed while on the hard disk, movement of the disk head to
the requested disk block/blocks where information is stored is necessary.
Practices such as disk defragmentation and cleanups can help improve file retrieval and overall
performance of your applications. It is therefore crucial to manage the disk storage carefully with
the retrieval and processing of data in mind. You also need to identify a suitable file system for your
operating system to ensure the best performance possible.
Understanding the main architectural differences and issues that may occur with different computer
hardware profiles can help identify a suitable hardware performance and disaster management
strategy that would be suitable for your needs.
On Windows the task manager and performance monitor can be helpful in identifying system
performance bottlenecks while in unix based operating systems top and ps are used for the same
purpose. Linux distributions such as Red Hat Enterprise Linux and Fedora provide a graphical user
interface System Monitor that is useful to monitor system performance.
Overall operational performance metrics that are critical for the business such as response time to
user requests, database, network, CPU and memory performance among other metrics should be
identified and tested and logged in real-time where possible or with system deployments
266
Networking
For clustered environments, understanding and monitoring your cluster's performance and identifying
overloads early is critical to system failure prevention.
24.3.1. Networking
Network configurations may contribute to performance bottlenecks and may be hard to detect. For
example a user may get an error on their browser when trying to load a web application on a dial up
connection while the same page may load on a broadband internet connection. The main issue in this
scenario may be bandwidth and may not be obviously displayed in the error message displayed.
Identifying network architecture and infrastructure is therefore critical in performance tuning and fixing
system bottlenecks.
Most modern operating systems provide you with network hardware configuration tools while some
hardware manufacturers may also provide extended network hardware configuration tools with their
drivers.
Most operating systems support different communication protocols which you can tweak. Factors such
as TCP buffer memory space, connection buffer limits and acknowledgment options among others
should be take into account in your network design.
Deciding to turn DNS lookup on or off in your web servers can also affect your performance but
may be necessary to turn on for high security environments. Factoring this and allocating necessary
resources or hardware can help improve system performance.
In addition, using bench-marking tools to test your applications may be a quick way to pinpoint issues
in your code which can often be causes for performance bottlenecks. Iterative tests are recommended
to identify cache and other hardware issues that may arise due to start up or other factors.
267
Chapter 24. JBoss Enterprise Application Platform 5 Performance Tuning
It is also important to take caution not to create bottlenecks for other applications while fixing a
performance issue in one application.
24.5.1. Instrumentation
Applications should be instrumented for performance analysis. In most cases, the actual production
workload is different than the expected workload. Without instrumentation of your applications, you
will lack accurate tracking data. Workloads on your applications can also change over time, as the
business size, models or environment changes.
Instrumentation in the past would have had to be embedded in the application. Today, there are many
solutions for instrumentation that do not require developers to code. Commercial products, and the
JBoss AOP framework can be used for just this purpose. You can also turn on call statistics in the
containers, and Hibernate statistics. For more on this please refer to the AOP and Hibernate project
pages.
Taking successive thread dumps (includes the current call stack for each Java Enterprise Application
Platform thread) can give the application developers enough information to get a sense for what
is going on in the application. This is something that you might do after the application has hit a
performance wall. If the performance problem lasts for five minutes, you might generate a thread
dump one a minute. You can use the JVM "jps -l" command to get a list of running Java applications
and the process ids for each. Note the process id for the "org.jboss.Main" application. You will then
run the "jstack ProcessID" command (replacing ProcessID with the "org.jboss.Main" process id) and
that will generate the thread dump. Of course, you should redirect the output of the jstack command to
save the output ("jstack ProcessID > threaddump1.txt").
The Java Virtual Machine (JVM) manages segments (generations) of memory. If a segment of the
heap space is exhausted, you will see a Java OutOfMemoryError (OOME). All bets are off, when you
268
Memory usage
get a Java OutOfMemoryError. The application should be restarted to correct any bad state. Part of
tuning is checking how much memory headroom you have while under load. If available memory is
too low, you will need to increase the max Java memory size (possibly switching to a 64-bit JVM if
needed).
Running out of memory generates an Error that is not likely to be masked in a Java catch block
because it is an Error rather than an Exception. An OOME is also thrown when the permanent
memory is exhausted and that is not part of the heap per se. That is a JVM specific area of memory
where information on loaded classes is maintained. If you have a mountain of classes (e.g, a lot of
EJBs and JSP pages) you can easily exhaust this area. Oftentimes an application will fail to deploy or
fail to redeploy. Increase your permanent memory space as follows to avoid OOMEs. The default with
the -server switch is 64 megabytes:
-XX:MaxPermSize?=256m
Note this is in addition to the heap. In this case we have 512M heap, 256M permanent space for a
total of 768 megabytes. Don't forget the JVM itself takes up a chunk of system memory and there
is also per thread stack space (size varies based on OS). That can add up with a lot of HTTP/S
processors.
-XX:MaxPermSize?=256m -Xmx512m (total of 768 megabytes allocated from system - this is not
the total size of the VM and does not include the space the VM allocates for the "C heap" or stack
space)
The HotSpot Java Virtual Machine consists of various garbage collection tools which you can use to
collect garbage collection information that you can use to tune your applications. You can find more
information on the HotSpot Virtual machine on http://java.sun.com/javase/technologies/hotspot/.
Java 6 includes new tools that help monitor Java applications. Jmap can generate a heap dump
file (http://java.sun.com/javase/6/docs/technotes/tools/share/jmap.html) that can easily be read by
the Eclipse Memory Analyzer tool (http://www.eclipse.org/mat/). The jstat tool (http://java.sun.com/
javase/6/docs/technotes/tools/share/jstat.html) can help give you a precise picture of your permanent
memory space and the other segments on the Java memory heap.
• jboss.vfs.forceCopy - has the options true and false, with the default being false.
This defines how nested jars should be handled. If forceCopy equals true, we create a temporary
copy of the nested jar, and re-wire VFS accordingly. If forceCopy equals false, we handle nested
jars in-memory, which doesn't create temporary copy, but is more memory consuming. Currently
JBoss Enterprise Application Platform forces temporary copy by default.
If the useCopyJarHandler property is used as part of URI query, you can configure force-copy at
runtime, per URI root (if it doesn't already exist).
• jboss.vfs.forceVfsJar has the options true and false, with the default being false.
By setting this property to true, you can implement the old JAR handling. Set to false by default, old
JAR handling was deprecated in favor of new ZIP handling code.
269
Chapter 24. JBoss Enterprise Application Platform 5 Performance Tuning
• jboss.vfs.forceNoReaper has the options true and false, with the default being false.
To gain a bit on performance, we close JAR files asynchronously via the separate reaper thread. If
you wish to close JAR files synchronously, you can force no usage of the reaper thread. This can
also be defined using the URI query and the noReaper query section.
• jboss.vfs.forceCaseSensitive has the options true and false, with the default being false.
With this enabled you can force differentiation between lower and upper cased file paths.
• jboss.vfs.optimizeForMemory has the options true and false, with the default being false.
With this enabled we re-order in-memory JAR handling, to gain on memory consumption.
Every VirtualFile lookup from the VFS class uses this singleton cache instance to check for
an existing matching cache entry. By matching we also consider any existing ancestor from which
you can use exact VirtualFile instance.
• LRUVFSCache: evicts cache entries based on LRU, keeping min and max entries.
• CombinedVFSCache: holds few permanent roots, any other new root is cached in its realCache
property.
In the JBoss Enterprise Application Platform we use CombinedVFSCache as we know which are our
permanent roots to watch and keep. This is how it's configured in MC's bean configuration file.
<bean name="VFSCache">
<constructor
factoryClass="org.jboss.virtual.spi.cache.VFSCacheFactory"
factoryMethod="getInstance">
<!-- Use the CombinedVFSCache implementation -->
270
Memory usage
<parameter>org.jboss.virtual.plugins.cache.CombinedVFSCache</
parameter>
</constructor>
<start ignored="true"/>
<property name="permanentRoots">
<map keyClass="java.net.URL"
valueClass="org.jboss.virtual.spi.ExceptionHandler">
<entry>
<key>${jboss.lib.url}</key>
<value><null/></value>
</entry>
<entry>
<key>${jboss.common.lib.url}</key>
<value><inject bean="VfsNamesExceptionHandler"/></value>
</entry>
<entry>
<key>${jboss.server.lib.url}</key>
<value><inject bean="VfsNamesExceptionHandler"/></value>
</entry>
<entry>
<key>${jboss.server.home.url}deploy</key>
<value><inject bean="VfsNamesExceptionHandler"/></value>
</entry>
</map>
</property>
<property name="realCache">
<bean class="org.jboss.virtual.plugins.cache.IterableTimedVFSCache"/
>
</property>
</bean>
Any new custom VFS root (for example, an additional deploy directory) should be added to this
configuration.
ScanningMetaData can come from the jboss-scanning.xml file placed in META-INF directory.
This is a simple example of this file:
<scanning xmlns="urn:jboss:scanning:1.0">
<path name="myejbs.jar">
<include name="com.acme.foo"/>
271
Chapter 24. JBoss Enterprise Application Platform 5 Performance Tuning
<exclude name="com.acme.foo.bar"/>
</path>
<path name="my.war/WEB-INF/classes">
<include name="com.acme.foo"/>
</path>
</scanning>
Here you list the paths inside your deployment, and which packages to include or exclude. If there
is no explicit include, everything that is not excluded is included. If there is no path element at all,
everything is excluded, as in the following example.
<scanning xmlns="urn:jboss:scanning:1.0">
<!-- Purpose: Disable scanning for annotations in contained deployment. -->
</scanning>
Another way to limit scanning is to provide the jboss-classloading.xml file. More information
about this can be found in the Class Loader documentation section as it covers a lot of other details as
well, not just scanning.
Resource limits set by your operating system may also set limits on your database management
system. A database administrator can analyze a database and identify performance bottlenecks
through taking the above factors into consideration and adjusting the necessary database
management system parameters such as writing dirty buffers to disk, checkpoints and log file
rotations. In some instances hardware upgrades may also be necessary to improve database
performance.
Database connections can be costly to establish and manage. Applications that create new
connections to the database with every transaction or query and then close that connection add a
great deal of overhead. Having a very small connection pool will also throttle the applications as
the JBoss Enterprise Application Platform by default queues the request for a default of 30,000
milliseconds (30 seconds) before cancellation and throwing an exception.
We recommend reliance on data source definitions you can setup in the deploy directory of the JBoss
Enterprise Application Platform and utilizing the connection pool settings. Connection pooling in the
JBoss Enterprise Application Platform allows you to easily monitor your connection usage from the
JMX console to determine proper sizing. Your database management system may also shipped with
tools that allow you to monitor connections.
Depending on the databases implemented, please ensure you create a data source file in the deploy
directory of your configuration as shown below:
<JBoss_Home>/server/<your_configuration>/deploy/
272
Clustering Tuning
<yourdatabasename>-ds.xml
Note
Please note that the name of the file must end with -ds.xml in order for the JBoss
Enterprise Application Platform to recognize it as a data source file. The Postgres
database data source file for example is named postgres-ds.xml.
Examples
Examples of datasource definition files for external databases can be found in the
<JBoss_Home>/docs/examples/jca directory.
The specific configuration steps needed to increase the maximum allowed buffer sizes are OS
specific. See your OS documentation for instructions on how to increase these. For Linux systems,
maximum values for these buffers sizes that will survive machine restarts can be set by editing the /
etc/sysctl.conf file:
273
Chapter 24. JBoss Enterprise Application Platform 5 Performance Tuning
In the above example, the -Djgroups.bind_addr setting tells the the Enterprise Application
Platform to run intra-cluster JGroups traffic over the 192.168.100.104 interface, with -b specifying that
all other traffic should use 10.0.0.104.
If your application uses high volume session replication (web sessions or EJB3 stateful session
beans), you might be able to increase performance by configuring the distributed caching layer
to use a JGroups channel configured with message bundling enabled. This is done by editing
the <JBoss_Home>/server/<your_configuration>/deploy/cluster/jboss-cache-
manager.sar/META-INF/jboss-cache-jboss-beans.xml file. For example, for the cache used
by default for web sessions:
. . .
. . .
. . .
For FIELD granularity web sessions, in the same file the same change can be made to the cache
configuration with the field-granularity-session-cache key. For EJB3 stateful session beans,
in the same file the same change can be made to the cache configuration with the sfsb-cache key.
274
Clustering Tuning
Note
Using the udp-async JGroups protocol stack for the session caches means an additional
JGroups transport protocol will be used. This means additional sockets will be opened
compared to a standard Enterprise Application Platform installation.
. . .
. . .
<property name="buddyReplicationConfig">
<bean class="org.jboss.cache.config.BuddyReplicationConfig">
. . .
For FIELD granularity web sessions, in the same file the same change can be made to the cache
configuration with the field-granularity-session-cache key. For EJB3 stateful session beans,
in the same file the same change can be made to the cache configuration with the sfsb-cache key.
275
Chapter 24. JBoss Enterprise Application Platform 5 Performance Tuning
So, be sure to carefully load test your clustered application when deciding whether to store items in
the Hibernate Second Level Cache. Avoid the temptation to turn on caching for all entity types; instead
rank your entity types based on how infrequent writes of each type are and how likely it is that more
than one transaction will read a particular entity. Then enable caching for one type at a time, testing for
the performance impact.
Be doubly cautious about enabling caching of query result sets. When query caching is enabled, any
time there is a database write, the clustered cache needs to send two messages around the cluster.
These messages are used to ensure that any query results that may have been affected by the write
are invalidated out of the cache. These messages need to be sent whether or not the entity type that
has been written is itself cached. The cost of these messages can easily offset the benefit of query
result caching. So, again, be sure to test the effect of caching.
jboss.jgroups:cluster=<cluster_name>,protocol=UDP,type=protocol
Provides statistical information on the sending and receipt of messages over the network, along
with statistics on the behavior of the two thread pools used to carry incoming messages up the
channel's protocol stack.
Useful attributes directly related to the rate of transmission and receipt include MessagesSent,
BytesSent, MessagesReceived and BytesReceived.
Useful attributes related to the behavior of the thread pool used to carry ordinary incoming
messages up the protocol stack include IncomingPoolSize and IncomingQueueSize.
Equivalent attributes for the pool of threads used to carry special, unordered "out-of-band"
messages up the protocol stack include OOBPoolSize and OOBQueueSize. Note that
OOBQueueSize will typically be 0 as the standard JGroups configurations do not use a queue for
OOB messages.
276
Other key configurations
jboss.jgroups:cluster=<cluster_name>,protocol=UNICAST,type=protocol
Provides statistical information on the behavior of the protocol responsible for ensuring lossless,
ordered delivery of unicast (i.e. point-to-point) messages.
jboss.jgroups:cluster=<cluster_name>,protocol=NAKACK,type=protocol
Provides statistical information on the behavior of the protocol responsible for ensuring lossless,
ordered delivery of multicast (i.e. point-to-multipoint) messages.
Use the XmitRequestsReceived attribute to track how often a node is being asked to
retransmit a messages it sent; use XmitRequestsSent to track how often a node is needing to
request retransmission of a message.
jboss.jgroups:cluster=<cluster_name>,protocol=FC,type=protocol
Provides statistical information on the behavior of the protocol responsible for ensuring fast
message senders do not overwhelm slow receivers.
Attributes useful for monitoring whether threads seeking to send messages are having to block
while waiting for credits from receivers include Blockings, AverageTimeBlocked and
TotalTimeBlocked.
JBoss Enterprise Application Platform 5 has a robust thread pooling, that should be sized
appropriately. The server has a jboss-service.xml file in the <JBoss_Home>/server/
<your_configuration>/conf directory that defines the system thread pool. There is a setting that
defines the behavior if there isn't a thread available in the pool for execution. The default is to allow the
calling thread to execute the task. You can monitor the queue depth of the system thread pool through
the JMX Console, and determine from that if you need to make the pool larger.
The new administration console can be used for configuring and managing different aspects of the
Enterprise Application Platform environment.
The default configuration is appropriate for development, but not necessarily for a production
environment. In the default configuration, console logging is enabled. Console logging is ideal for
development, especially within the IDE, as you get all the log messages to show in the IDE console
view. In a production environment, console logging is very expensive and is not recommended. Turn
down the verbosity level of logging if its not necessary. Please note that the less you log, the less I/O
will be generated, and the better the overall throughput will be.
Other performance tuning aspects include Caching, Clustering and Replication which are discussed in
the respective Chapters in this book.
277
278
Part V. Index
Index
JBossWS
Web Services, 15
JMS (see JBoss Messaging)
A
P
AOP (see JBoss AOP)
Performance
JBoss Enterprise Application Platform 5
C Performance Tuning, 265
Configuration Pooling
databases, 95 JBossJCA, 121
Profiles
D all, 10
DataSource default, 10
deployment type, 9 minimal, 10
production, 10
E standard, 10
EAR (see Enterprise Application) web, 10
Enterprise Application ProfileService
deployment type, 9 bootstrap, 6
Exploded Deployment, 10
R
F Remoting
Frequently Asked Questions, 125 about, 79
H S
Hot deployment SAR (see Service Archive)
disabling, 6 Server Configuration (see Server Profile)
implementation, 6 Server Profile
definition, 10
Service Archive
J *-service.xml deployment type, 9
JAX-WS (see Web Services)
deployment type, 9
JBoss AOP
applying aspects, 63
aspect oriented framework, 61 W
creating aspects, 63 WAR (see Web Application)
JBoss Enterprise Application Platform Web Application
architecture, 5 deployment type, 9
bootstrap, 6 Web Services
JMX Microkernel, 1 web services, 15
microcontainer, 1
performance tuning, 265
Server interface implementation, 6
JBoss Enterprise Application Platform 5
Performance Tuning
performance, 265
JBoss Messaging
about, 85
JBoss Microcontainer
*-jboss-beans.xml deployment type, 9
beans deployment type, 9
281
282