XML-enabling your Lotus Notes application

Skill Level: Introductory

Branavan Ganesan
IBM software developer
IBM

07 Feb 2003

This tutorial shows how the team at developerWorks used the XML capabilities of
Lotus Domino to extract data and transfer it to DB2. The same technique can be
used to transform and transfer data to any other relational database engine.

Section 1. Getting started

About this tutorial

This tutorial shows how we at developerWorks, working with the IBM eServer
Developer Domain team, used the XML capabilities of Lotus Domino to extract data
and transfer it to DB2. The same technique can be used to transform and transfer
data to any other relational database engine. This tutorial goes hand-in-hand with
"Managing and moving data using Java technology, XML, DB2, and Lotus Notes,"
which describes how to use forms, views, agents and page design elements to
facilitate XML representations of Domino data. It also covers how to invoke an
external Java servlet to read and transform the data in XML format. You should at
least skim the article before proceeding with this tutorial.

In this tutorial, you'll get a hands-on approach to the design techniques discussed in
"Managing and moving data using Java technology, XML, DB2, and Lotus Notes".
This tutorial shows you how to gather publish-ready metadata documents for
articles, surface them as XML, and invoke an external Java servlet from within Notes
that reads, transforms, converts the information, and updates DB2 accordingly.
Because this tutorial will focus on the Domino portion of this exercise, we won't
cover the external servlet in any detail.

XML-enabling your Lotus Notes application

© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 1 of 16
developerWorks® ibm.com/developerWorks

Getting the Notes database

In this tutorial, we'll be enhancing a Domino application. If you are an old hand at
Notes, and want to hone your skills on XML enabling a Domino app, you could
create a test database and build these design elements. If you are unfamiliar with
Notes or just don't feel like creating the forms, views, and so on, download the demo
database called xmlcptut.nsf, which has the base elements that we'll develop in this
tutorial.

You will be using the JDK 1.1 that ships with Domino to develop the agent within
Domino. If you are feeling adventurous, you might want to try your hand at an
external servlet that acts on the agent. If you need the Java 2 SDK, 1.3.0, you can
download it from http://java.sun.com/j2se/1.3.0_01/index.html. This tutorial does not
cover the coding of the external Java servlet.

Initial setup
Now that the database is downloaded, let's quickly check that everything is in place.

• Place the database on a Domino server. Be sure to include your name or

a group that includes you in the access control list (ACL). You will need a
minimum of designer level access to this database on the server.
• If you do not have internet passwords assigned at your company (or on
this server), be sure to add 'Anonymous' as type person with a minimum
of reader level authority in your ACL. This is essential to access our
documents over the HTTP protocol.
• Browse to the XML view in the database using the URL
http:/[hostname]/[dir]/xmlcptut.nsf/xmlview?OpenView. Substitute
'hostname' with the hostname of your Domino server. Substitute 'dir' with
the path where you have placed the xmlcptut.nsf database.
If everything has been set up correctly, you should see the rudimentary Domino view
containing the three documents that are set up to publish as XML documents.

Section 2. The content management form

Introduction

XML-enabling your Lotus Notes application

Page 2 of 16 © Copyright IBM Corporation 1994, 2008. All rights reserved.
ibm.com/developerWorks developerWorks®

Before we dive into the data transfer function, it's a good idea to understand exactly
what the data looks like. In this chapter we'll review:

• The content management form

• The fields that constitute the metadata of an article object
• The fields that support work management in this form

The content management form

1. Open the xmlcptut.nsf database in Domino Designer.

2. Open the Content submission form.

Figure 1. The content submission form

There is no rocket science here. The first two sections, "General document
information" and "Document searchable information," contain facts about the record.
The last section, "Document tracking information," contains fields that support and
facilitate work management.

XML-enabling your Lotus Notes application

© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 3 of 16
developerWorks® ibm.com/developerWorks

Data fields
The data fields in this form describe the characteristics of an article that is published
on the production Web site. Collectively, this constitutes the metadata for a given
article.

docID The 32-character Notes ID of the document

docTitle The title of the document
docAbstract An abstract that describes the article
activeURL The URL of the live article
docTargetDateLive The date the article moves to production
hardwarePlatform The hardware platforms this article pertains to
docType The kind of document this article is
contentTopic What topics this article falls under

Work management fields

Carrying out this same exercise for the work management fields yields the following:

docStatus The state of the current record in the editorial

workflow
serverStatus A field used to determine whether this is a new
document, an updated document, or an obsolete
document that is flagged for removal
publish_flag A simple flag field to mark a document as
publish-ready or not

What is of most importance to us is the publish_flag field. Any document with a

publish_flag value of 0 is deemed production ready.

Now that we have an overview of the content management form, we'll proceed to the
XML display form, which takes data created and edited using the content
management form and displays it in XML-ready format. Using a form that users are
familiar with in the Domino environment helps keep the back-office work a non-factor
when determining your solution. Users familiar with this form will not need any
training, or changes, for you to implement your data transfer solutions.

XML-enabling your Lotus Notes application

Page 4 of 16 © Copyright IBM Corporation 1994, 2008. All rights reserved.
ibm.com/developerWorks developerWorks®

Section 3. Setting up the XML display form

Displaying an XML document

In this chapter, you'll set up a display form that takes a Notes document and displays
it as an XML document. Once the form is set up, you will modify the fields so the
information displays as text when shown in the display form.

The XML display form

1. Open the xmlcptut.nsf database in Domino Designer.

2. Open the XML Display form form.

Figure 2. The XML display form

We will modify three fields here: doc_abstr, last_mod_date, and

doc_content_topic.

The goal is to code the field values so they are interpreted correctly both by the
Domino translation engine and by a parser that is expecting a well-formed XML

XML-enabling your Lotus Notes application

© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 5 of 16
developerWorks® ibm.com/developerWorks

document.

Note the following:

• All the fields in the form are set to be computed for display. This is to
avoid data overwrites during an accidental save when viewing documents
using this form.
• The second line is a processing instruction with a reference to the
stylesheet stored as a page design element within the database.
<?xml-stylesheet href="http://hostname.ibm.com/dir/xmlcptut.nsf/datatransfer.xsl"
type="text/xsl"?>

We will briefly look at that later in this tutorial.

• The Treat document contents as HTML option in the form properties is

checked. This is to ensure the Domino translation engine does not add
extraneous HTML tags when rendering the document over the browser.

The abstract field

The abstract field contains a general blurb about the article. It is likely to contain one
or more reserved characters, in particular & and '. These must be replaced with the
necessary escape characters. To do this, remove the quotes in the doc_abstrfield
and code it as follows:

tempvar := @ReplaceSubstring(docAbstract ; "&" :""":; "&amp;":"&quot;");

tempvar

The function @ReplaceSubstring takes the sourceList, which in this case is

docAbstract, and replaces all the values in the fromList (&, ") with the values in
the toList (&amp;, &apos;).

The modified date

We will do two things with the modified date field:

• The XML file will be read as text, with an encoding of UTF-8. Therefore, a
date/time datatype will not be recognized. We change the datatype to text
in the display formula.
• Our database engine is configured to accept dates in the mm-dd-yyyy
format.

XML-enabling your Lotus Notes application

Page 6 of 16 © Copyright IBM Corporation 1994, 2008. All rights reserved.
ibm.com/developerWorks developerWorks®

Enter the following code into the formula field in the last_mod_date field:

tempvar := @ReplaceSubstring(@Text(lastmodifieddate); "/" ; "-");

tempvar

In this case, the function @ReplaceSubstring takes the sourceList, which in this
case is lastmodifieddate, and replaces values in the fromList (/) with the values
in the toList (-). The one twist is that we change the datatype of the date field before
manipulating it.

The content topic

The doc_content_topic field is used to categorize all of the records in the
database into topics pertinent to users of the library. This field is set up as a keyword
field with a predefined set of values based on a pre-determined taxonomy. The issue
here is that any given document can pertain to one or more topics. This is solved in
Domino by allowing this field to accept multiple values. To render this in XML,
however, each of the values has to be tagged separately. For example, the two
values Server architecture and Parallel computing need to display as
follows:

<contenttopics>
<topic>614708929906D77186256B7B005C919D</topic>
<topic>FCD74466386C241386256B7B005E3AC5</topic>
</contenttopics>

Note the additional snag. In our DB2 tables, the content topics are stored in lookup
tables, so we need to supply the unique id, which will function as the foreign key into
the lookup table. The unique id is obtained using @DbLookup.

Following is the code for the content topic field:

shortval := @DbLookup("" : "Cache"; ""; "Content Topics"; contentTopic;4);

@If(@IsError(shortval); @Success;@Failure("Error"));
shortval = @If(contentTopic = ""; ""; shortval);
topicval := "<topic>" + shortval + "</topic>" + @NewLine;
topicval

We are beginning to get a handle on things. We now have a form that will display a
Notes record as an XML document, but we have more work to do before it can be
tested.

XML-enabling your Lotus Notes application

© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 7 of 16
developerWorks® ibm.com/developerWorks

Section 4. XML views

Introduction
Now that we have the display form set up, we'll set up the necessary views to
support it. In this chapter we will look at the main work management view, and build
a view that facilitates the servlet call to Domino.

The two views

The two supporting views function as:

• The search view, used by the servlet call to locate the document it is
supposed to read in as XML.
• The working view, which allows editors to manipulate which documents
are to be published, updated, deleted, and so on.
There is a third utility view that acts as a DBLookup view, and provides the ability to
substitute content topic value with the corresponding unique id. We won't examine it
in this tutorial because it's standard Domino fare.

Since we are using the form to display data as XML, the views will not be used to
perform this function. But, they will serve as necessary elements to form and
manipulate document collections.

The search view

First, we'll build the search view, or doc display view, used by the servlet.

1. Open the XML Capabilities Tutorial database in Domino Designer.

2. Build a new view and call it "XML Document view" with an alias of
"servletview."

3. Set up the 'View selection' formula as follows:

SELECT Form = "Content Submission Form" & (progressStatus = "Completed" |

progressStatus = "Archived") & publish_flag = "0"

XML-enabling your Lotus Notes application

Page 8 of 16 © Copyright IBM Corporation 1994, 2008. All rights reserved.
ibm.com/developerWorks developerWorks®

This formula corresponds to the status fields in the content submission form, and
ensures that only documents intended to be published appear in this view.

1. Under Design-View-Properties, click the Treat view contents as HTML

box. This ensures that the Domino translation engine does not include
unwanted HTML tags when rendering this view.

2. Set up the "form formula" for this view to show documents using the XML
display form by entering "XML display form".

Note that this view and the editorial working view could be collapsed by making the
unique id the first sorted column in the view. However, conventional wisdom shows
that collapsing utility functions and work management functions in the same view
can cause complications during later development and enhancement efforts. To test
this view:

1. Open the database using the Domino client and go to the XML document
view.

2. Open a document by double clicking on it.

3. You should see the document opened up in the the display form, showing
the data in the record in XML format.

Good work! Now that this view is set up, you have the essential utility elements
necessary to access Notes data in XML format both within the Domino environment
and from any external call over HTTP.

Working view
It's now time to take care of the end users. Users of the database do not really care
that you are using Java and XML-based technologies to transfer and manipulate
their data. They just want a database that is reliable and easy to use. Creating
design elements to support this is essential for a successful application.

For this application, there are a few pre-built views to support end user work
management. These are by no means exhaustive, but are there to show the types of
supporting views that can be built. As developers of Domino applications, these
views should be familiar to you. Let's take a quick look at the view that shows
publishable documents. This view lets the end user control which documents are
going to be moved to be published.

1. Open the "Docs ready to publish - XML view" in Domino Designer.

XML-enabling your Lotus Notes application

© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 9 of 16
developerWorks® ibm.com/developerWorks

2. Open a document by double clicking on it.

3. Note that the view selection formula contains the condition

progressStatus = "Completed" & publish_flag = "0".

The progressStatus field and publish_flag field give end users two ways to facilitate
work management and to control which documents are moved for publication.
Showing additional values such as the title and content type give the documents
more meaning.

Section 5. Building XSLT stylesheet page elements

Introduction
This section of the tutorial takes a quick look at how to build and store Extensible
Stylesheet Language for Transformations (XSLT) stylesheets that transform your
XML documents and store them along with the data. XSLT intricacies and
frustrations constitute a whole subject area unto itself. While it is not covered
extensively in this tutorial, investigation into XSLT is a must for any developer doing
serious work involving XML.

A stylesheet page element

Let's create a simple stylesheet that acts on the XML document surfaced by our
now-polished display form. We will do a variation of the "Hello World," and a simple
text message that shows the document title.

1. Open the database in Domino Designer.

2. Go to the page design elements by clicking on Pages in the left

navigation area. You should see a pre-created page element called
articletransform.xsl.

3. Click on New Page and create a new page element.

4. Go to page properties and call this hello.xsl. Be sure to check Treat

contents as HTML for this element to ensure that unwanted HTML tags
are not added by the Domino translation engine during the rendering

XML-enabling your Lotus Notes application

Page 10 of 16 © Copyright IBM Corporation 1994, 2008. All rights reserved.
ibm.com/developerWorks developerWorks®

process.

Showtitle.xsl
Type the following code into the body of the page element:

<?xml version="1.0" encoding="UTF-8"?>

<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:fo="http://www.w3.org/1999/XSL/Format">
<xsl:output method="html"/>
<xsl:template match="/">
<xsl:apply-templates select="/articleinfo/tables/fields"/>
</xsl:template>
<xsl:template match="/articleinfo/tables/fields">
<html>
<body>
<b>
<xsl:text>Title: </xsl:text>
<xsl:value-of select="papertitle"/>
</b>
</body>
</html>
</xsl:template>
</xsl:stylesheet>

Save and close the page element.

This stylesheet can now be called from our XML document by using the processing
instruction:

<?xml-stylesheet href=
"http://hostname.ibm.com/dir/xmlcptut.nsf/datatransfer.xsl"

XML-enabling your Lotus Notes application

© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 11 of 16
developerWorks® ibm.com/developerWorks

type="text/xsl"?>

The transformation result

Running a transformation against any article record using the processing instruction
in either an XML-aware browser or an XML IDE, such as XMLSpy, looks like this:

An Easter egg
In addition to the stylesheet we just went through, I've included another stylesheet as
a page design element in the demo database. We used this stylesheet in the project
to take the data from the XML document we used and generate an SQL statement. It
is provided as is.

Section 6. The calling agent

Introduction
We will finish this tutorial by looking at the section of a Java agent that invokes an
external servlet. Information about the XML document is passed in the call. Using
this technique allows the inclusion of code outside the Domino environment in the
overall application solution. We will concentrate on the following aspects of the
agent:

• Because the invocation occurs over HTTP, it is stateless. Therefore, we

will pass as parameters the URL of the XML document that is to be acted
on, the unique id of the document (for logging and error tracking
purposes), and an identifier to facilitate the location of application-specific
information that is kept server side by the servlet.
• Two URLs will be constructed -- one to identify the XML document and

XML-enabling your Lotus Notes application

Page 12 of 16 © Copyright IBM Corporation 1994, 2008. All rights reserved.
ibm.com/developerWorks developerWorks®

one to make the invocation.

• The invocation will be placed within a loop that acts on a document
collection. The collection will comprise all records marked as ready for
promotion to a production environment.

The agent
To speed things along, the shell of the agent has been pre-coded. Much of this is
self evident. We will add the method that generates the Notes URL, and will add the
portion of the code that invokes our external servlet.

Notice that the while loop that holds the servlet invocation has been set up in the
main method. The document collection is generated by using the
AgentContext.getUnprocessedDocuments() method. This way, the collection
can change from selected documents to all the documents in the view, depending on
the setting you use for the Which document(s) should it act on property.

Because this code is executed from an agent, a check is added to ensure that the
publish_flag value is not 1. An agent can potentially be invoked from any view
against any document in the database.

Last but not least, note that prior to invoking the servlet, the status fields and
modified date value are changed within the agent. Doing this automatically within the
agent gives users a data value showing what has been promoted and when.

The generateURL method

1. Open the database in Domino Designer.

2. Go to the agents design elements and open the PublishCode agent.

3. Add the following method to the agent before the main method:

private String generateURL(String uniqueID, Database thisdb,

View viewName, Session s)
{
String fullURL= new String();
try {
String dbPath = new String ( thisdb.getFilePath().replace( '\\' , '/' ) );
Vector aliases = viewName.getAliases();
String view = new String((aliases.elementAt(0)).toString());
//Use alias in order to accommodate view name change if necessary
Name sname = s.createName(thisdb.getServer());
String server = new String( sname.getCommon());
fullURL = new String ("http://" + server + ".ibm.com/" + dbPath + "/"
+ view + "/" + uniqueID);
//substitute the ibm.com with your domain
} catch(Exception e) {

XML-enabling your Lotus Notes application

© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 13 of 16
developerWorks® ibm.com/developerWorks

e.printStackTrace();
}
return fullURL;
}

Invoking the servlet

Now let's move to the code that invokes the servlet. We have already generated the
URL of the XML document with the aptly named generateURL() method. Now we
will generate a second URL, this time the URL of the calling servlet, feed it the three
parameters that we have built, and then invoke it.

Enter the following code under the comment in the main method:

URL servletURL =
new URL("http://hostname.ibm.com/servlet/DataTransfer?notesURL="
+ url + "UNID="+ uniqueid + "propID=IDENTIFIER");
URLConnection uc = servletURL.openConnection();
BufferedReader in = new BufferedReader(
new InputStreamReader(uc.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);
in.close();

This code makes use of the URL and URLConnection classes. Once the
connection is set up, the page contents are read by an input stream. We make use
of a BufferedReader, as it has a very helpful readLine() method.

Activating the agent

In our example, the "Activating the agent" property is set to "Manually from agents
list" to remove "(PublishCode)" from the actions list. The agent is then activated via
the "Publish" action button in the "Docs ready to publish - XML view". This
precaution is taken so users do not have the ability to run the agent against
documents from other views in the database.

If such a precaution is not necessary, then this step need not be taken. Allowing the
agent to be invoked from the actions list allows users greater flexibity in terms of
which views to use when identifying documents that need to be promoted.

XML-enabling your Lotus Notes application

Page 14 of 16 © Copyright IBM Corporation 1994, 2008. All rights reserved.
ibm.com/developerWorks developerWorks®

Section 7. Wrapup

Summary
This tutorial has shown you how to collect Domino documents and surface them as
XML documents. We reviewed the data that's in the Notes forms, built views and a
stylesheet to handle the data, and then looked at a Java agent that invokes an
external servlet. With this information in hand, you should be able to apply these
techniques to your own Notes documents.

XML-enabling your Lotus Notes application

© Copyright IBM Corporation 1994, 2008. All rights reserved. Page 15 of 16
developerWorks® ibm.com/developerWorks

Resources
• Read the article that accompanies this tutorial, "Managing and moving data
using Java technology, XML, DB2, and Lotus Notes."

Download the demo database called xmlcptut.nsf, which has the base elements
that we'll develop in this tutorial.

There are innumerable resources to guide you on your path to mastering XSLT.
Listed below are some resources that provide basic groundwork, along with
some that are specific to using the XML-enablement capabilities of Lotus
Domino.
• Exercising XML in Domino Designer (Lotus Developer Domain, January
2001)
• "Publishing XML Data using Notes/Domino 6" (Lotus Developer Domain)
• Take the tutorial, "Manipulating data with XSL" (developerWorks, October
2001).
• Take the Introduction to XML tutorial (developerWorks, August 2002)

About the author

Branavan Ganesan
Branavan Ganesan is a Lotus Domino developer who remembers when Domino was
just a pizza franchise. He has been working with the developerWorks development
team for the last two years and had contributed to the content management system
and the publishing environment of developerWorks. He holds a very useful Bachelors
degree in Comparative Religion from Haverford College and a Masters degree in
South Asia Regional Studies from the University of Pennsylvania. When he is not
playing foosball or attempting to work, he plays tablas and practices surviving
triathlons. You can reach Bran at [email protected]. This tutorial highlights the
XML enablement of Domino, but is based on a collaborative project between the
developerWorks infrastructure team and the eServer Developer Domain (eSDD)
team. The members of the team are Jay Allen, Barry Busler, Elizabeth Dunnagan,
Tim Janikowski, Keith Purcell, Adam Reeves.

XML-enabling your Lotus Notes application

Page 16 of 16 © Copyright IBM Corporation 1994, 2008. All rights reserved.