Java Sample App Backend Tutorial

The Java SDK tutorial bridges the gap between simple and advanced concepts by walking through a complete web application.

The full source code for the tutorial is available on GitHub couchbaselabs/try-cb-Java. The primary focus of the tutorial is to explain the function and theory behind the Couchbase Java client and how it works together with Couchbase Server, and especially new features in versions 4.0/4.5 like N1QL, FTS and sub-document. It makes use of the travel-sample data set. The code that generates the web application is provided with the source code, but is not discussed in this tutorial.

Specific Java prerequisites and set up

In addition to the prerequisites mentioned in the Sample Application, you’ll also need:

Your favourite IDE with a JDK 1.6+ installed (this tutorial assumes IntelliJ with JDK 1.8)
Maven 3
That’s it!

To get set up for the tutorial, follow these steps:

git clone https://github.com/couchbaselabs/try-cb-java.git or download the source
if your Couchbase Server is not running on localhost you can update the configuration’s hostname in src/main/resources/application.properties
open the project in your IDE, import as needed and let it build
alternatively, go straight to running the project by executing the following command in a terminal from inside the project’s root directory: mvn spring-boot:run

If you want to code it yourself, the real work is done in the following files:

There’s currently no "fill-in-the-blanks" branch so you’ll have to delete method bodies and try to take it from there.

This tutorial focuses on querying through N1QL and FTS rather than views. If you want information about using views, see Views.

Walking Through the API

The following sections lead you through the primary functions of the sample application. This shows you how to use the various features and services of Couchbase including: connecting to a cluster and bucket, key/value interactions, document query through N1QL and full text searches.

Configure and Bootstrap the SDK

Where: src/main/java/trycb/config/Database.java

Goals: Connecting to a Cluster, getting a reference to a Bucket and learning to reuse it.

Relevant Documentation Topics: Managing Connections using the Java SDK with Couchbase Server

The first step is to let the application connect to your cluster and obtain a reference to a Bucket (this is your entry point for the whole storage API). Spring Boot automatically configures a Cluster using the @autowired keyword by looking for the following properties in src/main/resources/application.properties:

spring.couchbase.bootstrap-hosts
spring.couchbase.bootstrap.bucket.name
spring.couchbase.bootstrap.bucket.password

Note that both Cluster and Bucket are thread safe and must be reused across your application (if you don’t, you’ll see a warning in the logs). Spring already knows the Cluster is a singleton because it’s @autowired and the Bucket can be set up with the @Bean keyword so that it will also be managed as a singleton in the application.

Connecting to the Cluster

public @Bean Cluster cluster() {
    return CouchbaseCluster.create(hostname);
}

The cluster() method creates the bean for the cluster reference. Here the application uses one of the simpler factory methods, with just a hostname. Without a hostname, the default is to connect to localhost. Note that you can tune the SDK through a CouchbaseEnvironment instance; you can pass this in as an additional first argument. This is particularly recommended if you need to connect to multiple clusters in the same application.

You could make the bootstrap process safer by providing a list of hostnames/IPs to bootstrap from (in case the one node you provided for bootstrap is unfortunately down when creating the Cluster reference). It is recommended for production deployments that at least 3 hostnames/IPs are used.

Getting a Bucket

public @Bean Bucket bucket() {
    return cluster().openBucket(bucket, password);
}

The second step is to connect to the Couchbase bucket you’ll be using. Here the application uses sample data from travel-sample (and the application.properties configuration reflects that). To obtain the corresponding Bucket from the Cluster and register it as a @Bean in bucket() method, simply open it using the configured bucket name and password.

Both the bucket and cluster can be managed through the SDK as well (e.g. add views or create new buckets), see Managing clusters using the Java SDK with Couchbase Server for more information.

With these steps, the application is ready to use the API. In the next step, the Key/Value (KV) storage part of the API will be covered.

Manage Users using Key/Value API

Where: src/main/java/trycb/service/User.java

Goals: Use Bucket's Key/Value operations and discover the Document API.

Relevant Documentation Topics: CRUD Document Operations Using the Java SDK with Couchbase Server, Asynchronous Progamming Using the Java SDK with Couchbase Server

Couchbase Server is a document oriented database which provides access to your data both through a document ID (for high performance access), as well as through views and N1QL (as powerful query languages).

This is noticeable in the API, where the methods reflect Key/Value operations (get, create, etc) and work with a Document interface that has an id() and a content. The default Document implementation, JsonDocument, accepts a simple representation of JSON as a content using JsonObject.

If you already have mechanisms in place to deal with marshalling/unmarshalling of your domain objects to/from JSON, you can skip the extra step of converting them to JsonObject and use a RawJsonDocument instead.

Creating New Users

Since this is a @Service, the createLogin method will be used by the REST API and return a Result object. This is a standardized wrapper for results returned by this application, that the frontend can interpret. This particular result will contain a Map<String, Object> with some more information about the user just created. Spring injects the Bucket reference for us, along with the request parameters username and password:

/**
 * Create a user.
 */
public Result<Map<String, Object>> createLogin(final Bucket bucket, final String username, final String password,   int expiry) {

Next the application will prepare the content for the new user (as a JsonObject) and the associated document (in order to give it an ID) and it will also set the document expiry, if one was provided:

String passHash = BCrypt.hashpw(password, BCrypt.gensalt());
JsonObject data = JsonObject.create()
    .put("type", "user")
    .put("name", username)
    .put("password", passHash);
JsonDocument doc;
if (expiry > 0) {
    doc = JsonDocument.create("user::" + username, expiry, data);
} else {
    doc = JsonDocument.create("user::" + username, data);
}

The "user::" prefix is arbitrary to this application, this is just a convention that the app uses to obtain unique keys and have additional information in it, but the key could have been anything else (even sequence numbers or UUIDs).

Here comes the part where the application use the Couchbase Server API to store the document, it’s rather simple:

bucket.insert(doc);

Actually, the application should send a response with the content and a success flag to the HTTP client. It should also indicate failure if the SDK throws an exception, so it wraps that in a try-catch block:

try {
    bucket.insert(doc);
    return Result.of(
            JsonObject.create().put("token", jwtService.buildToken(username)).toMap(),
            narration);
} catch (Exception e) {
    throw new AuthenticationServiceException("There was an error creating account");
}

When it comes to storing a document, you have broadly three choices of method:

insert() will only work if no document currently exists for the given ID, otherwise a DocumentAlreadyExistsException will be thrown.
replace() will only work if the document does already exist, otherwise a DocumentDoesNotExistException is thrown.
upsert() will always work, replacing or creating the document as needed.

So the result in fact just contains a JWT (Json WebToken) to identify the new user. If there is a problem, an AuthenticationServiceException will be thrown and correctly translated by the controller layer. But what is this narration object in the Result?

The frontend understands this second part of the Result, the narration, as something that it can display in a console, so that users of the application can get an idea of what is going on, on the server side while browsing the app. It is similar to a log, but sent to the frontend.

Checking Login by Getting the User’s Document

In the login method, the application checks a User’s credential and for that it needs to retrieve the corresponding document. Since user documents are identified by prefixing their username with user::, this is pretty simple:

JsonDocument doc = bucket.get("user::" + username);

If that particular key doesn’t exist, the get method returns null. That’s useful to check if the user exists:

if (doc == null) {
    throw new AuthenticationCredentialsNotFoundException("Bad Username or Password");
}

Otherwise it’s just a matter of checking the hashed password with the one provided by the user, and responding accordingly. Notice how the application gets the hash by calling content().getString("password"):

//...continued
} else if(BCrypt.checkpw(password, doc.content().getString("password"))) {
    return JsonObject.create()
        .put("token", jwtService.buildToken(username))
        .toMap();
} else {
    throw new AuthenticationCredentialsNotFoundException("Bad Username or Password");
}

A Quick Glance at the Async API with RxJava

The 2.x Java SDK relies on RxJava for its asynchronous API. It offers a powerful way of composing asynchronous streams for your processing. The getFlightsForUser() method can serve as a quick example of such an asynchronous call, it will return the result of a chain started with the async SDK call:

bucket.async().get("user::" + username)

RxJava’s Observable is a push model where you describe your stream (by composing and chaining rx operators) then subscribe to it (to consume the end data). You can also manage what to do with error notifications in the subscription.

The async() method on Bucket will switch to the async API. There, get will return an Observable in which the requested Document is emitted.

If the requested key doesn’t exist, the async API will instead result in an empty Observable and nothing will be emitted. See below for an example of how to deal with that particular case.

The next step in the chain is simply to extract the flight information that is needed and return it as a List. This is done by the transforming operator map. The application passes a function that will transform each emitted JsonDocument into a List<Object>:

.map(new Func1<JsonDocument, List<Object>>() {
     @Override
     public List<Object> call(JsonDocument doc) {
         JsonObject data = doc.content();
         JsonArray flights = data.getArray("flights");
         if (flights != null) {
             return flights.toList();
         } else {
             return Collections.emptyList();
         }
     }
 })

In order to detect that the document doesn’t exist, the application has to do things a bit differently since the map function won’t receive a null (it’s the enclosing Observable stream that is empty). Fortunately, RxJava provides a method to emit a single default value if an upstream Observable is empty:

.defaultIfEmpty(Collections.emptyList())

Finally, since in this example the application still must exit the method by returning a value in a synchronous manner, it can revert to a blocking behaviour and say "we only expect a single() value to be emitted and wait for it to return it":

.toBlocking()
.single();

To learn more about Observables, see the Asynchronous Progamming Using the Java SDK with Couchbase Server section.

A First N1QL Query: Finding Airports

Where: src/main/java/trycb/service/Airport.java

Goals: Use N1QL and the DSL to perform your first SELECT on Couchbase Server.

Relevant Documentation Topics: N1QL Queries Using the Java SDK with Couchbase Server.

In the SDK, there is a query method that accepts all variants of querying with Couchbase Server (views, spatial/geo views, N1QL and FTS). For N1QL, the N1qlQuery is expected. This allows you to wrap a N1QL Statement, provide query tuning through a N1qlParams and if necessary provide values for placeholders in the statement as a JsonObject or JsonArray.

N1QL is a super-set of SQL, so if you’re familiar with SQL you’ll feel at ease.

Statements can be provided either in String form or using the DSL.

The findAll method is expected to return a List (several matching rows) of Maps representing the JSON value. Once again, it is to be wrapped in a Result to standardize the JSON representation for the frontend. Spring will inject the bucket into it and the params attribute from the HTTP request. From that the application will start building a Statement:

/**
 * Find all airports.
 */
public static Result<List<Map<String, Object>>> findAll(final Bucket bucket, final String params) {
    Statement query;
//continued...

The application needs to select just the airport name from relevant documents in the bucket. Since it wants to filter relevant document on a criteria that depends on the input length, the query starts with "SELECT airportname FROM `travel-sample`".

AsPath prefix = select("airportname").from(i(bucket.name()));

Then it can choose the correct fields to look into depending on the length of the input. Notice the x() method that produces a token/expression from a string. From there you can apply operators like eq() (equals).

if (params.length() == 3) {
    query = prefix.where(x("faa").eq(s(params.toUpperCase())));
} else if (params.length() == 4 && (params.equals(params.toUpperCase()) || params.equals(params.toLowerCase()))) {
    query = prefix.where(x("icao").eq(s(params.toUpperCase())));
} else {
    query = prefix.where(i("airportname").like(s(params + "%")));
}

Use static imports on these methods of the Expression class:

x to create an Expression representing a plain token, like a field.
s to create a string literal (with appropriate quotes).
i to escape a token with back ticks (for instance when refering to the travel-sample bucket, you need to escape it because otherwise N1QL will interpret the dash as a subtraction operator).

The statement is ready! You can view (and log it) via its toString() method:

logQuery(query.toString());
//query.toString() example: SELECT airportname FROM `travel-sample` WHERE faa = "LAX"

Then the application needs to actually execute this statement by wrapping it in a N1qlQuery and invoking bucket.query(). Here is a very simple query with no placeholders and no particular tuning of the query is necessary, so we’ll use the N1qlQuery.simple() factory method:

N1qlQueryResult result = bucket.query(N1qlQuery.simple(query));
List<Map<String, Object>> data = extractResultOrThrow(result);

By looking at extractResultOrThrow, you can understand the structure of the N1QL response (as represented by N1qlQueryResult):

/**
 * Extract a N1Ql result or throw if there is an issue.
 */
private static List<Map<String, Object>> extractResultOrThrow(N1qlQueryResult result) {
    if (!result.finalSuccess()) {
        LOGGER.warn("Query returned with errors: " + result.errors());
        throw new DataRetrievalFailureException("Query error: " + result.errors());
    }

    List<Map<String, Object>> content = new ArrayList<Map<String, Object>>();
    for (N1qlQueryRow row : result) {
        content.add(row.value().toMap());
    }
    return content;
}

The N1qlQueryResult has two status flags: one intermediary parseSuccess() that indicates immediately if there is a syntax error (parseSuccess() == false) or not, and one that indicates the definite result of the query (finalSuccess()).

If the query is successful, it will offer a list of N1qlQueryRow through allRows(). Otherwise it will have JsonObject errors in errors(). That’s what the application inspects to respectively build a list of results or throw a DataRetrievalFailureException containing all the errors.

After is has extracted the rows, it once again packages them into a Result, augmented with the exact statement the application executed as a narration:

return Result.of(data, query.toString());

More Complex Queries: Finding Routes

Where: src/main/java/trycb/service/FlightPath.java

Goals: Let the DSL guide you into making more complex N1QL queries.

Relevant Documentation Topics: N1QL Queries Using the Java SDK with Couchbase Server.

In this class, there are two more complex queries. The first aims at transforming the human-readable airport name for the departure and arrival airports to FAA codes:

SELECT faa AS fromAirport FROM `travel-sample` WHERE airportname = "Los Angeles Intl"
  UNION SELECT faa AS toAirport FROM `travel-sample` WHERE airportname = "San Francisco Intl"

The second aims at constructing the result set of available flight paths that connect the two airports:

SELECT a.name, s.flight, s.utc, r.sourceairport, r.destinationairport, r.equipment
  FROM `travel-sample` AS r
  UNNEST r.schedule AS s
  JOIN `travel-sample` AS a ON KEYS r.airlineid
  WHERE r.sourceairport = "LAX" AND r.destinationairport = "SFO" AND s.day = 6
  ORDER BY a.name ASC

Yes, you read that right, N1QL can do joins (on a single bucket or on several). It works as long as the "foreign key" described by ON KEYS clause can be mapped to a document’s key in the joined bucket.

A specificity of N1QL that can be seen in the second statement is UNNEST. It extracts a sub-JSON and puts it at the same root level as the bucket (so its possible to do joins on each element in this sub-JSON as if they were entries in a left-hand side bucket).

For this final step, try to obtain the equivalent of these statements via the DSL and see how it guides you through the possibilities of the query language.

Indexing the Data: N1QL & GSI

Where: src/main/java/trycb/utils/StartupPreparations.java

Goals: Use the Index DSL to make sure data is indexed for N1QL to query it.

Index management is a bit more advanced (and is already done when loading the sample), so now that you’ve learned about N1QL, you can have a look at it. For N1QL to work, you must first ensure that at least a Primary Index has been created. For that you can use the DSL from the Index class:

Index.createPrimaryIndex().on(bucket.name())

The fluent API will guide you with the available options, you just have to declare that you want to createPrimaryIndex() and specify on(...) which Bucket.

You can also create secondary indexes on specific fields of the JSON, for better performance:

Index.createIndex(name).on(bucket.name(), x(name.replace("def_", "")))

In this case, give a name to your index, specify the target bucket AND the field(s) in the JSON to index.

Full Text Search: Finding Hotels

Where: src/mainjava/trycb/service/Hotel.java

Goals: Use FTS to search for matching Hotels and use subdoc API to fetch the relevant data for each result.

Relevant Documentation Topics: Full Text Search (FTS) Using the Java SDK with Couchbase Server, Sub-Document Operations.

In this service, hotels are searched for using more fuzzy criterias, like the content of the address or the description of an hotel. This is done using Full Text Search (FTS). When some results match the specified criteria, only the relevant data for each result to be displayed in the UI is fetched using the subdocument API.

The findHotels method accepts two parameters, location and description, which are the two possible refining criteria for an hotel search.

public Result findHotels(final String location, final String description) {
    ConjunctionQuery fts = SearchQuery.conjuncts(SearchQuery.term("hotel").field("type"));

A ConjunctionQuery allows to combine multiple FTS queries into one, in an AND fashion. That search always includes an exact match criteria that restricts it to the hotel data type (as reflected in the type field of the JSON document).

If the user provided a location keyword, a second component is added to the FTS query that will look for that keyword in several address-related fields of the document. That is done in an OR fashion, using a Disjunction this time:

if (location != null && !location.isEmpty() && !"*".equals(location)) {
    fts.and(SearchQuery.disjuncts(
                SearchQuery.matchPhrase(location).field("country"),
                SearchQuery.matchPhrase(location).field("city"),
                SearchQuery.matchPhrase(location).field("state"),
                SearchQuery.matchPhrase(location).field("address")
        ));
}

Similarly, if a description keyword was provided by the user, the application look at the freeform text description field and name fields of the document:

if (description != null && !description.isEmpty() && !"*".equals(description)) {
    fts.and(
        SearchQuery.disjuncts(
                SearchQuery.matchPhrase(description).field("description"),
                SearchQuery.matchPhrase(description).field("name")
        ));
}

The matchPhrase FTS query can contain several words and will search for variations of the words (e.g. including plural forms or words with the same root).

The compound FTS query is now ready to be executed. The application builds a SearchQuery object out of it, which also determines which FTS index to use ("hotels") and allows you to set various parameters (like a limit of maximum 100 hits to return). The query is logged (and kept for narration) then executed, returning a SearchQueryResult object:

SearchQuery query = new SearchQuery("hotels", fts)
    .limit(100);
logQuery(query.export().toString());
SearchQueryResult result = bucket.query(query);

The FTS results are then iterated over, and the document corresponding to each result is fetched. In actuality, only the parts of the document that will be displayed in the UI are required. This is where the sub-document API comes in.

The sub-document API allows you to fetch or mutate only a set of paths inside a JSON document, without having to send the whole document back and forth. This can save network bandwidth if the document is large and the parts that the application is interested in are small. So here the results of the FTS search are iterated over and appropriate subdoc calls are triggered:

for (SearchQueryRow row : result) {
    DocumentFragment<Lookup> fragment = bucket
            .lookupIn(row.id())
            .get("country")
            .get("city")
            .get("state")
            .get("address")
            .get("name")
            .get("description")
            .execute();

Each FTS result is represented as a SearchQueryRow which exposes the document’s id(). The application can use the sub-document API dedicated to fetching data (bucket.lookupIn(documentId)) and specify what parts it wants: country, city, state, address, name and description. It can then execute() the sub-document specification. In the rest of the code, the address-related fields are aggregated together and the data obtained is returned as a List<Map<String, Object>>.

Back in the findHotels method, the application artificially prepares a narration that reflects the subdocument specification and returns the list of data wrapped in a Result with two narration elements: the FTS query that was executed and the subdocument specification.

String ftsContext = ftsContext = query.export().toString();
String subdocContext = "DocumentFragment<Lookup> fragment = bucket\n" +
"                    .lookupIn(row.id())\n" +
"                    .get(\"country\")\n" +
"                    .get(\"city\")\n" +
"                    .get(\"state\")\n" +
"                    .get(\"address\")\n" +
"                    .get(\"name\")\n" +
"                    .get(\"description\")\n" +
"                    .execute();";

return Result.of(extractResultOrThrow(result), ftsContext, subdocContext);