Groovy

Business Process
Automation
O. David
Colorado State University
Dept. of Civil Engineering, Dept. of Computer Science

US Department of Agriculture
Objective
 Automate and integrate software project
management practices using CB with respect
to
◦ Trackers
◦ Forums
◦ Builds
◦ Documents
◦ Users

by using an efficient scripting approach.

Olaf David, CSU/USDA 2

A First Example
import cbscript.CB
import static cbscript.CB.*
import org.codehaus.groovy.runtime.TimeCategory

jf = CB.login("http://localhost:8080", “bond", “pass")

proj = jf.projects.find{it.name == "MyProj"}

use(TimeCategory) {
needsBuild = proj.trackers.find{it.name ==“Bugs"}.issues.any{
bug -> bug.status == "Fixed" && bug.submitted < 30.days.ago
&& bug.priority> LOW && bug.hasCommits
}

if (needsBuild) {
println “Starting build ....“
log = proj.builds.find{it.name == “BuildApp"}.invoke()
println " Build done: ${log.status} “
println " Output: ${log.stdOut} "
}
}

jf.logout();

Olaf David, CSU/USDA 3

 Principal Architecture
Groovy API
-High level, object centric, and productivity oriented
-For process automation

cbscript

cb-api CB Server

Java API
-Low level, session centric, stateless
-For a wide range of applications
Docs DB SCM

Olaf David, CSU/USDA 4

Why Scripting?
 Semantic density and flexibility of scripts
◦ Usability/Efficiency aspects of scripts vs. plain Java
vs. Web UI
 Remote Management of (multiple) CB server
 Leveraging general scripting productivity
(Groovy)
◦ Closures, Properties, Duck typing
◦ Builder (DSL)
◦ Intuitive Date management/use
 CB web UI does not allow complex queries
 No automation using the CB Web UI

Olaf David, CSU/USDA 5

Prerequisites
 Java 1.6+
 Groovy 1.6+
 cbscript-5.3.1.jar
◦ Contains cb-api.jar of the same CB version
 CB 5.3.x
◦ Network access to context
‘<cburl>/cb/remote-api’
◦ Project user access (account)

Olaf David, CSU/USDA 6

Login/Logout
import cbscript.CB
import static cbscript.CB.*

jf = CB.login("http://localhost:8080", “bond", "007")

// or
jf = CB.login(
url: "http://10.177.19.63:8080",
login: "bond",
password:“pass")

// do some work.

jf.logout();

 API respects CB licenses, access permissions

Olaf David, CSU/USDA 7

Session information
assert jf.login == "bond"
assert jf.url == "http://localhost:8080"
assert jf.user.realName == "Default System Administrator"

 meta information can be obtained about the

session.
 read/only

Olaf David, CSU/USDA 8

Getting Projects
// list all projects
jf.projects.each {
println " id: ${it.id} name : ${it.name}"
}

// find project(s)
assert jf.projects.find{it.name == "Moose"}.id == 2374
assert jf.projects.find{it.id == 2374}.name == "Moose“

// direct
jf.project(‘Moose’)

 ‘projects’ returns a list of projects

 Using closures to iterate, filter, search,…

Olaf David, CSU/USDA 9

Managing Tracker
// get all tracker of the ‘oms’ project.
jf.projects.find{it.name == "oms"}.trackers.each {
tracker ->
print tracker.name
print tracker.description
}

 ‘trackers’ returns a list of tracker

 Use closures to iterate, filter, search

Olaf David, CSU/USDA 10

Tracker Issues query
// Find issues that are not Closed
def bugTracker = jf.projects.find{
it.name == "oms“ }.trackers.find {it.name == "Bugs"}

bugTracker.issues.findAll{
it.status != 'Closed'
}.each{
println it.name
}

 Closures help traversing resources

 Obtain the Bug tracker in the oms project.
Then find all the ‘Closed’ issues

Olaf David, CSU/USDA 11

Tracker Issues query (cont.)
bugTracker.issues.findAll{
it.assignedTo == jf.user && it.status == 'New'
}.each {
println it.name
}

 Find all issues assigned to me and submitted

last week

Olaf David, CSU/USDA 12

Tracker Issues query (cont.)
bugTracker.issues.findAll{
it.summary.contains('big problem')
}.each{
println it.name
}

 Find a tracker issue that has the string 'big

problem' in summary

Olaf David, CSU/USDA 13

Tracker Issues query (cont.)
...
bugTracker.issues.findAll{
it.status == "Resolved" &&
it.assignedTo.name == "bond" &&
it.submitted < 10.days.ago
}.each{
println it.name
}

 All 'Resolved' issues assigned to user 'bond'

that were submitted the last 10 days
 Groovy date use is intuitive!

Olaf David, CSU/USDA 14

Tracker Issues query (cont.)
bugTracker.issues.findAll{
return it.submitter == jf.user && !it.hasCommits
}.each{
println it.name
}

 All tracker issues submitted by me and do

not have an attached SCM commit.

Olaf David, CSU/USDA 15

Tracker Issues query (cont.)
bugTracker.issues.findAll{
bug ->
bug.priority == HIGH && bug.status != “Closed”
&& bug.assigned == null
}.each{
println it.name
}

 All tracker issues with high priority that are

still open and are not assigned yet to a
developer.

Olaf David, CSU/USDA 16

Submit a new Issue
// get the bug tracker
def bugs = jf.projects.find{
it.name == “oms"}.trackers.find {it.name == "Bugs"}

bugs.submit(summary:"My Bug",
text:"This is a new Bug", priority:NORMAL)

 Use properties ‘summary’, ‘text’, ‘priority’…

 Priority: LOWEST, LOW, NORMAL, HIGH,
HIGHEST

Olaf David, CSU/USDA 17

Comment on an Issue
bug = bugs.submit(summary:"My Bug",
text:"This is a new Bug", priority:NORMAL)

bug.comment(text:”please specify”, format:PLAIN_TEXT)

 Comment on a existing issue.

Olaf David, CSU/USDA 18

Attach a file to an Issue
bug = bugTracker.issues.find{
bug -> bug.priority == HIGH
}

bug.attach(file:”/tmp/spec-1.txt”)

 Upload and attach a file to an issue.

 (file is not stored in Documents)

Olaf David, CSU/USDA 19

Forums
def newsForum= jf.projects.find{
it.name == "oms"}.forums.find {it.name == "News"}

newsForum.findAll{
post -> post.submitted > 8.hours.ago
}.each{
print it.text
}

 ‘forums’ contains all project forums

 Check for new posts today.

Olaf David, CSU/USDA 20

Post a new Forum message
news= jf.projects.find{it.name == "oms"}.forums.find {
it.name == "News“
}

news.post(subject:"New message",
text:"Content here ...")

 ‘forums’ contains all project forums

 Check for new posts today.

Olaf David, CSU/USDA 21

Check for Posts
news.findAll{
post ->
post.submitter.realName == “Joe Poster” &&
post.submitted < 1.week.ago
}.each{
print it.text
}

 Check for all new posts from “Joe Poster”

submitted this week and print out the
message text.

Olaf David, CSU/USDA 22

Documents
prj = jf.projects.find{it.name == "cbtest"}
doc = prj.artifact(‘download/EC2TW.txt’)

assert doc.bytes.size() == 709

assert doc.name == ‘EC2TW.txt’
assert doc.directory == false

 Use ‘artifact’ to reference a document or

folder in the project ‘Documents’
 Always absolute to ‘Documents’ root.

Olaf David, CSU/USDA 23

Documents (cont.)
prj = jf.projects.find{it.name == "cbtest"}
doc = prj.artifact("download/EC2TW.txt")

new File('c:/tmp/test1').withOutputStream {
it.write doc.bytes
}

 Copy a file from a CB server to the local FS

Olaf David, CSU/USDA 24

Builds
build = prj.builds.find{it.name == "build-lib"}
log = build.invoke();

 ‘builds’ contains all the builds of a project.

 Call ‘invoke’ to execute the build on a CB
server. (Invoke is a synchronous call)

Olaf David, CSU/USDA 25

Builds (cont.)
build = prj.builds.find{it.name == "build-lib"}
log = build.invoke();

assert log.output != null

assert log.error == null
assert log.status == “Successful”
assert log.successful == true
assert log.startedBy == jf.user

 Build ‘log’ captures the result of the build.

Olaf David, CSU/USDA 26

Associations
bug = bugTracker.submit(summary:"My Bug",
text:"This is a new Bug", priority:HIGH)

def lib = prj.artifact("builds/ngmf.jar")

bug.relate(lib, RELATED)

 Create associations by calling ‘relate’ on

tasks, artifacts, documents, etc.
 Pass in the ‘other’ task, artifact, document
and the kind of relationship (RELATED,
CHILD, PARENT, DEPENDS)

Olaf David, CSU/USDA 27

Associations (cont.)
readyToBuild = prj.trackers.find {
it.name == "Bug“
}.issues.findAll {
it.status == "Fixed" && thisMonth(it.submitted)
}

readyToBuild.each { bug -> milestone.relate(bug, DEPENDS) }

 Create a Association to many issues in one

‘bulk’ operation (use closure)
 ‘milestone’ is a tracker issue

Olaf David, CSU/USDA 28

InterWiki links
build = prj.builds.find{it.name == "build-lib"}
result = build.invoke();

String wikitext =
“””!!Milestone info
* Output Log Reference: ${CB.link(result)}
“””

// resolves to:
// “ Output Log Reference : [BUILDLOG:23456]”

 Use ‘CB.link()’ to create interwiki links for

task messages, forum posts, artifact
descriptions to reference other resources

Olaf David, CSU/USDA 29

Efficient and Direct Access
prj = jf.project(‘moose’) // get a project by name
prj = jf.project(1234) // get a project by If

tra = jf.tracker(2345) // get a tracker by id

iss = jf.issue(23467) // get a issue by id
bld = jf.build(23411) // get a build by id
...

 Direct access to a
project/forum/tracker/issue/post/build/.. using its
‘Id’ or ‘name’ (projects only)
 No traversing all higher level resources
necessary -> very efficient, but you need to
know the ID!!!
 Resource Access Efficiency vs. Query Flexibility
Olaf David, CSU/USDA 30
 Examples*
Problem elevation
SCRUM support
SCM commit control
Auditable Build Management
Tracker Status Statistics

* The examples demonstrate the support of

some aspects of the topics above

Olaf David, CSU/USDA 31

SCRUM support
 Objective
◦ Increase the priority of issues, based on some
dynamic rules

 Benefit
◦ Early problem detection, identification, and
communication

Olaf David, CSU/USDA 32

Simple Problem Elevation
bugs.findAll{
bug ->
bug.status == “Verified" &&
bug.assignedTo.name == “Joe” &&
bug.spendHours == 0 &&
!bug.hasCommits &&
bug.submittedAt < new Date(2009,4,1)
}.each{
it.priority++
}
 Increase the priority of all verified bugs
submitted before April that are assigned to
Joe that he did not fixed… so far.
 Call this above every day (e.g. cron)

Olaf David, CSU/USDA 33

 Examples*
Problem elevation
SCRUM support
SCM commit control
Auditable Build Management
Tracker Status Statistics

* The examples demonstrate the support of

some aspects of the topics above

Olaf David, CSU/USDA 34

SCRUM support
 Objective
◦ Support some aspects of SCRUM

 Benefit
◦ SCRUM coverage
◦ Sprint meeting preparation

Olaf David, CSU/USDA 35

SCRUM Support
jf.projects.find{it.name == "oms"}.trackers.find{
it.name == "Features"}.issues.findAll {
task ->
task.status == "Closed" && task.modifiedAt > 30.days.ago
}.each {
println "${it.summary} - ${it.description}"
}

 Prepare a Sprint review meeting

◦ Check which tasks in 'Feature' requests in the 'oms'
project were finished over the last 30 days.

Olaf David, CSU/USDA 36

SCRUM support (cont.)
jf.projects.find{it.name == "oms"}.trackers.find {it.name ==
"Bugs"}.issues.findAll{
it.status != "Closed" &&
it.submittedAt > new Date(2008, 1, 4)
}.each {
println "${it.summary} - ${it.description}"
}

 Prepare a Sprint review meeting

◦ Get all the bugs that are still not closed and were
submitted since April 1st 2008 (last sprint meeting)

Olaf David, CSU/USDA 37

SCRUM Support (cont.)
def issues = jf.projects.find{it.name=="oms"}.trackers.find{
it.name == "Features"}.issues.findAll {
it.category == ‘UI’
}
(-7..0).each { day ->
map[day] = issues.findAll {
it.status == "Closed" && it.modifiedAt < 8.hours.ago
}.size()
}

println map
// [-7:10, -6:3, -5:12, -4:4, -3:10, -2:5, -1:1, 0:2]

 Creating a sprint backlog for all

closed tasks in the UI category of
new features.
◦ The map contains the last seven days,
key: day diff to today (-1 is yesterday)
value: the number of closed tasks
Olaf David, CSU/USDA 38
SCRUM Support (cont.)
hours = 0
def fissues = jf.projects.find{it.name=="oms"}.trackers.find{
it.name == "Features“
}.issues.findAll {
it.status == “InDevelopment”
}.each {
hours += it.estimatedHours – it.spentHours
}

f = new File(“burndown.dat”)
today = new Date()
f.append("${today} , ${hours}")

 Creating daily burndown chart data

◦ Call this script every day to track the hours
being ‘burned’ in developing new features .
◦ Create the graph based on this file.

Olaf David, CSU/USDA 39

 Examples*
Problem elevation
SCRUM support
SCM commit control
Auditable Build Management
Tracker Status Statistics

* The examples demonstrate the support of

some aspects of the topics above

Olaf David, CSU/USDA 40

SCM Commit triggers status
change
 Objective
◦ A commit contains additional (meta) information
that triggers process actions such as an issue
status change

 Benefit
◦ This developer’s shortcut saves time

General Pattern for triggering of process actions by

commit message content

Olaf David, CSU/USDA 41

 SCM Commit triggers status
change
jf.projects.find{it.name == "OMSProject"}.trackers.find {
it.name == "Bugs“
}.issues.findAll{
it.hasCommits&& it.status != "Closed“
}.each { task ->
task.commits.each {
if (it.message ==~ /.*\(status=(.*)\).*/ && today(it.commitedAt)) {
task.status = str.replaceAll(/.*\(status=(.*)\).*/, {
all, status -> return status })
}
}
}
 A SVN commit message such as

svn commit –m “#1234 (status=Fixed) Changed loop order”

will trigger a status change on a task to fixed; no separate UI

operation needed. (Put this in cron and execute frequently)

Olaf David, CSU/USDA 42

 Examples*
Problem elevation
SCRUM support
SCM commit control
Auditable Build Management
Tracker Status Statistics

* The examples demonstrate the support of

some aspects of the topics above

Olaf David, CSU/USDA 43

Auditable Build
Management
 Objective
◦ Flexible build process triggering by tracker content
information
◦ Capture all information that relate to a build
Bugs <-> Build
Features <-> Build
Build process log <-> Build
Build product <-> Build
 Benefit
◦ Developer/PM independent build control
◦ Build result notification
◦ Automated build product storage and versioning
◦ Auditable and traceable

Olaf David, CSU/USDA 44

Pick up
Edit Developer changes task status
A to..
Comment
Status Bugs project Fixed
change

DE Feature Implemente
d
V Requests
Commit
assoc.

Commi
t
Update
Push
Pull
SC
Repo M

 The developer’s view

Olaf David, CSU/USDA 45

 Auditable Build Management
(1)
import cbscript.*
import static cbscript.CB.*

// login to CB
jf = CB.login("http://localhost:8080", "bond", “pass")

// find the project of interest

prj = jf.projects.find{it.name == "OMSProject"}

...

 Login and get the project reference

Olaf David, CSU/USDA 46

 A L
project ocal
Bugs

Feature
Identif
y (2
DE MG
V Requests ) R

Milestones

Build-lib

SC
M

Docume
nts

Olaf David, CSU/USDA 47

 Auditable Build Management
...(2)
// normal and higher priority bug fixed this month
readyBugs = prj.trackers.find {it.name == "Bug"}.issues.findAll {
it.status == "Fixed" && isThisMonth(it.submitted) &&
it.priority > LOW
}

// A feature is implemented and there is a commit

readyFeatures = prj.trackers.find {
it.name == “Features"}.issues.findAll {
it.status == "Implemented" && it.hasCommits
}

// combine all features and bugs into one single list

readyList = readyBugs + readyFeatures
...

 Find ‘candidates’ in ‘Bugs’ and ‘Features’

that should lead to a new build
 Combine the lists in to one.
Olaf David, CSU/USDA 48
 A L
project ocal
Bugs

DE Feature
Requests MG
V
Issue
(3 R

Milestones
build
)

3.2
Build-lib build

3.3
log

3.1
SC
checkout/update M

3.4
<cbrelease
Docume
nts

Olaf David, CSU/USDA 49

 Auditable Build Management
...(3)

// build check
if (!readyList.empty) {

// invoke the build.

build = prj.builds.find{it.name == "build-lib"}
result = build.invoke();

...

 If the list is not empty, we need to build

 Find the build and invoke it.
 The build will create the file ‘builds/ngmf.jar’
in ‘Documents’

Olaf David, CSU/USDA 50

 A L
project ocal
Bugs

DE Feature
V Requests New milestone
build
(4 MG
R
)
Milestones

Build-lib

SC
M

Docume
nts

Olaf David, CSU/USDA 51

 Auditable Build Management
...(4)
String wikiinfo = "!!Milestone info \n" +
"!Milestone Build:\n" +
"* Build Status: ${result.status}\n" +
"* Output Log Reference: ${CB.ref(result)}\n"

// create a new milestone in the milestone tracker

mt = prj.trackers.find {it.name == "Milestones"}
milestone = mt.submit(
new Task(name:"MS-1.2", text:wikiinfo, priority:LOW))
...

 Create a WIKI description with the result info

(status and link to the Buildlog)
 Submit a new milestone task with the WIKI
content.
 Notification will be sent on submission.

Olaf David, CSU/USDA 52

 A project L
ocal
Bugs
5.2
relate
DE Feature
V Requests (5 MG
R
)
Milestones 5.1 relate / link

Build-lib

SC
M

Docume
nts

Olaf David, CSU/USDA 53

 Auditable Build Management
(5)
...
// the build product
def lib = prj.artifact("builds/ngmf.jar")

// relate the milestone to all

// bugs/features/and the build product
milestone.relate(lib, RELATED)
readyList.each{ task -> milestone.relate(task, DEPENDS) }
...

 Relate the new milestone to all bugs and

features in the readyList.
 Also relate it to the build product (ngmf.jar)

Olaf David, CSU/USDA 54

 A project L
ocal
Bugs 6.1 Task Status change
..
Fixed -> ReadyForTesting
Implemented ->
ReadyForTesting
DE Feature
Requests MG
V R
(6
Milestones )
6.2
Build-lib copy

SC
M
ngmf.jar

Docume
nts

Olaf David, CSU/USDA 55

 Auditable Build Management
(6)
...
// copy the new lib from CB to my local FS (optional)
new File('c:/tmp/ngmf.jar').withOutputStream {
it.write lib.bytes
}

// change the status in all issues in readylistto

// "ReadyforTesting"
readyList.each{ task -> task.status = "ReadyForTesting" }
}

 Copy the build product to the local FS for testing

to skip manual download.
 Change the status for all bugs and features to
“ReadyForTesting” to indicate their processing

Olaf David, CSU/USDA 56

Before invocation and …

Olaf David, CSU/USDA 57

… After …

Olaf David, CSU/USDA 58

.. and After ..

Olaf David, CSU/USDA 59

.. and After ..

Olaf David, CSU/USDA 60

.. and After ..

Olaf David, CSU/USDA 61

Variation (Local Build)
 Build process locally executed
 Upload the build product as a milestone
attachment to the server
 Upload the local build console output to the
milestone as comment
 … rest stays the same.

Olaf David, CSU/USDA 62

Variation (cont.)
...

// build check
if (!readyList.empty) {

sout = new StringBuffer()

serr = new StringBuffer()

// invoke the build.

p = “make –f /home/me/makefile all”.execute()
p.consumeProcessOutput(sout, serr)
result = p.waitFor()

...

 Execute the build on the local machine

 Capture the process output

Olaf David, CSU/USDA 63

 Variation (cont.)
...
String wikiinfo = "!!Milestone info \n" +
"!Milestone Build:\n" +
"* Build Status: ${result}\n"

// create a new milestone in the milestone tracker

mt = prj.trackers.find {it.name == "Milestones"}
milestone = mt.submit(
subject:"MS-1.2", text:wikiinfo, priority:LOW)
if (result==0)
milestone.attach(“/tmp/libngmf.so”)
milestone.comment(text: result == 0 ? sout : serr)
...

 Attach the build product to the milestone and

the console output as comment

Olaf David, CSU/USDA 64

 Examples*
Problem elevation
SCRUM support
SCM commit control
Auditable Build Management
Tracker Status Statistics

* The examples demonstrate the support of

some aspects of the topics above

Olaf David, CSU/USDA 65

Tracker Status Distribution
 Objective
◦ Get the status distribution of all issues in all tracker
of a given project
◦ Visualize results ad-hoc
 Benefit
◦ Get a quick overview about the completeness and
work status across tracker

Olaf David, CSU/USDA 66

Tracker Status Distribution
(1)
def bugs = prj.trackers.find{it.name == "Bug"} // get the bug tr.

def smap= [:]

bugs.statusChoices.each{ smap[it]=0 } // (key:status, value:0)

bugs.issues.each { bug -> smap[bug.status]++ } // fill the map

println smap // print the content

// ["New":20, "Unconfirmed":10, "Verified":5, "Resolved":5, "Reopened":0,

"Closed":20, "ReadyForTesting":3]

 Create a map with each key as status choice

and the value for the # of issues with this
status.

Olaf David, CSU/USDA 67

Tracker Status Distribution
(2)
// data set
def data = new DefaultPieDataset();
smap.each{k, v -> data.setValue"$k", v } // smap to dataset

// chart
def chart = ChartFactory.createPieChart(“Bug Tracker Status", data,
false, true, true)
chart.plot.labelGenerator = new LG(' {0} - {1} ({2}) ')
// frame
def frame = new SwingBuilder().frame(title:’PieChart',
defaultCloseOperation:WC.EXIT_ON_CLOSE) {
widget(new ChartPanel(chart))
}
frame.pack()
frame.show()

 Graph the result using JFreeChart and

SwingBuilder

Olaf David, CSU/USDA 68

Tracker Status Distribution
(3)

Olaf David, CSU/USDA 69

References
 http://cbscript.javaforge.com
◦ CB script examples, download cbscript.jar
 https://codebeamer.com/cb/wiki/18830
◦ Codebeamer API
 http://groovy.codehaus.org
◦ All about Groovy
 https://codebeamer.com
◦ Codebeamer

Olaf David, CSU/USDA 70