GUI Testing On A Product Suite Using
On-The-Shelf Testing Software
Project Guide: Mrs. Poonam Singhal
Submitted to: ABES Institute Of Technology
Anubhav Jain
Enrl. No:2007CS007
B.Tech CSE (IVth yr)
Inst. Code: 290
ACKNOWLEDGEMENT
I am heartily thankful to my supervisor, Mrs. Poonam
Singhal, whose encouragement, guidance and support
from the initial to the final level enabled me to develop an
understanding of the subject.
Lastly, I offer my regards and respect to my project whose
incessant support in every respect during the the project
helped me complete the projected targets and garner a
treasure of invaluable industry knowledge and experience.
Anubhav Jain
INTRODUCTION
The product suite that I worked on consisted of three sub-
tools.These tools have been explained as following:
PAS/GTE: PAS (PDK Automation System) and GTE
(Graphic Textual Editor) is the first tool set which help
in the automation of PDK (Process Design
Kits).PDK’s are sets which contain all the elementary
electronic devices used to make microprocessor
circuits.
STeP: STEP (System for Testing PDKs) is the
second sub-tool used to test the PDK against a
variety of parameters for consistency. Using this
tool,PDKs using disparate architectures can also be
compared against one another.
DEFT: DEFT (Development Environment For
Technology Sub files) if the third sub-tool in this suite.
A tech file is an integral component of a PDK which
contains all the constraints which are to be
considered while designing a circuit. It is an eclipse
plug-in.
The team that I worked with was handed the responsibility
to make new builds for these three tools. Since the basic
functionality for every system and sub-system inside each-
tool is pre-defined, the test cases that I was developing
were specially designed keeping in mind regression
testing.
REGRESSION TESTING:
Regression testing is any type of software testing that
seeks to uncover software errors by partially retesting a
modified program. The intent of regression testing is to
provide a general assurance that no additional errors were
introduced in the process of fixing other problems.
Regression testing is commonly used to test the system
efficiently by systematically selecting the appropriate
minimum suite of tests needed to adequately cover the
affected change. Common methods of regression testing
include rerunning previously run tests and checking
whether previously fixed faults have re-emerged. One of
the main reasons for regression testing is that it's often
extremely difficult for a programmer to figure out how a
change in one part of the software will echo in other parts
of the software.
Regression testing is an integral part of the extreme
programming software development method. In this
method, design documents are replaced by extensive,
repeatable, and automated testing of the entire software
package at every stage in the software development cycle.
In the corporate world, regression testing has traditionally
been performed by a software quality assurance team
after the development team has completed work.
However, defects found at this stage are the most costly to
fix. This problem is being addressed by the rise of unit
testing. Although developers have always written test
cases as part of the development cycle, these test cases
have generally been either functional tests or unit tests
that verify only intended outcomes. Developer testing
compels a developer to focus on unit testing and to
include both positive and negative test cases
RESEARCH DESIGN
GUI (Graphical User Interface) testing done here has used
a on-the-shelf software known as QF-Test. This software
communicates with the jar file in the java component of the
tools tested in the suite. Since these tools have been
programmed in java, QF-Test was found to be sufficiently
adequate for the purposes of the stipulated testing. This
software creates an event-component mapping of every
action that is performed on the tools being tested. The
JDK of the concerned tool has to be instrumented with the
QF-Test module.
QF-Test from Quality First Software is a cross-platform
software tool for the GUI test automation limited to
Java/Swing, SWT, Eclipse plug-ins and RCP applications,
ULC, Java applets, Java Web Start and cross-browser test
automation of web-based applications.
QF-Test enables regression and load testing and runs on
Windows and all major UNIX systems. Its commercial
application is primarily performed by testers and
developers in the field of Quality Assurance. Since
December 2008 a webtest add-on is available which
allows for test automation of browser-based GUIs in
addition to its Java GUI testing functionality.
Its capture/replay function enables recording of tests for
beginners, while modularization allows for creating large
test suites in a concise arrangement. For the advanced
user who requires even more control over his application,
the tool offers access to internal program structures
through the standard scripting languages Jython, the Java
implementation of the popular Python language, and
Groovy.
The tool also offers a batch mode, allowing to run tests
unattended and then generate XML or HTML reports.
Thus the tool can be integrated into existing test
management or build/test frameworks like Ant or Maven.
As in the development of any software product, these tools
are continuously updated and improved according to user
needs and developer feedbacks.I was tasked to develop
QF test-cases and then develop QF-Test replay files for
further regression testing.
Quality First Software GmbH (QFS) is a German software
development company located south of Munich. It was
founded in 2001 by Managing Director Gregor Schmid and
specializes in quality assurance for the Java platform.
The name "Quality First Software" not only refers to quality
assurance in general, but also stresses the high demands
QFS makes on the quality of their own products.
The first product developed by QFS is QF-Test, a
professional tool for the automated testing of Java
applications with a graphical user interface. Version 1.0 of
qftestJUI, the predecessor of QF-Test, was launched in
October 2002. Since then, more than 1500 licenses have
been sold to over 500 customers from 40 countries.
Customers include blue-chips like BEA, HP, IBM, NEC,
and Philips, as well as medium and small companies,
academic and public institutions, such as the County of
Aarhus, Denmark or the federal tax and revenue bureaus
of Niedersachsen, Germany.
Since version 1.0 QF-Test has evolved continuously.
Version 2.0 with support for testing SWT/Eclipse based
applications was released in December 2006. After a
medium upgrade in 2007 and a long, thorough beta
period, version 3.0 with cross-browser web testing support
was released end of 2008
The department that I worked in was Custom IC(CIC)
dept. As the name suggests, this dept looked after IC’s
which were custom designed and special constraints were
given by the user for designing the product.
COMPANY PROFILE
Cadence Design Systems, Inc is an electronic design
automation (EDA) software and engineering services
company, founded in 1988 by the merger of SDA Systems
and ECAD, Inc. For years it had been the largest company
in the EDA industry.
Cadence, headquartered in San Jose, California, is one of
the world's leading suppliers of electronic design
technologies and engineering services in the electronic
design automation (EDA) industry. The primary corporate
product is software used to design chips and printed circuit
boards.
Cadence employs approximately 5,000 people and
reported 2008 revenues of approximately $1.04 billion. In
November 2007 Cadence was named one of the 50 Best
Places to Work in Silicon Valley by San Jose Magazine.
Cadence's product offerings are targeted at various types
of design and verification tasks which include:
Virtuoso Platform - Tools for designing full-custom
integrated circuits; includes schematic entry,
behavioral modeling (Verilog-AMS), circuit simulation,
full custom layout, physical verification, extraction and
back-annotation. Used mainly for analog, mixed-
signal, RF, and standard-cell designs, but also
memory and FPGA designs.
Encounter Platform - Tools for creation of digital
integrated circuits. This includes floorplanning,
synthesis, test, and place and route . Typically a
digital design starts from Verilog netlists..
Incisive Platform - Tools for simulation and functional
verification of RTL including Verilog, VHDL and
SystemC based models. Includes formal verification,
formal equivalence checking ,hardware acceleration ,
and emulation.
Verfication IP (VIP) Cadence provides the broadest
set of commercial VIP available with over 30 protocols
in its VIP Portfolio. They includeAMBA, PCI Express,
USB, SATA, OCP, SAS, SATA, MIPI and many
others. Cadence VIP also provides the unique
Compliance Managemnt System(CMS) to automate
protocol compliance verification.
Integration Optimised IP (Design IP) Cadence offers
Vertically Integrated IP, inclusive of Digital Controller,
Serdes Layer, and Device Driver. Protocols supported
include USB, DDR, PCI-Express, 10G-40G Ethernet,
and On Chip Bus Fabric.
Allegro Platform - Tools for co-design ofintegrated
circuits, packages, andPCBs.
OrCad/Pspice - Tools for smaller design teams and
individual PCB designers.
In addition to EDA software, Cadence provides contracted
methodology and design services as well as silicon design
IP, and has a program to make it easier for other EDA
software to interoperate with the company's tools.
Cadence Research Laboratories (CRL) was established in
1993 in Berkeley to focus on advanced research in
Electronic Design Automation. We study the mathematical
and algorithmic foundations for modeling, analyzing, and
optimizing complex integrated systems. Our mission is
broadly stated—Increase external visibility of Cadence as
a technology leader, and help make its products
successful in the marketplace.
We are actively engaged with Cadence's business units.
Our contributions include developing prototype
components for proving new concepts, consulting on
operational and strategic decisions, generating code for
products, and directing customer engagements to discover
new challenges and receive feedback on new ideas.
We are also involved in the external research community
through regular publications of scientific papers and active
leadership on conference committees and in other
organizations. CRL maintains a variety of collaborations
with other research institutions and universities; we are
particularly involved with the University of California at
Berkeley.
To keep pace with market demand for more performance
and functionality in today’s mobile phones, digital
cameras, computers, automotive systems and other
electronics products, manufacturers pack billions of
transistors onto a single chip. This massive integration
parallels the shift to ever-smaller process geometries,
where the chip’s transistors and other physical features
can be smaller than the wavelength of light used to print
them.
Designing and manufacturing semiconductor devices with
such phenomenal scale, complexity and technological
challenges would not be possible without electronic design
automation (EDA). It is essential for everything from
verifying that the myriad transistors do what the designer
intended to dealing with physical effects on electrons
traveling miles of wires with widths sometimes measuring
less than 100 nanometers.
Cadence Design Systems is the world's leading EDA
company. Cadence customers use our software,
hardware, and services to overcome a range of technical
and economic hurdles.
Our technologies help customers create mobile devices
with longer battery life. Designers of ICs for game
consoles and other consumer electronics speed their
products to market using our hardware simulators to run
software on a ‘virtual’ chip—long before the actual chip
exists. We bridge the traditional gap between chip
designers and fabrication facilities, so that manufacturing
challenges can be addressed early in the design stage.
And our custom IC design platform enables designers to
harmonize the divergent worlds of analog and digital
design to create some of the most advanced mixed-signal
system on chip (SoC) designs. These are just a few of the
many essential Cadence solutions that drive the success
of leading IC and electronic systems companies.
ANALYSIS & DATA INTERPRETATION
Majority of the testing data given were dummy files upon
which the tools could be operated and the desired effects
be tested and registered. Since there were 3 tools in the
suite, every tool had a different test suite. Inside every test
suite, there are multiple test cases and inside every test
case, there are multiple tests. These tests are of 2
varieties:
1. Internal GUI Tests: These kind of tests used the
tools which come in-built in the QF-Test software.
These are very primitive yet useful tools which are
used to cross-check things like text,images,directory
structures,etc.
2. Script Tests: These are the more complex, user-
oriented tests which were custom designed to meet
very specific features. Just to state an example, the
developers wanted that the date be checked in every
GTE file as to whether the date is the present one or
the date of origin of file. So using the internal
programming features of Jython, language used in
developing scripts, it was checked that the date
updates itself constantly.
These tests, after being verified are stored in the
repository for future use.
CONCLUSION
It is safe to say that I achieved all the objectives set out for
me at the beginning of my term at Cadence Design
Systems as a trainee. All the three test suites were
developed, tested and reviewed by the whole team. For
this purpose, the whole team had a meeting in which a
demo of the test suites, their functionality was given and
the intricacies of the entire system was disbursed into the
whole team so that troubleshooting could be made easy.
The test suites will now be run whenever there is a change
in the software suite or a new build is developed. This
saves the time to manually test the soft wares again and
again and automates the whole process.
My efforts were commended by the whole team and my
contribution lauded.
