International Journal of Scientific Research in _______________________________ Research Paper .

Computer Science and Engineering

Vol.5, Issue.2, pp.81-87, April (2017) E-ISSN: 2320-7639

Investigation of Manual and Automation Testing using Assorted Approaches

Shruti Malve1* and Pradeep Sharma2
1
Dept. of Computer Science, Govt. Holkar Science (Autonomous) College, Indore, India
2
Dept. of Computer Science, Govt. Holkar Science (Autonomous) College, Indore, India
Corresponding Author: [email protected]
Available online at: www.isroset.org
Received 15 Mar 2017, Revised 29th Mar 2017, Accepted 25th Apr 2017, Online 30th Apr 2017
th

Abstract- Testing means “revealing a person's capabilities by putting them under strain; challenging”. Software testing exist a
vast domain, but it may be classify into two way: manual testing and automated testing. In manual testing, test cases are
executed (run) manually without any help from tools/scripts, but with automated testing, test cases are executed (run) with the
help of tools, scripts, and software. Software testing is expensive, labor intensive and consume more time in a software
development life cycle. There was forever a need in software testing to decrease the testing time. The type of testing (manual or
automated) depends on various factors, adding project need, budget, timeline, expertise, and suitability. Three fundamental
factors of any project are of course time, cost, and quality – the objective of any successful project is to diminish the cost and
time required to complete it successfully while maintaining quality output. When it comes to testing, one type may accomplish
this goal better than the other. This paper presents the concept of automation and manual testing and problem with manual
testing and benefit of automatic testing. The main objective of this paper is to conduct a comparative study of Manual and
Automation testing Method based on Accuracy, Bug detection, Efforts, cost and Time.

Keyword-Software Testing, Manual Testing, Automation Testing

I. INTRODUCTION the necessities are frequently changing & huge amount of

regression testing is required to be performed. [1], [2], [3],
The motive of software testing technique is to recognize all [4].
the errors exists in a software product. It is a procedure to
exercise and evolution a system or its component by manual The overall work of Research Paper is divided into five
automatic means to authenticate that it satisfies particular parts, First we describe Related Work, second part conduct
requirements or to recognize differences between expected Methodology, third part present the comparison of software
and actual results. Testing activity can be conducted in to testing techniques, Fourth part presents the Result, and
two ways: Manual testing & automation testing. Any kind of finally fifth part conclude the Research Paper.
software testing can be complete by manually and by
automation tool. Manual testing is in which the testing II. RELATED WORK
activity is executed by testing persons.
SOFTWARE TESTING
Manual Testing is a procedure where in a tester usually Software testing is procedure that ensures the value of the
follows a documented test plan which leads them with in a product to its stakeholders with information about the
set of important test-cases. In a software testing a test-case superiority of the product. It is the method of evaluating a
is group of terms written for particular applications & tester system by manual automatic means to recognize differences
run all these Conditions to make sure the correct between expected and real results. [5][6].
functionality of the software applications. Automated
software testing’s the finest way to increase the Fig. 1. Shows Mode of Testing Based on test execution, is
effectiveness, efficiency and coverage of software testing. divided into two categories.
Automation testing requires considerable amount of 1. Manual Testing
investment for buying the software & compatible hardware 2. Automation testing
resources. Automation testing does what manual testing
does not. Automation testing improves the accuracy & it
saves the time of the tester & organization’s money.
Automation testing is best suited in the environment where

© 2017, IJSRCSE All Rights Reserved 81

 Int. J. Sci. Res. in Computer Science and Engineering Vol-5(2), Apr 2017, E-ISSN: 2320-7639

 White Box Testing – This is a testing method which is

based on information of the interior logic of an
application’s code and also identified as Glass box
Testing. It means its works on interior working of code
in the system.
 Integration Testing – In integration testing, the
individual parts of the system/application are joint and
tested as a group. Integration testing follow unit testing
and precedes system testing. [9]
Fig. 1 Types of software Testing
 System Testing – It is a method to test whole system.
System Testing (ST) is a black-box testing procedure
1. Manual Testing
Manual testing is a testing method in which a person initiate performed to assess the whole system the system's
every test, interacts with it, and compile, analyzes and fulfillment against specified necessities.
reports the outcome. When there is a functioning that the  Unit Testing – A unit may be defined as “the smallest
test-cases is run without tester than Software testing is called possible testable software component”. A unit is the
automated. [7] least testable element of an application. Unit tests focus
Manual testing is a testing procedure in which the test on functionality and reliability, and they typically
engineer organize test-case manually and execute them to consist of a cycle of stand-alone tests.
recognize bugs in the software. It is old way of software  Acceptance Testing – Acceptance testing, a testing
testing. Manual testing is a painstaking movement that procedure performed to find out whether or not the
requires the tester to have a certain set of qualities; to be software system has meet the requirement specifications.
observant, creative, innovative, open-minded, and skillful. The main motive of this test is to estimate the system's
Repetitive manual testing may be not easy to execute on big compliance with the business requirements and verify if
software applications. Manual testing have many problems it is has met the required criteria for delivery to end
like: it is very time consuming procedure, not reusable, has users.
no scripting capability, huge effort required, and some bugs
remain uncovered. [8]. A. Benefits of manual testing
1. If Test Cases have to be run a small number of times it's
We may categorize manual testing in following kinds of more likely to perform manual testing.
testing which is indicates in Fig. 2. These are, 2. It permit the tester to execute additional ad-hoc (random
testing).
3. It requires minimum cost

B. Drawbacks of manual testing

1. Manual tests may be extremely time consuming.
2. Manual testing requires more testers to perform testing.

C. Limitations of Manual Testing

1. Performing manual testing is a boring activity for a
testers. It needs to check each link on the site and every
facts of the product under test.
2. It requires so much time for testing. We have to do
same operations again & again. Regression testing is
being used for checking bugs after making modification
on one fraction of the software have negative effects on
Fig. 2 Types of Manual Testing the other fraction of software.
3. Manual testing is time consuming, error prone and need
 Black Box Testing – –Black-Box testing method is used lot of infrastructure and manpower[10].
to test the functionalities and needs of the
software/system. It doesn’t test the internal fraction of 2. Automation Testing
the system. Black-Box Testing is testing without In compare to manual testing, automated testing automates
awareness of the interior working of the application not only test-case execution, but also test-case generation
under test (AUT). Also known as functional testing or and test result verification. A completely automated testing
input output- driven testing. system is work without any user interference.[11].

© 2017, IJSRCSE All Rights Reserved 82

 Int. J. Sci. Res. in Computer Science and Engineering Vol-5(2), Apr 2017, E-ISSN: 2320-7639

Automated tests perform a series of events without user  Complexity

intervention. It is also defined as a testing a system with With the increase in the number of requirements that
different data sets continuously without intervention of are to be tested, this leads to more and more complexity
human. Simply automated testing is automating the manual which makes the maintenance of test data extremely
testing method currently in use. Automation is the utilize of difficult.
strategies, toolset. Automation testing uses automation
testing tools. The job of automation testing tool is to test the C. Limitations of Automation Testing
intended task and to cut down the human effort. Automation  Automation cannot replace manual testing: Not all
testing is faster than manual testing and it is more reliable testing tasks may be easily automated, especially those
since it performs every test case with precision. [12][13] that require widespread knowledge in a domain.
 Difficulty in maintenance of test automation: Change
A. Benefits of automation testing in technology and evolution of software products leads
 Fast: Automation is less time consuming than manual to difficulty in preserve automated tests.
testing.  Process of test automation needs time to mature:
 Repeatable: Repetition of data or content can be Creating the infrastructure and tests for automation
easily achieved by automation tools in no time. requires time, hence maturity of automation (and related
 Reusable: It can be operated on different version of benefits) requires time
the software.  False expectations: Firms/Companies have impractical
 Load & Performance Testing: No viable manual expectations when it comes to AST with the aim of
alternative exists [14] saving as much cost as possible.
 Programmable: Testers should be check hidden  Inappropriate test automation strategy: An
information at advance level. appropriate policy is hard to decide, hence leading to
 Comprehensive: Testers can make test suites of tests inappropriate strategies that do not allow to utilize the
that cover up each quality in software application. benefits of AST
 Reliability: Automation of test-cases reduces this  Lack of skilled people: To automate tests many skills
risk. The organization can put a procedure in place, are needed (e.g. knowledge of test tools, general
which can make sure that the test scripts written are software development skills, domain and system
correct knowledge[18]
 Cost Effective: In Automation method expenditure of
money is less than manual testing. [15][16] D. Motivation for the Automation of Testing
 Test Coverage: Wider test coverage of application Testing software is often times repetitive in nature. A list of
features [5]. steps is executed, the output is examined, the system is
 Performance: Automated testing reduces the testing restored to its pre-test state, and the list is executed again.
cycle time and hence the time-to-market for a product. Perhaps there is a room full of testers, all running tests on
[17]. the same system. Maybe the testers have programming
skills, or maybe not. In any case, running test cases by hand
B. Drawbacks of Test Automation is error prone, slow, and expensive. Fatigue and ennui have
 It’s Not Easy! the effect of reducing the level of attention to detail paid to
Writing test automation scripts is not an easy task. You test execution and critical test steps may be missed or
really require testers who are knowledgeable in doing haphazardly done. Automated tests, on the other hand, run
this otherwise it will go horribly wrong and you will the same way every time and reduce errors that result from
end up spending even more money and time than if these factors. Automated testing makes possible the kind of
done manually. short release cycles demanded by development processes
 Automation Script Errors such as Extreme Programming (XP). Bugs introduced by
If an error is exist in the test automation script and changes in the code are detected at an earlier stage of the
which is undetected would give wrong output development process, thus saving money and effort. [19]
respectively. In fact you can’t even realize the error The scope of the planned automation tasks also depends on
until the software launches and then falls over the existing and available test ware elements and tools. [20]
 Scope Changes  Increases in productivity: Ideally, an automated test
Every project will have to implement change request suite should run without any user intervention at all...
management. However despite best endeavors there will  Increases in reliability: The best tests are those that are
still be some which get through. easily repeatable, and having to do everything by hand
can undermines that. Automated tests are more reliable

© 2017, IJSRCSE All Rights Reserved 83

 Int. J. Sci. Res. in Computer Science and Engineering Vol-5(2), Apr 2017, E-ISSN: 2320-7639

than tests run manually because they run exactly the same 7 Static testing may detect bugs Dynamic testing may detect
way each time they are run. that dynamic testing may not bugs that static testing may
 Increases in coverage: Automated tests are fast. A
detect and it is low level task. not detect and it is a high
computer isn’t going to go back and forth between a
written script and execution of the tests. The test software level task.
already has those instructions built in. [21]. 8 It is not considered as time It is considered as a time

III METHODOLOGY consuming task. consuming task at all time

because it requires several
My research methodology for this Research Paper is
empirical and fundamental. Empirical methodology is used test-cases for execute.
for observing the fact and ideas of manual and automation 9 Static testing have one Dynamic Testing is not
testing techniques, and fundamental methodology is used for
describing the fundamental facts of both testing techniques. another name that is Dry Run known by any other name.
Testing.
IV.COMPARISON OF SOFTWARE TESTING
TECHNIQUES
B. Difference between box approach method
The software testing is a very wide topic. It includes varies
methods, types and levels or stages of testing. To find an TABLE II. Comparison between Black Box and White Box
appropriate result we analyze following difference table. testing
i. Difference between Static and Dynamic Testing # Black Box Testing White Box Testing
ii. Difference between box approach method
1 Black-box testing is the White box testing is the testing
iii. Manual and Automation Testing
testing method which is procedure which is work in
A. Difference between Static and Dynamic Testing used to test the software internal structure of the software
without the knowledge of and this internal structure needs
TABLE I. Comparison between Static and Dynamic testing
interior structure of knowledgeable tester
Static Testing Dynamic Testing
program/code.
1 In static testing we don’t Dynamic Testing facilities 2 This type of testing is Generally, this type of testing is
execute the code that we code at all time. carried out by testers. carried out by software
2 It means exploratory and It means testing, running, and developers.
reviewing the software. using the software. 3 In Black-Box testing there In White-Box testing there is
3 It definitely comes before It definitely follows after is no requirement of requirement of implementation
dynamic testing. static testing. implementation knowledge
4 This testing is not take more This testing takes more time knowledge.
time because its intention to at all time because its 4 It’s not need to know the Its need to know the
inspect the software or code. intention to execute the Programming Knowledge Programming Knowledge in
software or code. in Black Box Testing. White Box Testing.
5 This testing starts always This testing not starts early in 5 Testing is appropriate on Testing is appropriate on minor
early in the life cycle so it’s the life cycle so its defiantly upper levels of testing like level of testing like Unit
defiantly decrease the cost of expand the cost of System Testing, Testing, Integration testing.
project/Product. project/Product Acceptance testing.
6 It is always consider as least It is always consider as 6 Black box testing means White box testing means
cost effective task. additional cost effective task. functional test or external structural test or interior testing.
testing.
7 In Black Box testing is In White Box testing is mainly

© 2017, IJSRCSE All Rights Reserved 84

 Int. J. Sci. Res. in Computer Science and Engineering Vol-5(2), Apr 2017, E-ISSN: 2320-7639

mainly focus on the focus on the testing of program 7 It is not expensive. It is expensive.
functionality of the system code of the system under test 8 Manual testing needs more Automation testing needs
under test. like code structure, branches, testers because in manual testing some testers because it
all of the testing works are carry needs to execute all
conditions, loops etc.
out manually. testing work.
8 The main goal of this The main goal of White Box
9 It gives low accuracy result. It gives high accuracy
testing to check on what testing to check on how System
result.
functionality is performing is performing. 10 It is considered as low quality. It is considered as high
by the system under test. quality.
11 In this Testing we cannot do In this Testing we can
9 Black-box testing includes White-box testing includes
batch testing. do multiple types of
Functional testing, Structural testing, Logic testing, batch testing.
Behavior testing, Close- Path testing, Loop testing, Code 12 No require of programming It is necessary to know
box testing, so there is no coverage testing, Open box knowledge in Manual Testing. the programming
knowledge in
any requirement of testing, so there is compulsory
Automation Testing
programming knowledge. to know about programming
knowledge.
V.RESULTS
The Result table shows that Automation testing is better
C. Complete Difference between Manual and Automation
than Manual testing because if we think about that the
Testing
accuracy, bug detection percentage, time required, cost
required, effort estimation and people requirement then the
Automated testing is, well, automated. This differs from
graph shows that In every different thing Automation testing
manual testing where a human being is responsible for
is better than Manual testing.
single-handedly testing the functionality of the software in
the way a user would. The benefit of manual testing is that it
allows a human mind to draw insights from a test that might APR: According to People Requirement
otherwise be missed by an automated testing program. [22] ASICCT: According to spending in installation,
configuration, customization and training
TABLE III. Comparison between Manual and Automation testing
Table IV. Comparisons Based on Survey Report
# Manual Testing Automation Testing Keys Manual (in %) Automation (in %)
1 Manual Testing is a procedure Automation Testing is a
Accuracy 68 83.2
which is done manually. procedure which is done
by the automated tools. Bugs detection 36 15
2 In software testing we have to Automation Testing is a percent average.
first start Manual Testing, continuous part of
without starting the manual Manual Testing. Time Required 34 7.8
testing we can’t start Automation Cost requirement 20 8
Testing.
3 In Manual Testing testers are In Automation Testing APR
allowed to do Random Testing to we always test through Cost requirement 20 50
find the Bugs. Running Scripts.
4 Manual Testing requires more Automation testing
ASICCT
time to test test-cases requires less time to test Effort 87 56
test-cases.
People required 32 14
5 It is done without interaction of It is done using tools at
any Tool. all times.
6 Manual Testing would be run Automation Testing is
sequentially. done on different
machines at same time.

© 2017, IJSRCSE All Rights Reserved 85

 Int. J. Sci. Res. in Computer Science and Engineering Vol-5(2), Apr 2017, E-ISSN: 2320-7639

Result Graph Manual(in %)

Automation(in %)

90
80
Keysvalues in percent

70
60
50
40
30
20
10
0

Keys Names
Fig. 3 Result Graph Based on survey

Discussion of Graph: Note:In the above graph and comparion table show the
The entire graph shows the key values, here key means the survay data that conduct during research work.This resealt
properties of Manual and Automation testing. The vertical based on Dollop InfoTech, Labha Technologies, Hammer
InfoTech, Mainfest InfoTech and India InfoTech etc.
axis shows the keys values of Manual and Automation
testing in percent, and the horizontal axis shows the keys
VI. CONCLUSION AND FUTURE SCOPE
name of both testing techniques. The bars show the different
testing approaches, here blue bars indicate manual testing The simulation result and observations in can be
and red bars indicate Automation testing . summarized as follow:
From this graph it is show that the accuracy of Automation  Automation testing gives the result with high accuracy
testing is better than manual testing. It means is gives the than the manual testing.
result with high accuracy. This Graph also show that  Automation testing approach detect the bugs quickly
automation testing approach detect the bugs quickly. Also than the manual testing.
this graph shows that the Automation testing required less  Automation testing required less time to test the test
time to test the test case. Cost requirement is also less for the case than the manual testing.
automation testing according to People Requirement but cast  Cost requirement to test test case according to is testers
requirement is high according to spending in installation, is less but according to the cost of which is spending in
configuration, customization and training for development installation, configuration, customization and training of
of software. Effort estimation is also minimum for the those tools is high than Manual Approach.
automation testing than the manual testing. No of People  Effort estimation is also minimum for the automation
which works for the automation testing is minimum, it testing than the manual testing.
means some people is required for automation testing to test  No of People which works for the automation testing is
the test case. minimum, it means some people is required for
automation testing to test the test case.

© 2017, IJSRCSE All Rights Reserved 86

 Int. J. Sci. Res. in Computer Science and Engineering Vol-5(2), Apr 2017, E-ISSN: 2320-7639

[8] Shaveta, Sachin kumar, Nitika, Snehlata. “Comparative Study of

Form this observation it is shows that Automation testing is Automated Testing Tools: Quick Test Pro and Load Runner”.
International Journal of Computer Science and Information
good idea to test the test cases in software development
Technologies, Vol. 3 (4) , 2012.
process in this new generation word where people wants [9] Sami Torniainen. “IMPROVING EFFECTIVENESS OF
“better results in short time” so automation work is good for REGRESSION TESTING OF TELECOMMUNICATIONS
them than Manual work. SYSTEM SOFTWARE.” Master of Science thesis, HELSINKI
UNIVERSITY OF TECHNOLOGY, February 25th 2008.
This proposed Approach does not shows the low cost for [10] Sunil L. Bangare, Seema Borse, Pallavi S. Bangare, Shital
Nandedkar “Automated Api Testing Approach ” Stes’s Sinhgad
spending in installation, configuration, customization and Academy Of Engineering, Pune-48, Maharashtra India.
training of those tools in automation testing and also some [11] Andreas Leitner, Ilinca Ciupa, Bertrand Meyer, Mark Howard
depletion in automation approaches. So in Future work “Reconciling Manual and Automated Testing: the AutoTest
could involve both these left aspects. And one of the testing Experience”. ETH Zurich CH-8092 Zürich, AXA Rosenberg
fact is also involve that 75% company are using Manual Investment Management LLC Orinda, California 94563.
testing so in future we want that maximum all company are [12] Jagdish Singh, Monika Sharma.” A Comprehensive Review of
Web-based Automation Testing Tools”. International Journal of
used Automation approaches for their work of test to test Innovative Research in Computer and Communication
cases. So in this thesis we show the good properties of Engineering, Vol. 3(10), October 2015
Automation approach to show the company that automation [13] Prof. (Dr.) V. N. MAURYA , Er. RAJENDER KUMAR .
is good for them work and use it. “Analytical Study on Manual vs. Automated Testing Using with
Simplistic Cost Model ”. International Journal of Electronics and
Electrical Engineering, vol .2 (1), 2012.
ACKNOWLEDGMENT [14] Joe Fernandes (Oracle) Alex Di Fonzo (Synchronoss
Technologies “When to Automate Your Testing (and When Not
To)”.oracle 30 year.
I would like to express my gratitude to my supervisor [15] DAYU GUAN (Steven). ”Manual to Automated Testing.” Master
Dr.Pradeep Sharma for the useful comments, remarks and of Information Management thesis, Victoria Business School,
engagement through the learning process of this research. Victoria University of Wellington, October 2014.
Furthermore I would like to thank Prof.Shivlal Mewada for [16] Linda G. Hayes. The Automated Testing Handbook.
introducing me to the topic as well for the support on the [17] Amit Srivastava. ” Test Automation and Software Development.”
TATA Research development and design center, December 2002.
way. Also, I like to thank the participants in my survey, who
[18] Dudekula Mohammad Rafi, Katam Reddy Kiran Moses, Kai
have willingly shared their precious time during the process Petersen, Mika V. M¨antyl¨.” Benefit and Limitations of
of interviewing. I would like to thank my loved ones, who Automated Software Testing”. Systematic Literature Review and
have supported me throughout entire process, both by Practitioner Survey, Sweden.
keeping me harmonious and helping me putting pieces [19] Dr. Mike Bartley,” Achieving business benefits through
together. automated software testing”, Founder and CEO, TVS
([email protected])
[20] Douglas Hoffman. “Test Automation Architectures: Planning for
REFERENCE Test Automation” 24646 Heather Heights Place Saratoga,
California 95070-9710, 1999.
[1] Harpreet Kaur, Dr.Gagan Gupta. “Comparative Study of Automated [21] David Wayne Jansing .”Enhancing the Effectiveness of Software
Testing Tools: Selenium, Quick Test Professional and Test Automation.” Master of Science thesis, University of New
Testcomplete”. Int. Journal of Engineering Research and York Polytechnic Institute Utica, New York, December 2015.
Applications, vol, 3(5), pp.1739-1743, 2013, [22]Smart Bear Software,” Manual Tester’s Guide to Automated
[2] Vishawjyoti, Sachin Sharma. “STUDY AND ANALYSIS OF Testing” Inc. 100 Cummings Center, Suite 234N Beverly, 2013.
AUTOMATION TESTING TECHNIQUES”. Journal of Global
Research in Computer Science, Vol 3(12), 2012.
[3] VIVEK KUMAR. “COMPARISON OF MANUAL AND
AUTOMATION TESTING”. International Journal of Research in
Science And Technology, Vol. 1(V), 2012.
[4] Antawan Holmes and Marc Kellogg. “Automating Functional
Tests Using Selenium”. Proceedings of AGILE 2006 Conference
(AGILE'06), 2006,ISBN : 0-7695-2562-8/06 $20.00 IEEE.
[5] R. M. Sharma.” Quantitative Analysis of Automation and Manual
Testing.” International Journal of Engineering and Innovative
Technology (IJEIT), Vol. 4(1), 2014.
[6]Er. Rajender Bathla, Er. Shallu Bathla. “Innovative Approaches of
Automated tools in Software testing & current technology as
compared to Manual testing ”. Global Journal of Enterprise
Information System, Vol-1(1) , 2009.
[7] Bc. Dávid Chmurcˇiak. “Automation of regression testing of web
applications.” DIPLOMA THESIS, MASARYK UNIVERSITY
FACULTY OF INFORMATICS , Brno, spring 2013.

© 2017, IJSRCSE All Rights Reserved 87