Perl Testing
                       Consideration
                          (seen from other languages)


                               Masaki Nakagawa

                 YAPC::Asia 2011 Tokyo (yapcasia.org/2011/talk/80)



2011   10   14
Profile
                 • Masaki Nakagawa
                 • @ikasam_a
                 • metacpan.org/author/MASAKI
                 • MC of Yokohama.pm (yokohama.pm.org)
                 • github.com/masaki
                 • socialbios.com/ikasam_a
                 • “Pushing the Team B”
2011   10   14
Thank you !

                 ameblo.jp/yuki-kashiwagi-we/image-11002817849-11452563522.html
2011   10   14
Works
                 • Embedded Development w/ C++
                 • WebApp. Development w/ Ruby
                 • Project Management (small)
                 • New Business Development
                  •   www.ricoh.co.jp/ucs/

                  •   www.ricoh.co.jp/ucs/P3000/



2011   10   14
Embedded Development

                 • “Hardware Driven”
                 • “The Waterfall”
                 • Development Process
                 • QA Process
                 • Systematic Testing
2011   10   14
Classification of Testing


                  Perspective   Target

                     How         What




2011   10   14
Perspective
                      Developer or User (Customer)


                 • Developer Testing
                 • Acceptance Testing


2011   10   14
Target
                               One or More


                 • Unit Testing
                 • Integration Testing


2011   10   14
How
                        Techniques for writing tests


                 • Black Box Testing
                 • White Box Testing


2011   10   14
What
                           the Purpose of testing

                 • Functional Testing
                 • Non-Functional Testing
                  • Performance / Stress / Usability / ...

2011   10   14
Testing Framework


                 1. Helping to write tests
                 2. Executing tests
                 3. Reporting test results




2011   10   14
Ruby Testing




                 www.atmarkit.co.jp/fcoding/rails/articles/passionate/07/passionate07a.html
                                 survey.hamptoncatlin.com/survey/stats
2011   10   14
e.g.) RSpec

                 1. describe ... it ...
                 2. $ rspec ...
                 3. Spec::Runner::Formatter




2011   10   14
e.g.) Cucumber

                 1. Given ... When ... Then ...
                 2. $ cucumber ...
                 3. Cucumber::Formatter




2011   10   14
Many Frameworks


2011   10   14
Perl Testing

                 1. Test::More, Test::Class,
                    Test::Base, Test::Spec, ...
                 2. $ prove ... (TAP::Harness)
                 3. TAP (TAP::Formatter)



2011   10   14
Perl has TAP

2011   10   14
TAP
                 • testanything.org
                  • “the Test Anything Protocol”
                  • “is a simple text-based interface”
        1..4
        ok 1 -     Input file opened
        not ok     2 - First line of the input valid
        ok 3 -     Read the rest of the file
        not ok     4 - Summarized correctly # TODO Not written yet


                             testanything.org/wiki/index.php/Main_Page
2011   10   14
Perl <3 TAP
                 • From any test description
                  •   e.g.) Test::Class, Test::Base, Test::Spec

                 • Convert results to non-TAP format
                  •   e.g.) TAP::Formatter::JUnit

                 • Easy to write test helpers
                  •   e.g.) Test::Exception, Test::Deep, ...


2011   10   14
Rails Testing




                  gihyo.jp/magazine/wdpress/archive/2011/vol61
2011   10   14
Rails Testing
                 •   Perspective
                     •   UnitTest, End-to-End

                 •   Frameworks
                     •   Test::Unit (+ Shoulda), RSpec, Cucumber

                 •   Browser Integration
                     •   Capybara (rack-test, Selenium, Envjs, ...)

                 •   Test Double
                     •   rspec-mocks, rr, WebMock

2011   10   14
Catalyst

                 • Catalyst::Test
                 • Test::WWW::Mechanize::Catalyst
                 • Test::WWW::Selenium::Catalyst

2011   10   14
Plack


                 • Plack::Test
                 • Test::WWW::Mechanize::PSGI


2011   10   14
Browser Integration

                 • WWW:: + WAF Customization
                  • WWW::Mechanize
                  • WWW::Selenium (Selenium-RC)
                 • not yet for testing
                  • WWW::HtmlUnit
                  • Selenium::Remote::Driver (WebDriver)
2011   10   14
Test Double
                         xunitpatterns.com/Test%20Double.html




                 • Mock
                  • Test::MockObject
                  • Test::Mock::Recorder
                 • Stub
                  • Test::Mock::Guard
                  • Test::Mock::LWP::Dispatch
2011   10   14
Test Double
                                  xunitpatterns.com/Test%20Double.html


                 •   Test::Double (under construction)

                     •   0.01 released !

                         •   metacpan.org/release/Test-Double

                     •   github.com/masaki/Test-Double

                     •   Mocks and Stubs

                         •   mock out w/ mocha style

                         •   stub out w/ rr style

                     •   Collaborators Welcome!!!
2011   10   14
Test Double
                          xunitpatterns.com/Test%20Double.html




                 • Spy
                  • DBD::Mock
                 • Fake
                  • Test::mysqld
                  • Test::TCP
2011   10   14
Descriptive Frameworks

                 • Test::More
                 • Test::Class (xUnit style)
                 • Test::Declare / Test::More + “subtest”
                 • Test::Spec (RSpec 1 compat.)
                 • Test::Base (Parameterized Tests)
                 • Test::Cukes / Test::BDD::Cucumber
                   (Cucumber clone)
2011   10   14
Weakness
                 • Expectation and Verification
                  • vs. rr, rspec-mocks, mocha, ...
                 • Browser Integration API
                  • vs. Capybara, Webrat, ...
                 • Naming of Test Double modules
                  • Test::Mock::Guard is “Stub” module
2011   10   14
Comparision
                 • Pros
                  • TAP-based Test Description
                  • Many Helpful Modules
                 • Cons
                  • Non-Uniform Browser Integration
                  • Poor Expectation and Verification
2011   10   14
Conclusions
                 • About Testing
                  • 4 Classifications
                  • 3 Elements in Frameworks
                 • Perl and Ruby/Rails Testing Environments
                  • Perl has TAP and Many Great Modules
                  • Learn and Steal MORE from Ruby/Rails
                    • Write More Descriptively
2011   10   14
Thank you!
                 Questions?


2011   10   14

Perl Testing Consideration (seen from other languages)

  • 1.
    Perl Testing Consideration (seen from other languages) Masaki Nakagawa YAPC::Asia 2011 Tokyo (yapcasia.org/2011/talk/80) 2011 10 14
  • 2.
    Profile • Masaki Nakagawa • @ikasam_a • metacpan.org/author/MASAKI • MC of Yokohama.pm (yokohama.pm.org) • github.com/masaki • socialbios.com/ikasam_a • “Pushing the Team B” 2011 10 14
  • 3.
    Thank you ! ameblo.jp/yuki-kashiwagi-we/image-11002817849-11452563522.html 2011 10 14
  • 4.
    Works • Embedded Development w/ C++ • WebApp. Development w/ Ruby • Project Management (small) • New Business Development • www.ricoh.co.jp/ucs/ • www.ricoh.co.jp/ucs/P3000/ 2011 10 14
  • 5.
    Embedded Development • “Hardware Driven” • “The Waterfall” • Development Process • QA Process • Systematic Testing 2011 10 14
  • 6.
    Classification of Testing Perspective Target How What 2011 10 14
  • 7.
    Perspective Developer or User (Customer) • Developer Testing • Acceptance Testing 2011 10 14
  • 8.
    Target One or More • Unit Testing • Integration Testing 2011 10 14
  • 9.
    How Techniques for writing tests • Black Box Testing • White Box Testing 2011 10 14
  • 10.
    What the Purpose of testing • Functional Testing • Non-Functional Testing • Performance / Stress / Usability / ... 2011 10 14
  • 11.
    Testing Framework 1. Helping to write tests 2. Executing tests 3. Reporting test results 2011 10 14
  • 12.
    Ruby Testing www.atmarkit.co.jp/fcoding/rails/articles/passionate/07/passionate07a.html survey.hamptoncatlin.com/survey/stats 2011 10 14
  • 13.
    e.g.) RSpec 1. describe ... it ... 2. $ rspec ... 3. Spec::Runner::Formatter 2011 10 14
  • 14.
    e.g.) Cucumber 1. Given ... When ... Then ... 2. $ cucumber ... 3. Cucumber::Formatter 2011 10 14
  • 15.
  • 16.
    Perl Testing 1. Test::More, Test::Class, Test::Base, Test::Spec, ... 2. $ prove ... (TAP::Harness) 3. TAP (TAP::Formatter) 2011 10 14
  • 17.
  • 18.
    TAP • testanything.org • “the Test Anything Protocol” • “is a simple text-based interface” 1..4 ok 1 - Input file opened not ok 2 - First line of the input valid ok 3 - Read the rest of the file not ok 4 - Summarized correctly # TODO Not written yet testanything.org/wiki/index.php/Main_Page 2011 10 14
  • 19.
    Perl <3 TAP • From any test description • e.g.) Test::Class, Test::Base, Test::Spec • Convert results to non-TAP format • e.g.) TAP::Formatter::JUnit • Easy to write test helpers • e.g.) Test::Exception, Test::Deep, ... 2011 10 14
  • 20.
    Rails Testing gihyo.jp/magazine/wdpress/archive/2011/vol61 2011 10 14
  • 21.
    Rails Testing • Perspective • UnitTest, End-to-End • Frameworks • Test::Unit (+ Shoulda), RSpec, Cucumber • Browser Integration • Capybara (rack-test, Selenium, Envjs, ...) • Test Double • rspec-mocks, rr, WebMock 2011 10 14
  • 22.
    Catalyst • Catalyst::Test • Test::WWW::Mechanize::Catalyst • Test::WWW::Selenium::Catalyst 2011 10 14
  • 23.
    Plack • Plack::Test • Test::WWW::Mechanize::PSGI 2011 10 14
  • 24.
    Browser Integration • WWW:: + WAF Customization • WWW::Mechanize • WWW::Selenium (Selenium-RC) • not yet for testing • WWW::HtmlUnit • Selenium::Remote::Driver (WebDriver) 2011 10 14
  • 25.
    Test Double xunitpatterns.com/Test%20Double.html • Mock • Test::MockObject • Test::Mock::Recorder • Stub • Test::Mock::Guard • Test::Mock::LWP::Dispatch 2011 10 14
  • 26.
    Test Double xunitpatterns.com/Test%20Double.html • Test::Double (under construction) • 0.01 released ! • metacpan.org/release/Test-Double • github.com/masaki/Test-Double • Mocks and Stubs • mock out w/ mocha style • stub out w/ rr style • Collaborators Welcome!!! 2011 10 14
  • 27.
    Test Double xunitpatterns.com/Test%20Double.html • Spy • DBD::Mock • Fake • Test::mysqld • Test::TCP 2011 10 14
  • 28.
    Descriptive Frameworks • Test::More • Test::Class (xUnit style) • Test::Declare / Test::More + “subtest” • Test::Spec (RSpec 1 compat.) • Test::Base (Parameterized Tests) • Test::Cukes / Test::BDD::Cucumber (Cucumber clone) 2011 10 14
  • 29.
    Weakness • Expectation and Verification • vs. rr, rspec-mocks, mocha, ... • Browser Integration API • vs. Capybara, Webrat, ... • Naming of Test Double modules • Test::Mock::Guard is “Stub” module 2011 10 14
  • 30.
    Comparision • Pros • TAP-based Test Description • Many Helpful Modules • Cons • Non-Uniform Browser Integration • Poor Expectation and Verification 2011 10 14
  • 31.
    Conclusions • About Testing • 4 Classifications • 3 Elements in Frameworks • Perl and Ruby/Rails Testing Environments • Perl has TAP and Many Great Modules • Learn and Steal MORE from Ruby/Rails • Write More Descriptively 2011 10 14
  • 32.
    Thank you! Questions? 2011 10 14