keskiviikko 30. marraskuuta 2011

UI Test Automation with Watir

Watir is lightweight browser driver built in Ruby. Watir holds ATDD ideology where tests are easy to read and maintain. Tests are based to scenarios written with Gherkin. Step definitions will contain Ruby code which drives web browser like normal user e.g. open browser, goto web page, click some button, write text to fields and select checkbox. Very cool indeed. Here some example lines:

browser = Watir::Browser.new
browser.goto 'http://www.google.com'
browser.button(:name => 'submit').click
browser.text_field(:name => 'somefield').set "some text"
browser.checkbox(:value => 'Watir').set
See more examples here

Pair coding

We continued practices from last week. Our instructor has been done scenarios and code implementation to ready so we could test something. Good job. From those scenarios we created skeletons to step definitions which we started to complete with test implementations. Implementation was not so trivial even Ruby syntax look readable. Our computer was the Mac and seems it was not so familiar most of us ;) Also Ruby syntax and how to define methods with it etc. were uncertain. Implementation happened quite well after we get familiar with basics of Mac and Ruby. Purpose of this coding dojo were to test user account creation form. Tests should ensure that the required text fields are not empty after submit button click. After button click the test should also check that user account really exists in system.

Validation of required fields gives an error message to the screen which contains number of missing fields and under it the name list of missing fields. We could use the error message text as a criterion for the test result. In this way we cannot get information which fields doesn't meet the validation rules. This approach could be harmful in later if validation of fields are changed so that some field are not required anymore e.g. first name is optional. If not required and some required field has not been filled after submit process then we need in the test to find out which fields are required and not. So the better solution could be use the name list of missing fields to solve test result.


During implementation we found that table lookup were quite heavy and hard to done. We decide that system under test need to be updated for the test methods. We needed user identifier for each table row to find user account information from table easily to ensure that user data is really stored in the system. After the changes test implementation were easy to implement. This is a good idea to keep in the mind when making tests. It is pointless to implement complicated tests if the changes are easier to implement to the system under test.

This coding dojo was very good. I got many ideas how to improve my own working. I decide to study Watin soon. Watin is .NET implementation of Watir. Here is easy example:


[Test]
public void SearchForWatiNOnGoogle()
{
  using (var browser = new IE("http://www.google.com"))
  {
    browser.TextField(Find.ByName("q")).TypeText("WatiN");
    browser.Button(Find.ByName("btnG")).Click();
  
    Assert.IsTrue(browser.ContainsText("WatiN"));
  }
}

tiistai 22. marraskuuta 2011

Coding Dojo 1 Done Done


In last friday was first coding dojo where topics were Cucumber (and Gherkin) and ATDD (Acceptance Test Driven Development). Class was split to two groups where expert teams were introduced their topic. Expert team where I belong to introduced ATDD (see slides in Finnish).

ATDD

ATDD is team practice which concentrate to external quality and it contains TDD (Test Driven Development). Specification workshop consist of project key members e.g. developers, testers, product owner, architect and technical writers. These members can vary during software project. During specification workshop user stories are selected and specifications are written down. These specifications will be implemented to executable specifications. Next phase is feature implementation so tests can be executed successfully. Implementation will follow TDD-cycle. Last phase is demonstration and customer approval. There are many ATDD benefits but few to mention this practice will improve communication between team members and enable fast feedback. The customer is responsible for certification and approval of functionalities which facilitates team work. There are also many problems to use this practice. Customer teaching to test driven development could be tricky and customer doesn't have any resources or knowledge. Writing automatically executable tests could be hard e.g. embedded systems. There are many tools in different software environments e.g. FIT, Robot and Cucumber.


Cucumber and Gherkin

Other expert team highlights Cucumber and Gherkin. Cucumber is ruby based BDD tool which executes plain-text functional descriptions as automated tests. Cucumber can be used to “test” code written in Ruby or other languages including but not limited to Java, C# and Python. Cucumber understand Gherkin language. It is a Business Readable, Domain Specific Language that lets you describe software’s behavior without detailing how that behavior is implemented. These functional descriptions are written in .feature files. Each feature to own file. Communication is better if people can use their spoken language. Cucumber supports currently over 40 spoken languages and steadily growing.

Example of functional description from wiki pages:
Feature: Serve coffee
  In order to earn money
  Customers should be able to
  buy coffee at all times

  Scenario: Buy last coffee
   Given there are 1 coffees left in the machine
   And I have deposited 1$
   When I press the coffee button
   Then I should be served a coffee
NOTE: Keywords used to describe feature and scenario are bold ('But' is also supported keyword but not used in this example). These keywords should be written case sensitive and ':' should be added after Feature and Scenario.

From theory to practice

One of our course instructors act as business owner. He need few user account management features for Bowling company's web site e.g. create new user account, list all users and filter results. Our group ask questions to clarify requirements. We practice Gherkin description by pair coding with text editor. This pair is only allowed to speak freely. Other members can ask open questions to present own ideas but no more. One pair member was changed after 5 minutes and this way every group member can participate to pair coding.

We didn't implement step definitions because time of coding dojo was limited. Steps after writing feature scenarios aren't now familiar but I think this was very good way rather than implementing functionality. It seems to be quite simple to continue by my self after looking documentation from Cucumber's web page. Cucumber (and Gherkin) has been documented quite well and Internet offers a lot information.

We didn't get solved last feature due time was over. But here is my answer how it could be resolved. I am not sure can table include table and haven't test it yet. If you have better solution please be free and comment.
Feature: List users
  In order to modify user informations 
  Owner should be able to list users
  with free query filter
  Scenario: List users
    Given system have users <users>
    | username | fname | lname | picture |
    | foo      | bar   | baz   | nil     |
    | n00b     | l33t  | fff   | n00b.jpg|
    When I set search filter as <filter>
    | filter |
    | nil    |
    | foo    |
    Then I should see <result>
    | filter | users                                |
    | nil    || username | fname | lname | picture ||
    |        || foo      | bar   | baz   | nil     ||
    |        || n00b     | l33t  | fff   | n00b.jpg||
    | foo    || foo      | bar   | baz   | nil     ||

Conclusion

ATDD is powerful team practice and can managed with tools e.g. Cucumber and Gherkin. This combination can guarantee all features are done like customer wants to. There could be problems to take in use this team practice in software projects because it could make customer, managers, developers and other team members to think that this is yet another complicated practice. Actually it should enable all communicate same language during whole project and remove uncomfortably feelings has every requirement done done.






perjantai 4. marraskuuta 2011

First steps

I have been software developer four years. Software projects has been varying from mobile to client/server solutions. On every project I have been thinking how to be sure that all requirements has been implemented correctly. I haven't got enough knowledge of software testing so in few later projects I focused little bit more to unit testing during development. I don't mean Test Driven Developement. No. Just some desperate try to get some reliability to development. Maybe I should try TDD. But anyway, I realize that I can be more efective and get production more accuracy even after huge changes with unit test cases... but those do not cover everything. I still miss some practice of how to test UI and how to be sure that all requirements are done (and even right).

Now I have been enrolled to the Software Testing course in University of Turku. After this course I should have some clue of Acceptance Test Driven Development (ATDD) with Cucumber, UI Test Automation, Exploratory testing of a web application and Session based testing of an embedded device application. These things are handled in Coding and Testing Dojo's during end of this year where expert groups teach some of previous topic to others. I pick up the ATDD which seems to be hot and huge topic. This blog will be my learning diary where to collect information, links and coding dojo feelings overall from topics in Software Testing. Hopefully this blog might help someone.

Some links to start with ATDD: