Angry Birds

Our first quest lecturer was Raine Mäki from Rovio with topic testing a cross-platform mobile game. Angry Birds game is most known product from Rovio. Even my three and half year old son knows what it is and can play it. His grandmother buy in last year bird toy plush. That is fantastic concept which works from young to old. Lecture was good and gives real life view. He admits they have still lot improvements in their testing environments and processes. Actually engineers are needed. Lot testing works are done by manual. Markets are growing fast and automation is needed. There are so many platforms, OS and variations of those that every bug cannot track in proprietary time. But fatal bugs need still to be stopped because it can ruin this fantastic brand. Games are still main competence but other products growing e.g. toys and movies. Hopefully this growing continues, keeps doing funny products and for all that they can keep feet’s on the ground.

Session based testing

We had again very nice overview of Session Based Testing and tools: Session Tester and RapidReporter. Tools were quite simple to use and light weight. Basically those are timer, documentation and presentation helpers. Metrics and documentation during testing can be formed to html documentation.

We start our final dojo by creating smaller testing groups. Each group got own device either computer or Android tablet/phone. Groups with computer should test University of Turku web site and other groups test Fuugo application. I start in Fuugo testing group. At the start Fuugo testing groups start to explore application how it works etc. After 15 minutes all know what to test and quite well how. We decided session length should be 45 minutes. My group decides to test main view search functionality. We tried everything what comes to mind how to break searching functionality without finding any error. We keep documentation on paper of every testing and added comment of our tests. Suddenly our instructor says we had to change testing group and leave our documentation on the desk. Now we got open web site and documentation written by someone else. That was hard find out what they have been done. We check some points what they have been done so we understand better their documentation. We didn't expect that other groups start to read our documentation. Hopefully other group understands our writings. Web site was still buggy as earlier dojos. Bugs were again localization problems and some dead links. After session we change again back to original positions and start to continue where we leave. Other group has not found any error either. Some new test aspects have been written in documentation. After this session we didn't find any errors at all from main view search functionality. I admit time boxing help to focus to relevant things in testing.

Exploratory Testing and RedMind

Third dojo was interesting by giving eye refreshing view and we have some fun. I am happy that I can continue my old testing method. Even it have fancy name Exploratory Testing. We found that the ET is what every developer or users do naturally when testing some application. Searching information and trying to find bugs with different inputs and clicks. Inventing the wheel again of course is not purpose and best practices of these inputs can be found from Test Heuristics Cheat Sheet. We use these practices and try to find bugs from web site of University of Turku. We found several validation bugs from forms, language related bugs e.g. hyperlink redirect from page in English to page in Finnish. These bugs aren't showstoppers or neither high priority. But it is good to find these because implementation of validation to form is so easy that now I wonder why these haven't done. Validation expressions can be found from Internet quickly if can't create one by self. You can't make an omelet without breaking eggs.

I must admit that systematical testing is needed even ET is in use. Managers could want report on testing progress and without systematical testing that is quite hard to done. If falling into indiscriminate testing it probably cause some parts those should be tested will be ignored. This could be preventing with manuscript where test orders are thought already.  

We have also nice Redmind overall presentation. These presenters have some hands on experience and I feel they like to use this tool. Redmine seems to be versatile project management web application and development tools can be integrated seamlessly of e.g. version control. Also wiki pages and blogs can be hosted and supports plugins. So if some functionality has not been implemented you can write your own.

UI Test Automation with Watir

Watir is lightweight browser driver built in Ruby. Watir holds ATDD ideology where tests are easy to read and maintain. Tests are based to scenarios written with Gherkin. Step definitions will contain Ruby code which drives web browser like normal user e.g. open browser, goto web page, click some button, write text to fields and select checkbox. Very cool indeed. Here some example lines:

browser = Watir::Browser.new
browser.goto 'http://www.google.com'
browser.button(:name => 'submit').click
browser.text_field(:name => 'somefield').set "some text"
browser.checkbox(:value => 'Watir').set

See more examples here

Pair coding

We continued practices from last week. Our instructor has been done scenarios and code implementation to ready so we could test something. Good job. From those scenarios we created skeletons to step definitions which we started to complete with test implementations. Implementation was not so trivial even Ruby syntax look readable. Our computer was the Mac and seems it was not so familiar most of us ;) Also Ruby syntax and how to define methods with it etc. were uncertain. Implementation happened quite well after we get familiar with basics of Mac and Ruby. Purpose of this coding dojo were to test user account creation form. Tests should ensure that the required text fields are not empty after submit button click. After button click the test should also check that user account really exists in system.

Validation of required fields gives an error message to the screen which contains number of missing fields and under it the name list of missing fields. We could use the error message text as a criterion for the test result. In this way we cannot get information which fields doesn't meet the validation rules. This approach could be harmful in later if validation of fields are changed so that some field are not required anymore e.g. first name is optional. If not required and some required field has not been filled after submit process then we need in the test to find out which fields are required and not. So the better solution could be use the name list of missing fields to solve test result.

During implementation we found that table lookup were quite heavy and hard to done. We decide that system under test need to be updated for the test methods. We needed user identifier for each table row to find user account information from table easily to ensure that user data is really stored in the system. After the changes test implementation were easy to implement. This is a good idea to keep in the mind when making tests. It is pointless to implement complicated tests if the changes are easier to implement to the system under test.

This coding dojo was very good. I got many ideas how to improve my own working. I decide to study Watin soon. Watin is .NET implementation of Watir. Here is easy example:

[Test]
public void SearchForWatiNOnGoogle()
{
  using (var browser = new IE("http://www.google.com"))
  {
    browser.TextField(Find.ByName("q")).TypeText("WatiN");
    browser.Button(Find.ByName("btnG")).Click();
  
    Assert.IsTrue(browser.ContainsText("WatiN"));
  }
}

Coding Dojo 1 Done Done

In last friday was first coding dojo where topics were Cucumber (and Gherkin) and ATDD (Acceptance Test Driven Development). Class was split to two groups where expert teams were introduced their topic. Expert team where I belong to introduced ATDD (see slides in Finnish).

ATDD

ATDD is team practice which concentrate to external quality and it contains TDD (Test Driven Development). Specification workshop consist of project key members e.g. developers, testers, product owner, architect and technical writers. These members can vary during software project. During specification workshop user stories are selected and specifications are written down. These specifications will be implemented to executable specifications. Next phase is feature implementation so tests can be executed successfully. Implementation will follow TDD-cycle. Last phase is demonstration and customer approval. There are many ATDD benefits but few to mention this practice will improve communication between team members and enable fast feedback. The customer is responsible for certification and approval of functionalities which facilitates team work. There are also many problems to use this practice. Customer teaching to test driven development could be tricky and customer doesn't have any resources or knowledge. Writing automatically executable tests could be hard e.g. embedded systems. There are many tools in different software environments e.g. FIT, Robot and Cucumber.

Cucumber and Gherkin

Other expert team highlights Cucumber and Gherkin. Cucumber is ruby based BDD tool which executes plain-text functional descriptions as automated tests. Cucumber can be used to “test” code written in Ruby or other languages including but not limited to Java, C# and Python. Cucumber understand Gherkin language. It is a Business Readable, Domain Specific Language that lets you describe software’s behavior without detailing how that behavior is implemented. These functional descriptions are written in .feature files. Each feature to own file. Communication is better if people can use their spoken language. Cucumber supports currently over 40 spoken languages and steadily growing.

Example of functional description from wiki pages:

Feature: Serve coffee
  In order to earn money
  Customers should be able to
  buy coffee at all times

  Scenario: Buy last coffee
   Given there are 1 coffees left in the machine
   And I have deposited 1$
   When I press the coffee button
   Then I should be served a coffee

NOTE: Keywords used to describe feature and scenario are bold ('But' is also supported keyword but not used in this example). These keywords should be written case sensitive and ':' should be added after Feature and Scenario.

From theory to practice

One of our course instructors act as business owner. He need few user account management features for Bowling company's web site e.g. create new user account, list all users and filter results. Our group ask questions to clarify requirements. We practice Gherkin description by pair coding with text editor. This pair is only allowed to speak freely. Other members can ask open questions to present own ideas but no more. One pair member was changed after 5 minutes and this way every group member can participate to pair coding.

We didn't implement step definitions because time of coding dojo was limited. Steps after writing feature scenarios aren't now familiar but I think this was very good way rather than implementing functionality. It seems to be quite simple to continue by my self after looking documentation from Cucumber's web page. Cucumber (and Gherkin) has been documented quite well and Internet offers a lot information.

We didn't get solved last feature due time was over. But here is my answer how it could be resolved. I am not sure can table include table and haven't test it yet. If you have better solution please be free and comment.

Feature: List users
  In order to modify user informations 
  Owner should be able to list users
  with free query filter
  Scenario: List users
    Given system have users <users>
    | username | fname | lname | picture |
    | foo      | bar   | baz   | nil     |
    | n00b     | l33t  | fff   | n00b.jpg|
    When I set search filter as <filter>
    | filter |
    | nil    |
    | foo    |
    Then I should see <result>
    | filter | users                                |
    | nil    || username | fname | lname | picture ||
    |        || foo      | bar   | baz   | nil     ||
    |        || n00b     | l33t  | fff   | n00b.jpg||
    | foo    || foo      | bar   | baz   | nil     ||

Conclusion

ATDD is powerful team practice and can managed with tools e.g. Cucumber and Gherkin. This combination can guarantee all features are done like customer wants to. There could be problems to take in use this team practice in software projects because it could make customer, managers, developers and other team members to think that this is yet another complicated practice. Actually it should enable all communicate same language during whole project and remove uncomfortably feelings has every requirement done done.

First steps

I have been software developer four years. Software projects has been varying from mobile to client/server solutions. On every project I have been thinking how to be sure that all requirements has been implemented correctly. I haven't got enough knowledge of software testing so in few later projects I focused little bit more to unit testing during development. I don't mean Test Driven Developement. No. Just some desperate try to get some reliability to development. Maybe I should try TDD. But anyway, I realize that I can be more efective and get production more accuracy even after huge changes with unit test cases... but those do not cover everything. I still miss some practice of how to test UI and how to be sure that all requirements are done (and even right).

Now I have been enrolled to the Software Testing course in University of Turku. After this course I should have some clue of Acceptance Test Driven Development (ATDD) with Cucumber, UI Test Automation, Exploratory testing of a web application and Session based testing of an embedded device application. These things are handled in Coding and Testing Dojo's during end of this year where expert groups teach some of previous topic to others. I pick up the ATDD which seems to be hot and huge topic. This blog will be my learning diary where to collect information, links and coding dojo feelings overall from topics in Software Testing. Hopefully this blog might help someone.

Some links to start with ATDD:

• Introduction to ATDD
• Acceptance Test Driven Development 
• John Smart, ATDD Bring Developers and Testers Together