Acceptance Testing in Eclipse Using FIT


Laurie Williams, Dright Ho, Ben Smith and Sarah Heckman. [Contact Authors]
CSC 326 - Software Engineering
Department of Computer Science
North Carolina State University

Back to Software Engineering Tutorials


0.0 Contents
1.0 Introduction to FIT
2.0 FitRunner Plug-in
3.0 ColumnFixture
4.0 ActionFixture
5.0 Running FIT Tests
5.0 Exercise
6.0 Resources

1.0 Introduction to FIT

FIT is a tool developed by Ward Cunningham that helps with the writing of automated acceptance tests. FIT uses JUnit, but extends the testing functionality. The test cases are displayed in HTML tables, which makes it easy for customers to write their own test cases. Behind the HTML table is a Java class called a fixture. The fixture takes the contents of the HTML table and runs the test cases on the project being tested. A fixture can extend ColumnFixture, RowFixture, or ActionFixture.

FIT is packaged with JUnit in a stand alone jar file that can be added to the library of any application. However, FIT, by itself, can be difficult to use in the Eclipse environment. We will be using the FitRunner plug-in, developed by Chih-wei Ho, to easily run FIT tests in Eclipse.

When a FIT test is run, cells that contain test cases are color coded to represent the results of the test. If the cell is green the test passed; there was an exact match (It may be difficult to have an exact match with long Strings). If the cell is red, the test failed. The expected result (the value in the table) is displayed above the actual result (the value the fixture returns). If the cell is yellow, a Java exception occurred. The error message and stack trace will print out in the cell.

In this tutorial, we are going to learn about FitRunner, ColumnFixture, and ActionFixture. The exercise introduces CoffeeMaker, a project with built-in bugs for you to uncover.

Top | Contents

2.0 FitRunner Plug-in

FitRunner is an Eclipse plug-in written by Chih-wei Ho to facilitate the interaction between the FIT libraries and Eclipse. FitRunner can be found on SourceForge. Follow the installation instructions on the site to plug FitRunner into your copy of Eclipse, or see the Installing Eclipse Plugins Tutorial. FitRunner should already be installed on the computers in the Laboratory for Collaborative System Development.

It is considered a best practice in testing to separate the test case code from the application code. It is also a good idea to separate your FIT tests from your JUnit tests as well. (Since FIT is based on JUnit, JUnit will try to run the FIT tests when it is run, causing errors. By seperating the tests into different folders you can point FIT to the folder for FIT tests and jUnit to the unit tests folder). Here is an example file structure that helps with this:

File Structure Before Adding fit.jar and junit.jar to the Build Path
File Structure After Adding fit.jar and junit.jar to the Build Path

Most projects that you create will have a src/ folder that contain the main application code, a bin/ folder that contain the compiled .class files, and a lib/ folder that contains jar files that need to be on the project's build path. In order to use FitRunner, fit.jar needs to be on the build path. Since FIT requires the JUnit libraries, junit.jar must also be on the build path.

The acctest/spec/ folder will contain the specification HTML tables of your FIT acceptance tests, while acctest/result/ will contain the HTML tables that show the results of your FIT acceptance tests. It is best to keep a clean file of your test specification so that you can rerun the tests on a clean file. The acctest/fixtures/ folder will contain the Java fixtures that run your FIT acceptance tests. The unittests/ folder will contain your JUnit test cases. The models/ folder may contain your models that are generated by Omondo.

2.1 Putting fit.jar on the Build Path

2.1.1 Download fit.jar and put in the Project/lib/ directory.

2.1.2 Right click on the project and select Properties. In the left column select Java Build Path. To the right select the Libraries tab. Click the Add JARs... button. Select your project, and go down through the tree until you find fit.jar under the lib/ folder. Select fit.jar and Click OK.

Note: After you add fit.jar to the build path, it will no longer be displayed in the lib/ directory in the Package Explorer view of Eclipse. The jar file is still located in the lib/ directory when you explore it from outside of Eclipse, but since it is now on the build path inside of Eclipse, it has its own listing in the project.

2.2 Putting junit.jar on the Build Path

2.2.1 Right click on the project and select Properties. In the left column select Java Build Path. To the right select the Libraries tab. Click the Add External JARs... button. You should start off in the Eclipse install directory, otherwise search through the file system until you find the Eclipse install directory.

Note: The Eclipse install directory is usually under the C:/ on most computers in the Laboratory for Collaborative System Development. This is where it is suggested that you install Eclipse on the Eclipse website.

2.2.2 Under the Eclipse install directory select plugins/org.junit_3.8.1/junit.jar. Click OK.

OR

2.3.1 Download junit.jar from here, or from www.junit.org. Add junit.jar to your lib/ directory in your project.

2.3.2 Right click on the project and select Properties. In the left column select Java Build Path. To the right select the Libraries tab. Click the Add JARs... button. Select your project, and go down through the tree until you find junit.jar under the lib folder. Select junit.jar and Click OK.

Note: After you add junit.jar to the build path, it will no longer be displayed in the lib/ directory in the Package Explorer view of Eclipse. The jar file is still located in the lib/ directory when you explore it from outside of Eclipse, but since it is now on the build path inside of Eclipse, it has its own listing in the project.

For the exercise below, you will be downloading a project that already has some FIT and JUnit tests started. You should not have to change the build path configurations on the project, but in case you do, follow the above instructions to find the jar files that are stored in the lib/ directory of the CoffeeMaker project.

Top | Contents

3.0 ColumnFixture

In a ColumnFixture based HTML table, each column title should correspond to an attribute or a method in the associated fixture. A ColumnFixture executes the tests row by row, from top to bottom. Each row is executed column by column, left to right.

A column heading that does not end in ()s corresponds to an attribute in the fixture source code that has the same name as the column heading in the FIT table. The value that is in the cell is assigned to fixture's attribute. A column heading that does end in ()s corresponds to a method in the fixture source code that has the same name as the column heading in the table. The value that is in the cell is the expected result of running the method, and the fixture method must return a value of the specified type. Each cell under a method column heading is a test case.

There are two main steps to creating a ColumnFixture test in FIT. The first is to create an HTML table, and populate it with the tests. Second, a fixture is made that contains the code that runs the tests.

3.1 Creating an HTML table

Use your favorite HTML editor to create an HTML table. Dreamweaver or FrontPage are good applications to use, since they speed up the creation of tables. A ColumnFixture table uses the following format:

edu.ncsu.csc.ColumnClass
attribute1 attribute2 method1() method2()
1 a true 3
2 b false 6

The top row contains the package and class name of the fixture that is associated with this table. The second row contains the names of the attributes and methods. All subsequent rows contain the test information.

3.2 Creating a Fixture

All fixtures should be created in a separate source folder in your project in Eclipse. It is always good practice to separate the application code from the test code. We are going to store our fixtures in acctest/fixtures.

3.2.1 Create a new class. Name the class the same as the class name in the table, and make sure the packages match. Browse for ColumnFixture as a super class.

3.2.2 The code in the fixture should be similar to this:

package edu.ncsu.csc;


import fit.ColumnFixture;


public class ColumnClass extends ColumnFixture {

     public int attribute1;
public String attibute2; public boolean method1() {
//insert code here return true; } public int method2() {
//insert code here return attribute1;
} public void execute() {
//insert code here
} }

Each method that corresponds to a method heading in the HTML table should return a value that is the same type as that in the table. The execute method is inherited from ColumnFixture and is overridden if there is something outside of the fixture that you wish to execute, or that needs to be executed every time. It is good form to make all other attributes and methods that are not associated with the table private.

Top | Contents

4.0 ActionFixture

In an ActionFixture based HTML table, each table represents a set of steps used to execute one test case. An ActionFixture can correspond to testing a User Story. Each row is a command that is performed in order, top to bottom.

The first column in an ActionFixture represents a command that is executed. ActionFixture has 4 commands, but other commands can be created in subclasses of ActionFixture. These 4 commands are:

  • start: The name of the class that is the associated fixture.
  • enter: A method with an argument
  • press: A method with no argument
  • check: The returned value of a method with no arguments

Any time the check command is used is a test case.

3.1 Creating an HTML table

Use your favorite HTML editor to create an HTML table. Dreamweaver of FrontPage are good applications to use, since they speed up the creation of tables. An ActionFixture table uses the following format:

fit.ActionFixture
start edu.ncsu.csc.ActionClass  
press method name 1  
enter method name 2 1
check method name 3 true

The top row contains fit.ActionFixture, which states that the table is an ActionFixture. The start row is only called once and its argument is the name of the class that is the associated fixture. The press row contains the name of the method that is called. The enter row contains the name of the method and the value that is the parameter of that method. The check row contains the name of the method and the expected value the method will return.

3.2 Creating a Fixture

All fixtures should be created in a separate source folder in your project in Eclipse. It is always good practice to separate the application code from the test code. We are going to store our fixtures in acctest/fixtures/.

3.2.1 Create a new class. Name the class the same as the class name in the table, and make sure the packages match. Browse for ActionFixture as a super class.

3.2.2 The code in the fixture should be similar to this:

package edu.ncsu.csc;


import fit.ActionFixture;


public class ActionClass extends ActionFixture {


     public void methodName1() {
          //insert code here
     }
     public void methodName2(int num) {
          //insert code here
     }
     public boolean methodName3() {
          //insert code here
          return true;
     }     

}
      

Each method name is based off of the method name supplied in the ActionFixture table. If the method name has spaces in the table, then the spaces are removed, and the first letter of each new word is capitalized (unless it's a number).

Top | Contents

5.0 Running FIT Tests

There are several ways to run a program.

5.1 If the program has never been run before, you need to create run configuration for the program.

5.1.1 Select Run > Run..

5.1.2 Click the arrow on the icon in the tool bar and select Run..

5.1.3 Right click on the project you wish to run. Select Run > Run..

5.2 Select FIT, then click the New button at the bottom left of the screen. This creates a new run configuration. Give the setting a name. If the project is not specified, you can Browse for it. The output folder defaults to acctest/result/ but you can search for another output folder.

5.2.1 You can run a FIT test on just one file. Select the radio button beside Run FIT on a single file and Browse for the file.

5.2.2 You can also run FIT on a folder. Select the radio button beside Run FIT on a folder and Browse for a folder. The default folder is acctest/spec/.

5.3 Just double click on an output file in the Package Explorer, and the FIT test results will be displayed in the built in Eclipse browser. A textual representation of the results will display in the console view. The summary.html file gives you pass, failure, and exception information for each file under test.


Running FIT tests
Top | Contents

6.0 Exercise

For this exercise we will be using the CoffeeMaker project. Download the CoffeeMaker project from here. Unzip the CoffeeMaker project to your home directory and import the project into Eclipse. Please see the Eclipse Import/Export Guide for instructions on how to import a project into Eclipse.

We all know that most computer scientists love caffeine, so the Computer Science department is looking to put a coffee kiosk in our new building. The coffee kiosk must be able to make coffee for students to purchase. CoffeeMaker User Stories and Black Box Test Cases.

The CoffeeMaker code is complete; however, we need you to create and run acceptance tests on the first three functions of the CoffeeMaker. These functions are: 1) Add a recipe, 2) Add Inventory, and 3) Edit a recipe. FIT tables and FIT fixtures have been created for each of the 6 user stories in the CoffeeMaker project. However, there are only a few tests in each fixture. There are currently 5 (very obvious) bugs in the system. We need you to generate enough acceptance tests to find 3 of these bugs, one each in the three functions under test. Once you find the bugs, create a fix (These should be very simple fixes. If the fix takes longer than 5 minutes, you found a bigger bug than the one we wanted you to find!). You'll want to write more test cases in the FIT tables that have been provided (hint: run the FIT tests first) Find and fix all 3 of the bugs. Create a list of the bugs that you found.

Deliverables to the TA

  • List of 3 bugs found in the Add Recipe, Add Inventory, and Edit Recipe users stories in CoffeeMaker
  • All of your FIT acceptance tests, with all green cells. (Coloring the cells green inside of an HTML editor does not count! Your TA will want to see your run your tests and check the results from the proper output files.)
Top | Contents

7.0 Resources
Top | Contents

Back to Software Engineering Tutorials
Acceptance Testing Using FIT ©2003-2006 North Carolina State University, Laurie Williams, Dright Ho, Sarah Heckman, Ben Smith
Email the authors with any questions or comments about this tutorial.
Last Updated: Wednesday, August 20, 2008 1:47 PM