Acceptance Testing in Eclipse Using FIT
|1.0||Introduction to FIT|
|5.0||Running FIT Tests|
FIT is a tool developed by Ward Cunningham that helps with the writing of automated acceptance tests. FIT uses JUnit, but extends the testing functionality. The test cases are displayed in HTML tables, which makes it easy for customers to write their own test cases. Behind the HTML table is a Java class called a fixture. The fixture takes the contents of the HTML table and runs the test cases on the project being tested. A fixture can extend ColumnFixture, RowFixture, or ActionFixture.
FIT is packaged with JUnit in a stand alone jar file that can be added to the library of any application. However, FIT, by itself, can be difficult to use in the Eclipse environment. We will be using the FitRunner plug-in, developed by Chih-wei Ho, to easily run FIT tests in Eclipse.
When a FIT test is run, cells that contain test cases are color coded to represent the results of the test. If the cell is green the test passed; there was an exact match (It may be difficult to have an exact match with long Strings). If the cell is red, the test failed. The expected result (the value in the table) is displayed above the actual result (the value the fixture returns). If the cell is yellow, a Java exception occurred. The error message and stack trace will print out in the cell.
In this tutorial, we are going to learn about FitRunner, ColumnFixture, and ActionFixture. The exercise introduces CoffeeMaker, a project with built-in bugs for you to uncover.
FitRunner is an Eclipse plug-in written by Chih-wei Ho to facilitate the interaction between the FIT libraries and Eclipse. FitRunner can be found on SourceForge. Follow the installation instructions on the site to plug FitRunner into your copy of Eclipse, or see the Installing Eclipse Plugins Tutorial. FitRunner should already be installed on the computers in the Laboratory for Collaborative System Development.
It is considered a best practice in testing to separate the test case code from the application code. It is also a good idea to separate your FIT tests from your JUnit tests as well. (Since FIT is based on JUnit, JUnit will try to run the FIT tests when it is run, causing errors. By seperating the tests into different folders you can point FIT to the folder for FIT tests and jUnit to the unit tests folder). Here is an example file structure that helps with this:
Most projects that you create will have a src/ folder that contain the main application code, a bin/ folder that contain the compiled .class files, and a lib/ folder that contains jar files that need to be on the project's build path. In order to use FitRunner, fit.jar needs to be on the build path. Since FIT requires the JUnit libraries, junit.jar must also be on the build path.
The acctest/spec/ folder will contain the specification HTML tables of your FIT acceptance tests, while acctest/result/ will contain the HTML tables that show the results of your FIT acceptance tests. It is best to keep a clean file of your test specification so that you can rerun the tests on a clean file. The acctest/fixtures/ folder will contain the Java fixtures that run your FIT acceptance tests. The unittests/ folder will contain your JUnit test cases. The models/ folder may contain your models that are generated by Omondo.
2.1 Putting fit.jar on the Build Path
2.2 Putting junit.jar on the Build Path
For the exercise below, you will be downloading a project that already has some FIT and JUnit tests started. You should not have to change the build path configurations on the project, but in case you do, follow the above instructions to find the jar files that are stored in the lib/ directory of the CoffeeMaker project.
In a ColumnFixture based HTML table, each column title should correspond to an attribute or a method in the associated fixture. A ColumnFixture executes the tests row by row, from top to bottom. Each row is executed column by column, left to right.
A column heading that does not end in ()s corresponds to an attribute in the fixture source code that has the same name as the column heading in the FIT table. The value that is in the cell is assigned to fixture's attribute. A column heading that does end in ()s corresponds to a method in the fixture source code that has the same name as the column heading in the table. The value that is in the cell is the expected result of running the method, and the fixture method must return a value of the specified type. Each cell under a method column heading is a test case.
There are two main steps to creating a ColumnFixture test in FIT. The first is to create an HTML table, and populate it with the tests. Second, a fixture is made that contains the code that runs the tests.
3.1 Creating an HTML table
Use your favorite HTML editor to create an HTML table. Dreamweaver or FrontPage are good applications to use, since they speed up the creation of tables. A ColumnFixture table uses the following format:
The top row contains the package and class name of the fixture that is associated with this table. The second row contains the names of the attributes and methods. All subsequent rows contain the test information.
3.2 Creating a Fixture
All fixtures should be created in a separate source folder in your project in Eclipse. It is always good practice to separate the application code from the test code. We are going to store our fixtures in acctest/fixtures.
In an ActionFixture based HTML table, each table represents a set of steps used to execute one test case. An ActionFixture can correspond to testing a User Story. Each row is a command that is performed in order, top to bottom.
The first column in an ActionFixture represents a command that is executed. ActionFixture has 4 commands, but other commands can be created in subclasses of ActionFixture. These 4 commands are:
Any time the check command is used is a test case.
3.1 Creating an HTML table
Use your favorite HTML editor to create an HTML table. Dreamweaver of FrontPage are good applications to use, since they speed up the creation of tables. An ActionFixture table uses the following format:
The top row contains fit.ActionFixture, which states that the table is an ActionFixture. The start row is only called once and its argument is the name of the class that is the associated fixture. The press row contains the name of the method that is called. The enter row contains the name of the method and the value that is the parameter of that method. The check row contains the name of the method and the expected value the method will return.
3.2 Creating a Fixture
All fixtures should be created in a separate source folder in your project in Eclipse. It is always good practice to separate the application code from the test code. We are going to store our fixtures in acctest/fixtures/.
There are several ways to run a program.
5.1 If the program has never been run before, you need to create run configuration for the program.
5.2 Select FIT, then click the New button at the bottom left of the screen. This creates a new run configuration. Give the setting a name. If the project is not specified, you can Browse for it. The output folder defaults to acctest/result/ but you can search for another output folder.
5.3 Just double click on an output file in the Package Explorer, and the FIT test results will be displayed in the built in Eclipse browser. A textual representation of the results will display in the console view. The summary.html file gives you pass, failure, and exception information for each file under test.
For this exercise we will be using the CoffeeMaker project. Download the CoffeeMaker project from here. Unzip the CoffeeMaker project to your home directory and import the project into Eclipse. Please see the Eclipse Import/Export Guide for instructions on how to import a project into Eclipse.
We all know that most computer scientists love caffeine, so the Computer Science department is looking to put a coffee kiosk in our new building. The coffee kiosk must be able to make coffee for students to purchase. CoffeeMaker User Stories and Black Box Test Cases.
The CoffeeMaker code is complete; however, we need you to create and run acceptance tests on the first three functions of the CoffeeMaker. These functions are: 1) Add a recipe, 2) Add Inventory, and 3) Edit a recipe. FIT tables and FIT fixtures have been created for each of the 6 user stories in the CoffeeMaker project. However, there are only a few tests in each fixture. There are currently 5 (very obvious) bugs in the system. We need you to generate enough acceptance tests to find 3 of these bugs, one each in the three functions under test. Once you find the bugs, create a fix (These should be very simple fixes. If the fix takes longer than 5 minutes, you found a bigger bug than the one we wanted you to find!). You'll want to write more test cases in the FIT tables that have been provided (hint: run the FIT tests first) Find and fix all 3 of the bugs. Create a list of the bugs that you found.
Deliverables to the TA