Andy Meneely and Laurie Williams [Contact Authors]
CSC 712 - Software Engineering
Department of Computer Science
North Carolina State University

Back to Software Engineering Tutorials

HttpUnit is a Java API for interacting with websites. Acting as a browser, HttpUnit enables a programmer to automate testing on a website by sending HTTP requests, returning the responses, and parsing the HTML that is returned. HttpUnit also supports HTTP headers, cookies, Javascript execution, DOM, and many other features used in web development. Although HttpUnit is typically used as a utility for JUnit tests (hence its name), it can actually be used for any automated interaction over HTTP - test or not.

For our purposes, we will be using HttpUnit for security testing - specifically, penetration testing - of our J2EE web application, iTrust. While much of our penetration testing can be done with JUnit interacting directly with our Java code, many vulnerabilities lie in our as-of-yet untested JSPs. In this tutorial, we will write a test for a particularly bad Cross-Site Scripting (XSS) vulnerability in iTrust that lies in a JSP. One could, however, use HttpUnit to test almost any kind of web application vulnerability, such as SQL Injection or Denial of Service.

HttpUnit consists of a number of Java jars files used for various purposes. This section will demonstrate how to add new JAR files to the iTrust classpath in Eclipse WTP.

1. Download our zip file of jars: iTrust HttpUnit jars.zip.
2. Extract the zip and copy the four jar files into your iTrust folder. (Note: you can copy them directly into the Eclipse Package Explorer, or if you copy them directly to your iTrust folder in Explorer, make sure you refresh you project).
3. In the Package Explorer, select the four jar files, right-click and go to Build Path > Add to Build Path
   
   
   
   Adding jars to your classpath
   
   
4. If you expand the Referenced Libraries node, you should see the four new jars on your classpath
   
   
   Checking for the jars are on your classpath
   
   

HttpUnit uses the following key classes:

• WebConversation represents the browser. With WebConversation, you can make requests, store cookies, store frames, change general settings, etc. For iTrust, it's important to keep track of the same WebConversation in a test as it stores your session authentication info. If you want to simulate multiple users on the system, use multiple WebConversations.
• WebResponse represents the actual response from the website. Another way of thinking of the WebResponse is the actual page itself. The WebResponse class has accessors such as getText() which will return the actual HTML, and getLink(), getTitle, etc from parsing the HTML.
• HttpException represents an HTTP error (such as 404 - Not Found, or 403 - No Access). Be sure to use the com.meterware.httpunit.HttpException and not the other one!
• WebRequest is an abstract class representing a request. The PostMethodWebRequest represents a form posting (which is used throughout iTrust).

A typical HttpUnit session might look like the the following code. Can you see what it does?

WebConversation webConversation = new WebConversation();
WebResponse loginResponse = webConversation.getResponse("http://localhost:8080/iTrust");
assertEquals("iTrust Login", loginResponse.getTitle());
WebResponse homePage = loginResponse.getLinkWith("HCP").click();
assertEquals("iTrust - HCP Home", homePage.getTitle());
			

The code first requests iTrust, clicks on the first link with "HCP" as a substring (happens to be the testing-only link above login), and goes to the HCP homepage. Copy this into your own unit test and run it (be sure to have Tomcat running!)

Alternatively, you could use the actual form submission by obtaining the first (and, in this case, only) form from the response, setting the parameters, and submitting. The following snippet will do the trick:

WebConversation webConversation = new WebConversation();
WebResponse loginResponse = webConversation.getResponse("http://localhost:8080/iTrust");
assertEquals("iTrust Login", loginResponse.getTitle());
loginResponse.getForms()[0].setParameter("j_username", "9000000000");
loginResponse.getForms()[0].setParameter("j_password", "pw");
WebResponse homePage = loginResponse.getForms()[0].submit();
assertEquals("iTrust - HCP Home", homePage.getTitle());
			

A few comments worthy of note:

• HttpUnit tries to simulate the user experience by allowing you to chain commands in a single line of code - as if you're clicking through the website. As a result, however, not all of the details of HTML browswing are supported. Of particular absence is the ability to manipulate hidden fields (iTrust uses a formIsFilled hidden field ubiquitously). The HttpUnit website's FAQ provides a few tricks to get around this, and when all else fails, you can always encode your hidden fields in the URL (e.g. mypage.jsp?formIsFilled=true)
• Using WebConversation, you can request any diabolical URL of your choosing.
• The WebResponse has many different ways of finding the element you're looking for. Most methods that ask for a string will do a substring match (such as the getLinkWith() method)

iTrust has a particularly bad Cross-Site Scripting (XSS) vulnerability in one of its header files that allows an attacker to craft a URL which could result in an execution of Javascript in a client's browser. The exploit uses the "error" field used throughout the application to report an error at the top of a page. If you have iTrust running on your local system, you can test this out by going to the link and log in in as an HCP: http://localhost:8080/iTrust/auth/hcp/hcpHome.jsp?error=<script>alert('XSS');</script>

First, let's write a test case that will initially fail when it's run. Using a similar approach as our example above, use the following as your unit test:

public class ErrorXSS extends TestCase {
	private static final String XSS_EXPLOIT = "<script>alert('XSS');</script>";
	private static final String ESCAPED_XSS_EXPLOIT = "&lt;script&gt;alert('XSS');&lt;/script&gt;";

	public void testXSSInErrorMessage() throws Exception {
		WebConversation webConversation = new WebConversation();
		WebResponse loginResponse = webConversation
				.getResponse("http://localhost:8080/iTrust/auth/hcp/hcpHome.jsp?error="
						+ XSS_EXPLOIT);
		assertEquals("iTrust Login", loginResponse.getTitle());
		WebResponse homePage = loginResponse.getLinkWith("HCP").click();
		assertEquals("iTrust - HCP Home", homePage.getTitle());
		assertFalse("Should not contain an XSS exploit", homePage.getText()
				.contains(XSS_EXPLOIT));
		assertTrue("Should contain the escaped version of the exploit",
				homePage.getText().contains(ESCAPED_XSS_EXPLOIT));
	}

}
			

If you run this code as a unit test, you should get a red bar with the comment: Should not contain an XSS exploit. What this test is testing for is if the exact text - malicious text - that we passed to the error parameter comes back exactly how it was sent. If so, we know that Javascript will execute.

Now to fix the vulnerability. For this tutorial, we will simply escape the characters passed to the error parameter using a method already written in iTrust. Normally, however, for an error message mechanism such as this, we would want to create a whitelist of error messages so that an attacker couldn't insert custom error messages into URLs for phishing. But, for the sake of simplicity, we will just escape the characters.

Open up the file messages.jspf in the /WebRoot/resources folder. Look for the following chunk of code:

if (request.getParameter("error") != null) {
%>
	<span class=error> <%=request.getParameter("error")%> </span>
<%
}
			

Change the middle line to use the already-written HTMLEncoder method in the edu.ncsu.csc.itrust package

HtmlEncoder.encode(request.getParameter("error"))
			

Green bar!

You now have all of the tools you need to execute JUnit test via making HTTP requests using HttpUnit. Below are some other helpful HttpUnit code snippets that might help for security testing. They are not meant to be complete, so please adapt to taste.

The following helps for expecting an authorization (403) error.

try {
	new WebConversation().getResponse("URL YOU SHOULDN'T GET TO!");
	fail("exception should have been thrown");
} catch (HttpException e) {
	assertEquals(403, e.getResponseCode());
}
			

HttpUnit also parses tables - however, it handles nested tables rather peculiarly. I reccommend using getTableStartingWithPrefix which searches through tables and will match the first non-blank cell. Once you obtain the table, you can pull cells individually using zero-based row and column indices.

WebConversation webConversation = new WebConversation();
WebResponse loginResponse = webConversation
	.getResponse("http://localhost:8080/iTrust/auth/hcp-uap/editBasicHealth.jsp?pid=2");
WebResponse basicHealth = loginResponse.getLinkWith("HCP").click();
WebTable table = basicHealth.getTableStartingWithPrefix("Basic Health");
assertEquals("Smokes?", table.getCellAsText(1, 2));