Background and Related Work

Static analysis is the process of evaluating a system or component based on its form, structure, content or documentation [1] . Tools can be used to automate the process of performing static analysis. Static analysis may be run throughout the development process since this analysis does not require program execution [2] . Static analysis also allows for the entire program to be analyzed [2] .

The main problem with static analysis is that many of these tools have a high rate of false positives. A false positive is a reported alert that is not a fault in the program. Static analysis tools cannot be completely automated (meaning that the problem is reported and fixed) because human input is required to verify the accuracy of the alerts for fault fixes. [2] We wrote AWARE to mitigate the cost of false positives by presenting software engineers with static analysis alerts that are ranked. Alerts are ranked such that alerts more likely to be true positives will be noticed and corrected by a software engineer earlier in the development process.

AWARE v0.2 incorporates the CnC [3] static analysis and test case generation tool. CnC combines static analysis and automatic JUnit test case generation via the ESC/Java [4, 5] and JCrasher [6] tools, respectively. ESC/Java was initially developed at the Compaq Systems Research Center for Java [4] . ESC/Java 2 [5] , the next version of ESC/Java, incorporates support for the Java Modeling Language (JML) and compatibility with Java 1.4. CnC looks specifically for conditions that may cause runtime exceptions like a ClassCastException or ArithmeticException [3].

ESC/Java is an annotation-based static analysis tool that uses a theorem prover, Simplify [7], to perform formal verification of Java source code. JCrasher [6] generates test cases for methods that contain a static analysis alert. The test cases are generated by providing parameters to the method that may cause the specified runtime exception [6].

Alert Ranking

Alert ranking strategies are used to prioritize alerts returned from static analysis tools based on the confidence, or probability, in the classification of the alert a true positive [9] . Z-Ranking [9] is an algorithm that uses statistical analysis to rank alerts presented by three static analysis tools for C. The Z-Ranking algorithm is a non-adaptive strategy in that filtered alerts are not considered in adjusting future rankings [10] . Another ranking algorithm, FEEDBACK-RANK [10] adjusts the alert ranking after each alert is inspected based on the relationship between alerts and the alert's location in the source code. AWARE will utilize filtering to adjust the alert ranking strategy.

Kremenek et al. [10] have shown that true positives and false positive alerts tend to cluster by their location in the source code, or the alert's code locality. This clustering can occur at the function level, class level, and package level. The reasoning behind alert clustering is that programmers tend to make the same mistakes over and over [10] , causing the clustering of true positives. False positives cluster because analysis tools are not sound , leading to many false positive alerts [10] . If a programmer provides feedback that a fault is a true positive, this feedback increases the probability that other faults in the same population, or grouping of error reports – in this case by location , are also true positives and vice versa [10] .

[1] IEEE, "IEEE Standard 610.12-1990, IEEE Standard Glossary of Software Engineering Terminology," 1990.

[2] B. Chess and G. McGraw, "Static Analysis for Security," in IEEE Security and Privacy , November/December 2004 ed, 2004, pp. 32-35.

[3] C. Csallner and Y. Smaragdakis, "Check 'n' Crash: Combining Static Checking and Testing," presented at the 27th International Conference in Software Engineering, St. Louis, MO, USA, 2005.

[4] C. Flanagan, K. R. M. Leino, M. Lillibridge, G. Nelson, J. B. Saxe, and R. Stata, "Extended Static Checking for Java," presented at Programming Language Design and Implementation, Berlin, Germany, 2002.

[5] D. R. Cok, "ESC/Java2 Implementation Notes," Nov 2004 2004.

[6] C. Csallner and Y. Smaragdakis, "JCrasher: an automatic robustness tester for Java," Software - Practice and Experience , vol. 34, pp. 1025-1050, 2004.

[7] D. L. Detlefs, K. R. M. Leino, G. Nelson, and J. B. Saxe, "Extended Static Checking: A Technical Report," System Research Center at Compaq 1988.

[8] K. R. M. Leino, G. Nelson, and B. Saxe, "ESC/Java User's Manual," 2000.

[9] T. Kremenek and D. Engler, "Z-Ranking: Using Statistical Analysis to Counter the Impact of Static Analysis Approximations," in 10th International Static Analysis Symposium , 2003.

[10] T. Kremenek, K. Ashcraft, J. Yang, and D. Engler, "Correlation Exploitation in Error Ranking," presented at International Symposium on Foundations of Software Engineering (FSE), Newport Beach, CA, 2004.