Grafter

A tool to reuse tests and examine behavioral differences between similar programs (i.e., clones).

New Release (Mar 2017) Multiple bugs in test coverage analysis, loop synthesis, and exception handling have been fixed.

New Feature (Feb 2017) Now Grafter supports clones detected by Simian.

Setup

1. Grafter is packaged as an executable Jar file. Please download first. Grafter has been tested in Ubuntu 14.04. If you have issues with running it on other platforms, please email Tianyi Zhang (tianyi.zhang@cs.ucla.edu).

2. Follow instructions to config Grafter in the config file.

3. Run Grafter in the terminal.

$ java -jar grafter-2.0-jar-with-dependencies.jar /path/to/your/config/file

Usage

1. Load Clones to Grafter

Grafter requires users to provide clones. You can find clones either using clone detection tools (e.g., Deckard, Simian) or manually.

(Option 1) Load clones from Deckard: click "Deckard" and load clones (i.e., Deckard output file) to Grafter. You can view and inspect clones detected by Deckard in Grafter.

(Option 2) Load clones from Simian: make sure you save the clones detected by Simian in the xml format. Then click "Simian" and load clones (i.e., the Simian xml file) to Grafter. You can view and inspect clones detected by Simian in Grafter.

(Option 3) Load clones from other sources: Please format your clones following this sample xml format and load them by clicking "Load".

Note: it may take a while to process if there are a lot of clones.

2. Inspect Clones

Compare clones: right-click a clone group and select "Compare".

Rename clones: each clone in the group is named in the format of "(start line number, end line number)" by default. You can double-click and rename a clone, but it has to be in the format of "XYZ(start line number, end line number)".

Exclude Trivial Clone Groups: Grafter automatically excludes trivial clone groups (e.g., clones in test files) and mark them in red. If you believe Grafter mistakenly filters out a clone group, you can re-add this group by right-clicking on it and select "Include". When inspecting clones, you can discard a clone group by right-click -> Exclude.

Save your inspection result (required): Click "Export" and save all clone groups of interest to an xml file. All excluded clones will be discarded and not exported to the xml file.

2. Find Test Cases of Clones

Click "Coverage" and Grafter will automatically locate test cases for each clone.

Grafter requires at least one clone in the clone group to be executed by one or more tests in order to reuse tests on its counterpart clones. If a clone group has no test cases, you can discard it by right-clicking the group and then selecting "Exclude".

Please remember to export the updated clone groups with corresponding test cases to an xml file by clicking "Export" so that you can reload these clones and tests next time.

3. Reuse Tests and Examine Behavioral Differences

Test-level Comparison: compare test outcomes of two clones by right-clicking the clone group and selecting "Compare Test Behavior".

The comparison results are represented in a table and behavioral differences are highlighted in red for ease of investigation. The first column shows names of test cases. The second and third columns show whether each clone passes or fails the corresponding test case. The last column shows the comparison result. Green means both clones are consistent on this test and red means their test outcomes are different.

State-level Comparison: compare intermediate state values of two clones by right-clicking the clone group and selecting "Compare State Behavior".

In the state-level comparison table, the first and third columns show the name of corresponding variables referenced by code clones, and the second and fourth columns show the values of variables in the format of XML. Grafter prints the value of an object in XML using XStream.

Inspecting Grafted Clone: Grafter allows users to inspect the intermediate grafted code during test reuse.

      a. Compare two clones by right-clicking on the clone group and selecting "Compare".

      b. Graft the left clone to replace the right clone in the side-by-side view by clicking "Graft Left" in the top menu bar.

      c. Graft the right clone in place of the left one by clicking "Graft Right".

      d. You can undo and redo the previous graft by clicking the "Undo" and "Redo" buttons on the top menu bar respectively.

Replicate the Evaluation Results in the ICSE'17 Paper

Dataset

Download our dataset.

Our dataset contains 52 pairs of clones from three subject programs: Apache Ant, Java-APNS, and Apache XML Security. Apache Ant is a software build framework. Java-APNS is a Java client for the apple push notification service. Apache XML Security is an XML signature and encryption library. In particular, our dataset includes 38 pairs of Type II clones and 14 pairs of Type III clones.

Baseline

We compare Grafter with a static approach by Jiang et al. to assess Grafter's behavior comparison capability. Jiang et al.'s approach is designed to detect three types of cloning inconsistencies. Our hypothesis is that by checking for behavioral differences at runtime, Grafter can detect behavioral divergence more effectively than a static tool.

Because Jiang et al.'s implementation supports Java 1.4 only. We reimplemented Jiang et al.'s technique (download).

Test Reuse and Differential Testing Experiment

1. Update config files and run Grafter. In each subject program, update config files with paths in your own machine. Then run Grafter with the updated config file.

$ java -jar grafter-2.0-jar-with-dependencies.jar Grafter.config

2. Load clones and their tests. Load clones and corresponding tests from the xml file (./clone_report/clones_with_tests.xml).

3. Reuse tests and examine their behavioral differences. For each clone pair, right click and select "Compare Test Behavior" and "Compare State Behavior" respectively.

4. Run Jiang et al.'s static cloning inconsistency technique as baseline. Execute the command in terminal.

$ java -jar baseline.jar /path/to/config/file /path/to/clone/report/file

Mutation Experiment

Mutants are generated using a mutation analysis framework, Major. In total, 361 mutants are generated on 30 pairs of clones that do not have test behavioral divergence. You can these mutants in the mutants folder in each subject program.

1. Run Grafter to detect mutants in batch. Execute the command in terminal.

$ java -cp grafter-2.0-jar-with-dependencies.jar edu.ucla.cs.grafter.batch.BatchRunner /path/to/config/file /path/to/clone/report/file

2. Run Jiang et al.'s technique to detect mutants in batch. Execute the command in terminal.

$ java -cp baseline.jar edu.ucla.cs.clone.batch.BatchRunner /path/to/config/file /path/to/clone/report/file

The mutation experiment may take a while to finish. You can find logs of our evaluation in the grafter_log folder in each subject program, where mutation.log contains the test and state comparison results of Grafter and mutation_Jiang.log contains the mutation testing result of the baseline technique.

Publication

Our work is accepted to ICSE 2017. Please cite our paper if you find it related or if you use Grafter in your research.

@inproceedings{zhang2017automated,
  title={Automated transplantation and differential testing for clones},
  author={Zhang, Tianyi and Kim, Miryung},
  booktitle={Proceedings of the 39th International Conference on Software Engineering},
  pages={665--676},
  year={2017},
  organization={IEEE Press}
}
	

Contact

The project is currently maintained by Tianyi Zhang. Please drop an email to me (tianyi.zhang@cs.ucla.edu) if you have any questions.

Acknowledgement

The project is in collaboration with my advisor, Professor Miryung Kim and is supported by Air Force Research Laboratory Grant. Thanks to Todd Millstein and anonymous reviewers for helpful feedback on this work. Grafter's GUI is augmented from the JMeld program differencing tool.