ExampleCheck

A technique to mine API usage patterns from GitHub repositories and contrast Stack Overflow examples with mined API usage patterns.

Update (Nov 2018) We have released a stand-alone API misuse detector that is powered by the 180 common usage patterns of 100 popular Java and Android API methods mined from GitHub.

Update (Aug 2018) We have installed a new SSL certificate that is trusted by the Chrome browser by default. From now on, there is no need to manually load our previous self-signed certificate.

Empirical Study

We mine 380,125 Java repositories in GitHub and learn 180 validated API usage patterns for 100 popular Java methods.

Then we examine 217,818 Stack Overflow posts using this pattern set.

The key findings are:

Dataset

Our dataset contains:

The dataset is available for download (link).

API Misuse Detector

We release a stand-alone API misuse detector that is powered by the 180 common API usage patterns mined from GitHub (Download).

The API misuse detector relies on a SMT solver, Z3 to check whether a method call is guarded by a proper predicate. Please install Z3 first (Download). Then add the path of the Z3 executable to the system path so that ExampleCheck can find the Z3 command directly.

Use the following command to run our API misuse detector.

$ java -jar examplecheck-cmdline-tool.jar FILE_PATH

FILE_PATH is the path to the Java source code. It can be an incomplete code fragment or a Java class file.

Sample output:

Violation 1 : In the readJSON method, you may want to check whether the receiver of getAsString() is not equal to null before calling getAsString(). 119 Github code examples also do this.

Violation 2 : In the readJSON method, you may want to check whether match_number.isJsonPrimitive() returns true before calling getAsString(). 51 Github code examples also do this.

Violation 3 : In the readJSON method, you may want to handle the potential Exception thrown by getAsString() by using a try-catch block here. 48 Github code examples also do this.

Total number of detected API misuse: 3

Chrome Extension

Besides the command-line API misuse detector, we also build a Chrome extension that alerts users about API usage violations in a Stack Overflow page, when the page is loaded in a Chrome browser.

Download: You can download the Chrome extension via this link or by searching "ExampleCheck" in Chrome Web Store.

To test whether ExampleCheck connects to our server, go to https://stackoverflow.com/questions/29860000, and check whether the method call, getAsString is highlighted in the accepted answer.

Publication

Please cite our papers if you find them related.

@inproceedings{zhang2018stackoverflow,
   title={Are Code Examples on an Online Q\&A Forum Reliable? A Study of API Misuse on Stack Overflow},
  author={Zhang, Tianyi and Upadhyaya, Ganesha and Reinhardt, Anastasia and Rajan, Hridesh and Kim, Miryung},
  booktitle={Proceedings of the 40th International Conference on Software Engineering},
  year={2018},
  organization={ACM}
}
@inproceedings{reinhardt2018augmenting,
  title={Augmenting Stack Overflow with API Usage Patterns Mined from GitHub},
  author={Reinhardt, Anastasia and Zhang, Tianyi and Mathur, Mihir and Kim, Miryung},
  booktitle={Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
  pages={880--883},
  year={2018},
  organization={ACM}
}

Contact

The project is currently maintained by Tianyi Zhang. Feel free to email me (tianyi.zhang@cs.ucla.edu) if you have any questions.

Acknowledgement

The project is supervised by my advisor, Miryung Kim at UCLA. We collaborate with Ganesha Upadhyaya and Hridesh Rajan from Iowa State Univeristy and use the distributed software mining infrastracture, BOA, built by their group. Anastasia Reinhart helps us build the Chrome extension of ExampleCheck during her internship at UCLA. This project is supported by AFRL grant FA8750-15-2-0075, and NSF grants CCF-1527923, CCF-1460325, CCF-1423370, CNS-1513263, and CCF-1518897. Anastasia Reinhardt’s internship at UCLA is supported by CRA-W Distributed Research Experiences for Undergraduates (DREU) program.