Compiler fuzzing through deep learning

Chris Cummins, Pavlos Petoumenos, Alastair Murray, Hugh Leather

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

252 Downloads (Pure)

Abstract

Random program generation-fuzzing-is an effective technique for discovering bugs in compilers but successful fuzzers require extensive development effort for every language supported by the compiler, and often leave parts of the language space untested. We introduce DeepSmith, a novel machine learning approach to accelerating compiler validation through the inference of generative models for compiler inputs. Our approach infers a learned model of the structure of real world code based on a large corpus of open source code. Then, it uses the model to automatically generate tens of thousands of realistic programs. Finally, we apply established differential testing methodologies on them to expose bugs in compilers. We apply our approach to the OpenCL programming language, automatically exposing bugs with little effort on our side. In 1,000 hours of automated testing of commercial and open source compilers, we discover bugs in all of them, submitting 67 bug reports. Our test cases are on average two orders of magnitude smaller than the state-of-the-art, require 3.03× less time to generate and evaluate, and expose bugs which the state-of-the-art cannot. Our random program generator, comprising only 500 lines of code, took 12 hours to train for OpenCL versus the state-of-the-art taking 9 man months to port from a generator for C and 50,000 lines of code. With 18 lines of code we extended our program generator to a second language, uncovering crashes in Solidity compilers in 12 hours of automated testing.

Original languageEnglish
Title of host publicationISSTA 2018 - Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis
EditorsEric Bodden, Frank Tip
PublisherAssociation for Computing Machinery
Pages95-105
Number of pages11
ISBN (Electronic)9781450356992
DOIs
Publication statusPublished - 12 Jul 2018
Event27th ACM SIGSOFT International Symposium on Software Testing and Analysis - Amsterdam, Netherlands
Duration: 16 Jul 201821 Jul 2018

Publication series

NameISSTA 2018 - Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis

Conference

Conference27th ACM SIGSOFT International Symposium on Software Testing and Analysis
Abbreviated titleISSTA 2018
Country/TerritoryNetherlands
CityAmsterdam
Period16/07/1821/07/18

Keywords

  • Compiler Fuzzing
  • Deep Learning
  • Differential Testing

Fingerprint

Dive into the research topics of 'Compiler fuzzing through deep learning'. Together they form a unique fingerprint.

Cite this