v3 flag enabled
Log In
Log In

Fuzzy testing 2

News

Fuzz Testing

Introduction

What is fuzz testing? Fuzzing is a testing technique that injects random pieces of data to a software function to uncover crashes and vulnerabilities. It helps improving code security and reliability, since it can trigger edge cases that went unnoticed during unit testing.

How does it work? Fuzz testing relies on a fuzzing engine, a library that runs your code in a loop, injecting different inputs at each iteration. The fuzzing engine will instrument your code to measure coverage, and use this information to drive the generation of samples. Most of the samples will contain malformed input, and will test your code’s tolerance to ill-formed inputs.

Which kind of errors does fuzzing detect? The fuzzing engine will monitor your code for crashes. Fuzzing is often used with the address and undefined sanitizers. In short, fuzzing will make sure that your code doesn’t crash, leak or incur in undefined behavior, regardless of how malformed the input is. A lot of vulnerabilities in C++ code are related to the former kind of errors, so fuzzing can make your code more secure.

Should I use it? Fuzz testing is specially relevant for libraries that process potentially untrusted, user-controlled input, like network data. Libraries that implement parsers, decoders or network protocols usually benefit from fuzz testing.

Which Boost libraries use it? Libraries like https://www.boost.org/libs/json">Boost.Json, https://www.boost.org/libs/url">Boost.URL and https://www.boost.org/libs/mysql">Boost.Mysql use this technique - if you’re about to implement it in your library, have a look at what these libraries do.

Should I still write unit tests? Yes. Absolutely. Fuzzing does not replace unit tests, but complements them. Unit tests verify that your code produces the intended results by providing known inputs and running assertions on the outputs. In fuzz testing, inputs are generated randomly by the fuzzing engine, so no assertions are usually run on the outputs - fuzzing will only monitor for crashes and memory errors.

How can I add fuzzing to my library? We recommend using https://llvm.org/docs/LibFuzzer.html">LibFuzzer, since it’s the easiest fuzzing engine to use, and the one that other Boost libraries use. You can use other fuzzing engines if you prefer.

LibFuzzer Basics

Quoting documentation, "LibFuzzer is an in-process, coverage-guided, evolutionary fuzzing engine". LibFuzzer will run your code multiple times with different, random inputs. It will instrument your code to measure coverage, and will attempt to generate inputs that maximize it, effectively trying to discover new paths in your code.

LibFuzzer is included in clang, so you don’t need to install anything to get started.

Let’s say we want to fuzz a function that parses JSON data, like parse_json(string_view input). We will create a source file with the following code:

#include <string_view>
#include <your/parsing/function.hpp>

extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size)
{
    // The range [data, data+size) contains the data generated by the fuzzer
    std::string_view input_data (reinterpret_cast<const char*>(data), size);
    parse_json(input_data);
    return 0;
}

We can build a fuzzer executable by adding -fsanitize=fuzzer to clang’s compile and link flags. This will automatically link LibFuzzer to your code. It’s advised to also enable the address and undefined sanitizers, which increases the range of errors detected by the fuzzer. We recommend building in release mode with debug symbols enabled, so crashes are symbolized correctly.

Related Posts