本文共 23451 字,大约阅读时间需要 78 分钟。
afl过程
In this article, we're going to talk about not the classical AFL itself but about utilities designed for it and its modifications, which, in our view, can significantly improve the quality of fuzzing. If you want to know how to boost AFL and how to find more vulnerabilities faster – keep on reading!
在本文中,我们将不讨论经典的AFL本身,而是讨论为其设计的实用程序及其修改,我们认为这可以显着提高模糊测试的质量。 如果您想知道如何提高AFL以及如何更快地发现更多漏洞,请继续阅读!
AFL is a coverage-guided, or feedback-based, fuzzer. More about these concepts can be found in a cool paper, . Let's wrap up general information about AFL:
AFL是一种覆盖率导向或基于反馈的模糊器。 有关这些概念的更多信息,请参见 。 让我们总结一下有关AFL的一般信息:
Repeats the preceding step to find where the program crashes.
重复上一步,找到程序崩溃的位置。
It’s highly effective, which is proven by practice.
它非常有效,实践证明。
Here's a graphic representation:
这是一个图形表示:
If you don't know what AFL is, here is a list of helpful resources for you to start:
如果您不知道什么是AFL,以下是一些有用的资源供您入门:
.
。
— a short intro to AFL.
的简短介绍。
— a simple demo of fuzzing C++ programs with AFL.
—使用AFL模糊化C ++程序的简单演示。
— a collection of the vulnerabilities found with AFL (hasn't been updated since 2017).
-AFL发现的漏洞的集合(自2017年以来未更新)。
you can read about the stuff AFL adds to a program during its build.
您可以了解AFL在其构建过程中添加到程序中的内容。
useful tips about fuzzing network applications.
有关模糊网络应用程序的有用技巧。
At the moment this article was being written, the latest version of AFL was . The fuzzer is in active development, and with time some side developments are being incorporated into the main AFL branch and grow irrelevant. Today, we can name several useful accessory tools, which are listed in the following chapter.
在撰写本文时,AFL的最新版本是 。 模糊测试器正在积极开发中,随着时间的推移,一些侧向开发已被并入AFL的主要分支中,并且变得无关紧要。 今天,我们可以命名几个有用的附件工具,这些工具将在下一章中列出。
Some AFL users that its author, Michal Zalewski, had apparently abandoned the project since the last modifications date to November 5, 2017. This may be connected to him leaving Google and working on some new projects. So, users started to make new themselves for the last current version 2.52b.
一些AFL用户 ,自从上次修改日期至2017年11月5日以来,其作者Michal Zalewski显然已放弃了该项目。这可能与他离开Google并从事一些新项目有关。 因此,用户开始为最新的当前版本2.52b制作新的 。
There are also different variations and derivates of AFL, which allows fuzzing Python, Go, Rust, OCaml, GCJ Java, kernel syscalls, or even entire VMs.
AFL也有不同的变体和派生版本,允许对Python,Go,Rust,OCaml,GCJ Java,内核syscall甚至整个VM进行模糊测试。
— — for Python.
— —用于Python。
— — for fuzzing programs written on Rust.
— —用于模糊编写在Rust上的程序。
— — afl-fuzz for javascript.
— — JavaScript的afl-fuzz。
— — AFL fuzzing for Java.
— — Java的AFL模糊测试。
— — another fuzzer for Java (an on the topic).
— — Java的另一个器(有关该主题的 )。
— — AFL-like fuzzer for JVM.
— — JVM的类似于AFL的 。
— — for fuzzing programs written on swift.
— —用于模糊以swift编写的程序。
— — for OCaml.
— —用于OCaml。
— — fuzzer based on afl for .net.
— —基于.net的afl的模糊器。
For this chapter, we've collected various scripts and tools for AFL and divided them into several categories:
在本章中,我们收集了用于AFL的各种脚本和工具,并将它们分为几类:
Crash processing 崩溃处理— a set of utilities for automatic processing/analysis of crashes and reducing the number of test cases.
一组实用程序,用于自动处理/分析崩溃并减少测试用例的数量。
— another crash analyzer for AFL.
-AFL的另一个崩溃分析器。
— a set of scripts for the analysis of results.
一组用于分析结果的脚本。
— a simple triage tool.
—一个简单的分类工具。
— afl-cmin on Python.
-Python上的afl-cmin。
— a tool that automatically generates builds of debian packages suitable for AFL.
一种自动生成适用于AFL的debian软件包构建的工具。
— a set of tools for working with input data.
一组用于处理输入数据的工具。
— provides human-friendly data about coverage.
提供有关覆盖率的人性化数据。
— ratio assessment. Script counts the number of instrumentation blocks in the binary.
-比率评估。 脚本计算二进制文件中检测块的数量。
— is like afl-cov but uses a clang sanitizer.
—与afl-cov相似,但使用了叮当消毒剂。
— a script for covering code and analysis by Cisco Talos Group.
—一个脚本,用于覆盖Cisco Talos Group的代码和分析。
— something like a collection of patches for AFL that modify the code to make it easier for the fuzzer to find branches.
-类似于AFL补丁程序的集合,这些代码会修改代码以使模糊测试者更容易找到分支。
— a wrapper for afl-tmin that tries to speed up the process of the minimization of test case by using many CPU cores.
— afl-tmin的包装,它试图通过使用许多CPU内核来加速最小化测试用例的过程。
— a variation of afl-tmin based on the ddmin algorithm.
mod-基于ddmin算法的afl-tmin的变体。
— is a fast utility for minimizing test cases by Tavis Ormandy based on parallelization.
—是Tavis Ormandy基于并行化用于最小化测试用例的快速实用程序。
— distributed fuzzing for AFL.
-AFL的分布式模糊测试。
— AFL distributed fuzzing framework.
— AFL分布式模糊测试框架。
— a tool for the execution of many AFL instances.
执行许多AFL实例的工具。
— management and execution of many synchronized AFL fuzzers on AWS cloud.
在AWS云上管理和执行许多同步的AFL模糊器。
— another script for running AFL in AWS.
用于在AWS中运行AFL的另一个脚本。
— fuzzing testing of the open source libraries with libFuzzer and AFL.
—使用libFuzzer和AFL对开源库进行模糊测试。
Recently, there has been published a very good article titled .
最近,发表了一篇很好的文章,标题为 。
Deployment, management, monitoring, reporting 部署,管理,监控,报告— is a set of patches and scripts for easily adding support for various non-x86 architectures for AFL.
—是一组补丁程序和脚本,用于轻松添加对AFL的各种非x86架构的支持。
— a few small scripts to simplify the management of AFL.
一些简化AFL管理的小脚本。
— a script for monitoring AFL.
用于监视AFL的脚本。
— a web server on Python for managing multi-afl.
-Python上的Web服务器,用于管理多afl。
— an image of a docker with afl-latest, afl-dyninst, and Triforce-afl.
具有afl-latest,afl-dyninst和Triforce-afl的码头工人的图像。
— a web server for the remote management of AFL instances.
用于远程管理AFL实例的Web服务器。
AFL had a very strong impact on the community of vulnerability researchers and fuzzing itself. It's not surprising at all that after some time people started making modifications inspired by the original AFL. Let's have a look at them. In different situations, each of these modifications has its own pros and cons compared to the original AFL.
AFL对脆弱性研究人员的社区及其模糊性产生了非常强烈的影响。 一段时间之后,人们开始根据原始AFL进行修改,这一点也就不足为奇了。 让我们看看它们。 在不同情况下,与原始AFL相比,每种修改都有其优缺点。
Almost all mods can be found at
几乎所有mod都可以在上找到
What for?
做什么的?
Increase the speed and/or code coverage
提高速度和/或代码覆盖率
Environment
环境
Working without source code
无需源代码即可工作
Code instrumentation
代码检测
Before going on with examining different modifications and forks of AFL, we have to talk about two important modes, which also had been modifications in the past but were eventually incorporated. They are Syzygy and Qemu.
在继续研究AFL的不同修改和分支之前,我们必须谈论两种重要的模式,它们在过去也曾是修改,但最终被合并。 他们是Syzygy和Qemu。
mode — is the mode of working in instrument.exe 模式-是Instrument.exe中的工作模式instrument.exe --mode=afl --input-image=test.exe --output-image=test.instr.exeSyzygy allows to statically rewrite PE32 binaries with AFL but requires symbols and an additional dev to make WinAFL kernel aware.
Qemu mode — the way it works under QEMU can be seen in . The support of working with binaries with QEMU was added to upstream AFL in Version 1.31b. AFL QEMU mode works with the added functionality of binary instrumentation into qemu tcg (a tiny code generator) binary translation engine. For that, AFL has a build script qemu, which extracts the sources of a certain version of qemu (2.10.0), puts them onto several small patches and builds for a defined architecture. Then, a file called afl-qemu-trace is created, which is in fact a file of user mode emulation of (emulation of only executable ELF files) qemu-. Thus, it is possible to use fuzzing with feedback on elf binaries for many different architectures supported by qemu. Plus, you get all the cool AFL tools, from the monitor with information about the current session to advanced stuff like afl-analyze. But you also get the limitations of qemu. Also, if a file is built with toolchain using hardware SoC features, which launches the binary and is not supported by qemu, fuzzing will be interrupted as soon as there is a specific instruction or a specific MMIO is used.
Qemu模式-它在QEMU下的工作方式可以在看到。 在版本1.31b中向上游AFL添加了使用QEMU处理二进制文件的支持。 AFL QEMU模式与qemu tcg(微型代码生成器)二进制翻译引擎中添加的二进制检测功能一起使用。 为此,AFL具有一个构建脚本qemu,该脚本可提取某个版本的qemu(2.10.0)的源代码,并将其放到几个小补丁中并为定义的体系结构构建。 然后,创建了一个名为afl-qemu-trace的文件,它实际上是qemu-的用户模式仿真文件(仅可执行ELF文件的仿真文件)。 因此,对于qemu支持的许多不同体系结构,可以将模糊与对elf二进制文件的反馈一起使用。 此外,您还可以获得所有很棒的AFL工具,从带有有关当前会话的信息的监视器到afl-analyze等高级内容。 但是您也得到qemu的限制。 另外,如果使用硬件SoC功能通过工具链构建文件,该文件会启动二进制文件,而qemu不支持该文件,则一旦有特定指令或使用特定MMIO,模糊处理就会被中断。
Here's of the qemu mode, where the speed was increased 3-4 times with TCG code instrumentation and cashing.
这是qemu模式的 ,其中通过TCG代码检测和兑现将速度提高了3-4倍。
Forks 前叉The appearance of forks of AFL is first of all related to the changes and improvements of the algorithms of the classic AFL.
AFL的前叉的出现首先与经典AFL的算法的更改和改进有关。
— A modification for fuzzing PE files that have no source code in the Windows OS. For its operation, the fuzzer analyzes a target program with IDA Pro and generates the information for the following static instrumentation. An instrumented version is then fuzzed with AFL.
—一种用于模糊处理Windows OS中没有源代码的PE文件的修改。 为了进行操作,模糊器使用IDA Pro分析目标程序,并为以下静态仪器生成信息。 然后使用AFL对已安装的版本进行模糊处理。
— is an attempt to port the classic AFL to Windows with Cygwin. Unfortunately, it has many bugs, it's very slow, and the development of has been abandoned.
试图通过Cygwin将经典的AFL移植到Windows。 不幸的是,它有很多错误,非常慢,并且已被放弃。
(extends AFL with Power Schedules) — one of the first AFL forks. It has added heuristics, which allow it to go through more paths in a short time period.
(用功率计划表扩展了AFL)—第一个AFL分支之一。 它增加了启发式功能,使它可以在短时间内通过更多路径。
— an extension for AFL, that targets rare branches.
-AFL的扩展,针对稀有分支。
— is an extension for AFL meant for getting to certain parts of code instead of full program coverage. It can be used for testing patches or newly added fragments of code.
—是AFL的扩展,旨在获取代码的某些部分而不是完整的程序覆盖范围。 它可以用于测试补丁或新添加的代码片段。
— an extension for AFL, that looks for test cases which could significantly slow down the program.
— AFL的扩展,用于寻找可能会严重降低程序速度的测试用例。
— is an extension for AFL that is meant to forecast how hard it is to find new paths.
是AFL的扩展,旨在预测寻找新路径的难度。
— is one of the latest fuzzers, written on rust. It uses new strategies for mutation and increasing the coverage.
-是最新的模糊测试工具之一,用Rust书写。 它使用新策略进行突变并增加覆盖范围。
— fuzzing with neural netwoks.
用神经网络模糊。
— integration of AFl with UnTracer for effective tracing.
—将AFl与UnTracer集成以进行有效跟踪。
— Practical Concolic Execution Engine Tailored for Hybrid Fuzzing. Essentially, it is a symbolic execution engine (basic components are realized as a plugin for intel pin) that together with AFL performs hybrid fuzzing. This is a stage in the evolution of feedback-based fuzzing and calls for a separate discussion. Its main advantage is that can do concolic execution relatively fast. This is due to the native execution of commands without intermediate representation of code, snapshots, and some heuristics. It uses the old Intel pin (due to support problems between libz3 and other DBTs) and currently can work with elf x86 and x86_64 architectures.
—专为混合模糊测试而设计的实用Concolic执行引擎。 本质上,它是一个符号执行引擎(基本组件实现为intel引脚的插件),与AFL一起执行混合模糊测试。 这是基于反馈的模糊测试发展的一个阶段,需要进行单独的讨论。 它的主要优点是可以相对较快地执行condicate执行。 这是由于命令的本机执行而没有代码,快照和某些启发式的中间表示。 它使用旧的Intel引脚(由于支持libz3和其他DBT之间的问题),目前可以与elf x86和x86_64体系结构一起使用。
— Greybox fuzzer, an obvious advantage of which is that along with an instrumented program it also gets specification of input data using the ANTLR grammar and after that performs mutations with the help of this grammar.
— Greybox模糊器,一个明显的优点是,它与仪器化程序一起,也可以使用ANTLR语法获取输入数据的规范,然后在此语法的帮助下执行变异。
— Another Graybox fuzzer. As input, it gets specification of input data in the format used by the Peach fuzzer.
另一个Graybox模糊测试器。 作为输入,它以桃子模糊器使用的格式获取输入数据的规范。
There are many research papers dedicated to the implementation of the new approaches and fuzzing techniques where AFL is modified. Only white papers are available, so we didn't even bother mentioning those. You can google them if you want. For example, some of the latest are , , , for AFL.
有许多研究论文致力于在AFL修改后实施新方法和模糊技术。 只有白皮书可用,因此我们什至不用提那些。 您可以根据需要搜索它们。 例如,最新的一些是 , , ,AFL 。
Modifications based on Qemu 基于Qemu的修改— AFL/QEMU fuzzing with full emulation of a system. A fork by nccgroup. Allows fuzzing the entire OS in qemu mode. It is realized with a special instruction (aflCall (0f 24)), which was added in QEMU x64 CPU. Unfortunately, it's no longer supported; the last version of AFL is 2.06b.
— AFL / QEMU模糊化并带有完整的系统仿真。 nccgroup的一个fork。 允许在qemu模式下模糊整个操作系统。 它是通过特殊指令(aflCall(0f 24))实现的,该指令已添加到QEMU x64 CPU中。 不幸的是,它不再受支持。 AFL的最新版本是2.06b。
— the fuzzing of Linux system calls.
-Linux系统调用的模糊处理。
— a small demo project with QEMU Augmented Instrumentation (qai).
— QEMU增强仪器(qai)的一个小型演示项目。
An example of working with this modification and .
使用此修改的示例和 。
Before we go on to the modifications based on the frameworks of dynamic binary instrumentation (DBI), let's not forget that the highest speed of these frameworks is shown by DynamoRIO, Dynlnst and, finally, PIN.
在继续进行基于动态二进制工具(DBI)框架的修改之前,请不要忘记DynamoRIO,Dynlnst和PIN最终显示了这些框架的最高速度。
PIN-based modifications 基于PIN的修改— AFL with Intel PIN instrumentation.
—带有Intel PIN工具的AFL。
— another AFL instrumentation realized through Intel PIN.
—通过Intel PIN实现的另一种AFL仪器。
— AFL with PINtool.
—带PINtool的AFL。
— A clone (of the basic core) of AFL fuzzer.
— AFL (基本核心)克隆。
— the author of this tool tried to port AFL to Windows for the fuzzing of already compiled binaries. Seems like it was done overnight just for fun; the project has never gone any further. The repository doesn't have sources, only compiled binaries and launch instruction. We don't know which version of AFL it's based on, and it only supports 32-bit applications.
该工具的作者试图将AFL移植到Windows,以模糊已编译的二进制文件。 似乎是为了娱乐而过夜。 该项目从未进行过。 该存储库没有源,只有编译的二进制文件和启动指令。 我们不知道它基于哪个版本的AFL,它仅支持32位应用程序。
As you can see, there are many different modifications, but they are not very very useful in real life.
如您所见,有许多不同的修改,但是它们在现实生活中不是很有用。
Dyninst-based modifications 基于Dyninst的修改 — American Fuzzy Lop + Dyninst == AFL balckbox fuzzing. The feature of this version is that first a researched program (without the source code) is instrumented statically (static binary instrumentation, static binary rewriting) with Duninst, and then is fuzzed with the classic AFL that thinks that the program is build with afl-gcc/afl-g++/afl-as ;) As a result, it allows is to work with a very good productivity without the source code — It used to be at 0.25x speed compared to a native compile. It has a significant advantage compared to QEMU: it allows the instrumentation of dynamic linked libraries, while QEMU can only instrument the basic executable file statically linked with libraries. Unfortunately, now it's only relevant for Linux. For Windows support, changes to Dyninst itself are needed, which is -American Fuzzy Lop + Dyninst == AFL黑箱模糊测试。 此版本的特点是,首先使用Duninst对研究的程序(无源代码)进行静态检测(静态二进制检测,静态二进制重写),然后对经典的AFL感到困惑,认为该程序是使用afl-构建的结果是,它允许在没有源代码的情况下以非常好的生产率工作—与本地编译相比,它的速度为0.25倍。 与QEMU相比,它具有显着的优势:它允许检测动态链接库,而QEMU仅可以检测与库静态链接的基本可执行文件。 不幸的是,现在它仅与Linux有关。 为了获得Windows支持,需要对Dyninst本身进行更改,此操作 . 。There's yet another with improved speed and certain features (the support of AARCH64 and PPC architectures).
还有另一个速度提高,某些功能(AARCH64和PPC架构的支持)。
Modifications based on DynamoRIO 基于DynamoRIO的修改— AFl + DynamoRIO – fuzzing without sources on Linux.
— AFl + DynamoRIO –在Linux上没有源时进行模糊测试。
— another realization based on DynamoRIO which very well described on .
dr-基于DynamoRIO的另一种实现,在上有很好的描述。
— a modification by vanhauser-thc. Here's what he says about it: «run AFL with DynamoRIO when normal afl-dyninst is crashing the binary and qemu mode -Q is not an option». It supports ARM and AARCH64. Regarding the productivity: DynamoRIO is about 10 times slower than Qemu, 25 times slower than dyninst, but about 10 times faster than Pintool.
-vanhauser-thc的修改。 他是这样说的:“当普通afl-dyninst使二进制文件崩溃且qemu模式-Q不可行时,用DynamoRIO运行AFL”。 它支持ARM和AARCH64。 关于生产率:DynamoRIO比Qemu慢10倍,比dyninst慢25倍,但比Pintool快10倍。
— the most famous AFL fork Windows. (DynamoRIO, also syzygy mode). It was only a matter of time for this mod to appear because many wanted to try AFL on Windows and apply it to apps without sources. Currently, this tool is being actively improved, and regardless of a relatively outdated code base of AFL (2.43b when this article is written), it helped to find several vulnerabilities (CVE-2016-7212, CVE-2017-0073, CVE-2017-0190, CVE-2017-11816). The specialists from Google Zero Project team and MSRC Vulnerabilities and Mitigations Team are working in this project, so we can hope for the further development. Instead of compilation time instrumentation, the developers used dynamic instrumentation(based on DynamoRIO), which significantly slowed down the execution of the analyzed software, but the resulting overhead (doubled) is comparable to that of the classic AFL in binary mode. They also solved the problem of fast process launch, having called it persistent fuzzing mode; they choose the function to fuzz (by the offset inside the file or by the name of function present in the export table) and instrument it so that it could be called in the cycle, thus launching several input data samples without restarting the process. An came out recently, describing how the authors found around 50 vulnerabilities in about 50 days using WinAFL. And shorty before it was published, Intel PT mode had been added to WinAFL; detalis can be found .
最著名的AFL前叉Windows。 (DynamoRIO,也是syzygy模式)。 这个mod的出现只是时间问题,因为许多人想在Windows上尝试AFL并将其应用于没有源代码的应用程序。 目前,此工具正在积极改进中,无论AFL的代码库相对过时(撰写本文时为2.43b),它都有助于发现一些漏洞(CVE-2016-7212,CVE-2017-0073,CVE- 2017年1月90日,CVE-2017-11816)。 Google零项目团队和MSRC漏洞与缓解团队的专家正在从事此项目,因此我们可以希望进一步发展。 开发人员使用了动态工具(基于DynamoRIO)来代替编译时工具,这大大减慢了分析软件的执行速度,但是所产生的开销(增加了一倍)与二进制模式下的经典AFL相当。 他们还称其为持续模糊模式,从而解决了进程快速启动的问题。 他们选择要模糊化的函数(通过文件内的偏移量或通过导出表中存在的函数名称)并进行检测,以便可以在循环中调用它,从而在不重新启动过程的情况下启动了多个输入数据样本。 最近了 ,描述了作者如何使用WinAFL在大约50天内发现了大约50个漏洞。 而且在发布之前不久,Intel PT模式已添加到WinAFL中。 详细资料可以在找到。
An advanced reader could notice that there are modifications with all the popular instrumentation frameworks except for . The only mention of the use of Frida with AFL was found in . A version of AFL with Frida is really useful because Frida supports several RISC architectures.
高级读者可能会注意到,除了之外,所有流行的检测框架都进行了修改。 在中找到了将Frida与AFL一起使用的唯一提及。 带有Frida的AFL版本非常有用,因为Frida支持多种RISC体系结构。
Many researches are also looking forward to the release of DBI Scopio framework by the creator of Capstone, Unicorn, and Keystone. Based on this framework, the authors have already created a fuzzer (Darko) and, according to them, successfully use it to fuzz embedded devices. More on this can be found in .
许多研究还期待Capstone,Unicorn和Keystone的创建者发布DBI Scopio框架。 基于此框架,作者已经创建了一个模糊器(Darko),并据此成功地将其用于模糊嵌入式设备。 有关更多信息,请参见 。
Modifications, based on processor hardware features 根据处理器硬件功能进行修改When it comes to AFL modifications with the support of processor hardware features, first of all, it allows fuzzing kernel code, and secondly — it allows for much faster fuzzing of apps without the source code.
在支持处理器硬件功能的情况下进行AFL修改时,首先,它可以对内核代码进行模糊处理,其次,它可以在没有源代码的情况下对应用进行快速模糊处理。
And of course, speaking about processor hardware features, we are most of all interested in (Processor Tracing). It is available from the 6th generation of processors onwards (approximately, since 2015). So, in order to be able to use the fuzzers listed below, you need a processor supporting Intel PT.
当然,谈到处理器的硬件功能,我们最感兴趣的是 (处理器跟踪)。 从第六代处理器开始可用(大约从2015年开始)。 因此,为了能够使用下面列出的模糊器,您需要支持Intel PT的处理器。
— a third-party WinAFL modification that uses Intel PT instead of DynamoRIO.
—使用Intel PT而不是DynamoRIO的第三方WinAFL修改。
— is an academic project aimed at solving the coverage-guided problem for the OS-independent fuzzing of the kernel. The problem is solving by using a hypervisor and Intel PT. More about it can be found in the white paper .
—是一个学术项目,旨在解决与操作系统无关的内核模糊的覆盖率指导问题。 问题是通过使用管理程序和Intel PT解决。 有关更多信息,请参见白皮书 。
As you can see, the area of AFL modifications is actively evolving. Still, there is room for experiments and creative solutions; you can create a useful and interesting new modification.
如您所见,AFL修改领域正在积极发展。 尽管如此,仍有实验和创新解决方案的空间。 您可以创建一个有用且有趣的新修改。
Thanks for reading us and good luck with fuzzing!
感谢您阅读我们,祝您好运!
Nikita Knyzhov
尼基塔·克尼佐夫(Nikita Knyzhov)
P.S. Thanks to the research center team, without whom this article would be impossible. PS感谢研究中心团队,没有他们,本文将是不可能的。翻译自:
afl过程
转载地址:http://xmbwd.baihongyu.com/