May 1st, 2014 at 7:08 pm
that.
Clang is moving forward very fast. Today, to develop tools similar to the one presented in that post one should use the "tooling" library (libTooling). I hinted at its existence even in that post, but it was still at its infancy then. Now libTooling
And adds some descriptive comments:
The good thing about this repository is that it comes complete with a makefile which builds the tools & samples out of tree, meaning that they can reside anywhere as long as the paths to LLVM & Clang headers and libraries are
set up correctly. If you’ve ever tried to build an out-of-tree tool using LLVM or Clang as a library, you will not underestimate the utility of having this makefile available and continuously updated
Anyway, it’s time to explain the code.
The low-level details of the sample haven’t changed much. An implementation of the ASTConsumer interface
chooses which AST parsing events to handle. MyASTConsumer implements HandleTopLevelDecl,
so it will be invoked on each top-level declaration in the AST. It uses an implementation of RecursiveASTVisitor to
do the actual traversal over declaration AST nodes. MyASTVisitor implements VisitFunctionDecl to
be able to emit comments before and after function definitions, and VisitStmt to
detect if statements and analyze them.
Finally, a Rewriter is used
to manage the textual changes to the source code. This is how Clang-based source to source transformations work – analysis of the AST lets us find out where to apply changes, and a Rewriter is
The link between the two is the very accurate source location information managed by the AST for each node.
What’s really new about this sample is how it turns this low-level transformation logic into an actual tool. With libTooling, we no longer need to laboriously set up an instance of the compiler front-end manually. The ClangToolclass
does it all for us, while CommonOptionsParser simplifies
the command-line interface of the tool. All we need is to implement a FrontendAction –
MyFrontendAction is very simple. All it
does is provide our custom AST consumer in CreateASTConsumerand
sets up the Rewriter. Note that ClangTool will
create a new MyFrontendAction instance per file, so
it’s correct to have a new Rewriter in each instance.
This is enabled by adding another layer of abstraction – theFrontendActionFactory.
The factory pattern here is used to decouple the process of creating objects implementing FrontendAction from
the concrete FrontendAction subclasses used by our
tool. It also lets us customize the creation process to a greater extent – for example passing additional information into eachMyFrontendAction object.
In our sample, advanced features aren’t really needed so we use the conveniencenewFrontendActionFactory template
which creates a basic factory behind the scenes.
It’s important to spend a few minutes to discuss how to invoke the tool. libTooling has the concept of a compilation database, which tells tools about the compiler options used to build the sources under consideration.
Describing compilation databases in detail is out of scope here, and I may devote a separate post to it in the future, but in the meantime it helps knowing that a compilation database isn’t strictly required to run tools.
The libTooling command-line parser (CommonOptionsParser)
supports providing compiler flags on the command line, following the special flag --.
Think of it as a simplified, ad-hoc version of a compilation database.
you can invoke the sample discussed here as follows:
Note the -- at
the end: this invocation means there are no special flags the tool needs to know about while analyzing the file.
This post provides an introduction into building a source-to-source transformation tool based on Clang’s libTooling. It should provide a foundation for building custom tools, and nicely complements the existing Clang libTooling documentation
using libTooling these days.
<a target="_blank" href="http://eli.thegreenplace.net/2014/05/01/modern-source-to-source-transformation-with-clang-and-libtooling/#id1">[1]</a>
Clang tools based on libTooling.
<a target="_blank" href="http://eli.thegreenplace.net/2014/05/01/modern-source-to-source-transformation-with-clang-and-libtooling/#id2">[2]</a>
Note that the code presented here uses low-level APIs. libTooling has some more convenient higher-level APIs which I hope to talk about in future posts.
<a target="_blank" href="http://eli.thegreenplace.net/2014/05/01/modern-source-to-source-transformation-with-clang-and-libtooling/#id3">[3]</a>
A FrontendAction is used throughout the main Clang
compiler as well. For example, the Clang code generator – the component responsible for emitting LLVM IR from the AST – defines a subclass named CodeGenActionwhich
is its main entry point.
Related posts:
<a target="_blank" href="http://eli.thegreenplace.net/2012/06/08/basic-source-to-source-transformation-with-clang/">Basic source-to-source transformation with Clang</a>
<a target="_blank" href="http://eli.thegreenplace.net/2014/01/16/building-gcc-4-8-from-source-on-ubunu-12-04/">Building gcc 4.8 from source on Ubunu 12.04</a>
<a target="_blank" href="http://eli.thegreenplace.net/2011/07/03/parsing-c-in-python-with-clang/">Parsing C++ in Python with Clang</a>
<a target="_blank" href="http://eli.thegreenplace.net/2012/07/05/how-clang-handles-the-type-variable-name-ambiguity-of-cc/">How Clang handles the type / variable name ambiguity of C/C++</a>
<a target="_blank" href="http://eli.thegreenplace.net/2012/12/17/dumping-a-c-objects-memory-layout-with-clang/">Dumping a C++ object’s memory layout with Clang</a>