GCC Internals: A Conceptual View Part I

Abhijat Vichare
CFDVS, Indian Institute of Technology, Bombay

January 2008

A.Vichare

GCC Internals

Plan

Part I GCC: Conceptual Structure C Program through GCC Building GCC

Part II Gimple The MD-RTL and IR-RTL Languages in GCC GCC Machine Descriptions

A.Vichare

GCC Internals

Part I GCC Architecture Concepts

A.Vichare

GCC Internals

The GNU Tool Chain

Source Program

(1:1:2)

gcc

Target Program
A.Vichare GCC Internals

The GNU Tool Chain

Source Program

(1:1:2)

cc1

cpp

gcc

Target Program
A.Vichare GCC Internals

The GNU Tool Chain

Source Program

(1:1:2)

cc1

cpp

gcc

Target Program
A.Vichare GCC Internals

The GNU Tool Chain

Source Program

(1:1:2)

cc1

cpp

gcc

as

Target Program
A.Vichare GCC Internals

The GNU Tool Chain

Source Program

(1:1:2)

cc1

cpp

gcc

as

ld

Target Program
A.Vichare GCC Internals

The GNU Tool Chain

Source Program

(1:1:2)

cc1

cpp

gcc

as glibc/newlib ld

Target Program
A.Vichare GCC Internals

The GNU Tool Chain

Source Program

(1:1:2)

cc1

cpp

gcc

as

GCC glibc/newlib

ld

Target Program
A.Vichare GCC Internals

Usual Compilation Phase Sequence vs. GCC

A Typical Text Book Compiler Phase Sequence
Parsing Semantic Analysis Optimization Target Code Generation

A.Vichare

GCC Internals

Usual Compilation Phase Sequence vs. GCC

A Typical Text Book Compiler Phase Sequence
Parsing Semantic Analysis Optimization Target Code Generation

GCC is: Retargetable: Can generate code for many back ends Re-sourcable: Can accept code in many HLLs

A.Vichare

GCC Internals

Usual Compilation Phase Sequence vs. GCC

A Typical Text Book Compiler Phase Sequence
Parsing Semantic Analysis Optimization Target Code Generation

The GCC Phase Sequence looks like

Parsing Semantic Analysis Optimization Target Code Generation

GCC is: Retargetable: Can generate code for many back ends Re-sourcable: Can accept code in many HLLs

A.Vichare

GCC Internals

Usual Compilation Phase Sequence vs. GCC

A Typical Text Book Compiler Phase Sequence
Parsing Add HLL selection Theability Phase GCC Semantic Analysis Optimization Target Code Generation Add back end Code Gen. ability Target Code Generation

Sequence

Parametrise wrt. front and looksend back like

Parsing

Semantic Analysis

Optimization

GCC is: Retargetable: Can generate code for many back ends Re-sourcable: Can accept code in many HLLs

A.Vichare

GCC Internals

Implications of Retargetability in GCC

Retargetability Choose target at build time than at development time

Hence : there are Three time durations associated with GCC

1 2 3

tdevelop : The Development time (the gcc developer view) tbuild : The Build time (the gcc builder view) top : The Operation time (the gcc user view)

The downloaded GCC sources . . . . . . correspond to the gcc developer view, and . . . are ready for gcc builder view.

A.Vichare

GCC Internals

The GCC Compiler Generation Framework

(2:1:3)

GCC
HLL Specic Code, per HLL Language and Machine Independent Generic Code Machine dependent Generator Code Set of Machine Descriptions

tdev

Parser

Genericizer

Gimplier

Tree SSA Optimizer cc1/gcc

RTL Generator

Optimizer

Code Generator

A.Vichare

GCC Internals

The GCC Compiler Generation Framework

Choose HLL

(2:1:3)

GCC
HLL Specic Code, per HLL Language and Machine Independent Generic Code Machine dependent Generator Code Set of Machine Descriptions

tdev

Selected

tbuild
Tree SSA Optimizer cc1/gcc RTL Generator Code Generator

Parser

Genericizer

Gimplier

Optimizer

A.Vichare

GCC Internals

The GCC Compiler Generation Framework

Choose HLL

(2:1:3)

GCC
HLL Specic Code, per HLL Language and Machine Independent Generic Code Copied Machine dependent Generator Code Set of Machine Descriptions

tdev

Selected

tbuild
Tree SSA Optimizer cc1/gcc RTL Generator Code Generator

Parser

Genericizer

Gimplier

Optimizer

A.Vichare

GCC Internals

The GCC Compiler Generation Framework

Choose HLL Choose Target MD

(2:1:3)

GCC
HLL Specic Code, per HLL Language and Machine Independent Generic Code Copied Machine dependent Generator Code Set of Machine Descriptions

tdev

Selected

Generated

tbuild
Tree SSA Optimizer cc1/gcc RTL Generator Code Generator

Parser

Genericizer

Gimplier

Optimizer

A.Vichare

GCC Internals

The GCC Compiler Generation Framework

(2:1:3)

GCC
HLL Specic Code, per HLL Language and Machine Independent Generic Code Machine dependent Generator Code Set of Machine Descriptions

tdev

Parser

Genericizer

Gimplier

Tree SSA Optimizer cc1/gcc

RTL Generator

Optimizer

Code Generator

top

Source Program

Assembly Program

A.Vichare

GCC Internals

Is GCC complex?
As a Compiler . . . . . . Architecture? Not quite! . . . Implementation? Very much! Architecture wise:
1

Supercially: GCC is similar to typical compilers! Deeper down: Dierences are due to: Retargetability GCC can be (and is) used as a Cross Compiler !

Implementation wise: . . . ? (Next slides)

A.Vichare

GCC Internals

Some Interesting Facts about GCC 4.0.2

Pristine compiler sources (downloaded tarball) Lines of C code Lines of MD code Lines of total code Total Authors (approx) Backend directories 1098306 217888 1316194 63 34

(1:1:3)

For the targetted (= pristine + generated) C compiler Total Total Total Total Total lines of code lines of pure code pure code WITHOUT #include number of #include directives #include les 810827 606980 602351 4629 336

A.Vichare

GCC Internals

Some Interesting i386 MD Facts

(1:1:4)

General information Number of .md les Number of C les 8 72

Realistic code size information (excludes comments) Total Total Total Total lines lines lines lines of of of of code .md code header code C code 47290 23566 9986 16961

A.Vichare

GCC Internals

Part II C Program through GCC

A.Vichare

GCC Internals

C Program: Journey through GCC

Practically . . . The Source Conceptually Input int f(char *a) { int n = 10; int i, g; i = 0; while (i < n) { a[i] = g * i + 3; i = i + 1; } return i; }

(1:1:6,7)

A.Vichare

GCC Internals

C Program: Journey through GCC

Practically . . . Simplied AST Conceptually Input Parse (AST)
FnDecl RetType Decl Body StmtList

(1:1:6,7)

Args

Stmt1

modify expr i 0 while stmt bool expr Body

Stmt2

A.Vichare

GCC Internals

C Program: Journey through GCC

Practically . . . Gimple IR Conceptually Input Parse (AST) IR1 (Gimple)

(1:1:6,7)

f (a) { unsigned int i.0; char * i.1; char * D.1140; int D.1141; ... goto <D1136>; <D1135>: ... D.1140 = a + i.1; D.1141 = g * i; ... <D1136>: if (i < n) goto <D1135>; ... }
A.Vichare GCC Internals

C Program: Journey through GCC

Practically . . . Tree SSA form Conceptually Input Parse (AST) IR1 (Gimple) Optimization

(1:1:6,7)

f (a) { ... int D.1144; ... <bb 0>: n_2 = 10; i_3 = 0; goto <bb 2> (<L1>); <L0>: ... D.1140_9 = a_8 + i.1_7; D.1141_11 = g_10 * i_1; ... <L1>:; if (i_1 < n_2) goto <L0>; else ...; ... }
A.Vichare GCC Internals

C Program: Journey through GCC

Practically . . . Conceptually Input Parse (AST) IR1 (Gimple) Optimization IR2 (RTL) RTL IR (fragment)

(1:1:6,7)

(insn 21 20 22 2 (parallel [ (set (reg:SI 61 [ D.1141 ]) (mult:SI (reg:SI 66) (mem/i:SI (plus:SI (reg/f:SI 54 ...) (const_int -8 ...))))) (clobber (reg:CC 17 flags)) ]) -1 (nil) (nil))

A.Vichare

GCC Internals

C Program: Journey through GCC

Practically . . . Final ASM (partial) Conceptually Input Parse (AST) IR1 (Gimple) Optimization IR2 (RTL) ASM Code
.file "sample.c" ... f: pushl %ebp ... movl -4(%ebp), %eax imull -8(%ebp), %eax addb $3, %al ... leave ret ...

(1:1:6,7)

A.Vichare

GCC Internals

Front End Processing Sequence in cc1 and GCC

toplev_main () general_init () decode_options () do_compile () compile_file() lang_hooks.parse_file () c_parse_file () c_parser_translation_unit () c_parser_external_declaration () c_parser_declaration_or_fndef () finish_function () /* TO: Gimplification */ toplev.c toplev.c toplev.c toplev.c toplev.c toplev.c c-parser.c c-parser.c c-parser.c c-parser.c c-decl.c

(2:1:5)

Tip Use the functions above as breakpoints in gdb on cc1.

A.Vichare

GCC Internals

GIMPLE Phase sequence in cc1 and GCC

(2:1:10)

Creating GIMPLE representation in cc1 and GCC c_genericize() c-gimplify.c gimplify_function_tree() gimplify.c gimplify_body() gimplify.c gimplify_stmt() gimplify.c gimplify_expr() gimplify.c lang_hooks.callgraph.expand_function() tree_rest_of_compilation() tree-optimize.c tree_register_cfg_hooks() cfghooks.c execute_pass_list() passes.c /* TO: Gimple Optimisations passes */

A.Vichare

GCC Internals

The Tree passes list

(Partial) Passes list (tree-optimize.c) ( 70 passes) pass_remove_useless_stmts pass_lower_cf pass_all_optimizations pass_build_ssa pass_dce pass_loop pass_complete_unroll pass_loop_done pass_del_ssa pass_warn_function_return pass_expand pass_rest_of_compilation // // // // // // // // // // // // Pass Pass Optimiser Optimiser Optimiser Optimiser Optimiser Optimiser Optimiser Optimiser RTL Expander RTL passes

(2:1:11)

A.Vichare

GCC Internals

GCC Tree Passes: Code organisation

(2:1:13)

Tree Pass Organisation Data structure records pass info: name, function to execute etc. (struct tree opt pass in tree-pass.h) Instantiate a struct tree opt pass variable in each pass le. List the pass variables (in passes.c). Dead Code Elimination (tree-ssa-dce.c) struct tree_opt_pass pass_dce = { "dce", // pass name tree_ssa_dce, // fn to execute NULL, // sub passes ... // and much more };

A.Vichare

GCC Internals

RTL Pass Structure in cc1 and GCC

Gimple non-strict RTL translation non-strict RTL passes information extraction & optimisations non-strict strict RTL passes /* non strict RTL expander pass */ pass_expand_cfg expand_gimple_basic_block () expand_expr_stmt () expand_expr () /* TO: non strict RTL passes: * pass_rest_of_compilation */

(2:1:19)

cfgexpand.c cfgexpand.c stmt.c stmt.c

A.Vichare

GCC Internals

RTL Passes
Driver: passes.c:rest of compilation ()

(2:1:20)

Basic Structure: Sequence of calls to rest of handle * () + bookkeeping calls. (over 40 calls!) Bulk of generated code used here! (generated code in: $GCCBUILDDIR/gcc/*.[ch]) Goals:
Optimise RTL Complete the non strict RTL

Manipulate
either the list of RTL representation of input, or contents of an RTL expression, or both.

Finally: call rest of handle final ()

A.Vichare

GCC Internals

RTL Target ASM

(2:1:26)

passes.c:rest of handle final() calls assemble_start_function (); final_start_function (); final (); final_end_function (); assemble_end_function (); varasm.c final.c final.c final.c varasm.c

A.Vichare

GCC Internals

Part III Building GCC

A.Vichare

GCC Internals

Building a Compiler: General issues I

(1:1:14)

Some Terminology The sources of a compiler are compiled (i.e. built) on machine X X is called as the Build system The built compiler runs on machine Y Y is called as the Host system The compiler compiles code for target Z Z is called as the Target system Note: The built compiler itself runs on the Host machine and generates executables that run on Target machine!!!

A.Vichare

GCC Internals

Building a Compiler: General issues II

(1:1:15)

Some Denitions Note: The built compiler itself runs on the Host machine and generates executables that run on Target machine!!! A few interesting permutations of X, Y and Z are: X=Y=Z X=Y=Z X=Y=Z Native build Cross compiler Canadian Cross compiler

Example Native i386: built on i386, hosted on i386, produces i386 code. Sparc cross on i386: built on i386, hosted on i386, produces Sparc code.

A.Vichare

GCC Internals

Building a Compiler:

Bootstrapping A compiler is just another program It is improved, bugs are xed and newer versions are released To build a new version given a built old version:
1 2

Stage 1: Build the new compiler using the old compiler Stage 2: Build another new compiler using compiler from stage 1 Stage 3: Build another new compiler using compiler from stage 2 Stage 2 and stage 3 builds must result in identical compilers

Building cross compilers stops after Stage 1!

A.Vichare

GCC Internals

GCC Code Organization Overview

GCC Components are: Build conguration les Compiler sources Emulation libraries Language Libraries (except C) Support software (e.g. garbage collector)

(1:1:11)

Our conventions GCC source directory : $(GCCHOME) GCC build directory : $(GCCBUILDDIR) GCC install directory : $(GCCINSTALLDIR) $(GCCHOME) = $(GCCBUILDDIR) = $(GCCINSTALLDIR)

A.Vichare

GCC Internals

The GCC Build System I

(1:1:16)

Some Information Build-Host-Target systems inferred for native builds Specify Target system for cross builds Build Host systems: inferred Build-Host-Target systems can be explicitly specied too For GCC: A system = three entities
cpu vendor os

e.g. sparc-sun-sunos, i386-unknown-linux, i386-gcc-linux

A.Vichare

GCC Internals

The GCC Build System II

Basic GCC Building How To prompt$ cd $GCCBUILDDIR prompt$ configure <options>

(1:1:17,19)

Specify target: optional for native builds, necessary for others (option --target=<host-cpu-vendor string>) Choose source languages (option --enable-languages=<CSV lang list (c,java)) Specify the installation directory (option --prefix=<absolute path of $(GCCBUILDDIR)>) configure output: customized Makefile

prompt$ make 2> make.err > make.log prompt$ make install 2> install.err > install.log Tip Run configure in $(GCCBUILDDIR). See $(GCCHOME)/INSTALL/.
A.Vichare GCC Internals

Adding a New MD
To add a new backend to GCC Dene a new system name, typically a triple. e.g. spim-gnu-linux Edit $GCCHOME/config.sub to recognize the triple Edit $GCCHOME/gcc/config.gcc to dene
any backend specic variables any backend specic les $GCCHOME/gcc/config/<cpu> is used as the backend directory

(1:1:18)

for recognized system names. Tip Read comments in $GCCHOME/config.sub & $GCCHOME/gcc/config/<cpu>.
A.Vichare GCC Internals

The GCC Build Process I

(1:1:20)

GCC builds in two main phases: Adapt the compiler source for the specied build/host/target systems Consider a cross compiler:
Find the target MD in the source tree Include MD info into the sources (details follow)

Compile the adapted sources NOTE:

Incomplete MD specications Unsuccessful build Incorrect MD specication Run time failures/crashes (either ICE or SIGSEGV)

A.Vichare

GCC Internals

The GCC Build Process

Adapting the Compiler Sources

(1:1:21)

make rst compiles and runs a series of programs that process the target MD Typically, the program source le names are prexed with gen The $GCCHOME/gcc/gen*.c programs
read the target MD les, and extract info to create & populate the main GCC data structures

Example Consider genconstants.c:

<target>.md may dene UNSPEC * constants. genconstants.c reads UNSPEC * constants genconstants.c generates corresponding #defines Collect then into the insn-constants.h #include "insn-constants.h" in the main GCC sources
A.Vichare GCC Internals

The GCC Build Process

Adapting the Compiler Sources Pictorial view
GCC Sources Target Compiler Source Generation Target Compiler Data Structures

struct c_test insn_conditions[], size_t n_insn_conditions genconditions genconstants libiberty.a gensupport.c rtl.c readrtl.c printrtl1.c errors.c bitmap.c ggcnone.c genflags genconfig gencodes genattr genemit genextract genopinit genpeep insnconditions.c insnconstants.c insnflags.h insnconfig.h insncodes.h insnattr.h insnemit.c insnextract.c insnopinit.c insnpeep.c HAVE_ATTR_(md_inst_attribs) RTX exmission functions for every insn in MD file Extract operands of RTL instructions in MD file Writes a function that initialises an array with the code for each insn/expand in MD file Extract peephole optimisation information in MD files GCC_INSN_CONSTANTS_H HAVE_(md instructions) enum insn_code { CODE_FOR_(md inst)= ..
...

};

Target Machine Description

A.Vichare

GCC Internals

Building GCC Summary

Choose the source language: C (--enable-languages=c) Choose installation directory: (--prefix=<absolute path>) Choose the target for non native builds: (--target=sparc-sunos-sun) Run: configure with above choices Run: make to
generate target specic part of the compiler build the entire compiler

(1:1:23)

Run: make install to install the compiler Tip Redirect all the outputs: $ make > make.log 2> make.err
A.Vichare GCC Internals