COMPILER ENGINEERING

LAB # 4: SYNTAX ANALYSIS (PARSING)

LEXICAL ANALYSIS SUMMARY

1. Start New Token 2. Read 1st character to start recognizing its type according to the algorithm specified in 3. Slide 3 3. Pass its Token (Lexeme Type) and Value Attribute send to Parser 4. Repeat Steps (1-3) 5. Repeat Until End

10-14/3/12

Department of Computer Science Compiler Engineering Lab

Start New TOKEN Read 1st Character If is Digit? If is Letter? Read Following Characters If any is digit or _? If all letters? Is RELOP? >, <, !, = Is AROP? +, -. /, *, =
10-14/3/12

TOKEN = NUM

TOKEN = ID Is a Keyword ?

Is 2nd Char (=)? TOEKN = AROP

TOKEN= RELOP

TOKEN=KEYWOR D

Department of Computer Science Compiler Engineering Lab

SYNTAX ANALYSIS (PARSING)

is the process of analyzing a text, made of a sequence of tokens to determine its grammatical structure with respect to a given (more or less) formal grammar. Builds Abstract Syntax Tree (AST) Part from an Interpreter or a Compiler Creates some form of Internal Representation (IR) Programming Languages tend to be written in Context-free grammar Efficient + fast Parsers can be written for them
10-14/3/12 Department of Computer Science Compiler Engineering Lab 4

PHASE 2 : SYNTAX ANALYSIS

also called sometimes Syntax Checking Ensures that:
the code is valid grammatically (without worrying about the meaning) and will sequence into an executable program.

The syntactical analyzer applies rules to the code; For example:

checking to make sure that each opening brace has a corresponding closing brace, and that each declaration has a type, and that the type exists .. etc
10-14/3/12 Department of Computer Science Compiler Engineering Lab 5

CONTEXT-FREE GRAMMAR
Defines the components that forms an expression + defines the order they must appear in A context-free grammar is a set of rules specifying how syntactic elements in some language can be formed from simpler ones The grammar specifies allowable ways to combine tokens(called terminals), into higher-level syntactic elements (called non-terminal)

10-14/3/12

Department of Computer Science Compiler Engineering Lab

CONTEXT-FREE GRAMMAR
Ex.:
Any ID is an expression (Preferred to say TOKEN) Any Number is an expression (Preferred to say TOKEN) If Expr1 and Expr2 are expressions then:
Expr1+ Expr2 are expressions Expr1* Expr2 are expressions

If id1 and Expr2 are expressions then:

Id1 = Expr2 is a statement

If Expr1and Statement 2 then

While (Expr1) If (Expr1) then are statements
10-14/3/12

do Statement 2, Statement 2

Department of Computer Science Compiler Engineering Lab

GRAMMAR & AST

TOKEN (terminals) = Leaf Expressions, Statements (Non-Terminals) = Nodes
Lexical Analysis

Stream of Characters

Stream of TOKENs

Stream of TOKENs

Syntax Analysis

Abstract Syntax Tree (AST)

10-14/3/12

Department of Computer Science Compiler Engineering Lab

PHASE 2 : SYNTAX ANALYSIS

10-14/3/12

Department of Computer Science Compiler Engineering Lab

PHASE 2 : SYNTAX ANALYSIS

Token Token Tokens

Syntax Analyzer (Parser)

10-14/3/12

Department of Computer Science Compiler Engineering Lab

10

SYMBOL TABLE
A Symbol Table is a data structure containing a record for each identifier with fields for the attributes of an ID Tokens formed are recorded in the ST Purpose:
To analyze expressions\statements, that is a hierarchal or nesting structure is required Data structure allows us to: find, retrieve, store a record for an ID quickly.
For example: in Semantic Analysis Phase + Code Generation phase retrieve ID Type to Type Check and Implementation purposes
10-14/3/12 Department of Computer Science Compiler Engineering Lab 11

SYMBOL TABLE MANAGEMENT

The Symbol Table may contain any of the following information:
For an ID:
The storage allocated for an ID, its TYPE, Its Scope (Where its valid in the program)

For a function also:

Number of Arguments Types of Arguments Passing Method (By Reference or By Value) Return Type

Identifiers will be added if they dont already exist

10-14/3/12 Department of Computer Science Compiler Engineering Lab 12

SYMBOL TABLE MANAGEMENT

Not all attributes can always be determined by a lexical analyzer because of its linear nature E.g. dim a, x as integer
In this example the analyzer at the time when seeing the IDs has still unreached the type keyword

So, following phases will complete filling IDs attributes and using them as well
For example: the storage location attribute is assigned by the code generator phase

10-14/3/12

Department of Computer Science Compiler Engineering Lab

13

ERROR DETECTION & REPORTING

In order the Compilation process proceed correctly, Each phase must:
Detect any error Deal with detected error(s)

Errors detection:
Most in Syntax + Semantic Analysis In Lexical Analysis: if characters arent legal for token formation In Syntax Analysis: violates structure rules In Semantic Analysis: correct structure but wrong invalid meaning (e.g. ID = Array Name + Function Name)
Department of Computer Science Compiler Engineering Lab

10-14/3/12

14

COMPILER PHASES

10-14/3/12

Department of Computer Science Compiler Engineering Lab

15

LEXICAL ANALYZER & SYMBOL TABLE

Lexical Analyzer

Token Token ID Type Id1 expr1 1d2 Expr2 Id3 Expr3 ID AROP ID AROP ID AROP

Token Value position ASS Initial SUM Rate MUL

Location

N1
10-14/3/12

Num

60
Department of Computer Science Compiler Engineering Lab 16

SYNTAX ANALYZER & SYMBOL TABLE

10-14/3/12

Department of Computer Science Compiler Engineering Lab

17

SYNTAX ANALYZER & SYMBOL TABLE

A LEAF is a record with two or more fields One to identify the TOKEN and others to identify info attributes Token ID Id1 expr1 1d2 Expr2 Id3 Expr3 N1
10-14/3/12

Token Type ID AROP ID AROP ID AROP NUM

Token Value position ASS Initial SUM Rate MUL 60

Location

Department of Computer Science Compiler Engineering Lab

18

SYNTAX ANALYZER & SYMBOL TABLE

An interior NODE is a record with a field for the operator and two fields of pointers to the left and right children Operator Expr1 Expr2 Expr3 Left Child (Pointer) id1 id2 id3 Right Child (Pointer) Expr2 Expr3 N1

10-14/3/12

Department of Computer Science Compiler Engineering Lab

19

TASK 1: THINK AS A COMPILER!

Analyze the following program syntactically: int main() { std::cout << "hello world" << std::endl; return 0; }

10-14/3/12

Department of Computer Science Compiler Engineering Lab

20

LEXICAL ANALYZER OUTPUT

1 = string "int 2 = string "main 3 = opening parenthesis 4 = closing parenthesis 5 = opening brace 6 = string "std 7 = namespace operator 8 = string "cout 9 = << operator 10 = string ""hello world" 11 = string "endl 12 = semicolon 13 = string "return 14 = number 0 15 = closing brace
Department of Computer Science Compiler Engineering Lab 21

10-14/3/12

TASK 2: A STATEMENT AST

Create an abstract syntax tree for the following code for the Euclidean algorithm: while b 0 if a > b a := a b else b := b a return a

10-14/3/12

Department of Computer Science Compiler Engineering Lab

22

TASK 2: A STATEMENT AST

10-14/3/12

Department of Computer Science Compiler Engineering Lab

23

LAB ASSIGNMENT
Write the Syntax Analyzer Components and Ensure fulfilling the following : Create a Symbol Table (for all types including IDs, Functions, .. Etc) Fill the Symbol Table with Tokens extracted from the Lexical Analysis phase Differentiate between Node and Leaf Applying grammar rules (tokens, expressions, statements)

10-14/3/12

Department of Computer Science Compiler Engineering Lab

24

QUESTIONS?
Thank you for listening

10-14/3/12

Department of Computer Science Compiler Engineering Lab

25