Role of lexical analysis and its issues

The lexical analyzer is the first phase of compiler. Its main task is to read the input characters and produce as output a sequence of tokens that the parser uses for syntax analysis            

  • It is implemented by making lexical analyzer be a subroutine
  • Upon receiving a “get next token” command from parser, the lexical analyzer reads the input character until it can identify the next token
  • It may also perform secondary task at user interface
  • One such task is stripping out from the source program comments and white space in the form of blanks, tabs, and newline characters
  • Some lexical analyzer are divided into cascade of two phases, the first called scanning and second is “lexical analysis”.
  • The scanner is responsible for doing simple task while lexical analysis does the more complex task
Issues in Lexical Analysis:
There are several reasons for separating the analysis phase of compiling into lexical analysis and parsing:
  • Simpler design is perhaps the most important consideration. The separation of lexical analysis often allows us to simplify one or other of these phases.
  • Compiler efficiency is improved
  • Compiler portability is enhanced

Leave a Reply