241 Computer Science Certification Practice Exam

Lexical analysis is the process of breaking the source code into tokens. It reads the raw text and converts it into a sequence of meaningful symbols such as keywords, operators, and identifiers, while skipping whitespace and comments. This first phase uses patterns (like regular expressions) to recognize what each group of characters represents and often places identifiers into a symbol table along with their attributes. The token stream it produces is what the parser uses to check the program's grammar and structure. It’s not about turning tokens into machine code or optimizing something yet—that happens in later stages. For example, a line like int count = 42; would be turned into tokens representing the keyword, the identifier, the assignment operator, the numeric literal, and the semicolon, which the rest of the compiler then analyzes.