Improved Parallel Scanner for the Concurrent Execution of Lexical Analysis Tasks on Multi-Core Systems

Purpose: The processing power of machines will continue to accelerate massively. Modern eras of computing are driven by elevated parallel processing by the revolution of multi-core processors. This continuing trend toward parallel architectural paradigms facilitates parallel processing on a single machine and necessitates parallel programming in order to utilize the machine's enormous processing power. As a consequence, scanner generator applications will eventually need to be parallelized in order to fully leverage the throughput benefits of multi-core processors. This article discusses the way of processing the tasks in parallel during the scanning stage of lexical analysis. This is done by recognizing tokens in different lines of the source program in parallel along with auto detection of keyword in a character stream. Tasks are allocated line-by-line to the multiple instances of the lexical analyzer program. Then, each of the instances is run in parallel to detect tokens on different cores that are not yet engaged. Design/Methodology/Approach: Developing a theoretical and experimental approach for parallelizing the lexical scanning process on a multi-core system. Findings/Result: Based on the developed model, the theoretical and practical results indicate that the suggested methodology outperforms the sequential strategy in terms of tokenization consistently. It significantly decreases the amount of time spent on lexical analysis during the compilation process. It is clearly observed that the speedup should increase at or close to the same rate as the number of cores and keywords in the source program increases. This enhancement would improve the overall compilation time even more. Originality/Value: A hybrid model is developed for the concurrent execution of a lexical analyzer on multi-core systems using a dynamic task allocation algorithm and an auto-keyword detection method. Paper Type: Experimental Research