Automatic thread extraction with decoupled software pipelining

Until recently, a steadily rising clock rate and other uniprocessor microarchitectural improvements could be re-lied upon to consistently deliver increasing performance for a wide range of applications. Current difficulties in main-taining this trend have lead microprocessor manufacturers to add value by incorporating multiple processors on a chip. Unfortunately, since decades of compiler research have not succeeded in delivering automatic threading for prevalent code properties, this approach demonstrates no improve-ment for a large class of existing codes. To find useful work for chip multiprocessors, we propose an automatic approach to thread extraction, called Decou-pled Software Pipelining (DSWP). DSWP exploits the fine-grained pipeline parallelism lurking in most applications to extract long-running, concurrently executing threads. Use of the non-speculative and truly decoupled threads pro-duced by DSWP can increase execution efficiency and pro-vide significant latency tolerance, mitigating design com-plexity by reducing inter-core communication and per-core resource requirements. Using our initial fully automatic compiler implementation and a validated processor model, we prove the concept by demonstrating significant gains for dual-core chip multiprocessor models running a variety of codes. We then explore simple opportunities missed by our initial compiler implementation which suggest a promising future for this approach.