An approach towards a scalable lock-free asynchronous event loop with encapsulated blocking capabilities

High performance server systems tend to increasingly rely on asynchronous and non-blocking event loop architectures. The primary reason for this is that non-blocking programming structures are often more scalable with respect to the number of concurrent open connections that a single machine is able to handle. This because a single threaded system will usually have a significant lower memory consumption since no programming stack is allocated for each open connection. However, the use of a pure asynchronous architecture has a very negative impact on the programming complexity. Applications that, for example, use a traditional callback structure automatically result into very complex state machine implementations inside both the application logic and IO interaction logic layers. The reason for this is that the developer is burdened by the responsibility to control all state transitions and maintaining the active state between IO operations since no information can be stored on the programming stack. This also results into an increased software coupling between components, and even causes that not every protocol implementation can be parsed in an asynchronous system. In this research, two different hybrid frameworks were created that try to combine the scalability benefits of an asynchronous architecture with the ease of development of synchronous programming. In addition, both approaches remove a major part of the concurrency complexity involved in normal multithreaded environments by using non-preemptive scheduling instead. The use of coroutines in combination with future objects allows the developer to write normal synchronous code that is still able to parallelize certain parts of the communication. The first approach uses the Scala Continuations library that provides language support for suspending method calls. This resulted in a proof of concept that allows the developer to write synchronous code while still being able to control the asynchronous behavior. However, this approach is still incompatible with blocking stream libraries. Instead, the second approach solves this incompatibility issue by using low level context switching which allows to totally encapsulate the continuation-behavior of the system. This resulted into a framework that allows developers to use all existing blocking API's for IO interaction while the stack allocation overhead is still minimized.