High performance in software development

What are the ingredients of high-performing software? Software development, especially for large high-performance systems, is one the most complex tasks mankind has ever tried. Technological change leads to huge opportunities but challenges our old ways of working. Processing large data sets, possibly in real time or with other tight computational constraints, requires an efficient solution architecture. Efficiency requirements span from the distributed storage and large-scale organization of computation and data onto the lowest level of processor and data bus behavior. Integrating performance behavior over these levels is especially important when the computation is resource-bounded, as it is in numerics: physical simulation, machine learning, estimation of statistical models, etc. For example, memory locality and utilization of vector processing are essential for harnessing the computing power of modern processor architectures due to the deep memory hierarchies of modern general-purpose computers. As a rule of thumb, a highly modularized software with distributed state leads to poor performance. At the same time, structure of the system naturally aligns to the communication structure of the organization designing it (Conway’s law). To achieve good design and high performance of complex systems require diverse collaboration with an empirical mindset: rapid validation and ability to change direction when our assumptions become disqualified. This is very different from traditional process-oriented engineering and requires different organizational structure and behavior. In this talk we share our experiences from creating critical, large, fault-tolerant and high-performing systems during the last 20 years. We’ve been working with global companies such as HBO, Panasonic, Samsung and Supercell, that are all dependent on high-performing software. We show what has worked, what not and why. We’ll cover the topics of productivity, efficiency and quality what are the issues that have the biggest impact on these in software development. We’ll discuss about the common assumptions, beliefs and misconceptions that we see in the field. Other topics: Agile development How to make things easier in software development How to avoid doing unnecessary work How to deal with distributed teams Code quality and performance Maintenance and further development of existing systems. Data analysis at scale About the speakers Juha-Matti is a Systems architect with broad experience in different ICT technologies and fields of application, with focus on open source technologies, security, and embedded solutions. He also does a lot of training, coaching, concepting and review work. His specialty is in understanding things, and making things understandable to others. Petri is a product development generalist with 25 years of experience in a variety of industries globally. He has a broad experience in various product development positions ranging from hands-on engineering to management, coaching, change management, customer and vendor management to sales. Petri has been working with smaller startups as well as large 50 000+ employee global companies with multi-site development in Europe, Middle East, Asia and USA. Petri was one of creators of Lean and agile movement in Finland. He is the co-creator of the Agile conference in Finland (ScanAgile) and has been speaking at different companies and international conferences on agile product development. Petri is a pioneer of Lean and agile product development in Finland to practicing them from year 2001. He is an expert in scaling Agile in large companies. Jouni is Data Scientist, a seasoned specialist in statistical modeling and machine learning, with a solid academic background and experience in wide variety of business applications and analytical software development. Jouni is a doctor of philosophy from University of Essen, with major of theoretical physics and statistical physics.