Parallelization of meta-heuristics for NP-complete problems

In this work, we will look at a class of very hard practical problems which can, currently, only be solved with algorithms running in exponential time on deterministic Turing machine. Further, we will discuss the theory of NP-completeness, which allows us to classify problems based on their complexity. We proceed by looking at four NP-equivalent problems: maximum independent set problem, maximum clique problem, minimum vertex cover problem and traveling salesman problem. We will continue with a class of meta-heuristic algorithms, which provide suboptimal solutions -- however, their running time is usually substantially smaller than exponential. We will discuss six of such meta-heuristic algorithms: hill climbing, simulated annealing, scatter search, cross-entropy method, ant colony optimization and evolutionary algorithm. To solve problems faster, we can also deploy several parallel computers and implement our solution using all the deployed computers simultaneously. Therefore, we will look at two technologies allowing us to write parallel programs OpenMPI and Parallel Virtual Machine (PVM). Based on serial algorithms and using OpenMPI and PVM, we will build three parallel meta-heuristic algorithms: simulated annealing, scatter search and cross-entropy method. Subsequently, we present test results of the mentioned serial and parallel algorithms, implemented in programming language C++ on four systems: two different Dell servers, heterogeneous system of computers and supercomputer LSC Adria. For the analysis, we use standard DIMACS instances. Results show that serial simulated annealing algorithm outperforms currently best known implementation and it is also better than described cross-entropy method and scatter search. When we compare serial and parallel algorithms, we observe that parallel algorithms usually do improve solving time and improve solution quality. However, the speedup at the same quality is substantially worse than the theoretically expected value. We will also compare parallel environments OpenMPI and PVM and see that simulated annealing and scatter search algorithms using PVM perform considerably better than using OpenMPI. We conclude with directions for possible future work