Decoding linear codes via optimization and graph-based techniques

Low-density parity-check (LDPC) codes have made it possible to communicate at information rates very close to the Shannon capacity by combining sparsity with quasi- randomness, which enables the use of low-complexity iterative message-passing (IMP) decoders. So far, most systematic studies of IMP decoders have focused on evaluating the average performance of random ensembles of LDPC codes with infinite length. However, the statistical nature of IMP algorithms does not seem very suitable for rigorous analysis the decoding of individual finite-length codes. The need for finite-length studies are most critical in applications such as data storage, where the required decoding error rate is too low to be verifiable by simulation. As an alternative to IMP algorithms, linear programming (LP) decoding is based on relaxing the optimal decoding into a linear optimization. The geometric nature of this approach makes it more amenable to deterministic finite-length analysis than IMP decoding. On the other hand, LP decoding is computationally more complex than IMP decoding, due to both the large number of constraints in the relaxed problem, and the inefficiency of using general -purpose LP solvers. In this dissertation, we study several aspects of LP decoding, starting by some steps toward reducing its complexity. We introduce an adaptive implementation of LP decoding, where the relaxed problem is replaced by a sequence of subproblems of much smaller size, resulting in a complexity reduction by orders of magnitude. This is followed by a sparse implementation of an interior-point LP solver which exploits the structure of the decoding problem. We further propose a cutting- plane approach to improve the error-correcting capability of LP decoding. Along the way, several properties are proved for LP decoding and its proposed variations. We continue by investigating the application of an optimization-based approach to decoding linear codes in the presence of intersymbol interference (ISI). By relaxing the optimal detection problem into a linear program, we derive a new graphical representation for the ISI channel, which can be used for combined equalization and decoding by LP or IMP decoders. Finally, in a separate piece of work, we study the effect of nonlinearities on the multiuser capacity of optical fiber