Acknowledgments

First, I would like to express my deep gratitude to my advisor, Naftali Tishby. Tali has been, and still is, a constant source of inspiration for me. His wisdom, patience and inexhaustible knowledge have guided me safely through the last few years. This thesis is based on numerous fruitful conversations we have had, which in particular resulted in eleven papers we collaborated on. Moreover, many sections of this thesis have not been published elsewhere and are a direct result of our joint long discussions. His contribution to this thesis, and to me, cannot be overemphasized. Nir Friedman has been like a second advisor to me. I have had many enjoyable and fruitful discussions with him which have led to three published papers, and hopefully many more to come. He showed me how to combine Bayesian Networks with the Information Bottleneck and made the first important step toward establishing the multivariate IB framework, which constitutes Part III of this thesis. Yoram Singer guided me through the weird world of Information Retrieval. Moreover, whenever I needed advice, or some help, he was there to provide it. Yair Weiss made the first rigorous step to show the relationship between Maximum Likelihood and the Information Bottleneck. Amazingly, he even succeeded in explaining it to me, and perhaps even more surprisingly, convinced me that it is true. Our collaboration was both pleasant and educational, and resulted in a joint paper which is presented in Appendix A. Bill Bialek hosted me at NEC research institute at Princeton, for ten exciting days in May 1998, which both Tali Tishby and he used to introduce me to the Information Bottleneck idea. Thanks to him, this was the turning point of my studies. Until then, I was under the impression that I could earn my Ph.D. by writing sitcom scripts. I’m grateful to Bill for this visit which eventually determined the course of my thesis. I would like to thank the Ph.D. students of the Machine Learning group at the Hebrew University for thei