Differential privacy has emerged as a de facto standard of privacy notion. It is widely adopted in various domains, including data publishing, data mining, and interactive database queries. However, applying differential privacy on complex data still remains challenging due to the huge change of sensitivity. In this dissertation, we introduce three major topics about publishing information with high-dimensional and graph data under differential privacy. The first topic discusses the possibility of publishing column counts from high-dimensional data under differential privacy, with a proposed technique called sensitivity control. The idea is to limit the contribution of data records such that sensitivity can be limited. We solve the challeng...