In this paper, we present our joint efforts to design and develop parallel implementations of the GNU Scientific Library for a wide variety of parallel platforms. The multilevel software architecture proposed provides several interfaces: a sequential interface that hides the parallel nature of the library to sequential users, a parallel interface for parallel programmers, and a web services based interface to provide remote access to the routines of the library. The physical level of the architecture includes platforms ranging from distributed and shared-memory multiprocessors to hybrid systems and heterogeneous clusters. Several well-known operations arising in discrete mathematics and sparse linear algebra are used to illustrate the chall...