The masses of quarks and leptons suggest a strong hierarchical structure. We argue that their patterns can be reproduced through the introduction of a new Abelian symmetry. The data suggest that this symmetry is anomalous. We suggest that the cancellation of its anomalies occurs through the Green-Schwarz mechanism. An important check of this idea is that it links the Weinberg angle to a mass ration of the elementary fermions. The Green-Schwarz mechanism occurs naturally in many superstring compactifications, and produces a small parameter, which we use to determine the quark mass hierarchy. We show that hierarchy and mixings among the chiral fermions is a consequence of the Green-Schwarz mechanism. We present several models where this idea ...