We prove several new results about the topology of fibers of Gelfand--Zeitlin systems on unitary and orthogonal coadjoint orbits, at the same time finding a unifying framework recovering and shedding light on essentially all known results. We find completely explicit descriptions of the diffeomorphism type of the fiber in many instances a direct factor decomposition of the fiber, and a torus factor corresponding to the action given by the Thimm trick. The new description also gives us a weak local normal form for a coadjoint orbit, which we use to define a topological toric degeneration, new in the orthogonal case. We also compute the first three homotopy groups (new in the orthogonal case) and cohomology rings of a fiber (new in both cas...