The last decade has witnessed the emergence of magneto-active polymers (MAPs) as one of the most advanced multi-functional soft composites. Depending on the magnetisation mechanisms and responsive behaviour, MAPs are mainly classified as hard magnetic MAPs and soft magnetic MAPs. Polymeric materials are widely treated as fully incompressible solids that require special numerical treatment to solve the associated boundary value problem. Furthermore, both soft and hard magnetic particles-filled soft polymers are inherently viscoelastic. In this paper, we propose a unified simulation framework for magneto-mechanically coupled problems that can model hard and soft MAPs made of compressible and fully incompressible polymers, including the effect...