The relative arbitrage portfolio, formulated in Stochastic Portfolio Theory (SPT), outperforms a benchmark portfolio over a given time-horizon with probability one. This paper analyzes the market behavior and optimal investment strategies to attain relative arbitrage both in the $N$ investors and mean field regimes under some market conditions. An investor competes with a benchmark of market and peer investors, expecting to outperform the benchmark and minimizing the initial capital. With market price of risk processes depending on the market portfolio and investors, we develop a systematic way to solve multi-agent optimization problem within SPT's framework. The objective can be characterized by the smallest nonnegative continuous soluti...