Shah Jahan tended to have buildings made from white marble. When Shah Jahan suddenly fell ill, a bloody war of succession broke out between his sons, where Aurangazeb was victorious. Aurangazeb would go on to place his father under house arrest in Agra fort.

The fort was under the Jat rulers of Bharatpur