He rebuilt much of the earlier house in 1879 for a total cost of £10,000. Scargill left Luton and the house and property was let to J Forder who renamed the estate Wardown. Frank Chapman's last surviving son (Jasper Chapman Scargill) died in Ireland 22 October 2012 at the age of 97 (93 years after the death of his father).

By 1903 the then owners (The Stewart Family, whose famous son, Sir Malcolm Stewart, founded the London Brick Company) decided