Joseph died in 1845 the immediate heir was his younger brother Edward, who had assumed the surname of Doughty as a condition of a legacy. Edward's only son died in childhood, so James Tichborne became next in line to the baronetcy, and after him, Roger. As the family's fortunes had been greatly augmented by the Doughty bequest, this was a considerable material prospect.

After Roger's birth, James and Henriette had three more children: two daughters who died in infancy and a second son, Alfred, born in 1839. The marriage was unhappy,