William was the eldest of nine children, seven sons and two daughters, of Thomas and his wife Joan Halke.

Notable family connections include Heneage Finch, 1st Earl of Nottingham, who married William's niece Elizabeth Harvey, and the diplomat Sir Daniel Harvey. His great nephew was the naval hero Eliab Harvey, captain of HMS ''Temeraire''.