His elder sister was the children's writer Frances Freeling Broderip. After attending University College School and Louth Grammar School, he entered Pembroke College, Oxford, in 1853. There he studied for the Church and passed all the examinations for the degree of BA, but did not graduate.

At Oxford, he wrote his Farewell to the Swallows