J. B. Nicholson, who would become the boys' stepfather, reverted to her maiden name and appended her new married name. The brothers were raised in "an iconoclastic, intellectual household" where their family entertained such guests as Theodore Roosevelt and Rudyard Kipling.

Wheeler-Nicholson spent his boyhood both in Portland and on a horse ranch in Washington state.