Frye was born in Birmingham, Alabama. He first attended the University of Illinois, where he received a BA in history and philosophy in 1939. He received his MA from Harvard University in 1940 and his PhD from Harvard in 1946, in Asiatic history.

Frye served with the Office of Strategic Services during World War II.