Scott was born in Salem, Oregon, but fortunately for Medford, his father, Tom, a State Police Officer, was transferred to Medford during Scott’s school days. Scott was a three-sport athlete excelling in football, basketball and track. An All-Conference quarterback at Medford, he was chosen to play in the North South Shrine football game where he won the outstanding back award for the game.

As an All-Conference basketball player in high school, Scott was recruited by several colleges to play basketball. He chose Oregon State and was a three-year starter for the Beavers. During his tenure at OSU, Oregon State won three Far West Classic Championships. In 1966, Oregon State won the Pacific Coast Championship. In national playoffs they defeated Houston but lost to Utah State in the final game of the Regionals.

As a 5th year student at OSU, Scott was asked by Dee Andros to play defensive back for the Beavers. Scott had so much success as a defensive back, he was drafted by the New York Giants football team. For six years he was the starting strong safety for the Giants. His first season with New York he was chosen to the NFL All Rookie team.