Joseph Phillips' mother died when he was young. After this, he became close to the mother of Duane Evans, his cousin. Phillips is from Colorado. He and his wife Nicole have three sons, Connor, Ellis, and Samuel. In 2006, Phillips published an autobiography, He Talk Like A White Boy, which includes a foreword by Tavis Smiley. Phillips is a lover of cooking and Western films.