It was also the fashion of the day. He may have gotten rid of it and worn "trendy" facial hair, or none at all, depending what fashion would have dictated.
Has anyone ever done one of those age-enhancement programs on him, to see what he would have looked like at 20, 30, 40,50 years of age?