Soon after Lee’s surrender at Appomattox, Lafayette and Mary moved near Gowensville in the upper part of the county. In 1889, they built a home on Tugaloo Road just off the present-day Cherokee Foothills Scenic Highway (SC 11).