If These Walls Could Talk:
A History of the Smith-McDowell House

The 4-story mansion was built on a plantation south of Asheville approximately twenty years before the Civil War. Constructed on a hill’s summit, ringed by picturesque mountains, the manse was constructed by slave labor. During a time when most people lived in log cabins, the imposing structure was composed of rare brick. Slave labor was probably used to construct the impressive home. Today known as The Smith-McDowell House, it is the oldest surviving house in Asheville and the oldest brick house in Buncombe County, North Carolina.

Portrait of James McConnell SmithThe mansion sits upon a plot of land acquired by Colonel Daniel Smith (1757-1824) via a land grant for Revolutionary War soldiers. Among the earliest settlers to the region, the Colonel’s son, James, is said to have been the first white child born west of the mountains in North Carolina. It was this son, James McConnell Smith (1787-1856) who would wed Mary “Polly” Patton (1794-1853) and build the impressive brick home. In 1833, J.M. Smith built and operated a state-licensed toll bridge over the French Broad River on the Buncombe Turnpike. The bridge was part of the Drovers Road, linking Greenville, TN and western North Carolina farmers to markets in South Carolina and Georgia. His monopoly on the bridge gave Smith great wealth and he became one of North Carolina’s most influential and leading businessmen, becoming a judge and a mayor and owning a store, tavern, two plantations, a tannery, gold mine and the Buck Hotel. At one time, Smith owned more than 30,000 acres across Buncombe County.

When Smith died in 1856, his son, John Patton Smith (1823-1857), inherited the Smith-McDowell House. Upon John’s death a short year later, James’s business partner and son-in-law, William Wallace McDowell (1823-1893), and Sarah Lucinda Smith McDowell (1826-1905) purchased the house and 350 acres for $10,000. On the eve of the Civil War, McDowell organized the Buncombe Riflemen, the first group of Confederate volunteers from Western North Carolina. W.W. McDowell achieved the rank of Major. During the course of the War, the house was visited by Union troops.

Portrait of Mary “Polly” PattonWhen the Civil War ended, the McDowell family faced economic reversals. The Smith-McDowell House, along with 15 acres, was sold to Alexander Garrett (c. 1820-1895) in 1881. Garrett was an Irish immigrant who had made his fortune in St. Louis, Missouri. He came to Asheville in hope that the mountain air would cure his daughter-in-law’s tuberculosis. Garrett updated the home, connecting the summer kitchen to the main house. He also founded Victoria, the wealthy community surrounding the house, becoming its mayor and building the Victoria Inn. Garrett later sold the Smith-McDowell House to his son, Robert, for $1.00.

Robert sold the Smith-McDowell House to Dr. and Mrs. Charles Van Bergen in 1898. Van Bergen commissioned Frederick Law Olmsted, Jr. to create a landscape plan for the property while adding a new carriage house that, according to tradition, was designed by Richard Morris Hunt. (The formal garden area and the carriage house are pictured here circa 1913). Subsequent owners of the house were Henrietta Bates McKee (friend of the Vanderbilts and President Roosevelt), millionaire Brewster Chapman, and Herman Gudger (General Manager of DuPont during WWI and Vice-President of film-famed Goldwyn Corporation). Chapman hired Richard Sharp Smith, the supervising architecture for Biltmore Estate, to update the house in 1913.

Photo of the Carriage House and formal garden areaIn 1951, the Catholic Diocese purchased the house for a boy’s school dormitory. The then-dilapidated house and grounds were purchased by Asheville-Buncombe Technical Community College in 1974. That same year, the Western North Carolina Historical Association rescued the house from demolition by negotiating a lease to restore the house as a heritage center. Due to fund-raising efforts and extensive restorations, the Museum opened in 1981. Today, the restored Smith-McDowell House is a nonprofit museum and is included in the National Register of Historic Places. It is the finest surviving example of brick antebellum architecture in Western North Carolina.