Colonel Joseph Taylor was born in South Carolina around 1783, the son of Samuel Taylor, a Revolutionary War veteran, and Eleanor Cannon. He and his wife, Nancy Sloan, lived in Anderson, South Carolina, and had several children, including Ellen and Benjamin. Ellen Cannon Taylor married William Poe, a banker from Augusta, Georgia, around 1835, and they had six children: William (b. ca. 1836), Ellen Matilda (b. ca. 1840), Azalea (b. September 1842), Nancy (b. 1844 or 1845), Nelson, and Taylor. The Poe family moved to Baltimore around 1842, and were living in Montgomery, Alabama, in 1850. William Poe had two sons, Robert and Thomas, from a previous marriage.
From the guide to the William and Ellen C. Poe letters, Poe, William and Ellen C. letters, [1836]-1852, (William L. Clements Library, University of Michigan)
