Sir James Stephen (1789-1859), colonial under-secretary, was the son of James Stephen (1758-1832), Master in Chancery. His brother, Sir George Stephen (1794-1879) was a barrister and reformer. Sir James's marriage to Jane Catherine Venn (d.1875) produced five children, Herbert Venn (1822-1846), Frances Wilberforce (1824-1825), (Sir) James Fitzjames (1829-1894), (Sir) Leslie (1832-1904) and Caroline Emelia (1834-1909). Sir James Fitzjames Stephen, 1st Baronet and judge, married Mary Richenda, and had three children, Herbert (1857-1932), James Kenneth (1859-1892) and Harry Lushington (1860-1945).
From the guide to the Stephen Family: Letters and Papers, 19th century, (Cambridge University Library, Department of Manuscripts and University Archives)