Thomas Lyttelton, Baron Lyttelton, 1744-1779