Fermor, George, 2nd Earl of Pomfret, Lord of the Bedchamber