Duff, James, active 1751-1796, Earl Fife