He built himself a more modest residence which was later replaced with an opulent mansion by his Victorian descendant, Walter Montagu Douglas Scott, 5th Duke of Buccleuch: see Montagu House, Whitehall.
Han byggde sig en mer blygsam bostad som senare ersattes med en överdådig herrgård av hans viktorianske ättling, Walter Montagu-Douglas-Scott, 5:e hertig av Buccleuch: se Montagu House, Whitehall.