D’Arcy Boulton, Jr. (1785–1846) and his wife, Sarah Anne (née Robinson), (1789–1863) built the Grange House in 1817 and lived there with their eight children John Andrew (1810?-1830), William Henry (1812-1874), D’Arcy Edward (1814-1885), Beverley Robert (?-1840), Mary Sayer (1816-1837), Emma Robinson (1818-1890), Sarah Ann (1824?-1906), and John (1829-1882).
Both sides of the family were members of the powerful elite in Upper Canada - Sarah Anne’s brother was Chief Justice John Beverley Robinson, the leader of the Family Compact, and Boultons and their friends enjoyed influence, favourable business dealings, and Crown appointments.
D’Arcy had trained as a lawyer but worked as a merchant with his brother-in-law, Peter Robinson. Once settled in The Grange he retired and became a landowner. He also held minor government positions.
Their eldest surviving son, William (1812–1874), continued to live in the house. He also trained and practised as a lawyer. William was also an alderman, was appointed mayor of Toronto four times and was a member of parliament.
In 1846, William married Harriet Dixon (1825–1909), a Bostonian from a wealthy family. They had no children. After William’s death, Harriet married scholar and political writer Goldwin Smith (1823–1910). Harriet left the Grange House to the Art Museum of Toronto (now the Art Gallery of Ontario) in her will.