Bayfield was founded in 1832 by Carel Lodewijk, Baron van Tuyll van Serooskerken (1784-1835), a Dutch nobleman who had purchased 388 acres (157 ha) in the area, part of a much larger holding in the Huron Tract that he purchased from the Canada Company Over the next 10 years, a plan was laid out for the community and two mills (grist and lumber) were built. It was named for noted Great Lakes surveyor and British naval officer Admiral Henry Wolsey Bayfield, who had personally selected the site for the community.
The mouth of the Bayfield River is a natural sheltered harbour and by 1869, the settlement was a port of entry and vessels touched here on their route from Detroit to Goderich. Stages travelled daily to Seaforth. In 1869, when the population was only 125, the settlement had a wagon and plough making shop, two blacksmiths, a distiller, a brickyard and two tanneries. A large, permanent harbour was built during the 1870s. By 1876, Bayfield was incorporated as a village, with a population of 846. In 1982, Bayfield was declared a heritage conservation district.