The Freshwater Memorial Hall was built in 1899, as a Drill Hall for the use of the local Territorial Army. The building itself with its castellation and red bricks cost £900 to construct.

It was founded by public subscription, and Hallam Tennyson - the poet's son - laid the foundation stone, which can still be seen at the front of the building.

In her book on Freshwater history, Joy Lester records that the building was dedicated to the memory of photographer Julia Margaret Cameron, who started the fundraising appeal with a subscription of £100.

After the Second World War with the decline in the demand for the Territorial Army, and the centralisation of the Island Territorial Army unit at the Island's capital Newport, the drill hall was no longer required by the Territorial Army.

In 1959 the drill hall became available to purchase. Freshwater and Totland parish councils set up a joint drill hall committee from members of both parish councils, to investigate the possibility of acquiring the building for the use as a public amenity for the West Wight area.

In August 1960 the drill hall was purchased and the governorship was based on the relative population of the two parishes, two thirds Freshwater and one third Totland.

The Memorial Hall was opened to the public on 23 September 1961 and at that time it was resolved that the drill hall committee should be replaced by a properly constituted 'Memorial Hall Joint Committee' the membership of which would comprise of the parish councils of Freshwater and Totland.

From its construction at the end of the 18th Century as a territorial army drill hall to its use now as a popular venue for local events, the Memorial Hall continues to be a focal point for the community. The hall is now managed by a registered charity, the Memorial Hall Arts & Leisure Company. The directors of the Company are members of the parish councils of Freshwater and Totland.