King Gustav III of Sweden laid the foundation stone of the Crownwork Ehrensvärd in 1775. The construction of the wings started in 1776. By 1777, the northern end and the western front of the granite cellar floor of the building had been bricked.
Other parts of the building could not be bricked, as some navy boathouses still remained. The following year, however, the construction appropriations were withdrawn and it was 1785 before any work could begin. This time, the construction was executed swiftly, and the building was ready to be used by the navy docks the following year. At the beginning of the Russian period, there was a Lutheran church in the building.
The building was badly damaged in bombardments during the Crimean War. After the war, the topmost floor was torn down and the first floor remained as a ruined attic space. The northern end of the ground floor was used as a school. The first proper floor of the lavatory wing was renovated as a Russian Orthodox prayer house.
During the 1918–1919 war prison camp, the building served as prison barracks no. 3. Little by little the Swedish ’great storage’ was established as a conference and banquet facility. The renovation and conversion of the building for conference and banquet use started in 1992, when the cabinets at the southern end of the ground floor were renovated, the windows repaired and the building connected to the district heating network.
In 1994–1996, new toilet facilities were built in the old lavatory wing and a new kitchen was built for the banquet hall. As part of the renovation work, two studios and one apartment were created in the rooms at the northern end of the banquet hall floor.