The fortress was built by Galeazzo II in 1360 and expanded in 1372. Like other Visconti Castles, the Voghera Castle was located near the city walls, providing defense against external attacks and, most importantly, an escape route in case of internal uprisings.

Used as a prison in the 19th century, the Castle underwent significant alterations, but the original layout remains clear, characterized by a rectangular plan with corner towers and a gate tower, initially equipped with a drawbridge that allowed crossing a moat dug around the perimeter of the walls.