In 1390 the first real transformation of the castrum occurred into a fortified castle with a rectangular plan, with a tower at each corner, and the ancient tower in the centre.
In 1558 the estate and castle were bought by the Cesi family who transformed the fortress into a stately baronial mansion, the summer residence of the family. The rooms were frescoed by Zuccari (the same as Villa d’Este in Tivoli) and windows were opened along the defence walls.
Here, in 1603 was established the first headquarters of the Academy of the Lincei, founded by Federico Cesi and Galileo Galilei was hosted right here. In 1656, the plague that depopulated the country, the castle was ceded to the Borghese family and so began a decline so much so that at one point it was used as a granary. The restoration began in the postwar period by private interests and now the castle, while still being a private residence but it is made available for public events of particular importance.