The 2 symmetrical wings constitute the 18th century pavilions. It is a remnant of the old fortress (10th century). The fortress consists of a keep, a huge 4-storey building. There is a parish church from 1230. In 1362, the "écorcheurs" seized the town; to make them flee, the town was destroyed. King Henry IV stayed in the castle for 10 days. During the 10 years war, Pesmes was ravaged. Louis XIV conquered Pesmes and spent the night of 15 February 1668 there. At the beginning of the 18th century, the castle was modified and is one of the most beautiful in the province. It is undoubtedly from this period that the staircases linking the stables date. These were sold in 1794 and destroyed.