The castle of Planes is located in the highest part of the town, at 472m. altitude. The town’s houses are so close to the castle that they are almost part of it. It has a polygonal enclosure with a perimeter of 220 meters and an interior surface of approximately 2900m². The date of construction is situated in the Almohad period, between the 12th-13th centuries, although it is not known with total certainty if there was a continuous settlement inside or it was only used as a refuge in case of siege.

The truth is that the castle of Planes was one of those that presented a more complex defensive system, with a rampart and an elbow entrance, located in the southwest part. The wall directed towards the north is practically rebuilt with a masonry wall belonging to reforms of the Christian era, due to some accidental detachment. It is in this part where the largest tower of the eight currently existing is located and possibly the only one that was built in the entire north facing. After the Christian conquest the castle was the possession of the different lords of the Barony. In the 20th century it was still privately owned, until in 1992 the City Council acquired it definitively.