The church is like many of the other village churches in Djursland from the 13th century. It is a Late Romanesque brick building with a newer porch in the south and tower in the west.
On the north side of the choir, there is preserved a sloping pedestal and above it a series of semi-circular search poles. Here you can also see the old woman's door and the plug after an original round-arched window.
The church was extensively renovated in 1773, where it included got cornice and battened roofs. This is probably around the time that the tower and porch were added.
Skarresø Kirke is placed beside a lake. The church is a romanesque building with 2 wings. The baroque pulpit with the four evangelists is a beautiful woodcarver work.