What the pilgrims came to revere was the late Gothic statue of St. Anne. According to legend, in 1716, on his way through Saxony, Schlenkrich, a travelling merchant from Lobendava, saw a group of children pulling the statue of St. Anne on a rope, as if they were playing a game. Schlenkrich bought the statue from them and donated it to the Church of the Visitation of the Virgin Mary in Lobendava.
The statue of St. Anne was originally displayed on the column and later in the chapel on Jáchym hill. However, several times it miraculously moved to the top of Annaberg hill. A Baroque central chapel in the shape of a Greek cross was built here in 1775 – 1777, and extended in 1857. The main altar was donated by the owner of the estate, František Václav, Count of Salm – Reifferscheidt and his mother Karolina of Dietrichstein. In the 19th century the Stations of the Cross were added (1834), as well as a statue of Jesus Christ and the Apostles in the Garden of Gethsemane and the Chapel of the Holy Sepulchre.
On the feast day of St. Anne (26. July) the otherwise quiet village of Annaberg rang to the folk festivities held by the people of Lobendava and the nearby villages. During the summer, before the Second World War, thousands of pilgrims used to visit this place. Annaberg was also visited by Catholic Sorbs, who went on from here to the Porciunkule indulgence festival at the Loretta chapel in Rumburk.
Before the Second World War the fair at Annaberg was known as “Madlmorcht” (girls' market). This is perhaps because, like with so many other fairs, it was a great opportunity for young people to meet one another and choose a spouse.