Syria’s head of antiquities was quoted as saying the temple was blown up on Sunday. The UK -based Syrian Observatory for Human Rights (SOHR) reported that it happened a month ago.
IS took control of Palmyra in May, sparking fears for the site.
It is considered one of the ancient world’s most important cultural centres.
The ancient city, which is a Unesco World Heritage site, is famed for its well-preserved Greco-Roman ruins, and the Baalshamin temple, built nearly 2,000 years ago, is one of the city’s best-known buildings.
The Islamic State group has destroyed several ancient sites in Iraq. The militants believe any shrines or statues implying the existence of another deity are sacrilege and idolatry, and should be destroyed.
IS “placed a large quantity of explosives in the temple of Baalshamin today [Sunday] and then blew it up causing much damage to the temple,” Syrian antiquities chief Maamoun Abdulkarim told AFP news agency.
“The cella (inner area of the temple) was destroyed and the columns around collapsed,” he said.
Emma Loosley, a professor at Exeter University who lived near the ancient city for three years, said the temple’s cella was “pretty much perfect”.
“I can’t think of another temple as beautifully preserved as the temple of Baalshamin, and what was special about Palmyra was that it was a unique culture,” she told the BBC.
“It had its own gods, its own form of art and architecture that you don’t get anywhere else.”
Unesco World Heritage Site, known as Pearl of the Desert.
Site contains monumental ruins of great city, once one of the most important cultural centres of the ancient world from the 1st and 2nd Centuries.
Its art and architecture combines Greco-Roman techniques with local traditions and Persian influences
More than 150,000 tourists visited Palmyra every year before Syrian conflict
Site boasts a number of monumental projects, over 1,000 columns, and a formidable necropolis of over 500 tombs
The Baalshamin temple is dedicated to the Phoenician god of storms and fertilising rains, and was almost completely intact.
The oldest parts of the temple are thought to have dated from the year 17AD.
Residents who had fled from Palmyra also said IS had planted explosives at the temple, although they had done it about one month ago, according to the Syrian Observatory for Human Rights.
Mr Abdulkarim said the 81-year-old had refused to tell IS where some treasures had been hidden, in an effort to save them.
The group has also published photos of what they said was the destruction of two Islamic shrines near Palmyra, which they described as “manifestations of polytheism”.
The modern city of Palmyra – known locally as Tadmur – is situated in a strategically important area on the road between the Syrian capital, Damascus, and the eastern city of Deir al-Zour.
IS ransacks the central library in the Iraqi city of Mosul, burning thousands of books.
: A video emerges showing the destruction of ancient artefacts at the central museum in Mosul.
IS uses explosives and bulldozers on Nimrud, one of Iraq’s greatest archaeological treasures. Shortly after, IS militants destroy ruins at Hatra.