Mr Trump called the conversation “the worst by far” of his calls with world leaders that day, and cut it short.
Mr Trump later tweeted that he would “study this dumb deal“.
Struck with the Obama administration, it would see up to 1,250 asylum seekers to Autralia resettled in the USA.
Australia has controversially refused to accept the refugees – most of whom are men from Iran, Afghanistan and Iraq – and instead holds them in offshore detention centres on the Pacific nations of Nauru and Papua New Guinea.
PM Turnbull had been seeking clarification on the future of the deal after Mr Trump last Friday signed an executive order temporarily barring the entry into the US of refugees and people from seven Muslim-majority countries.
The phone call between Mr Trump and Mr Turnbull took place on Saturday, and was one of four the US president had with world leaders, including Russia’s Vladimir Putin.
The Washington Post quotes senior US officials, briefed on the call, as saying that the conversation should have lasted an hour but was abruptly ended after 25 minutes by Mr Trump.
Mr Turnbull was seeking assurances from Mr Trump that the deal would be honoured.
The US president reportedly said accepting the refugees would be like the US accepting “the next Boston bombers”, who were from the Caucasus region of Russia.
The offical versionfrom the US was brief, but said both leaders had “emphasised the enduring strength and closeness of the US-Australia relationship”.
On Monday Mr Turnbull confirmed he had spoken to Mr Trump and thanked him for agreeing to uphold the deal.
US presidential spokesman Sean Spicer has since also said Mr Trump intends to uphold the deal.
But Mr Trump’s tweet on Wednesday – coming after the Washington Post story – has thrown fresh doubt on the arrangement.
Mr Turnbull later said he was disappointed that details of the call – which he described as “very frank and forthright” – had been made public.
He told a Sydney radio station that “the report that the president hung up is not correct”.
stralia announced in November 2016 that the US had agreed to a one-off deal to resettle refugees currently being held on Nauru and Manus Island, in Papua New Guinea (PNG).
The UN refugee agency, UNHCR, would oversee the deal and the “most vulnerable” would be prioritised, Mr Turunbull said.
No numbers were given and Australian Immigration Department Secretary Mike Pezzullo later told a Senate inquiry that, while those who were eligible could express an interest, it was up to the US to decide how many people it wanted to take.
A total of 1,254 people were being held in the two camps, 871 on Manus Island and 383 in Nauru, as of 30 November 2016, accroding to Australia goverment statistics.
Mr Trump’s tweet incorrectly labelled refugees as illegal, and recast the number who might be resettled as “thousands”.
Australia refuses to accept refugees who arrive by boat, under a tough deterrent policy. It has already struck resettlement deals with Cambodia and PNG, but only a handful of refugees have been resettled. Critics say the two nations are completely ill-equipped to resettle refugees.
So the US deal was a boon to the Australian government from a close political and military ally.
Australia has faced fierce international criticism for its offshore detention policy and wants to close the Manus Island camp. Conditions in the offshore camps have been roundly condemned by rights groups, who say the policy is punitive and inflicts harm on refugees.
All of the occupants of Manus Island are male. By far the largest number are from Iran, followed by Afghanistan and Iraq. There are also sizeable contingents from Bangladesh, Pakistan and Myanmar.
The Nauru camp holds men, women and children. Again the largest number come from Iran, followed by Sri Lanka and those who are stateless.
Some of those being held have spent several years in the camps awaiting a decision on their fate.