The move came after Austria hosted a meeting with Balkan states on the migrant issue, to which Greece was not invited.
Greece called the move an “unfriendly act”. It said its ambassador was recalled for “consultations”.
Ministers from EU and Balkan nations are meeting in Brussels to try to heal rifts over the migrant issue.
Austria, Serbia and Macedonia have taken their own steps to limit entry to migrants, angering Greece, which fears the controls will cause a bottleneck.
In a statement, the Greek foreign ministry said that the ambassador was being recalled “in order to safeguard the friendly relations between the states and the people of Greece and Austria”.
More than one million migrants arrived in the EU last year, many of them fleeing the civil war in Syria. So far this year, 100,000 migrants have entered the EU illegally – nearly all of them arriving in Greece. More than 400 have died on the sea crossing from Turkey.
The surge in migration coupled with the failure to agree an EU-wide response has led to warnings about the bloc’s survival.
On Wednesday, representatives from 10 Balkan states attended talks in Austria and agreed to co-ordinate action to limit the flow of migrants.
The meeting drew an angry response from Greece, with the foreign ministry warning that “responsibility for dealing with the migration and refugee crisis cannot burden one country”.
EU interior ministers meeting in Brussels on Thursday will hear plans drawn up by Austria and Balkan countries that seek to restrict the numbers entering their borders.
Measures include fingerprinting all entrants and turning back anyone without a passport or holding fake documents.
The countries have also pledged to accept only those they deem to be in need of protection – interpreted by some governments as meaning only Syrians and Iraqis.
Over the weekend, Macedonia barred entry to Afghans at its border with Greece. Angry protests erupted at the border crossings and Greece was forced to transport hundreds of Afghan back to Athens.
The EU has warned of a humanitarian crisis if the disagreements are not resolved.
European Council president Donald Tusk also warned that the failure to make progress towards resolving the crisis could increase the likelihood of the UK voting to leave the EU this year.
Greece has threatened to block all decisions at EU migration summits next month if member states do not agree to take in quotas of migrants.
Filippo Grandi, the head of UN refugee agency UNHCR, said, “Everybody has to take a share of this burden.”
Sebastian Kurz, the Austrian foreign minister, told German TV Europe had to “abandon this goal of allowing in unlimited numbers”.
Hungary announced on Wednesday that it would hold a referendum on whether to accept mandatory EU quotas for relocating migrants.
Prime Minister Viktor Orban said the quotas “could redraw Europe’s cultural and religious identity”.
Last year, Hungary’s right-wing government built a razor-wire fence on its borders with Serbia and Croatia to try to limit the flow of migrants.
EU leaders have announced they will hold a summit in early March with Turkey to attempt to seek fresh solutions to the crisis.