The attackers entered the church in Saint-Etienne-du-Rouvray during Mass, taking the priest, Fr Jacques Hamel, 84, and four other people hostage.
Police later surrounded the church and French TV said shots were fired. Both hostage-takers are now dead.
The Amaq news agency, linked to so-called Islamic State, said “two IS soldiers” had carried out the attack.
President Francois Hollande said the men had claimed to be from IS.
Speaking in Saint-Etienne-du-Rouvray, he said the attackers had committed a “cowardly assassination” and France would fight IS “by all means”.
Pope Francis decried the “pain and horror of this absurd violence”.
French interior ministry spokesman, Pierre-Henri Brandet, said one of the hostages had been critically wounded.
He said the hostage-takers had been “neutralised” after coming out of the church. French prosecutors say one person has since been detained over the attack.
Police sources said it appeared the attackers had slit the priest’s throat with a knife.
The area was cordoned off while the church was searched for explosives, and police told people to stay away.
Mr Brandet said the investigation into the incident would be led by anti-terrorism prosecutors.
One of the men was known to the French intelligence services, French TV channel M6 has reported.
French Prime Minister Manuel Valls has expressed his horror at the “barbaric attack”.
“The whole of France and all Catholics are wounded. We will stand together,” he wrote on Twitter.
The Archbishop of Rouen, Dominique Lebrun, who was attending a Catholic gathering in Poland, said: “I cry out to God with all men of goodwill. I would invite non-believers to join in the cry.
“The Catholic Church cannot take weapons other than those of prayer and brotherhood among men.”
A woman who works at a local beauty parlour, Eulalie Garcia, said she had known the priest since her childhood.
“My family has lived here for 35 years and we have always known him,” she said.
“He was someone who was treasured by the community. He was very discreet and didn’t like to draw attention to himself.”
France has been on high alert since the Bastille Day attack in Nice earlier this month, when a man ploughed a lorry into celebrating crowds, killing more than 80 people.
The Nice attack was carried out by Tunisian Mohamed Lahouaiej-Bouhlel, who was not a known jihadist but so-called Islamic State said he had acted in response to its calls to target civilians in countries that have been attacking it.
The BBC’s Lucy Williamson in Paris says the French government has been under huge pressure to prevent further attacks.