The militia was one of several groups angered by a US commando raid on Libyan soil on Saturday which seized senior al-Qaeda suspect Anas al-Liby.
Many saw the US raid as a breach of Libyan sovereignty, and there is growing pressure on the government to explain if it was involved.
However the exact motives for Mr Zeidan’s detention remain unclear. In a statement, the Revolutionaries Operations Room (LROR) said its actions had not been related to Mr Liby’s detention.
Foreign Minister Mohammed Abdelaziz told AFP news agency that Mr Zeidan had been freed, but added: “We have no details so far on the circumstances of his release.”
A government spokesman quoted by official Libyan news agency Lana earlier said the prime minister was free and on his way to his office.
Mr Zeidan had been taken in a pre-dawn raid on the Corinthia Hotel by more than 100 armed men.
The LROR said it was acting on the orders of the prosecutor general and in accordance with Libya’s criminal code.
However, state-run National Libyan TV quoted Justice Minister Salah al-Marghani as saying that the prosecutor general had issued no arrest warrant.
State TV broadcast images showing Mr Zeidan surrounded by what it said were armed men as he was led away. There were no reports of violence during his capture.
The prime minister was reportedly being held at the interior ministry anti-crime department in Tripoli, where an official said he was being treated well.
The LROR is one of a number of militias operating in Libya – it is nominally attached to government ministries but often acts independently and, correspondents say, often has the upper hand over police and army forces.
The government has been struggling to contain such militias who control many parts of the country, two years after the revolt that overthrew Col Muammar Gaddafi.
In a news conference shortly before the release was announced, the government condemned the “criminal act” of his detention and said it would not give in to “blackmail”.
Libyan politician Guma al-Gamaty told the BBC earlier said the arrest of Mr Liby had sparked anger on the Libyan streets.
“Accusations have been pointed at the prime minister that there is some sort of a collusion – that the prime minister knew in advance that Abu Anas al-Liby was going to be kidnapped,” he said.
On Monday, Libya demanded an explanation from the US ambassador over the arrest of Mr Liby, who is wanted in the US over the 1998 bombings of US embassies in Kenya and Tanzania.
The prime minister had earlier this week appeal for Western help in tackling rising militancy in Libya.
In an interview with the BBC on Monday, he said Libya was being used as a base to export weapons throughout the region.