Mr Chavez’s coffin set off in a procession to the Military Academy, where he will lie in state till Friday.
The government has announced seven days of mourning for the president, who died aged 58 after 14 years in the post.
Mr Chavez, a controversial figure and staunch critic of the US, was seriously ill with cancer for more than a year.
A self-proclaimed revolutionary, he inspired a left-wing revival across Latin America.
Latin American leaders have begun arriving in Caracas to pay their respects – among them President Cristina Fernandez de Kirchner of Argentina, Jose Mujica of Uruguay and Evo Morales of Bolivia.
Ecuador, Argentina, Brazil, Bolivia, Uruguay, Chile, Cuba and the Caribbean island of Dominica have declared periods of official mourning.
Another Chavez ally, Iranian President Mahmoud Ahmadinejad, also announced a day of mourning, describing him as a “martyr”.
Military units across the country have fired a 21-gun salute in his honour. They will fire another cannon shot each hour until he is buried, the armed forces said.
All schools and universities have been shut for the week.
‘To the pantheon’
On Wednesday morning, a priest prayed for eternal rest for Mr Chavez in a brief ceremony at the hospital where he died on Tuesday.
Officials then put the flag-draped coffin on top of a waiting hearse surrounded by crowds.
The procession began its slow journey through the streets of the city, led by officials including Vice-President Nicolas Maduro and accompanied by cheering red-clad supporters.
Some shouted “Chavez to the pantheon”, referring to the mausoleum he built for revolutionary leader Simon Bolivar.
“I’m here to say my final goodbye to my president. There will never be another Chavez. He is the greatest man that this fatherland gave us,” said Jose Gregorio Conde, 34, an education worker, quoted by AFP news agency.
Mr Chavez’s illness prevented him from taking the oath of office after he was re-elected for a fourth term in October.
Announcing the president’s death on Tuesday, Vice-President Nicolas Maduro called on the nation to close ranks after its leader’s demise.
“Let there be no weakness, no violence. Let there be no hate. In our hearts there should only be one sentiment: Love.”
Police and troops would be deployed nationwide “to guarantee the peace”, he added.
A statement from the military said it would remain loyal to the vice-president and to parliament, it added, urging people to remain calm.
Vice-President Maduro will assume the presidency until an election is called
Foreign Minister Elias Jaua told state television that Mr Maduro would also be the candidate of the governing United Socialist Party (PSUV).
Opposition leader Henrique Capriles, whom Mr Chavez defeated in October’s election, offered his condolences to Mr Chavez’s family, saying “we were adversaries, but never enemies”.
Mr Capriles is widely expected to be chosen to stand against the vice-president.
The BBC’s Irene Caselli, in Caracas, says Mr Maduro will probably win, but the question remains whether he will be able to lead Venezuela following the loss of its charismatic president.
The exact nature of Mr Chavez’s cancer was never officially disclosed, leading to continuing speculation about his health, and he had not been seen in public for
Last May, the former army paratrooper said he had recovered from an unspecified cancer, after undergoing surgery and chemotherapy in 2011 and a further operation in February 2012.
Despite this, he won another six-year presidential term in October 2012.
Mr Maduro has mentioned a plot against Venezuela, saying he had no doubt that Mr Chavez’s cancer, first diagnosed in 2011, had been induced by foul play by Venezuela’s enemies. The US promptly rejected the accusations as “absurd”.
Two US diplomats had been expelled from the country for spying on Venezuela’s military, Mr Maduro added.
Mr Chavez burst onto Venezuela’s national stage in 1992 to lead a failed military coup.
After two years in prison, he returned to politics and was swept to power in a 1998 election.
A self-proclaimed socialist and revolutionary, he won enduring support among the poor and repeated election victories by using Venezuela’s oil wealth to pursue socialist policies.
His government has implemented a number of “missions” or social programmes, including education and health services for all.
But his opponents accused him of mishandling the economy and taking the country towards dictatorship. Inequality has been reduced but growth overall has been lower than in some other Latin American economies.
Internationally, he was a staunch critic of US “imperialism” and accused Washington of backing a failed coup against him in 2002.
The US described the death as a “challenging time”, reaffirming what it described as its support for the Venezuelan people and its interest in developing a constructive relationship with Caracas.
Analysts say Mr Chavez’s death could alter the political balance in Latin America – dealing a blow to leftist states while favouring more centrist countries.
There could also be an economic impact given that Venezuela sells oil at below market prices to some neighbouring countries, especially in the Caribbean.