The protesters say the president must have known about a corruption scandal in the state oil firm, Petrobras.
The political opposition say much of the alleged bribery took place when she was head of the company.
But Ms Rousseff has been exonerated in an investigation by the attorney general and denies involvement.
Most of the politicians accused of taking bribes in a kickback scheme come from the governing coalition.
After the protests, the government promised a series of measures to combat corruption and impunity.
Justice Minister Jose Eduardo Cardozo said the government saw the rallies as an “expression of democracy”.
Protests have taken place across 22 Brazilian states and the federal capital, Brasilia.
The largest demonstration went ahead in Sao Paulo, a major opposition stronghold.
The estimates of how many people attended the march varied widely.
Brazilian data analysts Datafolha say almost 200,000 people marched on Avenida Paulista on Sunday evening.
But police estimated the number of participants at one million, based on aerial photographs of the area.
Many of the protesters waved Brazilian flags and wore the yellow shirts of the national football team.
They shouted slogans against corruption and the Workers’ Party government.
Opposition parties have backed Sunday’s protests but have not openly called for impeachment of the president, says the BBC’s Gary Duffy in Sao Paulo.
Senator Aecio Neves, who was defeated by a narrow margin in October’s presidential vote, issued a statement praising the protests.
He said Brazilians “went to the streets to reunite with their virtues, their values and also with their dreams”.
In Brasilia a crowd of 40,000 demonstrated outside the Congress building.
In Rio de Janeiro, where Ms Rousseff won 55% of the vote in the October presidential election, police said around 25,000 people had joined a protest there.
“There’s no point in complaining only on social media, we have to be here and show that we are really fed up,” businesswoman Daniela Mello told AP news agency in Rio.
Friday saw supporters of President Rousseff out in force, with tens of thousands taking to the streets.
Her supporters say calls for an impeachment, less than five months after she was elected to a second four-year term, amount to a coup attempt.
The Workers’ Party has been in power since President Luiz Inacio Lula da Silva was sworn in for his first term in January 2003.
Earlier this month, the Supreme Court approved the investigation of 54 people for their alleged involvement in the kickback scheme.
The list was prepared by Attorney General Rodrigo Janot who alleged that private companies paid corrupt officials in order to get lucrative Petrobras contracts.
According to the investigation, high-profile politicians also took a share of the money siphoned off from the oil company.
Mr Junot’s list includes Senate President Renan Calheiros, President of the Chamber of Deputies Eduardo Cunha, former Energy Minister Edison Lobao and former President Fernando Collor de Mello.
All deny corruption allegations.