Hospital sources told reporters that 34 government soldiers had been killed and 150 wounded in fierce fighting.
The coastal city was seized by the militant group in February 2015.
Forces aligned to Libya’s government of national accord launched an operation in May to re-take Sirte.
Overnight, Libyan and US warplanes carried out air strikes and mortar bombardment of districts one and three in the north of the Mediterranean city.
Tanks and rocket-propelled grenades were used in the battle.
The brigades loyal to the government said IS militants had deployed five car bombs on Sunday to try to stop the advance.