Lochte, Gunnar Bentz, Jack Conger and Jimmy Feigen were stopped by people posing as armed police, the US Olympic Committee (USOC) said in a statement.
The robbers demanded money and other personal belongings.
Initially the International Olympic Committee (IOC) denied reports of the robbery, before the US committee confirmed it had taken place.
“All four athletes are safe and co-operating with authorities,” the USOC’s statement said.
The group had been attending a party at the French Olympic team’s hospitality house when their taxi was stopped on the way back to the athletes’ village.
News of the incident first emerged from Lochte’s mother, Ileana, who told USA Today her son was unharmed but shaken by the theft.
“I think they’re all shaken up. There were a few of them,” she said.
Earlier, IOC spokesman Mark Adams said reports of the robbery were “absolutely not true”.
Lochte swam in two events at the Rio Games, winning gold in the 4x200m freestyle relay.
In the build-up to the games, three members of the Spanish Olympics sailing team and an Australian paralympian were robbed at gunpoint.
The Brazilian authorities have deployed more than 80,000 police and soldiers to patrol Rio during the games.