The British number three, who reached the semi-finals last year, came into the draw as a lucky loser and beat Kiki Bertens in the first round.
But Vinci proved too strong and sealed a 6-4 6-1 victory over the 29-year-old.
Keothavong, ranked 83rd, needed a strong finish to the season to defend the ranking points she earned this time last year and stay in the top 100.
Vinci now faces a quarter-final against Venus Williams, who beat ninth seed Mona Barthel of Germany 7-6 (7-4) 6-4.