Ignace Murwanashyaka and Straton Musoni were accused of ordering militias to commit mass murder and rape between 2008 and 2009.
The trial took place under a law which allows the prosecution of foreigners for crimes committed outside Germany.
It was hailed as a breakthrough in bringing Rwandan rebels to justice.
Murwanashyaka, who is the leader of the Democratic Forces for the Liberation of Rwanda (FDLR), was given a 13-year sentence, while Musoni, his deputy, was sentenced to eight years.
Musoni was released immediately because of the time he has already spent in custody.
The presence of the FDLR in eastern DR Congo was one of the key factors behind years of conflict which have led to an estimated five million deaths in the region.
The two men, who were arrested in 2009, had both been living in Germany since the early 1990s and from there coordinated attacks on civilians in the east of the DR Congo.
They faced accusations that they ordered the killing of people because they were not cooperating with the FDLR.
At the start of the trial four years ago, German prosecutor Christian Ritscher said “the two defendants had knowledge of everything, at least most of it”.
The FDLR was established by ethnic Hutus accused of taking part in the 1994 Rwandan genocide, who later set up their base in DR Congo.
It became one of the most powerful rebel forces operating in the east of the country but has rarely infiltrated Rwanda.
Rwanda, now led by ethnic Tutsis, sees the FDLR as a threat to its stability and, along with the UN, has repeatedly demanded its disarmament.
It has twice invaded DR Congo, saying it is trying to prevent attacks by the FDLR.
The UN had been planning to launch an offensive with DR Congo’s army against the rebels at the beginning of this year, but pulled out because of a row over the human rights record of two Congolese generals who were involved in the mission.
April-June 1994: An estimated 800,000, mostly Tutsis, killed in Rwanda genocide
June 1994: Hutus – including some responsible for the genocide – flee into Zaire (now DR Congo) after rebels take power
1997: Rwanda crosses border in pursuit of Hutu rebels
2000: FDLR formed from previous Hutu militia
2009: Rwandan army goes into DR Congo to challenge FDLR
2009: FDLR accused of attacking civilians
2015: DR Congo army begins offensive against FDLR