Although Rome had held of the Barbarians for a long time, eventually groups such as the 'Goths' made their way past the borders. Cities were also sacked, such as in 410 when the Visigoth King Alaric sacked Rome.
Due to taxation and inflation, the gap between the poor and the rich grew very large, and many rich people fled to the countryside to avoid taxes. The economy also relied on slaves, so when expansion stopped, so did the supply of slaves.
Because the Roman Empire streched from the Atlantic ocean to the Euphrates river in the Middle east, the could not communicate quickly enough with their holdings. They also struggled to get enough troops to guard all the borders.
Rome's inconsistent leadership magnified all the governing problems they had previously. In just 75 years 20 Men took the throne. Bodies guards could murder the emperor at will.