How might the southern colonies reliance on slave labor eventually cause problems?

The southern colonies reliance on slave labor eventually could have caused problems because it kept the economy from developing manufacturing jobs or other jobs for the middle class. With the reliance on free manual labor there was little incentive for the development of employment sectors that the slaves were already filling. Additionally slavery was morally wrong and when it was ended it led to instability in the southern economy because of this reliance on free labor.