Help!

[miawallace.livejournal.com]

So, I'm knitting a striped scarf in a K1, P1 ribbing stitch. It seems to be coming out great, except for one problem: the right hand edge, where I drop/cut the first color and begin the second color. While the left hand side looks nice and tight, the right looks bumpy and loose. What am I doing wrong? I hold the yarn tightly when beginning the first row of the second color, but no matter what it always comes out looking AWFUL. Anyone have any tips or techniques to keep stripes looking neat?

(Cross-posted to KnittingPretty. Sorry if it ends up on your friends list twice!)