help!

[isolated-igloo-.livejournal.com]

I was always taught to go "with the grain" of the fabric but i am making pj pants and it makes so much more sense fabric wise to go the other direction. I would have enough left to make another pair, if i went with the grain i would barelly have enough to make one. Does it make that big of a difference which direction you go? My fabric is flannel. Please help! Thanks.