Then, I just serged it.
I decided the sleeves and bottom looked kind of funny to not match the neckline, so I serged all around them as well.
I think I like them better. They are looser and just seem to fit better. I'm not sure if the bare serging makes them look like pajamas though... oh well, I'm one of those people who don't mind wearing pajamas out in public!