Personally, I don't wear them...but, my friends who do, I've noticed are usually waring one of those white t's under it. I doubt it matters, but if I were to wear one, thats how I'd do it.
Just those crappy white t's...the ones you can see through. It doesn't matter, its an undershirt. Actually, MOST of the time I wear two shirts. Its just more comfy for me.