Is it okay to wear sweatpants in public?

I love sweatpants but I've always wondered if it's okay to wear them to places like the mall, out with friends, pretty nice restaurants. And I'm talking about loose, baggy ones.