Why Is It That When Someone Smiles at Me I Don't Smile Back?!

Honestly, even if it's this guy who's way older than you who looks to have been staring at you for a while, all I do when I catch someone smiling at me, I just sort of walk by. Like adults, they smile at you all the time(well, some of them do). Kids just make weird faces at you. And babies don't know what to think. But when someone smiles at me I just get all freaked out. Sometimes I can manage more than a grimace back, but most of the time I just stare, deer-in-the-headlights like. Like this guy at Whole Foods, (Dan, we'll call him, though I know his name), he's really nice and he's so awesome I'm going to put him into one of my stories as the awesome uncle, but I turn around after we check out and he's smiling at me, and it's really kind of freaky. I was like "creeper" though I could have smiled back and made the entire situation end. But I didn't. God, I always realize these things too late.