We live in a very mollycoddled society where the slightest bit of discomfort is seen as wrong, but that discomfort is there for a reason. It's supposed to trigger some form of action, some form of change, a realization of a truth - something, and I think the self-help world has you believing that you should be happy all the time.
Plenty of casual daters will throw you off with maddening phrases like 'I'm just enjoying having fun with you.' This doesn't make them a bad person, but it's your call now how to respond. Just don't assume 'having fun' or any such cliche means they're going to suddenly decide they want a relationship next week.