As a man who has been pursued by women (including "hot" women to whom I was very attracted), and has said no to them, there is a simple way to avoid them feeling rejected. I tell them that I need to be "friends first" before I have sex with someone.
In my case, it has the added benefit of being true. And it might surprise some men, but most women appreciate that. As someone said, maybe humans are not so different.