What neurological aspect causes males to like females and vice versa?

I've always wondered how this worked. But if we lost our sexual organs then would we still be attracted to them if the chemicals stayed there??? Hmmm.