Its pretty simple, sex before marriage is fornication and fornication is forbidden by the scripture.
So are a lot of other things and I don't see Christians raising a fuss about those. Remarriage is adultery. Do you call anyone who has been married 2 or 3 times guilty of adultery?