The context of Gen 4 makes Gen 6:2 clear.
They that called upon the name of the Lord were calling themselves by the name of the Lord (a marginal note in many KJV bibles) Sons of God. Because the city on the east of Eden was other than them. The sons of men. Theologically.
Cain was thrust out from the presence of the Lord and built a city with a generation of God rejecting wicked people.
Adam and Seth on the West of Eden clung to a theology of a redeemer. And this is where Enoch's prophesy fragment from Jude sheds light. They believed in a coming judge that would execute judgment upon these in Cain's camp that spoke harsh words against him.
Gen 4 also mentions the daughters in Cain's camp, (daughters of men) like Naamah which meant, Fair, Beautiful, as well as a couple of others whose names meant things similar to FAIR, beautiful, pleasant. No mention of Seth's daughters or grand daughters at all.
Gen 6 says that the sons of God (those that called themselves by the name of the Lord from Seth's lineage in Gen 4) saw the daughters of men were FAIR, (those women already mentioned in Cain's camp that were fair in Gen 4) they took them wives of all which they chose (meaning they chose to follow the lusts of their flesh and marry wicked women for the wrong reasons rather than to choose the godly women in their own camp) and this resulted in a falling away (as it always does in the rest of the bible) until only Noah and his family were left.
See how simple the interpretation is without all these elaborate conspiracy theories and imaginative speculations? And that interpretation is straight from what is said in the text.
Also notice that the statement "there were giants (nephilim) in the land in those day" was made before the mention of the children of the sons of God and daughters of men. It says "there were giants in the land in those days. And Also after that..." and mentions that the children of the sons of God and the daughters of men grew up to become mighty men men of renown NOT NEPHILIM which is what the science fiction proponents always try to trick you into thinking that the text says. Don't fall for that old trick.
I too will not argue or debate with those who do not agree with my interpretation on this. I just put it out there for those who may be genuinely confused. To those who are committed to their Angel/Human mating producing giant baby hybrid theories, just scroll on by if you don't appreciate my post.