Yes, of course their perspective was relevant. They saw Him go up from them, which is what He did. Your correlation is invalid however, because He ascended right in front of them. The sun is very far away, and that distance makes the perspective critical, because a different perspective would...