So there is scripture about this very thing. According to this passage one may esteem or keep a day or not keep it. Judging the other for doing so is sin.
As for the one who is weak in faith, welcome him, but not to quarrel over opinions. One person believes he may eat anything, while the weak person eats only vegetables. Let not the one who eats despise the one who abstains, and let not the one who abstains pass judgment on the one who eats, for God has welcomed him. Who are you to pass judgment on the servant of another? It is before his own master that he stands or falls. And he will be upheld, for the Lord is able to make him stand. One person esteems one day as better than another, while another esteems all days alike. Each one should be fully convinced in his own mind. The one who observes the day, observes it in honor of the Lord. The one who eats, eats in honor of the Lord, since he gives thanks to God, while the one who abstains, abstains in honor of the Lord and gives thanks to God. For none of us lives to himself, and none of us dies to himself. For if we live, we live to the Lord, and if we die, we die to the Lord. So then, whether we live or whether we die, we are the Lord’s.
Romans 14:1-8 ESV
So we celebrate Christmas unto Christ, you don't celebrate it unto Christ. There really is nothing left to be said about it.