I have never turned away from any family member, and I don't think I will. If someone is cruel to me, I go to the Bible, and I pray for them.
My sister was always playing mind games with me. I just prayed for her, and a month ago, God saved her. She is a very different person! Of course! She was always trying to be better, smarter, richer than me. Now, she writes wanting to know about different Bible questions, how to love and forgive our mother and brother, etc. I have been a Christian for 38 years, so I have never harboured ill will in my heart, towards them, like she has. They both are extremely dysfunctional, but I go out of my way to show God's love. My sister is now praying for them. It's a start!
I really don't like the word "toxic." It is a very pop psychology term, and it is all encompassing. It is also judgemental. I don't disagree there are people with personality disorders, or psychopaths and sociopaths, which would be a better label, if one is going to label someone. They are so hard to deal with. They are the types that's likely will not change. But, our duty is to love and pray for them. They may not change, but perhaps someone else will see us being a light, and God will help them understand that knowing Jesus is what makes the difference.
Of course, if we are talking actual abuse - domestic violence, and so forth, then you do need to get out of that situation, because it may escalate, and you might be in danger. I agree the Boundaries book by Cloud and Townsend would be a great book. They also have other books, that target specific things like marriage. They are both Christian, and are far better qualified than most of us to discuss these questions.