I noticed that the boys that actually caused the most trouble were in classes that had male teachers.
The male teachers did not come in with them during library time.
My previous school that had a male teacher did come with his class but even he could not handle certain boys, who challenged him. I was ok with them though, but yes I had to protect the girls from the boys.
It was always boys that got banned from the library. The older boys seemed alright, its just the young ones that didnt know any better.
I had a few girl bullies too but as long as I kept them away from others they didnt do too much damage. Usually did this by giving them jobs to do.