He´s right to be mad at you.
If you know you deserve better ask yourself WHY do you want to get back together with someone who treats you bad?
I dont know if he has feelings for you but honestly if my friend had a boyfriend like that – I would ask more about her as well. And I would be mad too if she wants to get back together with someone who treats her like shit and she knows it. This is what friends do.
Ask yourself – if your friend was in an abusive relationship – wouldnt you want her to leave the guy too? So are you worth less than anyone else? You said you´ve broken up repeatedly. It seems to me your boyfriend doesnt want to change. So why are you staying? Why are you going back to him?
If you´re worried about losing your best friend – take better care of yourself and stay away from your Ex.