Firstly, kudos to you for understanding that your ex wasn’t/isn’t good for you. Knowing that will help your reservations as time goes along. I’m not an expert in relationships or emotions (in fact I have a thread on here too about my own issues) but I would like to offer any advice that I can.
The first thing I would suggest is blocking your ex’s phone number. This is normally free to do through your phone company. Out of sight, out of mind has worked well for me in the past and since it sounds like the crux of the emotional fallout comes when he contacts you breaking that cycle is key. If you don’t want to outright block him then you can tell him that you cannot be in contact with him for awhile because you are trying to focus on yourself and you would appreciate it if he would honor that. Any contact, short of an emergency, after that you don’t have to look at or respond to. Personally, I would just block his number because I know that I would want to read the text messages and/or listen to the voice mails.
How open is your communication with your significant other? Have you told him about your worries? I also have a very caring and understanding boyfriend and I’m realizing that being open with him about how I’m feeling can be more helpful than anything because he understands why I might be withdrawn and offers reassurances. Maybe your boyfriend can do the same for you.
Lastly, I suggest remaining honest with yourself. Remind yourself of all of the things that were wrong with your past relationship, like the things you listed here, and list all of the things that you love about your current relationship. Daily reminders of the good in our lives has a way of spreading positivity and joy throughout the day.
I hope that at least some of this is helpful!
Blessings to you,
Marquita