Mika,
In contrast to Angelique, I don’t think this has to do with “love” as much as “family” and “home”. Consider that even though the relationship he had with the ex didn’t work out in terms of romance, that she is still part of his sense of family. Almost all of us need family, and with his bouncing from foster to foster, its no wonder why he put some roots in with her and her family.
Consider a few options. One, if its just too messy for you, walk away. It would make sense, and you have the responsibility to care for your heart first. If you are unwilling to do that, or if your heart says stay, then consider accepting the ex and her family as part of his family, his sense of safety. Consider that you can’t be his “everything”, because that would put waaaay to much pressure on you. He needs people he trusts, can go to for a hug, or some consoling.
It makes sense that you would feel threatened by the ex, and why arguments and demands may fly back and forth between you and your boyfriend. Consider that it is difficult to feel in love while shouting, or being shouted at. In my opinion, if you think the intimacy with him has potential, accept him as is. This is a tough life to walk, sometimes, and without a family its even harder. When you attack the ex or his connection to the ex, perhaps you threaten to uproot his sense of safety, of family. Because you two are more new, it might not seem appropriate to see you as family. That takes time, consideration, warm sharing, and patience.
With warmth,
Matt