Dear Christy:
Reads like the only way for you to not be exposed to his family’s poor opinion of you is to physically move away from them and have no contact with any. Because your husband, passively or actively, shares their opinion, probably better leave him behind as well.
It is not good for your son to live with an angry mother, so better you do what it takes to not have your anger triggered and retriggered, by moving away.
anita