Observe the thoughts that you have about him and eventually they will go away. Also, you’re literally feeling miserable about the past which does not exist, it’s all in your head. The past and future are illusions of your own mind, therefore it’s just as simple as letting go.