I would have a heart to heart with him and let him know how you are feeling. You may also want to find a good couples counselor and see if you can work out these issues.