Don't feel guilty or ashamed just because you feel a certain way. If you beat up on yourself for letting something bother you, you are being a perfectionist and you are denying that you are a human being. We all make mistakes. We all let others (especially people we are attached to like "significant others") drive us crazy sometimes. The solution is to acknowledge that you are making progress but that you are not perfect. Giving yourself a break will keep you from needing to escape from the pain by using because giving yourself a break will make you feel better. It relieves sadness, stress and is a positive personality change. It also helps you make decisions because you won't worry about beating yourself up if the results don't turn out as planned.
Give yourself a break. It is normal to feel annoyed at times. But as soon as possible (especially before going to sleep), give yourself a break and relax. Exercise acceptance. Ask yourself if you have done everything reasonably necessary to deal with the problem, then forget about the problem and breathe. Finally, use gratitude and perspective and loving thoughts to turn your attitude around.