I do believe that yes, feelings of guilt can stop you from having sex with your spouse. But there is probably a lot more also going on in his head, right now.
Are either of you seeing a counselor? You both should be. Individually. IMO, it is too early for a marriage counselor unless you are lucky enough to find one that not only specializes in infidelity, but also comes at the problem in a way that you are both comfortable with.
You are still very early in recovery. Two years is the average, with some able to feel normal again in 1 year and others taking 5 or more. So, while you are needing this reassurance right now, he may be nowhere near ready to give it. That doesn’t mean that he doesn’t still find you attractive. Just that his head is still a mess.
Without meaning to upset you further, it is possible (likely even) that he still has feelings for the AP. Or at least thinks he does. I would say that they still long for the affair, regardless of the partner, because the affair is usually the exciting part. The partner seems to be, more often than not, just the convenient person that showed them attention. Who the AP is IRL is irrelevant to the situation. (Not that we BS’s don’t hate everything about them regardless.)
You may have read by now the concept that the affair acts like a drug on their brain. That doesn’t excuse the behavior, just to be perfectly clear. But it takes a very long time to get over addiction. The brain has to learn again how to find happiness in normal things without the rush of the drug.
Are you both working through any of the classes offered here or elsewhere on reconciliation? You should be working through all of these things on a regular basis. While it may be great that he’s bringing you coffee in bed, there are much deeper issues that need to be addressed other than his not treating you right for the last however many years.
He needs to address the real issues that allowed him to become involved with someone other than his wife. Oftentimes, like you have mentioned, childhood traumas need to be healed. Other psychological issues such as narcissism might play a role.
I can’t speak for you, obviously, but I think it just may be too soon for you to expect physical intimacy. While you may desperately need it right now, it may be detrimental to your overall goal of a truly intimate relationship with your H at this point. Emotional intimacy really should be developed and strong before we move to the physical.
Start shopping for a therapist. You need someone in your court that can help you through this traumatic time that isn’t also going through their own trauma.