How do I go about rebuilding trust and my attraction towards my partner?
I'm going to try to answer your question as best as I can from the information that you shared with me. You stated that you and your partner have been together for going on three years but it sounds like there was some time that the two of you weren't together and I'm not sure how long of a time period that was. Being together for three years is a good foundational start to the relationship. It is during this time that the trust in the relationship is being built. When that trust is violated, it can be hard to regain it. You had stated that "he practically was pursuing another person" I'm not sure what behaviors were happening. Was he having a sexual affair, an emotional affair or was it something else? Depending on what you mean by that statement of pursuing another could have an implication on how much trust was violated in the relationship.
Gaining trust back does take time. Can a relationship survive in a worst case scenario having an affair? The short answer is yes, however it takes work and commitment from both people involved. Are both of you committed to doing what it takes to build that trust back up? As part of that work and commitment, both parties will need to be completely open and honest with each other as well and not have any secrets from one another. This is where it starts.
In regards to your statement about "losing attraction to him", I would pose this question to you, how does having facial hair and being overweight change him as a person? It sounds to me like he is trying to find out who he is as a person. If having the facial hair has helped him to be more confident in himself, is that a negative to the relationship? I would question if you are feeling threatened by the confidence that he is gaining. Another question that presents is if you are using the changes in his appearance as an excuse because you want out of the relationship at this point?
I would suggest that you put together a list of the pros and cons to remaining in the relationship and the pros and cons to ending the relationship and being completely honest with yourself. Should you want to remain in the relationship, I would recommend that there is some relationship counseling to help improve the relationship.