Rebuilding
For whatever it's worth, in your summary of recent life events and the identified toll it is taking on you, I must say you are likable. As odd as that seems to start the answer, I think it is essential that people know through how they choose to present themselves and their situation if it is read with favor for the person or if the conversation needs to change to accountability. For you, you have been wronged but seem to accept that wrong happens. You have every reason to be hurt, angry, resentful, and bitter, but bitter doesn't want a family that has hurt her back, even if it is the ideal version she imagines when she wishes for it.
How You Remember the Past Isn't Real
I must note that what your mind is telling you isn't true. As hard as this is, as much as you want to hold on to the past and cling to that as the idea, it wasn't. I cannot admit I would have been able to say that in person, not like that and not this soon in a meeting. However, it would be best if you moved on so that you do not get bogged down by a false narrative from the past that things were ever as good as you imagined.
Hurt has a way of activating a retrospective look at life, covering the nuances that weren't so favorable. Hurt stimulates what we have lost, what could have been, but rarely allows us to see things as they really were. The phrase hindsight is 20/20 comes from this reality. You remember a family that never was; now, it is the anchor bogging you down. If the "whole" family would have been good and not on the road to where it went, it wouldn't be here today. In other words, there were underlying things that may have been missed in him, you, both, that would not have allowed for a continued relationship.
Adult Children
Your adult kids should want to see you doing well. Since you being in their life is a primary concern, why would you think they wouldn't want to be in your life or that you are somehow not worthy of that? Does your ex-husband present a better narrative? Do they support you? Do they like your Ex because he blames you or something? Your adult children have options where to spend their time, and where do you think they will spend it, taking care of mom and her emotions or around a father who did wrong but can carry a conversation?
Sadness has its place here, and mourning for a loss is crucial. However, it does seems like you may be digging a deep hole of despair, possibly because of the man you are with now. You have found comfort, you say, but he is not what you want. Despite him being a "comfort," you aren't very comfortable. I believe you are going about the divorce improperly by seeking comfort instead of letting go and mourning your loss of identity in the marriage. Comfort allows you to hold on to old beliefs about yourself and your family that are no more.
There are assumptions I have to make for the sake of the question. What I have seen in your question is much hurt, accountability, and a realistic view of the future. With your foundation of insight, I think you will move on from this just fine if you start allowing what has happened to you to no longer be the reason you aren't happy. What has happened is tragic and unfair; you were wronged. However, in a marriage, there are two parties responsible for making it work; what do you think you can do to work on yourself so that you can grow and find yourself happy for others when they make choices you disagree with?
I do not know your current relationship with your children. However, children tend to go where they are accepted. Even adult children must be accepted for who they are, their autonomous persons, not just the children of their parents. This means that our adult children may say and do things that we disagree with, but they do so as an opportunity to figure out who they are, where their boundaries are, where they stop, and others start. Being a parent, you can listen to their lives as though you are the most curious person in the world, removing any preference or judgment. Your relationship from what has happened isn't necessarily affected. However, how you manage yourself from here on will influence their involvement in your life. Again, they won't want to hear if you are sad and mopey or talk about their father often. Learning more about your children would be best.
I suspect that if you hold your previous family in nostalgia, you hold your children similarly. But neither the family nor the children are the same today as it was; it is better. As wild as it sounds, have you allowed the children to grow up in your mind? If so, what are they like, their interests, etc.? Like the family you once had, there has to be a letting go of what our mind tells us and practice of being present and accepting current reality.
Do not be scared to talk about any topic; get used to being uncomfortable, and the belief that you need to correct the wrongs must be eliminated. No more thoughts from the past; they are hurting you. The rest of your life is waiting. A therapist can give you that space to talk through some things without getting burned out, as family and friends can. However, a good therapist will hold you accountable and ask what you can do now to help the future you recover from this.