Ride at a pace you can maintain mate and dont feel you have to keep up, I was where you are 9 months ago and for various reasons havent got much further but I have learnt not to try and keep up.
If it takes you 5 minutes to get up a hill do so, they will wait if its an open invite, if they dont then they arnt worth riding with.
Trying to keep up and ride everything at the same pace tires you out too quickly and your performace drops significantly more so. Fitnes takes time, I dont have the strenghth in the legs to drive up hill so dont try to do so, its something I need to work on when out on my own, especially as I may have stupidly agreed to do something involving riding the South downs way next year