I think you need to ask yourself where does the "Worry" come from. Are you worried that you do not have enough for retirement? Are you worried that your friends are doing better in life?
At the end of the day, i would say that you are already better than most people by being able to plan well and live below your means.
Maybe you feel like you are shortchanging yourself in life, but life is more than just money and work. Do something that will bring you joy now that your career and finances are well in tact. Perhaps spend more time with your friends and family or take up a new hobby?
It depends on whether you are fulfilled and happy with achieveing more, or with a stable life. From what you wrote, it seems that you are contented with your life. So what makes you feel that you should be achieving more?