On a given day, even walking down not-so-mean streets, you are apt to pass people who are either bigger, stronger, faster, more agile, better-trained, better-armed, smarter, meaner, and sneakier than you, just to name a few.
Should you get into a set-to with somebody who is all of these? The smart money isn't going to bet on you to win.
You understand why, don't you?
Chances are you aren't going to run into too many people who are all of these, not if you are serious in your training, and you have learned how to compensate.
Pump a lot of heavy iron, you can get bigger and considerably stronger, but not appreciably taller; if you are five-foot high, even if you can bench a Volvo, you won't ever get to six feet, with the reach of somebody that height. The amount of muscle you can pack on will be limited by your frame and testosterone, and a middleweight using strength alone can't outmuscle the heavyweights.
You probably can't get much smarter than you are. However bright you were on your ninth birthday, chances are you'll stay close to that, at least in terms of IQ. You can gain knowledge, but the wetware's processing speed doesn't get a lot quicker.
You might could get meaner, and by this, I mean develop an attitude that you are the person who gets to walk away, whatever it takes. It's what they teach the Marines and the SEALs and the Rangers–get it done and go home.
You can increase your agility. You can, to an extent, increase your useful speed. But where you can really improve is in your skill. You can be better-trained and better-armed.
You can surely be sneakier. That's my preferred route. I'm old, slow, weak, a pushover. So I have to come up with something else.
When you compensate for those things that are less amenable to change by fixing the ones you can fix, you could give yourself a useful advantage, should push come to shove. And if you can't run away and you have to defend yourself or your loved ones, then you want to play your game, using your strengths, and not those of your attacker.
There's an old saw: Don't get into a fight with an old man–he'll just kill you. It speaks to the point eloquently: You can be old, slow and weak, but if you have a gun or a knife or a lightsaber and the wherewithal to use it, that MMA champ's physical advantages count for a lot less. Assuming, of course, you can get to your weapon before he takes your head off.
All of this is to say that you need something that you can reach for when you need it. You need to figure out what that is, and develop it.