The process of developing yourself to become a Martial Artist that truly has the ability to defend oneself from great harm is not very much an easy process, at least not the unarmed combative portion of it. It requires a lot of physical fitness training, combative technique training, and spiritual work. What you have stated is a spiritual understanding, but the combative technique training which incorporates "sparring" which is like a fight simulation process of development takes time and is a delicate process. During the sparring, you will be facing up directly to your own ego and this challenges your spiritual level of development.
Through sparring, you can come to the ultimate realization that there is nobody you are really fighting but yourself, and you can come to the realization that there is no need to fight, you can just relax onto yourself. But then, you can see that even though there is no need to fight, you still fight anyway because it is fun to fight, it has turned into a game. Not to be taken too seriously. When sparring has turned into a game for fun, then your abilities in true self-defense will increase because you have engaged into this game so many times that it has become so natural to you, the fear is no longer there, you know what you are capable of because you have demonstrated your abilities to yourself day after day in sparring.
Then it becomes like swimming, biking, running, walking, eating, drinking, making love; there is no question of whether or not you can do it, you know you can do it because you have been doing it every single day in joy. Now you can defend yourself but you don't even know you can defend yourself, it has become such an integral part of your life that it is nothing special, nothing to brag about, nothing to feel prideful about, it is like breathing, you are not special over the next person because you can breathe, everyone can breathe!
You are not special because you can fight, everyone can fight, but those who put more time in development in their craft or art develop a certain level of proficiency and skill and when you notice somebody who has been working very hard at something for a very long time, some times you cannot help but stop, watch, and admire, but for the person engaged within the practice of his Art, he does not notice that he is doing anything out of the ordinary. Like a lion who is just relaxing at the Zoo wondering why the hell everybody is standing around watching him!? Well, he is not aware that he is a lion!