You are correct that's why I said to file the scratches down until they are gone then sand smooth. If you completely remove the gouge/scratch you eliminate the possibility of a stress riser. Simple mechanics, if you can't remove all signs of the gouge then yes, you could get a stress riser. Even if you have to use a 1/2 round file and leave a slight depression, as long as it has smooth transitions it should be OK.
