Removing the old paint “edges”

If your fingers are doing the sanding, its going to continue.
Block it with some 80 grit. Reprime and block your way up to 320/400 wet sanding.400 or 600 is all you need for paint.