Do I need to sand this whole thing down!

Sand and wipe ospho on it; that will kill rust in the areas you can't get sandpaper in.