Yep rubber cap and clamp is the simplest solution. To remove the nipple and install a threaded pipe plug is doable and might look a little better but... If/when you need to replace the pump you be doing this work again. Modified cores you exchange would be odd ball too.
