Occasional Oil Pressure Light for 2-3 seconds in my 318. Should I panic?

Just change the sender and see it it goes away.
An idiot light is just that, a light for idiots. Just because the light doesn't go on doesn't mean that you don't have oiling issues. The thing could be running with 10 lbs of oil pressure all the time and the light wouldn't come on. If the system is sucking air somewhere it would show up with an unstable reading on the gauge or with declining oil pressure as rpms increase (the opposite of what should happen). This is CHEAP insurance for your new engine!