I agree with that generally. If you replaced a 600cfm Carter with a 600cfm Holley you are still in stock.
If you replace the stock cam with one "slightly better".. IMHO.. no longer stock. As you say.. performance gain.
I don't consider headers an engine upgrade even with the sometimes massive gains you get over the stock logs. Wonder if I'm alone there.