replacing trunk extension

My trunk extension is like swiss cheese in one area. Can I change that panel without disturbing the adjacent panels? Would I be better to also buy the lower quarter patch panel and cut that off to better access the trunk extension?