Are you sure that charge was for brokerage and just not Duty/taxes?
Canada Post charges only $5 for Brokerage ($8 for Priority Packages) IF they charge you at all. In 14 years I've only had Canada Post charge me once and that was taxes on said item because it wasn't made in the US.