Car turns when rolling backwards

I know it’s not the brakes, they are Strang and have very little drag. I’m going with the caster, more on one side than the other maybe?