Solenoid Issue?

Just wired the starter and battery with new cables. The post on the solenoid that the wires connect too is loose inside but tightnens up after you put the wires on with a nut to hold them on. Also The solenoid is loose where it bolts to the body so if this is where the solenoid gets its ground it might be part of the problem too.

When I turned the ignition the starter kept cranking even after shutting the key off, had to pull battery cable to shut off starter. Now when you connect the battery cable the starter just cranks. Any ideas?