Bad to drive without a kickdown?

Is it bad to drive without a kickdown linkage? I have a 904 in a duster and have heard that it can ruin the trans. Also heard that it's fine, just doesn't shift automatically. Will it ruin the trans?