Engine Hard Starting After Days Of Rest

What do you guys think about a dirty fuel filter in regards to starting?
If it was a dirty fuel filter not only would it not run well but it wouldn't start after being off for a few minutes??? If the filter is dirty it will be dirty all the time, not just after it's been off for a few days.