What grit paper are you using? A little tip... The sharper the sandpaper the straighter your work will be. The quicker the filler comes off the straighter it will be. When doing mud work I start with 36 grit and do it while the mud is still "tacky" this will shred it off quickly. When it's 90 percent done let it fully cure hard and finish it off. You won't have dust, you won't be sore, it won't take forever, and it will turn out better. Obviously you'll need to glaze it and finish with 180 minimum before primer. You'll end up with a pile of "saw dust" under the panel not fine dust everywhere.