you're assuming gravity is limited by the speed of light.
I was not postulating how far away something could be to not fall back to earth, just that the effects are felt to the edge of the universe. Gravity is a force, without mass, so even though earth is 4.5 Bn years old, the gravitational effects would be theoretically measured, infinitesimally, out at the edge. Other objects with greater mass would mitigate some of the effects but the force itself, would be felt way farther than is measurable to us.