There is no "fabric" but Earth is surrounded by space. The difference is in the center of the Earth vs the crust. Not, the South Pole vs the Equator. The Moon is not the Earth so Earth's gravity has a smaller effect.
I think you have a great grasp on physics. Imagine if the earth was upside down so the north pole was on the divot! Then time would be faster on the moon instead of slower. Of course that couldn't happen because that would mean that time would go backwards on the south pole, not to mention that everyone on Earth would get vertgo.
The expansion of space-time does not include our solar system . THAT has not been explained by mainstream astrophysics .