Black holes line the edge of the Milky Way disk and are slowing down its rotation.
The Sun is currently in a slow portion of its orbit.
The Sun is slowing down in its orbit due to friction from collisions with galactic dust.
More material resides in the region outside the Suns distance from the galactic center than within it..
Light cannot reach us from regions outside the galactic plane, so we cannot include this material in our estimate.