So I did some basic tests today and narrowed it down to the gap that's there between the side view mirror and the housing it's attached to.
I put tape around the open gap (it's effectively the gap that's needed to retract the mirror in) to seal it and the noise went away.
Taking it into the dealership soon. If it was the side view mirror and it needed replacement, the service advisor said it'll likely need to come in from Germany so it could take a while.
This is a top view of the area I wrapped. I basically wrapped the bottom and part of the other side where the actual mirror is to cover that gap.
