I had the exact same symptoms when my wheel bearings began to fail about a year ago. I first noticed it after doing the A8 brake upgrade which led me to think I did something wrong installing the rotors. I spend a couple weekends disassembling/reassembling my brakes, the click did not go away but the car drove fine otherwise. There was no play on my wheels at the time.

Couple months goes by, the click got slightly louder and my wheels started to make droning sounds at highway speeds so I decided to check my wheels again and found a very slight wobble. At this point I was pretty convinced my wheel bearings are going bad so I changed them and it solved my wobbly wheel problem along with the annoying click while braking.

my thoughts on the click was that my bearings were so worn that there was significant play between the inner race and outer race of the bearings. This is why my wheels wobbles and clicks while braking.

 

OK, update of sorts:

Sometimes if I jam on the brake at low speeds to stop, it creaks like I just jumped on an old bed.

I am starting to think that it's the strut tower bushing that jiggleo had suggested.