How I know something is wrong:
When idling, the engine shudders for about 1 second, then recovers. This repeats every few seconds. This occurs when the car is stopped, about to come to a stop, or turning at low speeds without using the accelerator. Steering feels very sluggish until the accelerator is applied, then a sudden boost in power steering occurs. If I quickly tap the accelerator and release it, the engine loses rpms, shudders hard, then recovers. If I do this several times in a row I can nearly cause the engine to die completely.

What I've tried:
Vacuum lines and seals (e.g. dipstick seal, oil cap, gaskets) appear to be ok (no cracks or leaking sounds) and I've cleaned out the ISV and throttle body. The disitributor cap/rotor, spark plugs, and spark plug wires were recently replaced. The ISV seems to be working fine, it opens/closes according to the voltage applied; unplugging the ISV causes RPMs to drop and an even rougher idle. The air filter is relatively new, but I have not looked at the injectors or fuel filter.

Possibly related problems:
Unfortunately my tachometer is toast so I'm not sure what rpm it's idling at. Increasing the idle speed seems to fix most of the symptoms, but causes a bump and clunking sound when I come to a stop with anything more than very light braking. Not a good sound. My car also makes tapping noises (at all speeds) that are loudest at just above idle. I suspect the valve clearances need to be adjusted.

Any ideas would be greatly appreciated. I can take a video with sound if anyone thinks it would be helpful.

 

I just get rough idle - a little vibration - when I put A4 2002 in Drive. There is hardly any vibration when in Park.
What can cause that?

 

The fuel filter usually affects high speed not idle. Could be injectors.
I once had a problem with the TPS on My 200QT that caused a rough idle.
Also the air valve could be sticking a little.
But my best quess would be that one of the 2 zillion vacuum lines is leaking.