I learned the hard way that I needed to plug my stuff in. I'm unsure if it comes with one or if I need to order one myself.

 

I doubt the standard A4 comes with a block heater. There are only a few aftermarket types out there: dipstick heaters which probably won't work, a block heater that replaces a freeze plug in the outside water jacket - good luck finding one for an A4, and a heater that goes inline in your radiator hose - again, good luck finding one that fits neatly.

Did you resolve this another way?