I have a 2009 A3 and it seems like everything I touch just about breaks on the interior of my car. I was wondering if it's a common thing for these or its just my bad luck?

 

Maybe you should have posted a picture of your hands and biceps to give us some clues

But all joking aside I haven't had anything in the interior break on me yet. I see some paint flaking (armrest, headlight switch) but otherwise it has been holding up pretty good.
What exactly have you been able to break so far?

 

Hahaha true true!
Well when I got the car the lid to the center console was broke off. And since my rear view mirror, the bracket that holds the sun visor up, the plastic latch that holds the sunroof screen/cover, and the back center armrest door had broken. It's not like I'm rough with it or anything. They just break

 

That is all stuff that gets moved constantly and some of it had some cheap plastic - nothing unusual to break unfortunately.
I know that the bracket holding the sun visor, the latch for the arm rest and the latch for the sunroof screen are known to fail. The rear view mirror shouldn't be that bad - that was probably just bad luck.

I'd say from what you posted that is all to be expected to fail for an A3 that is older. I even had the armrest latch and sun visor bracket fail on the A6 within 10 years, it's just stuff that was not made to last.