As an alternative to peanuts, which get _really_ expensive, and peanut butter which is messy: I've had really good luck with the peanut suet. Anything I have that'll eat peanuts loves the peanut suet. A suet cake is $0.87 at Walmart here, and a cage is $1.27.
You can mix it up with Berry Suet, and even Bug-infused Suet (no kidding!) I found the bug-infused suet at one of the major chain pet stores. The Yellow-billed Magpies here go bonkers for it... Even with the hundreds of birds I get in my back yard, I only go through a couple suet cakes a week. But I also feed mixed seed (15lbs/wk), Nyger (1.5lbs/wk), sugar water (2lbs of sugar a month), fruit (whatever I have left over that is getting too soft to be appetizing). I avoid the big nuts because the squirrels will instantly reduce $50 to squirrel-poo... [sigh]
If you're luring birds in, don't forget a bath! A heated bath can be had on fleaBay for <$50. I built my own water feature with an inline heater for <$30, and it's a full-fledged waterfall onto a shallow rock pond. Get creative and DIY this stuff. You don't have to spend real money on feeders or water features - the birds just aren't that "gourmet". ;-)