iPhone games have redefined the meaning of cheap--in ways both good and bad --but you still need to drop at least $200 for an iPhone or iPod Touch. For real portable retro gaming with street cred, may we suggest the Retro Mini Handheld, which for only $49.99 plays old NES cartridges (once you've blown the dust out of them).
Considering NES carts can be had for pennies at Salvation Army thrift shops, this little beauty might be the best way to play games for next to nothing at all. Compare, for instance, the cost of a $2.99 eBay copy of Metroid to a $5 DSi download of WarioWare: Snapped. Which would you rather have? Of course, the cartridge is larger than the machine, but at least people will know what you're playing.
Running off 4 AA batteries, the Retro Mini gets a smooth eight hours of game time (according to ThinkGeek, where this bad boy is on sale). As to whether game details like text are discernible on the "hi-def" 2.4-inch LCD screen, we are uncertain. But a composite cable jack turns this into a plug-and-play console...just add a TV.
What's more important is how this only serves to remind us that this might still be the best way to play official NES games on the go. Nintendo, are you listening? Where is our Virtual Console for the DSi?