Let's say you have a TV you like. Maybe it's a few years old; maybe it's new. But you have the upgrade bug, and can't decide what's best to get.
The following guide should help you figure out where the most bang will come from your dollars.
A new TV
Every year TVs get better. Over the past few years this is mostly from added features, but if your TV is more than a few years old, the latest crop are likely brighter and better looking with a higher contrast ratio. CNET has a great TV Buying Guide.
Despite the marketing, the latest crop of televisions aren't significantly improved enough to justify buying a new one if yours is just a couple of years old. This is true for 3D as well.
Speaking of which...
I have yet to meet anyone so enamored with 3D that they'd replace their new-ish television for this feature. If you're in the market for a new TV, then sure, check out 3D TVs. But it's not worth upgrading just for this feature.
If you don't have a Blu-ray player, get one. They barely cost more $100 and will play your current DVD collection better than your old DVD player. This is easily the most cost-effective upgrade you can get. Most new Blu-ray players also have built-in streaming services like Netflix and Vudu. Here's CNET's page on the best Blu-ray players.
Or, if you just want to get something that does streaming, check out...
Digital media receivers
Apple TV, Roku, LG Smart TV Upgrader, and many others offer an inexpensive entry into the wonderful world of streaming. Netflix offers tons of content for just a few dollars a month. Here's CNET's page on Best Network Video Streamers article.
In my testing, Blu-ray players have better scalers, making the best out of Netflix's mediocre picture quality. So if you're choosing between a $100 Apple TV, or a $150 Blu-ray player, the latter will likely look better, and, you know, play discs.
If you're using the speakers in your TV, you are missing out. No TV has good audio; they're all terrible compared to any of the options below. If you're looking for a substantial upgrade in your home TV and movie watching experience, nothing will do that more than better audio.
Soundbars are a single box that hold multiple speaker drivers. They mount or sit below your television. Most are about a TV's width, and 5 or 6 inches tall. While they offer an increase in sound quality over TV speakers, they're still not that great. The best option here is one that comes with a subwoofer, or at least has the ability to add one later on. The better ones have HDMI switching so you can plug all your sources into it, and play the audio from them.
Home theater in a box
The next step up in sound quality is an HTIB. These range in quality from horrible to decent. Most people like them because of the simplicity. Everything they need is in one box. Most HTIBs are five speakers, a subwoofer, and a receiver to power them. Many new models come with wireless surround speakers. These mount or are placed behind where you sit, and communicate wirelessly with the receiver. Keep in mind they still need to be plugged into a power outlet. Here's CNET's Surround Sound Buying Guide.
The better HTIBs are really just a receiver and speakers. You can save money, or get better quality, by buying these separately.
Receiver and speakers
Those new to the home theater world shy away from buying a receiver and speakers separately as it seems dauntingly complex. True, for the neophyte, they aren't as easy to hook up as a soundbar or HTIB, but the increase in sound quality can be worth it.
The biggest advantage, after being able to chose whatever receiver and speakers you want, is the ability to upgrade. Want get better speakers down the road? No problem. Want to replace your receiver with a better model with new features? No problem. You generally can't do that with an HTIB. The difference in price between a decent HTIB and a separate receiver and speakers isn't substantial. For less than $1,000 you could put together an excellent system that blows away any HTIB.
Check out CNET's Best Home Audio Gear.
If you already have a receiver and speakers, Audiophiliac Steve recommends getting an amp to give your speakers some more juice.
I'd recommend adding a second subwoofer. Not for additional level, but to even out the frequency response. Brent Butterworth did a brilliant article on the benefits of multiple subs.