I can see your point, and there certainly is a lot of gratification in working the HF bannds, but handhelds are just too darn useful to be ignored. They are great for when you go to swapmeets, camping or similar, especially if other hams will be attending. I'm bringing mine to the Yuma Hamfest this weekend so that other hams can contact me and we can meet up.
Also, HTs are popular with emcomm groups such as ARES and RACES, and even CERT groups are beginning to use Amateur Radio. Most of the disaster communications are within a local area, and in the cases of major events such as an earthquake, hams in the field with a HT can relay messages to hams sitting on a HF base station so that the messages, such as personal welfare messages to a victim's family, can go out over traffic nets to other states.
A handheld can open a new user to weekly nets, round robins and rag chews, and with a little "local" experience on a HT, the new hams can "move up" to a HF rig and round out their experience.
Not to mention the fact that you can purchase a used 2-meter HT for around $60 or a brand new dual-band Chinese HT for not much over $100. Even a very basic and very old HF rig is going to cost $300 unless it needs work. I know that for a fact because I'm using an old Atlas rig - nothing fancy, but I'm having a lot of fun with it, and the price was right!
What would really be nice is a 100 watt handi-talkie with both HF and 2-meters that worked from some super built in battery and antenna system that could serve as both a local repeater HT and a worldwide HF rig! I've often thought about getting one of the battery operated HF mobiles, but I don't think I could do much with 5 watts and such a small antenna. Maybe 5 watts and over 200 feet of antenna up at least 30 feet might work!
I wouldn't recommend to any new amateur operator to purchase a handheld as a first radio. Being FM only (with very few exceptions), limits your experience of amateur radio to a very limited spectrum (if you'll excuse the pun).
There are a few amateurs who will also add, that FM isn't much more than senior CB. To be honest about it, I tend to agree.
If you were to purchased one of several different all-band transceivers (Icom, Kenwood, and Yaesu all make at least one), you open yourself up to SSB, CW, and even a window into the world of DX.
While a used all band, all mode radio will cost more than a basic 2 meter handheld, they are about on par with a medium featured one.
In short, please, do yourself a favor; don't limit yourself to a narrow segment of the panorama that is amateur radio.