Is it the fuel? Not really. Even though giant passenger jets do guzzle down fuel at a ridiculous 2km per gallon -- seriously, they need 8l of jet fuel for every mile travelled -- there areso many people on an aeroplane that the fuel cost gets split down to a much more reasonable price: a per person fuel efficiency of 169km per gallon. That's good! So why is flying so expensive? It's everything else.
And there's a lot of everything else: a thousand different kind of taxes to pay, a pilot's salary, the crew cost, airport fees, aeroplane cost, maintenance, the non-flying part of flying (the airline's cost of running business), and so on. All that can turn the $US2.50 ($3) of fuel it takes for a person on an aeroplane to get from NYC to Washington DC into $US70 ($96) of "everything else." And that figure is an estimate on the low end that doesn't even include whatever profit the airline would like to get.
Watch the video by Wendover Productions breaking down the cost of flying. It's a lot of little dollars here and there that adds up rather quickly.