There are basically two types of drivers: those who get worked up about people driving slowly in the left lane, and those who do it all the time and have no idea they're upsetting everyone else.
In case you're in the second group, some background: every state has some sort of law that discourages people from traveling in the left lane on multi-lane roads and highways. It's not that you're never allowed in the left lane, just that you should only use it when necessary, for passing, then get back over.
That's because even if you're driving fast, there's always someone going faster. If you promptly get back over after passing, that car will be able to pass you, allowing everyone on the road to get to their destinations as quickly as possible. If you don't, it'll inevitably lead to buildups of traffic and likely raise the chance of accidents.
The system works best when people aren't hogging the left lane. That's one reason why National Motorists Association has declared June Lane Courtesy Month in an effort to to raise awareness about the importance of getting out of the left lane.
Here's Vox's small contribution — an explanation of the reasons why traveling in the left lane is a bad idea.
State laws restrict driving in the left lane
Every state has some law on the books restricting use of the left lane. In 29 states, the law says any car that's moving slower than the "normal speed of traffic" should be in the right lane — so even if it's going at the speed limit, a car that's not moving as fast as the other cars around shouldn't be in the left. Georgia recently increased the penalty for violating this law to a misdemeanor.
In 11 states, the laws are even stricter — specifically saying the left lane is only for turning or passing. Most of the remaining states say cars need to get over if they're blocking traffic that wants to pass, or if they're traveling more slowly than the speed limit.
Police are cracking down on left lane drivers
Traditionally, these laws have seldom been enforced, and were often just used as a legal excuse for police to pull people over for unrelated reasons.
It impedes traffic and probably makes everyone less safe
There's a certain rationale for these laws and enforcement campaigns. When drivers travel in the left lane, it makes the road more congested and probably more dangerous for all parties involved.
Research shows that many traffic jams result from a surprisingly small number of slow cars obstructing traffic, with their effects rippling outward. A small buildup of cars that can't pass because someone is driving slowly in the left — right next to another car traveling slowly in the right — is the exact type of scenario that can start this cascade of traffic.
Now, some people counter that as long as they're going the speed limit, they don't have to move over — and by slowing down would-be speeders, they're making the roads safer.
Apart from the fact that in 44 states, simply going the speed limit doesn't permit drivers to travel in the left lane, this argument doesn't make a lot of sense based on research into how accidents occur.
Unfortunately, there isn't much research on the effect of impeding people from passing in the left lane specifically. But there is evidence that slowing down and changing lanes is more dangerous than speeding.
Lanes changes account for about four percent of all car accidents in the US, and perhaps as much as ten percent of accidents on highways. Meanwhile, research has generally shown that the strongest predictor of an accident isn't speeding, but variance from the average speed of traffic — and a car going five miles per hour slower than the surrounding traffic has a greater chance of causing an accident than one going five miles per hour faster than it.
If relatively slow drivers are scattered among the right and left lanes, faster drivers have to repeatedly slow down and weave back and forth, changing lanes many times to pass all of them. If the slower drivers are all driving in the right lane, a faster driver can pass several at a time, then get back into the right, cutting down on the total number of lane changes and eliminating the slow downs.
Update: There's even some evidence that this is why accident and fatality rates are so much lower on the German autobahn, compared to US interstates, even though speed limits are higher (or in some places nonexistent) in Germany. Lane discipline is much more strictly maintained there, allowing drivers to travel more safely at faster speeds.