I'm new to Ayn Rand (I'm about 300 pages into Atlas Shrugged) so I don't know a whole lot about Objectivism. But from what I have gathered so far from Atlas Shrugged, it seems like her philosophy *was* libertarian. So what's the problem between objectivism and libertarianism?
Well, that's just it.
Rand's political philosophy is nearly indistinguishable from libertarianism.
She heavily influenced the movement, both through her intellectual influences upon its leaders (such as Murray Rothbard, and Roy Childs, Jr.), and through her books, which probably attracted more people to libertarianism than any other single source. (For a nice overview of her impact on libertarianism, see the series of articles by Roy Childs, Jr., Ayn Rand and the Libertarian Movement.)
I think Rand disliked libertarians because they embrace anyone who believes that people should be free to do as they please so long as they don't physically harm someone else.
Do you believe in a free society because it will maximize most people's utility functions?
Do you believe in a free society because it is the only appropriate society for man qua man?
Do you believe in a free society because God appeared to you in a dream, in the form of a burning Teletubbie, and told you so?
All are welcome in the libertarian party.
Of course, very few libertarians come to libertarianism through visions of burning Teletubbies; the vast majority are persuaded by utilitarian arguments (a la Friedman, Hayek, and Mises) or by natural rights-style arguments (a la Rand, Rothbard). Note also that this doesn't mean that most libertarians think that it doesn't matter how you arrived at your political beliefs--many care passionately about the issue. However, libertarians argue that, for the purposes of political influence, freedom lovers should set aside their philosophical differences and band together to fight for their common goals.
Rand believed that it was inappropriate to attempt to promote a free society divorced from its proper philosophical underpinning (which she believed to be Objectivism). She also strongly disagreed with the anarcho-capitalist faction of the libertarian movement (led by Rothbard) calling them "hippies of the right).
In addition, "Orthodox" Objectivists [those associated with the Ayn Rand Institute] will argue that to support the Libertarian party gives moral sanction and respectability to the beliefs of some of the decidedly minority views within the libertarian party. For example, here's an excerpt from Peter Schwartz's essay On Moral Sanctions:
"...Thus, the "benefits" of speaking to Libertarian groups are as
nonexistent as the "benefits" of exhibiting books at an Iranian
fair. The Libertarian movement is not some innocuous debating club. It
is a movement that embraces the advocates of child-molesting, the
proponents of unilateral U.S. disarmament, the LSD-taking and
bomb-throwing members of the New Left, the communist guerrillas in
Central America and the baby-killing followers of Yassir Arafat. These
views have all been accepted under the Libertarian umbrella (and
remain accepted under it by everyone who still calls himself a
Libertarian). It is these types of vermin that one is lifting into
respectability whenever one sanctions Libertarianism - or whenever one
maintains that ideas can be analyzed without being evaluated...."
See Schwartz's essay Libertarianism: The Perversion of Liberty, for a more in depth explication of the problems "Orthodox" Objectivists perceive with libertarianism. For the other side of the issue, see Jim Peron's four part article Objectivists and Libertarians.
Plug: In my opinion, The Objectivist Center takes a much more rational view toward libertarians than The Ayn Rand Institute.