Palantir began in the mind of Peter Thiel, the Silicon Valley investor and PayPal founder. He began thinking about it in 2003, a year after he sold PayPal to eBay. It was two years after the terrorist attacks on the United States, and Mr. Thiel, a fierce libertarian, wondered if the world could be made safer without losing freedoms.
“I felt we were drifting to a place in the U.S. we’d have a lot fewer civil liberties and no real effective protection,” Mr. Thiel said. He enlisted Mr. Karp, a college friend, as well as veterans of PayPal and his investment fund.
A big fan of J.R.R. Tolkien, he named Palantir after a set of magic stones in “The Lord of the Rings” that grant powerful people the ability to see the truth from afar. The company headquarters are called the Shire, after the home of the Hobbits.
Palantir’s founders started with an idea from PayPal. At one point, PayPal was losing the equivalent of 150 percent of its revenue to stolen credit card numbers. It figured out how computers could spot activity — like a flurry of payments to a brand new account — at a global scale. The flagged actions would then be put before a PayPal employee to investigate.
Palantir’s founders thought the same approach would work for national security. Almost no one in the venture capital world agreed except the C.I.A.’s venture fund, which gave Palantir $2 million; Mr. Thiel eventually put in about $30 million of his own money. In-Q-Tel gave the founders introductions to the C.I.A. and other spy outfits.
Palantir’s first full-fledged C.I.A. job was in 2008. Mr. Karp got more work from word of mouth, and donated Palantir’s technology to cyberactivists, who mapped Russian hackers attacking the nation of Georgia in 2008. (The spyware was rumored to have found Osama bin Laden, but Palantir would not confirm or deny such jobs.)
To drum up private-sector business, Mr. Thiel called on Mr. Ovitz, whom he knew through Marc Andreessen, the former Netscape whiz kid turned venture capitalist. At first, Mr. Ovitz thought Palantir could be used in selling online ads, but the housing crisis changed his thinking. Banks had thousands of homes in foreclosure and no idea how to efficiently clear the backlog in a collapsing market.
"The idea was to pick one bank, and the rest would follow,” Mr. Ovitz said. JPMorgan was the first. Much as Palantir figured out navigating Baghdad by analyzing recent roadside attacks, satellite images and moon phases, it derived home-sale prices by looking at school enrollments, employment trends and retail sales. Data that JPMorgan thought would take two years to integrate was put into action in eight days.
JPMorgan still uses Palantir for cybersecurity, fraud detection and other work, loading half a terabyte of data onto a Palantir system each day, according to a Palantir video. A spokesman for JPMorgan said the bank uses Palantir, but would not comment on specific projects.
Morgan Stanley is another customer.
“No human can look at all the data sources at one time,” said Jim Rosenthal, Morgan Stanley’s chief operating officer. The company uses Palantir to spot money laundering and employee theft, as well as for cybersecurity.
Government clients also struggle with a data explosion. “Everything becomes more difficult, the more crime becomes global, the more state actors are involved, the more trades there are around the globe,” said Preet Bharara, United States attorney for the Southern District of New York. “It’s malpractice to have records and not search them.” He has used Palantir for several cases, including the SAC investigation.
Investors are growing restless. Mr. Karp says he hears from them nearly every day. Joe Lonsdale, a co-founder who left to form his own investment firm, still has shares in the company and has stated in online forums that Palantir will go public. But while an I.P.O. may be hard to resist forever, Mr. Thiel said in an email that Palantir “has no plans to I.P.O. in the next few years.”
Palantir is now Palo Alto’s biggest tenant after Stanford, occupying about 250,000 square feet in downtown buildings, which hold many of Palantir’s 1,500 employees. Contracts around the world have surged as everyone’s data increases in size and diversity. Mr. Karp thinks the firm can grow to 5,000 employees for its maximum effectiveness, without, he says, the possibly corrupting influence of going public.
In Tolkien’s tale, the world was saved from darkness. The Shire, though, became an industrial wasteland. Mr. Karp hopes that Palantir can save itself, along with the world.