The Long Tail

From Wikipedia, the free encyclopedia

An example of a power law graph being used to demonstrate ranking of popularity. To the right is the long tail, to the left are the few that dominate (also known as the 80-20 rule).

The phrase The Long Tail (as a proper noun) was first coined by Chris Anderson in an October 2004 Wired magazine article^[1] to describe the niche strategy of businesses, such as Amazon.com or Netflix, that sell a large number of unique items, each in relatively small quantities. Anderson elaborated the Long Tail concept in his book The Long Tail: Why the Future of Business Is Selling Less of More (ISBN 1-4013-0237-8).

A frequency distribution with a long tail — the concept at the root of Anderson's coinage — has been studied by statisticians since at least 1946.^[2] The distribution and inventory costs of these businesses allow them to realize significant profit out of selling small volumes of hard-to-find items to many customers, instead of only selling large volumes of a reduced number of popular items. The group that purchases a large number of "non-hit" items is the demographic called the Long Tail.

Given a large enough availability of choice, a large population of customers, and negligible stocking and distribution costs, the selection and buying pattern of the population results in a power law distribution curve, or Pareto distribution. This suggests that a market with a high freedom of choice will create a certain degree of inequality by favoring the upper 20% of the items ("hits" or "head") against the other 80% ("non-hits" or "long tail").^[3]

The Long Tail concept has found a broad ground for application, research and experimentation. It is a common term in the online business and mass media, but also of importance in micro-finance (see Grameen Bank), user-driven innovation (Eric von Hippel), social network mechanisms (e.g., crowdsourcing, crowdcasting, Peer-to-peer), economic models, and marketing (viral marketing).

[edit] Statistical meaning

The tail becomes bigger and longer in new markets (depicted in red). In other words, whereas traditional retailers have focussed on the area to the left of the chart, online bookstores derive more sales from the area to the right of the line.

The long tail is the name for a long-known feature of some statistical distributions (such as Zipf, Power laws, Pareto distributions and general Lévy distributions). The feature is also known as heavy tails, power-law tails, or Pareto tails. In "long-tailed" distributions a high-frequency or high-amplitude population is followed by a low-frequency or low-amplitude population which gradually "tails off" asymptotically. The events at the far end of the tail have a very low probability of occurrence.

As a rule of thumb, for such population distributions the majority of occurrences (more than half, and where the Pareto principle applies, 80%) are accounted for by the first 20% of items in the distribution. What is unusual about a long-tailed distribution is that the most frequently-occurring 20% of items represent less than 50% of occurrences; or in other words, the least-frequently-occurring 80% of items are more important as a proportion of the total population.

Power law distributions or functions characterize an important number of behaviors from nature and human endeavor. This fact has given rise to a keen scientific and social interest in such distributions, and the relationships that create them. The observation of such a distribution often points to specific kinds of mechanisms, and can often indicate a deep connection with other, seemingly unrelated systems. Examples of behaviors that exhibit long-tailed distribution are the occurrence of certain words in a given language, the income distribution of a business or the intensity of earthquakes (see: Gutenberg-Richter law).

Chris Anderson's and Clay Shirky's articles highlight special cases in which we are able to modify the underlying relationships and evaluate its impact on the frequency of events. In those cases the infrequent, low-amplitude (or low-revenue) events — the long tail, represented here by the portion of the curve to the right of the 20th percentile — can become the largest area under the line. This suggests that a variation of one mechanism (internet access) or relationship (the cost of storage) can significantly shift the frequency of occurrence of certain events in the distribution. The shift has a crucial effect in probability and in the customer demographics of businesses like mass media and online sellers.

[edit] Chris Anderson and Clay Shirky

The phrase The Long Tail was, according to Chris Anderson, first coined^[4] by himself. The concept drew in part from an influential February 2003 essay by Clay Shirky, "Power Laws, Weblogs and Inequality",^[5] which noted that a relative handful of weblogs have many links going into them but "the long tail" of millions of weblogs may have only a handful of links going into them. Beginning in a series of speeches in early 2004 and culminating with the publication of a Wired magazine article in October 2004, Anderson described the effects of the long tail on current and future business models. Anderson later extended it into the book The Long Tail: Why the Future of Business is Selling Less of More (2006).

Anderson argued that products that are in low demand or have low sales volume can collectively make up a market share that rivals or exceeds the relatively few current bestsellers and blockbusters, if the store or distribution channel is large enough. Anderson cites earlier research by Erik Brynjolfsson, Yu (Jeffrey) Hu, and Michael D. Smith, that showed that a significant portion of Amazon.com's sales come from obscure books that are not available in brick-and-mortar stores. The Long Tail is a potential market and, as the examples illustrate, the distribution and sales channel opportunities created by the Internet often enable businesses to tap that market successfully.

An Amazon employee described the Long Tail as follows: "We sold more books today that didn't sell at all yesterday than we sold today of all the books that did sell yesterday."^[6]

Anderson has explained the term as a reference to the tail of a demand curve.^[7] The term has since been rederived from an XY graph that is created when charting popularity to inventory. In the graph shown above, Amazon's book sales or Netflix's movie rentals would be represented along the vertical axis, while the book or movie ranks are along the horizontal axis. The total volume of low popularity items exceeds the volume of high popularity items.

[edit] Research by Brynjolfsson, Hu, and Smith

[edit] Effects of online access

In his Wired article, Chris Anderson cites earlier research^[8] by Erik Brynjolfsson, Yu (Jeffrey) Hu, and Michael D. Smith, who first used a log-linear curve on an XY graph to describe the relationship between Amazon.com sales and sales ranking. They found that a large proportion of Amazon.com's book sales come from obscure books that were not available in brick-and-mortar stores.

They then quantified the potential value of the long tail to consumers. In an article published in 2003, these authors showed that, while most of the discussion about the value of the Internet to consumers has revolved around lower prices, consumer benefit (a.k.a. consumer surplus) from access to increased product variety in online book stores is ten times larger than their benefit from access to lower prices online. Thus, the primary value of the internet to consumers comes from releasing new sources of value by providing access to products in the long tail.

[edit] Goodbye Pareto Principle, the new distribution

In a 2006 working paper titled "Goodbye Pareto Principle, Hello Long Tail",^[9] Erik Brynjolfsson, Yu (Jeffrey) Hu, and Duncan Simester found that, by greatly lowering search costs, information technology in general and Internet markets in particular could substantially increase the collective share of hard-to-find products, thereby creating a longer tail in the distribution of sales.

They used a theoretical model to show how a reduction in search costs will affect the concentration in product sales. By analyzing data collected from a multi-channel retailing company, they showed empirical evidence that the Internet channel exhibits a significantly less concentrated sales distribution, when compared with traditional channels. An 80/20 rule fits the distribution of product sales in the catalog channel quite well, but in the Internet channel, this rule needs to be modified to a 72/28 rule in order to fit the distribution of product sales in that channel. The difference in the sales distribution is highly significant, even after controlling for consumer differences.

[edit] Economics and markets

[edit] Demand-side and supply-side drivers

The key supply-side factor that determines whether a sales distribution has a Long Tail is the cost of inventory storage and distribution. Where inventory storage and distribution costs are insignificant, it becomes economically viable to sell relatively unpopular products; however, when storage and distribution costs are high, only the most popular products can be sold. For example, a traditional movie rental store has limited shelf space, which it pays for in the form of building overhead; to maximize its profits, it must stock only the most popular movies to ensure that no shelf space is wasted. Because Netflix stocks movies in centralized warehouses, its storage costs are far lower and its distribution costs are the same for a popular or unpopular movie. Netflix is therefore able to build a viable business stocking a far wider range of movies than a traditional movie rental store. Those economics of storage and distribution then enable the advantageous use of the Long Tail: Netflix finds that in aggregate, "unpopular" movies are rented more than popular movies.

An MIT Sloan Management Review article titled "From Niches to Riches: Anatomy of the Long Tail"^[10] examined the Long Tail from both the supply side and the demand side and identifies several key drivers. On the supply side, the authors point out how e-tailers' expanded, centralized warehousing allows for more offerings, thus making it possible for them to cater to more varied tastes.

On the demand side, tools such as search engines, recommendation software, and sampling tools are allowing customers to find products outside their geographic area. The authors also look toward the future to discuss second-order, amplified effects of Long Tail, including the growth of markets serving smaller niches.

[edit] Crowds

The "crowds" of customers, users and small companies that inhabit the Long Tail distribution can perform collaborative and assignment work. Some relevant forms of these new production models are:

• The peer-to-peer collaboration groups that produce open-source software or create wikis such as wikipedia.
• The crowdsourcing model, in which a company outsources work to a large group of market players using a collaborative online platform.
• The model of crowdcasting, is the process of building a network of users and then delivering challenges or tasks to be solved with the purpose of gaining insights or innovative ideas.
• Work performed by individuals in commons-like, non-market networks, described in the work of Yochai Benkler.^[11]

[edit] Turnover within the long tail

The long tail distribution applies at a given point in time, but over time the relative popularity of the sales of the individual products will change.^[12] Although the distribution of sales may appear to be similar over time, the positions of the individual items within it will vary. For example, new items constantly enter most fashion markets. A recent fashion-based model ^[13] of consumer choice, which is capable of generating power law distributions of sales similar to those observed in practice,^[14] takes into account turnover in the relative sales of a given set of items, as well as innovation, in the sense that entirely new items become offered for sale.

There may be an optimal inventory size, given the balance between sales and the cost of keeping up with the turnover. An analysis based on this pure fashion model^[15] indicates that, even for digital retailers, the optimal inventory may in many cases be less than the millions of items that they can potentially offer. In other words, by proceeding further and further into the long tail, sales may become so small that the marginal cost of tracking them in rank order, even at a digital scale, might be optimised well before a million titles, and certainly before infinite titles. This model can provide further predictions into markets with long-tail distribution, such as the basis for a model for optimizing the number of each individual item ordered, given its current sales rank and the total number of different titles stocked.

[edit] Business models

[edit] Competitive impact

The Long Tail may threaten established businesses.^[16] Before a Long Tail works, only the most popular products are generally offered. When the cost of inventory storage and distribution fall, a wide range of products become available. This can, in turn, have the effect of reducing demand for the most popular products. For example, Web content businesses with broad coverage, such as Yahoo! or CNET, may be threatened by the rise of smaller Web sites that focus on niches of content, and cover that content better than the larger sites.

The competitive threat from these niche sites is reduced by the cost of establishing and maintaining them and the bother required for readers to track multiple small Web sites. These factors have been transformed by easy and cheap Web site software and the spread of RSS. Similarly, mass-market distributors like Blockbuster may be threatened by distributors like Netflix, which supply the titles that Blockbuster doesn't offer because they are not already very popular.

[edit] Internet companies

Some of the most successful Internet businesses have leveraged the Long Tail as part of their businesses. Examples include eBay (auctions), Yahoo! and Google (web search), Amazon (retail) and iTunes Store (music and podcasts) amongst the major companies, along with smaller Internet companies like Audible (audio books) and Netflix (video rental).

[edit] Video and multiplayer online games

The adoption of video games and massively multiplayer online role-playing games such as Second Life as tools for education and training is starting to show a long-tailed pattern. It costs significantly less to modify a game than it has been to create unique training applications, such as those for training in business, commercial flight, and military missions. This has led some to envision a time in which game-based training devices or simulations will be available for thousands of different job descriptions.

[edit] Microfinance and microcredit

The banking business has used internet technology to reach an ever increasing number of customers. But the most important shift in business model due to the Long Tail has come from the various forms of microfinance developed.

As opposed to e-tailers, micro-finance is a distinctly low technology business. Its aim is to offer very small credits to lower-middle to lower class and poor people, that would otherwise be ignored by the traditional banking business. The banks that have followed this strategy of selling services to the low-frequency long tail of the sector, have found out that it can be an important niche, long ignored by consumer banks.^[17] The recipients of small credits tend to be very good payers of loans, despite their non-existent credit history. They are also willing to pay higher interest rates than the standard bank or credit card customer. It also is a business model that fills an important developmental role in an economy.^[18]

Grameen Bank in Bangladesh has successfully followed this business model. In Mexico the banks Compartamos and Banco Azteca also service this customer demographic, with an emphasis on consumer credit. Kiva.org is an organization that provides micro credits to people worldwide, using a distinct direct business model.

[edit] User-driven innovation

According to the user-driven innovation model, companies can rely on users of their products and services to do a significant part of the innovation work. Users want products that are customized to their needs. They are willing to tell the manufacturer what they really want and how it should work. Companies can make use of a series of tools, like interactive and internet based technologies, to give their users a voice and to enable them to do innovation work that is useful to the company.

Given the diminishing cost of communication and information sharing (in analogy to the low cost of storage and distribution, in the case of e-tailers), long-tailed user driven innovation in will gain importance for businesses.

In following a long-tailed innovation strategy, the company is using the model to tap into a large group of users that are in the low-intensity area of the distribution. It is their collaboration and aggregated work that results in an innovation effort. Innovation communities formed by large groups of users can perform rapidly the trial and error process of innovation, share information, test and diffuse the results.

Eric von Hippel of MIT's Sloan School of Management defined the user-led innovation model in his book Democratizing Innovation.^[19] Among his conclusions is the insight that as innovation becomes more user-centered the information needs to flow freely, in a more democratic way, creating a "rich intellectual commons" and "attacking a major structure of the social division of labor".

[edit] Marketing

The drive to build a market and obtain revenue from the consumer demographic of the Long Tail has led businesses to implement a series of new marketing techniques, most of them based on extensive use of internet technologies. Among the most representative are:

• New Media Marketing: a term to describe the building and managing of social networks and online or virtual communities, and extend the reach of marketing to the low-frequency, low-intensity consumer in a cost effective way.
• Buzz Marketing: The strategic use of word of mouth, the transmission of commercial information from person to person in an online or real-world environment.
• Viral Marketing: The intentional spreading of marketing messages using preexisting social networks, with an emphasis of the casual, non-intentional and low cost.

[edit] Cultural and political impact

The Long Tail has possible implications for culture and politics. Where the opportunity cost of inventory storage and distribution is high, only the most popular products are sold. But where the Long Tail works, minority tastes become available and individuals are presented a wider array of choices. The long tail presents opportunities for various suppliers to introduce products in the niche category. These encourage the diversification of products. These niche products open opportunities for suppliers while concomitantly satisfying the demands of many individuals — therefore lengthening the tail portion of the Long Tail. In situations where popularity is currently determined by the lowest common denominator, a Long Tail model may lead to improvement in a society's level of culture. The opportunities that arise because of the Long Tail greatly affect society's cultures because suppliers have unlimited capabilities due to infinite storage and demands that were unable to be met prior to the Long Tail become realized.

[edit] Cultural diversity

Television is a good example of this: TV stations have a limited supply of profitable or "prime" time slots during which people who generate an income will watch TV. These people with money to spend are targeted by advertisers who pay for the programming so the opportunity cost of each time slot is high. Stations, therefore, choose programs that have a high probability to appeal to people in the profitable demographics in order to guarantee a return. Twin Peaks, for example, did not have broad appeal but stayed on the air for two seasons because it attracted young professionals with money to spend. Generally, as the number of TV stations grows or TV programming is distributed through other digital channels, the key demographic individuals are split into ever smaller groups. As the targeted groups become smaller niches, and the quantity of channels becomes less of an opportunity cost, previously ignored groups become profitable demographics in the long tail. These groups along the long tail then become targeted for television programming that might have niche appeal. As the opportunity cost goes down with more channels and smaller niches, the choice of TV programs grows and greater cultural diversity rises as long as there is money in it.

[edit] Distribution of independent content

Often presented as a phenomenon of interest primarily to mass market retailers and web-based businesses, the Long Tail also has implications for the producers of content, especially those whose products could not — for economic reasons — find a place in pre-Internet information distribution channels controlled by book publishers, record companies, movie studios, and television networks. Looked at from the producers' side, the Long Tail has made possible a flowering of creativity across all fields of human endeavour. One example of this is YouTube, where thousands of diverse videos — whose content, production value or lack of popularity make them inappropriate for traditional television — are easily accessible to a wide range of viewers.

[edit] Contemporary literature

The intersection of viral marketing, online communities and new technologies that operate within the Long Tail of consumers and business is described in the novel by William Gibson, Pattern Recognition.

[edit] Military applications and security

In military thinking, John Robb applies the Long Tail to the developments in insurgency and terrorist movements, showing how technology and networking allows the Long Tail of disgruntled groups and criminals to take on the nation state and have a chance to win.

[edit] Criticism

A 2008 study by Anita Elberse, professor of business administration at Harvard Business School, calls the Long Tail theory into question, citing sales data which shows that the Web magnifies the importance of blockbuster hits.^[20] On his blog, Anderson responded to the study, praising Elberse and the academic rigor with which she explores the issue but drawing a distinction between their respective interpretations of where the "head" and "tail" begin.^[21]

Also in 2008, a sales analysis of an unnamed UK digital music service by economist Will Page and high-tech entrepreneur Andrew Bud found that sales exhibited a log-normal distribution rather than a power law; they reported that 80 percent of the music tracks available sold no copies at all over a one-year period. Anderson responded by stating that the study's findings are difficult to assess without access to its data.^[22][23]

[edit] See also

• Black swan theory
• Mass customization
• Micropublishing
• Swarm intelligence

[edit] Notes

[edit] References

• Anderson, Chris (2006). The Long Tail: Why the Future of Business Is Selling Less of More. New York: Hyperion. ISBN 1-4013-0237-8. http://books.google.com/books?id=O2k0K1w_bJIC&printsec=frontcover. 
• The Long Tail a computer model by Fiona Maclachlan, The Wolfram Demonstrations Project.

[edit] External links

• "The Long Tail" by Chris Anderson, Wired, Oct. 2004
• "Consumer Surplus in the Digital Economy" by Erik Brynjolfsson, Yu (Jeffrey) Hu, and Michael D. Smith, November 2003
• "Goodbye Pareto Principle, Hello Long Tail" by Erik Brynjolfsson, Yu (Jeffrey) Hu, and Duncan Simester, December 2006
• "From Niches to Riches: Anatomy of the Long Tail" by Erik Brynjolfsson, Yu (Jeffrey) Hu, and Michael D. Smith, July 2006
• "Power Laws, Weblogs and Inequality" by Clay Shirky
• "Zipf, Power-laws, and Pareto - a ranking tutorial" by Lada A. Adamic
• The Rise and Fall of the Hit by Chris Anderson, Wired, Jul. 2006
• Interview with Russ Roberts, Library of Economics and Liberty
• "A practical model for analyzing long tails" by Kalevi Kilkki, First Monday, May 2007.
• Mexican microlending bank surges in market debut
• Asian Development Bank - Microfinance
• Businessweek, The Ugly Side of Microlending
• Eric von Hippel´s MIT Homepage
• Yochai Benkler, The Wealth of Networks
• netwiki Scientific wiki page dedicated to network theory