Digital Dark Matter and the Economic Contribution of Apache

Shane  Greenstein; Frank  Nagle

Article | Research Policy | May 2014

Digital Dark Matter and the Economic Contribution of Apache

Abstract

Researchers have long hypothesized that research outputs from government, university, and private company R&D contribute to economic growth, but these contributions may be difficult to measure when they take a non-pecuniary form. The growth of networking devices and the Internet in the 1990s and 2000s magnified these challenges, as illustrated by the deployment of the descendent of the NCSA HTTPd server, otherwise known as Apache. This study asks whether this experience could produce measurement issues in standard productivity analysis, specifically, omission and attribution issues, and, if so, whether the magnitude is large enough to matter. The study develops and analyzes a novel data set consisting of a 1% sample of all outward-facing web servers used in the United States. We find that use of Apache potentially accounts for a mismeasurement of somewhere between $2 billion and $12 billion, which equates to between 1.3% and 8.7% of the stock of prepackaged software in private fixed investment in the United States and a very high rate of return to the original federal investment in the Internet. We argue that these findings point to a large potential undercounting of the rate of return from IT spillovers from the invention of the Internet. The findings also suggest a large potential undercounting of "digital dark matter" in general.

Keywords: open source; Apache; Economic measurement; Digital economics; Measurement and Metrics; Open Source Distribution; Internet; Information Technology; Software; Economic Growth; Research and Development; Web Services Industry; Information Technology Industry; United States;

Format: Print

Find at Harvard Register to Read Purchase

Citation:

Greenstein, Shane, and Frank Nagle. "Digital Dark Matter and the Economic Contribution of Apache." Research Policy 43, no. 4 (May 2014): 623–631. (Lead Article.)

Knowledge work is becoming increasingly challenging as pace of change in the knowledge frontier is increasing. Organizations have created multiple mechanisms to minimize knowledge gaps and increase learning such internal training, mentorship programs as well as encouraging their professionals to exchange knowledge outside the boundaries of the organization. Reaching outside the organizational boundaries for knowledge exchange has changed dramatically in the last decade with the rise of online professionals communities. In the case of computer science and software engineering it has become a norm to learn and as questions that used to be kept within the boundaries of the organizations, externally, with a community of external professionals help answer and learn. It is less clear whether contributing to such online communities helps improve professionals’ knowledge work or becomes a competing or distracting force from their work place knowledge work. This is the empiric puzzle our study has set to explore. We conducted a mixed method study combining qualitative data from more than 100 interviews with software professionals with a unique quantitative dataset measuring software professionals’ code productivity in their workplace by the hour/every week for one to two years.

Case | HBS Case Collection | August 2019

Twiggle: E-commerce with Semantic Search

Shane Greenstein and Danielle Golan

Four years after being founded, in 2014, by former Google executives Amir Konigsberg (CEO) and Adi Avidor (CTO), Twiggle had developed a search enhancement that plugged into an online merchant’s existing framework. The company utilized advanced structuring and linguistic tools to build search technology that understood shopper intent and matched it with the right products. Konigsberg and Avidor believed e-tailers were losing enormous revenue due to poor search results. The team built a set of proprietary tools that created a human-like understanding of customer queries in the e-commerce search experience, using natural language processing that translated text into meaning. The team also constructed a semantic model of the e-commerce world for three product domains: fashion, home and electronics. The resulting “ontology” was a set of concepts and categories in a subject area that showed their properties and the relations between them. Twiggle’s tools could also unlock significant online sales revenue by increasing click-through and add-to-cart rates, ultimately improving sales conversions. So far, the company had secured deals with more than half a dozen large e-commerce retailers, and could improve search results in three product categories: fashion, home, and electronics.
The company initially targeted large direct-to-consumer e-commerce players. Yet the cofounders encountered challenges pursuing this type of customer. The customer acquisition process included lengthy, intensive face-to-face sales cycles, expensive and technologically complex proof-of-concept testing, and requests for customization. Konigsberg and Avidor wondered if they should expand their customer focus to target a wider set of smaller e-tailers that still had significant volumes of online search queries, but whose search quality tended to be less robust and might be easier to improve. They also contemplated a variety of growth opportunities.

Keywords: e-commerce; search technology; customer acquisition; Online Technology; Technological Innovation; Commercialization; Growth and Development Strategy; Technology Industry; Israel;

Citation:

Greenstein, Shane, and Danielle Golan. "Twiggle: E-commerce with Semantic Search." Harvard Business School Case 620-025, August 2019. View Details

Cite View Details Educators

Teaching note is meant to accompany Zebra Medical Vision case, which offers a look at a company’s decisions as a small startup competing with other startups and major technology companies. It also demonstrates the challenges faced by a machine learning company working on brand-new technology, including difficulties finding training data, hiring competition, and negotiating FDA approvals and other government regulations. Teaching Note for HBS No. 619-014.

Faculty & Research

Digital Dark Matter and the Economic Contribution of Apache

Abstract

Citation:

About the Authors

More from these Authors