Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • 2011
  • Article
  • Emerging Health Threats Journal

Scalable Detection of Anomalous Patterns With Connectivity Constraints

By: Skyler Speakman, Edward McFowland III and Daniel B. Neill
  • Format:Electronic
ShareBar

Abstract

We present GraphScan, a novel method for detecting arbitrarily shaped connected clusters in graph or network data. Given a graph structure, data observed at each node, and a score function defining the anomalousness of a set of nodes, GraphScan can efficiently and exactly identify the most anomalous (highest-scoring) connected subgraph. Kulldorff’s spatial scan, which searches over circles consisting of a center location and its k − 1 nearest neighbors, has been extended to include connectivity constraints by FlexScan. However, FlexScan performs an exhaustive search over connected subsets and is computationally infeasible for k > 30. Alternatively, the upper level set (ULS) scan scales well to large graphs but is not guaranteed to find the highest-scoring subset. We demonstrate that GraphScan is able to scale to graphs an order of magnitude larger than FlexScan, while guaranteeing that the highest-scoring subgraph will be identified. We evaluate GraphScan, Kulldorff’s spatial scan (searching over circles) and ULS in two different settings of public health surveillance. The first examines detection power using simulated disease outbreaks injected into real-world Emergency Department data. GraphScan improved detection power by identifying connected, irregularly shaped spatial clusters while requiring less than 4.3 sec of computation time per day of data. The second scenario uses contaminant plumes spreading through a water distribution system to evaluate the spatial accuracy of the methods. GraphScan improved spatial accuracy using data generated from noisy, binary sensors in the network while requiring less than 0.22 sec of computation time per hour of data

Keywords

Biosurveillance; Event Detection; Graph Mining; Scan Statistics; Spatial Scan Statistic

Citation

Speakman, Skyler, Edward McFowland III, and Daniel B. Neill. "Scalable Detection of Anomalous Patterns With Connectivity Constraints." Emerging Health Threats Journal 4 (2011): 11121.

About The Author

Edward McFowland III

Technology and Operations Management
→More Publications

More from the Authors

    • October–December 2022
    • INFORMS Journal on Data Science

    Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem

    By: Mochen Yang, Edward McFowland III, Gordon Burtch and Gediminas Adomavicius
    • 2022
    • Journal of Computational and Graphical Statistics

    Nonparametric Subset Scanning for Detection of Heteroscedasticity

    By: Charles R. Doss and Edward McFowland III
    • Pattern Recognition Letters

    Pattern Detection in the Activation Space for Identifying Synthesized Content

    By: Celia Cintas, Skyler Speakman, Girmaw Abebe Tadesse, Victor Akinwande, Edward McFowland III and Komminist Weldemariam
More from the Authors
  • Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem By: Mochen Yang, Edward McFowland III, Gordon Burtch and Gediminas Adomavicius
  • Nonparametric Subset Scanning for Detection of Heteroscedasticity By: Charles R. Doss and Edward McFowland III
  • Pattern Detection in the Activation Space for Identifying Synthesized Content By: Celia Cintas, Skyler Speakman, Girmaw Abebe Tadesse, Victor Akinwande, Edward McFowland III and Komminist Weldemariam
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College