Patterns and Predictions, a predictive analytics company, has announced its Big Data initiative "The Durkheim Project*" - to predict suicide at scale using its machine learning data fabric built on technologies from Attivio and Cloudera.

Patterns and Predictions is now providing technology enabling opt-in participation from more than 100,000 U.S. veterans to build a 'big' medical database that will help military mental health experts fight a complex and challenging enemy: the alarming incidence of suicide among veterans.

Cited by a 2012 TIME Magazine cover story ("One A Day"), as a problem of epidemic proportions, suicide rates among veterans are roughly double those of adults in the general U.S. population. TIME also noted that military efforts have struggled to successfully address the suicide crisis due, in part, to a lack of understanding why nothing has worked to stop it.

Patterns and Predictions' founder Chris Poulin, has been working with Dartmouth researchers to address this problem since 2010. "The next generation of predictive analytics tools gives scientific and clinical investigators new hope and resources for solving even the most intractable problems," said Poulin.

In 2011, Patterns and Predictions engaged Big Data analytics experts Attivio and Cloudera and secured a contract with the Defense Advanced Research Projects Agency (DARPA). Concluded in February 2013, staff from Patterns and Predictions, Dartmouth, and the U.S. Veterans Administration conducted an investigation to validate the machine learning data fabric. Initial findings indicated the data model's predictive accuracy was statistically significant (consistent accuracies of 65 percent or more) in predicting suicidality risk among a veteran control group.

Having confirmed the theoretical underpinnings of this work, Patterns and Predictions is leading "Phase 2" technology for The Durkheim Project, a tightly integrated Big Data initiative with one objective: suicidality prediction at scale. "The promise of Durkheim lies in its ability to collect and monitor a diverse repository of complex data, with the hope of eventually providing a real-time triage of interventional actions upon detection of a critical event," said Poulin. At this time, the associated medical study (Dartmouth CPHS #23781) is authorized as observation only and non-interventional.

Tightly Integrated Technologies

Durkheim's event-driven predictions are made possible by Patterns and Predictions' integration of its suicide risk prediction technology (including a Facebook® app, and mobile app for iPhone® and Android®) and database connections with complementary capabilities from Attivio, the leading unified information access platform, and Cloudera, the category leader in Apache Hadoop-based software and services for the enterprise.

Attivio: Attivio provided code that enables the indexing and dynamic joining of diverse information on a massive scale to make it highly accessible. "Attivio Active Intelligence Engine® offers a schema-less index and patented JOIN technology that our projects have relied on time and again to power our cutting-edge text analytics," said Poulin.

Cloudera: Patterns and Predictions engaged Cloudera Professional Services to co-develop code in the area of real-time prediction on CDH (Cloudera's Distribution Including Apache Hadoop), the market-leading open source distribution of Hadoop and related projects. "With Cloudera's unique software expertise, we can focus on making The Durkheim Project's risk assessments faster across larger data sets," said Poulin.