Causal Testing and Discrimination Testing
Yuriy Brun, Ph.D. and Alexandra Meliou, Ph.D.
With the ubiquity of software-driven products and services, it is critical that decisions made by software, such as loan approvals and employee screening, be fair and nondiscriminatory. However, current methods and software packages for testing for discrimination in software are both difficult to implement and ineffective at detecting causal discrimination. To better address discrimination detection in software, Professors Yuriy Brun and Alexandra Meliou from the College of Information and Computer Science have developed a technique and its software implementation called Themis. To use Themis, the user provides: the software to be studied for discrimination, a desired confidence level and error bound, and a schema giving the format for valid inputs. Themis will then generate a test suite that will test the software for group and causal discrimination and provide scoring for each. Themis has been tested on 20 software packages and was shown to be effective at detecting discrimination and was able to detect instances of discrimination in software designed to be anti-discriminatory. More broadly, Themis may be applied to measure causal relationships between inputs and outputs in software.
• Detects discrimination where existing methods have failed to do so • First time causal discrimination has been able to be measured • Fully automated whereas existing methods are manually implemented • Highly efficient - Minimizes size of test suite required to determine discrimination through optimizations: test caching, adaptive, confidence driven sampling, and sound pruning
• Finance • Advertising • Data management systems • Employment screening • Criminal justice
Professor Brun's research focuses on making it easier to build and deploy software systems. His research is centered around automation and software behavior. He develops techniques that automatically enforce behavior on systems, automatically mine behavioral models of software to help developers understand system behavior, and automatically repair systems to satisfy the behavioral requirements imposed on them. He works closely with developers to understand the challenges they face and to build tools to help them. He works closely with systems to understand where they go wrong and how to automate preventing that from happening. The long-term goal of this research is self-adaptive systems that self-monitor, self-manage, and self-correct their own behavior to achieve high-level goals in dynamic, constrained environments. Professor Brun's research is multidisciplinary, often combining advances in distributed systems, information theory, theoretical computer science, security, and machine learning. The research is highly collaborative and involves open-source development. Alexandra Meliou is an Assistant Professor in the College of Information and Computer Science, at the University of Massachusetts, Amherst. She has held this position since September 2012. Prior to that, she was a Post-Doctoral Research Associate at the University of Washington, working with Dan Suciu. Alexandra received her PhD and MS degrees from the Electrical Engineering and Computer Sciences Department at the University of California, Berkeley, in 2009 and 2005 respectively. She is the recipient of an ACM SIGMOD Research Highlight Award, an ACM SIGSOFT Distinguished Paper Award, an NSF CAREER Award, a Google Faculty Research Award, and a Siebel Scholarship. Her research interests are in the area of data and information management, with a current emphasis on provenance, causality, and reverse data management.
Available for Licensing and/or Sponsored Research
UMA 18-014
Patent Pending
|
