The New Jersey Institute of Technology's
Electronic Theses & Dissertations Project

Title:	New techniques for improving biological data quality through information integration
Author:	Herbert, Katherine Grace
View Online:	njit-etd2004-088 (xiii, 124 pages ~ 6.6 MB pdf)
Department:	Department of Computer Science
Degree:	Doctor of Philosophy
Program:	Computer Science
Document Type:	Dissertation
Advisory Committee:	Wang, Jason T. L. (Committee chair) Gehani, Narain (Committee member) McHugh, James A. (Committee member) Shih, Frank Y. (Committee member) Oria, Vincent (Committee member) Yin, Michael M. (Committee member)
Date:	2004-05
Keywords:	Data integration Data cleaning Bioinformatics Data mining Phylogeny Databases
Availability:	Unrestricted
Abstract:	As databases become more pervasive through the biological sciences, various data quality concerns are emerging. Biological databases tend to develop data quality issues regarding data legacy, data uniformity and data duplication. Due to the nature of this data, each of these problems is non-trivial and can cause many problems for the database. For biological data to be corrected and standardized, methods and frameworks must be developed to handle both structural and traditional data. The BIG-AJAX framework has been developed for solving these problems through both data cleaning and data integration. This framework exploits declarative data cleaning and exploratory data mining to help improve biological data quality. Within the framework, the problems and difficulties for cleaning biological data are discussed, specifically concerning phylogenetic data. Subsequently, it is demonstrated how BIG-AJAX can be used to improve the quality of the data in phylogeny. Moreover, within the cleaning architecture, data integration plays a key role in improving data quality. Data integration, due to the distributed and heterogeneous nature of the data sets becomes an enabling technology for improving data quality. From these concepts, two tools have been built, BIG-AJAX for TreeBASE and BIG-AJAX for Lineage Paths. Each system approaches the data quality problem using the conceptual BIG-AJAX framework while implementing different methods working within the framework to improve the data quality.