DATA EXPLORATION, CLEANING, AND INTEGRATION FOR DATA SCIENCE

COMPSCI 774

Course Description

Big Data is often said to deal with four Vs: volume, velocity, variety, and veracity. The focus is on variety and veracity challenges, which often arise in data science projects. In many such projects, data is often incorrect, hard to understand, and come from a variety of sources. Data scientists often spend 80% of their effort to explore, clean, and integrate this data, before analysis can be carried out to extract insights. As a result, managing variety and veracity has received significant attention. Study these topics, understand their challenges, and discuss solutions. These solutions often require data management, machine learning, big data scaling, cloud, crowdsourcing, and user interaction techniques. Knowledge of machine learning/AI [COMP SCI 540], databases [COMP SCI 564] and Python [COMP SCI 320] recommended.

Prerequisties

Graduate/professional standing

Satisfies

This course does not satisfy any prerequisites.

Credits

Not Reported

Offered

Not Reported

Grade Point Average

Not Reported

Could not calculate change

Completion Rate

Not Reported

Could not calculate change

A Rate

Not Reported

Could not calculate change

Class Size

Not Reported

No change from Historical

No data available

Instructors (2025 Fall)

Sorted by ratings from Rate My Professors

Similar Courses

FOUNDATIONS OF DATA MANAGEMENT

COMPSCI 784

Foundational concepts in databases and data management. The first part of the course discusses topics on query languages (conjunctive queries, Datalog), their expressivity and complexity of evaluation. The second part studies advanced topics in modern data management, including data streams, massive parallelism, provenance, uncertain data management and privacy. There are no specific course prerequisites. It is strongly encouraged that the students are familiar with databases and relational algebra (COMP SCI 564or equivalent). Knowledge of algorithms, complexity theory and probability will also be helpful.

BIG DATA SYSTEMS

COMPSCI 744

Issues in the design and implementation of big data processing systems, including: an overview of cluster architecture, key design goals (flexibility, performance and fault tolerance), popular execution frameworks, basic abstractions, and applications (e.g., batch analytics, stream processing, graph processing, and machine learning).

THEORY & ALGORITHMS FOR DATA SCIENCE

COMPSCI 541

Theoretical methods for data science. Topics include: review of probability background, concentration inequalities, geometry of high dimensional random variables, parametric and non-parametric estimation, selected topics from optimization (optimality conditions; deterministic and stochastic gradient descent), PAC learning, sample complexity and algorithms for linear classification and regression, and property/distribution testing. Uses Python programming language.

DATA SCIENCE PROGRAMMING I FOR RESEARCH

COMPSCI 319

Introduction to Data Science programming using Python. In addition to a survey of programming basics (control flow and data structures), web scraping, database queries, and tabular analysis will be introduced. Projects will emphasize analyzing real datasets in a variety of forms and visual communication using plotting tools. Similar toCOMP SCI 220but the pedagogical style of the projects will be adapted to graduate students in fields other than computer science and data science. No previous programming experience required.

INTRODUCTION TO BIG DATA SYSTEMS

COMPSCI 544

Deploy and use distributed systems to store and analyze large datasets. Unstructured and structured approaches to storage will be covered. Analysis will involve learning new query languages, processing streaming data, and training machine learning models. Most programming will be done in Python.