Why Classification?
Very large databases are becoming the norm in today’s world of big data. Imagine a database with terabytes of data—a terabyte is one trillion bytes of data. Facebook alone crunches 600 terabytes of new data every single day (as of 2014, the last time it reported these specs). The primary challenge of big data is how to make sense of it. And sheer volume is not the only problem: big data also tends to be diverse, unstructured and fast-changing. Consider audio and video data, social media posts, 3D data, or geospatial data. This kind of data is not easily categorized or organized. To meet this challenge, a range of automatic methods for extracting useful information has been developed, among them classification.
How Classification Works
An analyst’s goal is to create a set of classification rules that answer a question, make a decision, or predict behavior. To start, a set of training data is developed that contains a certain set of attributes as well as the likely outcome. The job of the classification algorithm is to discover how that set of attributes reaches its conclusion. Consider a credit-card company trying to determine which prospects should receive a credit card offer. The company’s training data might include: Obviously, this is a simple example, and the algorithm would need a far larger data sampling than the two records shown here. Further, the prediction rules are likely to be far more complex, including sub-rules to capture attribute details. Next, the algorithm is given a “prediction set” of data to analyze, but this set lacks the prediction attribute (or decision):
Day to Day Examples of Classification
Classification and other data-mining techniques are behind much of our day-to-day experience as consumers. Weather predictions use of classification techniques to report whether the day will be rainy, sunny, or cloudy. The medical profession analyzes health conditions to predict likely medical outcomes. A type of classification method, Naive Bayesian, uses conditional probability to categorize spam emails.
