Data Science: From Data to Knowledge

Introductory Example

Data Science is a collection of methods for discovering the knowledge hidden in data. As an example, let us imagine a company that has released a new product and does not yet know who its customers are. Unfortunately, marketing neglected to identify possible customer segments before the product launch. The company now wants to know who its customers are. To this end, it collected the following data in a supermarket:

Each row contains the data for one customer. Each column contains the observed data. For the sake of simplicity, very coarse “bins” are used here to classify the individual columns.

• Customers were categorized by age into YOUNG, MIDDLE, and RETIREE.
• MORNING, AFTERNOON, and EVENING were used as times of day.
• The filling of the shopping cart was recorded as NONE, LITTLE, MEDIUM, and FULL.
• The weather was simplified to RAIN, SUN, or CLOUDY.
• And finally, whether the new product was purchased or not: YES or NO.

These are only data from 13 purchases and 5 observed variables (the columns). In a data analysis, the first question is which unobserved variables might also exist. What data could improve the analysis? Do other factors play into the purchase decision? In reality, this is very likely. What about the customer’s clothing? Their hairstyle? If the new product is hairspray, the hairstyle is certainly an important feature. What about the weather? If the product is ice cream, the weather is certainly important as well.

One reason for the corporate “data hoarding” often criticized in the media is simply that, before the analysis, one does not know which features are important. So companies collect everything in reserve, according to the motto “it might still be useful.” From the perspective of information theory, this also makes sense, because reducing uncertainty requires information.

The data in our example are already in good condition. They are all consistent. In reality, as discussed in Chapter 6.4, data first have to be cleaned and standardized. Otherwise, the rule in data analysis is “Garbage in, Garbage out”.

An initial analysis of data is usually done with statistics and visualizations. Excel’s pivot tables and charts have become very widespread for this purpose. When working with new data, it is useful first to become familiar with them by looking at the usual statistical indicators.

Classification and Regression

Since the 1990s, work has been done on automatically extracting knowledge from data. The field used to be called “Knowledge Discovery in Databases” (KDD), was then often referred to as “Data Mining”, and is now called “Data Science”.

In the example above, the problem is to find a connection between sales and the other data in order to answer the question “Who are our customers?” The goal is to predict the “Product Purchased” column from the values in the other columns. So-called decision trees are suitable for this. Software can create these trees optimally from the sample data according to information-theoretic criteria. The following decision tree, for example, predicts the results of the “Product Purchased” column using the “Shopping Cart” and “Time of Day” columns.

This tree is applied to every record. Such a tree is read from top to bottom. The elongated nodes containing a question mark are decisions; the nodes without successors are results. The tree above corresponds to the following logic: “If the value in the ‘Shopping Cart’ column is ‘NONE’, then ‘No’; if it is ‘MEDIUM’, then also ‘NO’; if it is ‘LITTLE’, then check the ‘Time of Day’ column, and so on.” A decision tree corresponds to nested IF-THEN-ELSE rules. The tree above is, for example, equivalent to:

IF (Shopping Cart = LITTLE) THEN
    IF (Time of Day = AFTERNOON) THEN Yes
    ELSE No
ELSE
    IF (Shopping Cart = FULL) THEN Yes
    ELSE No

Such a decision tree contains the “knowledge” about the “Product Purchased” column. The company now knows that the product is bought by customers with full shopping carts or in the afternoon. The company also knows that “Age” and “Weather” are not needed. Data have become knowledge stored in a data structure, in a model.

In this example, the value of a binary variable, “Product Purchased,” was predicted. Such an assignment to discrete symbols is called classification. Other examples of classification include [CM16]:

• Identifying the bird species from a bird’s song
• Detecting malignant tumors and diseases

If the “Product Purchased” column contained not YES and NO but the number of products purchased, it would no longer be classification, but regression. Regression uses techniques other than decision trees. Since numbers have to be predicted, the methods are mostly mathematical and have their roots in mathematical statistics. The simplest regression is linear regression, which can be explained very intuitively. The following figure shows a set of points and the so-called “regression line.”

Colloquially speaking, the line is as close as possible to all points. To describe this procedure more precisely, however, one would have to become much more mathematical. That would go beyond the scope of this book. It is enough to know that regression is a mathematical-statistical procedure for bringing various data points “onto a line.” As a rule of thumb, the points all lie on the line. One could now store only the line instead of the individual points and, if necessary, calculate the points from the line.

In mathematics, lines have the advantage that they are relatively easy to calculate with. Their disadvantage is that they approximate very imprecisely. If a regression is performed with a third-degree polynomial, the points can be fitted much more closely, as the next figure shows:

But the mathematical formulas become more difficult. There is a tradeoff between accuracy and mathematical complexity. A line can be described with ax + b, but a third-degree polynomial requires ax^3 + bx^2 + cx + d. It is much easier to calculate with linear equations. Polynomial regression belongs to the so-called nonlinear regressions.

Regression has already been successfully applied in many areas, such as [CM16]:

• Finding new diamond mines based on data from the environments of previous mines.
• Which parts of a car are likely to fail next?
• How much money will a specific person spend next month?

The neural networks used in image recognition and self-driving cars are data structures that can learn nonlinear regressions from examples. We will explain neural networks in more detail later in Section 8.5.

Data Mining

The Data Mining Process

In the example above, with the decision tree for supermarket purchases, a working model was found on the first run. In reality, unfortunately, things are often quite different, because there is usually much more data and the data are often “noisy” and contain errors. Often the right data are not available yet and have to be obtained first. The various classification methods have many parameters and weights that have to be set correctly in order to obtain good results. Decision trees, for example, can be “pruned” if they become too large.

Creating good models with data mining is a lengthy and labor-intensive process consisting of the following steps [PF13]:

1. Analysis of the business situation and existing data: What is to be achieved? What is the goal? Is the data good enough? (“business and data understanding”)
2. Preparation of the data (“data preparation”): Cleaning, unification as in a data warehouse, and conversion into the “correct” format.
3. Model creation: The actual “data mining”
4. Evaluation and testing: Is the model good enough?
5. Deployment of the model: Analysis or prediction

These steps do not always have to be worked through in sequence. Often it only becomes clear after step 3 that more data are actually needed, and one has to return to step 1.

During testing, the model that has been found is checked against new data. What would the tree do with the following record?

It would say ‘NO’ because the shopping cart is ‘NONE’. But this would produce a wrong answer. The sample data used to train the tree did not contain this case. It is therefore important that the data used for training contain as many combinations as possible. In addition, different data are needed for training and testing.

Suppose the company knows the supermarket’s customers and wants to use the decision tree to predict how many products it has to deliver to the supermarket. It uses the tree for prediction.

If a classification returns a wrong result, there are two different possibilities:

If the model says ‘NO’ but the customer buys, the prediction is “false negative”. If the model says ‘YES’ but the customer does not buy, the prediction is “false positive”.

Both cases cause trouble for the company, but with very different consequences. Here, an error is not just an error. In a false positive, one product too many was delivered to the supermarket, but revenue will be lower than expected. In a false negative, there is one product too few in the supermarket’s warehouse. Overall, more could have been sold.

In other scenarios, the costs of the two classification errors are even higher. Imagine a model designed to detect debit-card fraud. The system examines every record and can either sound an alarm or not.

The false positives are now false alarms. Normal operations are interrupted, the police are called, and so on. All of this incurs costs. The false negatives are undetected cases of fraud, which can also cause immense damage.

The error rate of a classifier is the number of incorrectly classified records. In general, the goal is to make this error rate as low as possible. If incorrect classifications cause higher costs, the models must of course be tested and refined for a long time.

Important: Data mining is an iterative process in which the error rate is gradually improved.

Creating a working model for prediction is therefore anything but simple. It is a time- and labor-intensive task requiring a great deal of knowledge about the application domain and the methods of data mining. Another difficulty is that learning is easiest when feedback is received as often as possible. Take weather forecasting as an example. A forecast can be made every day, and the model can be checked every day. The error rate can be improved daily. Retail revenue during the Christmas season, by contrast, can be predicted and checked only once a year. Here it will probably take decades before the model has a good error rate [TG15].

The media often give the impression that, with Big Data and Artificial Intelligence, computers suddenly know everything about people and can monitor them. This knowledge, however, has to be extracted from the raw data for each specific question through a laborious data-mining process. It is not the case that one simply shows the data to the computer and “insights and wisdom fall from the sky.” Data-mining programs are a means of handling large amounts of data. They are based on statistical and information-theoretic algorithms.

Important: The intelligence lies in the data scientists who apply the methods intelligently to selected data.

Other Techniques

The two procedures discussed, classification and regression, are “supervised learning” methods. It is called “supervised” learning because the algorithm is told what it is supposed to learn. It is learning from an example [PF13]. In the example above, the target was the “Product Purchased” column. In addition to decision trees, there are many other techniques for classification and regression, such as neural networks, Bayesian networks, and support vector machines [PF13, SB14].

In “unsupervised learning” methods, no target to be learned is specified. These methods therefore serve more as tools for analysis [PF13].

In cluster analysis, the attempt is made to divide the records into k different groups. The records within a group should be as similar as possible. Cluster analysis is used, for example, to divide customers into customer groups.

In profiling, a model of the behavior of users or customers is created to predict their future behavior or to identify deviant behavior. A typical use case is an Intrusion Detection System (IDS) that creates a profile for each user. If a hacker breaks into the computer and behaves differently than the user, the IDS notices atypical behavior and can sound an alarm.

In market basket analysis, the attempt is made to find out which products are bought together. This also forms the basis for recommender systems following the motto “Customers who bought this product were also interested in…”.

Theory of Machine Learning

Behind all these learning methods stands a sophisticated mathematical theory, “computational learning theory”, or the theory of machine learning [SB14]. It examines, for example, which concepts can be learned at all by which methods, how many examples are required, and what error rate can be achieved at best. It is a highly mathematical theory between statistics and computer science. One result of the theory is that not every concept can simply be learned in order to make error-free predictions. The situation is much more complicated. As a rule of thumb, the simpler the concept to be learned and the more extensive the data, the better the prediction.

Important: Learning has information-theoretical limits. Complex systems remain complex and unpredictable even for machine learning methods.

Text Mining

Processing texts in natural language requires different techniques from traditional data analysis, because texts belong to the “unstructured” data. Of course, natural languages have structure, such as syntax and grammar rules, but this structure is not on the semantic level, the level of meaning. A computer cannot (yet?) understand the meaning behind language. But it can process language on the basis of syntax and rules, and partial successes have already been achieved in many areas [CM16, PF13].

In text classification, for example, emails are divided into categories based on keywords and their frequencies. This is used in customer support at large companies to forward complaints to the right department. In sentiment analysis, the attempt is made to determine the author’s mood or attitude. Text mining can detect forgeries and plagiarism, and it can also write standardized reports, for example on stock prices.

Most tools in the field of text mining are “ad hoc”. They are not based on linguistic theories, but “trick” their way into apparent understanding. Today, text mining is a mixture of linguistics, computer science, statistics, and machine learning.

More on natural language processing follows later in Section 8.2.

Software

There are very good software packages for trying out Data Science and Data Mining:

• Weka 3 is open source and written in Java, see http://www.cs.waikato.ac.nz/ml/weka/
• Orange is open source and uses Python, http://orange.biolab.si/
• RapidMiner Studio is available as a Community Edition, https://rapidminer.com

Data-Intensive Science

The new possibilities opened up by Big Data and Data Science have led to data-intensive science. In broad terms, the sciences have passed through the following stages (following [HTT09]):

1. Experimental
2. Theoretical
3. Computational
4. Data-Intensive

About 1000 years ago, science was “experimental”. Natural phenomena were observed and reproduced in experiments. About 400 years ago, science became “theoretical”, and people began forming theories about the observed phenomena, creating models, abstractions, and generalizations. Examples include Newton’s laws and Maxwell’s equations. Since the 1960s, science has been “computational”: theoretical models were calculated and checked with computers. Simulations began, leading among other things to agent-based modeling. Since around 2010, there has been “data-intensive” science. Data are collected during experiments or simulations and then processed with Big Data and analyzed with Data Science.

Important: Big Data and Data Science are a means of reducing uncertainty so that organizations can make better decisions.

This applies to all organizations: companies, science, environmental organizations, and state institutions alike.