The previous post talked about our failed attempt at using a decision tree to represent how people predicted the results of football matches. In this post, I describe how we actually went about solving the problem of finding a computational representation.
Let me reiterate a point that I've made many times. Interviews are the gold standard. Therefore, you start from the interviews. In our case, we went back to the interview transcripts and tried to see them not just as data with sets of predictions but as a representation of a process. The process in question was: given two teams, how do you reach a prediction about the match between the two given teams? It was fascinating how changing the lens helped us understand the process. While previously, we were looking at the interviews just as training data and test data, now we realized that the interviews basically documented the whole process followed, and they already contained the answer that we were looking for.
What immediately jumped out from the interviews was that the process did not always start at the same point (thus rejecting the idea of anything like a root node in a decision tree). Instead, based on different teams being considered, people started their thought process with a different feature. Further, thoughts about one feature led to another feature being considered. So, something like a spreading activation of thoughts. Finally, a lot of the thinking was comparative in nature. So, the two teams were being compared on the same feature for most of the time leading to the team with the stronger feature inhibiting the other team's chances of winning.
These observations pointed toward the following realizations: 1) Features form the core of the decision making process, 2) Features can link to other features, 3) Activation and inhibition are part of the process, and 4) There is no hierarchy or ordering of features to consider. With these realizations, we decided that a spreading activation network would be able to model the prediction process. Details about the spreading activation network in the next post.
Let me reiterate a point that I've made many times. Interviews are the gold standard. Therefore, you start from the interviews. In our case, we went back to the interview transcripts and tried to see them not just as data with sets of predictions but as a representation of a process. The process in question was: given two teams, how do you reach a prediction about the match between the two given teams? It was fascinating how changing the lens helped us understand the process. While previously, we were looking at the interviews just as training data and test data, now we realized that the interviews basically documented the whole process followed, and they already contained the answer that we were looking for.
What immediately jumped out from the interviews was that the process did not always start at the same point (thus rejecting the idea of anything like a root node in a decision tree). Instead, based on different teams being considered, people started their thought process with a different feature. Further, thoughts about one feature led to another feature being considered. So, something like a spreading activation of thoughts. Finally, a lot of the thinking was comparative in nature. So, the two teams were being compared on the same feature for most of the time leading to the team with the stronger feature inhibiting the other team's chances of winning.
These observations pointed toward the following realizations: 1) Features form the core of the decision making process, 2) Features can link to other features, 3) Activation and inhibition are part of the process, and 4) There is no hierarchy or ordering of features to consider. With these realizations, we decided that a spreading activation network would be able to model the prediction process. Details about the spreading activation network in the next post.
No comments:
Post a Comment