On the one hand, this is just a story of another researcher using a web game to obtain data – in this case, the ESP Game, which anonymously pairs two users, shows them the same images, and asks them to enter descriptive words until they get a match. But there’s both some good and some bad computer science that seems to be going on here.
On the good side, there’s definite value in finding ways to use people for data collection in places computers fall down, and this program seems engineered to do it well. Asking people to agree on terms is good, because it allows some confidence that they aren’t just based on a single view of the world, so I like that aspect of the system. But I’ve got my doubts about the quality of the data, and being a mainstream media article, that isn’t discussed here. Basically, I played the game a few times, and people soon learn that if there isn’t an obvious object in the picture, throwing out the names of the major colors will usually hit a match. Same with just typing in any words that appear in the picture. You can tell from the list of “taboo” words (these are, one assumes, words which have already been identified to be relevant to the image), that they vary in quality. Now, there seems to be some ranking based on how quickly the shared term was agreed upon, so that may help, but by making it a game you are asking for people to … well … game the system.
I also have my doubts about a comment from late in the article about future applications:
Blum suggested another problem that might be solved with this approach: Internet searches concerning mathematics. Different mathematicians use different symbols to represent the same variable in equations — what one labels X, another might label T. Humans can recognize that it’s the relationships of the variables that matters, not the labels, but computers can’t.
Using people to link words to images is clearly a good idea – there is a great deal of human intuition about the world involved in seeing what is important in a picture and giving it a name. Furthermore, this is something simple for people to do, because we must do it to navigate the world. The math problem, however, is difficult in a different way. First off, if you’re just talking about the case of knowing that “x^2 + 3x – 5” is the same equation under a different variable as “t^2 + 3t – 5”, this is not difficult for computers. Mathematics processing systems exist which can identify these and even significantly more complex relations where x in one equation is an entire expression in the other equation. This type of differentiation is essentially pattern matching, and what a computer lacks in intuition about real-world objects it makes up for in excellent pattern matching.
The problem I suspect being referred to (and it is possible that this was expressed, but the reporter simply didn’t understand the distinction), is coming up with the ability to type a mathematics equation into a search engine and have all of the occurences of that equation, up to equivalent variables/substitutions, come out the other side. But this is hard in part because of the difficulty of just finding the equations. Images are tagged as such on the web, but equations are sometimes rendered in ascii, sometimes in images, and sometimes as a combination. I suspect many of them are in pdf files viewable as webpages. Once the problem were broken down to the point that a user could be asked if two mathematical equations related or not, I believe a computer could check that as well.
Perhaps the real application would be a tool that presents a line of a webpage and asks if there’s an equation in it or not….