Sunday, May 31, 2009

Testing Wolfram Alpha

Wolfram Alpha is Stephen Wolfram’s much hyped search engine project. We decided to give it a little test run to see how it can be of use in market research, and wanted to share our experiences with you. We are probably not the only ones in the market intelligence community that are curious about Alpha: http://www.wolframalpha.com

It is important to distinguish Wolfram Alpha from Google and other text-based search engines. From the hype one might almost be led to believe Alpha has artificial intelligence (AI) and can compute an answer to almost anything. It’s definitely not an AI, but uses a series of engineered scripts to provide some interesting results in select areas. The scripting is based on Wolfram’s Mathematica software, and numbers is what Alpha is good at.

Alpha is not a universal search engine. It does not return information on for example “EU regulations” or “Adam Smith’s invisible hand”, giving you the quickly annoying null result message “Wolfram Alpha isn't sure what to do with your input”. Alpha excels at numeric data and performing calculations, though, and it can conjure up useful tables and charts in instants, but only based on facts it has stored in its databases. In some instances, Alpha utilizes external sources too, typically for frequently updated data such as stocks.

For instance, we entered “DnB NOR”, which is the largest financial institution in Norway, and Alpha returned latest trade data and returns, a chart with price history, performance comparisons and correlations with various indices in table and various chart formats, projections for future pricing and alpha, beta and R Square values for daily returns for DnB NOR compared to S&P 500. Wow. Impressive, assuming the information is correct. To verify some of the data, we went to the Oslo Stock Exchange website and checked the latest trade value and weekly, monthly and year-to-date returns. The latest trade value was identical, but the return values were a bit off, as in Alpha reported an additional +5% on 30 day-return and -10% on year-to-date return. Hmm. That’s a lot.

This led us to check out the Source data link provided on Alpha’s search result page. It gave a long list of finance-related sources ranging from academic books to sites such as Morning Star, with a disclaimer that not necessarily all sources were used in the results. This can make quoting a source for particular data difficult, as the source list seems to apply for the search result page as a whole rather than for individual elements on the page. The Oslo Stock Exchange was not on the list, by the way, but several sources for aggregated financial data were.

We decided to try one of the recommended searches Alpha provides on its front page, and thus entered “EU populations.” Alpha returned a summary with the total EU population, the country with the highest population and the lowest, a chart with historical developments for the total EU population (does not say whether it includes ascension member states), a list of top 5 and bottom 5 countries ranked by population size – which can be expanded to a full list with a few clicks, a distribution plot with unspecified data (what does it show?), along with some demographic data measures for the countries that have the highest, lowest and median values for the population, population density, population growth, life expectancy and median age. This part didn’t strike us as intuitively very practically organized. A few clicks at a More option for yielded some additional demographic variables, such as annual deaths and birthrate fractions.

Of high importance, we were unable to find references to which years the various population data stem from – in fact there seems to be hardly any meta data available. We could be comparing years-old figures for some parameters and countries with very recent data for others without knowing. The source list doesn’t provide us much help in this case either. This is a major issue that needs to be addressed if Alpha is to be used by market intelligence professionals.

Summarizing briefly, Wolfram Alpha currently can produce substantial amounts of data on select subject matters of quantitative nature. For market researchers looking for quick access to on-the-fly generated charts and tables on data otherwise obtained and structured via other sources, Alpha could be a low-cost solution. However, seeing the source references are vague and Alpha seems to lack very important meta data, it seems more like a tool for a) the general public with simple data requirements, b) students and specialists in certain fields with a need to perform mathematical and statistical computations without using desktop applications.

We’ll definitely keep tabs on Alpha and play around with it some more, but currently it doesn’t strike us as something that will revolutionize the market intelligence industry. If anything, it is a supplement to other search engines and tools. As far as search engines go, Wolfram Alpha is certainly not a replacement for Google, but then again, it doesn’t intend to be either. We look forward to seeing Alpha evolve.