Comments on: Search terms and the flu: preferring complex models http://www.deaneckles.com/blog/233_search-terms-and-the-flu-preferring-complex-models/?utm_source=rss&utm_medium=rss&utm_campaign=search-terms-and-the-flu-preferring-complex-models Dean Eckles blogs on people and technology Wed, 28 Apr 2010 05:13:23 -0700 http://wordpress.org/?v=2.8.6 hourly 1 By: Not just predicting the present, but the future: Twitter and upcoming movies | Ready-to-hand http://www.deaneckles.com/blog/233_search-terms-and-the-flu-preferring-complex-models/comment-page-1/#comment-12550 Not just predicting the present, but the future: Twitter and upcoming movies | Ready-to-hand Fri, 02 Apr 2010 19:02:08 +0000 http://www.deaneckles.com/blog/?p=233#comment-12550 [...] Search queries have been used recently to “predict the present“, as Hal Varian has called it. Now some initial use of Twitter chatter to predict the future: The chatter in Twitter can accurately predict the box-office revenues of upcoming movies weeks before they are released. In fact, Tweets can predict the performance of films better than market-based predictions, such as Hollywood Stock Exchange, which have been the best predictors to date. (Kevin Kelley) [...] [...] Search queries have been used recently to “predict the present“, as Hal Varian has called it. Now some initial use of Twitter chatter to predict the future: The chatter in Twitter can accurately predict the box-office revenues of upcoming movies weeks before they are released. In fact, Tweets can predict the performance of films better than market-based predictions, such as Hollywood Stock Exchange, which have been the best predictors to date. (Kevin Kelley) [...]

]]> By: Dean Eckles http://www.deaneckles.com/blog/233_search-terms-and-the-flu-preferring-complex-models/comment-page-1/#comment-9937 Dean Eckles Fri, 11 Dec 2009 23:55:08 +0000 http://www.deaneckles.com/blog/?p=233#comment-9937 Your comment strikes me as the more intelligent, personalized future of blog comment spam -- only you should have linked to your fan site, not Google Image Search ;-) I think it is intuitive. And model averaging has been a significant research topic in Bayesian statistics. Part of the challenge comes in other constraints and roles for our models besides prediction in a very narrow domain. We fit models to understand and explain phenomena. There is also the sense that larger more complex models may be overfit -- even if one does cross-validation -- to the specific context of this system, study, etc. So the balance can have to do with what one want to predict: do you want to generalize to just another case (another user on Netflix) or something a bit farther away (film preferences not expressed through a rental service). And these combined models can have computational disadvantages. The Netflix models would not be dropped into a system as-is. Your comment strikes me as the more intelligent, personalized future of blog comment spam — only you should have linked to your fan site, not Google Image Search ;-)

I think it is intuitive. And model averaging has been a significant research topic in Bayesian statistics. Part of the challenge comes in other constraints and roles for our models besides prediction in a very narrow domain.

We fit models to understand and explain phenomena. There is also the sense that larger more complex models may be overfit — even if one does cross-validation — to the specific context of this system, study, etc. So the balance can have to do with what one want to predict: do you want to generalize to just another case (another user on Netflix) or something a bit farther away (film preferences not expressed through a rental service).

And these combined models can have computational disadvantages. The Netflix models would not be dropped into a system as-is.

]]> By: Streeter http://www.deaneckles.com/blog/233_search-terms-and-the-flu-preferring-complex-models/comment-page-1/#comment-9936 Streeter Fri, 11 Dec 2009 22:50:32 +0000 http://www.deaneckles.com/blog/?p=233#comment-9936 For me, this seem intuitive; crowd-source models. Then by combining them, one ends up taking the best predictors from each to create a super model (not be confused with the other <a href="http://images.google.com/images?q=adriana+lima" rel="nofollow">supermodel</a>). For me, this seem intuitive; crowd-source models. Then by combining them, one ends up taking the best predictors from each to create a super model (not be confused with the other supermodel).

]]>