Informal SIGIR Test of Time Award

I have the fortune of attending ICML this year and hope to report on that next week.   Like other conferences, ICML includes a Test of Time award “given to papers that time and hindsight proved to be of lasting value to the Machine Learning community.”  This year, the award went to `Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data‘ by John Lafferty, Andrew McCallum, and Fernando Pereira.

As an exercise, I scanned the list of titles from SIGIR 2001 and created a poll to see which papers readers would nominate for an informal SIGIR 2011 Test of Time award.  The poll can be found here.  Three votes per person.  Poll closes on July 24, 2011.

Update: Looks like I did not read the fine print of the polling site close enough and the polls will close on July 24, 2011 or when 100 votes have been received, whichever comes first.  Currently at 63 votes total.  I did specify “informal”, didn’t I?

Update: The poll closed over the weekend and the top three papers captured 50% of the votes,

  1. “Relevance based language models”, Victor Lavrenko, W. Bruce Croft (24.39%; citations: 252/ACM,618/G)
  2. “A statistical learning learning model of text classification for support vector machines”, Thorsten Joachims (15.85%; citations: 60/ACM,215/G)
  3. “A study of smoothing methods for language models applied to Ad Hoc information retrieval”, Chengxiang Zhai, John Lafferty (13.41%; citations: 281/ACM,709/G)

The best paper at SIGIR 2001 was “Temporal summaries of new topics”, James Allan, Rahul Gupta, Vikas Khandelwal (1.22%; citations: 43/ACM,132/G). I cannot find an easy to get the most cited paper at the conference.

SIGIR 2011 ACCEPTED PAPERS THREAD

Please visit Ian’s post on Not Relevant for pre-prints of SIGIR 2011 accepted papers.

“Economic Impact Assessment of NIST’s Text REtrieval Conference (TREC) Program”

Thanks to your feedback,

“…this study estimates that TREC’s existence was responsible for approximately one-third of an improvement of more than 200% in web search products that was observed between 1999 and 2009.”

More here.

SIGIR 2010 Best Paper Nominees

SIGIR has posted best paper nominees.

  • A comparison of general vs personalized affective models for the prediction of topical relevance, I. Arapakis, K. Athanasakos, J. Jose
  • Assessing the Scenic Route: Measuring the Value of Search Trails in Web Logs, R. White, J. Huang
  • Caching Search Engine Results over Incremental Indices, F. Junqueira, R. Blanco, E. Bortnikov, R. Lempel, L. Telloli, H. Zaragoza
  • Comparing the Sensitivity of Information Retrieval Metrics, F. Radlinski, N. Craswell
  • Extending Average Precision to Graded Relevance Judgments, S. Robertson, E. Kanoulas, E. Yilmaz
  • Information Based Model for ad hoc information retrieval, S. Clinchant, E. Gaussier
  • Multi-style language model for web scale information retrieval, K. Wang, J. Gao, X. Li
  • Properties of Optimally Weighted Data Fusion in CBMIR, P. Wilkins, A. Smeaton

TREC Survey

There is a survey being conducted about the impact of TREC on information retrieval research.  This feedback is important for organizers and I encourage researchers to participate.  If you are outside of the IR community and have ever used TREC collections, this is feedback is also valuable.

SIGIR 2009 ACCEPTED PAPERS THREAD

SIGIR Poster decisions have been mailed. Full paper decisions should be soon as well. Authors are encouraged to post preprints/drafts of accepted publications in the comments section.

PAPERS

The program committee reviewed 494 full paper submissions and accepted 78, about a 16% acceptance rate.

In 2008, 497 submitted, 85 accepted, about a 17% acceptance rate.

POSTERS

The program committee reviewed 256 poster submissions and accepted 86, about a 34% acceptance rate.

In 2008, 173 submitted, 91 accepted, about a 53% acceptance rate.

UPDATE

Accepted papers published.

SIGIR 2009 Information

Conference URLs:

Important Dates:

Jan 19, 2009 Abstracts for full research papers due
Jan 26, 2009 Full research paper submissions due
Feb 2, 2009 Workshop proposals due
Feb 23, 2009 Posters, demonstration, and tutorial proposals due
Mar 2, 2009 Doctoral consortium proposals due
Mar 9, 2009 Notification of workshop acceptances
Apr 11, 2009 All other acceptance notification

WSDM Accepted Papers Posted

WSDM 2009 accepted papers posted here.