There is a survey being conducted about the impact of TREC on information retrieval research. This feedback is important for organizers and I encourage researchers to participate. If you are outside of the IR community and have ever used TREC collections, this is feedback is also valuable.
SIGIR Poster decisions have been mailed. Full paper decisions should be soon as well. Authors are encouraged to post preprints/drafts of accepted publications in the comments section.
PAPERS
The program committee reviewed 494 full paper submissions and accepted 78, about a 16% acceptance rate.
In 2008, 497 submitted, 85 accepted, about a 17% acceptance rate.
POSTERS
The program committee reviewed 256 poster submissions and accepted 86, about a 34% acceptance rate.
In 2008, 173 submitted, 91 accepted, about a 53% acceptance rate.
UPDATE
Accepted papers published.
Conference URLs:
Important Dates:
| Jan 19, 2009 | Abstracts for full research papers due |
| Jan 26, 2009 | Full research paper submissions due |
| Feb 2, 2009 | Workshop proposals due |
| Feb 23, 2009 | Posters, demonstration, and tutorial proposals due |
| Mar 2, 2009 | Doctoral consortium proposals due |
| Mar 9, 2009 | Notification of workshop acceptances |
| Apr 11, 2009 | All other acceptance notification |
WSDM 2009 accepted papers posted here.
In 2006, I was studying information retrieval at the University of Massachusetts and, during a Friday of extreme impatience, I installed WordPress, started apached and created a blog called “Information Retrieval”. After a handful of posts over the course of six months, the comments queue filled with spam and WordPress stopped working. It is with this dubious evidence that I have been asked my esteemed colleagues to write the first post of “Probably Irrelevant”. The talent represented by those nominating me will ensure that “Probably Irrelevant” will see a little more life than “Information Retrieval” (if it has not already based on the title alone).
Now, it seems appropriate that the inaugural post of an information retrieval blog should address the subject of “blog search”. Unfortunately, I am dreadfully less qualified than my co-authors to discuss the state of the art. So, I apologize in advance for errors, omissions, or general ridiculousness and lay blame on Kevyn and Jonathan.
Now, when I started “Information Retrieval”, one of the first messages I received was from a senior member of the IR community. He wrote,
Maybe you could blog about why anyone is interested in blogs :-)
I replied,
I’ll keep this in mind when you’re chairing a session on blog search at SIGIR 2010.
I will not identify the original commenter but encourage conference attendees to pay attention in Geneva.
Of course, this comment deserves some thought. One of the issues with blog search is the under-defined taxonomy of queries. The TREC Blog Track defines the following tasks
One question I hope will be resolved in the comments is where these query types came from. Are they derived from actual blog searchers? Or are they merely contrived by the track organizers while trading pints at the Gaithersburg Marriot? These are questions, not criticisms. I think these are fine tasks but we have to be careful to define queries which are representative of those being issued blog search engines or, more generally, fulfill some desire users have. The problem with a new corpus is that how users interact with it is still not completely developed. What users will actually use these systems? Casual blog readers? Marketers? Political scientists? Sociologists?
The majority of time in an “Introduction to Information Retrieval” course is devoted to modeling documents. And, yes, we have sophisticated models of documents. We decompose individual documents using passages, sentences, or other exploitable structure. We also model the corpus as a whole either explicitly (e.g. cluster-based retrieval, latent semantic indexing, regularization) or implicitly (e.g. pseudo-relevance feedback).
For an information retrieval researcher, a corpus without queries is a corpse. Queries make information retrieval different from unsupervised learning. Also, because they are so short, queries make information retrieval different from traditional text classification. While information retrieval research has focused on ranking documents given a query, prior to the late 1990s, there were very few (published) results on modeling queries in aggregate. However, with the advent of web search engines, there has been a growing body of work on such models. These include descriptive studies of web query frequencies and user clicking behavior as well as models for query similarity and clicking behavior. These results have mainly been presented for web users and queries; I would be very interested in seeing whether the results generalize to non-web search scenarios.
To come back to blog search, I believe we need a better understanding of both the corpus and the queries before defining tasks. Blog corpora exist and are actively being studied. I am less certain about blog queries. One approach would be to inspect query logs to blog search engines for different retrieval scenarios and then improve performance for those scenarios. Of course, some of us are engineers who sometimes desire to build a tool because we believe it would be used. However, if there is a mismatch between what we believe will be useful and what users find useful, then we have wasted time.*
I’ve touched on a lot in this first post and hope it serves as a starting point of discussion. So, welcome to “Probably Irrelevant”.
*I just became aware of a paper to be presented at CIKM entitled “What Should Blog Search Look Like?” which I hope will answer some of these questions.
Editor’s Note: Many thanks to Fernando for authoring our first post. He couldn’t have chosen a more timely topic, the TREC 2008 Blog Track judgements are underway, Iadh Ounis as recently posted a call for suggestions for the 2009 tasks, Jeff Dalton has an insightful response, and Marti Hearst’s paper is now online.
