Jeffrey P. Bigham
@jeffbigham
This is a brief history of my CHI deadline days:
CHI 2008:
This was the first time that I submitted a paper to CHI. The paper I submitted was about a system that I built and studied called WebAnywhere. This was actually a resubmission from a failed ASSETS 2007 paper, which admittedly wasn’t ready at the time. I thought this paper was really great. I remember that a few days after I submitted the paper, I was asked to review it. Of course, I declined.
WebAnywhere would go on to be one of my most well-known projects. It led to a number of awards (MIT TR35, Microsoft Imagine Cup, etc.) and funding, and still to this day is responsible for much of my funding.
It was rejected soundly from CHI 2008. I think its highest score was a 2.
CHI 2009:
My second CHI submission was on a project that I did to make CAPTCHAs more accessible to blind users. I did the research for this paper on the weekends during a crazy summer in which I worked at both Benetech and IBM. I wrote this paper mostly on a train in England, where I was visiting for some sort of accessibility conference and also to run the Nottingham marathon.
This paper received either all 5s or maybe all 5s and a 4.5. I remember thinking it might get nominated for an award, but it didn’t. Just a few months ago, I received a call from a journalist asking me to comment on the accessibility of CAPTCHAs because of this paper.
I was unfortunately too sick to present this paper (end of a grueling job hunting season), and so my coauthor Anna Cavender was nice enough to do it for me. It was in the dreaded accessibility session at CHI, so basically nobody was there. This is one thing that has changed since my first CHI paper. Things have changed now, and the accessibility sessions are now pretty full.
CHI 2010:
I remember this CHI deadline. It was pretty crazy. I was at MIT, and I was trying to get a note ready for what would become VizWiz, but it was too early and that failed. I ended up submitting a paper on a system for recording accessibility problems that people have recorded “retro-actively” -- basically, a WebAnywhere user could reach an accessibility problem, push a button, and the system would record a script of all the steps they took to get there. It was rejected with 2s.
I ended up getting a note accepted with my frequent grad school collaborator Anna Cavendar on a system for collecting videos of signing rare technical terms called the ASL-STEM Forum. I remember my part of that paper was documenting how terms first appeared in the Urban Dictionary before they eventually made it into the Oxford English Dictionary. I thought that was pretty cool.
Anyway, the WebAnywhere recording paper was rejected from CHI but then accepted to ASSETS 2010. And, VizWiz ended up winning Best Paper at UIST that year. I remember feeling awful about my failure of a CHI deadline. But, then I went to John Harvard’s with the Boston HCI crew, which was fun.
CHI 2011:
CHI 2011 was not a great year… I had worked with some undergraduates to write a paper that I thought was pretty cool. It was about the adoption of assistive technology in high schools. We had conducted a focus group with some relevant people, but looking back the number of participants wasn’t nearly large enough for this paper to ever have had a chance. Still really useful.
That was my first year on the CHI Program Committee. I was in the Special Applications subcommittee. My paper was soundly rejected.
CHI 2012:
I remember at one point thinking we were going to have four submissions to CHI 2012. One of those was Scribe, a system for real-time captioning that I worked on with Walter Lasecki. In the end, none of them got submitted. I was not particularly happy.
Scribe went on to be submitted to UIST, where it received a Best Paper Honorable Mention. The night of the CHI deadline, I left for Accessibility Camp Toronto. I remember being pretty upset that none of the CHI projects had worked out.
CHI 2013:
CHI 2013 was a pretty easy one. Erin Brady, Merrie Morris, and I submitted what I think was a really interesting paper looking at what people asked VizWiz. Walter drove this other paper on what we ended up calling “TimeWarp” for making it easier for Scribe captionists to capture their parts of the speech. Both papers scored pretty well, and the TimeWarp paper received a Best Paper nomination. 100% acceptance rate!
CHI 2014:
This was a kind of crazy deadline. I had just moved to CMU, and everything was kind of in transition. I worked on a kind of crazy paper called Crowd Storage (storing information in the crowd) and Walter led a paper on having the crowd figure out dependencies in action in domains like activity recognition. Both papers received very low scores, but both miraculously got in … I still think Crowd Storage is one of the more interesting projects I’ve worked on lately, even if it’s fairly crazy, not all that useful in practice, and no one agrees with me…
CHI 2015:
This was the deadline of super dooper crazyness. I think we submitted 9 papers, and 6 of them were accepted. One of the papers was a resubmit (although I ended up personally reworking it substantially), and one paper was led by Gierad Laput (Zensors) so I didn’t have to do much. But, the other 7 (or so) I was actively coding, writing, making figures, etc., right up until 20 minutes after the deadline. Walter led 3 or 4 of these papers, and didn’t even have results or much of anything written for one of them until 3am the night before (go crowdsourcing!). Incidentally, that paper received a Best Paper Honorable Mention, and further cemented another story of the mythical Walter Lasecki. Another paper by Erin Brady received the highest scores I’d seen since 2009 (all 5s and a 4), and eventually received a Best Paper Honorable Mention.
As crazy as it was submitting 9 papers, it felt even crazier preparing their rebuttals all in the space of the ~5 days that we’re allowed to do so. Most of those rebuttals were written either in the car to Ohio for Thanksgiving, or in a mall (it has free Internet!) in a little rural town near where my parents live.
CHI 2016:
I think we submitted 4 papers. One with Merrie Morris has been mostly done for about a week. The others we worked on right until the deadline. Two were accepted, including a really fun paper about writing papers from smartwatches using crowdsourcing with Michael Nebeling.
CHI 2017:
This was the year of over-extending. I think we submitted 11 paper overall, and only 4 were accepted :/ Some of the accepted and rejected papers were very great, although I have to say I wouldn’t have predicted most of the 4 that were accepted. Already, two of the rejections have been resubmitted and accepted at DIS and ASSETS, respectively.
CHI 2018:
This is my first CHI post-tenure. I’ll end up submitting 6 papers, all of which I think are really cool, but I suspect as usual I won’t do a very good job at predicting which of the papers will be accepted. My daughter’s school had a Pittsburgh Pirates night tonight, and I long ago agreed to take her. So, it’s probably the first time in 10 years I did something fun on the night before CHI!
In the end, 2 of the papers we submitted were accepted with very high scores (like multiple 5s for each! Much wow!), and 4 of them were soundly rejected. As much as people complain about CHI reviews, this was more or less what I expected, and it was a good outcome. A 33% acceptance rate isn’t great, but it’s still better than the 23% or so of the overall conference.
CHI 2019:
I honestly don’t even remember what 4 papers were rejected last year. This year I think I’m (only) submitting 4 papers. Perhaps I’m truly getting old.
I had quite some fun trolling people with a study that we were running, about a paper that isn’t going to be submitted to CHI. Haha.
This page and contents are copyright Jeffrey P. Bigham except where noted.
Blog posts are not intended to be final products, but rather a reflection of current thinking and/or catalysts for discussion, like tweets but longer.
