CSCW

Constraint can spur creativity. Approaching the 26th anniversary of the CSCW conference, we were asked to move the submission deadline two months earlier to reduce the reviewing conflict with CHI 2012. In response, we used the additional time to innovate and to increase participation while maintaining or increasing quality, by allowing authors to revise and resubmit their papers and notes. Conference attendees and proceedings readers will ultimately assess the outcome, but overall the feedback from associate chairs, reviewers and authors has been very positive.

Submissions increased 57% from CSCW 2011 to a record 290 papers and 125 notes. We increased the program committee to 67 associate chairs (ACs) to handle them; 21 were women and 20 were from outside North America. They comprised representatives from computer science, social media and social networks, psychology, anthropology, software engineering, management, and design. Fifty were from universities.

Our process most resembled a journal special issue, with a submission deadline, a team of four reviewers, and one revision cycle under a firm time constraint (in this case five weeks). Each submission was assigned to two committee members, designated AC-Coordinator and ACReviewer. Each AC assigned one external reviewer. In the first round, each paper or note was blind-reviewed by the AC-Reviewer and the two external reviewers. Reviewers assessed the likelihood that it could be revised within one month to be acceptable. Based on the outcomes, submissions were considered 'conditional accept' (CA, 8.7%), 'revise=resubmit' (RR, 46.5%) or 'reject' (44.8%). In general, papers that received one or more positive assessments were offered the opportunity to revise.

Papers that had no positive assessment in the first round were rejected. We based this decision on an analysis of CSCW 2011 data, which revealed that submissions that did not get an advocate in the initial reviewing had 0% chance of acceptance. Sixty percent of CSCW 2011 submissions had no advocate, compared to 45% of CSCW 2012 submissions. The difference is that the CSCW 2012 submissions had a month in which they could be improved. By immediately rejecting the 45% that had no chance, we streamlined and focused the process, and reduced the number of reviews per submission despite the revision cycle.

The AC Coordinator relied on the reviews in providing written guidance to authors on how to revise RRs. Authors of RR and CA submissions had five weeks to prepare a revision and an explanation of how they had addressed reviewer concerns. Resubmissions of RRs underwent a full second review from the same four-reviewer team; most CAs were checked by just one AC. Reviewing teams were strongly encouraged to resolve decisions prior to the program committee meeting and indeed, only 6% of initial submissions had to be discussed in the meeting.

The program committee split into two subgroups followed by a plenary session to make the final decisions. 127 papers and 37 notes, 39.5% of initial submissions, reached acceptable quality through revision. This is almost twice the number presented in any previous CSCW, in part due to record submissions. ACs and reviewers nominated papers and notes for consideration for Best Paper awards; 4 were selected, with 17 awarded Honorable Mention.

The accepted work represents diverse topics. Relatively new topics for CSCW include crowdsourcing, civic engagement, and social media; more traditional CSCW topics include distributed work, coordination, ethnography, and collaborative software development.

This collaborative effort was an example of computer supported cooperative work. The entire CSCW community provided input throughout the process-many ideas and changes came from outside of the committee. Committee members had to adapt to new ways of working. Our analysis indicates that the work required of reviewers and associate chairs did not exceed that of past years, and authors benefited from two rounds of feedback from committed ACs and reviewers.

Our goal was to maintain a high quality threshold while enabling more papers to reach that threshold. We believe there are many benefits to a review process with more opportunity for constructive interaction between reviewers and authors. Authors benefit by learning more about the CSCW culture, and by producing papers that are better than they could have produced on their own. Reviewers benefit by seeing their work lead to improved papers. The community benefits, because more of its members are successful at producing high quality research each year.

As a community, we must consider new ways to assess conference quality-when a larger volume of high quality work is produced, older metrics no longer hold. Considering the fraction of submitted work that is of low quality to be a measure of conference quality is backwards. We need new measures of process quality and the resulting quality of accepted work. Through continued innovation, CSCW will remain the top venue for leading edge research on collaboration and technology. As with any new process, mistakes were made and much was learned. We look forward to participating as community members in ongoing innovation.

The other technical tracks contribute immeasurably to the program. A record number of interactive posters will be presented. The 14 workshops drew a remarkable number of submissions. With demos, horizon, panels, videos, special sessions, and three plenary keynotes, CSCW 2012 is a great start to the second quarter-century of the CSCW conference.