Are the Peers Peers?

policy research. My experience from inside this peer review process fits precisely with the authors' external observations. Most section members were behavioral researchers, with ex- pertise in studying and modifying tobacco use behavior. This area of study and intervention constitutes less than half of current research and practice in tobacco control: the larger remainder includes studies of relevant tobacco industry and government behavior, innovative interventions, and the development and analysis of comprehensive government policies. To appreciate the range of concerns and methodologies reflected in state\x=req-\ of-the-art tobacco control work, one need only peruse any issue of Tobacco Control: An International Journal (including the special supplement on tobacco control research), any of the fre¬ quent issues of JAMA or of the American Journal of Public Health devoted to tobacco control, or the proceedings of any of the recent world conferences on tobacco and health. Though all of the Behavioral and Public Health Research Study Section members were thoughtful scholars learned in their fields, they were simply not the professional peers of most scholars working in the tobacco control arena. Aside from the limited intersections of their behavioral interests with tobacco control concerns, most had not contributed to the tobacco con¬ trol literature or participated in tobacco control conferences. Unfortunately, when it comes to evaluating tobacco control proposals, the irrelevance ofthe section members' experience and expertise mattered. One year, a tobacco control proposal reviewer announced that s/he did not believe in this type of research and was accordingly giving it a very low rating. The following year, the primary reviewer of a tobacco control proposal succeeded in eliminating one study from the pro¬ posal with a simple observation to the effect that s/he didn't like this type of study. My impression both years was that subjective comments like these were generally not offered and almost certainly did not carry the day with respect to the many behavioral in¬ tervention proposals. As the sole section member with pro¬ fessional experience in tobacco control policy research, my ability to respond to the criticisms of reviewers who don't do this kind of research, and don't like it, in a way that would be persuasive to a group of "peers" who also don't do this kind of research and have very little interest in it, was very lim¬ ited. The result, however unintentional, was a substantial degree of bias against most kinds of tobacco control research, regardless of their quality. Richard A. Daynard, JD, PhD Northeastern University Boston, Mass 1. Glantz SA, Bero LA. Inappropriate and review. JAMA. 1994;272:114-116. appropriate selection of 'peers' in grant To the Editor.\p=m-\Intheir article, Drs Glantz and Bero1 suggest that Glantz's initial research applications on tobacco control policies received poor priority scores from two study sections "because their members were not peers." They used this limited experience to launch an unsubstantiated critique of peer review policies. As a member of one of these study sections (Agency for Health Care Policy and Research, Health Services Research Dissemination [HSRD]), but not a primary or secondary reviewer of the application, I wish to point out some serious flaws and objections that bias the article's conclusions and undermine its credibility. For the record, the reason the initial application that we reviewed received a 92% priority score had nothing to do with the composition and interests of the study section. The main point, which the authors fail to mention, is that the methods section was only a sketchy description of an interesting, but complex, study. Incomplete protocols are rarely scored well by any federal study section on any topic. The article further implies that a letter accompanying the revised application describing the "problems with the peer review [the proposal] received" was related to the study section's subsequent favorable review. In fact, the executive secretary appropriately excluded this letter from the revised application sent to the study section. This application fared well the second time because the investigators provided more details on their methods and responded in detail to the study section's concerns described in its initial review. The data on publication rates presented as evidence of inad¬ equate peer review are also severely biased. The case-control analysis compared publication rates on "tobacco" by members of a general health services research study section with pub¬ lications indexing "heart" by a cardiovascular study section. Obviously, this is an inappropriate comparison group. Unlike the disease- or organ-based emphases of some National Insti¬ tutes of Health study sections, HSRD emphasizes health services/policy research and dissemination across a range of disciplines, including consumer, clinician, and organizational behaviors related to health. Taking the authors' arguments to the extreme would require our study section to have experts on every significant health hazard. Clearly, this is inappropri¬ ate, given the importance of representation of disciplines such as the behavioral sciences, biostatistics, and health economics. In fact, members of the study section were highly interested in the subject matter of the application and expert in many of the general domains it encompasses. Nevertheless, collateral re¬ viewers with specific expertise in tobacco control policy were also invited to evaluate the application; they reached similar conclusions regarding its potential significance pending sub¬ stantial revision and additional details. In summary, the article's biased critique of current study section review procedures (in the absence of any verifiable data on these processes) calls into question the appropriate¬ ness of basing an evaluation of peer review on one interpre¬ tation of a single case. Stephen B. Soumerai, ScD Harvard Medical School Boston, Mass 1. Glantz SA, Bero LA. Inappropriate and review. JAMA. 1994;272:114-116. appropriate selection of 'peers' in grant In Reply.\p=m-\Thestatement by Dr Gruder and colleagues that tobacco policy applications were funded by the TRDRP at a higher rate than applications as a whole is misleading. By our count, in the last review cycle only two tobacco policy grants were funded (out of a total of five submissions), and one of these was a grant application submitted by Dr Bero, which had the policy components of the proposal deleted by the study section. This compares with 90 funded grants out of 391 applications for nonpolicy grants. One of our points was that the inappropriate selection of peers discouraged submission of policy grants. We were, nonetheless, surprised that Gruder et al criticized our article, because he and his colleagues at TRDRP had already taken affirmative steps to remedy the problems in the peer review process we identified. In particular, last year TRDRP convened a new study section to review tobacco policy grants that was made up of appropriate reviewers. Unfortunately, we do not yet know the outcome of this process because the California legislature gutted the TRDRP. Likewise, the National Cancer Institute also appears to have taken our criticisms to heart, as opposed to the view advanced by Dr Soumerai, by establishing a special study section to review recently solicited proposals on tobacco policy research, rather than relying on existing study sections. Tobacco is the leading preventable cause of death in the United States today, and public policy interventions are the most effective way to deal with this problem. We are en- Downloaded From: http://jama.jamanetwork.com/ by a Florida International University Medical Library User on 06/15/2015