Dissecting a Manuscript Series

Panel 2 Highlights: The Methods

Written by Jess Schmerler (Education Committee)

Panelists

Highlights

Dr. Ronald A. Navarro is the Deputy Editor for Evidence-Based Orthopaedics for JBJS, and is a Professor of Orthopedic Surgery and Director of Orthopedic Affairs at Kaiser Permanente School of Medicine. Over his career, Dr. Navarro has served as the Regional Assistant Medical Director of Business Management for Retail Strategy for the Southern California Permanente Medical Group, and continues to serve as Regional Chief of Orthopedic Surgery.

His research interests include the role of shoulder arthroplasty registries in tracking patient outcomes, orthopedic treatment in ethnic groups, and how data can adapt surgeon behavior in this time of healthcare reform.

Dr. Marc Swiontkowski is the Editor-in-Chief of JBJS, and is a Professor in the Department of Orthopaedic Surgery at the University of Minnesota. Over his career, he has served as president of the Orthopaedic Trauma Association, president of the Mid America Orthopaedic Association, and representative to the Board of Directors of the American Board of Orthopaedic Surgery. Dr. Swiontkowski also has a podcast called OrthoJoe. Definitely check that out if you haven’t already.

The panelists provided us with an enormous amount of information in one hour. The most salient tips, quotes, and highlights are provided below:

  • You are aiming to have enough information that an individual coming along after you can reproduce the same experiment

  • Findings not being reproducible is often due to not enough detail being presented in the methods

  • The methods section allows the reader to judge the merit of the work on the basis of the methodology that was utilized

  • Methods should be able to be readable by the average orthopaedic reader, and should be straightforward, but still needs to be inclusive enough to contain a litany of information

  • If there is no ability to reference a specific part of the technique used, it has to be in there with high level of detail. If you run out of word count, you can always use the appendices to accomplish this

  • There are published guidelines for different experimental designs for clinical research:

    • STROBE guidelines for cohort studies

    • CONSORT guidelines for RCTs

    • PRISMA guidelines for systematic reviews

  • Dr. Swiontkowski recommends you always work with a statistician and write the methods section before the first data element is collected or first patient is evaluated

  • “The methods section should be the biggest section of the manuscript”

  • Search for JBJS levels of evidence that are based on rigor involved in the question as well as the potential for bias – the idea is that the level of evidence can drive treatment choices in big systems. This doesn’t mean that levels below I or II are not useful, since especially in the case of rare conditions, RCTs may not be possible. Level IV evidence can still be impactful.

  • Tables can be a good way to avoid lengthy descriptions in the methods section

  • In order to apply multivariate analysis and have it be believable, you need to have roughly 600 subjects in each arm; if it’s less, the confidence intervals are so wide there isn’t a lot you can take home from it.

    • Additionally, you should always report the confidence intervals, not just the p-values.

  • Acceptance rate is 5-6% for large database studies (less than ~17% overall) due to lack of indications as well as variable follow-up and other weaknesses – JBJS symposium on datasets and what their weaknesses are

  • Be sure to explain your inclusion and exclusion criteria either explicitly or via references

  • Do not over-explain nuances - keep the methods section separate from the discussion

  • If specific methods have been described in the literature, reference those rather than over-explaining

  • Do not describe results in the methods section

  • Do not assume everyone understands your study and methodology

    • Example: Do not forget to list the indications for treatment when demonstrating why patients in different groups received different treatments

Common Mistakes:

Overview

“Methods” was the second panel in the “Dissecting a Manuscript” series hosted by the Education Committee. The purpose of the panel was to learn from the JBJS panelists about the best ways to write a methods section through the lens of an analysis of three published manuscripts. We are incredibly grateful for the insights shared by our panelists.