How To Read a Summary Statement with Examples

June 2008 – Presentation delivered at NCCAM Grantsmanship Workshop

Dale L. Birkle, Ph.D
Scientific Review Officer, NCCAM

Reading your summary statement

Agenda

• Factors that determine your priority score
• What a summary statement is
• And what it isn't
• Is the project worth fixing?
• Is it fixable?
• How to amend, aka "revise"

Factors that determine the priority score

• Overall merit. The final priority score reflects a judgment of overall merit
• Ideal application in your field of science
• This is similar to a dog show, where dogs are judged for "best of breed," and different breeds do not compete with each other

Standard Review Criteria

• Significance
• Approach
• Innovation
• Investigators
• Environment

Other Critical Factors

• Special areas: NIH policies for
• recombinant DNA research
• human subjects
• research animals
• select agents
• biohazards

A common problem

• Presentation: Your presentation can make or break your application
• Though reviewers assess science, they are also influenced by the writing and appearance of your application
• If your application has lots of errors and internal inconsistencies, or is hard to follow, your score can suffer

PRESENTATION!

• Reviewers do their best based on the knowledge they have and the information you present
• Poor presentation can cause them to
• Miss a point
• Misunderstand
• Conclude you are a careless scientist
• You must do an outstanding job of writing and organizing your application

Know What a Summary Statement Means

• Your summary statement holds a wealth of information
• Reviewer critiques
• Summary of the discussion
• Priority score
• Percentile (for NCCAM R01s only)
• Recommended budget
• Human and animal subjects codes
• Administrative comments

A summary statement is not

• An exhaustive critique
• A teaching tool containing every point reviewers found to be problematic
• A document with you stating what you need to do to get a better score

A summary statement is not a complete guide to fixing your application

• Once reviewers identify significant weaknesses, they may stop discussing the application because time is short
• The flaw could be something simple to correct

But

• Once the reviewers stop discussing the application, their feedback ends

Resume versus Critiques

• After you get your summary statement, read it carefully and analytically.
• The Resume is the summary of the discussion and basis of the priority score.
• The Resume will state the essential strengths and weaknesses
• The Critiques may or may not reflect the outcome of the discussion.

Before you can decide what to do, you need to determine whether

• the application is worth fixing
• its faults are fixable

Contact Your Program Officer for Feedback

• Ask about the level of reviewer enthusiasm for your idea
• Ask about the program officer's level of enthusiasm
• Ask about points not addressed in the summary statement
• Ask about your options

Ask yourself: Is It Worth Fixing?

• Each unfunded application falls into one of three categories:
• Has major flaws
• Does not generate enthusiasm (dull topic)
• Has fixable problems

Examples of major flaws

• Philosophical issues, e.g., the reviewers do not think the work is important
• Hypothesis is not sound
• Work has already been done
• Methods are not suitable

If You Get Little Criticism

• Often this means reviewers were not excited about your idea. They may not state this explicitly
• Try to get honest feedback from coworkers or mentors
• Don't shoot the messenger
• If a dull topic was the problem, amending won't help
• Start over with a new hypothesis

If Reviewers Noted Many Fixable Problems

• This may be a good sign
• It often shows they are interested in the idea
• It often indicates that the application is worth fixing

Common Fixable Problems

• Problem: Poor writing.
• Solution: Rewrite; get help
• Problem: Insufficient information, experimental details, or preliminary data.
• Solution: Assess what's missing; add it to the Research Plan.
• Problem: Significance not convincingly stated.
• Solution: Beef up that section; show the importance to NCCAM's mission, your area of science, and public health.
• Problem: Research not shown to be feasible by the proposed staff.
• Solution: Get consultants with the required expertise
• Problem: Insufficient discussion of obstacles and alternative approaches.
• Solution: Describe what you'll do if you get negative results or an approach doesn't pan out. Include decision trees.
• Problem: Reviewers are not interested in the subject.
• Solution: Were they the appropriate reviewers? If not, request a different review group for your next submission. If yes, then this problem is not fixable.

Pick a Strategy That Suits the Problem

• For fixable problems
• Amend the application and resubmit it to the same study section
• Amend the application and resubmit it to a different study section
• For non-fixable problems
• Create a new application
• Look for funding outside NIH

New or Amended?

• If you revise more than 50 percent, it's a new application.
• If less, follow the rules for an amended application.

Can a Resubmission Hurt You?

• Most resubmissions score higher than the initial application, though, of course, there's no guarantee
• During fiscal years 1996 to 2006, more than 80 percent of resubmissions got better scores
• Less than 5 percent got significantly worse scores.

Amend and Resubmit

• You retain most of your original application
• Can address the reviewers' concerns
• The review group looks at the application in the context of their critiques
• NIH has anecdotal evidence that applicants who resubmit to the same review group have better chance of success

Resubmission caveat

• Even if you amend, your priority score may not improve
• The reviewers may focus on aspects that were not mentioned in the prior review
• The changes make things worse
• The response to criticisms are
• Inadequate
• Hostile

Resubmission Tips

• Capitalize on your strengths
• Throw out or revise the weak parts
• Check that your Specific Aims line up with your hypothesis.
• Respond point by point to the reviewers' comments and suggestions even if you disagree.
• If you disagree, explain why, and provide additional information if possible.
• Even better, change your application.
• If several years have elapsed since you submitted the application, the reviewers' comments may no longer be relevant.

Resubmission Tips

• Identify all changes. Use arrows, brackets, indents, or a new font. You can use Arial, Helvetica, Palatino Linotype, Georgia typeface, or a combination of the above. All fonts used should be black, 11 points or larger.
• If most of the text has changed, state that in the introduction rather than doing the above.
• Do not underline or shade changes.
• Avoid italicizing large blocks of text.

Resubmission Tips

• Add new findings and your own changes
• Though you should revise items mentioned in the summary statement, you aren't limited to those items
• In the Preliminary Studies/Progress Report section, add any new data obtained since the previous application
• Don't hesitate to make other changes. Strengthen the application as much as you can

Resubmission Timing

• Consider amending and resubmitting right away if your application scores above the payline, and the problems are fixable.
• Many people start amending even before getting the summary statement because waiting may cause them to miss the next receipt date.
• If you are on a list for possible selective pay or end-of-year funding, don't wait — amend and resubmit.

Should You Start Amending Quickly?

• You could start amending before you get your summary statement if you have the following information:
• Insights into peer review from your program officer, who attended the review meeting and told you about the discussion.
• Promising new data or other improvements you want to include.
• Then after you get your summary statement, add to the amendments you've already made to address reviewer concerns.

However…

• A one-round delay may have little impact on timing of an award
• At the end of the fiscal year (June-July receipt date, September-October Council), you often have to wait several extra months before you get an award because the Institute does not yet have a budget for the following fiscal year.
• If you wait to submit for the October-November receipt date instead, you could lose just a month or two before you actually get an award.

When Not to Resubmit Early

• Always wait to resubmit until you have the strongest application possible
• Better to wait for the next receipt date than to send an application before it is ready.
• Spend the time you need polishing your application
• Don't rush to meet a receipt date

Helpful Websites

NCCAM nccam.nih.gov/research/

NIAID's NIH Grant Cycle decision tree www.niaid.nih.gov/ncn/grants/cycle/default.htm

---

Note: PDF files requires a viewer such as the free Adobe Reader.