Lecture: Guidelines in Evaluating the Validity and Applicability of a Research study

1. What is the research design? 2. What is the level of quality of the Research study according to the U.S. Preventive Services Task Force? 3. Are the results of the study valid? a. Randomization b. Follow-up (80% or better) c. Blinding of the clinicians, patients, research personnels (the more blinding the better d. Baseline similarities (established at the start of the trial) 4. What are the results of the research study? 5. Will the results help in caring for my patient? 6. What is the Level of Recommendation according to the U.S. Preventive Services Task Force

DESIGN OF THE RESEARCH STUDY

Meta Analysis Examines a number of valid studies on a topic and combine the results using accepted statistical methodology as if they were from one large study. Double blind study A study which neither the patient nor the physician knows whether the patient is receiving the treatment of interest or the control treatment. Randomized controlled clinical trials Planned projects that study the effect of a therapy on real patients. They include methodologies that reduce the potential for bias (randomization and blinding) and that allow for comparison between intervention groups and control groups (no intervention). Cohort Studies This study involves a large population and follow patients who have a specific condition or receive a particular treatment over time and compare them with another group that has not been affected by the condition or treatment being studied. Cohort studies are observational and not as reliable as randomized controlled studies, since the two groups may differ in ways other than in the variable under study. Case Control Studies Studies in which patients who already have a specific condition are compared with people who do not. They often rely on medical records and patient recall for data collection. These types of studies are often less reliable than randomized controlled trials and cohort studies because showing a statistical relationship does not mean than one factor necessarily caused the other. Case series and Case reports Collections of reports on the treatment of individual patients or a report on a single patient.

Ideas, Editorials, Opinions Exchange of ideas-Subjective Animal Research Trial that involves animals as samples Test Tube research trial that tests a substance in a test tube

LEVEL of EVIDENCE
Level I Evidence obtained from at least one properly designed randomized controlled trial Level II-1 Evidence obtained from well-designed controlled trials without randomization Level II-2 Evidence obtained from well-designed cohort or case-control analytic studies, preferably from more than one center or research group. Level II-3 Evidence obtained from multiple time series with or without the intervention. Dramatic results in uncontrolled trials might also be regarded as this type of evidence. Level III Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees.

LEVEL OF RECOMMENDATION
The U.S. Preventive Services Task Force (USPSTF) assigns one of five letter grades to each of its recommendations (A, B, C, D, or I). What the Grades Mean and Suggestions for Practice A The USPSTF recommends the service. There is high certainty that the net benefit is substantial. Offer or provide this service. B The USPSTF recommends the service. There is high certainty that the net benefit is moderate or there is moderate certainty that the net benefit is moderate to substantial. Offer or provide this service. C Clinicians may provide this service to selected patients depending on individual circumstances. However, for most individuals without signs or symptoms there is likely to be only a small benefit from this service. Offer or provide this

service only if other considerations support the offering or providing the service in an individual patient. D The USPSTF recommends against the service. There is moderate or high certainty that the service has no net benefit or that the harms outweigh the benefits. Discourage the use of this service. I The USPSTF concludes that the current evidence is insufficient to assess the balance of benefits and harms of the service. Evidence is lacking, of poor quality, or conflicting, and the balance of benefits and harms cannot be determined. Read the clinical considerations section of USPSTF Recommendation Statement. If the service is offered, patients should understand the uncertainty about the balance of benefits and harms.

LEVELS OF CERTAINTY REGARDING NET BENEFIT

Description HIGH: The available evidence usually includes consistent results from well-designed, well-conducted studies in representative primary care populations. These studies assess the effects of the preventive service on health outcomes. This conclusion is therefore unlikely to be strongly affected by the results of future studies. MODERATE: The available evidence is sufficient to determine the effects of the preventive service on health outcomes, but confidence in the estimate is constrained by such factors as: The number, size, or quality of individual studies. Inconsistency of findings across individual studies. Limited generalizability of findings to routine primary care practice. Lack of coherence in the chain of evidence. As more information becomes available, the magnitude or direction of the observed effect could change, and this change may be large enough to alter the conclusion. LOW: The available evidence is insufficient to assess effects on health outcomes. Evidence is insufficient because of: The limited number or size of studies. Important flaws in study design or methods. Inconsistency of findings across individual studies. Gaps in the chain of evidence. Findings not generalizable to routine primary care practice. Lack of information on important health outcomes. More information may allow estimation of effects on health outcomes. The USPSTF defines certainty as "likelihood that the USPSTF assessment of the net benefit of a preventive service is correct." The net benefit is defined as benefit minus harm of the preventive service as implemented in a general, primary care population. The USPSTF assigns a certainty level based on the nature of the overall evidence available to assess the net benefit of a preventive service.