Discussion Question
1- Discussion Question: How can you guarantee validity and feasibility in your research study with a mixed method?
2- 3 paragraphs 3 sentences each
3- APA style
4- 2 references not older than 2015
Chapter 29
Feasibility Assessments and Pilot Tests of Interventions Using Mixed Methods
Copyright © 20
1
7 Wolters Kluwer Health | Lippincott Williams & Wilkins
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
1
Question #1
Tell whether the following statement is true or false:
The terms pilot study and feasibility study are interchangeable.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer to Question #1
False
Often, the terms pilot study and feasibility study are used interchangeably. Recently, however, some experts are drawing a distinction between the two.
A feasibility study is research completed prior to a main intervention study to test specific and discrete aspects of an emerging intervention or the anticipated trial.
A pilot study is an investigation designed to test the feasibility of, and to support refinements of, the protocols, methods, and procedures to be used in a larger scale trial of an intervention.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Basic Issues in Piloting Interventions
Feasibility study
Tests specific and discrete aspects of an emerging intervention
Using a fairly simple design
Pilot study
A small-scale version of a full trial
Assesses an entire set of procedures for implementing
Evaluates an intervention
Ideally involves a randomized design
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Feasibility Study
Completed prior to a main intervention study
Tests specific and discrete aspects of an emerging intervention or the anticipated trial
Does not focus on the outcome of interest
Typically does not use a randomized design
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
5
Overall Purpose of Pilot Work
Goal is to avoid a costly full-study failure.
Questionable as to whether it should be used to test a hypothesis
Sample size is too small.
Typically 30-40 subjects
Pilot study results are meant to inform the decision about whether to
Move forward with a full trial
Make revisions that require an additional pilot
Abandon the project altogether
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
6
Question #2
Tell whether the following statement is true or false:
An external pilot is a stand-alone study, whereas an internal pilot is an early phase of a larger trial.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer to Question #2
True
An external pilot is a stand-alone study, the findings from which inform the design and implementation of a full randomized control trial. An internal pilot is an early phase of a large trial, the findings from which are typically used to make adjustments to sample size projections.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Objectives and Criteria in Pilot Work
Stated at the onset of study
Focus on the
Process
Resources
Management issues
Scientific issues
Retention
Ethics
Recruitment
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
9
Scientific-Related Objectives
Focuses on the substantive aspects of the intervention
Safety and tolerability
Clinical significance
Preliminary evidence of efficacy
Clinical significance
Focuses on the methodologic aspects of a trial
Randomization
Sample size (computed with confidence intervals [CIs])
Fidelity
Instruments
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
10
Mixed Method Designs
Especially well suited for a pilot study.
Quantitative data can be used to assess whether feasibility criteria were met.
Qualitative data can elucidate
Why they were not met
How the intervention of study protocols could be improved
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
11
Lessons from Pilot Work
Lessons learned
Major product provides
Information from which protocols for a full trial can be created
If interventions have been found to be feasible, acceptable, and promising
List of proposed outcomes
The findings will be published so that others can benefit from learning about both the successes and the failures
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
12
Critical Appraisal of Feasibility and Pilot Studies
Provides descriptions of sample characteristics, design elements, and instruments
Intervention theory and development of the intervention
References should be provided to any previously published papers on intervention development
Researchers’ description of the pilot objectives, the criteria used to make decisions about feasibility, and the methods associated with feasibility assessments
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
13
Question #3
What term is used to identify practical information about the real-life application of a pilot study?
Responder analysis
Exit interview
Lessons learned
Statement of purpose
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer to Question #3
C
A major product from pilot work is a description of “lessons learned” that informs the final protocols for a full trial as well as providing information about the usefulness and achievability of the proposed intervention(s) in the “real world.”
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Chapter 27
Basics of Mixed Methods Research
Copyright © 20
1
7 Wolters Kluwer Health | Lippincott Williams & Wilkins
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
1
Question #1
Tell whether the following statement is true or false:
Mixed methods research is a research involving the collection, analysis, and integration of both qualitative and quantitative data within a study or series of studies.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer to Question #1
True
Mixed methods research is a research involving the collection, analysis, and integration of both qualitative and quantitative data within a study or series of studies, often with an overarching goal of achieving both discovery and verification.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Overview of Mixed Methods Research #1
Rationale for mixed methods studies
Complementarity of qualitative and quantitative data
Practicality
Enhanced validity
Paradigm issues and mixed method studies
Research question
Creswell and Plano Clark
Seven types
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Overview of Mixed Methods Research #2
Confirmation and explication
Instrumentation
Intervention development
Program evaluation
Realist evaluations
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question #2
Tell whether the following statement is true or false:
The paradigm most often associated with MM research is pragmatism.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer to Question #2
True
The paradigm most often associated with MM research is pragmatism, which has as a major tenet “the dictatorship of the research question.”
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Getting Started on a Mixed Methods Study
Purpose and intent
Research questions
Different types of data
Integrative
Focus on linking two strands
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
8
Mixed Methods Designs #1
Key decisions
Fixed versus emergent designs
Sequencing
Sequential/current
Prioritization
Equal versus dominant status
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question #3
Tell whether the statement is true or false:
Sequential designs involve both strands occurring in one simultaneous phase.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer to Question #3
False
Concurrent designs (both strands occurring in one simultaneous phase); sequential designs (one strand occurring prior to and informing the second strand).
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Mixed Methods Designs #2
Notation and diagramming in mixed methods designs
All capital letters for the dominant strand
All lowercase letters for the nondominant strand and sequence
An arrow is used for sequential designs
A “+” is used for concurrent designs
Parentheses can be used to show an embedded structure
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Mixed Methods Designs #3
Core mixed methods designs
Convergent
QUAN + QUAL → Results merged → Interpretation
Explanatory sequential
QUAN → qual (QUAN results explained by qual) → Interpretation OR quan → QUAL (quan results explained by QUAL) → Interpretation
Exploratory
QUAL → Development (e.g., qual + quan) → QUAN (Testing) → Interpretation
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Sampling and Data Collection in a Mixed Methods Studies
Identical: same participants are in both strands.
Nested: some of the participants from one strand are in the other strand.
Parallel: participants are either in one strand or the other but drawn from a similar population.
Multilevel: participants are not the same and are drawn from different populations at different levels in a hierarchy.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question #4
Tell whether the following statement is true or false:
Data collection in MM research can involve all methods of structured data only.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer to Question #4
False
Data collection in MM research can involve all methods of structured and unstructured data.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Analysis of Mixed Methods Data #1
Decisions
Integration intent
Procedures
Iterative analysis
Meta-matrices
Transforming quantitative and qualitative data
Qualitizing
Quantitizing
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Analysis of Mixed Methods Data #2
Joint displays
Statistics-by-theme
Side-by-side joint display
Meta-inferences
Six ways of exploring discrepancy
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Chapter 28
Developing Complex Nursing Interventions Using Mixed Methods Research
Copyright © 20
1
7 Wolters Kluwer Health | Lippincott Williams & Wilkins
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
1
Question #1
Tell whether the following statement is true or false:
Nursing intervention research refers to a distinctive process of developing, implementing, testing, and disseminating nursing interventions.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer to Question #1
True
Nursing intervention research refers to a distinctive process of developing, implementing, testing, and disseminating nursing interventions—particularly complex interventions.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Nursing Intervention Research #1
Distinctive process
Developing, implementing, testing, and disseminating nursing interventions
Particularly complex interventions
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Nursing Intervention Research #2
Complex intervention
The number of different behaviors required by those delivering or receiving the intervention, and the difficulty level of those behaviors
The number of different groups or organizational levels targeted by the intervention
The number and diversity of intervention outcomes targeted
The degree to which the intervention can be tailored to individual patients
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Nursing Intervention Research #3
Frameworks for developing and testing complex interventions
Phase 1: Development
Phase 2: Feasibility and piloting
Phase 3: Evaluation
Phase 4: Implementation
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question #2
Tell whether the following statement is true or false:
Studies to assess effectiveness of interventions in real-world clinical settings (Phase 4) are common in nursing.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer to Question #2
False
Studies to assess effectiveness of interventions in real-world clinical settings (Phase 4) are rare in nursing.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Nursing Intervention Research #4
Key features of complex intervention research
Strong support for mixed methods approaches
Coordinated teamwork
Ongoing, dedicated program
Patient and public involvement (PPI)
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Nursing Intervention Research #5
Desirable features of nursing interventions:
Salient
Efficacious
Conceptually sound
Cost-effective
Feasible
Developmentally appropriate
Culturally sensitive
Accessible
Acceptable
Adaptable
Readily disseminated
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question #3
Tell whether the following statement is true or false:
An important product during Phase 2 is a carefully conceived intervention theory from which the design of the intervention flows.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer to Question #3
False
An important product during Phase 1 is a carefully conceived intervention theory from which the design of the intervention flows.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Phase 1: Intervention Development #1
Key issues
Conceptualization of the problem
Understanding of current practices and why they are deficient
Articulation of an evidence base for the intervention
Conceptualization of the context
Conceptualization of solutions, strategies, and outcomes
Construct validation of the intervention
Identification of potential pitfalls within the implementation context
Cultivation of relationships
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Phase 1: Intervention Development #2
Major activities
Critical synthesis of relevant literature
Concept and theory development
Exploratory and descriptive research
Consultation with experts; content validation
Brainstorming with colleagues, team building, partnerships with stakeholders
Modeling and designing the intervention
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Phase 1: Intervention Development #3
Products and outcomes
An intervention theory
Preliminary specification of the content, intensity, dose, timing, setting, and delivery method of the intervention
Preliminary identification of key outcomes
Strategies to overcome pitfalls in implementing and testing the intervention
An implementation plan
A design for a pilot study
A plan for sponsorship of the pilot study
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Phase 1: Intervention Development #4
Intervention theory
Dose and intensity
Timing
Outcomes to target and when to measure them
Setting
Agents
Mode of delivery
Individualization
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Phase 2: Pilot Testing an Intervention
Feasibility
Acceptability
Pilots
Qualitative components
Recruitment and retention
Feasibility of more rigorous testing
Lessons
Refine the intervention
Improve its acceptance and appeal
Enhance delivery
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Phase 3: Evaluation of the Intervention
Full test of intervention using a randomized design
Assessing efficacy
Assessing intervention fidelity
Clarifying the intervention
Understanding the context
Probing for clinical significance
Interpreting results
Summary report
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Phase 4: Implementation
Implementation
Normalization process theory (NPT)
A measure called NoMAD (Normalization MeAsure Development)
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question #4
Tell whether the following statement is true or false:
A mixed methods approach can strengthen the test of the intervention during the Phase 2 controlled trial.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer to Question #4
False
A mixed methods approach can strengthen the test of the intervention during the Phase 3 controlled trial. The inclusion of qualitative components can shed light on intervention fidelity, variation in effects, clinical significance, and interpretive ambiguities.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins
Mixed Methods Designs for Intervention Research
Appropriate for all phases
Phase 1 QUAL has priority.
Phases 2 and 3 QUAN are usually dominant.
Copyright © 2021 Wolters Kluwer Health | Lippincott Williams & Wilkins