Skip to main content
Public Health Workforce Training
Link to beginning section of Advanced Program Evaluation

Research Design 1

ShapeTracker Intervention Stakeholders

  • Commissioner Middleton County (MC)
  • Middleton County (MC) Health Department (HD) ShapeTracker (ST) team
  • MC HD Information Technology staff
  • MC HD marketing/communications staff
  • Other MC HD staff
  • CDC
  • Group that developed ST for CDC
  • MC community members
  • Wellness advocates in MC
  • Obesity prevention programs in MC
  • Healthy eating related resources in MC
  • Exercise resources in MC
  • Smartphone providers in MC
  • California communities that implemented ST
  • Smartphone companies
  • MC government

Logic Model

Inputs Activities Outputs Short-term Outcomes Intermediate Outcomes Long-term Outcomes
ShapeTracker App Mobilize local resources and customize ST for MC # of times ST app downloaded in MC Decrease perceived barriers to healthy eating and physical activity Increase healthy eating in MC Reduce obesity in MC
Funding from HD and CDC Market ST in MC # of MC residents actively using ST Increase self-monitoring of healthy eating and physical activity Increase physical activity in MC Reduce obesity related health problems in MC
Team of experienced staff Keep app updated and maintain content Amount of use for different ST functions Increase self-efficacy for healthy eating and physical activity    
High smart phone use in MC community Provide technical assistance to users User satisfaction with ST Increase social supprt for healthy eating and physical activity    
Marketing & communication resources in HD   # of friends referred to ST      
IT resources in HD   # of MC residents actively using ST long-term      
Local obesity prevention resources in MC          

ShapeTracker Intervention Evaluation Questions

Process Questions
  1. Was a broad range of local resources successfully engaged in the project?
  2. Was ST successfully disseminated to a broad segment of MC community?
  3. What proportion of MC residents have:
    • downloaded the app?
    • became active users of the app?
    • became active long-term users of the app?
  4. For those who stopped using ST, why did they discontinue?
  5. How much are ST users using different functions in ST?
  6. How satisfied are MC residents with the ST app and its specific features?
Outcome Questions
  1. Does use of ST increase:
    • self-monitoring of healthy eating and physical activity?
    • self-efficacy for healthy eating and physical activity?
    • social support for healthy eating and physical activity?
  2. Does ST decrease perceived barriers to healthy eating and physical activity?
  3. Does use of ST increase healthy eating and physical activity among county residents?
  4. Does use of ST lead to lower BMI among those who are obese or overweight?
  5. Does use of ST help those with normal BMI prevent overweight?

The evaluation questions we have chosen to address in our evaluation plan.

I’ll walk you through this first example.

Our team is considering a design that uses our neighbor county, Upperton, as a comparison group. We want to aggressively market ShapeTracker to everyone in our county, and we hope a high proportion of Middleton residents will sign up. We do not think we can find a comparable comparison group within Middleton county.

We will utilize annual representative household surveys in both counties as an opportunity to collect data for this evaluation. We’ll add our outcome measures of physical activity, healthy eating and self-monitoring to these surveys. This year’s annual surveys, which will be collected before ShapeTracker is launched later this year, will be the baseline. Next year we will collect the same measures again in both counties. And for Middleton county household survey respondents who are using ShapeTracker, we’ll be able to include an additional section in the survey to collect some data we need for our process evaluation.

Question 1:

We first need to ask whether we need more than one row.
How many rows do we need?

Show answer

The correct answer is B:
We need two rows to represent the intervention county and the comparison county. When there is a control group or comparison group you need two rows.

Question 2:

If there are two rows, the next question is whether the rows are R or NR? Which one is our design?

Show answer

The correct answer is B:
If we have two rows, we need to indicate whether we have random assignment. In the notation system, R in the beginning of each row stands for randomized assignment and NR for non-randomized assignment. Ours is NR.

Question 3:

Do we need an X?

Show answer

The correct answer is A:
We always have at least one X to indicate the intervention. The first row for Middleton county has an X but the second row for Upperton county has a blank space.

Question 4:

How many O’s do we need for our diagram?

Show answer

The correct answer is B:
We are getting baseline measures in both counties, so we need O’s for the baselines. Both counties are also providing post intervention measures, so we need O’s for those. We need a total of 4 O’s.

Here is our design:

NR
O
X
O
NR
O
 
O

This is a comparison group design with baselines and post-tests for both groups (Middleton and Upperton). If we see more positive change from baseline to post-test in Middleton, compared to Upperton, we have evidence that ShapeTracker (the X) is having an impact on the outcomes.