St. Anthony Main 219 Main Street SE, Suite 302 Minneapolis, Minnesota 55414 612.623.9110
(f) 612.623.8807 |
By
PDA_News
|
Published
January 30, 2012
Lija Greenseid, Michael Luxenberg, and Matthew Christenson are co-authors with colleagues at ClearWay MinnesotaSM and the University of Minnesota on a new peer-reviewed journal article. The article is entitled, “The relation between media promotions and service volume for a statewide tobacco quitline and a web-based cessation program.” The article examines the impact of anti-tobacco media campaigns while controlling for other environmental factors on weekly calls to a state quitline and registrations for an online cessation intervention program. PDA conducted a multivariate regression analysis with two outcomes to examine the impact of media on these two standalone programs.
The article is published in BMC Public Health, an open-source, peer reviewed journal. The article is available online at: http://www.biomedcentral.com/1471-2458/11/939/abstract
By
PDA_News
|
Published
January 17, 2012
PDA was recently awarded a contract from the North Dakota Department of Health to evaluate the Quitline and QuitNet tobacco cessation programs. The North Dakota Tobacco Quitline and QuitNet Cessation Programs are components of a comprehensive tobacco control effort being developed and implemented by the North Dakota Department of Health and guided by best practices as outlined by the Centers for Disease Control.
PDA will partner with the North Dakota Department of Health to develop a comprehensive process and outcome evaluation to provide recommendations to enhance the Quitline and QuitNet programs. North Dakota is the sixth state for which PDA currently conducts tobacco control and other public health evaluations. We look forward to working with our neighbors to the northwest!
By
PDA_News
|
Published
January 10, 2012
PDA Vice President, Anne Betzner, and Senior Evaluator, Melissa Chapman-Haynes, each gave birth to healthy baby boys about 36 hours apart last week in the same hospital in Minneapolis. Anne’s son, William Connor Bradford, was born at 7:43 am on Thursday, January 5th and Melissa’s son, David Antoine Haynes III, was born at 9:30 pm on Friday, January 6th. This is the first child for each of them, and they are both very excited to become mothers. All of us at PDA are thrilled for Anne, Melissa, their husbands, and their expanding families!
By
MChapman
|
Published
December 19, 2011
PDA is happy to announce the addition of a new Associate Evaluator to our staff! Angie Ficek comes to us from the Minnesota Institute of Public Health where she contributed to the evaluation of Minnesota’s Statewide Health Improvement Program and several substance abuse prevention programs. Prior to working at the Minnesota Institute of Public Health, Angie served as a public health program coordinator at South Dakota State University in the Office of Nursing Research. In this role, she contributed to the evaluation of South Dakota’s tobacco Quitline as well as their cancer control coalition. At PDA, Angie is working on tobacco quitline evaluation projects for the states of Hawaii, Florida, and Connecticut. She is also working on obesity prevention programs for youth with the Connecticut Department of Public Health. Angie received her BA in Psychology from Minnesota State University – Moorhead and MPH in Community Health Education from the University of Minnesota. She is thrilled to be at PDA and to work with quitlines again!
PDA has maintained our high standards of data security by utilizing features in the LimeSurvey online surveying software. Specifically, because LimeSurvey is open source and can be installed on a company’s servers, PDA is able to control access to the survey data so that no one else, including a web hosting company, can access it. We also use PDA’s SSL certificate to protect data entered through online surveys. We have found that many other survey programs store the data on their own servers, which would have limited our ability to monitor or increase the protection of sensitive data. PDA is committed to maintaining a high level of data security and by storing all survey data in PDA’s encrypted databases we are able to do that.
For more information about LimeSurvey, visit their website, www.limesurvey.com, or watch for future posts on how PDA has been using LimeSurvery.
By
MChapman
|
Published
November 28, 2011
PDA is happy to announce the addition of a new Program Evaluator to our staff! Vanessa Klisch comes to us after moving home to Minnesota after several years in the San Francisco Bay area. Before her return, she worked at Blue Shield of California managing mental health, EAP, and tobacco cessation programs. She also spent several years at the Minnesota Institute of Public Health coordinating their Service to Science Program Evaluation Academy and worked on evaluation projects related to chemical dependency. At PDA, Vanessa is working on tobacco quitline evaluation projects for ClearWay Minnesota and the Florida Department of Health. Vanessa received her BA in American Indian Studies and MPH in Community Health Education from the University of Minnesota and was a MacArthur Scholar. She is excited to be at PDA and overjoyed at being back in the Twin Cities!
PDA has been using LimeSurvey’s template design feature to improve web surveys. Specifically, this feature allows us to create web surveys using associated design – matching the survey design with the brand of the organization associated with the evaluation (the evaluand). Our experience in survey design has shown that survey participants are much more likely to complete a survey about an organization if they can visually associate the survey with the evaluand. The LimeSurvey template design feature allows PDA to create surveys that include recognizable color schemes, banners, logos and browser tab icons to help survey participants associate the survey with the program being evaluated.
LimeSurvey is a free, open source web survey software suite used by PDA for our web survey needs. For more information about LimeSurvey, visit their website, www.limesurvey.com, or watch for future posts about how PDA has been using LimeSurvey.
On Saturday, November 5, a member of PDA’s staff will be presenting at the American Evaluation Association conference in Anaheim, CA. Dr. Melissa Chapman Haynes will present on how the evaluation team created logic models in collaboration with project staff in a community-based tobacco cessation program. Logic models are one key tool evaluators have to engage stakeholders and project leaders in determining values, and how those values relate to project activities and intended results. PDA specifically used logic models as a tool to better understand the broader context and scope of this project’s tobacco-cessation work, including in-person cessation classes, referrals to the tobacco Quitline, community partnerships, and training of health professionals related to tobacco cessation. PDA used the DoView program to create, share, and revise logic models with program stakeholders (http://doview.com/). The authors of this presentation are Dr. Melissa Chapman Haynes, Dr. Lija Greenseid, and Julie Rainey.
Also on Saturday, Dr. Lija Greenseid will give a demonstration on minimizing total survey error. Surveys are common tools for gathering evaluation information; however, they are only as useful as the quality of the data that are collected. This demonstration will provide a multi-faceted framework for thinking about survey quality that includes both the accuracy of the data and other important considerations such as usability, timeliness, and credibility. Dr. Greenseid will discuss advances in survey questionnaire design, sampling, and administration and share examples from her experience in evaluating tobacco control programs.
At PDA we have begun using LimeSurvey for many of our online surveying needs. LimeSurvey is a free, open source web survey software suite that has an ever growing feature list and has allowed us to create more elaborate, complex surveys than what we were able to do with other paid online survey subscriptions. We are excited about the flexibility of LimeSurvey, as this provides us with expanded options to collect tailored data.
For more information about LimeSurvey, visit their website, www.limesurvey.com, or watch for updates about how PDA has been using LimeSurvey on future posts.
The old joke goes … “A mathematician will add 2 plus 2 and get exactly 4. A physicist will say 2 plus 2 is approximately but not certainly 4. But a statistician will smile and say that 2 plus 2 is whatever you want it to be …”
Fortunately, the field of tobacco cessation and organizations like the North American Quitline Consortium (NAQC) are building agreement on methods for measuring quit rates. However, agreement on other elements of study design is less developed. A prime example of this is sampling strategy, or the process of systematically selecting a subset of participants to be included in a study. Program staff should examine the sampling strategies used in outcome studies to better understand what the resulting rates mean and how to use them.
Selecting participants by program intensity
Sampling criteria that are often considered include enrollment date, demographics (gender, race, etc.), and program-level variables (site, counselor, etc.). Program intensity is a sampling criterion that is often overlooked.
If a quitline offers several different interventions, you must decide which will be included in the sample. If a quitline offers an intensive multi-call program with free stop-smoking medications, as well as a single 10-minute session to answer questions about quitting, should both be included in the sample? This depends on the purpose and scope of the study. However, it is important to note that including participants enrolled in the brief intervention would result in a more conservative quit rate because evidence suggests that those who receive less intervention are less likely to quit.
A second facet of program intensity is the degree to which a participant completes the intervention. Some participants don’t complete intake before leaving the program or never speak with a counselor. Likewise, including these participants would result in a more conservative quit rate because they are less likely to have quit. On the other end of the spectrum, including only participants who completed two counseling sessions, for example, would result in more liberal or optimistic quit rates.
Choosing who to include in quit rates
Once the sample has been drawn and data is collected, program staff must decide who to include in the quit rates. A quit rate can include either the entire sample or only those who responded to the survey. Unfortunately, many participants who initially consent to take part in evaluations are later unreachable or refuse to participate when called for follow-up. Whether or not these participants are included in quit rate calculations can greatly influence the estimated quit rate and determine how conservative it is.
With the intention to treat rate, everyone in the survey sample is included in the quit rate, even those who did not respond to the survey. This is because the program intended to treat everyone, regardless of response status. However, no information is available on non-responders’ smoking status at follow-up. But because several studies
suggest that non-responders are more likely than responders to be using tobacco at follow-up, the intention to treat rate assumes all non-responders are still smoking. Because of this conservative assumption, this rate is considered to be the lower bound on the “true” quit rate.
A completer rate includes only those who completed the follow-up survey because this is the only group that reported their smoking status. Unfortunately, those who complete surveys are more likely to have quit than those who do not, so this rate may be somewhat inflated.
Best to report a range
Neither rate is “better” than the other. Calculating both completer and intention to treat rates provides an upper and lower bound on the “true” quit rate and provides a more complete picture of a cessation program’s success.
|
|
