This post is part of a new blog series called Transformative Change Initiative (TCI) Featured Evaluator, that includes interviews with members of TCI’s Evaluation Collaborative. This community of evaluators has a wealth of knowledge, experience and insights into evaluation of the TAACCCT grants that are being implementation throughout the United States. Want to be profiled or know someone who would make a great feature? Email us at occrl@illinois.edu.
Name: Sandra Staklis
Current position: Senior Research Associate, RTI International’s division of Education and Workforce Development
Short bio: Sandra Staklis, Ph.D. is Senior Research Associate in RTI International’s division of Education and Workforce Development. She has expertise in quantitative and qualitative data analysis and survey design, and experience in the areas of secondary and postsecondary student outcomes, community colleges, workforce education and development, and program evaluation. Dr. Staklis is currently leading the evaluation of Trade Adjustment Assistance Community College and Career Training (TAACCCT) grant programs in Colorado, Hawaii, Kansas, and Montana, as well as a multi-year program evaluation of a middle-skills retail training program for Goodwill International sponsored by the Walmart Foundation in eight sites nationwide. For the Office of Career, Technical, and Adult Education in the U.S. Department of Education, Dr. Staklis has led program evaluations and research studies on tracking secondary, postsecondary, and adult student outcomes using data collected through state longitudinal data systems.
Questions
Q. What is the design and predominant methods for your TAACCCT evaluation?
A. All of our TAACCCT outcome evaluations have a quasi-experimental design using propensity score matching to select matched comparison groups from other programs and historical cohorts. Data for these analyses are drawn from institutional or state student record data systems and match to data from state departments of labor. The implementation (or formative) evaluations are mixed-method studies that use multiple sources of data collected from a variety of stakeholders, including project staff, employers, community and workforce system partners, state agency staff, and students. The evaluation teams’ site visits, interviews, surveys, and observations seek to provide not only a comprehensive overview of the project’s implementation processes, challenges, and successes, but also actionable data to inform program refinements and improvements.
Q. What advice do you have for new TAACCCT evaluators?
A. Find multiple strategies for keeping up with grant activities! Particularly for consortium projects, TAACCCT grants involve many colleges that are engaged in a variety of activities, and keeping up and documenting progress, changes, and setbacks takes persistence. In addition to regular evaluation meetings with the grant leadership team, attend any regularly scheduled project meetings as an observer, join in-person project meetings when feasible, and also check in regularly with project leadership and the college leads by phone and email.
Q. What questions do you have for others about TAACCCT evaluation?
A.The sustainability of grant-funded programs is sometimes dependent on connections to state and local initiatives and local and institutional priorities, that can direct resources and support to program activities after the grant ends. Has the TAACCCT grant that you are evaluating succeeded in making such connections and, if so, in what ways do these connections have the potential to support the sustainability of grant activities?
Q. Do you have tools, reports or other products you are willing to share?
A. We work in cross-disciplinary and cross-subject-matter teams at RTI, which has exposed me to useful strategies and tools outside the area of education. One of these, the Consolidated Framework for Implementation Research (http://cfirguide.org/), is from my colleagues in public health. Although not all of the framework’s constructs are appropriate for education, I have found it helpful for ensuring that formative evaluations consider the multiple and multi-layered contextual factors that can affect a program’s successful implementation.
Sandra can be reached at sstaklis@rti.org.