Skip to content
CU Denver & CU Anschutz logo Colorado State University logo University of Northern Colorado logo

colorado school of public health

coloradoSPH

  • Apply
  • Visit
  • Give
 

Centers for American Indian & Alaska Native Health

  • About
    • Location
    • Directory
    • Contact Us
    • News
    • Events
  • Projects
    • AIM-AHEAD
    • Center for American Indian and Alaska Native Diabetes Translation Research
    • Center for American Indian and Alaska Native Health Disparities
    • Culturally Tailoring a Diabetes Nutrition Education Program
    • Native-CHART
    • Native Skin Health
    • Native WYSE CHOICES
    • Strong Lakota Families
    • Suicide Prevention Hub
    • Tribal Early Childhood Research Center
    • TIPCAP
    • Past Work
  • Training
    • Certificate in AI/AN Health
    • NCRE Scholars Program
    • Native Children's Research Exchange Network & Conference
    • Native Elder Research Center
    • GUMSHOE Program
  • Journal
    • Join Our Mailing List
    • Current Volume
    • Submission Guidelines
    • Past Volumes
University Quick Links
  • Projects
  • Past Work
  • Circles of Care

Circles of Care Evaluation Technical Assistance Center

Artwork of Native shield with feathers and buffalo skull

Description


The Circles of Care initiative provides funding to plan, design, and assess the feasibility of implementing a culturally appropriate mental health service model for American Indian/Alaska Native (AI/AN) children with Serious Emotional/Behavioral Disturbances (SED) and their families. This program is funded by the Federal Center for Mental Health Services (CMHS, part of the Substance Abuse and Mental Health Services Administration, or SAMHSA) with additional support from the Indian Health Service and the National Institute of Mental Health.

It was out of the growing appreciation for the high prevalence of mental health problems among American Indian and Alaska Native children and adolescents, the inadequacy of existing services for these children, adolescents, and their families, and the need for community-based strategic planning to improve these services that Circles of Care was born. CMHS initially set the following goals for the initiative: 

  • to support the development of mental health service delivery models that are designed by AI/AN communities to achieve outcomes for their children that they chose for themselves;
  • to position tribal and urban Indian organizations advantageously for future service system implementation and development;
  • to strengthen tribal and urban Indian organizations’ capacity to evaluate their own service system’s effectiveness; and
  • to develop a body of knowledge to assist tribal and urban Indian organizations, and other policy makers and program planners from all child-serving systems in improving systems of care for the AI/AN population overall.

As a part of this effort, from 1998-2014 CMHS funded two organizations to provide technical assistance to the Circles of Care grantees. The National Indian Child Welfare Association (NICWA) was charged with providing technical assistance in regard to the program development activities of the grantees. CAIANH was charged with providing technical assistance in regard to the program evaluation activities of the grantees. The description of the evaluation provided here applies to the period of time that the Circles of Care Evaluation Technical Assistance Center (CoCETAC) supported grantees’ program evaluations (1998-2014).

The Circles of Care initiative required an Evaluation Plan that both informs and assesses the strategic planning process. The overarching goals of the Circles of Care Evaluation Plan were as follows: 

  • to provide a knowledge base for the planning effort;
  • to facilitate the process for developing the capacity for ongoing evaluation efforts;
  • to examine the feasibility of the service system models; and
  • to document and disseminate the results of the initiative.

Thus, evaluation activities are designed to assure that the final service delivery models developed through the Circles of Care initiative are consistent with community needs, developed through community consensus building, and practical and feasible given the resources available.

 

Evaluation components


The Circles of Care evaluation consisted of six specific components designed to achieve the four overarching goals noted above: 1) to describe the need for services within each participating community; 2) to develop a locally-relevant definition of SED; 3) to describe the current services available in these same communities; 4) to design a plan for measuring outcomes for the new service delivery model; 5) to assess the feasibility of this model system of care; and 6) to describe and assess the strategic planning process. 

 

Technical assistance activities


CoCETAC utilized a variety of technical assistance activities that are aimed at achieving the goals outlined above. The complex nature of the evaluation effort requires careful orientation to the evaluation goals and components, attention to each grantee’s priorities, coordination of efforts across grantee communities, and ongoing feedback and troubleshooting. Rather than rely on a single activity to achieve the goals for the Circles of Care evaluation, CoCETAC employed the following menu of activities:

  1. Grantee Meetings. In many ways, the Grantee Meetings were the core of CoCETAC’s technical assistance activities. CoCETAC used didactic sessions to orient the grantees to each of the evaluation components as well as evaluation-related tasks (such as working with Institutional Review Boards). These didactic sessions were invariably followed by a series of facilitated large and small group discussions (which include program staff, evaluators, and family representatives) aimed at increasing the grantees’ appreciation of the nature of these evaluation efforts, including both the specific expectations and individual flexibility embedded in these activities. CoCETAC also facilitated special sessions for evaluators that allow the grantees to share their experiences with each other and problem solve as a group around difficult issues that arise during their work. The Grantee Meetings were convened jointly by CoCETAC and NICWA, the program development technical assistance provider, with the goal of demonstrating and conveying the important relationships between evaluation and program development for this initiative.
  2. Evaluator Meetings. While not envisioned in CoCETAC’s Scope of Work, the need for meetings focusing specifically on evaluation became quite apparent. These meetings consisted mostly of large-group discussions that focus on developing a common set of domains that each of the grantees is expected to address in their evaluation efforts. These meetings also provided additional opportunities for discussion of exemplary evaluation practices and group problem solving. While intended primarily as an opportunity for the evaluators to meet as a group, the vast majority of grantees send at least one representative from their program staff to participate. Representatives from CMHS, IHS, and NICWA participate in these meetings as well.
  3. Site Visits. While the Grantee and Evaluator meetings were the primary avenues for group-oriented technical assistance to the grantees, the annual site visits to the grantees is the key activity for engaging the grantees in intensive, individualized discussions. Topics covered in these 1½ day visits include survey development, conducting focus groups and case studies, database development, data analysis and interpretation, as well as strategies for building support for the initiative and its evaluation effort among families, youth, service providers, and tribal leadership.
  4. Scheduled Technical Assistance Conference Calls. Every month, CoCETAC’s Associate Director conducted scheduled conference calls with each grantee’s Program Director and Evaluator. These calls were another opportunity for intensive, individualized discussions with grantees, and assisted the grantees in addressing specific evaluation issues as they arise, tracking progress, and identifying issues requiring further discussion.
  5. Ad Hoc Conference Calls and Email Exchanges. In addition to the scheduled conference calls, CoCETAC staff conducted a number of ad hoc conference calls and email exchanges to address a variety of issues that arise during the grantees’ evaluation activities. 
  6. Detailed Reviews and Critiques of Evaluation Reports. During the course of their programs, all of the grantees submitted draft reports that were reviewed in detail by CoCETAC staff. This provided the opportunity for CoCETAC to help the grantees further refine their evaluations and tailor their reports for both local and national use.
  7. Referral to and Provision of Key Reference Materials. CoCETAC assisted the grantees in identifying reference materials that support their evaluation activities. In addition, CoCETAC provides copies of many key reference materials that are highly relevant to all grantees.

This wide variety of evaluation activities allows CoCETAC to meet the diverse evaluation technical assistance needs of the Circles of Care grantees and supports their efforts to produce results relevant to their strategic planning efforts.

 

Publications



Freeman BJ, Bess G, Fleming CM, Novins DK. Transforming through leadership: a qualitative study of successful American Indian Alaska Native behavioral health leaders. BMC Public Health. 2019;19(1):1-11. doi: 10.1186/s12889-019-7600-9

Walden AL, West AE. American Indian researcher perspectives on qualitative inquiry about and within American Indian communities. Am J Community Psychol. 2019;64(1-2):83-95. doi: 10.1002/ajcp.12357

Hirchak KA, Murphy SM. Assessing differences in the availability of opioid addiction therapy options: rural versus urban and American Indian reservation versus nonreservation. J Rural Health. 2017;33(1):102-109. doi: 10.1111/jrh.12178

Julian DA, Smith T, Hunt RA. Ethical challenges inherent in the evaluation of an American Indian/Alaskan Native Circles of Care project. Am J Community Psychol. 2017;60(3-4):336-345. doi: 10.1002/ajcp.12192

Dennis MK, Momper SL. An urban American Indian health clinic's response to a community needs assessment. Am Indian Alsk Native Ment Health Res. 2016;23(5):15-33. doi: 10.5820/aian.2305.2016.15

Freeman BJ, Coll KM, Two Dogs R, Iron Cloud Two Dogs E, Iron Cloud E, Robertson P. The value of Lakota traditional healing for youth resiliency and family functioning. J Aggress Maltreatment Trauma. 2016;25(5):455-469. doi: 10.1080/10926771.2015.1079282

Olson CK. Yurok Circles of Care project community strengths and resources analysis, Humboldt State University; 2013.

Moghaddam JF. Assessing Need across Provider and Community Member Dimensions in an Urban American Indian Center [Thesis]. 2012. 

West AE, Williams E, Suzukovich E, Strangeman K, Novins D. A mental health needs assessment of urban American Indian youth and families. Am J Community Psychol. 2012;49(3-4):441-453. doi: 10.1007/s10464-011-9474-6

Novins DK, Bess G. Systems of mental health care for American Indian and Alaska Native children and adolescents. In P Spicer, P Farrell, MC Sarche, HE Fitzgerald (eds), American Indian and Alaska Native children and mental health: Development, context, prevention, and treatment. 2011:189-204.

Johnson CV, Bartgis J, Worley JA, Hellman CM, Burkhart R. Urban Indian voices: A community-based participatory research health and needs assessment. Am Indian Alsk Native Ment Health Res. 2010;17(1):49-70. doi: 10.5820/aian.1701.2010.49

Sahota PC. Community-based participatory research in American Indian and Alaska Native communities. Washington DC: NCAI Policy Research Center. 2010.

Novins D, Freeman B, Thurman PJ, et al. Principles for participatory research with American Indian and Alaska Native communities: lessons from the circles of care initiative. Paper presented at: Poster Presented at the 2006 Conference indigenous suicide prevention research and programs in Canada and the United States: setting a Collaborative Agenda, Albuquerque, New Mexico, 2006.

Coll KM, Mohatt G, LeMaster PL. Feasibility assessment of the service delivery model. Am Indian Alsk Native Ment Health Res. 2004;11(2):99-108. doi: 10.5820/aian.1102.2004.99

Duclos CW, Phillips M, LeMaster PL. Outcomes and Accomplishments of The Circles of Care Planing Efforts. Am Indian Alsk Native Ment Health Res. 2004;11(2):121-138. doi: 10.5820/aian.1102.2004.121

Freeman B, Dogs IC-T. Contextual issues for strategic planning and evaluation of systems of care for American Indian and Alaska Native communities: an introduction to Circles of Care. Am Indian Alsk Native Ment Health Res. 2004;11(2):1-29. doi: 10.5820/aian.1102.2004.1

Novins D, LeMaster P, Thurman PJ, Plested B. Describing Community Needs: Examples From the Circles of Care Initiative. Am Indian Alsk Native Ment Health Res. 2004;11(2):42-58. doi: 10.5820/aian.1102.2004.42

Novins DK, King M, Stone LS. Developing a plan for measuring outcomes in model systems of care for American Indian and Alaska Native children and youth. Am Indian Alsk Native Ment Health Res. 2004;11(2):88-98. doi: 10.5820/aian.1102.2004.88

Simmons TM, Novins DK, Allen J. Words have power:(re)-defining serious emotional disturbance for American Indian and Alaska Native children and their families. Am Indian Alsk Native Ment Health Res. 2004;11(2):59-64. doi: 10.5820/aian.1102.2004.59

Thurman PJ, Allen J, Deters PB. The Circles of Care evaluation: doing participatory evaluation with American Indian and Alaska Native communities. Am Indian Alsk Native Ment Health Res. 2004;11(2):139-154. doi: 10.5820/aian.1102.2004.139

Nebelkopf E, King J. A holistic system of care for Native Americans in an urban environment. J Psychoactive Drugs. 2003;35(1):43-52. doi: 10.1080/02791072.2003.10399992

Cross TL, Earle K, Solie HE-H, Manness K. Cultural Strengths and Challenges in Implementing a System of Care Model in American Indian Communities. Systems of Care: Promising Practices in Children's Mental Health, 2000 Series. 2000.

Funding Information

Sponsor: Federal Center for Mental Health Services (CMHS)​

Years Funded: 1998-2014

Principal Investigator: Douglas K. Novins, MD, CAIANH, University of Colorado

Centers for American Indian & Alaska Native Health

Colorado School of Public Health

CU Anschutz

Nighthorse Campbell Native Health Building

13055 East 17th Avenue

Mail Stop F800

Aurora, CO 80045


CAIANH
  • Contact Us
  • Join Our Mailing List
ColoradoSPH
  • Contact Us
  • Locations
  • Employment
  • Accreditation
Information for
  • Prospective Students
  • Current Students
  • Alumni
  • Faculty & Staff
  • Contact Us
  • Website Feedback
  • CU System
  • Privacy Policy
  • Terms of Use
  • Accessibility
  • Accreditation
  • Employment
  • Give Now
 

© 2023 The Regents of the University of Colorado, a body corporate. All rights reserved.

Accredited by the Higher Learning Commission. All trademarks are registered property of the University. Used by permission only.

CMS Login

Apply

Visit

Give

Opens in a new window Opens document in a new window