The Value of Evaluation

Home>News and Events>The Value of Evaluation - Fulbright

By Tim Carey, 2017 Fulbright Northern Territory Scholar

Being awarded a Fulbright Senior Scholarship enabled me to spend four months at the Center for Behavioral Health Innovation (BHI), Department of Clinical Psychology, Antioch University New England. The University is a very small postgraduate university of approximately 1000 students. The Department of Clinical Psychology is one of five departments at the University and offers a Doctorate of Clinical Psychology. The University has a very strong and explicit social justice focus with many messages on noticeboards about the importance of equality, acceptance, and understanding.

It was a privilege to spend time at BHI learning about the work that they do. They are a small center led by two of the faculty members of the Department of Clinical Psychology and six other staff. They partner with organisations in rural and underserved communities helping these organisations to establish systems of ongoing monitoring and feedback in order to promote the effectiveness of the services they deliver. During my time there BHI staff were working with schools, health centres, and philanthropic organisations on various projects. The evaluation context in the US is quite different from Australia with the Federal Government in the US mandating that a proportion of funding (commonly 20%) for services and programs be allocated to the evaluation of those interventions. Funding for BHI largely comes from this requirement.

While working at BHI, we developed a project to investigate the factors that might be important in establishing and sustaining ongoing monitoring and feedback within an organisation. I interviewed 15 people from organisations with whom BHI had partnered at some time over approximately an eight year period. The people I interviewed had different roles within the organisations from leaders to service providers but they all spoke emphatically about the value of evaluation. They explained that, through ongoing monitoring and evaluation, they could make sure that the help that was being delivered really was experienced as help by the service recipients. They also emphasised the importance of having buy-in from the leadership of the organisation and of embedding systems within daily routines so that “the right thing to do is the easy thing to do”. Unanimously, interview participants spoke of the importance of the data they collected in terms of being able to engage with communities, communicate with funders, and “tell stories that touch people’s hearts as well as their heads”.

As a complementary aspect to my project I was fortunate to be able to speak with Dr Richard McKeon, who is Chief of the Suicide Prevention Branch of the Substance Abuse and Mental Health Services Administration. Like the people I interviewed, Dr McKeon emphasised the importance of collecting data and basing program decisions upon the data that are collected. Dr McKeon argued strongly that having data to highlight an issue can become an important way of persuading people that something needs to be done and to answer basic questions.

The work at BHI and the learnings from the project I conducted have important policy and practice implications for service delivery in remote health settings of Australia.

In addition to the work at BHI I had the opportunity to participate in the Oz to Oz program at Kansas State University. I was hosted by their Master of Public Health program and got to meet senior university personnel and learn about some aspects of their approach to public health. Their public health program is interesting because it is not delivered within a medical program or department. While I was at Kansas State University I delivered a colloquium titled “Defining Health: Issues and solutions”. I provided a related colloquium back at Antioch University titled “Health as Control: An organising principle for the biopsychosocial model”. These speaking engagements enabled me to discuss ideas from my recently published book “Patient-Perspective Care: A new paradigm for health systems and services”. A further invitation to speak to residents and students of the Family Medicine Residency at the Eastern Maine Medical Center and to staff of Acadia Hospital in Bangor, ME allowed me to explore these ideas in greater detail with a talk titled “Improving Health in Rural and Remote Contexts with Patient-Perspective Care”. Further teaching opportunities arose with three classes provided in the Brief Psychotherapies course of the Doctor Clinical Psychology program at Antioch.

I was fortunate to have my family accompany me to Keene, NH for the duration of the Fulbright Scholarship. I think we benefitted just as much personally as I did professionally from the time we spent involved in the community life of Keene. It was wonderful to experience occasions such as Halloween, Thanksgiving, and Christmas in the US with new-found friends and their families. Among other things, we picked blueberries and apples, carved pumpkins, turkey trotted at our son’s elementary school, trick-or-treated around the neighbourhood, and did some snow tubing. We also discovered that the saying “I’ll be there with bells on” originated in New England and we had many interesting discussions about New Hampshire’s state motto: Live free or die.

It’s an enormous privilege to become a Fulbright Scholar and tremendously humbling to be a part of the legacy Senator Fulbright started more than half a century ago. There was a real sense as I was undertaking the Fulbright Program that I was “living the dream”. There wasn’t ever a point when I wanted the dream to end. I still don’t. Thankfully, as a Fulbright Alumni, the dream can live on.

Share your Fulbright story too! Find out how by writing to: alumni@fulbright.org.au