Transforming Psychological Assessment Tools for the Online Age

Measuring social and psychological constructs effectively is critical to research and practice. Many methods are available – some briefer than others – some more qualitative than quantitative. Utilising standardised assessments, with proven reliability and validity, gives the necessary ‘kite’ mark for their optimal use.

At CATS we have our own bespoke psychological assessment measures which have been developed in a research context to investigate lifespan vulnerability and adversity. Thus measures of childhood neglect/abuse, adult adversity and life events, attachment style and parenting have been standardised by our team and are available as interviews with predetermined scoring procedures for use in both research and practice. In some instances we have briefer questionnaires to do a similar job (in childhood and attachment) but have sought ways of mimicking aspects of interviews with their attention to details of context, meaning and timing. For this we have sought novel ways of making measures digital and web-based.

Benefits of online measuring tools:

  • Ease of use & affordability
  • Anonymity
  • Scaleability
  • Robustness
  • Evidence-based

Who are these tools suitable for

  • Schools & Educators
  • Academic Researchers
  • Mental Health Practitioners
  • Social Work Agencies
  • Health Agencies

Our projects – the past & the future

We have recently partnered with Prof Mike Smith at Youth in Mind in order to develop online assessment tools which can deliver some of the benefits of interview approaches, taking less time and resource.

Along with Prof Robert Goodman at Kings College London, child and adolescent diagnostic tools such as the Strengths and Difficulties Questionnaire (SDQ) and clinical interview (Difficulties and Well-Being Assessment – DAWBA) were successfully developed to ascertain national rates of psychiatric disorder. These were further utilised for vulnerable children in care, and are now being rolled out to schools in the context of new policies on mental health workers in schools.

The online system provided a much more detailed and time-based approach to assessment, was easy to roll out in schools and similar locations and could provide an instant summary report as well as collecting anonymised group data. The online systems are robust and secure.

We worked with Mike Smith on our CLEAR – Computerised Life Events Assessment Records project. The project, which was ESRC funded, developed a complex online tool for assessing life event stress. This worked almost as well as the interview approach and much better than simple checklist questionnaires. We are currently collaborating to develop our Q pack, which utilises SDQ, for children in care and to extend our use of the CLEAR life events assessment.

  • Bifulco, A., Kagan, L., Spence, R., Nunn, S., Bailey-Rodriquez, D., Hosang, G. M., . . . Fisher, H. L. (2019). Characteristics of severe life events, attachment style and depression – Using an online approach. The British Journal of Clinical Psychology, 1-13. doi:10.1111/bjc.1221.
  • Bifulco, A., Spence, R., Nunn, S., Kagan, L., Rodriguez D., Hosang G.M., . . . Fisher, H. L. (2019). The Computerised Life Events and Assessment Record (CLEAR) online measure of life events: reliability, validity and association with depression. JMIR Mental health, 6(1), e10675. doi:10.2196/10675.
  • Goodman, R. (1999). The extended version of the Strengths and Difficulties Questionnaire as a guide to child psychiatric caseness and consequent burden. Journal of Child Psychology and Psychiatry and Allied Disciplines, 40(5), 791-799.
  • Meltzer, H., Gatward, R., Corbin, T., Goodman, R., & Ford, T. (2002). The mental health of young people looked after by local authorities in England. HMSO.
  • Meltzer, H., Gatward, R., Goodman, R., & Ford, T. (2000). Mental health of children and adolescents in Great Britain. Retrieved from Office for National Statistics: https://www.ncbi.nlm.nih.gov/pubmed/12745331.
  • Spence, R., Bunn, A., Nunn, S., Hosang, G. M., Kagan, L., Fisher, H. L., . . . Bifulco, A. (2015). Measuring Life Events and Their Association With Clinical Disorder: A Protocol for Development of an Online Approach. JMIR Res Protoc, 4(3), e83. doi:10.2196/resprot.4085

CLEAR

The Computerised Life Events Assessment Record (CLEAR) identifies recent life event changes and longer-term problems by users of the tool who provide all their responses online. It also assesses depression, wellbeing and attachment style. Other brief assessments can be added to the CLEAR site for a more comprehensive assessment. Severe (or negative) life events are part of the causal picture for onset of depression, anxiety and other disorders. Not only the events themselves and their timing, but also related dimensions of loss, danger, trauma etc are captured. An automatic brief report is produced for the user, and group data can be captured.

Researchers can use this tool as an assessment of stressors when examining psychological disorder in adolescents or adults. Clinicians or social workers can use it as part of their assessment in work in mental health or with children and families to ascertain individual or family stress. It will enable an overview of the domain of such stress (eg work, finance, parenting etc) as well as its likely impact on the individual.

We have tested both the reliability (consistency over time) and validity (how similar to more intensive interview approaches) and found both acceptable (Bifulco, Spence, et al., 2019). We have now published findings on the CLEAR tool in relation to depression (Bifulco, Kagan, et al., 2019; Spence et al., 2015), and our analyses on positive events and wellbeing, and life events in student achievement are being submitted for publication. We found the tool easy to administer and analyse.

  • Spence, R., Bunn, A., Nunn, S., Hosang, G. M., Kagan, L., Fisher, H. L., . . . Bifulco, A. (2015). Measuring Life Events and Their Association With Clinical Disorder: A Protocol for Development of an Online Approach. JMIR Res Protoc, 4(3), e83. doi:10.2196/resprot.4085.
  • Bifulco, A., Kagan, L., Spence, R., Nunn, S., Bailey-Rodriquez, D., Hosang, G. M., . . . Fisher, H. L. (2019). Characteristics of severe life events, attachment style and depression – Using an online approach. The British Journal of Clinical Psychology, 1-13. doi:10.1111/bjc.1221.
  • Bifulco, A., Spence, R., Nunn, S., Kagan, L., Rodriguez D., Hosang G.M., . . . Fisher, H. L. (2019). The Computerised Life Events and Assessment Record (CLEAR) online measure of life events: reliability, validity and association with depression. JMIR Mental health, 6(1), e10675. doi:10.2196/10675.

The use of CLEAR is given a unique login number to ensure privacy and anonymity. The instructions for use are onsite and easy. There are instructions and video guides throughout to clarify the meaning of questions. Three section of lifestyle, relationships and health are covered with subdivisions for 11 different areas of life. In each, users are asked if there have been changes in the last year, the nature of these changes and how stressful these were for them etc. The measure takes about an hour to complete, but breaks can be taken at any time, and this is done in the privacy of one’s own home. At the end a summary report is produced to alert the user to their key life events, their depression and attachment style. If relevant they will be advised to seek GP or clinical help.

Please contact the CATS team if you wish to use this measure as a university-based researcher or as a clinician or other practitioner. The measure will require some adaptation to your precise context (eg title, logo’s, consent etc) and the automatic summary report can be personalised for the particular project or service. We can undertake this for you and would need to make some charge. We can also provide a long list of logins (randomised letters). The tailored CLEAR can then be used. The group data captured can be downloaded for your (e.g. to SPSS) or we can offer a service to analyse it for you.

Q pack (in development)

This is a questionnaire combining the Strengths and Difficulties tool, the Vulnerable Attachment Style Questionnaire and a life events checklist. It is being used with children in care by our partners St Christopher’s Fellowship and Action for Children. This enables a monitoring of child/young persons symptoms, attachment style and life events stress. Whilst currently a paper-based approach we are seeking to develop it into an online tool by combining with Youth in Mind SDQ approach.

Whilst currently set up for children or young people in residential care, fostering or 6+ hostels, it could also be used in schools or in clinical settings to determine risk or resilience.

We have used the Q pack over a number of years in residential care and fostering and found it useful in assessing child risk and resilience (Bifulco A., Jacobs, Ilan-Clarke, Spence R., & Oskis, 2017; Jacobs, Boyce, Ilan-Clarke, & Bifulco, 2019) as well as indicating positive change over time.

  • Bifulco A., Jacobs, C., Ilan-Clarke, Y., Spence R., & Oskis, A. (2017). Adolescent Attachment Style in Residential Care: The Attachment Style Interview and Vulnerable Attachment Style Questionnaire. British Journal of Social Work 47, 1870–1883. doi:10.1093/bjsw/bcw117.
  • Jacobs, C., Boyce, N., Ilan-Clarke, Y., & Bifulco, A. (2019). Assessing attachment style in traumatised adolescents in residential care: A case study approach. Maltrattamento e Abuso all’Infanzia, 21, 87-103.

The measure is easy to complete and takes about 12 minutes. At present the measure is paper-based and completed by young person, and carer. It is then scored and analysed by the social worker or residential care home worker. The group data is analysed by CATS research team. We running briefings in the use of the tool for both social workers, residential care workers and foster parents.

Please contact our CATS office if you want to use this tool. Our online version is currently under development and we can let you know when it will also be available. Like the SDQ (see Youth in Mind above) it will then represent a simple online anonymised access with brief report and data capture.