You are here

Measuring Uncertainty Qualitatively (MUQ)

The Measuring Uncertainty Qualitatively (MUQ) Working Party is developing reserve uncertainty thinking beyond percentiles and bootstrap style techniques

The 2014 GIROC survey (GIROC UK Reserving Survey 2014) highlighted the wide differences in approaches to measuring and communicating uncertainty in reserves that are currently employed in the UK and identified that this was an area the profession would benefit from focusing on. 

The MUQ working party was set up in the wake of this finding and has undertaken work in two stages.

Outputs:

Stage 1 - Bringing together work that has been done before

Stage 1 was carried out in 2015 and focused on reviewing the various qualitative methods already employed. This work has brought together information on current market practice (both in the UK and around the world) and previous research into one place - please see the reference materials section below.

Stage 2 - Development of a reserve uncertainty framework

Stage 2 commenced in 2016 and used the Stage 1 research to propose ideas for how the measurement of reserve uncertainty could be moved forward in the UK.  As a result, the MUQ working party has proposed a framework for assessing uncertainty beyond statistical techniques. 

The purpose of this framework is to provide actuaries with a best practice approach for thinking about aspects of uncertainty that are not covered by quantitative factors such as percentiles.  This will enable actuaries to take a wider view of uncertainty and identify qualitative issues that may be missed in a purely quantitative approach.  It has been developed with a focus on reserving uncertainty, but the principles could be applied to many other areas of actuarial work.  

Feedback from members of the profession is encouraged to help develop the framework further.  Please send feedback to professional.communities@actuaries.org.uk

For further information, including additional reading, resources on individual framework elements and ideas on how to deal with specific elements of uncertainty, please the ‘Reference Materials’ section below.

Chair Keith Brown
Membership 17
Established 2014

Reference Materials:

Contents

  1. Experience from other countries
    1. Australia
    2. Ireland
  2. Additional resources for framework elements
    1. Internal/Process – Data
    2. Internal/Process – Control Risk
    3. Internal/Process - Communication
    4. Human – Expert Opinion
    5. Statistical – Model risk
  3. Additional useful resources
    1. Beyond ‘Bootstrap’ measures of uncertainty
    2. Previous GIROC papers

1. Experience from other countries

1.1 Australia

1.2 Ireland

2. Additional resources for framework elements

2.1 Internal/Process – Data

  • Data uncertainty matrix - The Forest of Brocéliande has a Knowledge Quality Assessment module, which provides an uncertainty matrix.  This can serve as a high-level qualitative register for data uncertainties in a model and was used to inform the MUQ framework.
  • IAN on data for "CP73" – guidance from the Society of Actuaries in Ireland on questions a Signing Actuary could ask to help form a view on whether data used for providing a Statement of Actuarial Opinion is sufficiently accurate, complete and appropriate.

2.2 Internal/Process – Control Risk

  • S&P GI ERM Rating Criteria – pages 22-25 cover risk controls assessment from a general insurance perspective, where the extent (or level of detail) that this is in place may cause additional (or lower) degree of uncertainty on the reserves.

2.3 Internal/Process - Communication

Language used in communicating uncertainties:

  • The Intergovernmental Panel on Climate Change has various types of reports on climate change demonstrating good use of language: IPCC_summary-for-policymakers

2.4 Human – Expert Opinion

  • Expert judgement elicitation
    • Dr Joseph Lo, Dr Ed Tredger and Bernadette Hlavka discuss the tricks and possible pitfalls when conducting a successful expert judgment elicitation meeting, Article from The Actuary, 2013
  • Uncertain factors surrounding expert judgements:  Expert Judgement WP, 2015
    • Expert judgement process has long been in use in actuarial work, but it has lacked transparency. This paper aims to provide a practical framework regarding expert judgement processes, and how those processes may be validated.

2.5 Statistical – Model risk

  • ‘Ersatz model tests’ – Presented at the 2016 reserving seminar, it looks at ways of testing stochastic models, with an aim to developing deeper understanding of when and why different models are more or less appropriate.

Effectiveness of methods working party

3. Additional useful resources

3.1 Beyond 'Bootstrap' measure of uncertainty

3.2

More information on the work of GIROC.

Related documents

Contact Details

If you want more information about this research working party please contact the Communities Team.

professional.communities@actuaries.org.uk

Filter or search events

Start date
E.g., 23/09/2021
End date
E.g., 23/09/2021

Events calendar

  • Spaces available

    A panel discussion with investment representatives discussing the impact of investment on climate change and vice versa.

    This event is part of The Road to Glasgow: IFoA’s Sustainability Thought Leadership Series which will provide a platform for prominent contributors so that our members and others can understand the perspectives of a wide range of parties interested in the climate debate.

  • Spaces available

    Climate change is one of the greatest risks facing our world today. Addressing it will require multi-faceted solutions. Through this panel session, we will explore the different levers that can be used to meet net-zero targets including climate science and data, government engagement, and mobilising green finance.

  • Spaces available

    How governments and societies collectively respond to the climate crisis is increasingly being analysed through the lens of intergenerational fairness. Our expert panel will assess and explore the extent to which inter-generous fairness is being considered in climate policies, the lessons from those countries who are leading the way on fairness and justice in climate interventions, and more.

  • The Growth Mindset for Actuaries

    13 October 2021 - 8 December 2021

    Spaces available

    This practical course is aimed at actuaries at any stage of their career who want to develop their own growth mindset and apply it to their work setting and personal or professional lifelong learning. The content of the course builds on the lecture given by Dr Helen Wright on Growth Mindset as part of the President’s 2021 Lecture series, and will be delivered over a period of 2 months, from mid-October to early December.

  • Spaces available

    The climate crisis and the degradation of our planet will affect societies everywhere. How we address these threats will require solutions that transcend borders. As a global profession, the actuarial community is well-placed to consider and propose effective risk management solutions to help manage the climate crisis.

  • Spaces available

    Content will be aimed at all actuaries looking to understand the issues surrounding mental health in insurance and in particular those looking to ensure products and processes widen access for, and are most useful to, those experiencing periods of poor mental health.