You are here

Measuring Uncertainty Qualitatively (MUQ)

The Measuring Uncertainty Qualitatively (MUQ) Working Party is developing reserve uncertainty thinking beyond percentiles and bootstrap style techniques

The 2014 GIROC survey (GIROC UK Reserving Survey 2014) highlighted the wide differences in approaches to measuring and communicating uncertainty in reserves that are currently employed in the UK and identified that this was an area the profession would benefit from focusing on. 

The MUQ working party was set up in the wake of this finding and has undertaken work in two stages.

Outputs:

Stage 1 - Bringing together work that has been done before

Stage 1 was carried out in 2015 and focused on reviewing the various qualitative methods already employed. This work has brought together information on current market practice (both in the UK and around the world) and previous research into one place - please see the reference materials section below.

Stage 2 - Development of a reserve uncertainty framework

Stage 2 commenced in 2016 and used the Stage 1 research to propose ideas for how the measurement of reserve uncertainty could be moved forward in the UK.  As a result, the MUQ working party has proposed a framework for assessing uncertainty beyond statistical techniques. 

The purpose of this framework is to provide actuaries with a best practice approach for thinking about aspects of uncertainty that are not covered by quantitative factors such as percentiles.  This will enable actuaries to take a wider view of uncertainty and identify qualitative issues that may be missed in a purely quantitative approach.  It has been developed with a focus on reserving uncertainty, but the principles could be applied to many other areas of actuarial work.  

Feedback from members of the profession is encouraged to help develop the framework further.  Please send feedback to professional.communities@actuaries.org.uk

For further information, including additional reading, resources on individual framework elements and ideas on how to deal with specific elements of uncertainty, please the ‘Reference Materials’ section below.

Chair Keith Brown
Membership 17
Established 2014

Reference Materials:

Contents

  1. Experience from other countries
    1. Australia
    2. Ireland
  2. Additional resources for framework elements
    1. Internal/Process – Data
    2. Internal/Process – Control Risk
    3. Internal/Process - Communication
    4. Human – Expert Opinion
    5. Statistical – Model risk
  3. Additional useful resources
    1. Beyond ‘Bootstrap’ measures of uncertainty
    2. Previous GIROC papers

1. Experience from other countries

1.1 Australia

1.2 Ireland

2. Additional resources for framework elements

2.1 Internal/Process – Data

  • Data uncertainty matrix - The Forest of Brocéliande has a Knowledge Quality Assessment module, which provides an uncertainty matrix.  This can serve as a high-level qualitative register for data uncertainties in a model and was used to inform the MUQ framework.
  • IAN on data for "CP73" – guidance from the Society of Actuaries in Ireland on questions a Signing Actuary could ask to help form a view on whether data used for providing a Statement of Actuarial Opinion is sufficiently accurate, complete and appropriate.

2.2 Internal/Process – Control Risk

  • S&P GI ERM Rating Criteria – pages 22-25 cover risk controls assessment from a general insurance perspective, where the extent (or level of detail) that this is in place may cause additional (or lower) degree of uncertainty on the reserves.

2.3 Internal/Process - Communication

Language used in communicating uncertainties:

  • The Intergovernmental Panel on Climate Change has various types of reports on climate change demonstrating good use of language: IPCC_summary-for-policymakers

2.4 Human – Expert Opinion

  • Expert judgement elicitation
    • Dr Joseph Lo, Dr Ed Tredger and Bernadette Hlavka discuss the tricks and possible pitfalls when conducting a successful expert judgment elicitation meeting, Article from The Actuary, 2013
  • Uncertain factors surrounding expert judgements:  Expert Judgement WP, 2015
    • Expert judgement process has long been in use in actuarial work, but it has lacked transparency. This paper aims to provide a practical framework regarding expert judgement processes, and how those processes may be validated.

2.5 Statistical – Model risk

  • ‘Ersatz model tests’ – Presented at the 2016 reserving seminar, it looks at ways of testing stochastic models, with an aim to developing deeper understanding of when and why different models are more or less appropriate.

Effectiveness of methods working party

3. Additional useful resources

3.1 Beyond 'Bootstrap' measure of uncertainty

3.2

More information on the work of GIROC.

Related documents

Contact Details

If you want more information about this research working party please contact the Communities Team.

professional.communities@actuaries.org.uk

Filter or search events

Start date
E.g., 03/07/2022
End date
E.g., 03/07/2022

Events calendar

  • Spaces available

    The Actuaries’ Carbon Collaboration (ACC) is a group of actuaries and other professionals working towards a coherent understanding of the issues around greenhouse gas (GHG) emissions by considering them in an actuarial context.

    Emissions into and absorption from the atmosphere can be modelled in the same way that we model variable cash flows, including the impacts of uncertain quantities and timing. The ACC’s work investigates both the insights that this analogy provides and its limitations.

  • Spaces available

    This talk presents the work of the Data and Modelling workstream of the IFoA Mental Health Working Party. We explore data and modelling considerations in the risk assessment and underwriting of mental health conditions. We will also consider how newer risk factors and improved data availability might open up opportunities for additional underwriting and product designs.

  • Pensions Dashboards

    12 July 2022

    Spaces available

    As the industry prepares for Pensions Dashboards, we have lined up a specialist panel who is at the heart of the ongoing work in this area. Hear the latest developments, future proposals and have the opportunity to put forward your questions to our speakers.

  • Spaces available

    Crisis management, the process by which an organization deals with a disruptive, sudden and unexpected event that threatens to harm the organization or its stakeholders, has become a key concern in the light of the Pandemic and Geopolitical risks.

  • Spaces available

    Artificial Intelligence is heralded as a game-changer to the ways industries use data, with the insurance industry especially looking to embrace these new technologies. This brings many questions – not least around data privacy, ethics, potential biases and so on – with regulators increasingly interested in developments.

    Moreover, whilst the stunning progress of specific technologies such as Watson and Alphazero made headlines some years ago, current commercial applications of AI remain far from mature.

  • Spaces available

    This webinar will cover:

    • Some background on the risks of misselling in an ESG context, including the DWS case

    • Achieving positive impact is a strong antidote to the risks of greenwashing or ESG misselling, however this risks having a tension with fiduciary responsibilities

    • This tension can be resolved with a concept called Universal Ownership

    • Under Universal ownership, investors have an appetite to make a loss in order to achieve positive impact, and yet still have no compromise on their fiduciary responsibilities

  • Spaces available

    This session will focus on the transformation roadmap of the healthcare sector in KSA and the role of actuarial capabilities in enhancing its evolution to the desired end stage as per the objectives of the Vision 2030. The discussion will focus how the system has evolved so far and shed light on  the expected future changes. Through examining  the transformation, we will highlight how the sector is and can use actuarial  expertise to not only assist with this transformation but also use basic actuarial principles to identify the key risks and their respective mitigation strategies.

  • Spaces available

    The purpose of this research paper is to explore enterprise risk management lessons which can be learnt from the Covid-19 pandemic in preparation for potential future pandemics as well as other “gray rhino” or “black swan” events. This paper is not intended to be an all-encompassing solution to the issues presented by Covid-19; rather, the content has been provided to help drive discussions regarding how risk management processes may need to evolve in line with the dynamic nature of the underlying risks that they sometimes need to capture.

  • Spaces available

    The IFoA's Infrastructure Working Party, led by Chris Lewin, will present its new introductory guide to infrastructure investment, which will be published on the IFoA web-site prior to the webinar.   Those readers whose institutions have already taken the plunge into infrastructure will know that it is a highly complex and diverse field of activity.   This guide does not explore all the matters which investors take into account, but it does discuss many of the more important points, including the risks and past returns, benchmarking, and ESG and SDG considerations.    Attendees will be invi

  • Spaces available

    Health contributes to happiness at the personal, family, community and societal level.  Health, importantly underpins all our economic security. This talk will explore the drivers of our health, the measurement of health and the steps we can take to improve health – most of which lie outside the NHS.