You are here

Measuring Uncertainty Qualitatively (MUQ)

The Measuring Uncertainty Qualitatively (MUQ) Working Party is developing reserve uncertainty thinking beyond percentiles and bootstrap style techniques

The 2014 GIROC survey (GIROC UK Reserving Survey 2014) highlighted the wide differences in approaches to measuring and communicating uncertainty in reserves that are currently employed in the UK and identified that this was an area the profession would benefit from focusing on. 

The MUQ working party was set up in the wake of this finding and has undertaken work in two stages.

Outputs:

Stage 1 - Bringing together work that has been done before

Stage 1 was carried out in 2015 and focused on reviewing the various qualitative methods already employed. This work has brought together information on current market practice (both in the UK and around the world) and previous research into one place - please see the reference materials section below.

Stage 2 - Development of a reserve uncertainty framework

Stage 2 commenced in 2016 and used the Stage 1 research to propose ideas for how the measurement of reserve uncertainty could be moved forward in the UK.  As a result, the MUQ working party has proposed a framework for assessing uncertainty beyond statistical techniques. 

The purpose of this framework is to provide actuaries with a best practice approach for thinking about aspects of uncertainty that are not covered by quantitative factors such as percentiles.  This will enable actuaries to take a wider view of uncertainty and identify qualitative issues that may be missed in a purely quantitative approach.  It has been developed with a focus on reserving uncertainty, but the principles could be applied to many other areas of actuarial work.  

Feedback from members of the profession is encouraged to help develop the framework further.  Please send feedback to professional.communities@actuaries.org.uk

For further information, including additional reading, resources on individual framework elements and ideas on how to deal with specific elements of uncertainty, please the ‘Reference Materials’ section below.

Chair Keith Brown
Membership 17
Established 2014

Reference Materials:

Contents

  1. Experience from other countries
    1. Australia
    2. Ireland
  2. Additional resources for framework elements
    1. Internal/Process – Data
    2. Internal/Process – Control Risk
    3. Internal/Process - Communication
    4. Human – Expert Opinion
    5. Statistical – Model risk
  3. Additional useful resources
    1. Beyond ‘Bootstrap’ measures of uncertainty
    2. Previous GIROC papers

1. Experience from other countries

1.1 Australia

1.2 Ireland

2. Additional resources for framework elements

2.1 Internal/Process – Data

  • Data uncertainty matrix - The Forest of Brocéliande has a Knowledge Quality Assessment module, which provides an uncertainty matrix.  This can serve as a high-level qualitative register for data uncertainties in a model and was used to inform the MUQ framework.
  • IAN on data for "CP73" – guidance from the Society of Actuaries in Ireland on questions a Signing Actuary could ask to help form a view on whether data used for providing a Statement of Actuarial Opinion is sufficiently accurate, complete and appropriate.

2.2 Internal/Process – Control Risk

  • S&P GI ERM Rating Criteria – pages 22-25 cover risk controls assessment from a general insurance perspective, where the extent (or level of detail) that this is in place may cause additional (or lower) degree of uncertainty on the reserves.

2.3 Internal/Process - Communication

Language used in communicating uncertainties:

  • The Intergovernmental Panel on Climate Change has various types of reports on climate change demonstrating good use of language: IPCC_summary-for-policymakers

2.4 Human – Expert Opinion

  • Expert judgement elicitation
    • Dr Joseph Lo, Dr Ed Tredger and Bernadette Hlavka discuss the tricks and possible pitfalls when conducting a successful expert judgment elicitation meeting, Article from The Actuary, 2013
  • Uncertain factors surrounding expert judgements:  Expert Judgement WP, 2015
    • Expert judgement process has long been in use in actuarial work, but it has lacked transparency. This paper aims to provide a practical framework regarding expert judgement processes, and how those processes may be validated.

2.5 Statistical – Model risk

  • ‘Ersatz model tests’ – Presented at the 2016 reserving seminar, it looks at ways of testing stochastic models, with an aim to developing deeper understanding of when and why different models are more or less appropriate.

Effectiveness of methods working party

3. Additional useful resources

3.1 Beyond 'Bootstrap' measure of uncertainty

3.2

More information on the work of GIROC.

Related documents

Contact Details

If you want more information about this research working party please contact the Communities Team.

professional.communities@actuaries.org.uk

Filter or search events

Start date
E.g., 18/04/2021
End date
E.g., 18/04/2021

Events calendar

  • Spaces available

    Climate-Related Risk - This free to view webinar on Climate-Related Risk is the first in a series focusing on some of the ‘Hotspots’ identified in the JFAR Risk Perspective bringing the Risk Perspective to life with practical illustrations and insights from subject experts from the IFoA and other Regulators

  • Spaces available

    Recent decades have seen institutions, such as employers and financial services, give people more choice and flexibility, but these freedoms have come with more responsibilities. Individuals are now responsible for managing more of their own financial risks, from ensuring they put enough money into their pension to securing affordable protection to be financially resilient.

  • Actuarial Innovation in the COVID-19 era

    This event is online. 
    26 April 2021 - 7 May 2021
    Spaces available

    Join us for this brand new IFoA webinar weries comprising of a fortnight of webinars, panel sessions and a hackathon, that showcase the range of ways in which the actuarial profession has added value, in the public interest, to the understanding and management of the current and future pandemics through insight and learning.

  • Spaces available

    This event is now temporarily closed on Monday 26 April, but the session will be repeated on Tuesday 27 April, 09.00-10.30. Please click here to register your place. 

    Actuaries have a lot to offer biodiversity management over the next decade as the world develops more depth to its response to this global challenge. This sessional offers an opportunity to learn about this emergent risk, to contribute to our thinking as a profession and help us develop the next steps forward.

  • Europe Town Hall

    28 April 2021

    Spaces available

    IFoA Immediate Past President John Taylor would like to invite you to the Institute and Faculty of Actuaries’ (IFoA) virtual Europe Town Hall, hosted by John Taylor with IFoA Council Members Alan Rae, Jennifer Hartley, Maribel Vasquez Flores and IFoA Chief Executive, Stephen Mann.

  • Spaces available

    Sessional Meeting - Free to viewMis-estimation risk is a key element of demographic risk, and past work has focused on mis-estimation risk on a run-off basis.  However, this does not meet the requirements of regulatory regimes like Solvency II, which demands that capital requirements are set through the prism of a finite horizon like one year.  This paper presents a value-at-risk approach to mis-estimation risk suitable for Solvency II work

  • Finance & Investment Virtual Conference 2021

    Available to watch globally in May.
    10-12 May 2021
    Spaces available

    This year's Finance and Investment Virtual Conference takes on the timely theme of ‘resilience’, something we have all learnt a lot more about in the last year! Our diverse range of talks will explore the theme of resilience in a variety of ways including in building robust investment portfolios, in the incorporation of ESG factors, in govern

  • Spaces available

    This talk will explore the potential benefits that wearable tech can bring to health & protection insurers and their customers. The traditional approach of integrating wearables into insurance has largely focused on measuring steps and using rewards-based incentive programs to encourage more activity.

  • Spaces available

    Join us for this talk with Professor Sir Adrian Smith as part of the 'Dr Patrick Poon Presidential Speaker Series'. Professor Smith joined The Alan Turing Institute as Institute Director and Chief Executive in September 2018. In November 2020, he became President of the Royal Society, in addition to his leadership of the Turing. He is also a member of the government's AI Council, which helps boost AI growth in the UK and promote its adoption and ethical use in businesses and organisations across the country. He received a knighthood in the 2011 New Year Honours list.

     

  • CILA 2021

    Available to watch globally in May.
    19-21 May 2021
    Spaces available

    We continue to live in a world of global uncertainty. Survival depends on our ability to simultaneously navigate through the diverse root-causes, ranging from: the consequences of Climate Change; on-going financial consequences of the COVID pandemic; or self-imposed changes in regulatory requirements and accounting standards.

  • Pensions Conference 2021

    Online webinar series
    16-22 June 2021
    Spaces available

    Welcome to the programme for our 2nd Virtual Pensions Conference. This year's conference features 11 webinars offering members and non-members the opportunity to get up to date content from leading experts in the pension industry. There will also be opportunity to ask questions and contribute to the discussion.