You are here

Measuring Uncertainty Qualitatively (MUQ)

The Measuring Uncertainty Qualitatively (MUQ) Working Party is developing reserve uncertainty thinking beyond percentiles and bootstrap style techniques

The 2014 GIROC survey (GIROC UK Reserving Survey 2014) highlighted the wide differences in approaches to measuring and communicating uncertainty in reserves that are currently employed in the UK and identified that this was an area the profession would benefit from focusing on. 

The MUQ working party was set up in the wake of this finding and has undertaken work in two stages.

Outputs:

Stage 1 - Bringing together work that has been done before

Stage 1 was carried out in 2015 and focused on reviewing the various qualitative methods already employed. This work has brought together information on current market practice (both in the UK and around the world) and previous research into one place - please see the reference materials section below.

Stage 2 - Development of a reserve uncertainty framework

Stage 2 commenced in 2016 and used the Stage 1 research to propose ideas for how the measurement of reserve uncertainty could be moved forward in the UK.  As a result, the MUQ working party has proposed a framework for assessing uncertainty beyond statistical techniques. 

The purpose of this framework is to provide actuaries with a best practice approach for thinking about aspects of uncertainty that are not covered by quantitative factors such as percentiles.  This will enable actuaries to take a wider view of uncertainty and identify qualitative issues that may be missed in a purely quantitative approach.  It has been developed with a focus on reserving uncertainty, but the principles could be applied to many other areas of actuarial work.  

Feedback from members of the profession is encouraged to help develop the framework further.  Please send feedback to professional.communities@actuaries.org.uk

For further information, including additional reading, resources on individual framework elements and ideas on how to deal with specific elements of uncertainty, please the ‘Reference Materials’ section below.

Chair Keith Brown
Membership 17
Established 2014

Reference Materials:

Contents

  1. Experience from other countries
    1. Australia
    2. Ireland
  2. Additional resources for framework elements
    1. Internal/Process – Data
    2. Internal/Process – Control Risk
    3. Internal/Process - Communication
    4. Human – Expert Opinion
    5. Statistical – Model risk
  3. Additional useful resources
    1. Beyond ‘Bootstrap’ measures of uncertainty
    2. Previous GIROC papers

1. Experience from other countries

1.1 Australia

1.2 Ireland

2. Additional resources for framework elements

2.1 Internal/Process – Data

  • Data uncertainty matrix - The Forest of Brocéliande has a Knowledge Quality Assessment module, which provides an uncertainty matrix.  This can serve as a high-level qualitative register for data uncertainties in a model and was used to inform the MUQ framework.
  • IAN on data for "CP73" – guidance from the Society of Actuaries in Ireland on questions a Signing Actuary could ask to help form a view on whether data used for providing a Statement of Actuarial Opinion is sufficiently accurate, complete and appropriate.

2.2 Internal/Process – Control Risk

  • S&P GI ERM Rating Criteria – pages 22-25 cover risk controls assessment from a general insurance perspective, where the extent (or level of detail) that this is in place may cause additional (or lower) degree of uncertainty on the reserves.

2.3 Internal/Process - Communication

Language used in communicating uncertainties:

  • The Intergovernmental Panel on Climate Change has various types of reports on climate change demonstrating good use of language: IPCC_summary-for-policymakers

2.4 Human – Expert Opinion

  • Expert judgement elicitation
    • Dr Joseph Lo, Dr Ed Tredger and Bernadette Hlavka discuss the tricks and possible pitfalls when conducting a successful expert judgment elicitation meeting, Article from The Actuary, 2013
  • Uncertain factors surrounding expert judgements:  Expert Judgement WP, 2015
    • Expert judgement process has long been in use in actuarial work, but it has lacked transparency. This paper aims to provide a practical framework regarding expert judgement processes, and how those processes may be validated.

2.5 Statistical – Model risk

  • ‘Ersatz model tests’ – Presented at the 2016 reserving seminar, it looks at ways of testing stochastic models, with an aim to developing deeper understanding of when and why different models are more or less appropriate.

Effectiveness of methods working party

3. Additional useful resources

3.1 Beyond 'Bootstrap' measure of uncertainty

3.2

More information on the work of GIROC.

Related documents

Contact Details

If you want more information about this research working party please contact the Communities Team.

professional.communities@actuaries.org.uk

Filter or search events

Start date
E.g., 26/10/2020
End date
E.g., 26/10/2020

Events calendar

  • GIRO 2020 Virtual Conference

    Available to watch globally in November.
    02-13 November 2020
    Spaces available

     

    This year's GIRO has been re-designed as a virtual conference to offer members and non-members the opportunity to get up to date content from leading experts in the general insurance field via online webinars. All sessions will be recorded and made available to purchase and re-watch post-event on the IFoA's GI Online Learning Resource area

  • Spaces available

    The webinar will discuss the challenges and opportunities schemes face in evaluating end game options, choosing a target state and understanding the impact this strategic decision could have on member outcomes long after the “end state” is reached. Adolfo, Kevin and Rhian bring over 60 years of experience in the industry and a variety of perspectives as scheme actuary, covenant adviser, trustee, de-risking adviser and insurer.

  • Spaces available

    Retail banking is going through a period of substantial change as it moves into the digital age. Banks have large amounts of data about their customers and about their risks. Open data application programming interface (APIs) and data science are enabling banks to use their data to offer innovative and sometimes personalised services. Data science is also adding value in risk areas such as fraud detection and cyber security. At the same time, the move to online banking is making it easier for firms including fintechs to enter banking without having to establish branch networks.

  • Spaces available

    Cash-flow driven investing is a game-changer for DB pension funds navigating their end-game. Suitable for sponsors who want to reduce risks on their balance sheets. And for trustees, it shifts the focus to providing greater certainty of returns, managing funding level volatility and ensuring they have enough income to pay cash-flow requirements.

  • Spaces available

    Patrick Kennedy, Partner at Gateley Legal and Founding Director of Entrust (a leading professional pensions trustee company), will be delivering an update on the latest legal developments during the course of 2020. With both a pensions legal perspective and over 25 years of trustee service, Patrick will seek to highlight how the letter of the law has continued to evolve against the backdrop of a difficult and challenging year

  • Spaces available

    The talk will provide an understanding of the priorities and relationships between deficit reduction contributions, in the context of wider scheme funding, and different types of value outflow from the employer based on the working party’s recently published report. 

  • Spaces available

    Covid-19 has required an urgent and cross-practice initiative to facilitate the extensive impact this pandemic has across all industries. IFoA members have been keen to contribute in a different way, so we developed the IFoA Covid-19 Action Taskforce [ICAT] to coordinate our effort, with a more efficient governance.

    We have over 500 volunteers and countless topics which we have amalgamated into 93 workstreams.