The risks of over reliance on self-assessments

Self-assessments are widely used for quality management and for the evaluation of the Internal Control System (ICS). This article summarizes some of the benefits of ICS self-assessments, but also underlines its risks and limitations, and suggests the internal yet independent performance of testing of both the design and the effectiveness of controls.

Self-assessments, including in the evaluation of the ICS

In many organizations and areas, self-assessments are valued tools for quality management and continuous improvement. There are proven methodologies such as TQM or the Baldrige Excellence Framework, that support the development of a culture of quality, improvement, and excellence.

Self-assessments are also valued by many in evaluating the ICS. As far as I have seen it or learned from peers, such self-assessments have several steps:

- the identification of the control objectives and the definition of the assessment criteria,

- the self-assessment process itself with interviews, testing etc. both on design and effectiveness,

- the eventual weakness identification, and remediation, including the re-performance of testing,

- the documentation, that helps to create evidence of the done work and an audit trail.

Overall, self-assessment is a proactive step for an organization to evaluate its ICS: identification of control deficiencies and remediation measures.

Self-Assessments and SOX

Under the Sarbanes-Oxley Act Section 404 (SOX), which specifically focuses on internal controls over financial reporting (ICFR), companies subject to SOX requirements are required to assess the effectiveness of their ICFR annually and to report annually on the results to the Securities and Exchange Commission (SEC).

SOX does not preclude companies from using self-assessment processes as part of their internal control evaluation and the assurance provided on the effectiveness of ICFR. However, SOX defines strict requirements on the effectiveness of internal controls: organizations must ensure that the self-assessments are rigorous, robust, objective, and well-documented, so that they can provide evidence to external auditors and regulators when requested.

The limitations of self-assessments

In my years as CAE, but also in my business positions, I have observed several problems with self-assessments:

- lack of objectivity: self-assessments are influenced by biases, such as overconfidence or a desire to show results in a favorable light. This is particularly true in a decentralized organization with a large geographical footprint, with cultural differences, possible miscommunications on the self-assessment process and the expectations. Self-assessments can also suffer from a limited perspective, as individuals or departments may not have access to comprehensive data or may overlook certain risks or deficiencies due to their proximity to the processes being assessed. The lack of objectivity undermines the reliability and credibility of the assessment.

- conflict of interest and inadequate accountability: individuals or departments conducting self-assessments can have a vested interest in the outcome and be conflicted. Without an independent oversight, there can be insufficient accountability for the results of self-assessments. This compromises the integrity of the process, of its results, and the quality of follow-up actions.

- inadequate expertise: some assessments require specialized knowledge and skills, for example in areas such as compliance or cybersecurity. Self-assessments may lack the necessary expertise, leading to incomplete or inaccurate evaluations.

- flawed rationale for self-assessments: I have seen several cases where self-assessment are used as a cheaper solution than an external (within the company or even from outside the company) and independent testing.

These limitations have two (at least!) major consequences:

- internally, most self-assessments are time-consuming. If they fail to provide meaningful insights, it is a significant waste of resources and a missed opportunity to improve the ICS.

- externally, an over-reliance on self-assessments can expose companies to a compliance risk. External stakeholders, including regulators, will question the validity and rigor of internally generated assessments.

A functional validation approach

In the Allianz Group, we developed several solutions to try to address these weaknesses:

- internal audit. It is not uncommon to see organizations asking IA to perform the testing. Much has been said and written on the matter. As Allianz Group CAE, I have made sure that IA focuses rather exclusively on Entity Level Controls (ELCA) related to ICFR, with a cycle congruent to our risk-based planning. And I have insisted that the Second Line (2LOD) grows muscle in control testing.

- 2LOD. In the insurance industry and under the Solvency 2 regime, the 2LOD key control functions are the actuarial, risk and compliance functions. Over the years, the Group actuarial and risk functions have developed strong strategies and methodologies to perform on site reviews and controls to ensure the compliance of the local actuarial and risk functions with their respective functional standards. This was very weak in compliance, and one of my priorities when I become CCO has been to build an entirely new team dedicated to compliance reviews in the different entities of the Group. This allowed to challenge compliance self-assessments, sometimes in a rather drastic way, and this led to a truer and fairer view on the status of compliance related controls in compliance.

- The aligned approach between risk, actuarial and compliance pushed 1LOD to in turn organize local controls performed by each relevant central function at Group level (for example for Underwriting, Claims processes, etc.), and to restrict the reliance over self-assessments.

As much as possible we have avoided seeking external assurance from auditors or consultants. We did this not only to reduce external costs, but also to reinforce the internal culture of responsibility and accountability over the components of the ICS.

While self-assessments can be a valuable tool for organizations to identify risks and opportunities for improvement, they are prone to biases, conflicts of interest, or inadequate expertise, which can undermine their effectiveness and credibility. The development of functional verifications mostly performed by central functions in local operations seem to be an effective and culturally adequate solution.

Download this article for free