We created a data fluency program for staff auditors at a large regional accounting firm. Included in that program was a lengthy discussion of data quality, including the importance of internal controls. We provided all participants with techniques and practice aids to help them identify conditions that may impair the client’s analytics readiness.
For the most experienced auditors, we followed that training with a four-hour fully immersive role-play that simulated an actual audit. The participants were given limited information and had to interview “clients” and obtain documentation to achieve the following mission: make a presentation to the engagement partner about the client’s analytics readiness and how effective the client’s information systems would be at ensuring real-time, high-quality data needed for analysis.
The scenario was designed for the participants to practice their data fluency and use the tools and techniques we provided to evaluate the client’s ability to produce data that was complete and accurate.
Building Data Fluency Will Take Time
The classroom experience was positive. The materials were well-designed and well-presented; the participants were engaged. And yet, when it came time to apply what they’d learned in an audit situation, they struggled. They had difficulty connecting the dots between data quality and internal control design. They used the techniques and tools we went over in the seminar, and they got good information. But they were challenged to understand and analyze that information.
The firms concluded that the method of analyzing data is so new to most staff members that it will take time for staff across the firm to become sufficiently fluent.
They also concluded that auditors at this level need to improve their critical thinking and problem-solving skills. Too many years of relying on last year’s work papers and standard audit programs have dulled these skills, which must be reinvigorated if firms are to be successful in their own data analytics efforts.
Data Analytics Makes Internal Control More Meaningful
While the participants’ mastery of data quality was disappointing, their evaluation of internal control—which we viewed as a secondary learning objective—blew us away.
Framing their mission as “helping the client assess data quality” changed their motivation for conducting walkthroughs and obtaining internal control documentation. Last year’s control narratives were quickly pushed aside. The teams’ conversations about control design and its strengths and weaknesses were in-depth and thoughtful.
They were curious and excited to talk to their clients about controls.
The presentations to the engagement partner were outstanding. Even the weakest of the presentations offered insights into the client’s information systems and controls that far exceeded anything I’ve seen in the classroom or on the job. Without even realizing it, they had applied the COSO framework, describing control objectives and linking them to control procedures—or, in some cases, the lack of procedures.
From this insight about how controls affect data quality, most teams were able to take that critical next step of linking control strengths and weaknesses to the design of substantive tests. They finally “got” that internal control matters.
Two participants told us that our data analytics training was the best internal control training they’d ever had. I agree. The data analytics training that was just so-so turned out to be great training for internal control.