Process Mining 101 Questions

• ‘Process mining 101: The key to digital transformation‘ webinar recording now available.

Process Mining is a data analytical technique that leverages data from IT systems to construct detailed representations of actual business processes. Process Mining tools can automatically summarise detailed process outputs to higher levels using built-in aggregation and abstraction features. These tools can create high-level views of processes, though some customisation or manual input may be required to accurately reflect specific business contexts or to adjust the granularity of the summaries.

Process Mining differs from traditional process mapping by using data from IT systems to automatically generate a detailed, real-time view of the actual process flows. Traditional process mapping relies on manual input and subjective interpretation, while Process Mining provides an objective, data-driven analysis. Process Mining identifies pain points by highlighting inefficiencies, bottlenecks, deviations, and compliance issues based on actual performance data. This allows for more accurate and actionable insights into where and how to improve the process.

There are several tools available for Process Mining, each catering to different needs and expertise levels. Vendor products like Celonis, UiPath, SAP Signavio, and Microsoft Process Advisor offer comprehensive and user-friendly Process Mining solutions. Additionally, there are plug-ins for dashboarding tools such as Power BI that perform basic Process Mining tasks. For those with more technical skills, open-source solutions like the Python library PM4Py offer similar functionalities but require more effort to set up and use.

Process Mining tools can ingest data from diverse sources, including various IT systems, data lakes, data warehouses, and manual extracts. These tools analyse activity logs, typically autogenerated by IT systems, to create a consolidated view of a process. As a bare minimum, to facilitate effective Process Mining, these logs must have a unique identifier to follow a data point across the process, the activity that has occurred, and a timestamp.

However, challenges may arise when dealing with processes that sit across multiple systems. These challenges include data silos, inconsistent data formats, and accurately mapping data across different systems. Ensuring data quality and consistency emerges as a critical consideration in mitigating these challenges. Moreover, variations in data granularity across systems necessitate thoughtful transformation to align with the desired level of process analysis.

Process Mining could also be used to test data completeness and process integrity management as it provides a comprehensive view across different systems and data warehouses/lakes. It helps in identifying discrepancies, inconsistencies, and detecting incomplete data as well as deviations from standard processes.

The compatibility to extract data from a variety of sources including older systems may depend on the ability to directly access and integrate data from these systems. Some tools offer connectors to pull data from a system, for example Celonis. In cases where direct data extraction via connectors is not feasible, data can be manually extracted from legacy systems. This involves exporting data into formats compatible with process mining tools, such as CSV or Excel files. Although this method requires more manual effort, it ensures that essential data from older systems can still be utilised for process analysis.

Process Mining primarily relies on an activity log created across the IT systems, so capturing manual steps can be challenging. However, these steps can be included by integrating data from manual records or by adding manual event logs to the dataset. Some tools allow users to monitor user activities on a desktop level to annotate steps that are not captured electronically, ensuring that the entire process, including manual activities, is represented. Task mining by Celonis captures user interaction data at the desktop level, which allows for a detailed analysis of how tasks are performed outside of core IT systems. This includes activities such as interacting with applications, consulting files, and other desktop actions. Communications mining by UiPath focuses on analysing communication data from various channels such as emails, chat logs, and collaboration tools. This approach helps in capturing the context and content of manual activities that occur during process execution and can lead to better identification of inefficiencies and areas for process optimisation.

The effectiveness of Process Mining heavily depends on the quality of the data. Good quality data ensures accurate process models and reliable analysis. Poor data quality, including missing, inconsistent, or incorrect data, can lead to misleading results and obscure insights. Just like with any data project, it's key to ensure data integrity, completeness, and accuracy to leverage the full potential of Process Mining tools.

Process Mining could be used for testing process integrity management as it provides a comprehensive view across different systems and data warehouses/lakes. It helps in identifying discrepancies, inconsistencies, and detecting deviations from standard processes. This continuous oversight ensures data integrity and enhances the overall reliability of the information.

Just as with any data-led project, the risks of adopting Process Mining include data privacy concerns, data quality issues, and potential resistance to change. To mitigate these risks, organisations should:

• Ensure robust data governance and security measures to protect sensitive information.
• Validate and cleanse data before analysis to ensure accuracy.
• Engage stakeholders early and provide training to facilitate adoption and address concerns.

One of the biggest pushbacks against process mining is that it can be seen as “opening a can of worms” - revealing a lack of consistent processes and exposing poor controls through a spaghetti diagram of tasks. While process mining can be a powerful tool in identifying some of those control gaps and process failures, it does require careful, transparent communication with stakeholders so that expectations are managed. It also benefits from a strong initial understanding of the process being analysed and the different ways in which that process could (or should) be navigated.

Ineffective ITGCs on the systems used for Process Mining can significantly impact the reliability of the outputs. The event logs that Process Mining typically relies on are, in the cases of larger ERPs, not accessible to most users. Therefore, the critical controls are around privileged user access, which if poorly managed can lead to unauthorised changes or data corruption, compromising data quality and integrity, resulting in inaccurate process models and misleading analyses. Process Mining can be used to identify ITGC issues by monitoring system activities, analysing process deviations and anomalies that may indicate control failures, such as unauthorised access patterns or irregular system modifications.

Conversely, when using cloud-based Process Mining tools, it is important to maintain robust data security and compliance measures. Celonis’ Trust Center and Data Security documentation highlight strict security protocols, regular audits, and comprehensive monitoring, which are crucial for maintaining the reliability of Process Mining outcomes and keeping sensitive information safe.

End users, ranging from operational risk and audit to business leaders, can "self-serve" Process Mining deliverables with little to no technical background, facilitating decision-making and enhancing process resilience. Process Mining can be used to support operational risk management by identifying control weaknesses and potential failure points, ensuring robust risk controls. It can also enhance resilience by providing real-time visibility into process performance, enabling quick identification and mitigation of disruptions. From a Consumer Duty perspective, Process Mining ensures that business processes align with regulatory requirements and consumer expectations, fostering trust and transparency.

The technical side of Process Mining is typically performed by data analysts, process analysts, and business intelligence professionals. These individuals – requiring a level of technical competence in the tool(s) being used – often create Process Mining views and analyses that are used by all functions within a company.

Defining the "Happy Path" in Process Mining can be done through several approaches:

• Interview the process owner and collaborate to map the expected path, ensuring that the defined path aligns with business expectations.
• If documented following the Business Process Model and Notation (BPMN) standard, the expected process model can be uploaded in most Process Mining tools. Alternatively, you can manually recreate the "Happy Path" from other documented process diagrams, e.g., Visio.
• Use a data-driven approach by analysing the most frequent and efficient process flows in the data to identify which activities and variations should be included in the "Happy Path".

Process Mining can analyse data against user-defined rules and conditions to identify exceptions. This could be an activity occurring or not, values exceeding a threshold, or transactions not following the expected "Happy Path" in the process. Exceptions identified will be remediated by

1. Writing back to the source system(s) to correct the issue,
2. Sending emails or notifications to relevant users,
3. Raising an incident in a ticketing system or documenting results in an audit management system, or
4. Triggering Robotic Process Automation (RPA) bots or executing a workflow.

Traditional audit activities involve interviews to understand processes or follow small samples of the process, which can be subjective and less comprehensive. Process Mining provides a data-driven approach which can be deployed across the entire audit lifecycle and beyond:

• In planning and walkthroughs: Process Mining can visually map out the actual processes, providing a comprehensive view, unlike traditional walkthroughs that rely on interviews and samples. This enables auditors to identify and map key controls within their business processes and better scope the overall audit.
• In fieldwork and control testing: By testing 100% of the population and applying advanced data analytics, auditors can test the effectiveness of controls, perform root cause analysis and highlight audit findings.
• In reporting and documentation: The integration of Process Mining tools allows for the seamless export of findings to Audit Management or GRC platforms. This streamlines the reporting process while findings are documented automatically. The data-driven insights enable reports that inform decision-making and strategy at the highest levels, unlike traditional reporting methods.
• In risk assessment and continuous auditing: Analysing the processes across the audit universe with real-time data allows organisations to anticipate and mitigate risks before they escalate. Process Mining allows audit functions to become more agile and base their strategic decisions on solid, data-driven insights.

• Eliminating process bottlenecks: Process Mining can help organisations identify inefficiencies and bottlenecks in their workflows. Process Mining can pinpoint areas where tasks are delayed or where resources are underutilised. By addressing these bottlenecks, organisations can improve overall efficiency and reduce operational costs.
• Streamlining procurement processes: Process Mining can be used to analyse procurement processes and identify areas for improvement. For example, organisations can use process mining to identify instances of maverick spending or to optimise supplier selection and negotiation processes. By optimising procurement processes, organisations can avoid overstocking, reduce inventory holding costs, and negotiate better prices with suppliers.
• Optimising resource allocation: Process Mining can provide insights into resource usage patterns within an organisation. By analysing data on resource allocation and usage, organisations can identify opportunities to reallocate resources from areas of low demand to areas of higher demand.

These examples can transfer to a multitude of industries and services. In the NHS, Process Mining has been effectively employed within NHS pathways and processes, showcasing its potential for improving healthcare delivery. It has been used to reduce missed appointments, identify bottlenecks in A&E pathways and enhance overall patient experience by maximising utilisation of available resources.