Neil Ward-Dutton on Process Mining: Why Now?

If you can't tell, I've been intrigued by Neil's series of YouTube videos covering a range of topics around AI and Automation, as part of his work at IDC. I think it is a great way to humanize and make relatable the work that IDC does in this space.? It also reveals how Neil is able to turn over an issue or space and see it from many angles at once.  

In a recent video, he discusses process mining:

Traditionally the use of tools to analyze logs of Enterprise systems and from them, reverse engineer what processes were running to result in those logs.  

However, it stayed a bit niche over the next few years for a number of reasons. One of which was, that if I'm using a BPMN engine to write my process, I already know natively what process was followed, and the process mining will just confirm that my BPMN diagram is adhered to by the software that interprets it.

So what changed in the last few years to make process mining such a hot space? Neil has five reasons why:

 
1. The rise of RPA which has re-ignited interest in process improvement and automation.
 
2. Focus on ease of use of the tools and ease of implementation.
 
3. Availability of SaaS models for adoption of these tools.
 
4. There's a flow of venture capital money into the market which has raised awareness via marketing spend.
 
5. Vendors are expanding their scope to cover more types of analysis.? Now also looking to effect change or automation into a process.

This collectively creates a more interesting mainstream market, in Neil's view and I have to agree. I think he listed the largest effect first. The biggest change across these tools is the change in the market context into which they play. Process mining has hit its stride in parallel to RPA picking up the pace these last few years.

One hypothesis I have is that if you build your new process with a process engine, you don't assign a lot of value to process mining. It becomes, primarily, a discovery tool to help you understand what is happening before you build the new process. After you build the new process, you know what it is (assuming no backdoors or side doors to access the functionality the process entails).

But with RPA, you have a collection of automations, without a governing process being required. Which means process mining doesn't just have value for discovery, it has value in validating the results, counting the ROI, and finding the next set of automation opportunities within the processes as captured by log files in enterprise systems.

Effectively, Mining becomes part of your governance and discovery over time, rather than just being a point-in-time effort.

I think a factor holding process mining back which was not discussed is pricing. The pricing needs to come down to make this functionality ubiquitous. I think it is likely that a large firm will release a free or nearly free process mining capability in the future, or that an open-source version of one of these tools will be sponsored by a major vendor to take cost out.

I also think it is likely for process mining companies to merge with or be acquired by process or RPA firms to increase the completeness of the offering, and we're already seeing some of that consolidation.