Beyond AI Proof Of Concept

A proof of concept demonstrates that there is some merit to an idea from a technical perspective on a very focused area of application. For a POC to be a meaningful indicator of value it needs:
‍
Executive Support: Good ideas without an executive willing to help maintain momentum and to fund the initiative often do not get their feet off the ground.
‍
Success Criteria and Scope: Stakeholders’ version of what good means tends to shift whereas success criteria should represent what is sufficiently good to justify further investment. It is common for people to slide their thinking from an indication of value in a pilot versus their expectations of a production solution. Similarly, stakeholders’ expectations for what needs to be in a pilot to prove value also needs to be defined early to prevent creep.
‍
Production Considerations: Even in a pilot stage, engineers with a background in production capabilities need to be involved so they can ensure that design is built in a way that scales that is feasible to operate within the broader technical ecosystem.
‍
Change Management and Adoption: If there is not a clear process for adopting the new capability sometimes production ready solutions are built that never create any meaningful values because users stick with the way they know.
‍
Accountability and Responsibility: For both development and maintenance there must be both clear accountable and responsible parties so each role knows what they need to do (and not do) and those people can be held accountable to complete those tasks.
‍
Data Readiness and Access: Pilots may succeed on a small curated dataset, but if the same quality information isn’t in the same format, stale, inconsistently available, or not available for many applications, the idea will not scale.

AI should only be implemented to achieve a particular impact for the organization. A technical metric might be a strong leading indicator that the desired impact will be achieved but that impact as it supports the mission of the organization is what is really important. Impact may be measured as additional revenue, cost savings, lives saved, new customers on-boarded, etc.

It must be measured to ensure that the model is achieving what was really intended to.
‍
For example, an application fraud detection model may have 99% accuracy but if it isn’t meeting the success criteria (desired real-world impact) of reducing fraud by $1M monthly, it isn’t successful according the organization.

This evolution has already started but the days of D&A leaders just building core data products to support key Finance dashboards (obviously a hyperbolic oversimplification) are behind us.

D&A leaders must think of themselves and act not as “data people” or “AI people” but business leaders who truly understand the industry and organization who happen to have the D&A background.
‍
They need to enable safe AI innovation so they average person can safely access AI capabilities. They need to work with business leaders to understand their current and long-term aspirations because the D&A leader might/will be able to suggest new methodologies to achieve that value delivery that the business leader on their own may not have thought about.
‍
D&A Leaders are key strategic leaders who have a seat at the table whose ideas and solutions will and should be expected to have a measurable impact for the business.