August 14, 2020

Predictive Analytics

Get notified whenever a new podcast is released!


AI presents tremendous opportunities for organisations and humanity as a whole. However, the Economist Technology Quarterly recently summarized the progress made so far and also discussed the current ‘reality check’ in the world of machine learning. Significant investment has not always yielded results and has led to disappointment. Nonetheless, the opportunities to reap the significant benefits of AI for your business are still there and the good news is that the challenges look quite a lot like those that have faced all technological progress within organisations over the years. 

So as one wave of AI development comes to an end and as the next one starts, this article highlights the importance of Agile fundamentals in order to build on the achievements of recent years, using the area of predictive analytics as an example.

Back to the Future

Technology development has traditionally involved translating a set of business requirements into a solution that meets them. Methodologies and tools have evolved in an attempt to make that process as effective as possible and to maximise the chances of solving the business problem. 

Developing a solution based on analytics adds additional dimensions to the mix, so what are they?

Analytics, Legal

In order to develop a predictive analytics solution the roles of the statistical modeller and lawyer come into play.

When developing an attribute that is predictive of the behaviour of a person, such as is used increasingly in motor insurance, the level of statistical correlation between variables is not known at the outset and can’t be specified as a requirement. 

Secondly, there is a legal question. Even if it is permissible to use the data sources, there will be other laws to protect citizens from discrimination. For example, there are laws in place that enable people to switch motor insurance and not to be penalised for this behaviour. 

So Analytics and Legal functions add two more dimensions to the traditional two-dimensional Business-Technology model. There are two more.  

Data Quality

The quality of the data and its source and can be thought of as a fifth dimension. A solution is useless unless the process of ingesting and cleansing the data is completely understood because the correlation discussed earlier will only be as good as the underlying data. Again, a similar problem to the age-old challenge of data quality with much higher stakes. How much time and effort should you invest in this before you have figured out which data is useful?  


The sixth dimension concerns the technology platform for performing statistical analysis and testing. Again, the level of investment and time devoted to robust development and testing environments is a challenge that is not new. But because AI is in its infancy, relatively speaking, the likelihood is that these environments will not be in place and effort to make them available will run in parallel with all the other activities. Cloud offerings create possibilities to create environments rapidly for data analysis and testing although can be costly if not controlled.

Way Forward

The six dimensions of Business-Technology-Analytics-Legal-Data-Environment are redolent of age-old situations with a degree of additional complexity. It’s as if the stakes of solution development have been raised, leading to bigger failures if not managed correctly. Is there a way forward out of the AI ‘reality check’? 

The answer is to be found in the principles. Alignment of the endeavour with the vision of the organisation becomes more important than ever so that all participants can step through the gates in the development process with confidence. Make sure you have a set of principles in place that support the vision and that you can test all decisions against.

How do you do that? An Agile mindset provides the answer because the approach can be found in Agile Manifesto. The principles there can act as the basis for defining your own set of principles as a group. Then, when faced with a challenge, revisit those principles, derive the benefit of Agile and ensure that the group collaborates to work through the complex issues.

Remember that it’s still about people. In his book ‘Principles’, Ray Dalio comments on the role that AI will play in our futures. He stresses the importance of valuing the role of a person with a deep understanding of an area of expertise in order to interpret the results of an AI or machine-learning system. 


As you grapple with the myriad possibilities that AI offers, the answer will be found by revisiting your core principles. What is the organisation trying to achieve and how does the current AI effort support that?

Armed with that level of alignment, use the Agile Manifesto and add in the perspectives of your analytics and legal teams in order to govern decision making and so create the learning environment required for success.