By Venkat Srinivasan, Managing Director at Innospark Ventures
AI is now ubiquitous! Everywhere we turn, we hear people talk about it. While the headlines on driverless cars and computers beating humans in Go capture the common human imagination, AI can and will transform every walk of life whether personal or business.
AI is not new. So why all the excitement? What is new is the availability of data, connectedness, the dramatic reduction in computing costs and advances in computational algorithms. The resurgence in AI’s popularity can be attributed to the ability to apply enormous ‘compute’ power to very large amounts of data which in turn has allowed for innovations in newer computational approaches.
But if you ask people what AI means, you will be surprised at the range of responses you get. Most people not familiar with AI technology think it is magic! And let their imagination run wild including becoming fearful of the term. They think AI is like an artificial human and is able to think and act like one. Technologists who have been introduced more recently to AI think of it in terms of machines learning intelligence from data, often referred to as ‘machine learning’ or ‘deep learning’. People who have been exposed to AI in its various avatars over several decades have a broader, richer view of AI — as intelligence embedded in machines to do tasks that humans normally do.
The notion that AI can learn and reason like humans, without having to program all the logic underlying such learning and reasoning, is referred to as Artificial General Intelligence or AGI. We are far from AGI. However, we have progressed significantly in embedding intelligence in machines to automate many of the tasks humans do. We will define AI as computational intelligence embedded in machines to automate tasks humans perform.
Intelligence Acquisition for AI Models
Computational intelligence can be acquired in three ways — from human experts, data, or both. In its initial wave, intelligence was acquired largely from human experts encoded as hand crafted rules (‘symbolic intelligence’), and as needed, coupled with statistical modeling of sample data if it could be obtained. This was tedious, time consuming and non-scalable at that time. In contrast, neural networks are an attempt to acquire intelligence from data and argued to mimic the way humans store and process intelligence. The explosive growth of content on the internet has paved the way for neural networks to gain tremendous popularity often to the exclusion of symbolic intelligence.
The popularity of neural networks has also been accompanied by tremendous hype which has fueled unrealistic expectations from AI. Perhaps the most highly visible casualty of this hype has been the staggering collapse of IBM Watson Health. We believe excessive reliance and unrealistic expectations on the completeness and reliability of the intelligence from data alone is a major reason.
We can summarize the current challenges with deep learning in 4 categories — explainability/traceability, context awareness, data adequacy and the implied intelligence acquisition model. All these challenges stem from a belief that explicitly observed data mirrors the intelligence in the human brain. And with vast amounts of data we may not need experts or human intelligence because the data will contain all the knowledge needed. Conversely, there is an implied assumption that sparse data environments are not conducive to AI applications. We have also seen people disregard symbolic intelligence acquisition and computationally accessible knowledge representation in sparse data environments as not being AI!
There is now widespread recognition that reliance on deep learning models based on data alone will not be sufficient for robust AI¹ ² ³ ⁴. Astoundingly, even Yann Lecun, who pioneered Convolutional Neural Networks (CNN), has recently changed his position a full 180 degrees⁵.
While many of these views are from an AGI perspective, they are equally valid for specific AI applications. It will behoove people working on building AI models to recognize that models based only on data will be incomplete. They are well advised to integrate expert intelligence whether it be feature selection, sample stratification or hand crafted rules.
Our view⁴ ⁶ of an effective approach for intelligence acquisition is to adopt a hybrid approach combining symbolic and data driven intelligence. The best approach for a specific situation depends on factors like how much humans know about the phenomenon being studied, whether explainability is important, and how much data is available. In many problems, starting with a symbolic intelligence implementation and coupling it with intelligence from data often yields the most effective path to robust, deployable AI applications. It is critical for high value problems that intelligence be explainable and tractable. In all cases, there should be a mechanism for the machine to either continuously learn new intelligence from data without any assistance and/or for the human expert to update the machine with new intelligence learned from data/experience.
The Future of AI
Where is AI headed? First, most people have realized that they have underestimated what it takes to acquire intelligence fit enough for everyday use. Like all statistical model building from the dawn of time, it takes many iterations and tinkering with data, features, and approaches to arrive at a generalizable AI model. Second, enterprises realize it will take time for an AI solution to gain acceptance as people get comfortable with changes to their established processes.
We are witness every day to interesting and sophisticated use of AI for solving real world problems. Precision medicine is becoming a reality. Covid vaccines in record time is a harbinger of what is to come. We will find better cures faster. Learning and skilling will be personalized to enable career linked life-long learning. Physical is increasingly merging with digital and enabling us to control the physical environment better. Brain computer interface is beginning to enable machines to understand and anticipate what and how we are thinking. Intelligent human aware robots are beginning to co-exist with humans and reshape the future of work. What will it take for AI to thrive in the future?
First, our approach to intelligence acquisition has to change from a blind reliance on data to integrating computational intelligence from experts where available. So the hype around ‘deep learning’ with CNNs has to give way to a hybrid approach which combines symbolic and connectionist methods. Whether one starts with experts or data will be dependent on the situation. In our experience, if we have robust expert intelligence, always start with expert intelligence and supplement it with data rather than the other way around. Starting with expert intelligence has several advantages including minimizing bias from data, speeding up model development and minimizing the amount of data needed.
Second, we need to adopt models that are explainable. The higher the value of the problem being solved, the greater the need for explainability.
Third, AI models have to be tractable. You have to be able to understand and identify how to improve the model. It can’t just be ‘let’s provide more data’ and then hope the models improve! Fourth, we need a new architecture for machine intelligence. The architecture has to recognize the humans ability to abstract and transfer learning from one field to the other. More on this in a future post.
Finally, we believe explainable, pre-built computational knowledge in different industries will become a ‘category’. Dare we say, ‘data’ is not the new oil; it never was. Computational intelligence is the new oil and will be more so in the future. A good example of such pre-built computational intelligence at a primitive level is WordNet in computational linguistics. While large language models are another example, they are not explainable. With explainable, pre-built computational expert knowledge, AI models can work with sparse data.
Fortunately, after a decade characterized by hype, there is widespread realization that it is imperative to integrate computational forms of expert (human) intelligence with intelligence from data, in order for AI solutions to achieve acceptable levels of performance. With an integrated approach, powerful intelligent machines can be created and deployed across all walks of business and society. We think this is something to get excited about!
Footnotes
¹Bengio, Y., Marcus, G., Boucher, V., 2019. Yoshua Bengio and Gary Marcus on the best way forward for AI. https://medium.com/@Montreal.AI/ transcript-of-the-ai-debate-1e098eeb8465.
²Marcus, G., 2020. The Next Decade in AI: Four Steps towards Robust Artificial Intelligence. CoRR abs/2002.06177.
³Marcus, G. 2022. Form, function and the giant gulf between drawing a picture and understanding the world. https://garymarcus.substack.com/p/form-function-and-the-giant-gulf
⁴Srinivasan, V. (2017) The Intelligent Enterprise in the Era of Big Data, John Wiley & Sons, New York.
⁵Lecun, Y. 2022. Most of today’s approaches will never lead to true intelligence. https://www.zdnet.com/article/metas-ai-guru-lecun-most-of-todays-ai-approaches-will-never-lead-to-true-intelligence/
⁶Srinivasan, V. Towards a New Architecture for Intelligence Acquisition, 2021, Working Paper, Available from the author.
2