Microsoft says AI and machine learning driven by open source and the cloud

Artificial intelligence and machine learning are rapidly gaining importance, and Mark Russinovich, Microsoft Azure chief technology officer, believes it's because of open-source software and the cloud.

Yes, Microsoft just announced that the next major edition of Windows 10 will support artificial intelligence (AI) and machine learning (ML). But, marketing hype aside, Microsoft knows darn well that the real heavy lifting for AI and ML happens on the cloud with open-source software. That was the message Mark Russinovich, Microsoft's Azure CTO, brought to The Linux Foundation's Open Source Leadership Summit (OSLS) in Sonoma, CA.

Russinovich opened by saying:

AI technologies and techniques are experiencing a renaissance. Open-source technologies and communities have fostered the growth of self-taught machine learning developers with libraries and frameworks. The computing power of the cloud has made the processing of large data sets cost effective and commonplace. As more research continues to be done and shared throughout the communities we will continue to see more intelligent apps driving even greater adoption of open-source technologies across all processing platforms.

Specifically, he mentioned two examples where Microsoft is using the cloud and open source to help provide solutions with customers. The first is with Rolls-Royce aircraft engineswhich use ML to track their wear and tear. This data is then used with AI to proactively maintain the engines.

Microsoft is also using the potent one-two of the cloud and open source to power DiagnosticX Intelligent Disease Predictive Architecture. This is a beta program. Its first use is to examine X-ray images from the National Institute of Health (NIH) chest X-ray data repository. This data is then fed to such open-source ML and AI analysis programs as Core ML, Google TensorFlow, and ONNX using Visual Studio for AI and Azure Machine Learning. The end result, which can be read using a web-interface, is a program that can diagnose pneumonia.

Why bother? Because there are many more X-ray machines than there are radiologists. When 50,000 children die in the US alone of pneumonia, any tool that can help spot the killer disease in time for a cure is a win.

What enables these programs to be created, said Russinovich, is the cloud. We hadn't made much progress before because it's only now that the cloud gives us what AI and ML developers always needed. This is on-demand, scalable compute with low cost, virtually infinite storage, and high-speed GPU processors such as the Nvidia's Tesla K80 and P100s. Combined, these give AI and ML developers the resources they've always needed to let them experiment. And, as Russinovich said, "AI and ML are an art, we need to experiment."

The tools, programs, and frameworks to make this happen, according to Russinovich, are almost all open source. "It starts with MySQL PostgreSQL, Hadoop, Cassandra and the NoSQL databases. They're all totally open source. For analytics and forecasting, you use R and SPARK. And, the most popular deep learning libraries -- TensorFlow, Keras, and Caffe -- are also all open source."

Russinovich continued, "Back in 2007, ML experts picked up that open source was the right way for AI and ML. Today, both are based on open source."

Between open source and cloud computing, Russinovich sees AI and ML transforming every industry and that's not marketing hype.