MSBA Professor Zachary Lipton: From Jazz Musician to Machine Learning Innovator

You may not think to look for predictive analytics algorithms and lines of code among the flickering candlelight of a jazz bar. But there isn’t as much distance between data science and jazz music as you might expect. In fact, CMU Tepper School of Business professor Zachary Lipton holds passions for both, at least when he’s not founding new start-ups or teaching machine learning courses in our online business analytics master’s degree program.

“Even though I really love music…after a while, I noticed there was a math- and science-shaped hole in my life,” Lipton says in describing his transition from jazz musician (you can find some of his music on YouTube) to machine learning and analytics academic.

Getting Jazzed About Machine Learning and Predictive Analytics

Jazz pianist Dave Brubeck once famously said, “Jazz stands for freedom. Get out there and improvise, and take chances, and don’t be a perfectionist – leave that to the classical musicians.” For Lipton, freedom came from the ability to pursue his intellectual curiosity and scholarship on a full-time basis.

“One of the great luxuries of being an academic is that you have a relatively broad license to research and explore a wide range of disciplines if you choose,” Lipton continues. “Over the course of my academic career, I’ve been able to research topics including machine learning methodologies, applied machine learning with healthcare applications, ethical concerns with automated systems and much more.”

One of the qualities that Lipton has kept from his days in jazz clubs is a tendency to challenge convention. Where jazz surprises and delights with musical improvisation, Lipton does the same for data science by creatively applying machine learning techniques to solve industry-wide problems.

Finding the Signal Through Noisy Data and Machine Learning Hype

When the machine learning community and many in tech media got a little overly enthusiastic in response to an advancement in natural language processing (NLP) from OpenAI, Lipton’s insight provided a measured perspective: Yes, it was a step forward, but it was a step on a path the rest of the artificial intelligence community was on already.

Skepticism is a healthy quality for analytics professionals. Machine learning algorithms and predictive analytics offer untapped opportunity, but as research firm Gartner points out, the hype around it can sometimes make it difficult to find the value for IT and business leaders. Creativity and innovation help push machine learning further, but only when guided by a purpose and a clear understanding of the goals driving the application of the technology.

This is one of the areas where professor Lipton’s perspective has proven invaluable. His background as an economist has helped him to hone-in on the practical side of predictive and machine learning technology.

Using Machine Learning to Find Answers in Medical Data

As natural language processing and machine learning were rapidly gaining traction, Lipton saw potential for the technology that few, if any, others had noticed. His research, published in 2015, became the first to use a specific technique for leveraging a machine learning algorithm in historical health information.

“When I started working in Deep Learning in 2014, a lot of people were getting excited about using sequential models for mining patterns in text data—people started using recurrent neural networks to develop automatic translation and speech recognition systems,” Lipton says. “When I was working in this area, I was more interested in applying a similar approach to finding patterns in medical data.”

The model Lipton and other researchers developed was able to classify 128 different diagnoses when given 13 clinical measurements. His work has since inspired more than 100 other publications and continues to serve as a foundation for exploring other machine learning applications.

“I forged a collaboration with David Kale, a Ph.D. student at USC, and data scientist affiliated with Children’s Hospital Los Angeles. Together, we got access to multivariate clinical time series data,” Lipton explains. “Basically, this type of data can be messy because it’s often recorded in different formats, on different time scales and with varying levels of completeness. So, traditional analytics methods at the time struggled to gain insights from it. We were the first to apply modern recurrent neural networks to this type of data for the purpose of recognizing diagnoses associated with specific patients. Our technique significantly outperformed other methods, and our work opened a promising new line of research.”

Where a lot of jazz music is about finding the right fusion of different styles, analytics is often about finding the right fusion between technology tools, business goals and data. Using the wrong method for the job results in conclusions that fall flat (or worse, conclusions that aren’t accurate at all). Finding the right approach, however, can incite applause in the conference room.

Find your analytics muse by applying to our online M.S. in Business Analytics.

The Transition from Music to Machine Learning

In the first year of earning his Ph.D., Lipton balanced playing saxophone four nights a week while taking classes and studying. It didn’t take long, though, before his focus shifted and he devoted more of his attention to his passion for finding answers in data.

“There was this kind of huge explosion in interest for machine learning right around the same time as I was beginning my Ph.D.,” Lipton says. “As the field took off and I was finding success in it, I started spending more time focusing on the technology side of my career. I spent the summer of 2014 living in India. Then the following summer, I was living in Seattle and worked as part of a core machine learning team, helping to build the technology that went into Amazon’s recommendation engine for products like Prime Video. The year after that I worked in Microsoft Research. In my last year, I jumped into simultaneously finishing my Ph.D. as a full-time data scientist and working on a really talented team with Amazon AI.”

The explosive rise of industry-wide interest for machine learning, neural networks and artificial intelligence helped to propel Professor Lipton’s passion as well as his career. He continues to make contributions in machine learning, from work in named entity recognition to critically analyzing trends in the academic machine learning community.

Despite his busy academic life and career as a technology entrepreneur, Lipton does still find time to play music as a hobby.

“I still practice a bit, and recently, we got my old proper keyboard in our apartment,” Lipton says. “My girlfriend, who’s a much better musician than I am and a composer, gives me some pointed looks when I miss a key signature. Music has filled an important part of my life, but I have to admit at this stage that I’m in all-in on machine learning.”

Bringing Machine Learning Together with Predictive Analytics

Similar to the incredible growth in machine learning and analytics as a whole, investment in predictive analytics tools is set to increase by leaps and bounds.

In the same way that data science is a broad field that often uses machine learning, predictive analytics can encompass a vast array of technologies, tools and methods. When applied correctly, though, machine learning can help to create or tailor predictive models more quickly and solve problems that would otherwise be impossible.

The Advantages of Machine Learning

“One major challenge Amazon has is having to decide how many items to stock,” Lipton explains, referencing his work with Amazon AI. “It’s a classic inventory problem, and there are myriad factors to consider. You could develop a predictive model manually, but machine learning can use historical data to build and optimize the model faster and make predictions about not only which items to stock but which fulfillment centers to stock them in.”

Machine learning is not purely about efficiency. Yes, it is more efficient to have an algorithm do the work, but that speed and accuracy also allows technology and business professionals to solve problems that are too complex for traditional software. Image recognition is a perfect example.

“There’s not a single person in the world who can write a set of rules in code that would allow software to recognize a person in a photograph…it’s just way too complicated of a task,” Lipton says. “The way we get around the fact that we don’t know how to write that kind of program is we have examples of the task being done correctly, so we can give an adaptive machine learning system a million photographs where we know who’s in them and it learns from that. If we can tune the model in the right way, we end up getting predictions that don’t just agree with the data we’ve already seen, but make accurate predictions about data we’ve never seen before.”

What’s Next for Machine Learning?

While many of the consumer-facing platforms and features we use every day wouldn’t be possible without machine learning, the technology is equally important in the behind-the-scenes areas of business, such as operations and decision-making. Businesses have always had to collect operational data and make forecasts to form decisions about everything from inventory to hiring. What has been missing, according to Lipton, has been the sophistication offered by predictive analytics technology.

To some extent, the estimation and prediction elements of analytics have been neglected. As Lipton mentioned, many of the problems associated with predictive analytics are impossible to solve using traditional methods because they are too complex and involve too many variables. Lipton sees machine learning’s potential to open opportunities in numerous sectors and industries, including areas like demand forecasting, power grid management and financial forecasting.

“There’s a huge amount of work being done with machine learning,” Lipton says. “One of the original and most successful applications of machine learning was in marketing, and although it’s become a more mature application of the technology, it’s still one with immense interest. There are also many reasonable concerns about negative externalities resulting from personalizing various services and automating consequential decisions. A significant fraction of my time is spent researching the societal impacts of machine learning with a goal of building more socially-aligned technology and providing clear guidance to policy-makers.”

What Analytics Skills Do You Need to Be Successful?

It’s no secret that machine learning and analytics technologies are set to disrupt entire industries. So, how can professionals prepare for a future where algorithms are integral across so many aspects of business?

The answer is to build technical skills that go beyond specific tools and extend to understanding how to conduct data analysis. This is what makes the online Master of Science in Business Analytics (MSBA) offered by the Tepper School of Business unique. It is designed to prepare students for the modern challenges that not only machine learning professionals will face but that nearly all successful businesses will need to address.

“The curriculum of the MSBA program covers multiple aspects that you need to be productive in machine learning,” Lipton says. “Our courses establish a fluency with statistics and probability, and we help you build programming confidence with Python and R. But, even if you’re not planning to go deep into machine learning, these courses and the focus on rigorous data analysis are skills that are rapidly becoming cornerstones of the modern economy.”

It is also not enough to develop advanced proficiency in programming, data science tools and data analysis techniques. Today’s analytics job marketplace needs professionals who can not only build software or that understand statistics. It is ripe with opportunity for those who also have the business skills to translate their technical proficiencies into actionable information and decisions.

“The demand for data science and machine learning jobs has grown so much that the supply of expertise can’t keep up,” Lipton says. “In the time that I’ve been in the field, salaries have risen by probably a factor of four or more. You can find a lot of people who are competent mathematicians, or decent software developers and you can find people who can communicate well. The problem is that if you’re really great at mathematics or software development, but you don’t know how to apply those techniques to a business problem, or you can’t communicate to other stakeholders in your organization, you’re not going to be very good at an applied machine learning or applied data science job. The highest demand is for professionals who have all three skillsets.”

About Tepper’s Online M.S. in Business Analytics

The online Master of Science in Business Analytics (MSBA) from the Tepper School of Business at Carnegie Mellon University helps students to develop proficiency in the full range of state-of-the-art business analytics techniques. With a comprehensive curriculum that encompasses data visualization, machine learning and optimization, large-scale data management and more, graduates leave the program positioned for advancement in their careers.