Machine Learning Systems
Course Description
Machine Learning and Deep Learning systems are at the core of many of today’s most exciting products and applications; they power everything from autonomous cars to online chat assistants, and from investment portfolio allocation to the drought warning systems in food-insecure regions. Regardless of the application, all of these systems share a set of key challenges during their life cycle. This research-centric and code-driven advanced course will help you appreciate and gain a practical, hands-on, knowledge of how these challenges can be mitigated, or even leveraged and exploited. The course will provide you with knowledge of both, the fundamental concepts, as well as the state-of-the-art developments in the field of ML Systems and their code-level implementation. Students will be provided instruction that will allow them to develop the skills necessary to understand, implement, and reason about ML systems. In the process, they will develop their very own programming portfolio.
All students who complete the course will receive a course completion certificate issued by the Cambridge AI Academy and signed by Nicholas Lane. Furthermore, students will be eligible to receive a personalized letter of performance written by Dr Nicholas Lane, suitable for use as support for applications to further study. This letter will comment on the students demonstrated abilities, skills, and progress made during the course.
Instructor - Nicholas D. Lane
Professor at the Department of Computer Science and Technology at the University of Cambridge where he leads the Machine Learning Systems Lab. Prior to joining Cambridge, Dr Lane was an Associate Professor at the University of Oxford (2017 to 2020) and Senior Lecturer at the University of London (2016 to 2017). Nicholas also has more than 10 years of experience in industrial research. Alongside his academic position, he is currently a Director at the Samsung AI Centre in Cambridge. Previously, he has been a Principal Scientist at Nokia Bell Labs and a Lead Researcher at Microsoft Research in Beijing.
Course syllabus
Lecture 1 (free, week 0) will introduce the background behind the forces that made ML and DL systems omnipresent in our daily lives. It is free and held two weeks before the primary course material begins.
Lecture 2 (week 1) will recapitulate the basics of machine and deep learning’s architecture and training and then introduce the ML/DL lifecycle. Model development, training, and inference will be discussed using examples from a range of different applications.
Lecture 3 (week 2) will further build up the students’ knowledge of deep learning by taking a deep dive in to the latest state of the art attention architectures. Applications to sequence processing will take the centre stage and will be explored using real world industrial case studies and code demos.
Lecture 4 (week 3) will introduce the crucial concept of machine and deep learning mapping onto hardware. First, the students will develop a thorough understanding of ML and DL accelerators. Then, this knowledge will be combined with the previously taught material to deliver a strong understanding and a thorough appreciation of ML/DL’s mapping onto hardware. Practical training-based examples will be used to demonstrate the point.
Lecture 5 (week 4) will enhance student’s understanding of the hardware-software interaction by focusing on the strategies that can aid the algorithm’s execution on constrained hardware. A range of advanced and automatic hardware-side as well as software-side solutions will be covered.
Lecture 6 (week 5) will investigate the opposite problem to what lecture 5 looked at – the scaling up of deep learning systems. Distributed, as well as Federated Learning systems will be introduced and contextualized as an extension to a more conventional hardware exploration. An understanding these will be derived from a close examination of their hardware organization and its inefficiencies.
Lecture 7 (week 6) will operationalize the students’ knowledge by discussing machine and deep learning compilers, and machine and deep learning stack. Thus, it will combine the knowledge of the preceding weeks into a holistic and practical model of machine and deep learning lifecycle. Industrial as well as academic case studies will be used to test and demonstrate this model.
The course will conclude with a multiple-choice exam conducted in real-time online and a 10-minute personalized one-to-one feedback session. Students will be provided with a certificate of accomplishment and any number of letters of recommendation based on their in-course performance for up to 24 months after finishing the course.