AI Course Selection IITB

Inspired by this article A survivor’s guide to Artificial Intelligence courses at Stanford, I wanted to write something akin for course structure at IITB.

This post is a repository of various courses you can do at IITB for Artificial Intelligence (AI) Track. While there is nothing officially called as AI track, it is more like an overview of different courses offered at the institute on AI topics. While the target audience is non-CS students I am pretty sure CS students can benefit a thing or two from this. Mtechs and PhD at IITB might also find something which can be useful for them. I completed my B.Tech in Electrical Engineering so there will be few things specific to EE but most of it is general enough for other streams as well. I will also be highlighting when I took the course, an ideal time to take those courses and some good practices or tips here and there most of which come from personal experience.

Before I start I want to link a fantastic post on Computer Science Opportunities by my friend Kalpesh Krishna and I definitely encourage everyone to read his blog as well. However this post will focus on a subset of the courses which is the AI part.

I also want to highlight that there are few courses which have counterparts in other departments. As I have taken only one of them I will mention the other but will mainly describe the one I have done. I will also include some of my friends comments on the courses I haven’t done but I can’t personally vouch for them.

AI Courses:

1. Basic Machine Learning Course:
   This is the foundational course on Machine Learning. There are two courses CS 403/725: Foundations of Machine Learning by Prof. Ganesh Ramakrishnan and another EE769 Introduction to Machine Learning. I took the former course (CS725) at the start of my fourth year (7th semester), and haven’t taken the latter. This is a foundational course but requires decent amount of comfort with probability. I would suggest taking this anytime after you have done a course of probability (for EE students it would be after EE 223 which is core course to be done in 3rd semester). Perhaps a good time would be 5th or 6th semester otherwise one might often miss out on appreciating many details. The course typically comes with an open ended project and I highly suggest trying to implement a research paper. Broadly it covers supervised learning and some parts of unsupervised learning. The course is fairly math-heavy and gives gentle introduction to linear regression, support vector machines, classifications, neural networks, bagging and boosting. Quite a few things are non-intuitive. Luckily the Professor has all this in video lecture format and you can review them once at home. There are also regular quizzes to keep you grounded.

   Do note that there is a newly introduced minor for the Machine Learning course in the CS department and accordingly CS725 might be restricted to graduate students and for UGs taking the minor course would be the only option. Please check with the department for course offerings.

2. Advanced Machine Learning Course:
   This a course in the CS department CS 726: Advanced Machine Learning offered by Prof. Sunita Sarawagi. I took this in my 8th semester. I don’t feel there is need to do a Basic ML course before doing this if you have done similar course online or worked on any relevant project. However, prior knowledge and understanding of neural networks, cross entropy loss etc are a must. The course starts by introducing Graphical Models in the first half. The second half comprises of neural networks and in particular delves into the cutting edge research topics which are actively being developed and I think the course content is pretty fantastic. This too comes with a course project and one can explore quite a few domain areas. I would suggest not taking this course unless you are fairly familiar with neural networks and have at least some coding experience with them otherwise the course project will be extremely difficult. A good time would be 6th semester if you have done CS725 and have good coding experience or 8th semester.

3. Foundations of Intelligent Learning Agents:
   CS 747: Foundations of Intelligent and Learning Agents is an extremely important course offered by Prof. Shivaram Kalyanakrishnan. It starts out with the Bandit Problem, to Markov Decision Process, then gently introduces concept of reinforcement learning till midsem. Post midsem is mainly a broad coverage of wide range of topics. This too comes with a project and the professor expects some novelty in the work which is a good way to explore a particular topic. The professor doesn’t allow a third year student to take the course, so the only option is to take this in the 7th semester like me.

4. Computer Vision:
   There are two courses for computer vision one in the CS dept CS763: Computer Vision by Prof. Arjun Jain and the other in EE dept EE702: Computer Vision by Prof. Subhasis Chaudhuri. I have only done the latter one. EE702 doesn’t cover the deep learning module for computer vision and focuses on classical vision techniques. CS763 instead focuses mostly on the deep learning part since it gives state of the art results and spends some time on classical techniques as well. I took EE702 in my 6th semester and the pre-requisite would be solid understanding of signals and systems and probability theory and ofcourse some maths. In some sense EE702 doesn’t fall into the category of AI but it is a good course to do. 6th semester is an ideal time to do this course.

   The following is a short review by Parth Kothari for the course CS 763. I will put it down verbatim. This course makes a decent attempt to blend the content of classical vision as well as Neural Networks related Vision into its curriculum (In Stanford, two separate courses are taken to cover these two huge domains). As a result of the combination, the course becomes a little heavy but gives you a really good idea of what existed pre and post NN era. Just like the other CS courses, the main learning happens through the assignments (there are 6 assignments in the course) and I would strongly urge to focus on them and not just on the theory. If ever you feel like giving computer vision a chance, please take this course seriously and not just let it pass away as yet another ALC course especially if you are a newcomer into the world of Vision. If you want to dig deeper, a great follow-up of this course would be to do the assignments of the equivalent course of Stanford (CS231n). They are wonderfully prepared and many of the questions explore the recent trends in the NN driven Vision domain.

   There are a couple of related courses to computer vision. While the coursework is completely different I will mention a few courses which complement Vision courses.

   1. Digital Image Processing:
      Again there are two courses one in the EE dept and the other in CS dept CS 663: Digital Image Processing. The CS course is jointly taken by Prof. Suyash Awate and Prof. Ajit Rajwade. I can vouch for the CS course in that it covers many fundamental topics. The second half of the course is especially useful as it covers a very wide range of topics like Face Recognition, JPEG compression etc. There is also a course project involved and you have plenty to learn. I took this course in my fifth semester. And I would recommend taking it around the same time.

   2. Advanced Image Processing:
      This is a course in the CS dept CS 754 - Advanced Image Processing. It covers a broad range of topics like tomography, compressed sensing, dictionary learning etc. While quite a few of them are not very relevant from AI perspective, they make a fine addition to the list of interesting courses you could do. I took it in my 6th Semester.

   3. Medical Image Processing:
      This is also offered in the CS dept CS 736: Medical Image Computing. I haven’t taken the course and the review is by courtesy of Parth Kothari. I will put it down verbatim. This course explores the math that goes into advance image processing applications. The best way to describe this course is a math-ier version of CS 663. Medical field is just an application of the theory taught in the course, so don’t fall into the trap of thinking that it has only medical applications. Because of the rigorous math involved, it might get a little tough to get along liking the course but personally, many of the core statistics and ML concepts are nicely taught in the course and the slides are really good to revise in your room. If you are up for some theoretical stuff related to IP (in contrast to the CS 763 NN module), this is the course for you. In this course, I found learning the theory much more important in comparison to doing the assignments.

5. Speech Recognition:
   Similar to Computer Vision, there are two courses one in CS dept CS753: Automatic Speech Recognition by Prof. Preethi Jyothi and EE679: Speech Processing by Prof. Preeti Rao. While the former focuses on advances in automatic speech recognition, the other focusses in general speech processing and the two have very little overlap. I have only done the former (CS753) course in my 7th semester. Basic ML is again recommended but not compulsory. It covers basics of automata theory, to Hidden Markov Models and introduction to language models till the midsem. The latter half is mainly a tour of neural networks applied to speech recognition and there were a few lectures devoted to speech synthesis as well. The course expects a strong background in probability theory and while it covers basic of Neural Networks it expects a decent level of comfort with them. The assignments are fairly interesting and have both a theory and a practical component. A good thing about the course is the course project starts fairly early which gives the opportunity to explore variety of things. I would recommend taking this in 5th or 7th semester.

6. Optimization:
   This is broadest of all topics. There are a large number of courses centered around this in different departments. I am linking 3 of the courses I know.

   • EE659: First Course in Optimization by Prof. Vivek Borkar:
     I did EE659 in my 7th semester and it is one of the most math heavy courses I have ever encountered. In theory, you don’t need probability theory to do this and can ideally be done in 5th semester as well. The only reason I would not recommend doing in 5th semester is that it is usually quite loaded and this one requires more concentration than you can imagine. I had a blast doing this course. It supported many of the intuitions that I had developed over the time I had developed in doing computer vision and machine learning stuff. There were 3 quizzes each and no midsem or endsem and the quizzes were easier than what was taught in the class. It starts off with the Bolzano Weistrauss Theorem and uses it as a base to teach all of the Convex Optimization, goes to different methods of Convex Optimization giving a flavour of Non-Convex optimization as well. Concludes with distributed optimization but those are not in the exams.

   • CS 709: Convex Optimization by Prof. Ganesh Ramakrishnan :
     I haven’t taken the course and the review is courtesy of Kalpesh Krishna. A substantial time was spent on pre-requisite stuff (linear algebra MA106 level) and it was difficult to follow the Professor at times. Also, the course content is a bit weird in that the most useful content was covered in the last month. The project is a good experience and involves a paper implementation. Assignments were a bit random though. Exams on the other hand were very well thought of and on the harder side. Unfortunately class attendance was generally on the lower side. Kalpesh took the course in his 8th semester.

   • SC 607: Optimization by Prof. Ankur Kulkarni

7. Information Retrieval:
   I am adding this for completeness but since I haven’t taken this course I cannot really add anything. It would be a good idea to look at the course content. From what I have heard (courtesy of Siddhant Garg) it is a pretty good course and a very important topic in CS. There are two courses a Basic and and Advanced course both taken by Prof. Soumen. It covers a broad range of topics relevant to Web Search, Information Retrieval and Natural Language Processing.

8. Natural Language Processing:
   The course is taken by Prof. Pushpak Bhattacharyya and it runs occassionally. Keep an eye out for this. Unfortunately I wasn’t able to do this course during my stay.

9. Math Courses:
   It is a good idea to have a strong base in Mathematics if you want to do anything in AI. There is no doubt that the amazing libraries like Keras and Pytorch make it extremely easy to code, most papers require you to understand some basic maths. Here are some of the courses which I think are quite useful.

   1. Real Analysis:
      Real Analysis is a course in the Maths department and compulsory course if you want to get minor in Maths. For EE there is another course called EE759: Applied Mathematical Analysis In Engineering. I have taken EE759 and it is an amazing course taken by Prof. Debasattam Pal. He teaches every concept very gradually and takes time to deliver the subject. It clears up many misunderstandings one might already have and paves path for all the future AI courses. I took this course in my 7th semester, but this can be easily done in the 3rd or 5th semester as well.

   2. Linear Algebra, Differential Equations and Matrices:
      MA106, MA108 also suffice for most cases. There is a course on EE636: Matrix Computation by Prof. Harish Pillai. Not at all compulsory but does provide a nice base to cover. Most importantly, stuff like Matrix norms and Eigen Value and Singular Value decomposition are taught in great detail which are kind of fundamental to all of Machine Learning. I took this in my 4th semester. Definitely helped a lot with what came latter.

   3. Probability:
      As earlier stated probability is the main pre-requisite for almost all of the courses. Every department has their own version of Probability course and mostly they are core courses. For EE dept there are two courses being EE223: Data Analysis and Interpretation and EE325: Probability And Random Processes which are core courses. Pay extreme attention to these courses as it will pay off in the long run.

10. Research Projects:
    I highly encourage and recommend students especially in their 3rd year to take a RnD project. It is an extremely important learning experience which can boost your confidence. It further helps you in deciding what you want to do in the future, if research is at all what you want to pursue. It is a good idea to contact professors early on and schedule a meeting with them to discuss if they are willing to give you a project. Make sure to be open minded when discussing projects and try to communicate with the professor frequently.

That’s all folks. Special thanks to Kalpesh Krishna, Siddhant Garg, Parth Kothari, Sudeep Salgia for their valuable inputs and proof-reading the post. If you have any comments for my post feel free to shoot me an email at ark.sadhu2904@gmail.com