Deep Learning for Natural Language Processing

By Prof. Pawan Goyal | IIT Kharagpur

Learners enrolled: 2543 | Exam registration: 38

ABOUT THE COURSE:

Natural Language Processing finds its applications nearly everywhere -- be it Machine Translation,Question answering, Text Summarization, Dialogs, etc. In the last decade, Deep Learning basedmethods have given very good performance across a variety of NLP tasks, and have become a default choice for NLP problems. This course aims to give a thorough understanding of various deep learning architectures along with their specific use-cases in NLP.The course will also introduce the fundamental ideas behind training as well as fine-tuning/prompting the Large Language Models, which include in-context-learning, Parameter-efficient-fine-tuning,Reinforcement Learning through Human Feedback (RLHF). The course will also offer hands-on tutorials to help students master this subject.

INTENDED AUDIENCE: Advanced UG and PG students

PREREQUISITES: Participants should have done a course on MachineLearning, and should know basics of PythonProgramming

INDUSTRY SUPPORT: Google,Microsoft, Amazon, Flipkart, Adobe

Summary

Course Status :	Upcoming
Course Type :	Elective
Language for course content :	English
Duration :	12 weeks
Category :	Computer Science and Engineering Data Science Artificial Intelligence
Credit Points :	3
Level :	Undergraduate/Postgraduate
Start Date :	20 Jan 2025
End Date :	11 Apr 2025
Enrollment Ends :	27 Jan 2025
Exam Registration Ends :	14 Feb 2025
Exam Date :	27 Apr 2025 IST

Note: This exam date is subject to change based on seat availability. You can check final exam date on your hall ticket.

Page Visits

Course layout

Week 1:

Introduction to NLP: What is Natural Language Processing? A brief primer on word and sentence level tasks and n-gram language Model.

Week 2: Introduction to Deep Learning

Shallow and Deep Neural Networks
Representation Learning

Week 3: Word Representations

Word2Vec
Glove
fastText,
Multilingual representations with emphasis on Indian Languages

Week 4: Recurrent Neural Networks

RNN LMs
GRUs, LSTMs, Bi-LSTMs
LSTMs for Sequence Labeling
LSTMs for Sequence to Sequence

Week 5: Attention Mechanism

Sequence to Sequence with Attention
Transformers: Attention is all you need

Week 6: Self-supervised learning (SSL), Pretraining

Designing SSL objectives
Pretrained Bi-LSTMs: ELMO
Pretrained Transformers: BERT, GPT, T5, BART

Week 7:

Applications: Question Answering, Dialog Modeling, TextSummarization
Multilingual extension with application to Indian languages

Week 8: Instruction Fine-tuning, FLAN-T5, Reinforcement Learningthrough Human Feedback (RLHF)

Week 9: In-context learning, chain-of-thought prompting. ScalingLaws. Various Large Language Models and unique architectural differences

Week 10: Parameter Efficient Fine-tuning (PEFT) - LoRA, QLoRA

Week 11: Handling Long Context, Retrieval Augmented Generation(RAG)

Week 12: Analysis and Interpretability, ethical considerations

Books and references

Daniel Jurafsky and James H. Martin. 2024. Speech andLanguage Processing. 3rd Edition.https://web.stanford.edu/~jurafsky/slp3/

Instructor bio

Prof. Pawan Goyal

IIT Kharagpur

Pawan Goyal is an Associate Professor at the Department of Computer Science and Engineering, IIT Kharagpur. Prior to joining IIT Kharagpur, he received his B. Tech. degree in Electrical Engineering from IIT Kanpur in 2007 and his PhD degree from University of Ulster, UK in 2011. He was then a post doctoral fellow at INRIA Paris Rocquencourt. His research interests include Natural Language Understanding, Information Retrieval and Sanskrit Computational Linguistics. He has published his research work in various top conferences including ACL, EMNLP, NAACL, KDD, SIGIR, AAAI, IJCAI, etc. He was the recipient of Google India AI/ML research awards 2020 and INAE Young Engineers Award 2020.

Course certificate

The course is free to enroll and learn from. But if you want a certificate, you have to register and write the proctored exam conducted by us in person at any of the designated exam centres.

The exam is optional for a fee of Rs 1000/- (Rupees one thousand only).

Date and Time of Exams: EXAM DATE : April 27, 2025 Morning session 9am to 12 noon; Afternoon Session 2pm to 5pm.

Registration url: Announcements will be made when the registration form is open for registrations.

The online registration form has to be filled and the certification exam fee needs to be paid. More details will be made available when the exam registration form is published. If there are any changes, it will be mentioned then.

Please check the form for more details on the cities where the exams will be held, the conditions you agree to when you fill the form etc.

CRITERIA TO GET A CERTIFICATE

Average assignment score = 25% of average of best 8 assignments out of the total 12 assignments given in the course.
Exam score = 75% of the proctored certification exam score out of 100

Final score = Average assignment score + Exam score

Please note that assignments encompass all types (including quizzes, programming tasks, and essay submissions) available in the specific week.

YOU WILL BE ELIGIBLE FOR A CERTIFICATE ONLY IF AVERAGE ASSIGNMENT SCORE >=10/25 AND EXAM SCORE >= 30/75. If one of the 2 criteria is not met, you will not get the certificate even if the Final score >= 40/100.

Certificate will have your name, photograph and the score in the final exam with the breakup.It will have the logos of NPTEL and IIT Kharagpur .It will be e-verifiable at nptel.ac.in/noc.

Only the e-certificate will be made available. Hard copies will not be dispatched.

Once again, thanks for your interest in our online courses and certification. Happy learning.

- NPTEL team

SWAYAM Helpline / Support