X
Home > Courses >

Introduction to Large Language Models (LLMs)

By Prof. Tanmoy Chakraborty, Prof. Soumen Chakraborti   |   IIT Delhi, IIT Bombay
Learners enrolled: 4494   |  Exam registration: 56
ABOUT THE COURSE:
This course introduces the fundamental concepts underlying Large Language Models (LLMs). It starts with an introduction to the various problems in NLP, and discusses how to approach the problem of language modeling using deep learning. It describes the architectural intricacies of Transformers and the pre-training objectives of the differen Transformer-based models. It also discusses the recent advances in LLM research, including LLM alignment, prompting, parameter-efficient adaptation, hallucination, bias and ethical considerations. This course prepares a student to comprehend, critique and approach various research problems on LLMs.

INTENDED AUDIENCE: UG and PG students in CSE, EE, ECE, IT, Maths, etc.

PREREQUISITES: Mandatory: Machine Learning, Python Programming Optional: Deep Learning

INDUSTRY SUPPORT: All those industries whose work involves machine learning, such as Google, Microsoft, Adobe, IBM, Accenture, Adobe, JP Morgan, Wipro, Flipkart, Amazon, etc
Summary
Course Status : Upcoming
Course Type : Elective
Language for course content : English
Duration : 12 weeks
Category :
  • Electrical, Electronics and Communications Engineering
  • Information Technology
  • Communication and Signal Processing
  • Data Science
  • Artificial Intelligence
Credit Points : 3
Level : Undergraduate/Postgraduate
Start Date : 20 Jan 2025
End Date : 11 Apr 2025
Enrollment Ends : 27 Jan 2025
Exam Registration Ends : 14 Feb 2025
Exam Date : 27 Apr 2025 IST

Note: This exam date is subject to change based on seat availability. You can check final exam date on your hall ticket.

Page Visits



Course layout

Week 1
  1. Course Introduction
  2. Introduction to NLP (NLP Pipeline, Applications of NLP)
Week 2
  1. Introduction to Statistical Language Models
  2. Statistical Language Models: Advanced Smoothing and Evaluation
 Week 3
  1. Introduction to Deep Learning (Perceptron, ANN, Backpropagation, CNN)
  2. Introduction to PyTorch
 Week 4
  1. Word Representation 
    2. a. Word2Vec, fastText
    b. GloVe
  2. Tokenization Strategies
Week 5
  1. Neural Language Models
    2. a. CNN, RNN
    b. LSTM, GRU
  2. Sequence-to-Sequence Models, Greedy Decoding, Beam search
  3. Other Decoding Strategies: Nucleus Sampling, Temperature Sampling, Top-k Sampling
  4. Attention in Sequence-to-Sequence Models
Week 6
  1. Introduction to Transformers
    2. a. Self and Multi-Head Attention
    b. Positional Encoding and Layer Normalization
  2. Implementation of Transformers using PyTorch
Week 7
  1. Transfer Learning: ELMo, BERT (Encoder-only Model)
  2. Transfer Learning: GPT (Decoder-only Model), T5 (Encoder-decoder model)
  3. Introduction to HuggingFace
Week 8
  1. Instruction Fine-tuning
  2. In-context Learning and Prompting Techniques  
  3. Alignment with Human Feedback (RLHF)
Week 9
  1. Parameter-efficient Adaptation (Prompt Tuning, Prefix Tuning, LoRA) 
  2. An Alternate Formulation of Transformers: Residual Stream Perspective
  3. Interpretability Techniques
Week 10
  1. Knowledge graphs (KGs)
    a. Representation, completion
    b. Tasks: Alignment and isomorphism
    c. Distinction between graph neural networks and neural KG inference
Week 11
  1. Open-book question answering: The case for retrieving from structured and unstructured sources;retrieval-augmented inference and generation
  2. Retrieval augmentation techniques
    a. Key-value memory networks in QA for simple paths in KGs
    b. Early HotPotQA solvers, pointer networks, reading comprehension
    c. REALM, RAG, FiD, Unlimiformer
    d. KGQA (e.g., EmbedKGQA, GrailQA)
Week 12
  1. Overview of recently popular models such as GPT-4, Llama-3, Claude-3,Mistral, and Gemini
  2. Ethical NLP – Bias and Toxicity
  3. Conclusion

Books and references

  • Tanmoy Chakraborty, Introduction to Large Language Models, Wiley India, 1st Edition, 2025. ISBN : 9789363864740
  • Dan Jurafsky and James H. Martin, Speech and Language Processing, 2nd edition, Pearson Press, 2008.
  • Jacob Eisenstein, Natural Language Processing, First edition,The MIT Press, 2019.
  • Research papers published in conferences/journals likeAssociation for Computational Linguistics (ACL), Empirical Methods in Natural Language Processing (EMNLP), North American Chapter of the Association for Computational Linguistics (NAACL), Association for the Advancement of Artificial Intelligence (AAAI), International Joint Conference on Artificial Intelligence (IJCAI), Neural Information Processing Systems (NeurIPS), International Conference on Learning Representations (ICLR), Transactions of the Association for Computational Linguistics (TACL).

Instructor bio

Prof. Tanmoy Chakraborty

IIT Delhi
Professor Tanmoy Chakraborty holds the Rajiv Khemani Young Faculty Chair in AI and an Associate Professor at IIT Delhi. He leads the Laboratory for Computational Social Systems (LCS2), a research group that primarily focuses on building economical, interpretable and faithful language models and applying them specifically to mental health and cyber-informatics. He served as visiting professor at MPI Saarbrucken, TU Munich and TU Darmstadt. Tanmoy did his PhD as a Google PhD Scholar at IIT Kharagpur and postdoc at University of Maryland. Tanmoy has received numerous faculty fellowships, including the Ramanujan, DAAD, Humboldt, ELISE, PECFAR, and several faculty awards from industries like Google, LinkedIn, JP Morgan, IBM, and Adobe. He has authored two textbooks -- "Social Network Analysis" and "Introduction to Large Language Models", on which he has been offering NPTEL courses. He is an ACM Distinguished Speaker. More details may be found at tanmoychak.com.


Prof. Soumen Chakraborti

IIT Bombay
Prof.Soumen Chakrabarti is a Professor of Computer Science at IIT Bombay. He works on linking unstructured text to knowledge bases and exploiting these links for better search and ranking. Other interests include link formation and influence propagation in social networks, and personalized proximity search in graphs. He has published extensively in WWW, ACL, EMNLP, NeurIPS, ICML, AAAI, IJCAI, SIGKDD, VLDB, SIGIR, ICDE and other conferences. He won the best paper award at WWW 1999. He was coauthor on the best student paper at ECML 2008. His work on keyword search in databases got the 10-year influential paper award at ICDE 2012. He got his PhD from University of California, Berkeley and worked on Clever Web search and Focused Crawling at IBM Almaden Research Center. He has also worked at Carnegie-Mellon University and Google. He received the Bhatnagar Prize in 2014 and the Jagadis Bose Fellowship in 2019.

Course certificate

The course is free to enroll and learn from. But if you want a certificate, you have to register and write the proctored exam conducted by us in person at any of the designated exam centres.
The exam is optional for a fee of Rs 1000/- (Rupees one thousand only).
Date and Time of Exams: April 27, 2025 Morning session 9am to 12 noon; Afternoon Session 2pm to 5pm.
Registration url: Announcements will be made when the registration form is open for registrations.
The online registration form has to be filled and the certification exam fee needs to be paid. More details will be made available when the exam registration form is published. If there are any changes, it will be mentioned then.
Please check the form for more details on the cities where the exams will be held, the conditions you agree to when you fill the form etc.

CRITERIA TO GET A CERTIFICATE

Average assignment score = 25% of average of best 8 assignments out of the total 12 assignments given in the course.
Exam score = 75% of the proctored certification exam score out of 100

Final score = Average assignment score + Exam score

Please note that assignments encompass all types (including quizzes, programming tasks, and essay submissions) available in the specific week.

YOU WILL BE ELIGIBLE FOR A CERTIFICATE ONLY IF AVERAGE ASSIGNMENT SCORE >=10/25 AND EXAM SCORE >= 30/75. If one of the 2 criteria is not met, you will not get the certificate even if the Final score >= 40/100.

Certificate will have your name, photograph and the score in the final exam with the breakup.It will have the logos of NPTEL and IIT Madras .It will be e-verifiable at nptel.ac.in/noc.

Only the e-certificate will be made available. Hard copies will not be dispatched.

Once again, thanks for your interest in our online courses and certification. Happy learning.

- NPTEL team


MHRD logo Swayam logo

DOWNLOAD APP

Goto google play store

FOLLOW US