X
Home > Courses >

Applied Natural Language Processing

By Prof. Ramaseshan R   |   Chennai Mathematical Institute
Learners enrolled: 4342

Natural Language Processing (NLP) is an important area of Artificial Intelligence concerned with the processing and understanding (NLU) of a human language. The goal of NLP and NLU is to process and harness information from a large corpus of text with very little manual intervention.

This course will introduce various techniques to find similar words using the context of surrounding words, build a Language model to predict the next word and generate sentences, encode every word in the vocabulary of the corpus into a vector form that represents its context and similar words and encode a sentence for machine translation and conversation purposes.

The course will help learners to gather sufficient knowledge and proficiency in probabilistic, Artificial Neural Network (ANN) and deep learning techniques for NLP.

INTENDED AUDIENCE: Any interested learners

PER-REQUISITES: Essential – Algorithms, Python proficiency, elementary probability and statistics, Linear Algebra, basic understanding of machine learning

NOTE: Only English corpus is considered throughout this course.


Summary
Course Status : Completed
Course Type : Elective
Language for course content : English
Duration : 12 weeks
Category :
  • Computer Science and Engineering
Level : Postgraduate
Start Date : 29 Jul 2019
End Date : 18 Oct 2019
Exam Date : 16 Nov 2019 IST

Note: This exam date is subject to change based on seat availability. You can check final exam date on your hall ticket.

Page Visits



Course layout

WEEK 1:   Introduction, terminologies, empirical rules
WEEK 2:   Word to Vectors
WEEK 3:   Probability and Language Model
WEEK 4:   Neural Networks for NLP
WEEK 5:   Distributed word vectors (word embeddings) 
WEEK 6:   Recurrent Neural Network, Language Model
WEEK 7:   Statistical Machine Translation
WEEK 8:   Statistical Machine Translation, Neural Machine Translation
WEEK 9:   Neural Machine Translation
WEEK 10: Conversation Modeling, Chat-bots, dialog agents, Question Processing
WEEK 11: Information Retrieval tasks using Neural Networks- Learn to Rank, Understanding Phrases, analogies
WEEK 12: Spelling Correction using traditional and Neural networks, end notes

Books and references

Books
  1. Niladri Sekhar Dash and S. Arulmozi, Features of a Corpus. Singapore: Springer Singapore, 2018, pp. 17–34. isbn: 978-981-10-7458-5. doi: 10.1007/978- 981- 10- 7458- 5_2, url:https://doi.org/10.1007/978981-10-7458-5_2.
  2. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning, http://www.deeplearningbook.org. MIT Press, 2016.
  3. Nitin Indurkhya and Fred J Damerau, "Handbook of natural language processing," Chapman and Hall/CRC, 2010.
  4. Daniel Jurafsky and James H. Martin "Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition," 1st. Upper Saddle River, NJ, USA: Prentice Hall PTR, 2000. isbn: 0130950696.
  5. C.D. Manning et al,  "Foundations of Statistical Natural Language Processing," Mit Press. MIT Press, 1999. isbn: 9780262133609. url: https://books.google.co.in/books?id=YiFDxbEX3SUC. 
  6. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schutze, "An Introduction to Information Retrieval," Cambridge UP, 2009. Chap. 6,pp. 109–133.
  7. Jacob Perkins, "Python 3 text processing with NLTK 3 cookbook," Packt Publishing Ltd, 2014.
  8. Noah A. Smith, "Linguistic Structure Prediction. Synthesis Lectures on Human Language Technologies," Morgan and Claypool, May 2011.
Articles
  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. “Neural machine translation by jointly learning to align and translate”. English (US). In: arXiv (2014).
  2. Yoshua Bengio et al. “A Neural Probabilistic Language Model”. In: Journal of Machine Learning Research 3 (Mar. 2003), pp. 1137–1155. issn:1532-4435. 
  3. Peter F. Brown et al. “Class-based N-gram Models of Natural Language”. In: Comput. Linguist. 18.4 (Dec. 1992), pp. 467–479. issn: 0891-2017. 
  4. Peter F. Brown et al. “The Mathematics of Statistical Machine Translation: Parameter Estimation”. In: Comput. Linguist. 19.2 (June 1993), pp. 263–311. issn: 0891-2017.
  5. KyungHyun Cho et al. “On the Properties of Neural Machine Translation:Encoder-Decoder Approaches”. In: CoRR abs/1409.1259 (2014). arXiv:1409.1259. 
  6. Scott Deerwester et al. “Indexing by latent semantic analysis”. In: JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE 41.6 (1990), pp. 391–407.
  7. Chris Dyer. “Notes on Noise Contrastive Estimation and Negative Sampling”. In: CoRR abs/1410.8251 (2014). arXiv: 1410.8251. 
  8. Yoav Goldberg. “A Primer on Neural Network Models for Natural Language Processing”. In: CoRR abs/1510.00726 (2015). arXiv: 1510.00726.
  9. Nils Hadziselimovic et al. “Forgetting Is Regulated via Musashi-Mediated Transnational Control of the Arp2/3 Complex.” In: Cell 156.6 (Mar. 2014),pp. 1153–1166. issn: 1097-4172.
  10. Sepp Hochreiter and Jürgen Schmidhuber. “Long Short-Term Memory”. In: Neural Comput. 9.8 (Nov. 1997), pp. 1735–1780. issn: 0899-7667.
  11. Chiori Hori and Takaaki Hori. “End-to-end Conversation Modeling Track in DSTC6”. In: CoRR abs/1706.07440 (2017). arXiv: 1706.07440. 
  12. Andrej Karpathy, Justin Johnson, and Fei-Fei Li. “Visualizing and Understanding Recurrent Networks.” In: CoRR abs/1506.02078 (2015). 
  13. Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. “Effective Approaches to Attention-based Neural Machine Translation”. In: CoRR abs/1508.04025 (2015). arXiv:1508.04025. 
  14. Tomas Mikolov et al. “Efficient Estimation of Word Representations in Vector Space”. In: CoRR abs/1301.3781 (2013). 
  15. Franz Josef Och and Hermann Ney. “The Alignment Template Approach to Statistical Machine Translation”. In: Computational Linguistics 30.4 (Dec. 2004), pp. 417–449. issn: 0891-2017. 
  16. F. Pedregosa et al. “Scikit-learn: Machine Learning in Python”. In: Journal of Machine Learning Research 12 (2011), pp. 2825–2830.
  17. Xin Rong. “word2vec Parameter Learning Explained”. In: CoRR abs/1411.2738 (2014). arXiv: 1411.2738. url: http://arxiv.org/abs/1411.2738.
  18. Fraser W. Smith and Lars Muckli. “Nonstimulated early visual areas carry information about surrounding context”. In: Proceedings of the National Academy of Sciences 107.46 (2010), pp. 20099–20103.
Proceedings
  1. Kyunghyun Cho et al. “Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation”. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar: Association for Computational Linguistics, Oct. 2014, pp. 1724–1734.
  2. Rafal Jozefowicz, Wojciech Zaremba, and Ilya Sutskever. “An Empirical Exploration of Recurrent Network Architectures”. In: Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37. ICML’15. Lille, France: JMLR.org, 2015, pp. 2342–2350. 
  3. Quoc Le and Tomas Mikolov. “Distributed representations of sentences and documents”. In: International conference on machine learning. 2014,pp. 1188–1196. 
  4. Edward Loper and Steven Bird. “NLTK: The Natural Language Toolkit”, In: Proceedings of the ACL-02 Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics - Volume 1. ETMTNLP ’02. Philadelphia, Pennsylvania: Association for Computational Linguistics, 2002, pp. 63–70.
  5. Tomas Mikolov et al. “Distributed Representations of Words and Phrase and Their Compositionality”. In: Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2. NIPS’13.Lake Tahoe, Nevada: Curran Associates Inc., 2013, pp. 3111–3119.
  6. Andriy Mnih and Geoffrey Hinton. “A Scalable Hierarchical Distributed Language Model”. In: Proceedings of the 21st International Conference on Neural Information Processing Systems. NIPS’08. Vancouver, British Columbia, Canada: Curran Associates Inc., 2008, pp. 1081–1088. isbn:978-1-6056-0-949-2.
  7. Frederic Morin and Yoshua Bengio. “Hierarchical probabilistic neural network language model.” In: Aistats. Vol. 5. Citeseer. 2005, pp. 246–252.
  8. Kishore Papineni et al. “Bleu: a Method for Automatic Evaluation of Machine Translation”. In: Proceedings of 40th Annual Meeting of the Association for Computational Linguistics. Philadelphia, Pennsylvania, USA: Association for Computational Linguistics, July 2002, pp. 311–318. 

Instructor bio


Prof. Ramaseshan R. He is currently working as a Visiting faculty at CMI and handling NLP. He has more than 30 years of experience in research and development, teaching, product development, information technology, innovation, and convergence.

Course certificate

  • The course is free to enroll and learn from. But if you want a certificate, you have to register and write the proctored exam conducted by us in person at any of the designated exam centers.
  • The exam is optional for a fee of Rs 1000/- (Rupees one thousand only).
  • Date and Time of Exams:16th November 2019 Morning session 9am to 12 noon; Afternoon Session 2pm to 5pm.
  • Registration url: Announcements will be made when the registration form is open for registrations.
  • The online registration form has to be filled and the certification exam fee needs to be paid. More details will be made available when the exam registration form is published. If there are any changes, it will be mentioned then.
  • Please check the form for more details on the cities where the exams will be held, the conditions you agree to when you fill the form etc.

CRITERIA TO GET A CERTIFICATE
  • Average assignment score = 25% of average of best 8 assignments out of the total 12 assignments given in the course. 
  • Exam score = 75% of the proctored certification exam score out of 100
  • Final score = Average assignment score + Exam score

YOU WILL BE ELIGIBLE FOR A CERTIFICATE ONLY IF AVERAGE ASSIGNMENT SCORE >=10/25 AND EXAM SCORE >= 30/75. 
  • If one of the 2 criteria is not met, you will not get the certificate even if the Final score >= 40/100.
  • Certificate will have your name, photograph and the score in the final exam with the breakup.It will have the logos of NPTEL and IIT Madras. It will be e-verifiable at nptel.ac.in/noc.
  • Only the e-certificate will be made available. Hard copies are being discontinued from July 2019 semester and will not be dispatched.


MHRD logo Swayam logo

DOWNLOAD APP

Goto google play store

FOLLOW US