Skip to content
IT Onboarding Wiki
Artificial Intelligence
Fundamentals & Terms
Natural Language Processing (NLP)

Natural Language Processing (NLP)

Natural Language Processing (NLP) refers to the subfield of Artificial Intelligence concerned with the automated processing, interpretation, and generation of human language. The goal is to enable machines to understand and generate natural language.1 The development of powerful NLP systems is closely tied to advances in machine learning, especially deep learning. With the introduction of word embeddings such as word2vec2, GloVe3, or contextual representations like BERT4, it became possible to model linguistic meanings in vector spaces. This enables machines to perform semantic reasoning and context-sensitive understanding.

With the rise of transformer-based architectures such as GPT1 5, NLP has reached a new level. Such models handle tasks like question answering, summarization, translation, or text classification with high accuracy, sometimes without specific fine-tuning.6 7 Models like GPT-4 show how generative language processing can be combined with retrieval-based methods to improve text understanding.8 Modern NLP systems also benefit from RAG approaches that integrate external knowledge sources into the generation process. This increases reliability for complex, knowledge-intensive questions. Building on these methods, generative AI systems can be further differentiated, as described in the next section.

  1. Yenduri, G., Ramalingam, M., Selvi, G. C., Supriya, Y., Srivastava, G., Maddikunta, P. K. R., Raj, G. D., Jhaveri, R. H., Prabadevi, B., Wang, W., Vasilakos, A. V., & Gadekallu, T. R. (2024). GPT (Generative Pre-Trained Transformer)—A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions. IEEE Access, 12, 54608–54649. https://doi.org/10.1109/ACCESS.2024.3389497 ↩︎ ↩︎

  2. Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space (No. arXiv:1301.3781). arXiv. https://doi.org/10.48550/arXiv.1301.3781 ↩︎

  3. Pennington, J., Socher, R., & Manning, C. D. (2014). GloVe: Global Vectors for Word Representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, 1532–1543. https://doi.org/10.3115/v1/D14-1162 ↩︎

  4. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (No. arXiv:1810.04805). arXiv. https://doi.org/10.48550/arXiv.1810.04805 ↩︎

  5. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2023). Attention Is All You Need (No. arXiv:1706.03762). arXiv. https://doi.org/10.48550/arXiv.1706.03762 ↩︎

  6. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2021). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (No. arXiv:2005.11401). arXiv. https://doi.org/10.48550/arXiv.2005.11401 ↩︎

  7. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., … Amodei, D. (2020). Language Models are Few-Shot Learners (No. arXiv:2005.14165). arXiv. https://doi.org/10.48550/arXiv.2005.14165 ↩︎

  8. OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., Belgum, J., … Zoph, B. (2024). GPT-4 Technical Report (No. arXiv:2303.08774). arXiv. https://doi.org/10.48550/arXiv.2303.08774 ↩︎

Last updated on
Deep Learning (DL)Generative AI (GenAI)