Question 1

What is Natural Language Processing (NLP)?

Accepted Answer

Natural Language Processing (NLP) is a subfield of artificial intelligence, computer science, and linguistics concerned with the interactions between computers and human language, in particular how to program computers to process and analyze large amounts of natural language data.

Question 2

What is Tokenization?

Accepted Answer

Tokenization is the process of breaking down a stream of text into smaller components called tokens. These could be words, characters, or subwords. It is usually the first step in an NLP pipeline.

Question 3

What is Stemming?

Accepted Answer

Stemming is a crude heuristic process that chops off the ends of words to find their base or root form, even if the root is not a valid word. For example, 'running' becomes 'run'.

Question 4

What is Lemmatization?

Accepted Answer

Lemmatization uses vocabulary and morphological analysis of words to remove inflectional endings and return the base or dictionary form of a word, which is known as the lemma. For instance, 'better' becomes 'good'.

Question 5

What is Stop Word Removal?

Accepted Answer

Stop words are high-frequency words that often add little lexical value to sentences, such as 'is', 'the', and 'at'. Removing them can significantly reduce the size of the vocabulary and improve model training speed and accuracy.

Question 6

What is a Bag-of-Words (BoW) model?

Accepted Answer

A Bag-of-Words model is a simple representation of text used in NLP. It describes the occurrence of words within a document, ignoring word order and grammar but keeping multiplicity (frequency).

Question 7

What is TF-IDF?

Accepted Answer

TF-IDF stands for Term Frequency-Inverse Document Frequency. It's a numerical statistic intended to reflect how important a word is to a document in a collection or corpus. Words that appear frequently in one document but rarely across all documents get the highest score.

Question 8

What are Word Embeddings?

Accepted Answer

Word embeddings are a type of word representation that allows words with similar meaning to have a similar representation. They map words or phrases to vectors of real numbers in a continuous vector space (e.g., Word2Vec, GloVe).

Question 9

What is Word2Vec?

Accepted Answer

Word2Vec is a popular algorithm used for generating dense word embeddings. Created by researchers at Google, it uses a two-layer neural network (either Skip-Gram or CBOW) to reconstruct linguistic contexts of words.

Question 10

What is Part-of-Speech (POS) Tagging?

Accepted Answer

POS tagging is the process of marking up a word in a text as corresponding to a particular part of speech based on both its definition and its context (e.g., tagging a word as a noun, verb, adjective).

Question 11

What is Named Entity Recognition (NER)?

Accepted Answer

NER is an information extraction technique that seeks to locate and classify named entities mentioned in unstructured text into predefined categories such as person names, organizations, locations, medical codes, time expressions, quantities.

Question 12

What is Sentiment Analysis?

Accepted Answer

Sentiment analysis (or opinion mining) uses NLP to identify, extract, and quantify subjective information from text, typically classifying the polarity as positive, negative, or neutral.

Question 13

What are N-grams?

Accepted Answer

An N-gram is a contiguous sequence of n items from a given sample of text or speech. When n=1 it's a unigram, n=2 is a bigram, and n=3 is a trigram. It captures context that BoW misses.

Question 14

What is Levenshtein Distance?

Accepted Answer

Levenshtein Distance is a string metric for measuring the difference between two sequences. It is the minimum number of single-character edits (insertions, deletions, substitutions) required to change one word into the other.

Question 15

What is an RNN (Recurrent Neural Network)?

Accepted Answer

An RNN is a class of artificial neural networks where connections between nodes can create a cycle, allowing output from some nodes to affect subsequent input to the same nodes. This makes them good for sequential data like text.

Question 16

Why use LSTM instead of a basic RNN?

Accepted Answer

LSTMs (Long Short-Term Memory networks) are a special kind of RNN capable of learning long-term dependencies. They solve the vanishing gradient problem of standard RNNs through specialized memory cells and gating mechanisms.

Question 17

What is the Vanishing Gradient Problem?

Accepted Answer

During backpropagation in deep neural networks (especially RNNs), gradients are recursively multiplied. If these gradients are small, the resulting gradient value shrinks exponentially, stopping earlier layers from learning.

Question 18

What is perplexity in Language Modeling?

Accepted Answer

Perplexity is a measurement of how well a probability distribution or probability model predicts a sample. In NLP, a lower perplexity score indicates the language model is better at predicting the next word.

Question 19

What is BLEU score?

Accepted Answer

BLEU (Bilingual Evaluation Understudy) is an algorithm for evaluating the quality of text which has been machine-translated from one natural language to another based on n-gram overlap between candidate and reference texts.

Question 20

What is ROUGE score?

Accepted Answer

ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is a set of metrics used for evaluating automatic summarization and machine translation, focusing primarily on recall metrics.

Question 21

What is a Language Model?

Accepted Answer

A language model learns the probability distribution over sequences of words. It tries to predict the next word in a sequence given the previous words.

Question 22

What is Text Generation generation constraints?

Accepted Answer

Constraints like temperature, top-k sampling, and top-p (nucleus) sampling help control how deterministic or creative the output sequence from a language model is.

Question 23

Calculate Document Frequency.

Accepted Answer

Document Frequency (DF) is the number of documents in which a term appears. It helps determine the uniqueness of the term globally.

Question 24

What is the GloVe embedding?

Accepted Answer

GloVe (Global Vectors) is an unsupervised learning algorithm for obtaining vector representations for words based on aggregated global word-word co-occurrence statistics.

Question 25

Why sequence padding is important?

Accepted Answer

Neural networks process batches in matrices, so all input sequences must be the same length. Padding adds neutral tokens (like 0) to shorter sequences so they match the maximum sequence length.

Question 26

Explain the Attention mechanism.

Accepted Answer

Attention mechanisms allow models to focus on specific parts of an input sequence when predicting the output, rather than relying on a single fixed-length hidden vector. It computes a weighted sum of the inputs based on relevance.

Question 27

What is Self-Attention?

Accepted Answer

Self-attention, or intra-attention, is an attention mechanism relating different positions of a single sequence in order to compute a representation of the sequence. It's the core component of Transformers.

Question 28

Explain the Sequence-to-Sequence (Seq2Seq) Model.

Accepted Answer

Seq2Seq models take an input sequence and generate an output sequence. They typically use an Encoder to read the input into a context vector, and a Decoder to generate the output sequence (e.g., for translation).

Question 29

What is Teacher Forcing?

Accepted Answer

Teacher forcing is a fast and effective training technique for RNNs/Seq2Seq models where the model receives the ground truth output from the previous time step as input for the current time step, instead of its own prediction.

Question 30

What is Subword Tokenization?

Accepted Answer

Subword algorithms define tokens as characters or subwords, allowing models to mitigate the Out-Of-Vocabulary (OOV) problem efficiently. Examples include Byte-Pair Encoding (BPE), WordPiece, and SentencePiece.

NLP & LLM Interview Master

Difficulty Filters

Tools & Options

What is Natural Language Processing (NLP)?

What is Tokenization?

What is Stemming?

What is Lemmatization?

What is Stop Word Removal?

What is a Bag-of-Words (BoW) model?

What is TF-IDF?

What are Word Embeddings?

What is Word2Vec?

What is Part-of-Speech (POS) Tagging?

What is Named Entity Recognition (NER)?

What is Sentiment Analysis?

What are N-grams?

What is Levenshtein Distance?

What is an RNN (Recurrent Neural Network)?

Why use LSTM instead of a basic RNN?

What is the Vanishing Gradient Problem?

What is perplexity in Language Modeling?

What is BLEU score?

What is ROUGE score?

What is a Language Model?

What is Text Generation generation constraints?

Calculate Document Frequency.

What is the GloVe embedding?

Why sequence padding is important?

Explain the Attention mechanism.

What is Self-Attention?

Explain the Sequence-to-Sequence (Seq2Seq) Model.

What is Teacher Forcing?

What is Subword Tokenization?

How does Byte-Pair Encoding (BPE) work?

What is WordPiece?

Explain the concept of Word Sense Disambiguation (WSD).

What is an LLM (Large Language Model)?

What is BERT?

Explain the Masked Language Modeling (MLM) task in BERT.

What is Next Sentence Prediction (NSP) in BERT?

What is fine-tuning in NLP?

What is the difference between Extractive and Abstractive Summarization?

How do Convolutional Neural Networks (CNNs) perform in NLP?

Explain the Skip-gram architecture in Word2Vec.

Explain Continuous Bag of Words (CBOW) architecture.

What is Negative Sampling in Word2Vec?

What is Coreference Resolution?

Difference between Generative and Discriminative models.

What is the ELMo algorithm?

What is Top-K vs Top-P sampling?

What are the common evaluation metrics for NER?

How do you handle Class Imbalance in NLP?

What are Zero-shot and Few-shot learning?

Explain the architecture of a Transformer model.

What is the purpose of Positional Encoding in Transformers?

Explain the calculation of Scaled Dot-Product Attention.

What is Multi-Head Attention?

Difference between GPT architectures and BERT?

Explain Retrieval Augmented Generation (RAG).

What is LoRA (Low-Rank Adaptation)?

What is RLHF (Reinforcement Learning from Human Feedback)?

What is Prompt Engineering vs Prompt Tuning?

Explain the difference between T5 and standard Seq2Seq models.

What are Vector Databases?

Explain the 'Hallucination' problem in LLMs and mitigation strategies.

What are KV (Key-Value) Caches in LLM decoding?

What is FlashAttention?

What is Quantization in neural networks? (e.g. 4-bit, 8-bit)

Differentiate between RoPE and Absolute Positional Encodings.

Explain ALiBi (Attention with Linear Biases).

Explain Directed Acyclic Graphs (DAGs) in Dependency Parsing.

What is the Conditional Random Field (CRF) layer used for in NER?

What is Knowledge Distillation?

Explain Mixture of Experts (MoE) architecture.

How does Semantic Search differ from Lexical/Keyword Search?

What is Cross-Encoder vs Bi-Encoder?

What is Token Healing in LLMs?

Explain Direct Preference Optimization (DPO).

Implement Cosine Similarity in Python.

Calculate TF-IDF for word 'data'.