
Kiến thức học máy cơ bản
Welcome to this comprehensive course on the basic concepts of machine learning (ML). Whether you are a student of computer science, an AI enthusiast, or a professional looking to refresh…

Hệ thống học từ dữ liệu đầu vào đi kèm nhãn đúng thuộc kiểu nào?
Chuẩn hóa Z‑score của một đặc trưng trên tập train nhằm mục tiêu nào?
Trong gradient descent, learning rate quá lớn thường gây hiện tượng gì?
Khi nào cần chuẩn hóa đặc trưng trước khi dùng k‑NN?
Trong Naive Bayes, giả định nào đúng về các đặc trưng?
Một nhóm thử 30 cấu hình và chọn cấu hình có Accuracy test cao nhất. Vấn đề chính là gì?
Nếu một đặc trưng có giá trị ngoại lệ rất lớn, phương pháp scaling nào thường bền vững hơn Min‑max?
Trong hồi quy tuyến tính, hệ số θ₁ trong mô hình ŷ = θ₀ + θ₁x biểu diễn gì?
Trong quyết định cây, giảm max_depth giúp gì?
Fundamentals of Machine Learning
Welcome to this comprehensive course on the basic concepts of machine learning (ML). Whether you are a student of computer science, an AI enthusiast, or a professional looking to refresh your knowledge, this guide will walk you through the essential ideas, terminology, and best practices that underpin modern ML systems. The content is organized around key questions that often appear in quizzes, providing clear explanations, examples, and practical tips.
1. Relationship Between AI, Machine Learning, and Deep Learning
Artificial Intelligence (AI) is the broadest field, encompassing any technique that enables computers to perform tasks that normally require human intelligence. Machine Learning (ML) is a sub‑field of AI that focuses on algorithms that learn patterns from data. Within ML, Deep Learning (DL) is a specialized approach that uses multi‑layer neural networks to automatically extract hierarchical features.
- AI → ML → DL: Deep learning belongs to machine learning, and machine learning belongs to AI.
- Not all AI systems use ML (e.g., rule‑based expert systems).
- Not all ML models are deep neural networks; many classic algorithms such as decision trees, k‑NN, and SVM are non‑deep.
Understanding this hierarchy helps you choose the right tool for a given problem and communicate effectively with stakeholders.
2. Supervised Learning: The Core Paradigm
When a dataset includes input features together with correct labels, the learning task is called supervised learning. The algorithm receives a pair (x, y) for each training example, where x is the feature vector and y is the target label. The goal is to learn a mapping f(x) ≈ y that can predict labels for unseen data.
- Typical supervised tasks: classification (e.g., spam detection) and regression (e.g., house‑price prediction).
- Contrast with unsupervised learning (no labels) and reinforcement learning (learning from rewards).
Choosing supervised learning when labels are available usually yields the most accurate models, provided that the data is representative and properly pre‑processed.
3. Feature Scaling with Z‑Score Normalization
Many algorithms (e.g., gradient‑based optimizers, k‑NN, SVM) are sensitive to the scale of input features. Z‑score normalization (also called standardization) transforms each feature x as follows:
z = (x - μ) / σ
where μ is the mean of the feature on the training set and σ is its standard deviation (assuming variance ≠ 0). This operation centers the data at zero and rescales it so that the variance becomes one.
- It does not force the data to follow a normal distribution.
- It preserves outliers, unlike min‑max scaling which compresses them into a fixed interval.
- Always compute
μandσon the training set and apply the same parameters to validation and test sets.
Standardization is especially useful for algorithms that assume features are centered, such as logistic regression and neural networks.
4. Learning Rate in Gradient Descent
Gradient descent iteratively updates model parameters θ using the rule:
θ ← θ - η ∇L(θ)
where η is the learning rate and ∇L(θ) is the gradient of the loss function. Setting η too high can cause the optimization to oscillate or even diverge, because the update step overshoots the minimum.
- Symptoms of a large learning rate: loss values that bounce wildly, exploding gradients, or NaN values.
- Solutions: use learning‑rate schedules, adaptive optimizers (Adam, RMSprop), or perform a learning‑rate search (e.g., using a logarithmic grid).
Choosing an appropriate learning rate is a critical hyper‑parameter tuning step that directly impacts convergence speed and final model performance.
5. Why Feature Normalization Matters for k‑Nearest Neighbors
The k‑NN algorithm classifies a query point based on the majority label among its k nearest neighbors, where distance is typically measured with Euclidean or Manhattan metrics. If one feature has a much larger numeric range than others, it will dominate the distance calculation, effectively drowning out the contribution of smaller‑scale features.
- Example: height measured in centimeters (≈180) vs. binary gender (0/1). Without scaling, height will outweigh gender in distance calculations.
- Solution: apply standardization or min‑max scaling to bring all features onto a comparable scale.
Proper scaling ensures that each feature contributes fairly, leading to more reliable neighbor selections and better classification accuracy.
6. Naïve Bayes and the Conditional Independence Assumption
Naïve Bayes classifiers are built on Bayes' theorem:
P(C|X) = \frac{P(X|C)P(C)}{P(X)}
where C is the class label and X = (x_1, x_2, …, x_n) is the feature vector. The "naïve" part refers to the assumption that features are conditionally independent given the class:
P(X|C) = \prod_{i=1}^{n} P(x_i|C)
This simplification makes computation tractable even with many features, though it is rarely true in real data. Despite the strong assumption, Naïve Bayes often performs surprisingly well, especially in text classification and spam detection.
7. Proper Model Evaluation: Avoiding Test‑Set Leakage
When you experiment with many model configurations (e.g., 30 different hyper‑parameter settings) and select the one with the highest test accuracy, you inadvertently use the test set for model selection. This leads to optimistic bias because the test data has influenced the final model choice.
- Best practice: split the data into three parts—training, validation, and test. Use the validation set for hyper‑parameter tuning, and reserve the test set strictly for final performance reporting.
- Alternatively, employ cross‑validation to obtain a more robust estimate of model performance.
Maintaining a clean separation between model development and final evaluation ensures that reported metrics reflect true generalization ability.
8. Robust Scaling for Outlier‑Heavy Features
When a feature contains extreme outliers, min‑max scaling (which maps the minimum and maximum values to 0 and 1) becomes unstable because the outlier stretches the scaling range, compressing the majority of data into a narrow band. Robust scaling mitigates this issue by using statistics that are less sensitive to outliers:
z_robust = (x - median) / IQR
where IQR (inter‑quartile range) is the difference between the 75th and 25th percentiles. This approach centers the data around the median and scales it according to the spread of the middle 50% of values, preserving the relative ordering of typical observations while reducing the impact of extreme values.
- Use robust scaling when you suspect heavy‑tailed distributions or when preprocessing pipelines must handle occasional spikes.
- Combine robust scaling with outlier detection techniques (e.g., isolation forest) for even greater resilience.
9. Summary of Key Takeaways
- AI > Machine Learning > Deep Learning: each level is a subset of the previous one.
- Supervised learning requires labeled data; it is the most common paradigm for predictive tasks.
- Z‑score standardization centers data at zero and scales variance to one, improving algorithm stability.
- A learning rate that is too large causes oscillations or divergence in gradient descent.
- Feature scaling is essential for distance‑based methods like k‑NN to prevent domination by high‑scale attributes.
- Naïve Bayes assumes conditional independence of features given the class label.
- Never use the test set for model selection; keep it untouched for the final unbiased evaluation.
- Robust scaling (median & IQR) handles outliers better than min‑max scaling.
10. Further Reading and Resources
To deepen your understanding, explore the following resources:
- Andrew Ng’s Machine Learning Course (Coursera)
- Scikit‑learn Preprocessing Documentation
- A Survey on Gradient Descent Optimization Algorithms
- Naïve Bayes from Scratch (Python)
By mastering these foundational concepts, you will be well‑equipped to tackle more advanced topics such as ensemble methods, deep neural architectures, and real‑world deployment pipelines.
