Lesson 3: Machine Learning and Neural Networks

Introduction

From examples to learning

In Lesson 1, you learned what artificial intelligence is and where it appears. In Lesson 2, you compared traditional software with systems that learn patterns from data. Now we are going one level deeper.

Imagine a computer receiving hundreds of customer messages. People describe the same problem in different ways. One person writes, "I was charged twice." Another writes, "Two payments came out." A third says, "There is a duplicate charge on my account." A fixed list of keywords may miss some of these messages. A machine-learning model can be trained on examples so it can recognize patterns in new messages.

That is the central idea of machine learning: use data and a training process to build a model that can make useful predictions or decisions about new inputs.

Neural networks are one family of machine-learning models. They are made of connected mathematical operations arranged in layers. They are not miniature human brains, and they do not become reliable simply because they have many layers. Their performance depends on the data, the training process, the model design, and the way people evaluate and use the results.

You do not need to code or understand advanced mathematics to follow the main lesson. The Beginner path focuses on the ideas. The Business path applies them to a fictional company. The Advanced path introduces the technical details and small Python examples.

By the end, you should be able to explain how a model learns from examples, describe what a neural network does, and ask the questions that matter before trusting a model.

What you will learn

Explain the difference between an algorithm, a model, training, and inference.
Identify examples, features, labels, and predictions in a simple machine-learning task.
Compare supervised, unsupervised, reinforcement, and self-supervised learning at a high level.
Describe the basic parts of a neural network, including neurons, weights, biases, layers, and activation functions.
Explain how a model is trained and why it must be evaluated on data it did not train on.
Recognize overfitting, underfitting, data leakage, bias, and other reasons a model may fail.
Identify a low-risk business task where machine learning might help and describe how a person would review its output.

The lesson

1. Machine learning: from examples to a model

Machine learning is an approach to artificial intelligence in which a computer system uses data and a training procedure to create a model. The model captures patterns that can help it make a prediction, classify an input, rank possibilities, or support another task.

It helps to distinguish four terms:

Algorithm: A defined procedure used to carry out a task. In machine learning, an algorithm describes how a model is trained or how a calculation is performed.
Data: The examples or information used to train, evaluate, or operate the system.
Model: The result of training. It contains learned parameters that are used to produce an output from an input.
Training: The process of adjusting the model using data so that its outputs improve according to a chosen objective.

After training, the model can be used on new information. This is called inference. For example, a model may be trained using labeled customer messages. Later, when a new message arrives, the system uses the trained model to estimate its category.

Training and inference are different activities. Training changes the model's learned parameters. Inference uses the current model to produce an output. Some systems can be updated or retrained over time, but a model does not automatically learn from every conversation or every prediction. That depends on how the system has been designed.

Machine learning is not a computer writing its own software without direction. People still define the task, choose and prepare data, select a learning method, set an objective, evaluate results, and decide how the model may be used.

A useful mental model: training is the practice process; the trained model is what the system has learned from that process; inference is using that model on a new example.

2. The ingredients: examples, features, labels, and predictions

A machine-learning project begins with a clearly defined task and suitable data. A dataset is usually made up of examples. Each example contains information the model can use.

A feature is an input measurement or piece of information used by a model. Features might be numbers, categories, words, image pixels, or other representations. A label is the answer or outcome supplied for an example in a supervised-learning task.

Imagine a fictional business called Cedar Trail RV. It wants to sort incoming service messages into categories so employees can find the right queue.

One training example might contain:

Message text: "The refrigerator stopped cooling during our trip."
Features: words or numerical representations derived from the message.
Label: "service issue."

Another example might say, "Can I move my appointment to Friday?" Its label might be "scheduling."

During supervised training, the model uses many examples and their labels to learn patterns associated with each category. Later, a new message arrives without a label. The model produces a prediction, such as "scheduling" or "service issue." A person or a separate application rule may then confirm the correct routing.

Good data is not simply a large pile of records. It should be relevant to the task, permitted for the intended use, sufficiently representative of real inputs, and labeled consistently when labels are required. Missing examples, incorrect labels, duplicated records, or data collected in an unrepresentative way can create misleading results.

Features also matter. A feature can be technically available but inappropriate to use. For example, information that reveals a person's identity may be unnecessary for a simple message-routing task. Use only the information needed, and follow applicable privacy rules and organizational policies.

Before asking which algorithm to use, ask: What is one example? What information is available at the moment of prediction? What outcome are we trying to predict? How will we know the outcome is correct?

3. Four common ways machine-learning systems learn

Machine learning includes several learning approaches. The right approach depends on what information is available and what the system is expected to do.

Supervised learning

In supervised learning, training examples include both inputs and known target answers, called labels. The model learns a relationship between the inputs and the labels.

Two common supervised tasks are:

Classification: Predict a category, such as whether a message concerns billing, scheduling, or service.
Regression: Predict a numerical value, such as an estimated delivery time or expected demand.

The model's prediction can be compared with the known label or value during training.

Unsupervised learning

In unsupervised learning, the training data does not include the target labels the model is meant to predict. The method looks for structure or patterns in the data.

For example, a business might explore whether product descriptions naturally form groups based on shared characteristics. Clustering is one technique for grouping similar examples. The resulting groups still need human interpretation. A cluster is not automatically a meaningful business category.

Reinforcement learning

In reinforcement learning, an agent interacts with an environment, takes actions, and receives rewards or penalties according to a defined reward signal. The learning process aims to improve the agent's choices over time.

This approach is used in areas such as simulated control tasks, robotics research, and game-playing systems. Designing the reward matters. If the reward does not reflect the real goal, an agent may learn behavior that technically earns reward but is undesirable.

Self-supervised learning

In self-supervised learning, a training signal is created from the data itself rather than requiring a person to label every example. A language model, for example, can be trained to predict missing or next pieces of text from large collections of text.

This idea will become important in Lesson 4, when we study large language models.

These categories are useful starting points, not four sealed boxes. Modern systems may combine learning approaches, and the same application can use different methods at different stages.

4. Neural networks: layers of connected calculations

A neural network is a machine-learning model made from connected units that perform mathematical calculations. These units are often called neurons or nodes. The design was loosely inspired by some ideas about biological nervous systems, but an artificial neural network is a mathematical and computational system. It is not a digital copy of a human brain.

A simplified network has three kinds of layers:

Input layer: Receives the information supplied to the model. The inputs are represented as numbers.
Hidden layers: Transform information through learned calculations. A network may have one or many hidden layers.
Output layer: Produces the model's result, such as a category score, probability-like value, or numerical prediction.

A basic artificial neuron receives input values. Each input is multiplied by a learned weight, the weighted values are combined, and a bias is added. An activation function then transforms the result. The output can be passed to another layer.

Weights determine how strongly different inputs contribute to a calculation. A bias shifts the calculation. An activation function helps a network represent more complex, nonlinear relationships. Without suitable nonlinear transformations, stacking certain simple linear operations would still behave like one linear operation.

The hidden layers allow a network to build intermediate representations of the input. In an image task, early transformations may respond to simple visual patterns while later transformations combine information into more complex patterns. That is a simplified illustration; the exact representations depend on the architecture and training.

The word deep commonly refers to a neural network with multiple layers of learned transformations. More layers do not automatically make a model better. A larger network can require more data, computation, careful training, and evaluation. The design should fit the task.

Remember: a neural network is a collection of learned calculations. It does not become correct, fair, or wise merely because it is called a neural network.

5. How a neural network is trained

Training a neural network involves repeatedly making predictions, measuring errors, and adjusting learned parameters.

A simplified training cycle looks like this:

Receive an example. The network is given an input, represented as numbers.
Make a prediction. Information moves through the layers. This calculation is called a forward pass.
Measure the difference. During supervised training, the prediction is compared with the known target using a mathematical measure called a loss function.
Calculate how to adjust the parameters. Backpropagation calculates how changes to the network's parameters relate to the loss.
Update the parameters. An optimizer uses those calculations to adjust weights and biases.
Repeat. The process continues across examples and training rounds.

The loss is a measure used by the training process to represent how far a prediction is from the target according to the chosen objective. Lower loss during training can indicate that the model is fitting its training examples better. It does not, by itself, prove that the model will perform well on new data.

Backpropagation is a method for calculating how the loss changes with respect to parameters throughout a network. An optimizer, such as a gradient-based optimizer, uses those calculations to update parameters. You do not need to calculate derivatives by hand for this lesson. The important idea is that the training process has a feedback mechanism for adjusting the model.

An epoch is one pass through the training dataset. A batch is a group of examples processed together during an update. Training may use many epochs and batches, depending on the task and method.

Several settings influence training. These are called hyperparameters. Examples include the learning rate, number of layers, number of units, batch size, and number of training epochs. They are choices made around the training process, rather than learned weights in the same sense as model parameters.

Training is an optimization process, not a guarantee of understanding. A model can reduce its training loss while learning patterns that do not hold in the real world. That is why evaluation on separate data is essential.

6. Training, validation, and testing

A model should not be judged only by how well it performs on the examples used to train it. It must also be evaluated on examples it did not use to learn.

A common approach divides available data into separate groups:

Training set: Used to fit the model's parameters.
Validation set: Used during development to compare choices, tune settings, or decide when to stop training.
Test set: Held aside for a final evaluation after the development choices have been made.

The test set is meant to provide an estimate of how the model may perform on new examples from a similar setting. If people repeatedly inspect the test results and use them to change the model, the test set is no longer a clean final check.

This is one reason data leakage is dangerous. Leakage occurs when information that should not be available during training or model selection influences the process. For example, a model might accidentally receive a field that was created only after the event it is supposed to predict. Or information from the test set might influence training decisions. The evaluation can then look better than real-world performance.

The way data is divided also matters. If several messages come from the same customer, placing near-duplicates in both training and test sets may make the test easier than the real task. For time-based predictions, it may be more appropriate to train on earlier records and test on later records.

The evaluation measure should fit the problem. Accuracy measures the share of predictions that are correct. Precision asks how many predicted positive cases were actually positive. Recall asks how many of the actual positive cases the model found. Other tasks need other measures.

A single score rarely tells the whole story. Review errors by category, consider the cost of different mistakes, and compare the model with a simple baseline. A model is useful only if it performs well enough for the actual task and operating conditions.

7. When models fail: overfitting, underfitting, and changing data

A model can perform well during training and still fail when it encounters new examples. Understanding why is a core part of machine learning.

Overfitting happens when a model learns details or noise that are too specific to its training examples. It may perform very well on training data but poorly on new data. One analogy is memorizing the answers to a practice worksheet without learning the method needed for a different question.

Underfitting happens when a model is too limited, insufficiently trained, or otherwise unable to capture the useful patterns in the task. It performs poorly even on the training examples. One analogy is trying to describe a complicated picture with only one simple line.

The goal is not to memorize the training set. The goal is to generalize, meaning perform usefully on new examples that are relevant to the task.

Data can also change after a model is deployed. Customers may use different language, products may change, a process may be updated, or the environment may shift. Data drift usually means the distribution of model inputs has changed. Concept drift means the relationship between inputs and the target has changed. Either can make patterns learned from older data less useful.

A model should therefore be monitored after deployment. Teams need a way to detect poor results, collect appropriate feedback, investigate changes, and decide whether retraining or redesign is needed. Retraining is not automatically the right answer if the underlying task, labels, or data collection process are flawed.

A practical warning: a model that performed well in a demonstration is not automatically ready for production. Test it with representative examples, examine the mistakes, define a fallback, and monitor the system under real operating conditions.

8. The human role: data quality, fairness, and responsible use

Machine-learning systems are shaped by human choices. People decide what problem to solve, which data to collect, how examples are labeled, which features are used, what counts as success, and how outputs are acted upon.

If the training examples do not represent the people or situations where the model will be used, performance may be uneven. If labels are inconsistent, the model may learn inconsistent patterns. If the data reflects past unfair practices, the model may reproduce or amplify them. A technically accurate prediction can still be inappropriate for a particular decision.

Privacy matters as well. Data should be collected and used only for a legitimate, permitted purpose. Sensitive information should not be added simply because it is available. A team should understand where data is stored, who can access it, how long it is retained, and whether it may be sent to an external service.

The consequences of error should guide the level of oversight. A model that groups fictional product descriptions for exploration is different from a system that influences access to credit, employment, healthcare, housing, or another consequential opportunity. High-impact uses require specialized legal, technical, and governance review. A beginner exercise or small demonstration is not a basis for making such decisions.

A responsible workflow includes:

A clearly defined task and intended users.
Data that is appropriate, representative, and permitted.
A baseline and an evaluation plan.
Review of errors and uneven performance.
Clear limits on what the model may decide or do.
Human escalation and a fallback when the model is uncertain or unavailable.
Monitoring after deployment.

AI can support human work, but people and organizations remain responsible for how systems are designed and used.

9. What to remember

Machine learning uses data and a training process to create a model that can make predictions or support decisions about new inputs. A model is not the same thing as the algorithm used to train it, and training is not the same as inference.

Features are the information a model receives. Labels are known target answers used in supervised learning. Different learning approaches use different kinds of feedback: supervised learning uses labeled examples, unsupervised learning looks for structure without target labels, reinforcement learning uses rewards from interaction, and self-supervised learning creates training signals from the data itself.

Neural networks are models built from connected calculations arranged in layers. Their weights, biases, and activation functions are adjusted during training. Backpropagation helps calculate parameter updates, and an optimizer applies them.

The most important lesson is that training performance is not enough. A model must be evaluated on data it did not train on, checked for overfitting and leakage, and monitored as conditions change.

In the next lesson, we will examine large language models. You will see how the ideas of data, neural networks, and self-supervised learning connect to systems that generate and work with language.

Try it

Four ways a model can learn

Compare what information is available during learning and what the system is trying to discover. These are simplified examples, not complete descriptions of every modern system.

Approach	Learning signal	Example task	Typical output
Supervised	Examples with known labels or target values	Classify a customer message	A category or numerical estimate
Unsupervised	Data without supplied target labels	Group similar product descriptions	Clusters or discovered structure
Reinforcement	Rewards or penalties from actions in an environment	Learn a control strategy in a simulation	A policy for choosing actions
Self-supervised	A training target derived from the data itself	Predict a missing or next piece of text	A model trained to represent or predict patterns

Follow a model through the training cycle

Move through the steps in order. Notice that preparing data and evaluating the result are part of the process, not optional details added after the model is built.

Step 1 of 7
Define the task
State exactly what the model should predict or produce. Decide who will use the output and what happens next.
Step 2 of 7
Prepare suitable examples
Collect permitted, relevant data. Check missing values, duplicates, representation, and label consistency.
Step 3 of 7
Choose a learning approach
Decide whether the task is supervised, unsupervised, reinforcement-based, self-supervised, or a combination.
Step 4 of 7
Train the model
Use the training data and a chosen algorithm to adjust the model's learned parameters.
Step 5 of 7
Validate and improve
Use separate validation data to compare settings and identify problems such as overfitting.
Step 6 of 7
Test and review
Evaluate the selected model on held-out test data. Inspect the kinds of errors it makes, not just one overall score.
Step 7 of 7
Deploy and monitor
Set permissions, human review, and fallback steps. Monitor real-world results and investigate changes.

Identify the part of the learning example

Read each item and decide whether it is a feature, a label, a model, or an inference-time prediction before revealing the answer.

The words and numerical representation from a service message

Features. They are the input information supplied to the model.

The known category 'scheduling' attached to a training message

Label. It is the target answer for that supervised training example.

The trained system that has adjusted weights based on many examples

Model. It contains learned parameters used to produce outputs.

The model estimates that a new message is about a repair

Prediction. This is an output produced during inference.

Ask for an explanation that teaches the process

Compare the vague request with the more useful one. The stronger request gives the AI a learner level, a sequence, and a way to check understanding.

Weaker prompt

Explain neural networks.

Stronger prompt

I am a beginner learning about machine learning. Explain how a neural network makes a prediction and how training changes its weights. Use a simple analogy, define input layer, hidden layer, output layer, weight, bias, activation function, loss, and backpropagation in plain language. Do not assume I know calculus. Finish with three short questions to check my understanding, and explain the answers.

The stronger prompt names the audience, learning goal, required concepts, level of mathematics, and desired practice. It also asks for explanations rather than a list of unexplained terms.

Map a low-risk machine-learning idea

Use this prompt with an AI assistant to explore a possible project. Do not include confidential records or personal information.

Help me evaluate a low-risk machine-learning idea. The task I am considering is: [describe the task without private information]. Explain what the model would predict, what examples and features might be needed, whether labels would be available, what a simple baseline would be, how to split and evaluate the data, what mistakes could happen, and where a person should review the output. Do not invent data or claim the project will work. Ask me for missing information before making assumptions.

Hands-on exercises

Beginner AI Beginners

Beginner: build a learning example

Choose a simple classification task, such as sorting fictional messages into "billing," "scheduling," and "service."

Create six fictional examples. For each example, write the input, identify two possible features, and assign a label. Then write one new message that was not part of your six examples and explain what prediction a model might make.

Finish by answering:

What makes a feature different from a label?
Why could a model make a mistake on the new message?
What would a person check before acting on the prediction?

Do not use real customer messages or personal information.

What a good result looks like: A small fictional set of labeled examples, a new example, and a short explanation of features, labels, prediction, and human review.

Beginner: explain a neural network without jargon

Explain a neural network to a friend who has never studied AI. Your explanation must use an everyday analogy and correctly include the role of the input layer, hidden layers, output layer, weights, and training.

Then explain one thing the analogy does not capture. Avoid saying that a neural network thinks or learns exactly like a human brain.

What a good result looks like: A short, accurate explanation that distinguishes an artificial neural network from a biological brain and describes the flow of information through layers.

Business Business Owners and Professionals

Business: evaluate a fictional dealership use case

Use the fictional Cedar Trail RV business described in the Business section. It receives service messages with varied wording and wants to suggest a queue such as scheduling, parts question, or service concern.

Prepare a one-page project brief with:

The exact prediction task and who will use the result.
Three possible categories and examples of messages for each.
The features the model may need, and information it should not need.
How staff could label a starter dataset.
A simple rules-based baseline to compare against.
How you would separate training, validation, and test examples.
Which errors matter most and how a staff member would review uncertain cases.
A fallback if the model is unavailable or the prediction is unclear.
One measurable way to compare the pilot with the current process.

Do not propose using the model to make credit, financing, employment, safety, or other high-impact decisions.

What a good result looks like: A scoped, low-risk project brief that defines the task, data, baseline, evaluation, oversight, and fallback without promising that machine learning will improve the process.

Advanced Advanced AI and Technical Development

Advanced: compare a perceptron with a small neural network

Review the two Python examples in the Advanced companion. The first trains a single perceptron to represent the OR pattern. The second trains a small network with a hidden layer to represent XOR.

Run the code if you have a Python environment, or trace the steps on paper. Then answer:

What are the inputs and target labels?
Which values are learned parameters?
Why can a single linear threshold represent OR?
Why does XOR require a nonlinear decision boundary?
What does the hidden layer add?
Why do four training examples demonstrate an idea but provide no meaningful evidence of real-world generalization?

Change one setting in the second example, such as the number of training epochs or learning rate. Record what changes, and do not assume one run proves a general rule.

What a good result looks like: A technically grounded comparison of a single perceptron and a small multilayer network, including a clear statement of the toy dataset's limitations.

Business

Path B: Machine learning in a business

Machine learning can help a business when it has a clearly defined task and enough suitable examples to learn useful patterns. It is not a shortcut around unclear processes, poor data, or missing accountability.

Consider the fictional Cedar Trail RV, a dealership that receives service messages through a website, phone notes, and email. Employees spend time reading each message and deciding whether it concerns scheduling, a parts question, a warranty question, or a service concern.

A low-risk pilot could use supervised learning to suggest a category for each incoming message. The model would not decide whether a repair is covered, promise a completion date, approve a payment, or change a customer record by itself. It would make a suggestion for an employee to review.

Design the pilot before choosing the model

1. Define the task. Write one sentence: "Given a new service message, suggest which of four staff queues should review it." Avoid broad goals such as "use AI to improve service."

2. Define the categories. Write clear labeling instructions. If two employees would label the same message differently, the categories or instructions need improvement before training.

3. Check the data. Use only messages the business is permitted to use. Remove unnecessary personal details. Check that the examples reflect the language and request types the system will actually receive. Do not assume that more data automatically means better data.

4. Establish a baseline. First measure how the current process works. A simple keyword rule or existing manual routing process can provide a comparison. Record where the baseline succeeds and fails.

5. Evaluate the model. Keep separate training, validation, and test examples. Review errors by category. A missed urgent service concern may matter more than a harmless misclassification between two routine queues.

6. Keep a person in control. Display the suggested category with enough context for an employee to check it. Provide an "uncertain" or "send to manual review" route. Keep permissions and record changes in the ordinary business application, not inside an uncontrolled model response.

7. Measure the result. Compare the pilot with the baseline using a measure that reflects the actual goal, such as correct routing, time spent reviewing messages, or the rate of messages sent to the wrong queue. Include the cost of maintaining the model and reviewing its output.

Where machine learning may fit

Other possible business uses include grouping similar product descriptions, estimating demand from historical patterns, identifying unusual records for investigation, or helping prioritize work. Each idea needs its own data review and evaluation plan.

Do not use a small classroom model or an unvalidated prediction to make financing, credit, eligibility, employment, safety, or other consequential decisions. Those uses require specialized expertise, legal and policy review, stronger governance, and appropriate human accountability.

The business lesson is simple: start with a narrow problem, compare against the current process, inspect the mistakes, and expand only if the evidence supports it.

Common misconceptions

Myth A machine-learning model is the same thing as the algorithm that trains it.: Reality
The algorithm is a procedure used during learning. The model is the trained result with learned parameters that can be used to produce outputs.
Myth A model learns from every prediction it makes.: Reality
Most deployed models do not automatically update their parameters after each prediction. Updating or retraining requires a specific design and process.
Myth A neural network is a digital human brain.: Reality
Artificial neural networks are mathematical models loosely inspired by some biological ideas. Their structure and operation are very different from a human brain.
Myth More layers always make a neural network better.: Reality
More layers can increase capacity and complexity, but can also increase data, computation, and tuning requirements. The architecture must fit the task.
Myth A high training score proves that a model works.: Reality
A model can overfit training examples. It must be evaluated on separate data and monitored under real operating conditions.
Myth If a model gives a confident prediction, it must be correct.: Reality
A model's output can be wrong or poorly calibrated. Confidence is not a substitute for evidence, evaluation, and appropriate review.

Advanced Advanced AI and Technical Development

Advanced companion

Path C: Technical companion

This companion turns the beginner explanation into a more precise technical picture. It introduces the notation used to describe a dataset, a model, its parameters, and the objective used during training. It then examines a perceptron, a multilayer network, gradient-based optimization, and evaluation.

The goal is conceptual understanding, not a full course in calculus or deep learning. You can follow the architecture and training loop without deriving every equation. The Python examples use only the standard library and small, synthetic datasets. They are designed to make the mechanics visible, not to demonstrate production performance.

Keep three questions in view: what information enters the model, what objective changes its parameters, and what evidence shows that the trained model generalizes beyond the examples it saw?

1. A compact mathematical view of learning

A supervised dataset can be represented as a collection of input and target pairs:

D = \{(x_i, y_i)\}_{i=1}^{N}

Here, $x_i$ is the input for example $i$ , $y_i$ is its target, and $N$ is the number of examples. The input may be a vector of numerical features, a representation of text, image data, or another structured representation.

A model is often written as $f_{\theta}$ , where $\theta$ represents its learned parameters. Its prediction for an input is:

\hat{y}_i = f_{\theta}(x_i)

The training process searches for parameter values that perform well according to an objective. A loss function measures the difference between predictions and targets for the chosen task. A simplified empirical objective is:

L(\theta) = \frac{1}{N}\sum_{i=1}^{N}\ell(f_{\theta}(x_i), y_i)

The exact loss depends on the task. Regression, binary classification, and multi-class classification commonly use different objective functions. Not every learning method is supervised, so this notation is a useful starting point rather than a description of all machine learning.

2. The perceptron: a weighted sum and a decision

A simple artificial neuron receives numerical inputs, multiplies each input by a weight, adds a bias, and applies an activation function.

z = \sum_{j=1}^{d} w_j x_j + b

a = \phi(z)

The input values are $x_j$ , the learned weights are $w_j$ , the bias is $b$ , and the activation function is $\phi$ . In a basic perceptron, the activation may be a threshold function that returns one of two classes.

A single perceptron creates a linear decision boundary. It can represent simple linearly separable patterns, such as the logical OR relationship between two binary inputs. It cannot represent every possible pattern. The XOR pattern is a classic example that cannot be separated correctly by one linear threshold unit.

The first code example trains a perceptron on the four possible input pairs for OR. The dataset is deliberately tiny and complete for that toy rule. It is not a model of real-world language, images, or business behavior.

3. Hidden layers and nonlinear representations

A multilayer neural network composes several transformations. A simplified layer can be written as:

h^{(l)} = \phi(W^{(l)}h^{(l-1)} + b^{(l)})

The vector $h^{(l-1)}$ is the previous layer's output, $W^{(l)}$ is a matrix of weights, $b^{(l)}$ is a bias vector, and $\phi$ is an activation function. The first layer receives the input representation. The final layer produces the model output.

Nonlinear activation functions allow multiple layers to represent nonlinear relationships. The hidden units create intermediate representations that are learned during training. The architecture, activation functions, data, and optimization process all affect what can be learned.

The XOR example uses two inputs, two hidden units, and one output unit. Its hidden layer makes it possible to represent a pattern that a single linear threshold unit cannot. The example uses sigmoid activations and hand-set initial parameters so that the code is deterministic and reproducible.

4. Loss, gradients, and backpropagation

Training changes parameters to reduce a loss objective. In gradient descent, a parameter update can be represented as:

\theta_{t+1} = \theta_t - \eta \nabla_{\theta} L(\theta_t)

The learning rate $\eta$ controls the size of an update. The gradient $\nabla_{\theta} L$ indicates how the loss changes as the parameters change. A learning rate that is too large can make optimization unstable; one that is too small can make progress slow.

Backpropagation applies the chain rule to calculate gradients through the network. It moves information about the output error backward through the layers so that the training procedure can estimate how each parameter contributed to the loss. An optimizer then applies updates.

The second code example uses a small online update loop. It processes one training example at a time, calculates the output error, propagates the error through the hidden layer, and updates the weights and biases. This is a deliberately small teaching implementation. Production systems use tested libraries and additional methods for numerical stability, batching, regularization, and performance.

5. Generalization, splits, and data leakage

A model's purpose is to perform on examples beyond the training set. The ability to do this is called generalization.

Training data is used to fit parameters. Validation data supports model selection and tuning. A final test set is held aside to estimate performance after those choices are made. The exact split strategy depends on the data. Time-dependent problems may require chronological splits. Data grouped by customer, device, location, or source may need group-aware splitting so closely related examples do not appear on both sides.

Data leakage occurs when information unavailable at prediction time influences training or evaluation. Leakage can come from future information, duplicated examples, target-derived features, or repeated tuning against the test set. It can make a weak model appear stronger than it is.

The toy code examples train and display predictions on their tiny training datasets. They do not establish generalization. A real evaluation needs appropriate held-out data, representative sampling, clear labels, and a documented process.

6. Classification metrics and error analysis

For classification, a confusion matrix compares predicted classes with actual classes. In a binary task, it distinguishes true positives, true negatives, false positives, and false negatives.

Accuracy is the fraction of all predictions that are correct. Precision is the fraction of predicted positive cases that are truly positive. Recall is the fraction of actual positive cases that the model identifies. These measures answer different questions.

Accuracy can be misleading when classes are imbalanced. Imagine a dataset in which one class is very common and another is rare. A model that always predicts the common class may achieve a high accuracy score while missing every rare case. The appropriate metric depends on the cost and meaning of each error.

For multi-class tasks, teams may examine per-class precision and recall, a confusion matrix, macro or weighted averages, and examples of misclassified inputs. A metric should be selected before reviewing results, based on the intended use. Do not select whichever metric makes the model look best after the fact.

7. From experiment to responsible system

A trained model is only one component of a deployed machine-learning system. A real application also needs input validation, data handling, versioning, access controls, monitoring, logging, fallback behavior, and a defined owner.

Record which data version, code version, model configuration, and evaluation results were used. Keep a baseline so the team can compare the model with a simpler method. Monitor both technical behavior and task outcomes. A change in input patterns may reduce performance even when the model itself has not changed.

The team should decide which outputs are suggestions, which actions require approval, and which decisions must remain outside the model's authority. For high-impact uses, obtain qualified legal, domain, privacy, security, and technical review before deployment.

A responsible machine-learning project is an evidence-based process. It does not end when training finishes. It continues through evaluation, deployment, monitoring, review, and decisions about whether the system should be changed or retired.

A single perceptron learns the OR pattern

# A single perceptron learning the OR rule
training_data = [
    ((0, 0), 0),
    ((0, 1), 1),
    ((1, 0), 1),
    ((1, 1), 1),
]

weights = [0.0, 0.0]
bias = 0.0
learning_rate = 0.2

for epoch in range(10):
    for inputs, target in training_data:
        weighted_sum = sum(w * x for w, x in zip(weights, inputs)) + bias
        prediction = 1 if weighted_sum > 0 else 0
        error = target - prediction

        for index, value in enumerate(inputs):
            weights[index] += learning_rate * error * value
        bias += learning_rate * error

for inputs, target in training_data:
    weighted_sum = sum(w * x for w, x in zip(weights, inputs)) + bias
    prediction = 1 if weighted_sum > 0 else 0
    print(f"{inputs} -> {prediction} (target: {target})")

This example uses only Python's standard library. The model has two input weights and one bias. For each labeled example, it calculates a weighted sum, applies a threshold, compares the prediction with the target, and adjusts its parameters. After training, it prints the predictions for all four OR examples.

The OR pattern is linearly separable, so one perceptron can represent it. This is not a general-purpose classifier. It has only four synthetic examples and a deliberately simple threshold rule. Try changing the targets to XOR and observe that a single perceptron cannot represent that pattern correctly.

A tiny neural network learns the XOR pattern

import math

# Four labeled examples of the XOR pattern
training_data = [
    ([0.0, 0.0], 0.0),
    ([0.0, 1.0], 1.0),
    ([1.0, 0.0], 1.0),
    ([1.0, 1.0], 0.0),
]

def sigmoid(value):
    return 1 / (1 + math.exp(-value))

# Two input values, two hidden neurons, and one output neuron.
input_to_hidden = [
    [0.15, -0.20],
    [0.25, 0.10],
]
hidden_bias = [0.05, -0.05]
hidden_to_output = [0.20, -0.15]
output_bias = 0.02
learning_rate = 0.8

for epoch in range(20000):
    for inputs, target in training_data:
        hidden = [
            sigmoid(
                sum(inputs[i] * input_to_hidden[i][j] for i in range(2))
                + hidden_bias[j]
            )
            for j in range(2)
        ]
        output = sigmoid(
            sum(hidden[j] * hidden_to_output[j] for j in range(2))
            + output_bias
        )

        # Backpropagate the output error through the network.
        output_delta = (target - output) * output * (1 - output)
        hidden_deltas = [
            hidden[j] * (1 - hidden[j]) * hidden_to_output[j] * output_delta
            for j in range(2)
        ]

        for j in range(2):
            hidden_to_output[j] += learning_rate * output_delta * hidden[j]
        output_bias += learning_rate * output_delta

        for i in range(2):
            for j in range(2):
                input_to_hidden[i][j] += (
                    learning_rate * hidden_deltas[j] * inputs[i]
                )
        for j in range(2):
            hidden_bias[j] += learning_rate * hidden_deltas[j]

# Show the learned output for each training example.
for inputs, target in training_data:
    hidden = [
        sigmoid(
            sum(inputs[i] * input_to_hidden[i][j] for i in range(2))
            + hidden_bias[j]
        )
        for j in range(2)
    ]
    probability = sigmoid(
        sum(hidden[j] * hidden_to_output[j] for j in range(2))
        + output_bias
    )
    prediction = 1 if probability >= 0.5 else 0
    print(
        f"{inputs} -> probability {probability:.3f}, "
        f"prediction {prediction}, target {int(target)}"
    )

This example uses Python's standard math library and no external packages. It creates a network with two input values, two hidden neurons, and one output neuron. The sigmoid activation maps each neuron's weighted input to a value between zero and one. A simple backpropagation loop updates the parameters over repeated passes through four synthetic XOR examples.

The final output shows the model's predictions on the same four examples used for training. This confirms that the small network can represent the XOR pattern with these settings. It does not measure generalization, prove that the model is reliable, or demonstrate how a production neural network should be built. The example is for understanding the mechanics of a hidden layer and parameter updates.

Experiment: Try a pattern a single perceptron cannot represent

Run the OR perceptron example and confirm that it matches the four targets. Then change the targets to XOR: 0 for (0, 0), 1 for (0, 1), 1 for (1, 0), and 0 for (1, 1). Keep the same single-neuron architecture and training loop. Record the predictions and explain why changing the learning rate or adding more epochs cannot make a single linear threshold represent a non-linearly separable pattern.

Experiment: Explore a neural network visually

Open the TensorFlow Playground resource. Choose a simple dataset and observe the initial decision boundary. Add or remove a hidden layer, change the number of neurons, and adjust the learning rate. Train again and note how the boundary changes. Keep the comparison controlled by changing one setting at a time. Explain why a more complex boundary is not automatically better and why evaluation on separate examples matters.

Experiment: Design an evaluation plan

Choose a fictional classification or regression task. Write down the input, target, available examples, and the moment when a prediction would be made. Define a baseline. Explain how you would split the data without leakage, which metric best reflects the cost of errors, what examples need human review, and how the system should behave when it is uncertain or unavailable. State what evidence would cause you not to deploy the model.

Resources

Google Machine Learning Crash Course
A structured course with explanations, visualizations, and exercises covering machine-learning fundamentals, data, neural networks, and evaluation.
Neural Networks module from Google Machine Learning Crash Course
A deeper technical module on nodes, hidden layers, activation functions, inference, and backpropagation.
TensorFlow Playground
An interactive visualization for experimenting with neural-network structure and training behavior. It is an open-source project and is not an official Google product.

Completion checklist

I can explain the difference between an algorithm, a model, training, and inference.
I can identify features, labels, examples, and predictions in a simple task.
I can describe supervised, unsupervised, reinforcement, and self-supervised learning at a high level.
I can describe the input, hidden, and output layers of a neural network.
I understand that weights, biases, and activation functions are part of a network's calculations.
I can explain why separate evaluation data is needed and what overfitting means.
I can identify a low-risk machine-learning use case and describe data, evaluation, human review, and a fallback.

Machine Learning and Neural Networks