Lesson 2: Traditional Software vs. Artificial Intelligence

Introduction

Two different ways to make computers useful

Think about two tasks.

First, imagine a program that calculates the total cost of an order. It receives a price, adds tax according to a defined rate, subtracts an approved discount, and displays the total. The developer has written the rules. If the inputs and rules are the same, the program should produce the same result.

Now imagine a system that reads a customer's message and estimates whether the person is asking about a repair, a payment, a product, or a delivery. People do not all use the same words. They may misspell terms, describe the same issue in different ways, or leave out important details. Writing a separate rule for every possible sentence would be difficult. A machine-learning system can instead learn patterns from examples and use those patterns to classify a new message.

Both systems are software. Both can be useful. But they solve problems in different ways.

Traditional software generally applies instructions that people explicitly define. Machine learning uses examples and a training process to build a model that can recognize patterns or make predictions. Generative AI uses models to create outputs such as text, images, audio, or code.

This lesson explains the distinction without treating one approach as universally better. The goal is not to replace every traditional program with AI. The goal is to understand what each approach does well, where it can fail, and how to combine them responsibly.

You will learn to recognize the difference in everyday tools, business workflows, and technical systems. By the end, you should be able to look at a task and ask a better question: does this need exact rules, pattern recognition, generated content, or a combination?

What you will learn

Explain how traditional software follows explicit instructions written by developers.
Describe how machine-learning systems use data to learn patterns and make predictions.
Compare rule-based automation, machine learning, and generative AI using practical examples.
Identify tasks that are better suited to traditional software, AI, or a combination of both.
Evaluate an AI use case by considering data, uncertainty, risk, cost, and human oversight.
Design a simple workflow that combines predictable software rules with AI assistance.

The lesson

1. Traditional software: instructions written in advance

Traditional software is often described as rule-based or explicitly programmed software. A developer writes instructions that describe what the program should do when it receives particular inputs.

Consider a simple invoice calculator. It might:

Read the quantity of each item.
Multiply each quantity by its unit price.
Add the line totals.
Apply a defined tax rule.
Display the amount due.

The program does not need to learn what multiplication means from thousands of invoices. The calculation is specified directly. If the inputs and program rules are unchanged, the calculation should be repeatable.

This approach is excellent when the rules are known, the inputs can be represented clearly, and the desired result can be defined precisely.

Examples include:

Adding numbers and calculating totals.
Applying a fixed discount when a valid code is entered.
Checking whether a required field is blank.
Sending a reminder a specified number of days before an appointment.
Giving a user access when their account has the required permission.
Moving a record to a new status when an approved condition is met.

Traditional software can be sophisticated. It can contain databases, interfaces, workflows, search tools, and large networks of connected services. The important distinction is not whether the program is simple or complicated. It is whether its behavior is primarily specified through instructions and rules rather than learned from training data.

A rule can be wrong, incomplete, or outdated. Traditional software does not automatically know that a business policy has changed. Someone must update the rule or the data it uses.

A useful mental model: traditional software is like a recipe. A person defines the steps, and the computer follows them. The computer may execute those steps very quickly, but it is not independently discovering the recipe.

2. Machine learning: patterns learned from examples

Machine learning is a branch of artificial intelligence in which a system uses data and an algorithmic training process to build a model. Instead of a developer writing every decision rule by hand, the training process adjusts the model so that it can perform a task based on patterns in examples.

Imagine a fictional business that receives thousands of customer messages. The business wants to sort messages into categories such as billing, scheduling, product questions, and service requests.

A traditional approach might search for specific words. A message containing the word "invoice" could be sent to billing. This can work for obvious cases, but it can miss a message such as "I was charged twice" or incorrectly classify a message that says "I need an invoice for the repair appointment."

A machine-learning approach could be trained using a collection of messages that people have labeled with the correct category. During training, the model learns patterns that help distinguish the categories. When it receives a new message, it estimates which category is most likely.

The model is not memorizing a perfect rule for every possible sentence. It is learning statistical relationships from the examples it was given. This can help it handle wording it has not seen before, but it does not guarantee that every new message will be classified correctly.

A simplified machine-learning lifecycle:

Define the task, such as classifying incoming messages.
Gather examples that are relevant and permitted for the task.
Prepare and review the data.
Train a model using a suitable learning method.
Evaluate it on examples that were not used to fit the model.
Deploy it in a workflow.
Monitor its performance and respond when the data, task, or environment changes.

Machine learning is useful when a task involves patterns that are difficult to express as a manageable list of hand-written rules. Examples include identifying unusual transactions, recognizing objects in images, estimating demand, or ranking likely matches.

The quality of the result depends on more than the algorithm. Data quality, representative examples, evaluation, deployment conditions, and ongoing monitoring all matter.

A useful mental model: instead of writing every answer into a recipe, you show a learner many examples, provide feedback during training, and evaluate whether the learner can handle new examples.

3. Rules, machine learning, and generative AI are not the same thing

People sometimes use the word "AI" to describe any software that seems clever. It is more useful to distinguish several approaches.

Rule-based automation follows conditions and actions that people define. For example: if an appointment is tomorrow and the customer has not confirmed, send a reminder. This is predictable when the rule and data are correct.

Predictive machine learning estimates an outcome from patterns in data. For example: estimate which incoming message category is most likely, or estimate demand for a product. The result is a prediction, not a guarantee.

Generative AI produces new content in response to an input. A language model might draft a reply, summarize a document, explain a concept, or suggest a checklist. An image model might create an illustration from a description. Generated output can be useful, but it can also contain errors or unsupported details.

These categories can overlap. A generative AI product may use machine-learning models. A business application may combine an AI model with ordinary software rules, databases, permissions, and user interfaces.

Do not assume that a feature is AI simply because it has a modern interface or uses the word "smart." Ask what the system actually does. Does it follow a fixed rule? Does it estimate a pattern? Does it generate content? Does it combine several methods?

Example: a customer appointment workflow

A traditional rule checks the appointment date and sends a reminder at a specified time.
A machine-learning model might classify a customer's free-text response as likely confirmation, cancellation, or question.
A generative AI model might draft a polite response using the customer's message and approved appointment information.
Traditional software should still verify the appointment record, enforce permissions, and require approval before changing sensitive records if that is the chosen policy.

The strongest design may use all of these methods, each for the part of the task it handles best.

4. A side-by-side comparison

The differences become easier to understand when we compare the approaches across the same dimensions.

Question	Traditional software	Machine learning	Generative AI
Where does behavior come from?	Instructions and rules written by people	Patterns adjusted during training on data	A generative model trained to produce content
What does it do?	Executes defined operations	Predicts, classifies, ranks, or detects patterns	Generates text, images, audio, code, or other content
Is the output predictable?	Often highly repeatable for the same inputs and state	Usually probabilistic or score-based	Often variable, even for similar prompts
What does it need?	Defined logic and input data	Relevant data, training, and evaluation	A model, input context, and often a prompt
How can it fail?	Incorrect rules, missing cases, bad inputs, software defects	Biased or unrepresentative data, drift, poor generalization	Incorrect, incomplete, misleading, or fabricated content
How is it improved?	Change code, configuration, or data	Improve data, features, model, training, or deployment	Improve the model, context, instructions, tools, or review process

These are broad distinctions, not absolute laws. Some traditional systems contain statistical methods. Some machine-learning systems are embedded inside larger rule-based applications. Generative systems also rely on ordinary software to receive requests, retrieve information, enforce access, and display results.

The most important lesson is that different methods create different strengths and risks. A calculator should not improvise an answer. A system that interprets varied human language may benefit from machine learning. A tool that drafts a first version of a customer email may benefit from generative AI, provided a person reviews it when accuracy or customer impact matters.

There is no prize for using the most advanced technology. The right choice is the one that solves the actual problem reliably and responsibly.

5. When traditional software is the better choice

Traditional software is often the better choice when a task has clear rules and the result must be exact, repeatable, and auditable.

Use defined rules for calculations. If a finance workflow needs to total a contract, calculate a payment using approved inputs, or compare a number against a threshold, use tested calculation logic. A language model may explain the calculation, but it should not be the source of truth for arithmetic that must be exact.

Use rules for permissions and approvals. A system should not let an AI-generated suggestion bypass account permissions, approval limits, or required signatures. Access control should be enforced by the application.

Use rules for required business processes. If a record must not move to "completed" until required fields are present, enforce that condition in the software.

Use rules when the process is stable and well understood. A reminder sent three days before a scheduled event does not need a generative model to decide when three days have passed.

Traditional software also offers advantages in repeatability and testability. Developers can create tests for known inputs and expected outputs. When the requirements are precise, that can make defects easier to isolate.

However, rules can become difficult to maintain when exceptions multiply. A system that tries to handle every possible variation in natural language through manually written conditions may become fragile. That is one situation where a learned model may help.

A practical rule: if you can describe the task as a clear set of conditions and exact actions, start by asking whether ordinary software is sufficient. Add AI only when it provides a meaningful improvement.

6. When AI is useful, and why it still needs oversight

AI can be valuable when the input is varied, ambiguous, unstructured, or too complex to handle with a short list of rules.

Examples include:

Classifying messages that use many different phrases.
Finding patterns in a large collection of records.
Identifying objects or text in images.
Summarizing long documents.
Drafting a first version of a response.
Generating ideas, outlines, or alternative explanations.
Helping a person explore a dataset or write code.

But AI introduces uncertainty. A model's output should be treated according to the consequences of being wrong.

For a low-risk task, such as generating ten possible names for a fictional project, a person may only need to review the suggestions. For a high-impact task, such as deciding whether a customer qualifies for a financial product, an AI prediction should not silently make the decision. The organization needs appropriate rules, qualified human judgment, transparency, and controls.

Three questions to ask before trusting an AI result:

What evidence or source information supports this output?
What could happen if the output is wrong?
Who is responsible for checking it and deciding what action to take?

AI can assist with work without becoming the final authority. For example, it can summarize a customer message, suggest a category, and draft a response. A person can confirm the facts, select the correct next step, and approve the message.

The right amount of oversight depends on the task, the data, the model, and the potential impact. Do not use "AI is usually good at this" as a substitute for testing it in the actual workflow.

7. Combining software and AI into one workflow

Many useful applications combine traditional software and AI instead of choosing only one.

Consider a fictional company that receives service requests through a website.

Step 1: Traditional software receives the request. It validates required fields, records the submission, assigns a reference number, and stores the information.

Step 2: AI helps interpret the message. A model suggests whether the request concerns scheduling, a product question, billing, or technical support. It may also summarize the customer's description.

Step 3: Traditional software applies business rules. It checks which team handles that category, verifies required permissions, and routes the record according to approved rules.

Step 4: Generative AI drafts a response. It uses approved information to prepare a suggested reply. It should not invent appointment availability, prices, policies, or commitments.

Step 5: A person reviews the result. The employee checks the summary, category, and draft, corrects errors, and approves the next action when required.

Step 6: Traditional software records the outcome. It stores the approved response, updates the status, and keeps an audit trail.

This architecture separates responsibilities. AI handles interpretation and drafting where flexibility is useful. Ordinary software handles exact records, permissions, and repeatable actions. People handle judgment and accountability.

This pattern is sometimes called a human-in-the-loop workflow when a person reviews or approves an AI-assisted step. It is not necessary for every low-risk operation, but it is an important design option when mistakes could affect customers, money, privacy, safety, or legal obligations.

A well-designed workflow also includes a fallback. If the model is uncertain, unavailable, or returns an unusable result, the request can be sent to a person or handled through a simpler process.

8. A practical framework for choosing the right approach

Before adding AI to a process, describe the task clearly. Then work through these questions.

1. Are the rules already known? If the task can be expressed as exact conditions and actions, traditional software may be sufficient.

2. Does the task require recognizing patterns? If it involves many examples, changing patterns, or inputs that are difficult to describe with fixed rules, machine learning may be worth evaluating.

3. Does the task require creating a draft or explanation? If the output is text, images, code, or another form of content, generative AI may help.

4. How costly is a mistake? The greater the potential impact, the stronger the testing, review, and approval controls should be.

5. What data is required and may it be used? Check privacy, permissions, data quality, retention, and whether sensitive information may be sent to the chosen system.

6. How will success be measured? Define a baseline and a measurable result. Examples might include time saved, fewer routing errors, faster response preparation, or improved consistency. Do not assume that adding AI automatically improves performance.

7. What happens when the system cannot help? Create a fallback to a person or a reliable non-AI process.

A useful decision is not "AI or no AI?" It is "which part of this workflow benefits from which method, and how will we know it works?"

For your first project, choose one small task. Test it with realistic examples, compare it with the current process, record errors as well as successes, and expand only when the evidence supports doing so.

9. What to remember

Traditional software follows instructions and rules that people define. Machine learning uses data and a training process to learn patterns that support predictions or classifications. Generative AI creates content based on patterns learned during training and the context it receives.

These approaches are not enemies. They are tools with different strengths.

Use traditional software for exact calculations, permissions, required fields, and predictable workflows.
Consider machine learning for classification, prediction, ranking, and pattern recognition.
Consider generative AI for drafting, summarizing, explaining, brainstorming, and creating content.
Combine methods when a workflow needs both flexibility and reliable controls.
Keep people responsible for decisions that require judgment, accountability, or careful review.

The central skill is not learning to call everything AI. It is learning to identify the real problem and select the simplest reliable method that solves it.

In the next lesson, we will look more closely at machine learning and neural networks, including how systems learn from data and why training and evaluation matter.

Try it

Choose the method that fits the task

Read each task and compare the role of rules, machine learning, and generative AI. More than one method can appear in a complete workflow.

Task	Best starting approach	Why
Calculate an invoice total	Traditional software	The arithmetic and business rules should be explicit and testable.
Classify varied customer messages	Machine learning may help	Examples can teach a model patterns beyond a short keyword list.
Draft a first response to a customer	Generative AI may help	It can produce flexible language for a person to review.
Enforce user permissions	Traditional software	Access decisions need explicit, dependable controls.
Summarize a long document	Generative AI may help	The task involves interpreting and producing natural language.

Ask AI to help evaluate a workflow, not make unsupported promises

Compare the vague request with the stronger request. Notice how the stronger version asks for a decision framework and makes limitations visible.

Weaker prompt

Should my business use AI?

Stronger prompt

I run a fictional local service business with five employees. We receive customer requests by email. Compare three approaches for sorting requests: fixed keyword rules, a machine-learning classifier, and a generative AI assistant. For each approach, explain the setup, likely strengths, failure modes, privacy considerations, human review needs, and a small pilot test. Do not invent performance statistics. Recommend the simplest approach to test first and explain what evidence would change your recommendation.

The stronger prompt gives context, defines the comparison, asks for risks and evaluation, and explicitly prevents invented performance claims.

Inside a hybrid customer-request workflow

Step through the workflow and identify which responsibilities belong to ordinary software, AI, and people.

Step 1 of 5
Receive and record
Traditional software validates required fields, creates a reference number, and stores the request.
Step 2 of 5
Interpret the message
A machine-learning model may suggest a category. Generative AI may summarize the request. These are suggestions that can be wrong.
Step 3 of 5
Apply business rules
Traditional software checks permissions, routing rules, and required conditions.
Step 4 of 5
Draft and review
Generative AI can prepare a response from approved information. A person checks the facts and approves it when required.
Step 5 of 5
Record the result
Traditional software saves the approved response, status, and audit information.

Which approach is doing the work?

Try to answer each item before revealing the explanation.

A program sends a reminder exactly 24 hours before an appointment.

Primarily traditional software. It follows a defined timing rule.

A model estimates whether a transaction is unusual based on patterns in previous transactions.

Machine learning. It uses learned patterns to estimate or flag an outcome.

A tool drafts a friendly explanation of a complicated policy.

Generative AI. It creates natural-language content for review.

An application checks that an employee has permission before opening a record.

Traditional software should enforce the permission rule, even if AI helped interpret an earlier request.

Evaluate one task in your own work

Help me evaluate this task: [describe the task]. First, explain whether it is mainly a rules problem, a pattern-recognition problem, a content-generation problem, or a combination. Compare a traditional software approach, a machine-learning approach, and a generative AI approach where relevant. Identify required data, likely failure modes, privacy concerns, human review, and a small low-risk pilot. Do not invent statistics or claim that AI will improve results without evidence.

Hands-on exercises

Beginner AI Beginners

Beginner: sort everyday tasks

Create a list of ten tasks you encounter at home, school, or work. Examples include calculating a bill, organizing photos, setting a reminder, summarizing a letter, or recognizing a voice command.

For each task, write:

What information goes into the task?
What result should come out?
Are the rules exact and known, or does the task require interpreting patterns or creating content?
Would traditional software, machine learning, generative AI, or a combination be a reasonable starting point?
What could go wrong?

Finish by choosing one task where ordinary software is enough and one where AI might provide useful assistance. Explain the difference in two or three sentences.

What a good result looks like: A list of ten tasks with a reasoned approach for each. The learner should recognize that not every computer task needs AI.

Business Business Owners and Professionals

Business: map one workflow

Use a fictional business or a process you are authorized to discuss. Choose one workflow, such as handling incoming inquiries, scheduling appointments, preparing a quote, routing service requests, or following up after a purchase.

Map the current steps from beginning to end. Label each step:

RULE: exact conditions, calculations, permissions, or record updates.
PATTERN: interpreting varied messages, identifying categories, or estimating likely outcomes.
GENERATION: drafting text, summaries, ideas, or other content.
HUMAN: judgment, approval, customer conversation, or accountability.

Then design a small hybrid workflow. Keep exact calculations and permissions in ordinary software. Add AI only where it offers a plausible benefit. Define a human review point and a fallback if AI is unavailable or uncertain.

Use fictional customer names and sample data. Do not include confidential customer information.

What a good result looks like: A workflow map that separates rules, pattern recognition, content generation, and human judgment, with a small pilot and a fallback.

Advanced Advanced AI and Technical Development

Advanced: build and test a rule-based baseline

Use the code example in the advanced companion as a starting point. Run it in a local Python environment if available, or inspect it line by line.

Add at least five new sample messages to the labeled dataset.
Add one new category and at least two examples for it.
Test the keyword rules against messages that use different wording.
Record false positives and false negatives.
Explain why this small rule-based demonstration is not a trained machine-learning model.
Propose a fair evaluation plan for comparing the rules with a learned classifier. Include a held-out test set and a suitable metric for the task.

Do not describe the toy rules as a production-ready classifier. The purpose is to understand the baseline and the evaluation problem.

What a good result looks like: A working or carefully inspected rules baseline, documented error cases, and a technically sound plan for a future model comparison.

Business

Path B: Apply the distinction to a business

Start with the business problem, not the AI tool

A business owner may hear that AI can automate customer service, marketing, scheduling, reporting, and sales. Those are broad claims. A better starting point is to identify one repeated task and understand what makes it difficult.

Imagine a fictional company called Harborlight Home Services. It receives customer requests through a website and email.

Some parts of its workflow are straightforward:

Store the customer's contact details.
Check that required fields are present.
Create a service-request number.
Apply approved scheduling and routing rules.
Record who changed the request and when.

These are good candidates for ordinary software because the business can define the rules.

Other parts are less predictable:

A customer may describe the same repair in several different ways.
A message may contain multiple questions.
A customer may be frustrated and use emotional or unclear language.
A long description may need a short summary for the technician.

Machine learning may help classify or detect patterns in varied messages. Generative AI may help summarize the request or draft a response. Neither should be allowed to invent a price, promise a time slot, change a contract, or bypass a company policy.

A practical opportunity assessment

Before buying a tool or building an integration, write down:

Current process: What does the employee do today?
Volume: How often does the task occur? Use your own measured data.
Time: How much employee time does it take?
Quality problem: What errors, delays, or inconsistencies occur?
Candidate method: Could rules, machine learning, generative AI, or a combination help?
Risk: What happens if the system is wrong?
Control: Who checks the result, and what actions are blocked until approval?
Pilot: How will you test the idea on a limited set of cases?
Success measure: What evidence would justify keeping or expanding it?

Where a small business might start

Rules and automation: reminders, task creation, required-field checks, status updates, and approved routing.
AI-assisted interpretation: categorizing incoming requests or summarizing long messages.
Generative AI: drafting email responses, preparing first-pass marketing copy, or turning meeting notes into action lists.
Human judgment: final pricing, sensitive customer decisions, exceptions, commitments, and approvals.

A practical business often benefits from a hybrid approach. Use AI to reduce the effort involved in interpreting or drafting. Use tested software logic to protect records and enforce policies. Keep employees involved where a customer or business decision matters.

Measure before you scale

Run a small pilot with non-sensitive or properly authorized information. Compare the assisted workflow with the current process. Track completion time, correction rate, missed cases, employee feedback, and customer impact where appropriate. Include the cost of the tool and the time needed to review its output.

If the pilot does not improve the process, change the design or stop. A tool is not valuable merely because it uses AI.

Common misconceptions

Myth Traditional software is not intelligent, so it is always outdated.: Reality
Traditional software is often the safest and most effective choice for exact calculations, permissions, and predictable operations. Reliability matters more than whether a task is branded as AI.
Myth Machine learning means the computer understands a task exactly as a person does.: Reality
A model learns statistical patterns from data. Its behavior depends on training, data, task design, and evaluation. Pattern recognition is not proof of human-like understanding.
Myth Generative AI can replace every rule-based system.: Reality
Generative output can vary and can be wrong. Exact calculations, access control, and required business rules should remain explicit and testable.
Myth If an AI result sounds confident, it is probably correct.: Reality
Fluent wording is not evidence. Check important claims against reliable source information and use stronger review for higher-impact tasks.
Myth A business must choose either AI or traditional software.: Reality
Many useful applications combine ordinary software, AI models, and human review. Each part should have a clear responsibility.

Advanced Advanced AI and Technical Development

Advanced companion

This companion examines the computational distinction between deterministic programs and data-fitted models. The goal is to establish precise language before the next lesson on machine learning and neural networks.

A traditional program implements a specified mapping from inputs and state to outputs. A machine-learning workflow selects a model family, fits parameters using training data and an objective, then evaluates the resulting model on data not used for fitting. A generative model learns a distribution or conditional distribution from which it can produce samples or sequences.

These categories are not mutually exclusive at the application level. A deployed AI product is typically a software system containing a model plus data pipelines, interfaces, storage, policies, monitoring, and control logic.

The code below is intentionally a small rule-based baseline. It is not a machine-learning model. Its purpose is to make the limits of hand-written keyword rules visible and to create a baseline that a later classifier could be compared against.

1. Programs as specified functions

A conventional program can be modeled as a function or state transition system. Given an input $x$ , configuration $\theta$ , and current state $s$ , the program computes an output and possibly a new state:

(y, s') = P(x, \theta, s)

The implementation may contain branches, loops, database queries, and calls to other services. Its behavior is still specified by the code and configuration. For fixed inputs and state, a deterministic component should produce a repeatable result.

In real applications, nondeterminism can enter through clocks, network responses, concurrency, randomization, or external systems. The conceptual distinction remains: the behavior is implemented through explicit program logic rather than inferred by fitting a predictive model to examples.

This is why software tests can assert exact outputs for carefully selected inputs. A tax calculation, permission check, or required-field validation should generally have explicit expected behavior.

2. Learning as parameter estimation

A supervised learning problem begins with examples $(x_i, y_i)$ , where $x_i$ is an input and $y_i$ is a target label or value. A model family $f_\theta$ is selected, and training adjusts parameters $\theta$ to reduce an objective:

\theta^* = \operatorname*{arg\,min}_{\theta} \frac{1}{N}\sum_{i=1}^{N} L(f_\theta(x_i), y_i)

This expression is a simplified view of empirical risk minimization. Actual training can include regularization, optimization details, data weighting, constraints, and other design choices.

The developer still makes many explicit decisions: task definition, data collection, model architecture, objective, training procedure, thresholds, and deployment controls. The difference is that the model's predictive behavior is not entirely written as a list of hand-authored rules. Some of its behavior is represented by learned parameters.

A trained model does not automatically generalize to every new input. Generalization must be measured, and the evaluation data must be separated appropriately from training data.

3. Deterministic logic and probabilistic outputs

A rule such as if amount > limit: flag has a clear condition. If the input amount and limit are unchanged, the condition evaluates the same way.

A classifier may instead produce scores for possible labels. For example, a message classifier could output scores for billing, scheduling, and service. The application might select the highest-scoring label, apply a confidence threshold, or route low-confidence cases to a person.

A score is not automatically a calibrated probability. Even when a model exposes probabilities, calibration should be evaluated for the intended population and use case. A system should not treat an arbitrary confidence score as a guarantee.

Generative models add another distinction: they produce sequences or other content. Their outputs may vary with sampling settings and context. Reproducibility can depend on model version, parameters, prompt, and serving environment.

Application logic should define how to handle these outputs. For example, it can reject malformed results, check required fields, verify cited records, and require approval before executing consequential actions.

4. Data, representation, and generalization

A rule-based system can be written without a training dataset, although it still depends on valid inputs and representative testing. A learned system depends on data that meaningfully represents the task and the conditions in which it will be used.

For supervised learning, labels must be defined consistently. If two reviewers label the same message differently, the target itself may be ambiguous. Class imbalance, data leakage, duplicates, sampling bias, and changing language can distort evaluation.

A held-out test set estimates performance on examples that were not used to fit the model. It must be constructed to reflect the intended deployment setting. Random splitting is not always appropriate; time-based or group-based splitting may be needed to avoid leakage.

Generalization means performing usefully on new examples, not merely reproducing training examples. A model can fit its training data well and still fail on new data. This is one reason to establish a transparent baseline and evaluate alternatives on the same test cases.

5. Evaluation depends on the task

There is no single metric that is best for every machine-learning task.

For classification, accuracy is the fraction of examples classified correctly. It can be misleading when classes are imbalanced. Precision measures how many predicted positives are correct; recall measures how many actual positives are found. A confusion matrix helps show which categories are being confused.

For a customer-message classifier, the cost of misrouting a safety-related request may differ from the cost of misrouting a general question. The evaluation should reflect the real consequences, not only an attractive aggregate score.

For generative systems, evaluation may include factuality, completeness, relevance, instruction following, privacy, and human judgments. Some properties can be checked automatically; others require careful review. A few impressive examples do not establish system-wide reliability.

A production evaluation should include representative cases, edge cases, failure analysis, and monitoring after deployment.

6. Hybrid architecture and control boundaries

A robust application can treat a model as one component in a larger system:

Input validation and authentication happen before model use.
Retrieval supplies approved context when needed.
The model returns a prediction or generated result.
Output validation checks structure and allowed values.
Business rules enforce permissions and policy.
Human approval is required for designated actions.
Logs and monitoring support investigation and improvement.

This architecture avoids giving a model unrestricted authority over records or external actions. A model can propose a category or draft, while ordinary code verifies whether the action is permitted.

The boundary is especially important when AI interacts with tools. A model's suggestion to issue a refund, alter a contract, disclose a record, or make a commitment should not itself be treated as authorization. The application must enforce the relevant policy.

7. A fair comparison experiment

To compare a rule-based baseline with a learned classifier, define the task and evaluation set before inspecting results.

Create a labeled dataset using a written labeling guide.
Separate training, validation, and test data in a way that avoids leakage.
Implement a simple rules baseline.
Train a candidate model using only the training data and permitted tuning data.
Evaluate both approaches on the same held-out test set.
Report per-class precision and recall, confusion matrices, and error examples.
Compare operational factors such as latency, maintenance, explainability, data requirements, and review workload.
Decide whether any measured improvement justifies the added complexity and risk.

Do not choose a model solely because it is more sophisticated. A simple rules system can be preferable when it meets the requirements and is easier to inspect and maintain.

A transparent keyword-rule baseline

# Educational example only. This is a hand-written rules baseline,
# not a trained machine-learning model.

RULES = {
    "billing": ["invoice", "charged", "payment", "refund"],
    "scheduling": ["appointment", "reschedule", "available", "booking"],
    "service": ["repair", "broken", "not working", "technician"],
}

def classify_message(text):
    normalized = text.lower()
    scores = {
        label: sum(term in normalized for term in terms)
        for label, terms in RULES.items()
    }

    best_label = max(scores, key=scores.get)
    if scores[best_label] == 0:
        return "needs-human-review", scores

    tied = [label for label, score in scores.items()
            if score == scores[best_label]]
    if len(tied) > 1:
        return "needs-human-review", scores

    return best_label, scores


examples = [
    "I was charged twice for the same invoice.",
    "Can I reschedule my appointment?",
    "The kitchen faucet is broken.",
    "I need help with the thing we discussed."
]

for message in examples:
    label, scores = classify_message(message)
    print(f"{label:20} | {message} | {scores}")

This example makes the rules visible. It can miss synonyms and context, and it can match a word in the wrong context. The human-review outcome is deliberate. Do not present it as a trained classifier or a production-ready system.

A minimal train/test split concept

# Conceptual example using scikit-learn.
# This shows the shape of a supervised text-classification experiment.
# Install scikit-learn separately in a suitable local environment.

from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.metrics import classification_report

messages = [
    "Please send me the invoice",
    "I was charged twice",
    "Can you explain this fee on my bill?",
    "I need a copy of my receipt",
    "Can I move my appointment?",
    "What times are available?",
    "I need to cancel my visit on Friday",
    "Is there an opening next week?",
    "The appliance is broken",
    "I need a technician for a repair",
    "The unit is making a loud noise",
    "My heater stopped working",
]
labels = [
    "billing", "billing", "billing", "billing",
    "scheduling", "scheduling", "scheduling", "scheduling",
    "service", "service", "service", "service",
]

# 12 examples, 4 per category. A 25% test split holds out 3 messages,
# one per category, which stratification needs (at least one per class).
X_train, X_test, y_train, y_test = train_test_split(
    messages, labels, test_size=0.25, random_state=7,
    stratify=labels
)

model = make_pipeline(
    CountVectorizer(),
    LogisticRegression(max_iter=1000)
)
model.fit(X_train, y_train)

predictions = model.predict(X_test)
print(classification_report(y_test, predictions, zero_division=0))

This is a small illustration of a supervised text-classification pipeline, not a meaningful production benchmark. Twelve examples are far too few to establish reliability. A real experiment needs a larger, representative labeled dataset, careful split design, and analysis of per-class errors.

Experiment: Find the limits of keyword rules

Run the keyword baseline. Add messages that express the same intent without using the listed keywords, such as "money came out of my account twice" or "could we move our visit to Friday?" Record which cases the rules miss and which they classify incorrectly. Explain why adding more keywords may help some cases but can also create conflicts and maintenance burden.

Experiment: Design a model evaluation plan

Write a one-page plan for comparing a rules baseline with a learned message classifier. Define the categories, labeling guide, train/validation/test split, metrics, review threshold, and fallback. Explain what error is most costly for the fictional business and how you would measure it. Do not claim that a model is better until the experiment produces evidence.

Experiment: Draw the hybrid system boundary

Draw a system diagram with an input form, validation layer, model service, output validation, business rules, human approval, and record storage. Mark which component is allowed to make a suggestion and which component is authorized to execute a change. Include an error path for model timeout, invalid output, and low confidence.

Resources

NIST AI Risk Management Framework
A framework for considering and managing risks associated with AI systems.
Google Machine Learning Crash Course
A structured technical learning resource for machine-learning concepts. The next lesson will explore learning methods in more detail.

Completion checklist

I can explain how traditional software uses explicit instructions and rules.
I can explain how machine learning uses data to learn patterns.
I can distinguish predictive machine learning from generative AI.
I can identify tasks where traditional software is the better starting point.
I can describe a workflow that combines software rules, AI assistance, and human review.
I completed the exercise for my learning path.
I passed the knowledge quiz.

Traditional Software vs. Artificial Intelligence