AI introspection
Lab setup
First, make sure you have completed the initial setup.
If you are part of a course
-
Open Terminal. Run the update command to make sure you have the latest code.
$ mwc update -
Move to this lab's directory.
$ cd ~/Desktop/making_with_code/pilot/labs/lab_introspection
If you are working on your own
-
Move to your MWC directory.
$ cd ~/Desktop/making_with_code -
Get a copy of this lab's materials.
git clone https://git.makingwithcode.org/founders-studio/lab_introspection.git
With an AI agent, getting working code is often the easy part. Understanding it is harder, and understanding is what lets you judge whether the code is right, explain it to a client, or build on it later. In this lab, you will solve an algorithmic problem with an agent's help, and then use the agent as a tutor until you understand the solution well enough to explain it four different ways.
By the end of this lab, you will be able to:
- Analyze a network of air travel to measure how important each city is over time.
- Explain the PageRank algorithm as an iterative algorithm, as a random walk, as a stationary distribution, and in terms of linear algebra.
- Use an AI agent to learn a hard idea, rather than to skip it.
- Decide when to have an AI write code, and when to write it yourself.
The question
The data folder contains flights.csv, which records how many passengers flew between
pairs of U.S. cities each year from 1990 to 2025, from the U.S. Bureau of Transportation
Statistics. (data/README.md explains where the data comes from.)
๐ป Start Python and take a look.
>>> import pandas as pd
>>> flights = pd.read_csv("data/flights.csv")
>>> flights.head()
year origin destination passengers
0 1990 Abilene, TX Dallas/Fort Worth, TX 1533
1 1990 Abilene, TX Houston, TX 92
...
Here is our question: how has the popularity of cities changed over time?
๐ป Before writing any code, discuss with your group: what could "popular"
mean for a city in an air travel network? Write down at least three different ideas in
questions.md.
Deciding what to delegate
Before you start, decide which parts of the work to hand to an agent and which to do yourself. A useful rule: delegate code when you can easily check the result, and write it yourself when writing it is how you learn.
๐ Open plan.md, which lists the parts of this task:
- Loading and cleaning the data.
- Building a network for each year.
- Computing popularity scores.
- Plotting scores over time.
- Checking that the scores mean what we think they mean.
๐ป For each part, decide with your group whether to delegate it, write it
yourself, or both, and note why in plan.md.
Solving it
๐ป Start Claude Code. Explain the question and ask for help computing a popularity score for each city in 1990. Ask the agent to suggest an approach and explain it before it writes any code.
There's a good chance the agent will suggest PageRank, the algorithm Google originally used to rank web pages. PageRank treats a network as a set of nodes (here, cities) connected by edges (routes, weighted by passengers), and gives each node a score based on how much "flow" ends up there.
๐ป Have the agent write popularity.py, which builds the network for a
given year and computes PageRank using the networkx library. Run it.
$ python popularity.py 1990
Chicago, IL 0.0535
Atlanta, GA 0.0403
Dallas/Fort Worth, TX 0.0400
...
๐ป Have the agent extend the program to compute scores for every year, and
write plot.py to plot how the scores of the top cities change over time.
๐ Look at your plot. Which cities rise and fall? Can you connect any of the changes to real events (for example, airlines opening or closing hubs, or a sudden drop in travel)?
Four ways to understand PageRank
You now have working code. But can you explain what PageRank is? This is where the real work of the lab starts. Use the agent as a tutor: ask it questions, ask for small examples, ask it to explain again more simply, and ask it to check your explanations. Don't move on until each explanation makes sense to you.
For each explanation below, write a paragraph in explanations.md in your own words, and
test it with an experiment.
As an algorithm
PageRank starts by giving every city the same score. Then, over and over, each city passes its score along its outgoing routes, in proportion to the passengers on each route. (A small fraction of every city's score is also shared equally with all cities; more on that below.) Eventually the scores stop changing.
๐ป Write your own version of this in my_pagerank.py, without the agent
writing it for you. Start from a tiny network of three or four cities, which you make up and
can check by hand. When it works, run it on the real data and compare with networkx.
As a random walk
Imagine a traveler who never stops flying. At each airport, they pick their next flight at random, in proportion to the passengers on each route out. Every so often (with probability 0.15), they get bored and teleport to a random city instead. A city's PageRank is the share of time the traveler spends there.
๐ป Have the agent write random_walk.py, which simulates this traveler for
a million flights. Read the code until you understand it. Compare its results with your
PageRank scores.
As a stationary distribution
Now imagine not one traveler but millions, spread across all the cities. After everyone takes one flight, the travelers are spread out differently. Keep going, and eventually the spread stops changing. One flight later, the same share of travelers is in each city as before. This unchanging spread is called a stationary distribution.
๐ป Explain in explanations.md why the stationary distribution is the same
as the result of the iterative algorithm, and the same as the share of time spent by the
random traveler.
As linear algebra
We can write the chance of flying from each city to each other city in a table called a transition matrix. Multiplying the travelers' spread by this matrix moves everyone one flight forward. A stationary distribution is one that comes out unchanged after the multiplication. In linear algebra, a vector that a matrix leaves unchanged (except perhaps for scaling) is called an eigenvector; when it is left exactly unchanged, its eigenvalue is 1. PageRank is the eigenvector of the transition matrix with eigenvalue 1.
๐ป Have the agent show you how to build the transition matrix for your
small example network (including the 0.15 chance of teleporting to a random city) and
compute its eigenvectors with numpy.linalg.eig. Find the one with
eigenvalue 1, scale it so it adds up to 1, and compare it with your PageRank scores.
Interpreting the results
๐ป Discuss, and answer in questions.md: what does PageRank actually
measure for an airline network? What would make it misleading? For example, what happens to a
city that's a popular connection point but where few people actually want to go?
Deliverables
๐ป Also answer these questions in questions.md:
- Which of the four explanations helped you most, and why?
- Describe one moment when the agent's explanation didn't make sense to you. What did you do?
- What is your rule of thumb for when to have an AI write code, and when to write it yourself? Did it change during this lab?
- Note:
data/flights.csv(535,543 rows, 23 MB) is built from the BTS T-100 Domestic Segment table for U.S. carriers, 1990โ2025, aggregated to city pairs and years. Total passengers fall from 821 million in 2019 to 342 million in 2020. The example output above is real. networkx'spagerankneedsscipy, so it is a dependency.