⭐ The big idea: this assignment is all about getting to know a dataset before doing any math on it — peek at rows, check for junk (missing values, duplicates), slice out pieces you care about, and draw pictures. EDA = the "look before you leap" of data science. Here the data is clean (no missing, no duplicates), so the fun is in the slicing and the plotting.
🟦 The Setup
The College.csv dataset (from the ISLR textbook) holds 777 U.S. universities with 18 variables: applications received (Apps), accepted (Accept), enrolled (Enroll), tuition (Outstate), whether it's Private, graduation rate, and more. The first column holds each university's name, so we make that the index — every row is now labelled with its school.
🗂️ Index = name tag: instead of calling a row "row 42", we call it "Princeton University". Python won't treat the name as a number, which is exactly what we want.
1️⃣ Question 1 — Peek at the data
Use
head() to view the first few rows.✅ Answer:
college.head() → shows the first 5 rows (Abilene Christian, Adelphi, Adrian, Agnes Scott, Alaska Pacific).1
After loading, the very first column is the university name (labelled
Unnamed: 0 in the raw CSV).2
We rename it to
Names, then set it as the index with college.set_index('Names', inplace=True).3
Now
college.head() prints each school's name at the left edge instead of a row number.2️⃣ Question 2 — The info() summary
Use
info() to produce a numerical summary of the variables.✅ Answer:
college.info() → 777 rows, all no missing values, 1 categorical (Private: Yes/No) + 17 numeric columns.1
info() prints each column, its data type, and how many non-null (non-missing) entries it has.2
Every column shows 777 non-null → the dataset is complete. No missing values to worry about.
3
Only
Private is object (text). Everything else is a number, so most columns are ready for math.🔍 Health check:
info() is the doctor's checkup — it tells you if any variable is "sick" (missing data) before you start analysing.3️⃣ Question 3 — Check for duplicates
Examine for duplicates and drop them in place.
✅ Answer:
college.drop_duplicates(inplace=True) → 0 duplicates found, so nothing changes. (Still good practice to check!)1
college.duplicated().sum() counts how many duplicate rows exist.2
Here the count is 0 — every row is unique.
3
The
inplace=True argument tells pandas to modify college directly instead of returning a copy.🧹 Tidiness: duplicates often come from merging messy data or typing errors. Always sweep for them — even if the floor is already clean.
4️⃣ Question 4 — Replace missing Apps with 0
Replace any missing values in the
Apps column with 0.✅ Answer:
college['Apps'] = college['Apps'].fillna(0) → there were already no missing values, so nothing changes (and the test confirms no row has 0 apps).1
fillna(0) replaces every NaN (missing) in Apps with 0.2
Since
Apps had no NaN, the column is unchanged.3
The assertion
college[college['Apps']==0].empty checks that no real school has 0 applications — every school got some applications.☑️ Just in case: even clean data gets a safety check. Filling missing numbers with 0 is a simple default so downstream math doesn't crash.
5️⃣ Question 5 — Least out-of-state tuition
Find the college with the least out-of-state tuition (return the name).
✅ Answer:
college_least_tuition = college['Outstate'].idxmin() → "Brigham Young University at Provo".1
idxmin() returns the index label (the school name) of the row with the smallest value in that column.2
That's why we set the index to university names — now
idxmin() gives us the name directly, not a row number.🏷️ Name, not number:
idxmin() is like asking "who has the cheapest tuition?" and getting "Brigham Young" back instead of "row 87".6️⃣ Question 6 — Select the PhD column
Select the
PhD column as a dataframe; find its length.✅ Answer:
phd_column = college[['PhD']] → phd_column_length = len(phd_column) = 777.1
Use double brackets
[['PhD']] to get a DataFrame (neat table, keeps row names, 2-D).2
Use single brackets
['PhD'] to get a Series (a 1-D column, like a numpy array).3
Since there's one
PhD value per university, its length equals the number of rows = 777.📑 Table vs list: a DataFrame is a spreadsheet; a Series is one column of it. Double brackets = "give me the whole table with just this column".
7️⃣ Question 7 — Slice rows 15 & 16
Select
Private and Top10perc, keeping only rows 15 and 16 (0-indexed).✅ Answer:
private_top10 = college[['Private','Top10perc']].iloc[15:17] → rows are American International College (Top10perc 9) and Amherst College (Top10perc 83). Length = 2.1
First pick the two columns with double brackets.
2
.iloc[15:17] slices by position — rows 15 and 16 (17 is exclusive), giving exactly 2 rows.3
So
len(private_top10[['Private']]) = 2.🔢 Position vs label:
iloc counts rows like an index card (0,1,2…); loc looks them up by name. Both are useful — know which one you're using.8️⃣ Question 8 — All schools with "Penn"
Select all rows whose index contains "Penn" (capital P).
✅ Answer:
many_penns = college[college.index.str.contains("Penn")] → 8 schools.1
.index.str.contains("Penn") checks each school name for the substring "Penn".2
It's case-sensitive, so only "Penn" (capital P) matches — no lowercase "penn".
3
Result: 8 schools including University of Pennsylvania, Penn State, and several "of Pennsylvania" schools.
🔎 Search: this is a text filter — like Ctrl+F for "Penn" across all the name tags, grabbing every row that has it.
9️⃣ Question 9 — Pairplot of the first 10 columns
Use
seaborn.pairplot() on the first 10 columns.✅ Answer:
sns.pairplot(college[college.columns[:10]]) → a scatterplot matrix of every pair.
🎨 How to read this: each square is one pair of variables. The diagonal shows each variable alone (histogram). The key takeaway — Apps, Accept, Enroll, and F_Undergrad all grow together: big schools get more applications, accept more, and enroll more.
👀 Eyes on everything: a pairplot shows every two-variable relationship at once. The clear diagonal "clusters" between size variables tell you they're strongly correlated.
🔟 Question 10 — Tuition: private vs public
Boxplot of
Outstate vs Private. Find average out-of-state tuition for private and for public.✅ Answer:
private ≈ $11,802 · public ≈ $6,813.
🎨 How to read this: the blue box = private schools, amber box = public. Private schools charge far more out-of-state (median well above $10,000), and they spread higher too. The dashed lines mark each average.
1
Compute the means by filtering on the
Private column: college.loc[college['Private']=='Yes','Outstate'].mean().2
Private ≈ $11,802 — about 73% more than public ($6,813).
💸 Price gap: private universities charge out-of-state students roughly double what public schools do — the classic sticker-price difference everyone knows.
1️⃣1️⃣ Question 11 — Build the "large university" mask
Create
large_university: True where Enroll exceeds the mean of all enrollments.✅ Answer:
large_university = college['Enroll'] > college['Enroll'].mean() (mean ≈ 780).1
This creates a boolean Series (a "mask") — one True/False per university.
2
A school is "large" if it enrolls more than the average of ~780 new students.
🎚️ The divider: a mask is a row of switches — True keeps a row, False drops it. It's the tool we use to slice data by a condition.
1️⃣2️⃣ Question 12 — The large_universities dataframe
Create a dataframe of only the large universities.
✅ Answer:
large_universities = college[large_university] → 218 rows (out of 777, so ~28%).1
Passing the mask inside
college[...] keeps only the rows where the mask is True.2
218 universities have enroll > mean (~780).
🎣 Filtering: the mask is like a net that catches only "big" fish. Out of 777 schools, 218 are big by this rule.
1️⃣3️⃣ Question 13 — 75th percentile of Enroll
Use
describe(include='all') on large_universities; find the 75th percentile of Enroll.✅ Answer:
enroll = large_universities.describe(include='all').loc['75%','Enroll'] = 2,408.1
describe(include='all') gives a full summary (count, mean, min, quartiles, max) for every column.2
Grab the
75% row and the Enroll column → 2,408.3
Meaning: the top quarter of "large" universities enroll more than 2,408 new students.
📶 Upper rung: even among the "big" schools, the biggest 25% are enormous — reinforcing that enrollment is right-skewed (a few giant flagships dominate).
1️⃣4️⃣ Question 14 — Applications histogram
Histogram of
Apps with bin size 200. Is it true more universities get 0–10,000 apps than 10,000–20,000?✅ Answer:
applications = True — 732 schools get 0–10,000 apps, only 42 get 10,000–20,000.
🎨 How to read this: nearly all the bars are crammed against the left edge (the green 0–10,000 band). Only a tiny handful reach the amber 10,000–20,000 band. Distribution is heavily right-skewed.
1
Count apps in 0–10,000:
((college['Apps']>=0)&(college['Apps']<10000)).sum() = 732.2
Count apps in 10,000–20,000: = 42.
3
732 > 42, so
applications = True.📉 Long tail: most U.S. colleges are small or regional and get modest applications. Only elite/large flagships pass 10,000. Application volume is a long-tail distribution, not a bell curve.
1️⃣5️⃣ Question 15 — Acceptance rate
Add
acceptance_rate = Accept / Apps to a copy. Which college has the lowest rate? Highest?✅ Answer: lowest =
college_copy['acceptance_rate'].idxmin() → "Princeton University" (~15.5%). Highest = idxmax() → "Emporia State University" (~100%).
🎨 How to read this: red bars = most selective (Princeton ~0.15, Harvard, Yale, Columbia). Green bars = most open (Emporia State, MidAmerica, Wayne State, Southwest Baptist all accept essentially everyone).
1
Always copy first:
college_copy = college.copy() so we don't touch the original dataframe.2
Add the derived column:
college_copy['acceptance_rate'] = college_copy['Accept'] / college_copy['Apps'].3
idxmin()/idxmax() return the school name with the lowest/highest rate.🎯 Selectivity: acceptance rate = how picky a school is. Princeton is hyper-competitive (only ~15% in); Emporia State is open-admission (accepts basically everyone). The range spans 15% → 100%.
🎮 Play with it yourself
📋 Quick Answer Sheet
| # | Question | Answer |
|---|---|---|
| 1 | head() | First 5 rows (5 schools) |
| 2 | info() | 777 rows, no missing, 1 categorical + 17 numeric |
| 3 | Duplicates | 0 found — drop changes nothing |
| 4 | Missing Apps → 0 | No missing already; fillna(0) no-op |
| 5 | Least out-of-state tuition | Brigham Young University at Provo |
| 6 | Length of PhD column | 777 |
| 7 | Rows 15–16 (Private, Top10perc) | Length = 2 (Am. International, Amherst) |
| 8 | Colleges with "Penn" | 8 schools |
| 9 | Pairplot first 10 cols | Size variables all correlate |
| 10 | Avg outstate private / public | $11,802 / $6,813 |
| 11 | large_university mask | Enroll > mean (~780) |
| 12 | large_universities | 218 schools |
| 13 | 75th pct of Enroll | 2,408 |
| 14 | More 0–10k than 10k–20k? | True (732 vs 42) |
| 15 | Lowest / highest acceptance | Princeton / Emporia State |
🧠 Checklist (did you get it?)
- ✅
head()peeks;info()checks health (missing values, types). - ✅ Always check duplicates & missing values — even clean data gets a sweep.
- ✅ Double brackets
[['col']]→ DataFrame; single['col']→ Series. - ✅
ilocslices by position;loclooks up by index label. - ✅
idxmin()/idxmax()return the name of the min/max row. - ✅ A boolean mask
df['col'] > valuefilters rows. - ✅ Copy before deriving new columns — never mutate source data in place.
- ✅ EDA = look, slice, plot — the numbers tell the story (private costlier, apps right-skewed, enrollment skewed).