Predicting school performance from satellite imagery

A big part of what makes a school good cannot be seen from space, like its teachers or its community. But it is interesting to ask what is visible from above. So I gave the model satellite imagery of each school and asked it two yes-or-no questions.

First, is the school's math score above the national average? Second, is it above average compared only to schools with a similar socioeconomic levelA number that scores each school by the socioeconomic background of its students. I used it to group schools with similar economic conditions.?

The biggest challenge was that the government's school locations were often wrong. When I first downloaded the imagery, a lot of the tiles were empty forest or random city blocks with no school in sight.

So I looked the schools up by name on Google and found that the real coordinates were often different from the official ones. I used the Google Maps API to fix the locations before training.

I ended up giving the model two views of every school. A close-up view from Google Maps that shows the campus and the streets right around it, and a wider view from Landsat that shows the whole neighborhood.

Getting the close-up view right took some trial and error. I started with 256px tiles at the closest Google zoom, but that only covered about 200m around a point, which was not enough to actually see a school. Switching to 512px tiles captured more of the campus and its surroundings, and picked up far more real schools.

Close-up (Google Maps)

Wider view (Landsat)

Close-up (Google Maps)

Wider view (Landsat)

Each view goes into its own vision modelTwo DINOv2 ViT-S/14 vision transformers, one finetuned on the close-up view and one on the wider view.. A final simple modelA small logistic regression that takes the two vision models' outputs and produces the final answer. then takes both of their predictions and produces the two answers.

I trained the two vision models separately and kept the version from the epoch where the validation loss was lowest, before each model started to overfit.

Close-up model

Wider-view model

The combined model got the national-average question right about 75.1% of the time, and the harder within-group question right about 65.8% of the time. The wider neighborhood view carried most of the signal about a school's economic setting, while the close-up view helped tell apart schools that looked similar on paper.

Combining the two views with the simple model on topA small logistic regression that takes the two vision models' outputs and produces the final answer. also beat my earlier, more basic way of merging them.

One thing worth being clear about: this shows what the model can pick up on, not what makes a school good. It is a tool for spotting patterns and forming questions, not for judging any single school.

I also used a standard methodIntegrated gradients: a method that highlights which pixels pushed the model toward its answer. to check where the model was looking. Honestly it did not reveal much, but it does seem to focus on the center of the image, which is right where the school sits.