Public schools are a really nice idea. The government builds a building, right in your neighborhood, where anyone can send their kids to get an education for free. It's simple and appealing.
But in practice, it's quite a bit different. Land that is in the intake zone for a good school becomes more expensive, and you create a situation in which the school is open exclusively to the "public" of people who can afford a very expensive house.
Look at this chart showing the correlation between the price of a family-size house and the reading proficiency scores in the local school (the outlier, Garrison, where the reading scores are terrible and the houses are expensive anyway is my neighborhood public school):
Of course, in principle this situation could be mitigated. It is true that land near Eaton Elementary School in Cleveland Park is very expensive, but the technology exists to cram many dwellings onto a given parcel of land. You can build homes that share side walls ("townhouses"), and you can even use steel-frame construction to build vast vertical stacks of dwellings, accessible from the ground floor via elevator. That wouldn't address the housing cost problems of the truly poor, but a less-constrained building supply would at least improve the situation for the middle class.
In practice, however, affluent neighborhoods with good schools tend to enact zoning rules that mandate detached single-family homes. That ensures that expensive land leads to expensive houses, thus preventing undesirable types from accessing the "public" school.
The other possible policy reform is to change the relationship between geography and school access.
In DC, you are guaranteed the right to send your kid to your in-zone elementary school, but all charter schools admit students purely on the basis of a lottery. Convenience still counts in life, so the charter system hardly eliminates geographic sources of disadvantage. But it does mitigate them. Shifting to more reliance on charter schools or having public schools admit students without geographic preference would be good for equality. But in this case, equality really is a leveling measure that lifts up poorer households in part by dragging down richer ones.
Homeowners in pricey neighborhoods have the value of their house tied directly to both excluding others from access to the neighborhood school and also to the absence of quality educational alternatives to buying a house in Fancytown.