A cartoon is worth a thousand tweets when it comes to the debate over whether the dress is white and gold or blue and black. And, to that end, internet nerd-cartoon XKCD* posted this illustration that makes pretty clear what's going on with the mystery dress.
Color is a function of light
When you look at the original photo, your eye is naturally drawn to the dress in the center of the image. But your interpretation of the dress depends on the way your eye perceives the light around it.
Because both blue- and yellow-tinged light are present in the image background, some people's brains perceive the original image as being lit by blue light (or just the photograph being tinted blue) while others perceive the whole scene as being lit by yellow light. But while our brains are making that judgment we're actually focused on the dress, and so we're expressing the outcome of an automatic, unconscious judgment about the light as a judgment about the color of the dress.
XKCD's illustration forces that process into conscious thought. The dress colors are the same on both sides. It's the backgrounds that vary. And so what looks gold on the left looks much darker on the right, which is why some people see that color as blue and others as brown or black. Vox's Joe Posner annotated XKCD's illustration a bit to make this even clearer:
This is a well-known optical illusion
Let's take this beyond the dress. Look at this checkerboard with a weird cylinder casting a shadow over it:
The squares marked A and B are exactly the same shade of gray. Here's Posner's modification again:
"As with many so-called illusions, this effect really demonstrates the success rather than the failure of the visual system," writes Edward Abelson, who posted the illusion. "The visual system is not very good at being a physical light meter, but that is not its purpose. The important task is to break the image information down into meaningful components, and thereby perceive the nature of the objects in view."
The original photograph of the dress isn't meant to be an optical illusion. And so there's no comparison on the screen that helps you switch your perception of it back-and-forth. As such, people are getting locked into their first perception, because the brain is trying to help them perceive the object in view, not try to help them interrogate the nature of light and color.
"What's happening here is your visual system is looking at this thing, and you're trying to discount the chromatic bias of the daylight axis," neuroscientist Bevil Conway told Wired. "So people either discount the blue side, in which case they end up seeing white and gold, or discount the gold side, in which case they end up with blue and black."
Further reading: The color-changing dress, explained
*I give that description as a huge fan. I literally have prints of XKCD cartoons hanging in my house.