This chart of the world's tallest buildings through 1884, made by George F. Cram in the 1880s, is a beautiful illustration of the record-setting buildings of the time:
The tallest building there is the Washington Monument, at 555 feet tall (it was under construction and dedicated in 1885).
But that's nothing compared with modern buildings. This is what happens when we modified the chart to include the Burj Khalifa of Dubai, the tallest building in the world at 2,717 feet (reaching to 2,722 feet with the tip):
How did we manage to start building things so ridiculously tall? Steel and concrete, which have incredible strength that makes them good for the task, didn't become commonly used in construction until the 1880s (the first all steel-framed skyscraper showed up in 1889). Other improvements, like the elevator, made extremely tall buildings more practical. Better engineering and design processes helped, as well. If you want to wade into the complexities of the Burj Khalifa's construction, this article is a good start.
The tallest building in the world has reached a lot higher in the past 100 years — who knows what the next version of this chart will look like.