One of the first income tax evasion prosecutions, follow the passing of the 16th amendment (making income tax permanent), relied almost entirely on bank deposit records. Centralized reporting requirements are pretty important to the IRS. I couldn't find the specific case, but this paper [0] covers the topic pretty thoroughly. If you are incredibly bored and end up reading it, you'll notice that in their investigations the IRS commonly skips a lot of things that would be labor intensive (like finding property held under a different name), and sticks to the prewrapped financial records from the banks.

So yeah, taxes existed before the internet - they used paper deposit slips.

[0] http://digitalcommons.law.yale.edu/cgi/viewcontent.cgi?artic...