Developers of almost any skill level can write an HTTP/1.1 compatible implementation (and its fault tolerant enough that compatible-ish is ok). Soon we will live in a world with 3 or 4 HTTP/2.0 libraries and developers will have almost no knowledge of how the thing works under the hood.
It's not as if it needs to become opaque to developers. If people need a way to introspect a SPDY connection, someone will write a tool.
This is in the same vein as TCP, TLS, or DNS. Developers use these technologies all the time, and it exposes a simple interface to them, but all of them are quite complicated under the covers. I don't believe that Nagle's algorithm or the UDP framing for DNS is common knowledge amongst the majority of developers.
Similarly, SPDY or HTTP/2.0 will expose a simple interface to developers, but internally transport the content in an efficient, if somewhat inscrutable, manner.
While I appreciate the sentiment that HTTP is easy to understand as it stands, I'd rather not optimize for "developers of almost any skill level". While it would be totally OK for you to write a HTTP/1.1 parser as a toy, it would be somewhat crazy for you to use that implementation in a production scenario.
There, everybody uses a small set of implementations that have been vetted: Firefox, Chrome, Apache, Nginx, etc. These guys work extremely hard to make sure that their implementation is solid not merely "compatible-ish". I'd rather optimize for them, rather than the beginners. I'd rather make it easy for them to write correct implementations. I'd rather jettison the "fault tolerant enough that compatible-ish is ok" design of HTTP/1.1 that makes their life so difficult.
I think you made the point for me. TCP, TLS, and DNS (with the exception of OpenDNS a good 20 years later) see no innovation or hacking from startups. Compare to plain text protocols like HTTP.
