That's the most unfortunate Python 3 change I've seen. I use byte codecs like hex, zlib, and base64 quite a bit more than text codecs. In Python 2, a programmer with forward-compatible habits can write
from __future__ import unicode_literals
from io import open
with the understanding that migration to Python 3 will remove that boilerplate. But taking a similar approach for byte codecs requires knowledge and reference of the right module name (instead of the encoding name) and the names of the corresponding encode and decode functions (instead of just encode and decode). So we've got
and unlike the text boilerplate, it's a permanent uglification. I don't know of an idiomatic replacement for the last one off the top of my head. Hopefully it's something nicer and more symmetrical than
Just a nitpick on this. This is actually a Python "gotcha". You'll notice that the .encode('base64') method actually is a base-64 Content-Transfer-Encoding[1] which enforces a limit on the length of the line to 76 characters. Here's an example demonstrating the difference:
import base64
eighty_chars = ("X" * 80)
assert '\n' in eighty_chars.encode('base64').strip()
assert '\n' not in base64.b64encode(eighty_chars)
I don't agree with that. IMO removing the zlib, hex, and base64 encodings was a good thing.
While an argument can be made that they're technically "encoding", they're really outside the scope of the problem the encode and decode methods were meant to solve.
Just that codecs supported incremental operations and base64.b64encode did not. Handling HTTP transfer encoding in python 2 was a matter of two lines and worked on arbitrary stream data. In Python 3 that's now ~50 lines of code with different behavior for each transfer encoding and not all of them support stream processing or have the same interface.
It's less convenient, it's a different API for every type of transformation, and the change has made code demonstrably worse.
Further, the use of encode/decode is explicit. The only thing that was implicit was the automatic transformation of string -> unicode when people mistakenly used unicode codecs on string objects, or the reverse for string codecs on unicode objects. The proper answer to both of these is to just not do automatic type conversion... which is what was done in Python 3.
So actually, had we left all of the codec machinery intact, those codec errors described by Armin wouldn't ever occur again! Instead, you'd get a TypeError caused by passing the wrong type of object to the underlying encoder/decoder.
The API is nice, but it is difficult to maintain. To get encoders/decoders into the string class in the first place, you have to maintain a global registry. (I suppose you could pass them all to the constructor of the string object, but nobody's going to do that.) The global codec registry leads to naming conflicts. If you import a module that globally adds a "foo" encoder, then you import another module that globally adds a "foo" encoder, now what? Both modules break because of their dependency on the global name "foo". Because of the details of the codec.register implementation, you can't even catch the conflict at registration time and refuse to load the second module, you simply have to wait until your program returns subtly-incorrect results.
Compare this to the scheme where you import codec modules explicitly and just call their functions. Your imports are lexically-scoped, and if you happen to need two encoders that use the same name, you can just alias one of them at import time. This strategy can't introduce unexpected errors as your program grows larger, because the side effects are constrained to one module. It either works now and will always work, or doesn't work and fails quickly while you are developing.
Ultimately, people use Python because they want a bit of discipline in their lightweight language. This isn't Javascript or PHP, after all :)
Codec registration has never been an issue. Let me repeat that with some emphasis, because it's an important point. Codec registration has NEVER BEEN AN ISSUE.
And global registries are not inherently a bad thing. If you were to say "I don't want a json/pickle/messagepack decoder built into the codecs module by default", I would agree - because it's not a string/unicide <-> string/unicode transformation. But it wouldn't bother me for someone to add that support in their stuff because data.decode('json') is terribly convenient. Arguably better than peppering your code with the following (or loading it in a shared space, or injecting it into __builtins__, ...)
try: import simplejson as json
except ImportError: import json
But ultimately we are adults. If you would prefer to import a cluster of modules just to convert your strings to hex or compress your string with zlib, you are free to do so. It's just unfortunate that due to misunderstandings about the fundamental underlying problem (TypeErrors), functionality was removed.
Ultimately, adding random methods to classes introduces many subtle side effects. (See my reply to a sibling comment above.)
There's noting intrinsically obvious about making the string class responsible for encoding and decoding, other than the fact that help("") mentions the existence of that method. Most other useful utilities that operate on strings are separate classes or modules; re, for example.
But I agree with you and with Armin; removing string -> string and unicode -> unicode encodings and decodings were a mistake. I said as much when the discussions about Python 3 and codecs were going on ~5 years ago.