yEnc ! and B-News

B-News and yEnc: " *
The story of B-News and his younger brother, yEnc.


When B-News was 4 months old, its creator was contacted by someone who's name will not be disclosed here (just let it be said that it was not Jürgen Helbling), who suggested another encoding method based on 'quoted printable' escaping of control characters. A few tests were done, but B-News didn't adopt that method because its development had gone too far in another direction already, and for other reasons that will become clear in the next two paragraphs.

A few months later, in another part of the usenet world, the first work started on a "standard specification" that was based on that same method, slightly extended, and now it had also got a name: yEnc. You'll find the name of B-News' creator in the credits list on the general topics page on the yEnc site, even though he didn't do more than follow the initial discussions about it.

yEnc is less CPU-intensive than B-News and reaches about the same low level of overhead, but it doesn't avoid the use of all control codes, it just leaves out those that were (experimentally) observed to have undesired effects on some servers, which means that it's somewhat less RFC compliant than B-News.

B-News was designed to be as compliant to RFC's as possible (except for assuming 8 bit cleanness) to avoid any problems with existing news software, yEnc was designed with an attitude of "if it doesn't follow the RFC's, then they'll have to change the RFC's."

The reason of yEnc's success is that it was heavily plugged in newsgroups and with newsreader writers right from the start, something the author of B-News wanted to avoid until it had proven to work and had been tested on a broader basis (including mail).

On the server where it was born (telenet's), B-News has in the mean time grown to be the primary posting method. That server used to keep all binaries local to save on feed bandwidth until that was changed recently - to the discontentment of some usenet users who now start finding BN posts in "their" groups without knowing what they are.

Because of the huge success of yEnc and because it's useless to invent the wheel twice, development on B-News has come to a virtual standstill. It is expected that telenet's users will gradually move towards using yEnc over time: as it is now, some are heavily opposed and others say it's a dumb thing not to move right away, the kind of discussions that can be expected on usenet ;-)

----------------
------------http://b-news.sourceforge.net/ -----------
*
What is (or was) B[omma]News?

BommaNews was, in January of '01, the original development title of a newsreading (or rather not) program. It has its roots - development and betatesting - among the users of a Belgian ISP called Telenet (aka Pandora), and Pandora's original logo was a grandma, affectionately nicknamed "bomma" in Flemish, hence the name.

*
Why B-News?

B-News (or bommanews) was developed to lift the weight of the overhead inherent to UUEncode and Base64 encoding: it uses a new encoding method to stuff binary data in text messages. This method eats more CPU resources, but it manages to lower the loss from approximately 40% for UUEncode to 3.5% (the decimal point between those digits is not dirt on your monitor), while still avoiding the use of ANSI control codes in the message body.

For the rest, B-News (the application) is very limited in functionality: it was designed primarily to post files to a usenet server, and download files that were posted in its own format from there again. A stand-alone decoder was also developed to allow downloading with classic newsreaders, and for Mac and Linux users someone else created an encoder and decoder too.

New in version 1.1 are some [very primitive] text posting and reading capabilities.

*
Where does yEnc come in?

Find out how B-News and yEnc are related here.

*
How does it do that? Where are the drawbacks?

UUEncode and Base64, the encoding methods used by classical newsreaders and mail software, each split three bytes of binary data into four bytes of text, using 6 databits per byte - encoding is done by simply masking bits out of the bytes. Six bits also means that only 64 different characters out of a total of 256 are used in the text representation of binary data, even though more was already theoretically possible in the days when UUEncode and Base64 were developed.

Bommanews pushes it to the limit, and uses all available values that fit in a byte with exception of the ASCII control codes 00 to 1F hex.

Because it uses 224 different codes, this means that it requires a newsserver that treats incoming data in a perfect 8-bit clean way, not only allowing codes above 7F hex, but also not choking on control codes with the parity bit set (80-9F), and it requires that the server gives back exactly the same character sequence when such a message is downloaded later.

Using 224 different codes also means that encoding and decoding isn't any longer a matter of simple logical operations (shift, and, or & the family): in fact the processor is doing quite a bit of arithmetics behind the screens.

Because newsfeeds at telenet's servers usually suck to such an extent that passing out posts would be useless, and because the shared files often aren't interesting for anyone but the persons they are intended to reach, BommaNews posts all its files with a 'local' distribution by default - so the posts never leave the newsserver they are posted on if the server supports this - but not all servers do! This just might be the reason why you haven't seen any yet. [Added: telenet's servers stopped supporting local distribution in June 2001, but instead started filtering out all messages larger than 16K bytes from the newsfeed, which filtering it stopped in turn in september 2002.]

In the mean time, B-News has been tested and has already created a pretty high volume of "messages" without any sign of a problem on three types of newsservers: Microsoft IIS 5.0 and 6.0, and especially on Twister (which happens to be Telenet's newsserver). [Added: this was written in 2001, you can add a bunch to that.]

*
The encoding/decoding details, in plain English. .../...


------------