Subject: Unicode interpretation of SOFT HYPHEN breaks ISO 8859-1 compatibility

The recent clarication of the semantics of SOFT HYPHEN (U+00AD) in Unicode 4.0 [1, 2] did not undergo any public review [3] and appears problematic with regard to existing implementation practice. Changing its general category from \Pd " (punctuation, dash) to \Cf " (other, format) along with clarifying that it has no rendering breaks backwards compatibility with how this character has been widely used in ISO 8859-1 terminals for the past 15 years. The change now causes headaches for the designers of VT100-style terminal emulators with ISO 8859-1 and UTF-8 support. Modern text processing systems make a clear distinction between an unformatted content data stream (e.g., a Word, TeX or HTML le) and a formatted presentation datastream (e.g., a PDF, PostScript, or PCL le). This distinction is today given for granted by people who grew up with Word and similar content-data-stream editors. Unicode with its character/glyph model is clearly targeted primarily for use in content data streams, which still have to undergo line breaking before display. However, ISO 8859-1 was written in about 1984, when internationalization was all about adding diacritical characters for European countries, which preserved the one-to-one character/glyph mapping of tradi