git: locale - Add two new API functions

[Matthew Dillon]

commit 8a84c799639d2e842952ce56fad530a085b16712
Author: Matthew Dillon <dillon at apollo.backplane.com>
Date:   Sun Aug 16 15:03:03 2015 -0700

    locale - Add two new API functions
    
    * Add wcrtombin() and mbintowcr(), powerful buffer-to-buffer functions
      to translate UTF8, which can also perform escaping and de-escaping,
      allow legacy parsing, and do checks with varying levels of strictness.
    
      NOTE: If no locale or "C" locale, these functions are 1:1 (do not do
    	UTF8 processing).  A UTF8 locale must be used to access these
    	functions.
    
    * Add wcrtoutf8() and utf8towcr().  These work exactly the same as
      the above functions but are locale-agnostic.  They always operate in
      UTF8 and locale initialization is not necessary.
    
    * Escaping and de-escaping for 8-bit-clean round-trips with two of the
      modes.
    
    * Can construct UTF8B from unescaped 8-bit-clean data, can deconstruct
      UTF8B (restore the original clean data), and can also be flagged
      generically to take already-escaped UTF8B or wchars as input.
    
    * Can be used to perform strict checks.
    
    * Very powerful incremental buffer validation and processing capabilities.
      Far more visibility to the caller rather than hiding state in a tracking
      state structure.  No tracking state structure.
    
    * Does not do idiotic nul buffer termination.  Caller passes explicitly
      sized data, can opt to include or not include a terminator (depending
      on what is desired), and the byte buffers can contain any 8-bit code
      including 0x00.

Summary of changes:
 include/locale.h                  |   1 +
 include/wchar.h                   |  14 ++
 include/xlocale/_wchar.h          |   8 +
 lib/libc/locale/Makefile.inc      |   8 +-
 lib/libc/locale/Symbol.map        |   3 +
 lib/libc/locale/locale_isstrict.c |  52 ++++++
 lib/libc/locale/mbintowcr.3       | 262 +++++++++++++++++++++++++++++++
 lib/libc/locale/mbintowcr.c       |  44 ++++++
 lib/libc/locale/mblocal.h         |   6 +
 lib/libc/locale/mbtowc.3          |   2 +
 lib/libc/locale/multibyte.3       |   4 +
 lib/libc/locale/none.c            |  97 ++++++++++++
 lib/libc/locale/utf8.c            | 321 +++++++++++++++++++++++++++++++++++++-
 lib/libc/locale/wcrtomb.3         |   2 +
 lib/libc/locale/wcrtombin.c       |  44 ++++++
 lib/libc/locale/xlocale_private.h |   2 +
 test/debug/utf8bin.c              | 101 ++++++++++++
 17 files changed, 969 insertions(+), 2 deletions(-)
 create mode 100644 lib/libc/locale/locale_isstrict.c
 create mode 100644 lib/libc/locale/mbintowcr.3
 create mode 100644 lib/libc/locale/mbintowcr.c
 create mode 100644 lib/libc/locale/wcrtombin.c
 create mode 100644 test/debug/utf8bin.c

http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/8a84c799639d2e842952ce56fad530a085b16712


-- 
DragonFly BSD source repository