bomstrip — strip Byte-Order Marks from UTF-8 text

Description | Download | Acknowledgements

Description

The bomstrip utility reads a UTF-8 file on its standard input and copies it to the standard output, removing the three-byte BOM from the start if it is present. The fun part is that bomstrip is actually a collection of various implementations in many programming languages! For a better explanation of why strippping the BOM from UTF-8 files is good and necessary, please take a look at the real bomstrip page by Mechiel Lukkien.

This is a patchset that adds a couple of new implementations (C++, Awk, and a Perl one-liner so far), a manual page, an additional bomstrip-files utility to strip files in place, and a Makefile to install the whole thing.

Downloading bomstrip

The bomstrip utility may be fetched from its real homepage at Mechiel Lukkien's site — or you may get it here, along with my patchset :)

The latest release is bomstrip-9 — with the patchset making it bomstrip-9-roam-02:

• bomstrip-9.tgz — the original source archive (with a PGP signature);
• ChangeLog-roam.txt — the list of changes in my patchset;
• bomstrip-9-roam-02.patch — my second patchset to bomstrip-9 (with a PGP signature);
• bomstrip-9-roam-02-extra-files.tar.gz — four additional test files (with a PGP signature) that I've added to test handling of multi-line files, files without a linefeed at the last line, and files with a DOS-style CR/LF end-of-line marker. Those are packaged separately from the patch because some implementations of patch(1) cannot really handle files with no LF on the last line and files with varying line endings.
• a Debian GNU/Linux package: bomstrip-9-2 — in the Debian archive for both the unstable and testing distributions.

Older releases:

• bomstrip-9-roam-01

  • bomstrip-9-roam-01.patch — my first patchset to bomstrip-9 (with a PGP signature);
  • bomstrip-9-roam-01-extra-files.tar.gz — the additional test files (with a PGP signature).

• bomstrip-8-roam-06

  • bomstrip-8.tgz — the original source archive (with a PGP signature);
  • bomstrip-8-roam-06.patch — my sixth patchset to bomstrip-8 (with a PGP signature);
  • bomstrip-8-roam-06-extra-files.tar.gz — the additional test files (with a PGP signature).

• bomstrip-8-roam-05

  • bomstrip-8-roam-05.patch — my fifth patchset to bomstrip-8 (with a PGP signature);
  • bomstrip-8-roam-05-extra-files.tar.gz — the additional test files (with a PGP signature).

• bomstrip-8-roam-04

  • bomstrip-8-roam-04.patch — my fourth patchset to bomstrip-8 (with a PGP signature);
  • bomstrip-8-roam-04-extra-files.tar.gz — the additional test files (with a PGP signature).

• bomstrip-8-roam-03

  • bomstrip-8-roam-03.patch — my third patchset to bomstrip-8 (with a PGP signature);
  • bomstrip-8-roam-03-extra-files.tar.gz — four additional test files (with a PGP signature).

• bomstrip-8-roam-02

  • bomstrip-8-roam-02.patch — my second patchset to bomstrip-8 (with a PGP signature);
  • bomstrip-8-roam-02-extra-files.tar.gz — the additional test files (with a PGP signature).

• bomstrip-8-roam-01

  • bomstrip-8-roam-01.patch — my first patchset to bomstrip-8 (with a PGP signature);
  • bomstrip-8-roam-01-extra-files.tar.gz — the additional test files (with a PGP signature).

Acknowledgements

Thanks go to (in no particular order):

• Mechiel Lukkien for starting the bomstrip project;
• Andreas Gohr, Berteun Damman, Matthijs Bomhoff, Ruben Smelik, and Andrew Gerrand for contributing implementations in many other languages;
• the FreeBSD Project for providing me with a robust OS to play with stuff on;
• Debian GNU/Linux for pretty much the same;
• …and everybody else who feel they deserve gratitude (and this probably won't be the first time I've forgotten to express gratitude, either).

Description | Download | Acknowledgements