Byte order mark (BOM) header

FastCSV is capable of reading CSV files with a Byte order mark (BOM) header.

Note

A byte order mark (BOM) is a sequence of 2 to 4 bytes at the start of a text file that serves as a header to indicate the file’s Unicode encoding, such as UTF-8, UTF-16, or UTF-32. For UTF-16 and UTF-32, the BOM header also indicates the byte order (big-endian or little-endian).

While UTF-8 is the standard encoding for most text files today, some applications still use the BOM header to explicitly specify the file’s encoding.

Enabling automatic BOM header detection can impact performance. Due to the decreased usage of BOM headers in modern applications, BOM header detection is disabled by default. To enable BOM header detection, use the detectBomHeader method of the CsvReaderBuilder class.

You may also want to check out the corresponding Javadoc for more information.

The following table shows the BOM headers for different Unicode encodings that FastCSV can detect:

Encoding	BOM header (hex)
UTF-8	EF BB BF
UTF-16 (BE)	FE FF
UTF-16 (LE)	FF FE
UTF-32 (BE)	00 00 FE FF
UTF-32 (LE)	FF FE 00 00

Example

In the following example, a CSV file with a BOM header is created and read using FastCSV.

ExampleCsvReaderWithBomHeader.java

package example;

import static java.nio.charset.StandardCharsets.UTF_16LE;

import java.io.IOException;
import java.io.OutputStreamWriter;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.stream.Stream;

import de.siegmar.fastcsv.reader.CsvReader;
import de.siegmar.fastcsv.reader.CsvRecord;
import de.siegmar.fastcsv.writer.CsvWriter;

/**
 * Example for reading CSV files with a BOM header.
 */
public class ExampleCsvReaderWithBomHeader {

    public static void main(final String[] args) throws IOException {
        final Path testFile = prepareTestFile();

        final CsvReader.CsvReaderBuilder builder = CsvReader.builder()
            .detectBomHeader(true);

        try (Stream<CsvRecord> csv = builder.ofCsvRecord(testFile).stream()) {
            csv.forEach(System.out::println);
        }
    }

    // Create a file with content encoded in UTF-16 little-endian and
    // a corresponding BOM header
    static Path prepareTestFile() throws IOException {
        final Path tmpFile = Files.createTempFile("fastcsv", ".csv");
        tmpFile.toFile().deleteOnExit();

        try (var out = Files.newOutputStream(tmpFile);
             var csv = CsvWriter.builder()
                 .build(new OutputStreamWriter(out, UTF_16LE))) {

            // Manually write UTF-16LE BOM header
            out.write(new byte[]{(byte) 0xff, (byte) 0xfe});

            csv.writeRecord("a", "o", "u");
            csv.writeRecord("ä", "ö", "ü");
        }

        return tmpFile;
    }

}

You also find this source code example in the FastCSV GitHub repository.

Tip

To create CSV files with a BOM header using FastCSV, you have to write the BOM header manually, as shown in the example. FastCSV does not support creating CSV files with a BOM header by itself.