Byte order mark (BOM) header

FastCSV is capable of reading CSV files with a Byte order mark (BOM) header.

Note

A byte order mark (BOM) is a sequence of 2 to 4 bytes at the start of a text file that serves as a header to indicate the file’s Unicode encoding, such as UTF-8, UTF-16, or UTF-32. For UTF-16 and UTF-32, the BOM header also indicates the byte order (big-endian or little-endian).

While UTF-8 is the standard encoding for most text files today, some applications (like Microsoft Excel) still use the BOM header to explicitly specify the file’s encoding.

Enabling automatic BOM header detection can impact performance. Due to the decreased usage of BOM headers in modern applications, BOM header detection is disabled by default. To enable BOM header detection, use the detectBomHeader method of the CsvReaderBuilder class.

You may also want to check out the corresponding Javadoc for more information.

The following table shows the BOM headers for different Unicode encodings that FastCSV can detect:

Encoding	BOM header (hex)
UTF-8	EF BB BF
UTF-16 (BE)	FE FF
UTF-16 (LE)	FF FE
UTF-32 (BE)	00 00 FE FF
UTF-32 (LE)	FF FE 00 00

The UTF-8 BOM header is the most common one that is also used by Microsoft Excel.

Example

In the following example, a CSV file with a BOM header is created and read using FastCSV.

ExampleCsvReaderWithBomHeader.java

package example;

import static java.nio.charset.StandardCharsets.UTF_8;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;

import de.siegmar.fastcsv.reader.CsvReader;
import de.siegmar.fastcsv.reader.CsvRecord;
import de.siegmar.fastcsv.writer.CsvWriter;

/// Example for reading CSV files with a BOM header.
public class ExampleCsvReaderWithBomHeader {

    public static void main(final String[] args) throws IOException {
        final Path testFile = createTempFile();
        writeTestFile(testFile);
        readTestFile(testFile);
    }

    static Path createTempFile() throws IOException {
        final Path tmpFile = Files.createTempFile("fastcsv", ".csv");
        tmpFile.toFile().deleteOnExit();
        return tmpFile;
    }

    static void writeTestFile(final Path file) throws IOException {
        try (var out = Files.newOutputStream(file);
             var csv = CsvWriter.builder().build(out, UTF_8)) {

            // Manually write UTF-8 BOM header
            out.write(new byte[]{(byte) 0xEF, (byte) 0xBB, (byte) 0xBF});

            csv.writeRecord("a", "o", "u");
            csv.writeRecord("ä", "ö", "ü");
        }
    }

    static void readTestFile(final Path file) throws IOException {
        final CsvReader.CsvReaderBuilder builder = CsvReader.builder()
            .detectBomHeader(true);

        try (CsvReader<CsvRecord> csv = builder.ofCsvRecord(file)) {
            csv.forEach(System.out::println);
        }
    }

}

You also find this source code example in the FastCSV GitHub repository.

Tip

To create CSV files with a BOM header using FastCSV, you have to write the BOM header manually, as shown in the example. FastCSV does not support creating CSV files with a BOM header by itself.