What is WebP?
WebP is a next-generation image format released by Google in 2010. It offers better compression at the same level of quality as its older counterparts PNG and JPEG, while also supporting animation and transparency. Originally, it was only a lossy format but recently Google have added support for lossless compression too. This makes the WebP format a rival to both JPEG and PNG with Google’s own data showing it leads to smaller files by 25-34% and 26% respectively. As it’s also widely supported on all popular browsers except internet explorer, you can expect faster loading times and to use less storage. It’s a win-win.
The WebP format
The WebP format is based on the Resource Interchange File Format (RIFF), which is a generic container format which enables the storing of data into chunks.
At the start of the file is the WebP header itself, followed by one or more chunk headers and chunks. The chunk header lets you know what type of chunk it is - lossless, lossy, animation, extended, alpha, etc. and the chunk then contains the image data. So let’s take a look in detail at the format itself.
WebP file header
The file header is made up of 12 bytes of data which represent the following:
0 1 2 3
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 'R' | 'I' | 'F' | 'F' |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| File Size |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 'W' | 'E' | 'B' | 'P' |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Source: Google The first four bytes of the file spell out “RIFF” with the following four bytes making up a 32 bit unsigned integer representing the size of the file in bytes. The final four bytes then spell out “WEBP”. This layout is the same for all types of WebP files. Data that follows is then broken up into chunk headers and chunk data.
WebP Chunk Headers
The next 8 bytes of the file are dedicated to the chunk header with the first 4 bytes of this describing the type of chunk as a 4 letter string. There are a few options for this as follows:
- ‘VP8 ‘: Image data in the original lossy format (yes, there should be a space)
- ‘VP8L’: Image data in the lossless format. Some old readers may not support this.
- ‘VP8X’: The extended file format. Describes what chunks are present as well as height and width of image.
- ‘ANIM’ or ‘ANMF’: Animation chunks which are used to control the animation.
- ‘ALPH’: Contains data pertaining to the transparency of the image.
- ‘ICCP’: Contains data pertaining to the image’s colour profile.
- 'EXIF’ or ‘XMP ‘: Contains metadata about the image
For the purposes of this guide, we’re going to focus on VP8X: the extended file format, as this allows us to demonstrate how to read single bit flags using Rust as well as decode 24 bit integers. Including the Chunk header data, the file format now looks as follows:
0 1 2 3
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| WebP file header (12 bytes) |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ChunkHeader('VP8X') |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Rsv|I|L|E|X|A|R| Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Canvas Width Minus One | ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
... Canvas Height Minus One |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Source: GoogleThe first 12 bytes are the file header, as laid out in the previous section, followed by the 8 byte chunk header. The next byte then contains 6 flags denoting what other types of chunk will be present in the file. After 3 reserved bytes, the width and height of the image are encoded as two 24 bit integers (3 bytes each).
Decode the WebP file header in Rust
The full code is available here, on GitHub alongside the image used in the guide.
Let’s start by decoding just the file header. We’re going to doing everything using the Rust standard library so there’ll be no need for any dependencies. Just ensure you have Rust installed and then either clone Naurt’s Git repository or create a new project using:
cargo new read_webp_headersTo read the byte data from the WebP file we’ll be using Rust’s BufReader. This lets us keep track of our position in the file and read n bytes at a time. Let’s first take a look at the entire code snippet for reading in the WebP file header and then go through each stage step by step.
use std::{
fs::File,
io::{BufReader, Read},
};
fn main() {
let file = File::open("images/naurt_phone.webp").unwrap();
let mut reader = BufReader::new(file);
let mut four_byte_buffer = [0; 4];
reader.read(&mut four_byte_buffer).unwrap();
println!("{}", String::from_utf8(four_byte_buffer.to_vec()).unwrap());
reader.read(&mut four_byte_buffer).unwrap();
let file_size = ((four_byte_buffer[0] as u32)
| (four_byte_buffer[1] as u32) << 8
| (four_byte_buffer[2] as u32) << 16
| (four_byte_buffer[3] as u32) << 24)
+ 8;
println!("File size: {}", file_size);
reader.read(&mut four_byte_buffer).unwrap();
println!("{}", String::from_utf8(four_byte_buffer.to_vec()).unwrap());
}OUTPUT
RIFF
File size: 21380
WEBPOur output is exactly what we were expecting. The word RIFF, followed by the file size in bytes, followed by the word WEBP.
Reading RIFF
So how have we done this? First, we create a BufReader around the WebP file and then create a four byte buffer. We’ll use this to read the file 4 bytes at a time. In Rust a byte is represented by a single unsigned 8 bit integer (u8).
let file = File::open("path_to_my_image.webp").unwrap();
let mut reader = BufReader::new(file);
let mut four_byte_buffer = [0; 4];
reader.read(&mut four_byte_buffer).unwrap();reader.read returns a result containing the number of bytes read from the file. The bytes it reads are read into the buffer provided, hence the mutable reference to our buffer, &mut four_byte_buffer. Instead of using unwrap, some better error handling would usually be implemented but this is out of the scope of this tutorial.
Now we have the first four bytes of data, we need to convert them into a string. Rust makes this really easy with the String::from_utf8 method.
println!("{}", String::from_utf8(four_byte_buffer.to_vec()).unwrap());Bit Shifting for File Size
The trickiest part of this initial header reading is the file size. We’re told by the specification it’s a 32 bit unsigned integer, so we need to read the next 4 bytes and combine them from 4 u8 types into a single u32 type. We can do this using bit shifting. It’s a simple operation where each bit in your byte is shifted to the right or left. Let’s say your single byte has the value 5, represented in binary by 0000101. If you were to bit shift to the left 1, you’d move each bit to the left 1, resulting in 0001010 or 10. It’s the equivalent of saying $x \times 2^n$ where x is your value, and n is how many bits you’d like to shift.
We can then combine our 4 u8 bytes into a u32 by shifting our single bytes along to the correct position. The specification also tells us that our u32 is little endian encoded, meaning that the least significant bit comes first - the opposite of a Rust u32, so we need to make sure we reverse the order.
We can then represent this using the following rust code:
let file_size = ((four_byte_buffer[0] as u32)
| (four_byte_buffer[1] as u32) << 8
| (four_byte_buffer[2] as u32) << 16
| (four_byte_buffer[3] as u32) << 24) + 8;We use the logical OR operator | to combine the four values and then add 8 bytes as this value doesn’t account for the RIFF or WebP strings. Now we know how to read the file header, lets get started on the WebP chunk header.
Decoding the WebP chunk header in Rust
After the file header, the next data contained within the file is the chunk header type. This tells you what type of data you can be expecting in the chunk. Using the same reader as before (or a new one and using the seek function), we can read the next 8 bytes which contain the header type in the first four.
let mut eight_byte_buffer = [0; 8];
reader.read(&mut eight_byte_buffer).unwrap();
let chunk_type = String::from_utf8(eight_byte_buffer[0..4].to_vec()).unwrap();
println!("Chunk type: {}", chunk_type);Output
Chunk type: "VP8X"Now we know what type of chunk it is, we can decode the rest of the header. The VPX8 header contains a variety of boolean flags as well as the height and width of the image. So let’s start by making a struct that can hold all the data we’re interested in.
#[derive(Debug)]
pub struct ExtendedChunkHeader {
icc_profile: bool,
alpha: bool,
exif_metadata: bool,
xmp_metadata: bool,
animation: bool,
width: u32,
height: u32,
}Feature flags
As we’re interested in reading individual bits from the file we could use a Rust crate such as bitreader, but we’ve already covered bit shifting so we’re going to go ahead and use that method again. Here’s our full implementation:
impl ExtendedChunkHeader {
pub fn new_from_buf_reader<R>(reader: &mut R) -> Self
where
R: Read,
{
let mut four_byte_buffer = [0; 4];
reader.read(&mut four_byte_buffer).unwrap();
// Little endian encoded!
// First two bits are ignored. Reserved.
// ICC profile is the third bit, so we shift by 3. This is still within our first byte.
let icc_profile_mask = 1 << (8 - 3);
let icc_profile = (icc_profile_mask & four_byte_buffer[0]) > 0;
// Alpha profile is the fourth bit.
let alpha_mask = 1 << (8 - 4);
let alpha = (alpha_mask & four_byte_buffer[0]) > 0;
// Exif metadata is the fifth bit.
let exif_metadata_mask = 1 << (8 - 5);
let exif_metadata = (exif_metadata_mask & four_byte_buffer[0]) > 0;
// XMP metadata is the sixth bit.
let xmp_metadata_mask = 1 << (8 - 6);
let xmp_metadata = (xmp_metadata_mask & four_byte_buffer[0]) > 0;
// Alpha profile is the seventh bit.
let animation_mask = 1 << (8 - 7);
let animation = (animation_mask & four_byte_buffer[0]) > 0;
// This is the first byte finished as the last bit is reserved.
// The next 24 bits are reserved and just 0. Making up the rest of the 4 bytes.
// The width and height to come are the next 6 bytes. So let's read in three at a time now as they are 24 bit.
let mut three_byte_buffer = [0; 3];
reader.read(&mut three_byte_buffer).unwrap();
let width = ((three_byte_buffer[0] as u32)
| (three_byte_buffer[1] as u32) << 8
| (three_byte_buffer[2] as u32) << 16)
+ 1;
reader.read(&mut three_byte_buffer).unwrap();
let height = ((three_byte_buffer[0] as u32)
| (three_byte_buffer[1] as u32) << 8
| (three_byte_buffer[2] as u32) << 16)
+ 1;
return Self {
icc_profile,
alpha,
exif_metadata,
xmp_metadata,
animation,
width,
height,
};
}
}OUTPUT (Please refer to the GitHub repository for the full implementation)
WebpHeader { file_header: WebpFileHeader { riff: "RIFF", file_size: 21380, webp: "WEBP" }, chunk_header: Extended(ExtendedChunkHeader { icc_profile: false, alpha: true, exif_metadata: false, xmp_metadata: false, animation: false, width: 500, height: 1012 }) Let’s take a deeper look at how we read in an individual bit read in.
let icc_profile_mask = 1 << (8 - 3);
let icc_profile = (icc_profile_mask & four_byte_buffer[0]) > 0;The first line is creating a mask for our data. We only need one bit so we use the number 1, 0000001 and then shift it to the position we’re interested in. In this case it’s the 3rd bit in the byte - but remember, it’s little endian encoded so we have to reverse it using 8 - position. This results in our mask being 00100000 . We then use the & operator to check if the bit at that position is a 1 and therefore the flag is true. This then gets repeated for the four other flags in the file.
Width and Height
The width and height of the image are packed into 3 bytes each as a 24 bit unsigned integer. Decoding them is the same process as decoding the file size, but we’ll be using three bytes instead of four. You also need to add one as it’s ‘1-based’.
let width = ((three_byte_buffer[0] as u32)
| (three_byte_buffer[1] as u32) << 8
| (three_byte_buffer[2] as u32) << 16)
+ 1;And with that, we’ve read everything from both the file header and the first chunk’s header too!
Summary
Hopefully this guide has been a useful introduction to WebP in Rust. There’s plenty of scope to extend the code found on our GitHub to support other types of chunk header and even the image data itself. It’s worth noting that if you wanted a ready made package to read WebP images, convert them to and from JPEG and other formats, then we’d recommend using the image crate instead of going from scratch!
