Technology Stack
Python, NumPy, Parallel Programming, Bootstrap, PythonAnywhere
Overview
Duration: 4/2023
An implementation of the JPEG compression standard built from the specification rather than by calling an imaging library — the transform, the quantisation, and the entropy coding written directly, then parallelised and deployed as a web tool where an image can be uploaded and compressed at a chosen quality level.
The value of building a codec from scratch is that compression is where several disciplines meet at once: signal processing in the transform, human perception in the quantisation tables, and information theory in the entropy coding. Implementing it means understanding why each stage exists rather than which function to call.
The Pipeline
Colour space conversion. RGB is converted to YCbCr, separating luminance (Y) from the two chrominance components (Cb, Cr). This is the first perceptual decision in the format: human vision is substantially more sensitive to brightness detail than to colour detail, so separating them allows the colour channels to be discarded more aggressively than the brightness channel without a visible difference.
Block decomposition and DCT. The image is divided into non-overlapping 8×8 blocks, and each block undergoes a Discrete Cosine Transform from the spatial domain into the frequency domain. The output is a set of coefficients describing the block as a sum of frequency patterns, with the low-frequency components — the broad structure — concentrated in one corner.
Quantisation. This is where the loss happens, and where the compression comes from. Each coefficient is divided by a value from a quantisation table and rounded. The table applies coarse division to the high-frequency coefficients, which encode fine detail the eye barely resolves, and fine division to the low-frequency ones. Most high-frequency coefficients round to zero, and it is that field of zeros the next stage exploits. The quality setting is, precisely, a scale factor on this table.
Entropy coding. The quantised coefficients are read in a zigzag order that groups the zeros into long runs, then compressed with run-length and Huffman coding. This stage is fully lossless — it packs the data the previous stage decided to keep.
Storage. The coded blocks are written with the tables required to reverse the process into a standard-conforming file.
Parallelisation
Block independence is the property that makes JPEG parallelise cleanly: the transform and quantisation of any 8×8 block depend on no other block. The implementation exploits this by distributing blocks across workers, with the entropy coding stage handled after the parallel section, since Huffman coding depends on global symbol statistics.
Deployment
The codec runs behind a web interface where an image is uploaded, a quality level chosen, and the result returned alongside the original with the achieved compression ratio — so the perceptual cost of each quality setting is visible rather than described.
References
- Wallace, G. K. (1992). The JPEG still picture compression standard. IEEE Transactions on Consumer Electronics, 38(1).
- Pennebaker, W. B., & Mitchell, J. L. (1992). JPEG: Still image data compression standard. Van Nostrand Reinhold.
- Sayood, K. (2000). Introduction to data compression. Academic Press.
- Li, H., Zeng, B., Li, Z., & Wu, X. (2004). A fast DCT-SVD image compression algorithm. IEEE Transactions on Circuits and Systems for Video Technology, 14(3).
- Lee, K. M., & Ho, Y. S. (2003). A new fast and efficient image codec based on the discrete cosine transform. IEEE Transactions on Circuits and Systems for Video Technology, 13(7).