In 2005, during a routine conference on rock art, archaeologist Jon Harman saw two side-by-side images of the Martian surface. One showed the familiar, uniform rust-red terrain; the other, processed by a NASA algorithm, had turned into a vibrant mosaic of reds, greens, and yellows with razor-sharp contrast. The dramatic visual shift instantly struck his research intuition. On the spot, Harman ran the algorithm against a digital photograph he had taken inside a cave in Baja California, Mexico. From what had appeared to be a completely blank rock surface, a previously unseen yellow humanoid figure emerged.
Human Eyes Succumb to Time, but Linear Algebra Endures
The ancient rock paintings had never actually disappeared. Over thousands of years, wind erosion, sun exposure, and pigment degradation had flattened color contrasts well below the resolving power of the human retina. The original paints had been diluted into a narrow chromatic band imperceptible to the naked eye.
Figure: Jon Harman standing in front of an example of Native American rock art. Credit: NASA / Jon Harman
Against this physical compression of optical information, raw human vision was powerless. Harman, who holds a Ph.D. in mathematics from UC Berkeley and spent years working in medical imaging, realized that a byproduct of space exploration could be repurposed as a statistical transform to forcefully stretch pixel distances in color space.
Before this technique arrived, archaeologists had little choice but to rely on natural daylight, raking light, and subjective guesswork to trace stone-wall figures. Centuries of accumulated dust and mineral deposits had built what seemed like a physical wall of time. As long as human chromatic discernment remained constrained by biological limits, that barrier seemed insurmountable. The cross-disciplinary introduction of digital image processing dismantled that perceptual bottleneck once and for all.
Forcing Crowded Pixels Apart: The Decorrelation Stretch
The technique behind the Martian imagery is known as decorrelation stretch (DStretch). In 1978, Jim Soha, then head of the digital image processing group at NASA’s Jet Propulsion Laboratory (JPL), proposed applying the Karhunen–Loève transform (KLT)—a foundational tool in statistical signal analysis—to digital imagery for color enhancement. When multispectral satellite sensors transmit data, the red, green, and blue channels often exhibit intense physical cross-correlation, resulting in tightly clustered values where the human eye cannot discern subtle geological boundaries. The decorrelation stretch maps these compressed, correlated color distributions onto an expanded orthogonal space, cleanly isolating overlapping features.
The mathematical core of the Karhunen–Loève transform lies in optimal energy compaction. It projects multidimensional data onto an orthogonal coordinate system determined by the data’s covariance matrix, concentrating most of the variance into uncorrelated principal components. When remote-sensing instruments capture ground surfaces, the reflective spectra of mineral components are often heavily correlated across spectral bands, producing dull, low-contrast composite images. Decoupling these tightly bound signals through matrix transformation pulls subtle topographic and compositional boundaries out of the mud.
In 1996, Ronald Alley, another JPL engineer in the group who was developing applications for the Japanese ASTER instrument aboard NASA’s Terra satellite, refined the workflow for faster, more accurate calculation and documented the algorithm in an internal paper. Alley documented it out of a classic engineering ethic: someone had to write down the mechanics so users would actually understand how the tool worked. Years later, that exact paper served as the theoretical blueprint Harman discovered on Google.
Figure: Before processing: A series of faded humanoid figures painted along a rock wall. Credit: NASA / Harman
Figure: After processing: A new humanoid figure (yellow) emerges from behind the others in the Cave of San Borjitas, Baja California, Mexico. Credit: NASA / Harman
If faded rock paintings are thought of as a stack of semi-transparent sheets pressed tightly together, decorrelation stretch acts by prying apart the microscopic variances between them. Once folded pixels are forcefully unpacked across color space, the rock substrate, faded pigments, and mineral patinas are driven toward distinct color extremes, bringing forgotten artwork back into the light.
Uncovering 200 Lost Murals at Angkor Wat
Building on Alley’s work, Harman developed an image-processing plugin called Dstretch (initially built for ImageJ, the open-source software maintained by the NIH). He soon realized that color space selection was critical: different rock formations and pigment compositions responded dramatically depending on the coordinate representation. Standard RGB color space often failed to provide sufficient separation because naturally weathered pigments have RGB values nearly indistinguishable from their host rock. Harman integrated alternative color representations into Dstretch, such as YCbCr and LAB, calibrating dedicated transformation matrices to target the absorption spectra of common mineral pigments.
These purpose-built color spaces allowed field archaeologists with no background in computer science to strip away millennia of environmental weathering with a single click. The tool quickly demonstrated remarkable versatility worldwide. Across the temple complex of Angkor Wat in Cambodia, researcher Noel Hidalgo Tan used Dstretch to reveal more than 200 previously invisible paintings high on the central towers and surrounding structures. That milestone of 200 recovered artworks instantly proved that industrial-grade digital image processing could achieve what generations of flashlights and visual surveys never could.
In Egypt’s Beni Hassan tombs, the algorithm exposed faint paintings of bats and pigs—creatures seldom depicted in ancient Egyptian funerary art. At Writing-on-Stone Provincial Park in Alberta, Canada, Dstretch brought out a vivid petroglyph of a horse and rider, likely a calling-card taunt carved by a Crow warrior for Blackfoot rivals.
At the Årsand 1 site in western Norway, researchers not only identified 15 previously unrecorded figures beneath rock overhangs but also surfaced fine decorative details across 28 known motifs. Combined, 15 new images and 28 clarified patterns forced stone walls to surrender historical records sealed away for millennia.
Pixels Outlive Stone
The widespread adoption of Dstretch has spearheaded a quiet revolution in archaeology. Researchers no longer need to physically touch or sample fragile rock art surfaces; all that is required is a standard camera and a laptop running the plugin. This non-invasive digital data extraction ensures that open-air rock art, increasingly threatened by extreme climate shifts and human encroachment, can be permanently cataloged and preserved in digital form. Documentation work that once demanded months of laborious manual tracing and speculative reconstruction is now accomplished with a few clicks. Rock art research globally has received an unprecedented technological reset.
Harman once noted that, in many cases, modern observers still have little idea why ancient people drew these enigmatic symbols. The decorrelation stretch algorithm does not claim to decode meaning—its sole mandate is to recover the lost signal, leaving interpretation to human scholars.
NASA originally developed these algorithms in 1978 to peer through the atmospheric haze and iron dust of a planet hundreds of millions of kilometers away. Yet mathematical transformations treat all pixels with impartial neutrality. By forcing tightly clustered colors apart, long-invisible human figures and flying bats step forward from the stone on their own.
References:
- NASA Technology Transfer & Spinoffs
- Gizmodo Reporting
- Hacker News Discussion
