Practical guide
Build imaging pipelines
Image processing starts with a source buffer, a destination buffer, and one kernel. Add another destination and kernel to build a longer pipeline. The modules in examples/shaders provide ready-to-use Bayer, filtering, Fourier, corner-detection, and compression stages for this pattern.
Attach a module with shdr_target_shaders, create its filename_kernel factory and bind the named resources. Application pipelines execute the embedded module. The processing kernels use the common ES 3.1, ES 3.2 and desktop GL 4.3 compute subset.
Resource type float, uint, vec2, or vec4 in the tables means
shdr::buffer<float>, shdr::buffer<unsigned>, shdr::buffer<glsl::vec2>, or
shdr::buffer<glsl::vec4> in C++ respectively. Input and output storage must be
distinct for neighborhood filters, transforms, and compression stages.
Start with a filter
Attach examples/shaders/convolution2d.cpp to an application target. Its factory
then accepts two-channel pixels and a separate coefficient buffer:
const unsigned width=4, height=4; // Choose a four-by-four demonstration image.
shdr::buffer<glsl::vec2> pixels(width*height), filtered(width*height); // Allocate complex-valued input and output pixels.
shdr::buffer<float> coefficients = { // Create a normalized three-by-three averaging kernel.
1.f/9.f, 1.f/9.f, 1.f/9.f, // Set the coefficients for the first kernel row.
1.f/9.f, 1.f/9.f, 1.f/9.f, // Set the coefficients for the middle kernel row.
1.f/9.f, 1.f/9.f, 1.f/9.f // Set the coefficients for the final kernel row.
}; // Finish the nine-coefficient filter.
for (unsigned i=0; i<width*height; ++i) // Initialize each pixel in row-major order.
pixels[i] = glsl::vec2(float(i),0.f); // Store a real test value with a zero imaginary component.
auto filter = shdr::make_kernel("convolution2d_kernel"); // Create the embedded two-dimensional convolution kernel.
filter->bind("input_values",pixels); // Connect the source pixels.
filter->bind("output_values",filtered); // Connect the destination pixels.
filter->bind("coefficients",coefficients); // Connect the dynamic filter coefficients.
filter->set("dimensions",glsl::uvec2(width,height)); // Set the image width and height.
filter->set("kernel_size",glsl::uvec2(3u,3u)); // Set the filter width and height.
shdr::pipeline pipeline = shdr::dispatch(*filter, // Build a dispatch for the bound convolution resources.
shdr::extent((width+7)/8,(height+7)/8)); // Round image dimensions up to the shader's eight-by-eight groups.
pipeline.run(); // Run this convolution on the CPU.
Each vec2 can hold two image channels or one complex sample. Calling
pipeline.run(device) selects the GPU with the same resources and coefficients.
To add another filter, bind filtered as its input and connect its dispatch with
operator>>.
Bayer unpacking, demosaicing, and luminance
example_imaging checks all sixteen combinations of four packed formats and four Bayer phases against an independent pixel reference. Inputs use a fixed random seed. The linked path is unpack → bilinear demosaic → luminance.
| Shader factory | Input → output ports | Uniforms |
|---|---|---|
| bayer_unpack_kernel | packed_words:uint → samples:uint |
count:uint, packing:uint, bit_depth:uint |
| debayer_kernel | samples:uint → colors:vec4 |
width:uint, height:uint, pattern:uint, bit_depth:uint |
| luminance_kernel | colors:vec4 → gray:float |
count:uint |
Each packed_words element holds four consecutive bytes, least significant byte first.
Byte-addressed input is contiguous. Row stride and camera-container decoding are
the caller's responsibility. Format parameters are:
packing |
Sample storage | Group constraint |
|---|---|---|
| 0 | Bayer8: one byte per sample | Any sample count, use depth 8 |
| 1 | Bayer16: two little-endian bytes per sample | Any sample count, supported depths 1–16 |
| 2 | MIPI RAW10: four samples in five bytes | Count divisible by four, use depth 10 |
| 3 | MIPI RAW12: two samples in three bytes | Count divisible by two, use depth 12 |
pattern selects 0=RGGB, 1=BGGR, 2=GRBG, or 3=GBRG. Dimensions are positive and
count=width*height. Demosaicing keeps each sensor's measured channel and averages
nearby sites for the other channels, with explicit clamp-to-edge handling. Output
RGB is normalized by (2^bit_depth)-1 and alpha is one. Luminance uses
0.2126*R + 0.7152*G + 0.0722*B on linear RGB. Transfer-function decoding, black
level correction, and color calibration belong to upstream processing.
Unpack and luminance use (64,1,1) local groups. Demosaicing uses (8,8,1):
dispatch extent((width+7)/8,(height+7)/8).
Gaussian, Sobel, and Harris corners
example_vision builds a six-stage path: Gaussian horizontal → Gaussian vertical → Sobel → tensor → Harris response → nonmaximum suppression. A direct 25-tap reference verifies the two-pass Gaussian, coefficient-table references verify gradients and Harris values, and the final mask identifies four known corners of a bright square.
| Shader factory | Input → output element type | Additional uniforms |
|---|---|---|
| gaussian_horizontal_kernel | float → float |
width:uint, height:uint |
| gaussian_vertical_kernel | float → float |
width:uint, height:uint |
| sobel_kernel | float → vec2(dx,dy) |
width:uint, height:uint |
| harris_tensor_kernel | vec2 → vec4(dx²,dx*dy,dy²,0) |
count:uint |
| harris_response_kernel | vec4 → float |
width:uint, height:uint, harris_k:float |
| nonmax_kernel | float → uint mask |
width:uint, height:uint, threshold:float |
Every stage uses ports input_values and output_values, and local groups
(64,1,1). Gaussian passes use the normalized five-tap binomial kernel
[1,4,6,4,1]/16. Sobel derivatives are divided by eight. Harris response smooths
the symmetric structure tensor with the normalized 3×3 outer product of
[1,2,1], then computes det(M)-harris_k*trace(M)^2.
Nonmaximum suppression keeps responses strictly above the threshold and rejects a pixel when a 3×3 neighbor is greater. Equal-valued ties retain the smallest row-major pixel index. The output is zero or one per pixel. Neighborhoods use clamp-to-edge sampling, with separate immutable input buffers for each stage.
Dynamic 2D and 3D convolution
example_spectral includes two linked 2D filters and a
three-stage X/Y/Z volume filter. Coefficients and kernel sizes can be rebound
between runs. Both shaders perform true convolution with reversed coefficient
offsets: sample p + floor(kernel_size/2) - coefficient_position. Borders clamp
to the nearest input sample.
| Shader factory | Inputs → output | Uniforms and local size |
|---|---|---|
| convolution2d_kernel | input_values:vec2, coefficients:float → output_values:vec2 |
dimensions:uvec2, kernel_size:uvec2, (8,8,1) |
| convolution3d_kernel | input_values:float, coefficients:float → output_values:float |
dimensions:uvec3, kernel_size:uvec3, (4,4,4) |
Dimensions and kernel sizes must be positive. Allocate enough input, output, and
coefficient elements for their products. Index order is x fastest, then y, then
z. Two-dimensional filtering applies the same real coefficients to both vec2
channels. Those channels can also represent the real and imaginary parts of a
complex sample. A scalar image can use an imaginary channel initialized to zero.
Complex separable 2D Fourier transforms
dft_rows_kernel and
dft_columns_kernel read input_values:vec2
and write output_values:vec2, with uniforms dimensions:uvec2,
direction:float, and normalization:float. Both use (8,8,1) local groups.
vec2(real,imaginary) represents a complex number.
Use direction -1.f and normalization 1.f for both forward stages. For an
inverse transform, use direction 1.f, normalization 1.f/width for rows, and
1.f/height for columns. The example checks the forward result against an
independent double-precision complex 2D transform and checks inverse
reconstruction.
This direct DFT has cost O(width*height*(width+height)). It serves small two-dimensional transforms and portable numerical checks. The signal guide provides the radix-two FFT for longer one-dimensional blocks.
Lossless tiled run-length compression
example_imaging also checks exact reconstruction of
seeded random runs, including a partial final tile. Four independent dispatch
stages build and consume a compact stream of unsigned (value,run_length) pairs:
| Shader factory | Ports | Uniforms |
|---|---|---|
| rle_count_kernel | values:uint → sizes:uint |
count:uint, block_size:uint |
| rle_offsets_kernel | sizes:uint → offsets:uint |
blocks:uint |
| rle_encode_kernel | values:uint, offsets:uint → encoded:uint |
count:uint, block_size:uint |
| rle_decode_kernel | encoded:uint, offsets:uint → restored:uint |
count:uint, block_size:uint |
count and block_size are positive, and blocks=ceil(count/block_size). Allocate sizes
with blocks words, offsets with blocks+1 words, and encoded with worst-case
capacity 2*count words. The actual compact length is offsets[blocks] words.
Include the offset table when accounting for compressed size. The prefix stage
uses one invocation and is intended for small tile tables. Other stages use
local groups (64,1,1), one invocation per tile.
This lossless word codec compresses repeated values and can expand unrelated random values. Feed the decoder a valid stream and offset metadata produced by the matching encoder.