Practical guide

Decode EVT3 event-camera streams

The EVT3 example turns packed sensor words into compact, ordered events using the same compute shaders on CPU and GPU. It demonstrates a stateful stream with parallel batch processing: each batch carries the decoder state into the next.

Build the project, then run the independent reference checks:

# Run this example on the CPU.
./build/example_evt3
# Run this example on the requested headless GPU profile.
./build/example_evt3 --gpu es31

Recognize the input format

EVT3 uses 16-bit words with a four-bit type and twelve-bit payload. Coordinates, polarity and time are transmitted as state changes. A direct X word emits one event. Vector words emit selected positions relative to the current X base. Y contains an eleven-bit coordinate and a camera-system bit. Time is assembled from twelve-bit high and low words, with microsecond units. Sensor configuration determines byte order. See the primary Prophesee EVT3 format and OpenEB standalone decoder.

The application supplies uint16_t words converted from incoming bytes using the stream's configured byte order. The helper packs two words per uint: the earlier word occupies bits 0–15, and the next occupies bits 16–31. An odd final word is valid. The application handles file headers, transport acquisition and USB framing before loading a batch.

Connect the decoder

The reusable wiring is in evt3_support.hpp. Attach these native modules to the executable that uses it:

shdr_target_shaders(app PRIVATE SOURCES # Compile and embed every stage used by this decoder.
    examples/shaders/evt3_summary.cpp # Summarize each block's state changes and event counts.
    examples/shaders/evt3_seed.cpp # Carry decoding state between ordered blocks.
    examples/shaders/rle_offsets.cpp # Compute compact event output offsets.
    examples/shaders/evt3_emit.cpp # Expand encoded words into ordered event records.
    examples/shaders/evt3_reset.cpp # Clear persistent state for independent recordings.
    examples/shaders/evt3_flow.cpp) # Add the optional downstream normal-flow stage.

Include the helper and keep one decoder per independent stream:

#include "examples/evt3_support.hpp" // Use the example decoder wrapper and its typed resources.

shdr_examples::evt3_decoder decoder(4096); // Maximum words per batch.
std::vector<std::uint16_t> words = {0x8001, 0x6002, 0x0007, 0x2803}; // Provide a short valid EVT3 word sequence.
decoder.load(words); // Pack the words and resize the decoder's resources.
decoder.stream_pipeline().run();          // Or run(gpu) with a headless_device.

const unsigned count = decoder.event_count(); // Read the completed compact event count.
const shdr::buffer<shdr::glsl::uvec4>& result = decoder.events; // Read events through a const buffer view.
const shdr::glsl::uvec4* events = result.data(); // Obtain the completed host-visible event array.
// events[0] is X=3, Y=7, polarity=1, timestamp=4098 microseconds.

Call stream_pipeline() for consecutive batches. It retains Y, vector base, polarity, time and wrap state. Use reset_pipeline() for independent inputs. load updates the batch and dispatch sizes while preserving stream state. Empty batches produce zero events and preserve state.

The helper reserves worst-case storage for twelve events per input word. Its capacity is fixed at construction. Reject or split larger incoming batches. The optional block-size argument defaults to 128 words and accepts 1–4096. event_count() reads the compact length from the offset buffer. On a GPU this is a host synchronization point, as is downloading the event buffer. Downstream GPU stages can instead consume the output buffer and its offsets directly.

Understand the output

Each output event occupies one uvec4, with a 16-byte buffer stride:

Reconstruct time with (uint64_t(event.w) << 32) | event.z. All eleven Y bits are retained. Events follow input order. Set vector bits emit from low to high X offset. Trigger events remain interleaved at their input position.

Persistent state uses two uvec4 elements:

Process compact CD events on the CPU

A host consumer can convert decoded contrast-detection events to one uint64_t per event:

auto compact = shdr_examples::evt3_compact_cd_to64( // Convert CD records into the compact CPU event representation.
    decoder.events, decoder.event_count()); // Convert only the decoded events in this batch.

The helper skips trigger records and omits the camera-system bit. It validates X < 2048, Y < 1024 and timestamp < 2^42, throwing on a value that would be truncated. It also validates the supplied event count. For larger coordinates, longer timestamps or mixed-camera processing, retain the original uvec4 representation. A GPU-produced buffer is downloaded automatically for this CPU consumer.

Link an event optical-flow stage

evt3_flow.cpp fits a local plane to event timestamps. Append it to the decode pipeline and keep the event count in GPU storage:

shdr::buffer<shdr::glsl::vec4> flow(decoder.events.size()); // Reserve a flow result for every possible decoder output.
auto motion = shdr::make_kernel("evt3_flow_kernel"); // Create the embedded temporal-plane flow kernel.
motion->bind("events", decoder.events) // Connect ordered decoded events to the flow stage.
 .bind("offsets", decoder.offsets) // Connect the decoder's compact output offsets.
 .bind("flow", flow) // Write one normal-flow result per event.
 .set("blocks", decoder.blocks()) // Tell the shader how many offset blocks describe this batch.
 .set("radius", 3.f) // Search within a three-pixel spatial radius.
 .set("history", 128u) // Inspect at most 128 earlier events.
 .set("time_window", 10000u); // Restrict neighbor timestamps to a 10,000-microsecond window.
auto pipeline = decoder.reset_pipeline(); // Start with reset and the four decoding stages.
pipeline += shdr::dispatch(*motion, shdr::extent( // Append flow using the backend-resident event count.
 unsigned((flow.size() + 63u) / 64u))); // Dispatch enough groups for worst-case event capacity.
pipeline.run(); // Or pipeline.run(gpu).

The dispatch covers allocated capacity. The shader reads offsets[blocks] to select valid output indices, keeping the compact event count on the backend between decoding and flow. For consecutive batches, use stream_pipeline() and update the flow kernel's blocks uniform when loading a different block count.

For each CD event, the shader selects preceding CD events with matching polarity and camera-system bit, within the spatial radius and time window. It subtracts the low/high timestamp pair with an exact integer borrow before converting the bounded difference to float. The fit includes the center event and requires at least six neighbors, a nonsingular spatial fit and a nonzero time gradient.

The result is vec4(vx, vy, neighbor_count, valid). Velocities use pixels per second, and valid is zero or one. Failed fits and trigger records produce zero. Neighbor count measures sample support. The temporal-plane gradient determines normal flow, the velocity component constrained by that local plane. Estimating full two-dimensional motion requires additional spatial or motion information.

The flow neighborhood contains earlier events in the current batch. Decoder state persists across batches. Flow cost is bounded by event_count * min(history, 128) neighborhood checks. Use separate input and output buffers and choose parameters for the intended event density and motion.

The EVT3 example checks all flow outputs against an independent double-precision plane solver. A synthetic plane t = 100*x + 50*y microseconds has expected normal velocity (8000, 4000) pixels/second. Fixtures include timestamps crossing the low 32-bit boundary.

How four stages preserve stream order

1. Summary: independent invocations scan blocks of input words. Each writes its final state changes, time-wrap summary and event counts.
2. Seed: one invocation scans the smaller summary array, computes every block's starting state and output count, and updates persistent stream state.
3. Offsets: one invocation computes exclusive output offsets from those counts, using the same prefix module as tiled RLE compression.
4. Emit: blocks decode concurrently into disjoint compact output intervals. They visit only set vector bits and write events directly in stream order.

Resetting an independent batch adds a fifth stage. The two short serial scans process the block table, while summaries and event expansion run across blocks. Ordered dispatches and independent invocations express the sequential format state. This implementation processes complete input batches through compute dispatches.

Format policies and limits

The first high-time word establishes the output time base. Earlier state updates are retained. Each high-time word resets low time to zero, including a repeated high value. A high-time decrease of at least 4085 increments the wrap counter, matching OpenEB's large-jump tolerance. Smaller decreases update high time and preserve the wrap counter. Low-time values follow the encoded update sequence.

The 32-bit wrap counter extends the 24-bit time base to 56 bits and cycles after 2^32 time-base wraps. Vector X arithmetic is modulo 2048. VECT_8 uses payload bits 0–7. The decoder skips reserved types and OTHERS/CONTINUED extension records. Validate sensor geometry, input framing and recovery policy in the application before decoding.

Decode consecutive batches of a stream in order. Give concurrent streams and independently queued jobs separate decoder objects, persistent state and writable output storage. See asynchronous execution for background submissions.

The example checks direct and vector events, trigger ordering, camera bits, initial synchronization, repeated time updates, timestamp wraps, odd and empty batches, arbitrary block cuts and state carried across batch boundaries against an independent scalar interpreter.