Part 14 — Conformance TestingLink to the section Part 14 — Conformance Testing
On this page
Maturity: Draft · Profiles: all
14.1 PrinciplesLink to the section 14.1 Principles
- Conformance is claimed per target (Device or Client) and per profile.
- Every MUST in Parts 03–13 maps to at least one test case; test identifiers follow
T-<part>-<n>(for exampleT-07-04). - Tests are executed by an open-source UNIVIP Conformance Tool (to be built, roadmap Phase 3) that acts as a client when testing a device and as a device when testing a client.
- Mandatory alternatives (MQTT, SRT, RTSP, H.265) have their own optional test groups; passing them permits the additional claim "with MQTT real-time binding", etc.
- Results are published as a signed JSON report; a claim without a report is not a conformance claim.
14.2 Test groups (Device)Link to the section 14.2 Test groups (Device)
| Group | Covers | Profile |
|---|---|---|
| D-ID | Identity members, device_id stability across reconnect |
S |
| D-SEC | TLS 1.3 only, no plaintext, authentication methods, scope enforcement, replay rejection | S |
| D-SESSION | hello/welcome/capabilities ordering, ping/pong, reconnection with backoff, event replay after disconnect | S |
| D-CAP | Capability document validity, consistency with observed behavior (every advertised topic/command works; no unadvertised ones accepted) | S/C/N |
| D-TEL | Schema validity, rates, sequence continuity, units/frames sanity (altitude reference present, heading range), stale handling | S |
| D-EVT | Mandatory events fire under induced conditions (simulated low battery, link loss, state changes), envelope validity, stateful clear | S |
| D-MEDIA | WebRTC negotiation, H.264 profile/level, RTCP SR cadence, telemetry-to-frame alignment ≤ 100 ms using a test pattern with embedded timecode | S |
| D-CAM | Each advertised camera.* command: accept, lifecycle, limits rejection, continuous-move expiry, pilot override |
C |
| D-LEASE | Acquire/heartbeat/extend/release/expire/revoke; single vehicle lease; camera lease per payload | C/N |
| D-AIR | Each advertised vehicle.* command in a simulator or test range: preconditions, rejection reasons, hold within 1 s, lease expiry → hold, failsafe precedence, audit record completeness |
N |
| D-GEO | Footprint and target schema validity; footprint geometric sanity against known camera pose | optional |
| D-MQTT, D-SRT, D-RTSP, D-H265 | Alternative bindings | optional |
14.3 Test groups (Client)Link to the section 14.3 Test groups (Client)
| Group | Covers | Profile |
|---|---|---|
| C-SESSION | Accepts device sessions, validates auth, handles capabilities updates, deduplicates events | S |
| C-RENDER | Renders mandatory stream; displays position with altitude reference; alignment ≤ 100 ms measured via timecode pattern | S |
| C-RECORD | Recording retains telemetry, events, capabilities; replay reproduces flight path; export of ST 0601 if claimed | S |
| C-CAM | Offers only advertised controls; maps PTZ gestures to camera.* only; handles rejections visibly |
C |
| C-AIR | Lease acquisition before controls; lease state/expiry visible; ACCEPTED not shown as done; failsafe events surfaced immediately; separate permission for vehicle vs camera |
N |
| C-SEC | TLS 1.3, scope separation, token lifetimes, credential revocation | S |
14.4 Test environmentLink to the section 14.4 Test environment
- Device tests for Profile N MUST be run against a simulator (for example PX4 SITL via the reference gateway) and at least once against real hardware in a controlled environment under an operator's authority before a Profile N claim is published.
- The timecode pattern for alignment tests is a visible frame counter and a machine-readable timecode strip injected on the video source, so that the client's displayed telemetry time can be compared to the decoded frame's time. The strip encodes 72 bits, most significant first: the capture time on the device clock (48 bits, milliseconds since the Unix epoch), a frame counter (16 bits, modulo 2¹⁶) and a check byte (8 bits: the low byte of the sum of the 8 bytes of the 64-bit big-endian value
time << 16 | counter, XOR0x5A). The bits are drawn as square cells of sidec = max(4, ⌊width / 80⌋)pixels, white for 1 on a black background, in 2 rows of 36 cells, row by row; the strip's top-left corner is at (width − 37c,height − 3c). A reader samples the luma at each cell's centre (threshold 50 %) and discards a frame whose check byte does not match. The pattern survives H.264 compression at ordinary bitrates and needs no QR library; the reference gateway's synthetic camera draws it (gateway/univip_gateway/media.py).
14.5 Claim rules (Draft)Link to the section 14.5 Claim rules (Draft)
- Claims use the exact form
UNIVIP <version> <Target> Profile <S|C|N>[, <bindings>]. - A claim is valid for the tested firmware/software version; a MAJOR firmware change requires re-test.
- Misrepresentation of a claim is grounds for removal from the public conformance list.
14.6 Reference implementations and self-certificationLink to the section 14.6 Reference implementations and self-certification
Until the Conformance Tool exists, implementers MAY publish self-declared interoperability reports against the reference gateway and reference client, using the example messages in ../examples/ and the validator in ../tools/validate.py as the minimum bar. Such reports MUST NOT use the word "conformant".