salmanoff/docs/lcamera-dev-lib.md

liblcameraDev: libcamera device provider library

Overview

liblcameraDev.so (commonLibs/lcameraDev/) is a Salmanoff common library that wraps libcamera and exposes a small, dlopen-able C API for acquiring refcounted camera device handles. It is the camera analogue of libxcbXorg.so and liblivoxProto1.so: it owns transport and device lifecycle, not stim-buffer semantics.

The eventual lcameraBuff stim-buff-api (stimBuffApis/lcameraBuff/) loads this library dynamically (via SMO search paths), resolves cameras from DAP deviceSelector strings, and shares one capture session per physical camera across multiple DAP attachments (e.g. separate H/S/V channel lines).

+edev|cam0|colour-hsv-h()|lcameraBuff()|lcameraDev()|lcamera-id:...||
+edev|cam0|colour-hsv-s()|lcameraBuff()|lcameraDev()|lcamera-id:...||
+edev|cam0|postrin(...)|negtrin(...)|colour-hsv-v()|lcameraBuff()|lcameraDev()|lcamera-id:...||

The dev-identifier (cam0 above) is the same on all three lines: one logical camera device, three stim-feature attachments (H, S, V). The deviceSelector (provider column) is also typically identical across those lines so they resolve to the same physical camera and share one capture session.

Layering:

lcameraBuff (stimBuffApi)     SMO-facing producers, channel fan-out, intrins
    └─ dlopen ─> lcameraDev (commonLib)   libcamera manager, device refcounting
                    └─ libcamera0.2 + libcamera-ipa (system packages)

Channel splitting, colourspace conversion, threshold masks, and stencils belong in a separate shared raster library (rasterStimulus, future) and in lcameraBuff; lcameraDev stops at “here is a stable, acquired libcamera Camera you can configure and stream from.”

Why libcamera (for now)

Initial development targets libcamera rather than raw V4L2 so that complex Linux camera pipelines (Media Controller graphs, platform ISPs, Raspberry Pi CSI modules) are handled by libcamera’s pipeline handlers. Plain USB UVC webcams are also supported via libcamera’s uvcvideo pipeline on many distributions.

This is a Linux-only path. It does not provide cross-Unix portability by itself; a future v4l2Linux provider remains the fallback for generic /dev/videoN access without libcamera.

Runtime packages (Debian/Ubuntu naming):

Package	Purpose
libcamera0.2	Core libcamera.so, pipeline handlers, CameraManager
libcamera-ipa	IPA plugin modules (required on RPi, Rockchip, IPU3, etc.)
libcamera-dev	Headers and pkg-config (build only)

libcamera-v4l2 (v4l2-compat.so) is not required for native libcamera clients; it exists only to wrap legacy V4L2 applications via LD_PRELOAD.

DAP roles

General line shape (see also docs/deviceAttachmentPipelineSpec.md):

sensor-type|dev-identifier|postrin-spec|negtrin-spec|quale-iface-api|stim-buff-api|provider|deviceSelector

For camera colour channels:

DAP segment	Value	Role
dev-identifier	e.g. cam0	User-defined name for this device instance; same across H/S/V lines
quale-iface-api	colour-hsv-h(), colour-hsv-s(), colour-hsv-v()	Names the stim feature / greyscale channel plane
stim-buff-api	lcameraBuff()	Salmanoff producer plugin; maps qualeIface → channel
provider	lcameraDev()	Selects the libcamera-backed device access path
deviceSelector	see below	Identifies which physical camera to open (often repeated on each line)

Intrinsic thresholds on the brightness/value channel live in postrin(...) / negtrin(...) segments on the colour-hsv-v() line only, following docs/design/intrin-thresholds.md.

Example:

+edev|cam0|colour-hsv-h()|lcameraBuff()|lcameraDev()|lcamera-id:/base/soc/i2c@.../imx219@10||
+edev|cam0|colour-hsv-s()|lcameraBuff()|lcameraDev()|lcamera-id:/base/soc/i2c@.../imx219@10||
+edev|cam0|negtrin(interest-thr=40|distraction-thr=70|intolerable-thr=90)|postrin(interest-thr=30|distraction-thr=50|stupefaction-thr=80)|colour-hsv-v()|lcameraBuff()|lcameraDev()|lcamera-id:/base/soc/i2c@.../imx219@10||

Compound selector (multiple criteria on one line):

+edev|cam0|colour-hsv-h()|lcameraBuff()|lcameraDev()|lcamera-id:foo bar baz;model:aaaa||

All three lines share one underlying lcameraDev capture session keyed by the resolved libcamera camera ID. lcameraBuff must not open the same camera twice.

lcameraDev() provider params

Reserved for future options. Empty parentheses are valid:

lcameraDev()

Possible future params (not implemented in v1):

• ipa-dir=... — override IPA module search path
• log-level=... — map to LIBCAMERA_LOG_LEVELS for this provider instance

lcameraBuff() stim-buff-api params

Reserved for per-producer options (frame rate caps, pixel format preference). Channel selection is driven by the qualeIface name, not by stim-buff-api params.

Possible future params:

• fps-hz=30
• width=640|height=480
• pixfmt=NV12 — negotiation hint passed down to lcameraDev

Device selector format

libcamera’s primary stable identifier is Camera::id(): an opaque string, unique per camera on the system, stable across reboot when physical topology is unchanged. libcamera properties such as Model are descriptive but not guaranteed unique; there is no standard SerialNumber property in the core property set.

Single criterion

lcameraDev resolves each criterion using a typed prefix:value form. If the selector contains no ; and no prefix: delimiter, the entire string is treated as a literal lcamera-id: value (opaque libcamera ID).

Prefix	Example	Match rule
lcamera-id:	lcamera-id:/base/soc/i2c@1/imx219@10	Exact Camera::id()
(bare string)	/base/soc/i2c@1/imx219@10	Same as lcamera-id:
index:	index:0	Nth camera from CameraManager::cameras()
model:	model:imx219	Exact match on properties::Model
model-substr:	model-substr:Logitech	Substring match on properties::Model
location:	location:external	front, back, or external

The value after : runs to the next ; or end of string. Values may contain spaces (e.g. lcamera-id:foo bar baz). Leading/trailing whitespace on each value is stripped after parsing.

Compound selectors (semicolon-separated)

A deviceSelector may contain multiple criteria separated by ;. Every criterion must match the same enumerated camera (AND semantics). This lets operators combine a stable libcamera ID with a sanity-check on model name, or narrow a fuzzy selector with an extra constraint.

lcamera-id:foo bar baz;model:aaaa
location:external;model-substr:Logitech
index:0;model:imx219

Parsing rules:

1. Split the selector on unescaped ; into one or more clauses (a single clause with no ; is a compound selector of length 1).
2. Parse each clause as prefix:value (or bare opaque ID if no : is present in that clause only).
3. A camera is a candidate only if it satisfies all clauses.
4. Zero candidates → attach failure with diagnostics.
5. More than one candidate → attach failure (ambiguous); never pick arbitrarily.
6. Exactly one candidate → resolve session by that camera’s Camera::id().

If a value must contain a literal ;, escape it as \; in the selector string (same backslash-discard convention as whitespace escaping elsewhere in DAP tokens).

Resolution algorithm (summary):

1. Start CameraManager (once per process, inside lcameraDev).
2. Enumerate cameras; build an identity record per camera (id, model, location, systemDevices).
3. Parse deviceSelector into criterion clauses; apply all clauses (AND).
4. Zero matches → attach failure with diagnostic listing known cameras.
5. Multiple matches → attach failure (ambiguous selector); never pick arbitrarily.
6. Return or create a refcounted LcameraDevice for the winning Camera::id().

Discovery helper: lcameraDev should expose a list/print entry point (used by lcameraBuff verbose mode or a small CLI in libcamera-tools style) that prints each camera’s id(), model, and location so operators can copy a stable lcamera-id: into DAP specs after first install.

Shared session and refcounting

Unlike X11 windows, a camera has no “sub-objects” inside its feed — the selector always designates the whole camera stream. Multiple DAP lines for H, S, and V are views over one capture session, all under the same dev-identifier (e.g. cam0).

dev-identifier cam0 + deviceSelector (resolved) ──> LcameraDeviceSession (refcounted)
                                  │
                    ┌─────────────┼─────────────┐
                    ▼             ▼             ▼
              lcameraBuff     lcameraBuff     lcameraBuff
              (colour-hsv-h)  (colour-hsv-s)  (colour-hsv-v + intrins)

Rules:

• First getOrCreate for a resolved camera ID: enumerate, acquire() the libcamera::Camera, negotiate and start the shared stream.
• Subsequent getOrCreate for the same ID: increment refcount; return the same session handle.
• Last release: release() the libcamera camera, stop streaming, tear down buffers.
• Different deviceSelector strings that resolve to the same libcamera ID share one session (even if the selector text differs, e.g. one line uses lcamera-id:... and another uses a compound selector that resolves to the same camera).
• Attachments with the same dev-identifier and equivalent resolved camera are the same physical device from SMO’s perspective; channel differences come from the qualeIface name on each DAP line.

dlopen API (planned)

Exported from liblcameraDev.so using extern "C" symbols (mirroring livoxProto1). lcameraBuff loads the library with dlopen + dlsym and calls through function pointers.

Lifecycle

/* Start CameraManager; idempotent per process. */
typedef void lcameraDev_mainFn(void);

/* Stop manager, release all devices; called on SMO shutdown. */
typedef void lcameraDev_exitFn(void);

Device acquisition

struct LcameraDevGetOrCreateResult
{
    bool success;
    /* Opaque session handle; valid until matching release call. */
    void* deviceSession;
    /* Resolved libcamera camera ID (stable session key). */
    const char* resolvedCameraId;
};

typedef LcameraDevGetOrCreateResult lcameraDev_getOrCreateDeviceFn(
    const char* deviceSelector);

typedef void lcameraDev_releaseDeviceFn(void* deviceSession);

deviceSession is opaque to callers. lcameraBuff passes it back when detaching; it must not call libcamera APIs directly.

Enumeration (discovery)

struct LcameraDevCameraInfo
{
    const char* id;
    const char* model;      /* empty string if unavailable */
    const char* location;   /* "front", "back", "external", or "" */
};

struct LcameraDevEnumerateResult
{
    size_t count;
    LcameraDevCameraInfo* cameras; /* caller frees via lcameraDev_freeEnumerateResult */
};

typedef LcameraDevEnumerateResult lcameraDev_enumerateCamerasFn(void);
typedef void lcameraDev_freeEnumerateResultFn(LcameraDevEnumerateResult* result);

Frame access (boundary with lcameraBuff)

Exact frame-delivery API is TBD and will likely use a callback or a “dequeue latest frame buffer” method on the session handle. lcameraDev owns libcamera FrameBuffer allocation and Request requeue; lcameraBuff copies or maps the plane it needs for the active qualeIface channel.

Principle: one negotiated stream format per session (prefer native YUV such as NV12/YUYV). lcameraBuff + rasterStimulus derive H/S/V greyscale planes from that single stream.

Module layout (planned)

commonLibs/lcameraDev/
    CMakeLists.txt
    lcameraDev.h          Public C API + C++ internal headers
    lcameraDev.cpp        dlopen exports, CameraManager singleton
    cameraSession.cpp     Refcounted session, stream negotiation
    selectorResolve.cpp   deviceSelector parsing and matching
    cameraEnumerate.cpp   Discovery / identity records

Build links against libcamera (pkg-config). Does not link Salmanoff smocore; stays a standalone .so loadable by lcameraBuff and test tools.

Relationship to future work

Component	Responsibility
lcameraDev	libcamera lifecycle, selector resolution, shared capture session
lcameraBuff	StimBuffApiDesc, StimulusProducer, per-channel ring buffers, intrins
rasterStimulus (future)	YUV↔HSV, plane extraction, threshold masks, stencil geometry
xcbWindow / waylandWindow	Separate capture path; reuse rasterStimulus only

If libcamera IDs prove insufficient in practice, selector policies can gain by-id: / serial: fallbacks that scan udev/USB metadata without changing the lcameraBuff DAP surface — only lcameraDev selector resolution changes.

Open questions

1. Stream format defaults — prefer first supported YUV format, or allow lcameraBuff(pixfmt=...) override?
2. Hot-unplug — on camera removal, fail all attached lcameraBuff producers and drop the session, or attempt re-enumeration by stored lcamera-id:?
3. Thread model — libcamera callbacks vs polling in the Body thread production daemon; align with StimulusProducer::productionCDaemon timing (CONFIG_STIMBUFF_FRAME_PERIOD_MS).
4. IPA packaging — document per-platform apt install requirements in the main README when lcameraBuff lands (RPi needs libcamera-ipa).