salmanoff/docs/lcamera-dev-lib.md

liblcameraDev: libcamera device provider library

Overview

liblcameraDev.so (commonLibs/lcameraDev/) is a Salmanoff common library that wraps libcamera and exposes a small, dlopen-able C API for acquiring refcounted camera device handles. It is the camera analogue of libxcbXorg.so and liblivoxProto1.so: it owns transport and device lifecycle, not stim-buffer semantics.

The eventual lcameraBuff stim-buff-api (stimBuffApis/lcameraBuff/) loads this library dynamically (via SMO search paths), resolves cameras from DAP deviceSelector strings, and shares one capture session per physical camera across multiple DAP attachments (e.g. separate H/S/V channel lines).

+edev|cam0|colour-hsv-h()|lcameraBuff()|lcameraDev()|lcamera-id:...||
+edev|cam0|colour-hsv-s()|lcameraBuff()|lcameraDev()|lcamera-id:...||
+edev|cam0|postrin(...)|negtrin(...)|colour-hsv-v()|lcameraBuff()|lcameraDev()|lcamera-id:...||

The dev-identifier (cam0 above) is the same on all three lines: one logical camera device, three stim-feature attachments (H, S, V). The deviceSelector (provider column) is also typically identical across those lines so they resolve to the same physical camera and share one capture session.

Layering:

lcameraBuff (stimBuffApi)     SMO-facing producers, channel fan-out, intrins
    └─ dlopen ─> lcameraDev (commonLib)   libcamera manager, device refcounting
                    └─ libcamera0.2 + libcamera-ipa (system packages)

Channel splitting, colourspace conversion, threshold masks, and stencils belong in a separate shared raster library (rasterStimulus, future) and in lcameraBuff; lcameraDev stops at selector resolution, CameraManager lifecycle, and a refcounted, acquired libcamera::Camera handle per resolved device. Session mode negotiation (width, height, colour-space, fully-planar YUV requirement) happens in lcameraDev before capture starts. Frame buffers, request queues, and capture timing belong in lcameraBuff (and supporting libraries), not here.

Why libcamera (for now)

Initial development targets libcamera rather than raw V4L2 so that complex Linux camera pipelines (Media Controller graphs, platform ISPs, Raspberry Pi CSI modules) are handled by libcamera’s pipeline handlers. Plain USB UVC webcams are also supported via libcamera’s uvcvideo pipeline on many distributions.

This is a Linux-only path. It does not provide cross-Unix portability by itself; a future v4l2Linux provider remains the fallback for generic /dev/videoN access without libcamera.

Runtime packages (Debian/Ubuntu naming):

Package	Purpose
libcamera0.2	Core libcamera.so, pipeline handlers, CameraManager
libcamera-ipa	IPA plugin modules (required on RPi, Rockchip, IPU3, etc.)
libcamera-dev	Headers and pkg-config (build only)

libcamera-v4l2 (v4l2-compat.so) is not required for native libcamera clients; it exists only to wrap legacy V4L2 applications via LD_PRELOAD.

DAP roles

General line shape (see also docs/deviceAttachmentPipelineSpec.md):

sensor-type|dev-identifier|postrin-spec|negtrin-spec|quale-iface-api|stim-buff-api|provider|deviceSelector

For camera colour channels:

DAP segment	Value	Role
dev-identifier	e.g. cam0	User-defined name for this device instance; same across H/S/V lines
quale-iface-api	colour-hsv-h(), colour-hsv-s(), colour-hsv-v()	Names the stim feature / greyscale channel plane
stim-buff-api	lcameraBuff()	Salmanoff producer plugin; maps qualeIface → channel
provider	lcameraDev()	Selects the libcamera-backed device access path
deviceSelector	see below	Identifies which physical camera to open (often repeated on each line)

Intrinsic thresholds on the brightness/value channel live in postrin(...) / negtrin(...) segments on the colour-hsv-v() line only, following docs/design/intrin-thresholds.md.

Example:

+edev|cam0|colour-hsv-h()|lcameraBuff()|lcameraDev()|lcamera-id:/base/soc/i2c@.../imx219@10||
+edev|cam0|colour-hsv-s()|lcameraBuff()|lcameraDev()|lcamera-id:/base/soc/i2c@.../imx219@10||
+edev|cam0|negtrin(interest-thr=40|distraction-thr=70|intolerable-thr=90)|postrin(interest-thr=30|distraction-thr=50|stupefaction-thr=80)|colour-hsv-v()|lcameraBuff()|lcameraDev()|lcamera-id:/base/soc/i2c@.../imx219@10||

Compound selector (multiple criteria on one line):

+edev|cam0|colour-hsv-h()|lcameraBuff()|lcameraDev()|lcamera-id:foo bar baz;model:aaaa||

All three lines share one underlying lcameraDev capture session keyed by the resolved libcamera camera ID. lcameraBuff must not open the same camera twice.

lcameraDev() provider params

Reserved for future options. Empty parentheses are valid:

lcameraDev()

Possible future params (not implemented in v1):

• ipa-dir=... — override IPA module search path
• log-level=... — map to LIBCAMERA_LOG_LEVELS for this provider instance

lcameraBuff() stim-buff-api params

Reserved for per-producer options (frame rate caps, pixel format preference). Channel selection is driven by the qualeIface name, not by stim-buff-api params.

Possible future params:

• fps-hz=30
• width=640|height=480
• pixfmt=NV12 — negotiation hint for lcameraBuff stream setup (not lcameraDev)

Device selector format

libcamera’s primary stable identifier is Camera::id(): an opaque string, unique per camera on the system, stable across reboot when physical topology is unchanged. libcamera properties such as Model are descriptive but not guaranteed unique; there is no standard SerialNumber property in the core property set.

Single criterion

lcameraDev resolves each criterion using a typed prefix:value form. If the selector contains no ; and no prefix: delimiter, the entire string is treated as a literal lcamera-id: value (opaque libcamera ID).

Prefix	Example	Match rule
lcamera-id:	lcamera-id:/base/soc/i2c@1/imx219@10	Exact Camera::id()
(bare string)	/base/soc/i2c@1/imx219@10	Same as lcamera-id:
index:	index:0	Nth camera from CameraManager::cameras()
model:	model:imx219	Exact match on properties::Model
model-substr:	model-substr:Logitech	Substring match on properties::Model
location:	location:external	front, back, or external

The value after : runs to the next ; or end of string. Values may contain spaces (e.g. lcamera-id:foo bar baz). Leading/trailing whitespace on each value is stripped after parsing.

Compound selectors (semicolon-separated)

A deviceSelector may contain multiple criteria separated by ;. Every criterion must match the same enumerated camera (AND semantics). This lets operators combine a stable libcamera ID with a sanity-check on model name, or narrow a fuzzy selector with an extra constraint.

lcamera-id:foo bar baz;model:aaaa
location:external;model-substr:Logitech
index:0;model:imx219

Parsing rules:

1. Split the selector on unescaped ; into one or more clauses (a single clause with no ; is a compound selector of length 1).
2. Parse each clause as prefix:value (or bare opaque ID if no : is present in that clause only).
3. A camera is a candidate only if it satisfies all clauses.
4. Zero candidates → attach failure with diagnostics.
5. More than one candidate → attach failure (ambiguous); never pick arbitrarily.
6. Exactly one candidate → resolve session by that camera’s Camera::id().

If a value must contain a literal ;, escape it as \; in the selector string (same backslash-discard convention as whitespace escaping elsewhere in DAP tokens).

Resolution algorithm (summary):

1. Start CameraManager in lcameraDev_main (once per process).
2. Enumerate cameras; build an identity record per camera (id, model, location, systemDevices).
3. Parse deviceSelector into criterion clauses; apply all clauses (AND).
4. Zero matches → attach failure with diagnostic listing known cameras.
5. Multiple matches → attach failure (ambiguous selector); never pick arbitrarily.
6. Return or create a refcounted LcameraDevice for the winning Camera::id().

Discovery helper: lcameraDev should expose a list/print entry point (used by lcameraBuff verbose mode or a small CLI in libcamera-tools style) that prints each camera’s id(), model, and location so operators can copy a stable lcamera-id: into DAP specs after first install.

Shared session and refcounting

Unlike X11 windows, a camera has no “sub-objects” inside its feed — the selector always designates the whole physical camera. Multiple DAP lines for H, S, and V share one device session (acquired libcamera::Camera), all under the same dev-identifier (e.g. cam0).

dev-identifier cam0 + deviceSelector (resolved) ──> LcameraDeviceSession (refcounted)
                                  │
                    ┌─────────────┼─────────────┐
                    ▼             ▼             ▼
              lcameraBuff     lcameraBuff     lcameraBuff
              (colour-hsv-h)  (colour-hsv-s)  (colour-hsv-v + intrins)

Rules:

• First getOrCreate for a resolved camera ID: enumerate, acquire() the libcamera::Camera, create the session entry.
• Subsequent getOrCreate for the same ID: increment refcount; return the same session handle.
• Last release: release() the libcamera camera and erase the session entry.
• Different deviceSelector strings that resolve to the same libcamera ID share one session (even if the selector text differs, e.g. one line uses lcamera-id:... and another uses a compound selector that resolves to the same camera).
• Attachments with the same dev-identifier and equivalent resolved camera are the same physical device from SMO’s perspective; channel differences come from the qualeIface name on each DAP line.

Session mode configuration (Stage 1)

Before lcameraBuff starts capture, each producer calls lcameraDev_configureSessionModeCReq on the shared CameraSession with the requested mode:

Field	Role
width / height	Requested capture dimensions (non-zero)
colourSpace	Semantic colour model (Yuv in v1)
fullPlanarIsOptional	Default false — must select fully planar YUV

lcameraDev chooses a concrete libcamera pixel format (e.g. YUV420) from the camera’s supported formats. DAPS lines name the semantic colour-space, not a raw fourcc.

Rules:

• Configure only while the session exists and capture has not started.
• Identical configure request on an already-configured session is a no-op (multiple lcameraBuff producers sharing the same device selector each call get-or-create then configure with the same mode).
• Different configure request on an already-configured session throws (conflicting mode requests on the same physical camera).
• fullPlanarIsOptional == false (default): must select a fully planar YUV format or throw with candidate-format diagnostics.
• fullPlanarIsOptional == true: rejected at the configure API until lcameraBuff implements non-planar producer deinterleaving (Stage 2). The policy helper still accepts optional planar selection for future use.

Streaming, frame delivery, and per-frame colourspace work remain out of scope for lcameraDev; lcameraBuff uses the configured session to allocate buffers and start capture on attach.

dlopen API

Exported from liblcameraDev.so using extern "C" symbols (mirroring livoxProto1). lcameraBuff loads the library with dlopen + dlsym and calls through function pointers. Hot-path operations are *CReq coroutine invokers (sscl::co::ViralNonPostingInvoker); main / exit remain synchronous.

Header: commonLibs/lcameraDev/lcameraDev.h.

Lifecycle

typedef void lcameraDev_mainFn(
    const std::shared_ptr<sscl::ComponentThread>& componentThread);

typedef void lcameraDev_exitFn(void);

lcameraDev_main records the Body ComponentThread, starts libcamera's CameraManager (idempotent), and must succeed before any *CReq runs.

Device acquisition

struct LcameraDevGetOrCreateResult
{
    std::shared_ptr<CameraSession> deviceSession;
    CameraIdentityRecord resolvedIdentity;
};

typedef sscl::co::ViralNonPostingInvoker<LcameraDevGetOrCreateResult>
    lcameraDev_getOrCreateDeviceCReqFn(const std::string& deviceSelector);

typedef sscl::co::ViralNonPostingInvoker<void>
    lcameraDev_releaseDeviceCReqFn(
        const std::shared_ptr<CameraSession>& deviceSession);

Failures throw std::exception. lcameraBuff holds the returned shared_ptr<CameraSession> and passes it back to releaseDeviceCReq. The session wraps the acquired libcamera::Camera; higher layers configure the mode then stream from that handle.

Session mode configuration

enum class LcameraDevColourSpace { Yuv };

struct LcameraDevCameraModeRequest
{
    unsigned width = 0;
    unsigned height = 0;
    LcameraDevColourSpace colourSpace = LcameraDevColourSpace::Yuv;
    bool fullPlanarIsOptional = false;
};

struct LcameraDevConfiguredCameraMode
{
    unsigned width;
    unsigned height;
    LcameraDevColourSpace colourSpace;
    std::string pixelFormatName;
    bool isFullyPlanar;
    unsigned planeCount;
};

typedef sscl::co::ViralNonPostingInvoker<LcameraDevConfiguredCameraMode>
    lcameraDev_configureSessionModeCReqFn(
        const std::shared_ptr<CameraSession>& deviceSession,
        const LcameraDevCameraModeRequest& request);

lcameraDev_configureSessionModeCReq delegates to CameraSession::configureSessionModeCReq, which runs libcamera generateConfiguration + configure and stores the result on the session. Identical reconfigure is a no-op; conflicting reconfigure throws.

Enumeration (discovery)

struct LcameraDevCameraInfo
{
    std::string id;
    std::string model;
    std::string location;
};

typedef sscl::co::ViralNonPostingInvoker<std::vector<LcameraDevCameraInfo>>
    lcameraDev_enumerateCamerasCReqFn(void);

Manual verification tools

When built with -DENABLE_LIB_lcameraDev=ON:

• lcameraDev_list_cameras — runs enumerateCamerasCReq on a minimal probe ComponentThread.
• lcameraDev_probe <deviceSelector> — getOrCreateDeviceCReq, then releaseDeviceCReq (selector and session attach/detach only).
• lcameraDev_configure_probe <deviceSelector> <width> <height> [options] — getOrCreateDeviceCReq, configureSessionModeCReq, print resolved mode (or exception), then releaseDeviceCReq. Options: --colour-space=yuv, --opt-planar / --full-planar-is-optional, --reconfigure-twice.

Module layout

commonLibs/lcameraDev/
    CMakeLists.txt
    lcameraDev.h / lcameraDev.cpp       Public C API and dlopen exports
    cameraManagerState.h / .cpp         CameraManager singleton, session map
    cameraSession.h / .cpp              Refcounted acquired-camera session
    cameraIdentity.h / .cpp             Discovery identity records
    selectorParse.h / .cpp              Compound selector parsing
    selectorResolve.h / .cpp            AND-match resolution
    tools/                              lcameraDev_list_cameras, lcameraDev_probe,
                                        lcameraDev_configure_probe

Build links against libcamera (pkg-config). Does not link Salmanoff smocore; stays a standalone .so loadable by lcameraBuff and test tools.

Relationship to future work

Component	Responsibility
lcameraDev	libcamera lifecycle, selector resolution, refcounted acquired camera session
lcameraBuff	Stream setup, frames, StimBuffApiDesc, channel fan-out, intrins
rasterStimulus (future)	YUV↔HSV, plane extraction, threshold masks, stencil geometry
xcbWindow / waylandWindow	Separate capture path; reuse rasterStimulus only

If libcamera IDs prove insufficient in practice, selector policies can gain by-id: / serial: fallbacks that scan udev/USB metadata without changing the lcameraBuff DAP surface — only lcameraDev selector resolution changes.

Open questions

1. Hot-unplug — on camera removal, fail all attached lcameraBuff producers and drop the session, or attempt re-enumeration by stored lcamera-id:?
2. IPA packaging — document per-platform apt install requirements in the main README when lcameraBuff lands (RPi needs libcamera-ipa).

Stream format, frame timing, and libcamera callback threading are owned by lcameraBuff, not lcameraDev.