salmanoff/docs/lcamera-dev-lib.md

# liblcameraDev: libcamera device provider library

## Overview

`liblcameraDev.so` (`commonLibs/lcameraDev/`) is a Salmanoff common library that
wraps [libcamera](https://libcamera.org/) and exposes a small, dlopen-able C
API for acquiring refcounted camera device handles. It is the camera analogue of
`libxcbXorg.so` and `liblivoxProto1.so`: it owns transport and device lifecycle,
not stim-buffer semantics.

The eventual `lcameraBuff` stim-buff-api (`stimBuffApis/lcameraBuff/`) loads
this library dynamically (via SMO search paths), resolves cameras from DAP
`deviceSelector` strings, and shares one capture session per physical camera
across multiple DAP attachments (e.g. separate H/S/V channel lines).

```
+edev|cam0|colour-hsv-h()|lcameraBuff()|lcameraDev()|lcamera-id:...||
+edev|cam0|colour-hsv-s()|lcameraBuff()|lcameraDev()|lcamera-id:...||
+edev|cam0|postrin(...)|negtrin(...)|colour-hsv-v()|lcameraBuff()|lcameraDev()|lcamera-id:...||
```

The **`dev-identifier`** (`cam0` above) is the same on all three lines: one
logical camera device, three stim-feature attachments (H, S, V). The
**`deviceSelector`** (provider column) is also typically identical across those
lines so they resolve to the same physical camera and share one capture session.

Layering:

```text
lcameraBuff (stimBuffApi)     SMO-facing producers, channel fan-out, intrins
    └─ dlopen ─> lcameraDev (commonLib)   libcamera manager, device refcounting
                    └─ libcamera0.2 + libcamera-ipa (system packages)
```

Channel splitting, colourspace conversion, threshold masks, and stencils belong
in a separate shared raster library (`rasterStimulus`, future) and in
`lcameraBuff`; `lcameraDev` stops at selector resolution, `CameraManager`
lifecycle, and a refcounted, **acquired** `libcamera::Camera` handle per
resolved device. Session mode negotiation (width, height, colour-space,
fully-planar YUV requirement) happens in `lcameraDev` before capture starts.
Frame buffers, request queues, and capture timing belong in `lcameraBuff` (and
supporting libraries), not here.

## Why libcamera (for now)

Initial development targets libcamera rather than raw V4L2 so that complex Linux
camera pipelines (Media Controller graphs, platform ISPs, Raspberry Pi CSI
modules) are handled by libcamera’s pipeline handlers. Plain USB UVC webcams are
also supported via libcamera’s `uvcvideo` pipeline on many distributions.

This is a **Linux-only** path. It does not provide cross-Unix portability by
itself; a future `v4l2Linux` provider remains the fallback for generic
`/dev/videoN` access without libcamera.

Runtime packages (Debian/Ubuntu naming):

| Package | Purpose |
|---|---|
| `libcamera0.2` | Core `libcamera.so`, pipeline handlers, `CameraManager` |
| `libcamera-ipa` | IPA plugin modules (required on RPi, Rockchip, IPU3, etc.) |
| `libcamera-dev` | Headers and pkg-config (build only) |

`libcamera-v4l2` (`v4l2-compat.so`) is **not** required for native libcamera
clients; it exists only to wrap legacy V4L2 applications via `LD_PRELOAD`.

## DAP roles

General line shape (see also `docs/deviceAttachmentPipelineSpec.md`):

```text
sensor-type|dev-identifier|postrin-spec|negtrin-spec|quale-iface-api|stim-buff-api|provider|deviceSelector
```

For camera colour channels:

| DAP segment | Value | Role |
|---|---|---|
| `dev-identifier` | e.g. `cam0` | User-defined name for this **device instance**; same across H/S/V lines |
| `quale-iface-api` | `colour-hsv-h()`, `colour-hsv-s()`, `colour-hsv-v()` | Names the stim feature / greyscale channel plane |
| `stim-buff-api` | `lcameraBuff()` | Salmanoff producer plugin; maps qualeIface → channel |
| `provider` | `lcameraDev()` | Selects the libcamera-backed device access path |
| `deviceSelector` | see below | Identifies which physical camera to open (often repeated on each line) |

Intrinsic thresholds on the brightness/value channel live in `postrin(...)` /
`negtrin(...)` segments on the `colour-hsv-v()` line only, following
`docs/design/intrin-thresholds.md`.

Example:

```text
+edev|cam0|colour-hsv-h()|lcameraBuff()|lcameraDev()|lcamera-id:/base/soc/i2c@.../imx219@10||
+edev|cam0|colour-hsv-s()|lcameraBuff()|lcameraDev()|lcamera-id:/base/soc/i2c@.../imx219@10||
+edev|cam0|negtrin(interest-thr=40|distraction-thr=70|intolerable-thr=90)|postrin(interest-thr=30|distraction-thr=50|stupefaction-thr=80)|colour-hsv-v()|lcameraBuff()|lcameraDev()|lcamera-id:/base/soc/i2c@.../imx219@10||
```

Compound selector (multiple criteria on one line):

```text
+edev|cam0|colour-hsv-h()|lcameraBuff()|lcameraDev()|lcamera-id:foo bar baz;model:aaaa||
```

All three lines share one underlying `lcameraDev` capture session keyed by the
resolved libcamera camera ID. `lcameraBuff` must not open the same camera twice.

### `lcameraDev()` provider params

Reserved for future options. Empty parentheses are valid:

```text
lcameraDev()
```

Possible future params (not implemented in v1):

* `ipa-dir=...` — override IPA module search path
* `log-level=...` — map to `LIBCAMERA_LOG_LEVELS` for this provider instance

### `lcameraBuff()` stim-buff-api params

Reserved for per-producer options (frame rate caps, pixel format preference).
Channel selection is driven by the qualeIface name, not by stim-buff-api params.

Possible future params:

* `fps-hz=30`
* `width=640|height=480`
* `pixfmt=NV12` — negotiation hint for `lcameraBuff` stream setup (not
  `lcameraDev`)

## Device selector format

libcamera’s primary stable identifier is `Camera::id()`: an opaque string,
unique per camera on the system, stable across reboot when physical topology is
unchanged. libcamera properties such as `Model` are descriptive but **not**
guaranteed unique; there is no standard `SerialNumber` property in the core
property set.

### Single criterion

`lcameraDev` resolves each criterion using a typed `prefix:value` form. If the
selector contains no `;` and no `prefix:` delimiter, the entire string is
treated as a literal `lcamera-id:` value (opaque libcamera ID).

| Prefix | Example | Match rule |
|---|---|---|
| `lcamera-id:` | `lcamera-id:/base/soc/i2c@1/imx219@10` | Exact `Camera::id()` |
| *(bare string)* | `/base/soc/i2c@1/imx219@10` | Same as `lcamera-id:` |
| `index:` | `index:0` | Nth camera from `CameraManager::cameras()` |
| `model:` | `model:imx219` | Exact match on `properties::Model` |
| `model-substr:` | `model-substr:Logitech` | Substring match on `properties::Model` |
| `location:` | `location:external` | `front`, `back`, or `external` |

The value after `:` runs to the next `;` or end of string. Values may contain
spaces (e.g. `lcamera-id:foo bar baz`). Leading/trailing whitespace on each
value is stripped after parsing.

### Compound selectors (semicolon-separated)

A `deviceSelector` may contain **multiple criteria** separated by `;`. Every
criterion must match the **same** enumerated camera (**AND** semantics). This
lets operators combine a stable libcamera ID with a sanity-check on model name,
or narrow a fuzzy selector with an extra constraint.

```text
lcamera-id:foo bar baz;model:aaaa
location:external;model-substr:Logitech
index:0;model:imx219
```

Parsing rules:

1. Split the selector on unescaped `;` into one or more clauses (a single clause
   with no `;` is a compound selector of length 1).
2. Parse each clause as `prefix:value` (or bare opaque ID if no `:` is present
   in that clause only).
3. A camera is a candidate only if it satisfies **all** clauses.
4. **Zero candidates** → attach failure with diagnostics.
5. **More than one candidate** → attach failure (ambiguous); never pick
   arbitrarily.
6. Exactly one candidate → resolve session by that camera’s `Camera::id()`.

If a value must contain a literal `;`, escape it as `\;` in the selector string
(same backslash-discard convention as whitespace escaping elsewhere in DAP
tokens).

Resolution algorithm (summary):

1. Start `CameraManager` in `lcameraDev_main` (once per process).
2. Enumerate cameras; build an identity record per camera (`id`, `model`,
   `location`, `systemDevices`).
3. Parse `deviceSelector` into criterion clauses; apply **all** clauses (AND).
4. **Zero matches** → attach failure with diagnostic listing known cameras.
5. **Multiple matches** → attach failure (ambiguous selector); never pick
   arbitrarily.
6. Return or create a refcounted `LcameraDevice` for the winning `Camera::id()`.

Discovery helper: `lcameraDev` should expose a list/print entry point (used by
`lcameraBuff` verbose mode or a small CLI in `libcamera-tools` style) that prints
each camera’s `id()`, `model`, and `location` so operators can copy a stable
`lcamera-id:` into DAP specs after first install.

## Shared session and refcounting

Unlike X11 windows, a camera has no “sub-objects” inside its feed — the selector
always designates the whole physical camera. Multiple DAP lines for H, S, and V
share one **device session** (acquired `libcamera::Camera`), all under the same
`dev-identifier` (e.g. `cam0`).

```text
dev-identifier cam0 + deviceSelector (resolved) ──> LcameraDeviceSession (refcounted)
                                  │
                    ┌─────────────┼─────────────┐
                    ▼             ▼             ▼
              lcameraBuff     lcameraBuff     lcameraBuff
              (colour-hsv-h)  (colour-hsv-s)  (colour-hsv-v + intrins)
```

Rules:

* First `getOrCreate` for a resolved camera ID: enumerate, `acquire()` the
  `libcamera::Camera`, create the session entry.
* Subsequent `getOrCreate` for the same ID: increment refcount; return the same
  session handle.
* Last `release`: `release()` the libcamera camera and erase the session entry.
* Different `deviceSelector` strings that resolve to the same libcamera ID share
  one session (even if the selector text differs, e.g. one line uses
  `lcamera-id:...` and another uses a compound selector that resolves to the
  same camera).
* Attachments with the same `dev-identifier` and equivalent resolved camera are
  the same physical device from SMO’s perspective; channel differences come from
  the qualeIface name on each DAP line.

## Session mode configuration (Stage 1)

Before `lcameraBuff` starts capture, each producer calls
`lcameraDev_configureSessionModeCReq` on the shared `CameraSession` with the
requested mode:

| Field | Role |
|---|---|
| `width` / `height` | Requested capture dimensions (non-zero) |
| `colourSpace` | Semantic colour model (`Yuv` in v1) |
| `fullPlanarIsOptional` | Default `false` — must select fully planar YUV |

`lcameraDev` chooses a concrete libcamera pixel format (e.g. YUV420) from the
camera’s supported formats. DAPS lines name the semantic colour-space, not a
raw fourcc.

Rules:

* Configure only while the session exists and capture has not started.
* **Identical** configure request on an already-configured session is a **no-op**
  (multiple `lcameraBuff` producers sharing the same device selector each call
  get-or-create then configure with the same mode).
* **Different** configure request on an already-configured session throws
  (conflicting mode requests on the same physical camera).
* `fullPlanarIsOptional == false` (default): must select a fully planar YUV
  format or throw with candidate-format diagnostics.
* `fullPlanarIsOptional == true`: rejected at the configure API until
  `lcameraBuff` implements non-planar producer deinterleaving (Stage 2). The
  policy helper still accepts optional planar selection for future use.

Streaming, frame delivery, and per-frame colourspace work remain **out of scope**
for `lcameraDev`; `lcameraBuff` uses the configured session to allocate buffers
and start capture on attach.

## dlopen API

Exported from `liblcameraDev.so` using `extern "C"` symbols (mirroring
`livoxProto1`). `lcameraBuff` loads the library with `dlopen` + `dlsym` and
calls through function pointers. Hot-path operations are **`*CReq` coroutine
invokers** (`sscl::co::ViralNonPostingInvoker`); `main` / `exit` remain
synchronous.

Header: `commonLibs/lcameraDev/lcameraDev.h`.

### Lifecycle

```c
typedef void lcameraDev_mainFn(
    const std::shared_ptr<sscl::ComponentThread>& componentThread);

typedef void lcameraDev_exitFn(void);
```

`lcameraDev_main` records the Body `ComponentThread`, starts libcamera's
`CameraManager` (idempotent), and must succeed before any `*CReq` runs.

### Device acquisition

```cpp
struct LcameraDevGetOrCreateResult
{
    std::shared_ptr<CameraSession> deviceSession;
    CameraIdentityRecord resolvedIdentity;
};

typedef sscl::co::ViralNonPostingInvoker<LcameraDevGetOrCreateResult>
    lcameraDev_getOrCreateDeviceCReqFn(const std::string& deviceSelector);

typedef sscl::co::ViralNonPostingInvoker<void>
    lcameraDev_releaseDeviceCReqFn(
        const std::shared_ptr<CameraSession>& deviceSession);
```

Failures throw `std::exception`. `lcameraBuff` holds the returned
`shared_ptr<CameraSession>` and passes it back to `releaseDeviceCReq`. The
session wraps the acquired `libcamera::Camera`; higher layers configure the mode
then stream from that handle.

### Session mode configuration

```cpp
enum class LcameraDevColourSpace { Yuv };

struct LcameraDevCameraModeRequest
{
    unsigned width = 0;
    unsigned height = 0;
    LcameraDevColourSpace colourSpace = LcameraDevColourSpace::Yuv;
    bool fullPlanarIsOptional = false;
};

struct LcameraDevConfiguredCameraMode
{
    unsigned width;
    unsigned height;
    LcameraDevColourSpace colourSpace;
    std::string pixelFormatName;
    bool isFullyPlanar;
    unsigned planeCount;
};

typedef sscl::co::ViralNonPostingInvoker<LcameraDevConfiguredCameraMode>
    lcameraDev_configureSessionModeCReqFn(
        const std::shared_ptr<CameraSession>& deviceSession,
        const LcameraDevCameraModeRequest& request);
```

`lcameraDev_configureSessionModeCReq` delegates to
`CameraSession::configureSessionModeCReq`, which runs libcamera
`generateConfiguration` + `configure` and stores the result on the session.
Identical reconfigure is a no-op; conflicting reconfigure throws.

### Enumeration (discovery)

```cpp
struct LcameraDevCameraInfo
{
    std::string id;
    std::string model;
    std::string location;
};

typedef sscl::co::ViralNonPostingInvoker<std::vector<LcameraDevCameraInfo>>
    lcameraDev_enumerateCamerasCReqFn(void);
```

### Manual verification tools

When built with `-DENABLE_LIB_lcameraDev=ON`:

* `lcameraDev_list_cameras` — runs `enumerateCamerasCReq` on a minimal probe
  `ComponentThread`.
* `lcameraDev_probe <deviceSelector>` — `getOrCreateDeviceCReq`, then
  `releaseDeviceCReq` (selector and session attach/detach only).
* `lcameraDev_configure_probe <deviceSelector> <width> <height> [options]` —
  `getOrCreateDeviceCReq`, `configureSessionModeCReq`, print resolved mode (or
  exception), then `releaseDeviceCReq`. Options: `--colour-space=yuv`,
  `--opt-planar` / `--full-planar-is-optional`, `--reconfigure-twice`.

## Module layout

```text
commonLibs/lcameraDev/
    CMakeLists.txt
    lcameraDev.h / lcameraDev.cpp       Public C API and dlopen exports
    cameraManagerState.h / .cpp         CameraManager singleton, session map
    cameraSession.h / .cpp              Refcounted acquired-camera session
    cameraIdentity.h / .cpp             Discovery identity records
    selectorParse.h / .cpp              Compound selector parsing
    selectorResolve.h / .cpp            AND-match resolution
    tools/                              lcameraDev_list_cameras, lcameraDev_probe,
                                        lcameraDev_configure_probe
```

Build links against `libcamera` (pkg-config). Does **not** link Salmanoff
`smocore`; stays a standalone `.so` loadable by `lcameraBuff` and test tools.

## Relationship to future work

| Component | Responsibility |
|---|---|
| `lcameraDev` | libcamera lifecycle, selector resolution, refcounted acquired camera session |
| `lcameraBuff` | Stream setup, frames, `StimBuffApiDesc`, channel fan-out, intrins |
| `rasterStimulus` (future) | YUV↔HSV, plane extraction, threshold masks, stencil geometry |
| `xcbWindow` / `waylandWindow` | Separate capture path; reuse `rasterStimulus` only |

If libcamera IDs prove insufficient in practice, selector policies can gain
`by-id:` / `serial:` fallbacks that scan udev/USB metadata without changing the
`lcameraBuff` DAP surface — only `lcameraDev` selector resolution changes.

## Open questions

1. **Hot-unplug** — on camera removal, fail all attached `lcameraBuff` producers
   and drop the session, or attempt re-enumeration by stored `lcamera-id:`?
2. **IPA packaging** — document per-platform `apt install` requirements in the
   main README when `lcameraBuff` lands (RPi needs `libcamera-ipa`).

Stream format, frame timing, and libcamera callback threading are owned by
`lcameraBuff`, not `lcameraDev`.