salmanoff/docs/lcamera-dev-lib.md

# liblcameraDev: libcamera device provider library

## Overview

`liblcameraDev.so` (`commonLibs/lcameraDev/`) is a Salmanoff common library that
wraps [libcamera](https://libcamera.org/) and exposes a small, dlopen-able C
API for acquiring refcounted camera device handles. It is the camera analogue of
`libxcbXorg.so` and `liblivoxProto1.so`: it owns transport and device lifecycle,
not stim-buffer semantics.

The eventual `lcameraBuff` stim-buff-api (`stimBuffApis/lcameraBuff/`) loads
this library dynamically (via SMO search paths), resolves cameras from DAP
`deviceSelector` strings, and shares one capture session per physical camera
across multiple DAP attachments (e.g. separate H/S/V channel lines).

```
+edev|cam0|colour-hsv-h()|lcameraBuff()|lcameraDev()|lcamera-id:...||
+edev|cam0|colour-hsv-s()|lcameraBuff()|lcameraDev()|lcamera-id:...||
+edev|cam0|postrin(...)|negtrin(...)|colour-hsv-v()|lcameraBuff()|lcameraDev()|lcamera-id:...||
```

The **`dev-identifier`** (`cam0` above) is the same on all three lines: one
logical camera device, three stim-feature attachments (H, S, V). The
**`deviceSelector`** (provider column) is also typically identical across those
lines so they resolve to the same physical camera and share one capture session.

Layering:

```text
lcameraBuff (stimBuffApi)     SMO-facing producers, channel fan-out, intrins
    └─ dlopen ─> lcameraDev (commonLib)   libcamera manager, device refcounting
                    └─ libcamera0.2 + libcamera-ipa (system packages)
```

Channel splitting, colourspace conversion, threshold masks, and stencils belong
in a separate shared raster library (`rasterStimulus`, future) and in
`lcameraBuff`; `lcameraDev` stops at “here is a stable, acquired libcamera
`Camera` you can configure and stream from.”

## Why libcamera (for now)

Initial development targets libcamera rather than raw V4L2 so that complex Linux
camera pipelines (Media Controller graphs, platform ISPs, Raspberry Pi CSI
modules) are handled by libcamera’s pipeline handlers. Plain USB UVC webcams are
also supported via libcamera’s `uvcvideo` pipeline on many distributions.

This is a **Linux-only** path. It does not provide cross-Unix portability by
itself; a future `v4l2Linux` provider remains the fallback for generic
`/dev/videoN` access without libcamera.

Runtime packages (Debian/Ubuntu naming):

| Package | Purpose |
|---|---|
| `libcamera0.2` | Core `libcamera.so`, pipeline handlers, `CameraManager` |
| `libcamera-ipa` | IPA plugin modules (required on RPi, Rockchip, IPU3, etc.) |
| `libcamera-dev` | Headers and pkg-config (build only) |

`libcamera-v4l2` (`v4l2-compat.so`) is **not** required for native libcamera
clients; it exists only to wrap legacy V4L2 applications via `LD_PRELOAD`.

## DAP roles

General line shape (see also `docs/deviceAttachmentPipelineSpec.md`):

```text
sensor-type|dev-identifier|postrin-spec|negtrin-spec|quale-iface-api|stim-buff-api|provider|deviceSelector
```

For camera colour channels:

| DAP segment | Value | Role |
|---|---|---|
| `dev-identifier` | e.g. `cam0` | User-defined name for this **device instance**; same across H/S/V lines |
| `quale-iface-api` | `colour-hsv-h()`, `colour-hsv-s()`, `colour-hsv-v()` | Names the stim feature / greyscale channel plane |
| `stim-buff-api` | `lcameraBuff()` | Salmanoff producer plugin; maps qualeIface → channel |
| `provider` | `lcameraDev()` | Selects the libcamera-backed device access path |
| `deviceSelector` | see below | Identifies which physical camera to open (often repeated on each line) |

Intrinsic thresholds on the brightness/value channel live in `postrin(...)` /
`negtrin(...)` segments on the `colour-hsv-v()` line only, following
`docs/design/intrin-thresholds.md`.

Example:

```text
+edev|cam0|colour-hsv-h()|lcameraBuff()|lcameraDev()|lcamera-id:/base/soc/i2c@.../imx219@10||
+edev|cam0|colour-hsv-s()|lcameraBuff()|lcameraDev()|lcamera-id:/base/soc/i2c@.../imx219@10||
+edev|cam0|negtrin(interest-thr=40|distraction-thr=70|intolerable-thr=90)|postrin(interest-thr=30|distraction-thr=50|stupefaction-thr=80)|colour-hsv-v()|lcameraBuff()|lcameraDev()|lcamera-id:/base/soc/i2c@.../imx219@10||
```

Compound selector (multiple criteria on one line):

```text
+edev|cam0|colour-hsv-h()|lcameraBuff()|lcameraDev()|lcamera-id:foo bar baz;model:aaaa||
```

All three lines share one underlying `lcameraDev` capture session keyed by the
resolved libcamera camera ID. `lcameraBuff` must not open the same camera twice.

### `lcameraDev()` provider params

Reserved for future options. Empty parentheses are valid:

```text
lcameraDev()
```

Possible future params (not implemented in v1):

* `ipa-dir=...` — override IPA module search path
* `log-level=...` — map to `LIBCAMERA_LOG_LEVELS` for this provider instance

### `lcameraBuff()` stim-buff-api params

Reserved for per-producer options (frame rate caps, pixel format preference).
Channel selection is driven by the qualeIface name, not by stim-buff-api params.

Possible future params:

* `fps-hz=30`
* `width=640|height=480`
* `pixfmt=NV12` — negotiation hint passed down to `lcameraDev`

## Device selector format

libcamera’s primary stable identifier is `Camera::id()`: an opaque string,
unique per camera on the system, stable across reboot when physical topology is
unchanged. libcamera properties such as `Model` are descriptive but **not**
guaranteed unique; there is no standard `SerialNumber` property in the core
property set.

### Single criterion

`lcameraDev` resolves each criterion using a typed `prefix:value` form. If the
selector contains no `;` and no `prefix:` delimiter, the entire string is
treated as a literal `lcamera-id:` value (opaque libcamera ID).

| Prefix | Example | Match rule |
|---|---|---|
| `lcamera-id:` | `lcamera-id:/base/soc/i2c@1/imx219@10` | Exact `Camera::id()` |
| *(bare string)* | `/base/soc/i2c@1/imx219@10` | Same as `lcamera-id:` |
| `index:` | `index:0` | Nth camera from `CameraManager::cameras()` |
| `model:` | `model:imx219` | Exact match on `properties::Model` |
| `model-substr:` | `model-substr:Logitech` | Substring match on `properties::Model` |
| `location:` | `location:external` | `front`, `back`, or `external` |

The value after `:` runs to the next `;` or end of string. Values may contain
spaces (e.g. `lcamera-id:foo bar baz`). Leading/trailing whitespace on each
value is stripped after parsing.

### Compound selectors (semicolon-separated)

A `deviceSelector` may contain **multiple criteria** separated by `;`. Every
criterion must match the **same** enumerated camera (**AND** semantics). This
lets operators combine a stable libcamera ID with a sanity-check on model name,
or narrow a fuzzy selector with an extra constraint.

```text
lcamera-id:foo bar baz;model:aaaa
location:external;model-substr:Logitech
index:0;model:imx219
```

Parsing rules:

1. Split the selector on unescaped `;` into one or more clauses (a single clause
   with no `;` is a compound selector of length 1).
2. Parse each clause as `prefix:value` (or bare opaque ID if no `:` is present
   in that clause only).
3. A camera is a candidate only if it satisfies **all** clauses.
4. **Zero candidates** → attach failure with diagnostics.
5. **More than one candidate** → attach failure (ambiguous); never pick
   arbitrarily.
6. Exactly one candidate → resolve session by that camera’s `Camera::id()`.

If a value must contain a literal `;`, escape it as `\;` in the selector string
(same backslash-discard convention as whitespace escaping elsewhere in DAP
tokens).

Resolution algorithm (summary):

1. Start `CameraManager` (once per process, inside `lcameraDev`).
2. Enumerate cameras; build an identity record per camera (`id`, `model`,
   `location`, `systemDevices`).
3. Parse `deviceSelector` into criterion clauses; apply **all** clauses (AND).
4. **Zero matches** → attach failure with diagnostic listing known cameras.
5. **Multiple matches** → attach failure (ambiguous selector); never pick
   arbitrarily.
6. Return or create a refcounted `LcameraDevice` for the winning `Camera::id()`.

Discovery helper: `lcameraDev` should expose a list/print entry point (used by
`lcameraBuff` verbose mode or a small CLI in `libcamera-tools` style) that prints
each camera’s `id()`, `model`, and `location` so operators can copy a stable
`lcamera-id:` into DAP specs after first install.

## Shared session and refcounting

Unlike X11 windows, a camera has no “sub-objects” inside its feed — the selector
always designates the whole camera stream. Multiple DAP lines for H, S, and V
are **views** over one capture session, all under the same `dev-identifier`
(e.g. `cam0`).

```text
dev-identifier cam0 + deviceSelector (resolved) ──> LcameraDeviceSession (refcounted)
                                  │
                    ┌─────────────┼─────────────┐
                    ▼             ▼             ▼
              lcameraBuff     lcameraBuff     lcameraBuff
              (colour-hsv-h)  (colour-hsv-s)  (colour-hsv-v + intrins)
```

Rules:

* First `getOrCreate` for a resolved camera ID: enumerate, `acquire()` the
  `libcamera::Camera`, negotiate and start the shared stream.
* Subsequent `getOrCreate` for the same ID: increment refcount; return the same
  session handle.
* Last `release`: `release()` the libcamera camera, stop streaming, tear down
  buffers.
* Different `deviceSelector` strings that resolve to the same libcamera ID share
  one session (even if the selector text differs, e.g. one line uses
  `lcamera-id:...` and another uses a compound selector that resolves to the
  same camera).
* Attachments with the same `dev-identifier` and equivalent resolved camera are
  the same physical device from SMO’s perspective; channel differences come from
  the qualeIface name on each DAP line.

## dlopen API (planned)

Exported from `liblcameraDev.so` using `extern "C"` symbols (mirroring
`livoxProto1`). `lcameraBuff` loads the library with `dlopen` + `dlsym` and
calls through function pointers.

### Lifecycle

```c
/* Start CameraManager; idempotent per process. */
typedef void lcameraDev_mainFn(void);

/* Stop manager, release all devices; called on SMO shutdown. */
typedef void lcameraDev_exitFn(void);
```

### Device acquisition

```c
struct LcameraDevGetOrCreateResult
{
    bool success;
    /* Opaque session handle; valid until matching release call. */
    void* deviceSession;
    /* Resolved libcamera camera ID (stable session key). */
    const char* resolvedCameraId;
};

typedef LcameraDevGetOrCreateResult lcameraDev_getOrCreateDeviceFn(
    const char* deviceSelector);

typedef void lcameraDev_releaseDeviceFn(void* deviceSession);
```

`deviceSession` is opaque to callers. `lcameraBuff` passes it back when
detaching; it must not call libcamera APIs directly.

### Enumeration (discovery)

```c
struct LcameraDevCameraInfo
{
    const char* id;
    const char* model;      /* empty string if unavailable */
    const char* location;   /* "front", "back", "external", or "" */
};

struct LcameraDevEnumerateResult
{
    size_t count;
    LcameraDevCameraInfo* cameras; /* caller frees via lcameraDev_freeEnumerateResult */
};

typedef LcameraDevEnumerateResult lcameraDev_enumerateCamerasFn(void);
typedef void lcameraDev_freeEnumerateResultFn(LcameraDevEnumerateResult* result);
```

### Frame access (boundary with lcameraBuff)

Exact frame-delivery API is TBD and will likely use a callback or a “dequeue
latest frame buffer” method on the session handle. `lcameraDev` owns libcamera
`FrameBuffer` allocation and `Request` requeue; `lcameraBuff` copies or maps the
plane it needs for the active qualeIface channel.

Principle: **one negotiated stream format per session** (prefer native YUV such
as NV12/YUYV). `lcameraBuff` + `rasterStimulus` derive H/S/V greyscale planes
from that single stream.

## Module layout (planned)

```text
commonLibs/lcameraDev/
    CMakeLists.txt
    lcameraDev.h          Public C API + C++ internal headers
    lcameraDev.cpp        dlopen exports, CameraManager singleton
    cameraSession.cpp     Refcounted session, stream negotiation
    selectorResolve.cpp   deviceSelector parsing and matching
    cameraEnumerate.cpp   Discovery / identity records
```

Build links against `libcamera` (pkg-config). Does **not** link Salmanoff
`smocore`; stays a standalone `.so` loadable by `lcameraBuff` and test tools.

## Relationship to future work

| Component | Responsibility |
|---|---|
| `lcameraDev` | libcamera lifecycle, selector resolution, shared capture session |
| `lcameraBuff` | `StimBuffApiDesc`, `StimulusProducer`, per-channel ring buffers, intrins |
| `rasterStimulus` (future) | YUV↔HSV, plane extraction, threshold masks, stencil geometry |
| `xcbWindow` / `waylandWindow` | Separate capture path; reuse `rasterStimulus` only |

If libcamera IDs prove insufficient in practice, selector policies can gain
`by-id:` / `serial:` fallbacks that scan udev/USB metadata without changing the
`lcameraBuff` DAP surface — only `lcameraDev` selector resolution changes.

## Open questions

1. **Stream format defaults** — prefer first supported YUV format, or allow
   `lcameraBuff(pixfmt=...)` override?
2. **Hot-unplug** — on camera removal, fail all attached `lcameraBuff` producers
   and drop the session, or attempt re-enumeration by stored `lcamera-id:`?
3. **Thread model** — libcamera callbacks vs polling in the Body thread
   production daemon; align with `StimulusProducer::productionCDaemon` timing
   (`CONFIG_STIMBUFF_FRAME_PERIOD_MS`).
4. **IPA packaging** — document per-platform `apt install` requirements in the
   main README when `lcameraBuff` lands (RPi needs `libcamera-ipa`).