hayodea/salmanoff
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 2 Wiki Activity

Update docs on DAP specs and DA specs

Browse Source
This commit is contained in:
Hayodea Hekol
2025-10-01 13:27:32 -04:00
parent c7ca889e9c
commit b771856330
4 changed files with 100 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/api-drm-server-linux.md
+7 -6
View File
@@ -1,10 +1,11 @@
# DeviceSpec: API `drm()` from server `linux()`:
# DeviceSpec: stim-buff-api `drm()` from provider `linux()`:
The API is Linux DRM/KMS. The server is Linux itself. This server provides
direct capture of frames at the kernel so it works for both Linux and Wayland.
There's a program known as GPU Screen Recorder that is able to use this to
capture specific windows on X11, but the window-specific capture doesn't work
with Wayland. Irrespective, whole-screen capture works on both GFX servers.
The stim-buff-api is Linux DRM/KMS. The provider is Linux itself. This provider
enables direct capture of frames at the kernel level, so it works for both X11
and Wayland. There is a program called GPU Screen Recorder that can use this
API to capture specific windows on X11, but window-specific capture does not
work with Wayland. However, whole-screen capture works on both graphics
servers.
Notes:
* `modetest` utility in ubuntu package `libdrm-tests` is relevant.
docs/api-xcb-provider-xorg.md
+16 -15
View File
@@ -1,24 +1,25 @@
# DeviceSpec: API `xcb`, provider `xorg`
# DAP Spec: stim-buff-api `xcb`, provider `xorg`
## Overview
The `xcb` API with the `xorg` provider allows Salmanoff to interact with Xorg
The `xcb` stim-buff-api with the `xorg` provider allows Salmanoff to interact with Xorg
server windows. This can be used to capture visual data from specific windows
or entire screens managed by the Xorg server.
## DeviceSpec Format
## DAP Spec Format
The general format of a device-spec for the `xcb` API with the `xorg` provider
The general format of a DAP spec for the `xcb` stim-buff-api with the `xorg` provider
is:
```
sensor-type|implexor|xcb(api-params)|xorg(provider-params)|deviceSelector
sensor-type|dev-identifier|stim-iface-api|xcb(stim-buff-api-params)|xorg(provider-params)|deviceSelector
```
* `sensor-type` is always either '`+idev`' (interoceptor), '`+edev`'
(extrospector), or '`+adev`' (actuator).
* `implexor` is the name of the implexor algorithm that should be used with
the data that is provided by the `provider` via the `api`.
* `api` is `xcb` in this case, and the `api-params` in parentheses may be
* `dev-identifier` is a user-defined name for this specific device instance.
* `stim-iface-api` is the name of the StimIface library that should be used to
process the data from the stim buffer.
* `stim-buff-api` is `xcb` in this case, and the `stim-buff-api-params` in parentheses may be
omitted, in which case the parentheses will be empty, but the parentheses
must always be written out.
* `provider` is `xorg` in this case, and the `provider-params` in parentheses
@@ -28,11 +29,11 @@ sensor-type|implexor|xcb(api-params)|xorg(provider-params)|deviceSelector
identify the specific window or screen you want to access via that
`provider`.
## `api-params` and `provider-params`
## `stim-buff-api-params` and `provider-params`
### `api-params`
### `stim-buff-api-params`
The `api-params` for the `xcb` API can include the following:
The `stim-buff-api-params` for the `xcb` stim-buff-api can include the following:
* `dev-id` or `devid`: Specifies that the `deviceSelector` is a numeric window
ID. The ID can be specified in decimal or hexadecimal format.
@@ -97,25 +98,25 @@ xcb(dev-string)|My\ Exact\ Window\ Name
### To attach a specific window by name (substring match):
```
+edev|visual-implexor|xcb(dev-substring)|xorg(display=0|screen=0)|my-window
+edev|my-window|visual-stimiface|xcb(dev-substring)|xorg(display=0|screen=0)|my-window
```
This will attach to a window whose name contains "my-window" as a substring.
### To attach a specific window by exact name:
```
+edev|visual-implexor|xcb(dev-string)|xorg(display=0|screen=0)|My\ Exact\ Window\ Name
+edev|my-window|visual-stimiface|xcb(dev-string)|xorg(display=0|screen=0)|My\ Exact\ Window\ Name
```
This will attach to a window whose name exactly matches "My Exact Window Name".
### To attach a specific window by numeric ID:
```
+edev|visual-implexor|xcb(dev-id)|xorg(display=0|screen=0)|123456
+edev|my-window|visual-stimiface|xcb(dev-id)|xorg(display=0|screen=0)|123456
```
This will attach to a window with the numeric ID `123456`.
### To attach the entire screen:
```
+edev|visual-implexor|xcb()|xorg(display=0|screen=0)|all
+edev|my-screen|visual-stimiface|xcb()|xorg(display=0|screen=0)|all
```
This will attach to the entire screen `0` of display `0`.
docs/deviceAttachmentPipelineSpec.md
+65 -17
View File
@@ -2,11 +2,19 @@
## DAP Specs vs DA Specs:
**DAP (Device Attachment Pipeline) specs** are human-readable DSL specifications that describe a pipeline of steps to connect a particular device role on a particular device-identifier to Salmanoff. This document describes the DAP specification format.
**DAP (Device Attachment Pipeline) specs** are human-readable DSL
specifications that describe a pipeline of steps to connect a particular
device role on a particular device-identifier to Salmanoff. This document
describes the DAP specification format.
**DA (Device Attachment) specs** are compiled binary structs used internally by SMO after DAP specs have been parsed into binary format. DA specs are the internal representation that the system actually uses.
**DA (Device Attachment) specs** are compiled binary structs used internally
by SMO after DAP specs have been parsed into binary format. DA specs are the
internal representation that the system actually uses.
**Multiple Input Formats**: DAP specs can be parsed from multiple human-readable formats. For example, we intend to eventually extend ROS's URDF XML format to specify device attachment specs (URDFDA specs), which would also get compiled into the same DA spec binary format.
**Multiple Input Formats**: DAP specs can be parsed from multiple
human-readable formats. For example, we intend to eventually extend ROS's
URDF XML format to specify device attachment specs (URDFDA specs), which
would also get compiled into the same DA spec binary format.
## Attaching sensors:
@@ -34,7 +42,8 @@ sensor-type|dev-identifier|stim-iface-api|stim-buff-api(api-params)|provider(pro
(extrospector), or '`+adev`' (actuator).
* `dev-identifier` is a user-defined name for this specific device instance.
This represents a logical device that can be accessed through multiple
providers and may expose multiple stim features.
providers and may expose multiple stim features. In a sense it's like a
sense organ or sense modality.
* `stim-iface-api` is the name of the StimIface library that should be used to
process the data from the stim buffer. This replaces the previous implexor
concept.
@@ -44,7 +53,8 @@ sensor-type|dev-identifier|stim-iface-api|stim-buff-api(api-params)|provider(pro
`api-params` in parentheses may be omitted, in which case the parentheses
will be empty, but the parentheses must always be written out.
* `provider` may be a userspace daemon or an OS kernel that provides access to
the device's stim buffers via the `stim-buff-api`. The `provider-params` in
the device's I/O functionality; and thereby allows the `stim-buff-api` to
construct and present a stim-buffer to Salmanoff. The `provider-params` in
parentheses may be omitted, in which case the parentheses will be empty, but
the parentheses must always be written out.
* `dev-selector` is the idiosyncratic label/name used by the `provider` to
@@ -64,37 +74,75 @@ Each parameter must be in one of these forms:
* key=
* key
### Important Note on `stim-buff-api-params`:
The `stim-buff-api-params` should **never** include options related to the
stim buffer's type or format. The `stim-buff-api` must read and infer such
configuration details from the `stim-iface-api` portion of the DAP spec, and
configure itself accordingly to enable connection by the specified
stim-iface library in the way that it has been configured.
`stim-buff-api-params` are for options that are:
- Device-specific (not modality-wide)
- Specific to this particular stim-feature as provided by this device
- Configuration parameters needed by the stim-buff-api to properly interface
with the device
Examples of appropriate `stim-buff-api-params`:
- Buffer size settings
- Device-specific communication parameters
- Hardware-specific configuration options
- Connection timeouts or retry settings
Examples of **inappropriate** `stim-buff-api-params`:
- Data format specifications (should be inferred from stim-iface-api)
- Color space settings (should be determined by the stim-iface library)
- Processing algorithm parameters (belong to the stim-iface library)
## Logical View and Multiple Access Patterns:
### Single Device, Multiple Providers:
A single `dev-identifier` can unite several `dev-selectors` from multiple providers.
For example, a sound card device `soundcard0` could be accessed through:
A single `dev-identifier` can unite several `dev-selectors` from multiple
providers. For example, a sound card device `soundcard0` could be accessed
through:
* `ident: soundcard0, provider: alsa` - Provides access to the card via ALSA API for audio output
* `ident: soundcard0, provider: linux-driver-direct-file-ops` - Provides direct connection to Linux driver via read/write posix FD calls for beeper sound output
* `ident: soundcard0, provider: alsa` - Provides access to the card via ALSA for microphone input
* `ident: soundcard0, provider: alsa` - Provides access to the card via ALSA
API for audio output
* `ident: soundcard0, provider: linux-driver-direct-file-ops` - Provides direct
connection to Linux driver via read/write posix FD calls for beeper sound
output
* `ident: soundcard0, provider: alsa` - Provides access to the card via ALSA
for microphone input
So a single physical device is accessed via multiple providers, each with different selectors.
So a single physical device is accessed via multiple providers, each with
different selectors.
### Single Device, Same Provider, Different Stim-Buff-APIs:
A device could have different `stim-buff-apis`, possibly provided by different shared libraries:
A device could have different `stim-buff-apis`, possibly provided by different
shared libraries:
* `ident: soundcard0, provider: alsa, stim-buff-api: alsa-audio` - For audio output
* `ident: soundcard0, provider: alsa, stim-buff-api: alsa-mic` - For microphone input
* `ident: soundcard0, provider: alsa, stim-buff-api: alsa-audio` - For audio
output
* `ident: soundcard0, provider: alsa, stim-buff-api: alsa-mic` - For microphone
input
Different stim-buff-apis may be packaged into the same shared library, or multiple libraries may dlopen a common library behind the scenes.
Different stim-buff-apis may be packaged into the same shared library, or
multiple libraries may dlopen a common library behind the scenes.
### Stim Features and Buffers:
Logically, a `dev-identifier` represents a sense modality. Each device can export multiple stim features. For example, an eye can export:
Logically, a `dev-identifier` represents a sense modality. Each device can
export multiple stim features. For example, an eye can export:
- Color data
- Light intensity data
- Thermal heat data
- Pain input data
Each stim feature is exposed as a stim buffer, provided by a `stim-buff-api`. Stim-buff-apis rely upon providers to implement the device-specific operations required to effectuate the stim-buff controls.
Each stim feature is exposed as a stim buffer, provided by a `stim-buff-api`.
Stim-buff-apis rely upon providers to implement the device-specific operations
required to effectuate the stim-buff controls.
## Examples:
docs/livox-gen1-lidar-dap-spec.md
+11 -11
View File
@@ -1,14 +1,14 @@
# LivoxGen1Lidar Device Attachment Protocol (DAP) Specification
# LivoxGen1Lidar Device Attachment Pipeline (DAP) Specification
## Overview
The LivoxGen1Lidar DAP specification defines how to attach to Livox Gen1 LiDAR devices and access their various data streams using a unified API with mode-based parameter selection.
The LivoxGen1Lidar DAP specification defines how to attach to Livox Gen1 LiDAR devices and access their various data streams using a unified stim-buff-api with mode-based parameter selection.
## API Structure
## Stim-Buff-API Structure
The LivoxGen1Lidar DAP uses a unified API name `livoxGen1` with mode-based parameters to specify which data interface to present.
The LivoxGen1Lidar DAP uses a unified stim-buff-api name `livoxGen1` with mode-based parameters to specify which data interface to present.
## Device Attachment Specifications
## DAP Specifications
### 1. Point Cloud Intensity Data Device (Interoceptor)
@@ -16,13 +16,13 @@ The LivoxGen1Lidar DAP uses a unified API name `livoxGen1` with mode-based param
**Syntax**:
```
+idev | avia0 | structural-implexor | livoxGen1(mode=pointCloudIntensity) | livoxProto1(handshake-timeout-ms=1000,retry-delay-ms=3000,smo-ip=192.168.1.50,smo-subnet-nbits=24) | 3JEDK380010Z39
+idev | avia0 | structural-stimiface | livoxGen1(mode=pointCloudIntensity) | livoxProto1(handshake-timeout-ms=1000,retry-delay-ms=3000,smo-ip=192.168.1.50,smo-subnet-nbits=24) | 3JEDK380010Z39
```
**Alternative Syntax** (all equivalent):
```
+idev | avia0 | structural-implexor | livoxGen1(stim=pcloudIntensity) | livoxProto1(handshake-timeout-ms=1000,retry-delay-ms=3000,smo-ip=192.168.1.50,smo-subnet-nbits=24) | 3JEDK380010Z39
+idev | avia0 | structural-implexor | livoxGen1(affordance=pCloudI) | livoxProto1(handshake-timeout-ms=1000,retry-delay-ms=3000,smo-ip=192.168.1.50,smo-subnet-nbits=24) | 3JEDK380010Z39
+idev | avia0 | structural-stimiface | livoxGen1(stim=pcloudIntensity) | livoxProto1(handshake-timeout-ms=1000,retry-delay-ms=3000,smo-ip=192.168.1.50,smo-subnet-nbits=24) | 3JEDK380010Z39
+idev | avia0 | structural-stimiface | livoxGen1(affordance=pCloudI) | livoxProto1(handshake-timeout-ms=1000,retry-delay-ms=3000,smo-ip=192.168.1.50,smo-subnet-nbits=24) | 3JEDK380010Z39
```
**Mode Parameter Values** (synonymous):
@@ -38,7 +38,7 @@ The LivoxGen1Lidar DAP uses a unified API name `livoxGen1` with mode-based param
**Syntax**:
```
+edev | avia0 | structural-implexor | livoxGen1(mode=pcloud,format=xyz) | livoxProto1(handshake-timeout-ms=1000,retry-delay-ms=3000,smo-ip=192.168.1.50,smo-subnet-nbits=24) | 3JEDK380010Z39
+edev | avia0 | structural-stimiface | livoxGen1(mode=pcloud,format=xyz) | livoxProto1(handshake-timeout-ms=1000,retry-delay-ms=3000,smo-ip=192.168.1.50,smo-subnet-nbits=24) | 3JEDK380010Z39
```
**Mode Parameter Values** (synonymous):
@@ -62,7 +62,7 @@ The LivoxGen1Lidar DAP uses a unified API name `livoxGen1` with mode-based param
**Syntax**:
```
+idev | avia0 | gyro-implexor | livoxGen1(mode=gyro) | livoxProto1(handshake-timeout-ms=1000,retry-delay-ms=3000,smo-ip=192.168.1.50,smo-subnet-nbits=24) | 3JEDK380010Z39
+idev | avia0 | gyro-stimiface | livoxGen1(mode=gyro) | livoxProto1(handshake-timeout-ms=1000,retry-delay-ms=3000,smo-ip=192.168.1.50,smo-subnet-nbits=24) | 3JEDK380010Z39
```
**Mode Parameter Values**:
@@ -74,7 +74,7 @@ The LivoxGen1Lidar DAP uses a unified API name `livoxGen1` with mode-based param
**Syntax**:
```
+idev | avia0 | accel-implexor | livoxGen1(mode=accel) | livoxProto1(handshake-timeout-ms=1000,retry-delay-ms=3000,smo-ip=192.168.1.50,smo-subnet-nbits=24) | 3JEDK380010Z39
+idev | avia0 | accel-stimiface | livoxGen1(mode=accel) | livoxProto1(handshake-timeout-ms=1000,retry-delay-ms=3000,smo-ip=192.168.1.50,smo-subnet-nbits=24) | 3JEDK380010Z39
```
**Mode Parameter Values**: