Rename DSL: deviceSpec => deviceAttachmentSpec

This language is used broadly to specify how to attach (and thus
also how to detach) devices to/from Salmanoff. The next bit of work
we'll do is split off the DSL parsing from the management of the
list of parsed binary attached spec objects.

We'll be creating a PipeDeviceAttachmentParser, and later on when
we support URDF, we'll create a URDFDeviceAttachmentParser.
This commit is contained in:
2025-08-29 15:16:11 -04:00
parent 8f41e164a2
commit 6ef86eea05
10 changed files with 89 additions and 78 deletions
@@ -1,4 +1,4 @@
# Device manager: attaching sensors and actuators.
# Device Attachment Specification DSL: attaching sensors and actuators to SMO.
## Attaching sensors:
@@ -14,15 +14,17 @@ basically what conventional ML/LLM/ANN developers call an ROI ("Region of
Interest") extraction algorithm. An Implex algorithm is used to scan a frame
of input sensor data and detect objects and patterns within it.
## Sensor device spec:
## Sensor device attachment specification:
The general format of a device-spec for a sensor is:
The general format of a device attachment specification for a sensor is:
```
sensor-type|implexor|api(api-params)|provider(provider-params)|deviceselector
sensor-type|dev-identifier
|implexor|api(api-params)|provider(provider-params)|deviceselector
```
* `sensor-type` is always either '`+idev`' (interoceptor), '`+edev`'
(extrospector), or '`+adev`' (actuator).
* `dev-identifier` is a user-defined name for this specific device instance.
* `implexor` is the name of the implexor algorithm that should be used with
the data that is provided by the `provider` via the `api`.
* `api` is the interface that the provider uses to export perceptual data for
@@ -35,7 +37,7 @@ sensor-type|implexor|api(api-params)|provider(provider-params)|deviceselector
which case the parenthesis will be empty, but the parentheses must always be
written out.
* `device selector` is the idiosyncratic label/name used by the `provider` to
identify the specific device you want to access via that `provider`.
identify the specific device you want to attach via that `provider`.
## `API-params` and `provider-params`:
@@ -55,7 +57,7 @@ Some examples follow:
### To attach a particular window from a window manager:
```
+edev|visual-implexor|wayland()|wayland(server-socket)|window0
+edev|my-window|visual-implexor|wayland()|wayland(server-socket)|window0
```
Connect to the Wayland server that's listening on `server-socket`, using the
`wayland` api. Ask that Wayland server to give salmanoff read-access to all of
@@ -64,7 +66,7 @@ the frames composited into the window buffer for `window0`. Use salmanoff's
### To attach a window manager's entire rendered desktop:
```
+edev|visual-implexor|wayland()|wayland(listen-socket)|all
+edev|my-desktop|visual-implexor|wayland()|wayland(listen-socket)|all
```
In most cases, this is basically the same as attempting to attach all of the
underlying GFX server's output.
@@ -76,7 +78,7 @@ that Wayland server's compositor data.
### To attach all of an Xorg server's gfx output to all screens:
```
+edev|visual-implexor|x11()|xorg(listen-socket)|all
+edev|my-xorg-display|visual-implexor|x11()|xorg(listen-socket)|all
```
Connect to the Xorg server that's listening on `listen-socket`, using the `x11`
@@ -93,7 +95,7 @@ WM's output.
### To attach all of an Xorg server's gfx output to a particular screen:
```
+edev|visual-implexor|x11()|xorg(listen-socket)|:0
+edev|my-screen|visual-implexor|x11()|xorg(listen-socket)|:0
```
Connect to the Xorg server that's listening on `listen-socket`, using the `x11`
api. Ask that Xorg server to let Salmanoff read out all of the frames written
@@ -106,7 +108,7 @@ out to display `:0`. Use salmanoff's `visual-implexor` to implex from display
### To attach a camera device by connecting directly to its Linux driver:
```
+edev|visual-implexor|v4l()|linux()|/dev/video0
+edev|my-camera|visual-implexor|v4l()|linux()|/dev/video0
```
We specify that we want to use the `linux` kernel's loaded driver to connect
to communicate with `/dev/video0`, via the `Video4Linux` API. We want salmanoff
@@ -116,7 +118,7 @@ If `/dev/video0` is already consumed by another process, this may likely fail.
### To attach a microphone that's managed by ALSA server:
```
+edev|audio-implexor|alsa(shmem)|alsa()|cardname
+edev|my-microphone|audio-implexor|alsa(shmem)|alsa()|cardname
```
Connect to the ALSA server via `shmem`, using the `alsa` API. Request access to
@@ -125,7 +127,7 @@ the microphone function of the sound card with the name `cardname`. Use the
### To attach a thermal sensor managed by Linux:
```
+idev|thermal-implexor|thermal-zone()|linux()|/sys/class/thermal_zone0
+idev|my-thermal|thermal-implexor|thermal-zone()|linux()|/sys/class/thermal_zone0
```
Use the `thermal-zone` SysFS API provided by `linux` to connect to the sensor
@@ -147,18 +149,20 @@ types of devices will require different wilzor algorithms. You need to know
what type of wilzor algorithm needs to be used to enable salmanoff to control
your actuator device.
The general format for an actuator's device spec is:
The general format for an actuator's device attachment specification is:
```
WIP: TBD.
```
## Device specification files:
## Device attachment specification files:
Inside of a device spec file, you can list any number of device specs.
Separate individual device specs with two consecutive h-bar characters (`||`),
Inside of a device attachment specification file, you can list any number of
device attachment specifications.
Separate individual device attachment specifications with two consecutive h-bar
characters (`||`),
like this:
```
+edev|visual-implexor|wayland()|wayland(server-socket)|window0
|| +edev|visual-implexor|x11()|xorg(listen-socket)|all
|| +idev|thermal-implexor|thermal-zone()|linux()|/sys/class/thermal_zone0
+edev|my-window|visual-implexor|wayland()|wayland(server-socket)|window0
|| +edev|my-xorg-display|visual-implexor|x11()|xorg(listen-socket)|all
|| +idev|my-thermal|thermal-implexor|thermal-zone()|linux()|/sys/class/thermal_zone0
```
@@ -9,24 +9,26 @@
namespace smo {
namespace device {
class SenseDeviceSpec
class DeviceAttachmentSpec
{
public:
friend std::ostream& operator<<(
std::ostream& os, const SenseDeviceSpec& spec)
std::ostream& os, const DeviceAttachmentSpec& spec)
{
os << spec.stringify();
return os;
}
bool operator==(const SenseDeviceSpec& other) const
bool operator==(const DeviceAttachmentSpec& other) const
{
return sensorType == other.sensorType &&
return deviceIdentifier == other.deviceIdentifier &&
sensorType == other.sensorType &&
provider == other.provider &&
deviceSelector == other.deviceSelector;
}
public:
std::string deviceIdentifier;
char sensorType;
std::string implexor;
std::string api;
@@ -38,8 +40,10 @@ public:
std::string stringify() const
{
std::ostringstream os;
os << "Device: " << sensorType << ", Implexor: "
<< implexor << ", API: " << api << ", API Params: (";
os << "Device Identifier: " << deviceIdentifier
<< ", Sensor Type: " << sensorType
<< ", Implexor: " << implexor << ", API: " << api
<< ", API Params: (";
for (const auto& param : apiParams)
{
os << param.first;
@@ -63,11 +67,11 @@ public:
}
};
class InteroceptorDeviceSpec : public SenseDeviceSpec
class InteroceptorDevAttachmentSpec : public DeviceAttachmentSpec
{
};
class ExtrospectorDeviceSpec : public SenseDeviceSpec
class ExtrospectorDevAttachmentSpec : public DeviceAttachmentSpec
{
};
+4 -4
View File
@@ -1,12 +1,12 @@
#ifndef __USER_SENSE_API_LIB_H__
#define __USER_SENSE_API_LIB_H__
#include <preprocessor.h>
#include <stdbool.h>
#include <user/senseDeviceSpec.h>
#include <optional>
#include <string>
#include <memory>
#include <preprocessor.h>
#include <user/deviceAttachmentSpec.h>
namespace smo {
namespace sense_api {
@@ -14,9 +14,9 @@ namespace sense_api {
typedef int (sal_mlo_initializeIndFn)(void);
typedef int (sal_mlo_finalizeIndFn)(void);
typedef int (sal_mlo_attachDeviceReqFn)(
const std::shared_ptr<device::SenseDeviceSpec>& desc);
const std::shared_ptr<device::DeviceAttachmentSpec>& desc);
typedef int (sal_mlo_detachDeviceReqFn)(
const std::shared_ptr<device::SenseDeviceSpec>& desc);
const std::shared_ptr<device::DeviceAttachmentSpec>& desc);
/**
* @brief Hooks provided by Salmanoff to senseApi libraries.
+8 -8
View File
@@ -7,7 +7,7 @@
#include <dlfcn.h>
#include <xcb/xcb.h>
#include <user/senseApiDesc.h>
#include <user/senseDeviceSpec.h>
#include <user/deviceAttachmentSpec.h>
#include <xcbXorg/xcbXorg.h>
#include "xcbWindow.h"
@@ -55,9 +55,9 @@ std::string WindowSelector::stringify() const
}
AttachedWindow::AttachedWindow(
const std::shared_ptr<smo::device::SenseDeviceSpec>& spec
const std::shared_ptr<smo::device::DeviceAttachmentSpec>& spec
)
: deviceSpec(spec)
: deviceAttachmentSpec(spec)
{
// Validate required function pointers are available
if (!xcbXorg.fns.getOrCreateConnection ||
@@ -106,7 +106,7 @@ AttachedWindow::AttachedWindow(
}
void AttachedWindow::parseWindowSelector(
const smo::device::SenseDeviceSpec& spec
const smo::device::DeviceAttachmentSpec& spec
)
{
// Default match type
@@ -148,7 +148,7 @@ void AttachedWindow::parseWindowSelector(
}
}
int AttachedWindow::getRequiredParamAsInt(const smo::device::SenseDeviceSpec& spec,
int AttachedWindow::getRequiredParamAsInt(const smo::device::DeviceAttachmentSpec& spec,
const std::string& paramName)
{
auto it = std::find_if(
@@ -296,7 +296,7 @@ static int xcbWindow_finalizeInd(void)
}
static int xcbWindow_attachDeviceReq(
const std::shared_ptr<smo::device::SenseDeviceSpec>& desc
const std::shared_ptr<smo::device::DeviceAttachmentSpec>& desc
)
{
g_attachedWindows.emplace_back(
@@ -309,12 +309,12 @@ static int xcbWindow_attachDeviceReq(
}
static int xcbWindow_detachDeviceReq(
const std::shared_ptr<smo::device::SenseDeviceSpec>& spec
const std::shared_ptr<smo::device::DeviceAttachmentSpec>& spec
)
{
auto it = std::find_if(g_attachedWindows.begin(), g_attachedWindows.end(),
[&spec](const std::unique_ptr<xcb_window::AttachedWindow>& window) {
return window->getDeviceSpec() == spec;
return window->getDeviceAttachmentSpec() == spec;
}
);
+9 -7
View File
@@ -5,7 +5,7 @@
#include <vector>
#include <memory>
#include <user/senseApiDesc.h>
#include <user/senseDeviceSpec.h>
#include <user/deviceAttachmentSpec.h>
#include <xcbXorg/xcbXorg.h>
namespace xcb_window {
@@ -30,23 +30,25 @@ struct WindowSelector
class AttachedWindow
{
public:
AttachedWindow(const std::shared_ptr<smo::device::SenseDeviceSpec>& spec);
AttachedWindow(
const std::shared_ptr<smo::device::DeviceAttachmentSpec>& spec);
~AttachedWindow();
const std::shared_ptr<smo::device::SenseDeviceSpec>& getDeviceSpec() const
{ return deviceSpec; }
const std::shared_ptr<smo::device::DeviceAttachmentSpec>&
getDeviceAttachmentSpec() const
{ return deviceAttachmentSpec; }
const WindowSelector& getWindowSelector() const { return windowSelector; }
const std::string& getActualWindowName() const { return actualWindowName; }
void* getXcbConnection() const { return xcbConnectionShared.get(); }
std::string stringify() const;
private:
void parseWindowSelector(const smo::device::SenseDeviceSpec& spec);
void parseWindowSelector(const smo::device::DeviceAttachmentSpec& spec);
int getRequiredParamAsInt(
const smo::device::SenseDeviceSpec& spec,
const smo::device::DeviceAttachmentSpec& spec,
const std::string& paramName);
std::shared_ptr<smo::device::SenseDeviceSpec> deviceSpec;
std::shared_ptr<smo::device::DeviceAttachmentSpec> deviceAttachmentSpec;
WindowSelector windowSelector;
std::string actualWindowName;
std::shared_ptr<xcb_xorg::XcbConnection> xcbConnectionShared;
+3 -3
View File
@@ -11,11 +11,11 @@
namespace smo {
namespace device {
std::vector<std::shared_ptr<InteroceptorDeviceSpec>>
std::vector<std::shared_ptr<InteroceptorDevAttachmentSpec>>
DeviceManager::interoceptorDeviceSpecs;
std::vector<std::shared_ptr<ExtrospectorDeviceSpec>>
std::vector<std::shared_ptr<ExtrospectorDevAttachmentSpec>>
DeviceManager::extrospectorDeviceSpecs;
std::vector<std::shared_ptr<SenseDeviceSpec>>
std::vector<std::shared_ptr<DeviceAttachmentSpec>>
DeviceManager::senseDeviceSpecs;
const std::string DeviceManager::stringifyDeviceSpecs(void)
+20 -19
View File
@@ -7,7 +7,7 @@
#include <stdexcept>
#include <memory>
#include <functional>
#include <user/senseDeviceSpec.h>
#include <user/deviceAttachmentSpec.h>
#include <deviceManager/deviceManager.h>
#ifndef yylex
@@ -46,9 +46,9 @@ void yyerror(const char *message)
%union {
char* str;
char chr;
smo::device::SenseDeviceSpec* sensorSpec;
smo::device::InteroceptorDeviceSpec* interoceptorSpec;
smo::device::ExtrospectorDeviceSpec* extrospectorSpec;
smo::device::DeviceAttachmentSpec* sensorSpec;
smo::device::InteroceptorDevAttachmentSpec* interoceptorSpec;
smo::device::ExtrospectorDevAttachmentSpec* extrospectorSpec;
std::vector<std::pair<std::string,std::string>>* paramVector;
std::pair<std::string,std::string>* param;
}
@@ -83,8 +83,8 @@ sensor_spec:
interoceptor_spec:
KEYWORD_SPECTYPE_INTEROSPECTOR PIPE spec_body {
auto spec = std::make_shared<smo::device::InteroceptorDeviceSpec>(
*static_cast<smo::device::InteroceptorDeviceSpec *>($3));
auto spec = std::make_shared<smo::device::InteroceptorDevAttachmentSpec>(
*static_cast<smo::device::InteroceptorDevAttachmentSpec *>($3));
spec->sensorType = $1;
smo::device::DeviceManager::interoceptorDeviceSpecs.push_back(spec);
@@ -96,8 +96,8 @@ interoceptor_spec:
extrospector_spec:
KEYWORD_SPECTYPE_EXTROSPECTOR PIPE spec_body {
auto spec = std::make_shared<smo::device::ExtrospectorDeviceSpec>(
*static_cast<smo::device::ExtrospectorDeviceSpec *>($3));
auto spec = std::make_shared<smo::device::ExtrospectorDevAttachmentSpec>(
*static_cast<smo::device::ExtrospectorDevAttachmentSpec *>($3));
spec->sensorType = $1;
smo::device::DeviceManager::extrospectorDeviceSpecs.push_back(spec);
@@ -108,17 +108,18 @@ extrospector_spec:
;
spec_body:
STRING PIPE STRING LPAREN opt_params RPAREN PIPE STRING LPAREN opt_params RPAREN PIPE STRING {
$$ = new smo::device::SenseDeviceSpec();
$$->sensorType = '\0';
$$->implexor = std::string($1);
$$->api = std::string($3);
$$->apiParams = std::move(*$5);
$$->provider = std::string($8);
$$->providerParams = std::move(*$10);
$$->deviceSelector = std::string($13);
delete $5;
delete $10;
STRING PIPE STRING PIPE STRING LPAREN opt_params RPAREN PIPE STRING LPAREN opt_params RPAREN PIPE STRING {
$$ = new smo::device::DeviceAttachmentSpec();
$$->deviceIdentifier = std::string($1);
$$->sensorType = '\0'; // This will be set by the parent rule
$$->implexor = std::string($3);
$$->api = std::string($5);
$$->apiParams = std::move(*$7);
$$->provider = std::string($10);
$$->providerParams = std::move(*$12);
$$->deviceSelector = std::string($15);
delete $7;
delete $12;
}
;
@@ -7,7 +7,7 @@
#include <opts.h>
#include <utility>
#include <iostream>
#include <user/senseDeviceSpec.h>
#include <user/deviceAttachmentSpec.h>
namespace smo {
namespace device {
@@ -34,11 +34,11 @@ private:
public:
std::string allDeviceSpecs;
static std::vector<std::shared_ptr<InteroceptorDeviceSpec>>
static std::vector<std::shared_ptr<InteroceptorDevAttachmentSpec>>
interoceptorDeviceSpecs;
static std::vector<std::shared_ptr<ExtrospectorDeviceSpec>>
static std::vector<std::shared_ptr<ExtrospectorDevAttachmentSpec>>
extrospectorDeviceSpecs;
static std::vector<std::shared_ptr<SenseDeviceSpec>>
static std::vector<std::shared_ptr<DeviceAttachmentSpec>>
senseDeviceSpecs;
};
+3 -3
View File
@@ -8,7 +8,7 @@
#include <optional>
#include <functional>
#include <senseApis/senseApiLib.h>
#include <user/senseDeviceSpec.h>
#include <user/deviceAttachmentSpec.h>
namespace smo {
namespace sense_api {
@@ -39,9 +39,9 @@ public:
void attachAllSenseDevicesFromSpecs(void);
void attachSenseDevice(
const std::shared_ptr<device::SenseDeviceSpec>& spec);
const std::shared_ptr<device::DeviceAttachmentSpec>& spec);
void detachSenseDevice(
const std::shared_ptr<device::SenseDeviceSpec>& spec);
const std::shared_ptr<device::DeviceAttachmentSpec>& spec);
void detachAllSenseDevices(void);
std::string stringifyLibs() const;
+2 -2
View File
@@ -232,7 +232,7 @@ void SenseApiManager::finalizeAllSenseApiLibs(void)
}
void SenseApiManager::attachSenseDevice(
const std::shared_ptr<device::SenseDeviceSpec>& spec
const std::shared_ptr<device::DeviceAttachmentSpec>& spec
)
{
auto libOpt = getSenseApiLibByApiName(spec->api);
@@ -253,7 +253,7 @@ void SenseApiManager::attachSenseDevice(
}
void SenseApiManager::detachSenseDevice(
const std::shared_ptr<device::SenseDeviceSpec>& spec
const std::shared_ptr<device::DeviceAttachmentSpec>& spec
)
{
auto libOpt = getSenseApiLibByApiName(spec->api);