Docs: Add deviceSpec specification

Currently only has a spec for sensors. We'll create and add a spec for actuators later.
2025-01-05 16:51:12 -04:00
parent 6307d2869e
commit ef5022f815
1 changed files with 114 additions and 0 deletions
@@ -0,0 +1,114 @@
+# Device manager: attaching sensors and actuators.
+
+## Attaching sensors:
+
+Sensors are input devices to Harikoff. Harikoff will perceive them as perceptual
+inputs -- like your own sense organs. For example, if you attach a camera as a
+sensor, harikoff will experience it in the same way that you experience the
+visual sense data from your eyes.
+
+## Implexors:
+
+An implexor is an __Imp__licit __Ex__istent isolat__Or__ algorithm. It's
+basically what conventional ML/LLM/ANN developers call an ROI ("Region of
+Interest") extraction algorithm. An Implex algorithm is used to scan a frame of
+input sensor data and detect objects and patterns within it.
+
+The general format of a device-spec for a sensor is:
+```
+implexor|api|server(server-params)|deviceselector
+```
+
+* `implexor` is the name of the implexor algorithm that should be used with
+  the data that is provided by the `server` via the `api`.
+* `api` is the interface that the server uses to export perceptual data for
+  harikoff to read. Harikoff will run the `implexor` algorithm on the data from
+  this `api`.
+* `server` may be a userspace daemon or an OS kernel that provides perceptual
+  data via the `api`.
+* `device selector` is the idiosyncratic label/name used by the `server` to
+  identify the specific device you want to access via that `server`.
+
+Some examples follow:
+
+### To attach a particular window from a window manager:
+```
+visual-implexor|wayland|wayland(server-socket)|window0
+```
+Connect to the Wayland server that's listening on `server-socket`, using
+the `wayland` api. Ask that Wayland server to give harikoff read-access to all
+of the frames composited into the window buffer for `window0`. Use harikoff's
+`visual-implexor` to implex from that `window0`'s compositor data.
+
+### To attach a window manager's entire rendered desktop:
+```
+visual-implexor|wayland|wayland(listen-socket)|all
+```
+In most cases, this is basically the same as attempting to attach all of the
+underlying GFX server's output.
+
+Connect to the Wayland server that's listening on `listen-socket`, using
+the `wayland` api. Ask that Wayland server to give harikoff read-access to the
+entire compositor framebuffer. Use harikoff's `visual-implexor` to implex
+from that Wayland server's compositor data.
+
+### To attach all of an Xorg server's gfx output to all screens:
+```
+visual-implexor|x11|xorg(listen-socket)|all
+```
+
+Connect to the Xorg server that's listening on `listen-socket`, using the `x11`
+api. Ask that Xorg server to let Harikoff read out all of the frames written out
+to all screens. Use harikoff's `visual-implexor` to implex from the server's
+gfx framebuffer data.
+
+In most cases, this is basically the same as attempting to attach all of the
+WM's output.
+
+### To attach all of an Xorg server's gfx output to a particular screen:
+```
+visual-implexor|x11|xorg(listen-socket)|:0
+```
+Connect to the Xorg server that's listening on `listen-socket`, using the `x11`
+api. Ask that Xorg server to let Harikoff read out all of the frames written out
+to display `:0`. Use harikoff's `visual-implexor` to implex from display
+`:0`'s framebuffer data.
+
+### To attach a camera device by connecting directly to its Linux driver:
+```
+visual-implexor|linux|v4l|/dev/video0
+```
+We specify that we want to use the `linux` kernel's loaded driver to connect
+to communicate with `/dev/video0`, via the `Video4Linux` API. We want harikoff
+to use the `visual-implexor` algorithm to implex from `/dev/video0`'s data.
+
+If `/dev/video0` is already consumed by another process, this may likely fail.
+
+### To attach a microphone that's managed by ALSA server:
+```
+audio-implexor|alsa|alsa(shmem)|cardname
+```
+
+Connect to the ALSA server via `shmem`, using the `alsa` API. Request access to
+the microphone function of the sound card with the name `cardname`. Use the
+`audio-implexor` algorithm to implex from `cardname`'s microphone data.
+
+## Attaching actuators:
+
+Actuators are Harikoff's way of enacting changes in the external world. They're
+like your libs, or your mouth. Actuators enable harikoff to write outputs to the
+world outside.
+
+### Wilzors: 
+
+Actuator devices are analogous to your body's limbs. Harikoff controls these by
+using `wilzor` algorithms. Wilzor is a contraction of __Wil__lpower
+Actuat__Or__ but with a 'Z' in the middle to make it sound cooler. Different
+types of devices will require different wilzor algorithms. You need to know
+what type of wilzor algorithm needs to be used to enable harikoff to control
+your actuator device.
+
+The general format for an actuator's device spec is:
+```
+WIP: TBD.
+```