Synchronized live playback (2026-10-04, evening)
The main monitor now uses a single H.264/AAC HLS player for each selected camera,
with one-second segments and a three-segment browser target buffer. The rendered
lens and audio share a 44.1 kHz sample clock: every 2,205 audio samples determine
one 20 fps visual frame. Physical QuickCam evidence remains at its native capture
rate; the faster rate animates the effect, not additional camera detail.
sol_vision_av.py runs on-demand encoders in separate worker processes. One
worker is shared per camera/presentation mode and stops after 20 seconds without
requests. Queues and generated segment retention are bounded. Camera B captures
1280x720 and uses a 640x360 live preview. Rendering is decoupled from capture so
the USB reader cannot accumulate old frames behind lens work. OpenCV blends the
feathered layer without multiple full-frame NumPy float copies.
Routes are /api/dashboard/vision-av/{primary|secondary}/{public|local|raw}/live.m3u8.
The local and raw routes enforce the existing Vision network classification on
every playlist and segment request; public mode always applies its filter floor.status at the same path reports frame count, media-clock time, rendering cost,
late frames and drops. Main-page sound uses the video's audio track. The older
MP3 and MJPEG endpoints remain compatible clients, but cannot promise browser
alignment when consumed separately.
Fleet output and card updates are now 20 fps, with one-second A/V HLS segments.
Music gain follows Vision availability, independently of QR visibility. Expired
audio chunks are drained rather than replayed behind current video. Record Player
and Living Room use the same A/V source; receiver buffering can differ by room.
Bedroom sink mute and the Stranger Think repeat service are preserved.
Validation includes test_sol_vision_av: decodes a generated segment and checks
both stream types and audio/video PTS alignment. Sustained live tests of both
camera encoders reached 20 fps with no late frames or drops after the native
blend optimization. This verifies encoded timing, not acoustic latency on a
physical phone or television.
Public frontend revision: PR #48, merged as fe05ded, with pinned HLS.js 1.6.13
and its license. Deployment uses the reviewed site sync and a resolved backup.
Earlier dual-camera access and audio (2026-10-04)
The standalone monitor presents A (Record Player QuickCam, upright) and B (keyboard C615), one camera at a time, to local and public viewers. A is the default. Both default to custom filters. Filters-off controls require the server's Vision-only local-access decision; remote raw endpoints reject access. Legacy public grayscale URLs now deliver the filtered upright feed instead.
/api/dashboard/vision-access classifies direct LAN access and public-link access matching the current home IPv4 egress or on-link global IPv6 /64. Network proof refreshes and expires; unrecognized clients remain public. A VPN or Private Relay may prevent recognition. This does not expand Household Operations permissions.
The shared audio player uses /api/dashboard/vision-audio.mp3. Its encoder receives the exact PCM chunks processed by the Rhythmbox beat monitor for both lens effects. No webcam microphones, system-wide monitor, or synthetic audio fallback are used. Missing Rhythmbox audio produces silence. Tap Play to enable playback. The pre-existing fleet card audio also selects only Rhythmbox; household attention cues remain separate.
Validation: 45 unit tests passed; remote raw requests rejected; current home egress recognized via public link; A upright and B keyboard frames inspected; browser B telemetry live; audio endpoint decoded as 44.1 kHz stereo MP3. Audible source verification remains limited while Rhythmbox has no active stream.
The notes below describe earlier iterations and are superseded by this access policy.
SOL Vision Monitor
Document role: authoritative project README
Last implementation audit: August 27, 2026
Status: operational, with public classic-grayscale and full-color Vision Monitor modes
Camera: Logitech QuickCam Messenger Plus (046d:08f6)
Sensor maximum: 324x240 GRBG Bayer
Local presentation: complete 240x324 frame, rotated 90 degrees clockwise
Primary service: local-dashboard-api.service
SOL Vision Monitor is the camera, motion, scene-separation, and music-reactive
visualization system for the Sol-37 workstation. The public dashboard exposes
two explicit presentations: the original unwarped grayscale evidence feed and
the full-color portrait Vision Monitor with tracking labels, screen geometry,
crossing counts, and music-reactive effects. Raw track boxes, stream indices,
and other implementation detail remain local, while a bounded vision_context
object gives dashboard model conversations and summaries the same high-level
scene and effect metadata visible in the monitor.
The colorful pixel ring and dark lens artifact in the current camera image are
also used as source material for a Rhythmbox-reactive visual mode. The effect
warps duplicated camera pixels after analysis, so it cannot create motion,
person, crossing, or screen evidence.
Contents
- Design Principles
- Deployed State
- Architecture
- Physical Frames and Visual Frames
- Tracking and Scene Separation
- Music-Reactive Mode
- Privacy and Endpoints
- Project Components
- Operation
- Configuration
- Telemetry
- Verification
- Tests
- Recovery and Troubleshooting
- Measured Performance
- Current Boundaries
Design Principles
- The sensor frame is evidence. Tracking and scene analysis always consume
the unmodified camera reconstruction.
- Visualization happens after analysis. Music-driven pixel warps and TV-box
pulses never feed back into motion, body, screen, or crossing detectors.
- Raw internals stay local. Exact track boxes, screen quadrilateral points,
body-detector internals, PulseAudio stream indices, and audio samples remain
private loopback/LAN data.
- Public presentation is explicit. The default classic feed is monochrome.
The user-selected Vision Monitor feed is full color and includes the rendered
tracking, count, screen, and music-reactive overlays.
- Model context follows the monitor. Dashboard text-only and multimodal
turns include bounded scene, tracking, retrospective, and music-visual
metadata. Any attached image remains the canonical unwarped evidence frame.
- Candidates are not identities. A body detector can report a cautious
person_candidate; it does not identify a person.
- Crossings require repeated evidence. Moving blobs alone cannot increment
the persistent people-passed count.
- Visual animation is not new camera evidence. Physical frame cadence and
music-only render cadence have separate identifiers.
Deployed State
| Property | Current value |
| --- | --- |
| Camera device | /dev/video0 |
| Raw format | 324x240 GRBG, 8-bit Bayer |
| Local orientation | Full 240x324, 90 degrees clockwise |
| Camera profile | tv-screen |
| Controls | brightness 0, contrast 6, exposure 32, gain 16 |
| Physical-frame target | 6 fps software target; hardware delivery may be slower |
| Music visual target | 15 fps |
| Local page | http://127.0.0.1:8888/programs/sol-vision.html |
| API listener | 127.0.0.1:8896 |
| Dedicated browser class | sol-vision-monitor |
| Dedicated CDP port | 9231 |
| Persistent crossing state | ~/.local/state/sol-vision-camera/tracking.json |
The camera is demand-driven. A page, stream, snapshot request, or status touch
wakes acquisition and extends the interest window. When there is no recent
interest, the manager releases the camera and Rhythmbox monitor rather than
holding both indefinitely.
Architecture
text
/dev/video0: 324x240 GRBG Bayer
|
+--> webcam_feed_v4l2_gray_stream
| raw Bayer frames
|
+--> local_dashboard_api.py
|
+--> canonical tuned color reconstruction
| +--> private color PNG/MJPEG
| +--> public monochrome PNG/MJPEG
| +--> opt-in public full-color Vision Monitor PNG/MJPEG
| +--> bounded local vision-model captions
|
+--> sol_vision_tracking.py
| +--> full-frame 90 degree rotation
| +--> adaptive television quadrilateral
| +--> television mask for room motion
| +--> MOG2 motion objects
| +--> MediaPipe body candidates, HOG fallback
| +--> repeated-evidence crossing counter
|
+--> analyzed frame plus frozen tracking state
|
Rhythmbox sink input --> parec --monitor-stream
| 1024-frame audio windows
| bass/onset and mid/high envelopes
v
sol_vision_music_warp.py
+--> BGRA radial source-pixel remap of orb/ring
+--> independent TV quadrilateral pulse/glow
+--> approximately 15 fps private rerender
|
+--> private tracked PNG/MJPEG with TV-box pulse
+--> dedicated SOL Vision Monitor
+--> approved private Services Desk displays
|
+--> public full-color tracked PNG/MJPEG
+--> persistent quiet-floor radial censor
+--> AI Dashboard and public monitor togglePhysical Frames and Visual Frames
The runtime has two clocks by design.
| Field | Meaning |
| --- | --- |
| frame_id | A new physical camera frame was captured and analyzed |
| tracked_visual_frame_id | The most recent analyzed frame was rerendered with current music state |
| tracked_visual_target_fps | Requested private visualization cadence, currently 15 |
frame_id is the evidence clock. It must not advance merely because music
changed. tracked_visual_frame_id is the presentation clock and can advance
many times while the latest physical frame remains unchanged.
This separation makes the visual effect responsive even when the legacy USB
camera supplies only about one new physical frame per second. It does not
pretend that interpolated or duplicated frames are new observations.
Tracking and Scene Separation
sol_vision_tracking.py rotates the full frame upright before analysis. It
maintains an adaptive quadrilateral for the television and uses the calibrated
ROI only while adaptive acquisition recovers. The television polygon is masked
out of room-motion analysis so changing television content does not become a
moving person or room object.
Room tracking combines:
- OpenCV MOG2 foreground detection
- the local OpenCV Zoo MediaPipe person model
- OpenCV HOG only when the primary model is unavailable
- temporal association and duplicate suppression
- conservative crossing-line hysteresis
A crossing requires repeated scored body evidence across the line. A single
shape, a moving blob, model prose, or retrospective image review cannot change
the count.
Scene captions keep these claims separate:
IN FRAME: physical room contents, expressed as a model hypothesisON SCREEN: television content only, expressed as a model hypothesisMOTION: deterministic local tracker statePEOPLE: deterministic candidate/count state without identification
Music-Reactive Mode
The live audio analyzer listens only to Rhythmbox's current PulseAudio
sink-input. It resolves the live stream index with pactl and captures it withparec --monitor-stream. Other workstation sounds do not drive this effect.
The analyzer uses 1024-frame stereo PCM windows at 44.1 kHz with a requested
20 ms PulseAudio process interval. Its frequency roles are intentionally split:
| Audio feature | Visual role |
| --- | --- |
| Bass energy and onset | Orb and colorful pixel-ring diameter |
| Mid/high transient | TV-box expansion, brightness, glow, and border weight |
| Raw RMS silence gate | Orb collapse and long-silence restoration state |
Orb pixel compositor
The orb is not replaced with a drawn circle. Each render creates an explicit
feathered BGRA duplicate of the camera pixels around the lens artifact and
radially remaps that duplicate with nearest-neighbor interpolation. The
original frame remains the background outside the affected region.
When the orb shrinks below normal size, the vacated annulus is filled by
radially remapping exterior source pixels inward. This prevents the original
full-size orb from remaining visible underneath the smaller duplicate.
Silence state machine
| State | Trigger | Diameter behavior |
| --- | --- | --- |
| music_active | Raw audio exceeds the silence gate | Normal size plus bass/onset expansion |
| quiet_collapsing | Sound has just fallen quiet | Collapse toward minimum over 180 ms |
| quiet_minimum | Continuous silence under 10 seconds | Hold at 0.32x diameter |
| silence_restoring | Silence passes 10 seconds | Restore over 650 ms |
| long_silence_normal | Restoration is complete | Exact unwarped 1.0x size |
If sound resumes during the short quiet period, the next active audio window
returns the orb to its music-driven state. The private monitor follows the
normal ten-second restoration path. The public full-color renderer deliberately
holds silent_for_s below the restoration threshold whenever real Rhythmbox
sound is absent, keeping the strong quiet_minimum 0.32x radial remap as a
persistent low-level censor. This is compositor state, not an audible synthetic
track: no replacement audio is played, exposed, or stored.
Privacy and Endpoints
The API has deliberately different public and local surfaces.
| Endpoint | Content | Access |
| --- | --- | --- |
| /api/dashboard/camera.png | Monochrome snapshot | Public dashboard policy |
| /api/dashboard/camera.mjpg | Monochrome MJPEG | Public dashboard policy |
| /api/dashboard/camera-vision.png | Full-color annotated Vision Monitor snapshot | Public dashboard policy |
| /api/dashboard/camera-vision.mjpg | Full-color annotated Vision Monitor MJPEG | Public dashboard policy |
| /api/dashboard/camera-color.png | Tuned color snapshot | Local only |
| /api/dashboard/camera-color.mjpg | Tuned color MJPEG | Local only |
| /api/dashboard/camera-tracked.png | Annotated, music-reactive snapshot | Local only |
| /api/dashboard/camera-tracked.mjpg | Annotated, music-reactive MJPEG | Local only |
| /api/dashboard/camera-caption | Camera and caption telemetry filtered for access | Policy-filtered |
| /api/dashboard/status | Dashboard state with local tracking detail only for local clients | Policy-filtered |
The raw color and private tracked endpoints still return 403 to a simulated
public client. The public classic grayscale and full-color Vision Monitor
endpoints both remain available while dashboard access is enabled. The
dashboard starts on classic grayscale for first-time public visitors; the
selected mode is stored in that browser.
The Vision renderer rotates the complete reconstructed color frame, applies
the orb/ring remap and independent TV-box pulse, then draws current tracking,
screen-confidence, crossing-line, and count labels. It also applies the public
quiet-floor censor when real music is absent. These presentation pixels are
never submitted to the vision model: captioning and scene analysis continue to
use the canonical unwarped evidence frame.
Public status includes a bounded camera.vision_context summary with split
scene captions, conservative counts, screen confidence, retrospective-model
status, orb phase/scale, music envelopes, screen pulse, and censor-floor state.
It excludes raw audio samples, spectrum arrays, PulseAudio stream indices,
track boxes, exact screen quadrilateral points, and playback identifiers. The
dashboard serializes this object near the beginning of page context so both
ordinary summaries and camera-attached multimodal conversations receive it.
Project Components
| Component | Path | Role |
| --- | --- | --- |
| Monitor page | random/www/programs/sol-vision.html | Dedicated local UI and telemetry |
| Desktop launcher | random/bin/open_sol_vision_monitor | Opens/refreshes isolated Chrome window |
| Camera utility | random/bin/sol_vision_camera.py | Status, profiles, capture, verification, count reset |
| C615 bench utility | random/bin/c615_camera_bench.py | Dedicated manual-control bench for the newer Logitech C615 over UVC/V4L2 |
| C615 bench launcher | random/bin/open_c615_camera_bench | Opens the C615 bench with live preview, presets, snapshots, and metrics |
| V4L2 helper source | random/bin/webcam_feed_v4l2_gray_stream.c | Raw GRBG frame acquisition |
| Dashboard/camera manager | random/bin/local_dashboard_api.py | Device lifecycle, encoders, endpoints, visual rerender loop |
| Tracker | random/bin/sol_vision_tracking.py | Screen, motion, body, and crossing analysis |
| Music visualizer | random/bin/sol_vision_music_warp.py | Rhythmbox audio analysis and pixel compositor |
| Music tests | random/bin/test_sol_vision_music_warp.py | Audio, silence, shrink, and screen-pulse regressions |
| Tracker tests | random/bin/test_sol_vision_tracking.py | Geometry, detection, and crossing regressions |
| Dashboard tests | random/bin/test_sol_vision_dashboard.py | Caption and scene-separation regressions |
| User service | ~/.config/systemd/user/local-dashboard-api.service | Persistent API lifecycle |
Operation
Open or refresh the dedicated monitor
bash
/home/david/random/bin/open_sol_vision_monitorThe launcher uses the dedicated sol-vision-monitor window class, isolated
Chrome state under ~/.local/state/sol-vision-monitor/, and loopback CDP port9231. It must not select a shared Chrome window by title alone.
Inspect camera state
bash
/home/david/random/bin/sol_vision_camera.py status
/home/david/random/bin/sol_vision_camera.py tracking-statusApply or restore a camera profile
bash
/home/david/random/bin/sol_vision_camera.py apply --profile tv-screen
/home/david/random/bin/sol_vision_camera.py apply --profile room-defaultThe deployed service applies tv-screen before startup. room-default is the
original room-biased profile and is not the normal monitor setting.
Capture and verify the television profile
bash
/home/david/random/bin/sol_vision_camera.py verifyVerification writes a PNG and JSON report under~/.local/state/sol-vision-camera/. A passing clear_screen_candidate means
recoverable screen structure, not readable text.
Reset crossing counts
bash
/home/david/random/bin/sol_vision_camera.py reset-countsThis is destructive persistent-state mutation. Run it only when a reset was
explicitly requested.
Open the newer C615 control bench
bash
/home/david/random/bin/open_c615_camera_benchThe bench is intentionally separate from the deployed Vision Monitor stack. It
targets /dev/video1 by default and adds:
- one-click focus presets for autofocus, near, desk, and far
- manual exposure and gain controls with a motion guard that can defer changes
during transient close-range movement
- white-balance presets and manual temperature lock
- PTZ save/recall slots when the camera exposes those controls
- live preview metrics including measured FPS, frame-interval jitter, luma
histogram, focus score, clipping warnings, and motion level
- persistent full-state snapshots plus instant scene profiles such as
close-up electronics, desk, room, and low-light
Persistent bench state is stored under ~/.local/state/c615-camera-bench/.
Configuration
The main runtime variables are:
| Variable | Default | Meaning |
| --- | ---: | --- |
| LOCAL_DASHBOARD_CAMERA_DEVICE | /dev/video0 | V4L2 camera |
| LOCAL_DASHBOARD_CAMERA_WIDTH | 324 | Raw sensor width |
| LOCAL_DASHBOARD_CAMERA_HEIGHT | 240 | Raw sensor height |
| LOCAL_DASHBOARD_CAMERA_FPS | 6 | Physical processing target, not added resolution or guaranteed hardware rate |
| LOCAL_DASHBOARD_CAMERA_MUSIC_VISUAL_FPS | 15 | Private and public Vision rerender target, clamped to 5-30 |
| LOCAL_DASHBOARD_CAMERA_IDLE_TIMEOUT | 30 | Seconds to retain camera interest |
| LOCAL_DASHBOARD_CAMERA_VIEWER_STALE | 12 | Viewer stale threshold |
| SOL_VISION_BODY_DETECTION_INTERVAL | 6 | Frames between body-model runs, minimum 3 |
| SOL_VISION_SCREEN_DETECTION_INTERVAL | 3 | Frames between adaptive screen searches |
| SOL_VISION_TRACKING_STATE | ~/.local/state/sol-vision-camera/tracking.json | Persistent crossing state |
| SOL_VISION_PERSON_MODEL | ~/.cache/sol-vision-models/person_detection_mediapipe_2023mar.onnx | Primary body model |
Use a systemd user-service drop-in for durable overrides. After changing a
drop-in or implementation file:
bash
systemctl --user daemon-reload
systemctl --user restart local-dashboard-api.service
/home/david/random/bin/open_sol_vision_monitorRestart only the dashboard API. Do not broadly restart unrelated Sol services.
Telemetry
The main private music telemetry is under camera.tracking.music_warp:
json
{
"phase": "music_active",
"scale": 1.18,
"minimum_scale": 0.32,
"silent_for_s": 0.0,
"audio": {
"active": true,
"fresh": true,
"source": "rhythmbox_sink_input",
"sound_active": true,
"beat": 0.78,
"screen_beat": 0.63,
"age_s": 0.01,
"error": ""
},
"screen_overlay": {
"band_role": "mid_high_transient",
"pulse": 0.63,
"scale": 1.02,
"border_thickness_px": 4
}
}The dashboard-safe companion is camera.vision_context. It includes the
current scene_assessment, bounded tracking counts and screen confidence,music_visual phase/envelopes/pulse/censor state, and the non-countingretrospective_person_inference. It intentionally omits raw track geometry,
audio samples, spectra, and PulseAudio identifiers.
Useful health conditions are:
camera.frame_fresh=truecamera.camera_error=""tracking.body_detector.available=truetracking.body_detector.model_error=""tracking.music_warp.audio.fresh=truewhile Rhythmbox is presenttracking.music_warp.audio.error=""tracked_visual_frame_idadvances faster thanframe_idduring active viewing
Verification
Service, controls, and access boundary
bash
systemctl --user is-active \
local-dashboard-api.service \
caddy-sol37.service \
dashboard-access-daemon.service
v4l2-ctl -d /dev/video0 \
--get-ctrl=brightness,contrast,exposure,gain
curl -sS -o /dev/null -w '%{http_code} %{content_type}\n' \
http://127.0.0.1:8888/api/dashboard/camera-tracked.png
curl -sS -H 'CF-Connecting-IP: 8.8.8.8' \
-o /dev/null -w '%{http_code} %{content_type}\n' \
http://127.0.0.1:8896/api/dashboard/camera-tracked.png
curl -sS -H 'CF-Connecting-IP: 8.8.8.8' \
-o /dev/null -w '%{http_code} %{content_type}\n' \
http://127.0.0.1:8896/api/dashboard/camera.png
curl -sS -H 'CF-Connecting-IP: 8.8.8.8' \
-o /dev/null -w '%{http_code} %{content_type}\n' \
http://127.0.0.1:8896/api/dashboard/camera-vision.png
Expected results:
- local tracked:
200 image/png - simulated-public tracked:
403 application/json - simulated-public monochrome:
200 image/png - simulated-public full-color Vision Monitor:
200 image/png
Music state
bash
curl -sS 'http://127.0.0.1:8896/api/dashboard/status?touch=1' | \
jq '.camera | {
frame_id,
tracked_visual_frame_id,
tracked_visual_target_fps,
music_warp: .tracking.music_warp
}'Dedicated window
bash
wmctrl -lx | rg 'sol-vision-monitor'
curl -sS http://127.0.0.1:9231/json | \
jq '[.[] | select(.type == "page") | {title, url}]'A healthy state has exactly one dedicated page titled SOL Vision Monitor at
the local program URL.
Tests
Run the complete Vision Monitor regression suite from the runtime directory:
bash
cd /home/david/random/bin
python3 -m unittest -v \
test_sol_vision_music_warp \
test_sol_vision_tracking \
test_sol_vision_dashboardThe suite covers Rhythmbox stream selection, audio onset/release, raw silence
duration, 0.32x quiet shrink, long-silence restoration, independent TV-box
pulsing, adaptive screen geometry, television-motion exclusion, body evidence,
duplicate suppression, crossing gates, caption parsing, and retrospective
model boundaries.
Recovery and Troubleshooting
Monitor page is open but the feed is stale
bash
systemctl --user status local-dashboard-api.service --no-pager -l
journalctl --user -u local-dashboard-api.service -n 80 --no-pager
/home/david/random/bin/sol_vision_camera.py statusConfirm /dev/video0 exists and that the dashboard helper owns the active
capture. Restart only local-dashboard-api.service, then refresh the dedicated
monitor.
Camera is fresh but music does not react
bash
pactl -f json list sink-inputs | \
jq '[.[] | {index, corked, application: .properties."application.name"}]'
pgrep -a -f 'parec.SOLVisionMusicWarp'
Rhythmbox must have a live sink-input. The visual telemetry should showsource="rhythmbox_sink_input", a current stream index, fresh=true, and an
empty error. The monitor automatically re-resolves the index when Rhythmbox
creates a replacement stream.
Orb becomes small during a pause
This is expected for the first 10 seconds of continuous silence. It should
report quiet_minimum at 0.32x. After the ten-second threshold it transitions
through silence_restoring and returns to long_silence_normal at 1.0x.
Music reacts but looks slow
Compare frame_id and tracked_visual_frame_id. The visual ID should advance
at roughly 10-15 fps even if the physical ID advances near 1 fps. If both IDs
advance only at the physical cadence, inspect the service journal fordashboard-music-visual-loop render errors and confirmLOCAL_DASHBOARD_CAMERA_MUSIC_VISUAL_FPS was not overridden downward.
Raw color or private tracking returns 403 publicly
That remains expected for /camera-color. and /camera-tracked.. Use/camera-vision. for the deliberately published full-color annotated view;
the raw/private routes remain loopback/LAN-only.
Measured Performance
The following is a dated workstation observation, not a hardware guarantee:
| Measurement | August 26, 2026 observation |
| --- | ---: |
| Physical camera delivery | About 1.0 fps |
| Private tracked delivery before decoupling | About 1.0 fps |
| Private tracked delivery after decoupling | About 11.7 fps |
| Median tracked interval | About 80 ms |
| Configured visual target | 15 fps |
| Rhythmbox envelope age during playback | About 6-11 ms in sampled status |
| Dashboard process CPU after PNG throttling | About 68% of one CPU |
| Package temperature during verification | About 60 C |
The service remains pinned to background cores 0-2 and 6-8, leaving the
desktop-reserved foreground cores unchanged.
Current Boundaries
- The hardware maximum is 324x240; browser enlargement is not added detail.
- Bayer color is reconstructed and approximate. Weak chroma cues may be wrong.
- A passing screen verification means visible structure, not OCR-grade text.
- person_candidate is uncertain body evidence, never identity.
- Retrospective model review cannot create a live candidate or crossing.
- The visual rerender can be fast while physical scene evidence remains slow.
- Rhythmbox audio is analyzed in memory for envelopes; samples are not exposed
through the dashboard or stored by this feature.
- The orb effect manipulates existing camera pixels. It is an intentional
presentation layer, not a claim about a physical object changing size.
Household detection alignment — September 13, 2026
Vision and Services Desk now share bin/sol_vision_presence.py rather than
maintaining separate meaningful-activity rules. Each physical frame qualifies
repeated body evidence first, then persistent localized motion (three track hits,
12 pixels of travel, bounded object area, two fresh qualifying samples).
Screen-overlapping tracks are excluded; cached body detections are rechecked
against the current screen mask before association. No identity is inferred.
tracking.meaningful_activity reports the qualified signal separately from rawmotion_active and object counts. The monitor shows this qualified signal and
marks stale capture unknown. Public context contains only status, active, and
trigger, not geometry or household records. Existing crossing counts and quiet
reference refresh remain independent. Sound/flash cooldown (10 seconds) and QR
quiet return (600 seconds) remain display policies, not detection delays.
False-positive correction — September 13
Repeated Record Player person alerts exposed a static room-feature hypothesis.
Cached body results no longer increment independent person evidence on each
camera frame. Body candidates must change at least 20% of their unmasked box
relative to the quiet reference (24-level threshold, ambient-offset compensated).reference_rejected_people records the current-frame rejection count. The
10-second attention cooldown is unchanged. Physical walk-past sensitivity must
be checked after this conservative calibration; stationary scene hypotheses
are not sufficient to flash the display.
QR and arrival correction (latest September 13 request)
Meaningful arrival now requires corroborating movement for person hypotheses,
as well as fallback motion. Stationary body-shaped patterns cannot trigger cues.
QR presentation is separate: one arrival hides it for 30 seconds, then fades
it in over 30 seconds. A fixed 600-second lockout from the initial hide ignores
further motion. Continuous presence cannot hide it again at expiry; a fresh
arrival is required. Camera outages do not extend timers. Version-3 migration
restores the card immediately with a full lockout. Flash/chime remains 10 seconds.
Shared floating-card timing
Record Player now floats both QR and private tracked Vision cards. Both use one
opacity and timer: 30 seconds hidden, 30-second fade-in, 600-second disappearance
lockout starting at initial activity. The persistent dashboard is unaffected.
Vision uses a bounded background snapshot reader, preserving camera aspect ratio
and keeping network waits out of the HLS encoder. Stale frames are labeled.
Visible-card Rhythmbox playback
The Record Player private A/V compositor captures only the Rhythmbox PulseAudio
sink input selected by the existing music-visual selector. Audio is mixed with
the existing chime and minute tick at 45% gain, follows shared card opacity,
and is muted for hidden or stale Vision frames. A bounded background PCM queue
is continuously drained, preventing hidden-period replay or encoder blocking.
Telemetry exposes capture availability, gain, and errors without audio samples
or source IDs. HLS adds its normal playback latency to both card and audio.
Card effect latency optimization
Record Player consumes private tracked MJPEG continuously instead of polling
4-fps PNG output twice per second. The compositor and PCM blocks now share
a 12-fps clock (4000 samples at 48 kHz), with 24-frame GOPs preserving two-second
HLS segments. Only the newest Vision frame is retained. Audio buffering is
bounded to three 83-ms chunks, with stale samples rejected after 300 ms.
The existing receiver buffer remains; improved animation cadence does not
eliminate HLS transport latency. Verify physical perceived timing separately.
Close-camera traversal
A separate close-foreground pathway handles near-camera obstruction that cannot
satisfy small-object area/centroid-travel gates. It requires at least 55% of
unmasked room pixels to differ by 30 levels after removing median brightness
offset, for two fresh frames. Screen-only activity cannot qualify. This is
motion evidence, never a person identity or crossing count. Preserve existing
attention and shared-card cooldowns. Physical walk-by verification is pending.
Standing-test calibration correction
Actual captured frames showed close foreground despite low absolute differences
in the dark scene. A manually verified clear post-test image replaced the stale
reference. Close foreground additionally requires >=15% room residual >=18 and
room correlation <0.85 after brightness compensation. Recorded occupied samples
0/4/9/10 qualified; clear samples 12/15/25/34 did not. This replay calibration
is not a subsequent physical cue verification. Screen exclusion remains active.
Brief walk-by gate
The calibrated close-foreground path accepts one qualifying physical frame,
because the legacy camera delivers approximately one frame per second and a
brief traversal may not span two frames. Ordinary track-based motion still
requires repeated evidence. This supersedes the earlier two-frame requirement
for close foreground only; cooldowns and stationary-pattern rejection remain.
Dual-camera source contract (October 4, 2026)
The original QuickCam (046d:08f6) is the fixed default for every existing
primary SOL Vision endpoint and model context. The floating card is mostly primary with a bounded secondary view. Its stable
path is /dev/v4l/by-id/usb-046d_Camera-video-index0. The main API and tuning
utility share sol_vision_sources.PRIMARY_DEVICE. Missing hardware produces an
unavailable state; reconnecting another camera never replaces the primary.
The standalone /programs/sol-vision.html starts on the original camera tab.
Its additional HD C615 tab uses an independent SecondaryCamera capture owner
and local-only /api/dashboard/cameras/c615/{status,snapshot.jpg,stream.mjpg}
endpoints. It captures 1280x720 color at 10 fps, retries reconnections, and releases
the device after 15 seconds without interest. Selecting this tab does not mutate
the primary camera, fleet, tracking totals, or model context. The additional
camera has optional conservative screen candidates with no fallback polygon; it never inherits primary geometry or people counts. Its geometry runs before the music warp. Both cameras share the existing Rhythmbox analyzer and radial lens distortion; screen pulses only appear on an actual calibrated primary screen or a secondary candidate.
The private /api/dashboard/camera-fleet.png, .mjpg, and /camera-fleet/status
paths compose a 240x324 card: 50 seconds primary, up to 10 seconds secondary
per minute. A new primary motion/body activity signal returns to primary for
10 seconds. Sustained activity does not renew that presentation hold indefinitely;
the second camera still gets subsequent slots while motion detection continues.
The secondary camera prewarms even when a hold overlaps its slot. Each slot
displays exactly one camera, with its hardware-specific label; no inset. The
Vision card stays visible independently of QR fading.
Missing/stale secondary capture skips its slot; missing primary capture shows
primary unavailable rather than silently promoting the keyboard view. Original
motion detection and attention remain active through both presentation phases.
Both fleet compositor paths consume this dedicated card feed; standalone tabs
remain independent and default to primary. Household Operations HTML and data
are unchanged. Primary calibration is private in attention-reference.json,
with cameraId=primary; the October 4 calibration is based on fresh unwarped
screen corners, and is never read by the C615 detector.
Camera identity confirmed by David against direct, unwarped USB captures on October 4: A / QuickCam / 046d:08f6 is Record Player; B / C615 / 046d:082c is the keyboard. Card labels and standalone tabs use this mapping. Each card frame contains exactly one camera.
The active original-camera profile is now room-screen: brightness 3, contrast 8, exposure 256, gain 64, with LOCAL_DASHBOARD_CAMERA_SHADOW_GAMMA=0.65 for shadow visibility. The earlier tv-screen profile remains available for glare-heavy placement. Exposure and shadow lift were visually checked against the screen and surrounding room; this is brightness tuning, not added sensor resolution.