How a player or app uses adsvc is in Client integration; how the catalogue, and sync work is in the design notes.
u is the upstream URL (percent-encoded). ih/idx (or an opaque id) are optional and
only used as the file key. TorrServer URLs (/stream?link=…&index=…, /play/<ih>/<n>) are
recognised on their own. Otherwise the file is keyed by its URL without tokens, and then
by a hash of its own bytes once both ends of the file have passed through.
adsvc analyses the stream and reports what it finds; the player decides what to do with it.
The mpv client, mpv/adskip.lua, acts on them: press a at the start of an ad, or t at the start
of an intro, and A at its end to enroll it (, and . step frames). The fingerprint is taken
from what the proxy already analysed, so nothing is read again. From then on the confirmed
findings are acted on when playback reaches them (Ctrl+a toggles this).
Every reference is an ad or an intro. An intro is a title sequence that repeats in each
episode of a series; there is no type for credits. A type is
chosen when the reference is enrolled (type= of /ads/mark), and can be changed on the
Ad page.
The data directory holds catalogue.db (SQLite: the ads and the per-file ad maps),
tracking.csr (the ad index, rebuilt when the ads change) and identity.key. Ads are
identified by the hash of their fingerprint (a UUID-shaped id, the same on every machine).
The catalogue holds no secrets, URLs or file names; labels are whatever you type (an ad
marked without one is ad-<short id>, or intro-<short id>).
adsvc -media ~/Videos # proxy is the default command
The folder is served read-only on localhost:8000 (-media-listen, loopback only) and
added to the upstream allow-list. Open http://localhost:8000/ in a browser: every file
has a play URL to give the player, such as
http://localhost:8080/s?u=http://localhost:8000/episode1.mkv. With users configured, put
NAME:TOKEN@ after http://. Hidden files and symlinks that lead out of the folder are
not served. Keep the folder and address the same between runs: the play URL is the file’s
key until its content hash is known.
Bind to localhost (the default), or protect adsvc with users and restrict sources with
-route / -allow. Otherwise it will fetch any URL anyone asks it for.
adsvc proxy reads config.yml in the current directory when it exists, or the file named
by -config or $ADSVC_CONFIG; see config.yml.example for every
key. Flags and the environment win over the file. Users come from the file only. Unknown
keys are errors.
auth:
users:
alice:
tokens: [9vW3…] # any number; each belongs to one user
admin: true # its tokens also log in to the web page's admin pages
tv-box:
tokens: [Qm7c…]
A request carries Authorization: Bearer TOKEN, or the user and token in the URL:
http://alice:TOKEN@host:8080/s?u=…. Without a user that has a token, adsvc is open to
anyone who can reach it. adsvc token prints a new random token; add it to a user’s
tokens and reload.
kill -HUP <pid> reloads the file. Auth, routes, the allow-list, max_stall,
idle_timeout, confirm_score and logging apply at once; min_score, mark_window and
ffmpeg apply to new sessions; listen, data_dir, media and metrics need a restart. An invalid file
changes nothing and is reported in the log. Removing a token revokes it and ends web
sessions opened with it.
Routes accept id, ih+idx (aliases hash, index), u or key. With users
configured, all but /healthz need a token. Times are integer milliseconds on the
player’s media timeline.
| Route | Purpose |
|---|---|
GET /s?u=… |
the media itself, forwarded byte for byte |
GET /ads/file |
{key, aliases, container, size, durationMs, analysedToMs, analyzed: [[fromMs,toMs]…], ads: [{adId,label,type,startMs,endMs,score,confirmed}]} |
POST /ads/mark?startMs=S&endMs=E&label=L&type=ad\|intro |
enroll [S,E) of this file as an ad (the default) or an intro, from what was already analysed |
POST /ads/report?u=…&position=MS |
report an ad seen at MS, to mark it later |
GET /ads/library |
the reference ads |
GET /ads/maps |
stored per-file ad maps |
GET /healthz |
200 adsvc <version>, no auth |
type is ad or intro. A player that acts on detections automatically
should use only confirmed: true ones. A detection is confirmed once its whole interval
has been analysed, or when two consecutive blocks agree on the same alignment with a score
of at least confirm_score (default 2× min_score).
Nodes share ads, labels, votes and file maps as signed records. Each node has an ed25519
identity; a node decides for itself what to trust (trust.*: weights per node id, blocks,
quotas). Peers pull each other’s logs (sync.peers); a node behind NAT pushes to a
reachable one. /sync/* needs a user’s token unless sync.public: true.
adsvc identity # give this id to your peers
adsvc export -o snapshot.db # every record: a file anyone may have
adsvc import snapshot.db # bootstrap from someone's snapshot
Snapshots hold no secrets, URLs or file names, and never include the node’s own file maps.
Logs go to stderr (log/slog), command results to stdout. -log-level debug|info|warn|error
(or $ADSVC_LOG_LEVEL, log.level), -log-format text|json (or $ADSVC_LOG_FORMAT,
log.format); proxy -v is -log-level debug. Upstream URLs are logged with credentials
and signed-URL parameters redacted.
-metrics-listen localhost:9464 (or metrics.listen) serves Prometheus metrics at
/metrics on a port of its own, without auth. It is off by default. An address without a
host means localhost, so a scraper on another machine or container needs an explicit host
(0.0.0.0:9464). The metrics hold no URLs, file keys, tokens or user names; sync metrics
are labelled with the peer names from the config.
scrape_configs:
- job_name: adsvc
static_configs: [{targets: ["localhost:9464"]}]
| Area | Metrics |
|---|---|
| Build, process | adsvc_build_info, adsvc_config_reloads_total{result}, process_cpu_seconds_total, process_start_time_seconds; on Linux and Android also process_resident_memory_bytes (includes the mapped ad index; the ffmpeg processes are separate), process_virtual_memory_bytes, process_open_fds |
| Go runtime | go_goroutines, go_threads, go_memstats_heap_alloc_bytes, go_memstats_heap_inuse_bytes, go_memstats_sys_bytes, go_memstats_next_gc_bytes, go_gc_gomemlimit_bytes, go_gc_gogc_percent, go_gc_cycles_total, go_gc_pauses_seconds (a histogram; client_golang’s go_gc_duration_seconds is a summary, so it is not reused), go_sched_latencies_seconds |
| HTTP | adsvc_http_requests_total{handler,method,code}, adsvc_http_request_duration_seconds{handler} (not for streams), adsvc_http_requests_in_flight{handler} (for stream: open player connections). handler is stream, file, mark, report, reports, library, maps, healthz, admin, sync or other |
| Sources | adsvc_upstream_requests_total{result=2xx\|3xx\|4xx\|5xx\|error}, adsvc_upstream_response_seconds (time to headers), adsvc_stream_bytes_total, adsvc_stream_copy_errors_total (mostly players hanging up) |
| Sessions | adsvc_sessions_active, adsvc_sessions_opened_total{kind}, adsvc_sessions_closed_total{reason}, adsvc_containers_total{type}, adsvc_analysis_disabled_total{reason=unknown_container\|open_failed\|demux_error\|panic} |
| Keeping up | adsvc_analyser_queue_bytes, adsvc_analyser_stall_seconds_total (players waiting, up to max_stall), adsvc_analyser_skip_ahead_total and adsvc_analyser_dropped_bytes_total (gaps in coverage), adsvc_analysed_media_seconds_total |
| Tracking | adsvc_fingerprint_duration_seconds and adsvc_match_duration_seconds (per 10 s block), adsvc_detections_total, adsvc_detection_score, adsvc_confirmations_total, adsvc_file_map_requests_total{result=hit\|miss}, adsvc_marks_total{result}, adsvc_reports_total |
| Decoder | adsvc_ffmpeg_running, adsvc_ffmpeg_runs_total{result=ok\|failed\|start_failed\|read_error\|shutdown}, adsvc_ffmpeg_run_seconds, adsvc_decoder_restarts_total, adsvc_decoder_idle_frames_total (skipped: analysed before), adsvc_pcm_ring_bytes |
| Ad index | adsvc_library_ads, _index_ads, _postings, _overlay_ads, _overlay_postings, _dead_ads, _dead_postings, _mapped_bytes, _heap_bytes, _version, _last_build_timestamp_seconds, adsvc_library_builds_total{result}, adsvc_library_build_duration_seconds |
| Catalogue | adsvc_catalog_write_duration_seconds, adsvc_catalog_db_wait_seconds_total{pool}, adsvc_catalog_db_in_use{pool}, adsvc_catalog_db_bytes, adsvc_records_ingested_total{result,reason} |
| Sync | adsvc_sync_peers, adsvc_sync_rounds_total{peer,result}, adsvc_sync_round_duration_seconds{peer}, adsvc_sync_last_success_timestamp_seconds{peer}, adsvc_sync_pull_lag_records{peer}, adsvc_sync_records_total{peer,direction} |
Useful queries and alerts:
# sources failing (over 5% of requests)
sum(rate(adsvc_upstream_requests_total{result=~"error|5xx"}[5m])) / sum(rate(adsvc_upstream_requests_total[5m])) > 0.05
# the analyser cannot keep up with playback: coverage gaps for 10 minutes
rate(adsvc_analyser_skip_ahead_total[10m]) > 0
# media seconds analysed per second of wall time, per open stream (>= 1 keeps up)
rate(adsvc_analysed_media_seconds_total[5m]) / clamp_min(adsvc_http_requests_in_flight{handler="stream"}, 1)
# ffmpeg runs failing: often a codec missing from the ffmpeg build
rate(adsvc_ffmpeg_runs_total{result=~"failed|start_failed"}[10m]) > 0
# a demuxer crashed on some file
increase(adsvc_analysis_disabled_total{reason="panic"}[1h]) > 0
# matching slows down (index too big for the box)
histogram_quantile(0.99, rate(adsvc_match_duration_seconds_bucket[5m])) > 0.05
# a peer has not synced for three intervals (10 min default)
time() - adsvc_sync_last_success_timestamp_seconds > 1800
# close to the memory limit
go_memstats_heap_inuse_bytes / go_gc_gomemlimit_bytes > 0.9