feat: track currently watching videos

This commit is contained in:
2026-10-12 02:05:35 +02:00
parent 28c94eb70b
commit a6557047fe
4 changed files with 857 additions and 66 deletions
+19 -7
View File
@@ -78,11 +78,12 @@ These are the default bindings. Overrides in `[keys]` replace an action's bindin
| `W` | Choose combined or category watch later from any video view or preview | | `W` | Choose combined or category watch later from any video view or preview |
| `Alt-w` | Open the current category's watch later | | `Alt-w` | Open the current category's watch later |
| `f` | In watch later, cycle All → Unwatched → Watched | | `f` | In watch later, cycle All → Unwatched → Watched |
| `x` | In watch-later or saved-video lists, remove the selected video | | `x` | In watch-later, saved-video, or currently-watching lists, remove the selected video |
| `X` (Shift-X) | In either collection list, confirm bulk removal of the current category/filter subset | | `X` (Shift-X) | In collection lists, confirm bulk removal of the current list/category/filter subset |
| `v` | Save the selected video to saved videos (also works in previews) | | `v` | Save the selected video to saved videos (also works in previews) |
| `V` | Choose combined or category saved videos from any video view or preview | | `V` | Choose combined or category saved videos from any video view or preview |
| `Alt-v` | Open the current category's saved videos | | `Alt-v` | Open the current category's saved videos |
| `C` (Shift-C) | Open the combined Currently watching list |
| Tab / Shift-Tab | Switch between visible views | | Tab / Shift-Tab | Switch between visible views |
| `r` | Reload configuration and refresh the current view | | `r` | Reload configuration and refresh the current view |
| `?` | Open / close the scrollable keybinding reference | | `?` | Open / close the scrollable keybinding reference |
@@ -231,8 +232,9 @@ Files are siblings of `config.toml`:
- `watch-later.toml`: versioned saved-video metadata. - `watch-later.toml`: versioned saved-video metadata.
- `saved-videos.toml`: independent versioned saved-video metadata. - `saved-videos.toml`: independent versioned saved-video metadata.
- `currently-watching.toml`: versioned metadata for started, unfinished videos.
- `watch-state.toml`: versioned global watched video IDs. - `watch-state.toml`: versioned global watched video IDs.
- `playback-sessions/`: private per-launch progress streams and their expected video identities. - `playback-sessions/`: private per-launch progress streams, expected video identities, video snapshots, and durable admission markers.
State writes are atomic and owner-only. Invalid state files or detected external edits are reported rather than overwritten; repair the file or restart before retrying. Each state file is limited to 8 MiB. State writes are atomic and owner-only. Invalid state files or detected external edits are reported rather than overwritten; repair the file or restart before retrying. Each state file is limited to 8 MiB.
@@ -250,15 +252,25 @@ Saved titles are **green**, taking precedence over watch-later magenta when a vi
The collection is persisted independently in `saved-videos.toml` beside `config.toml`, using the same atomic, owner-only writes and validation as watch later. Saving repeatedly does not add duplicates. Configure saving and the collection selector with `video.saved.save` and `view.saved.browse`; direct category browsing uses `view.saved.category`. The corresponding watch-later navigation actions are `view.watch_later.browse` and `view.watch_later.category`. Selectors use the shared `selector.*` and `input.*` bindings. The live hints and `?` reference reflect overrides. Alt shortcuts require the terminal to transmit Alt/Meta; if macOS Option inserts symbols, enable Option-as-Meta or remap the actions. The collection is persisted independently in `saved-videos.toml` beside `config.toml`, using the same atomic, owner-only writes and validation as watch later. Saving repeatedly does not add duplicates. Configure saving and the collection selector with `video.saved.save` and `view.saved.browse`; direct category browsing uses `view.saved.category`. The corresponding watch-later navigation actions are `view.watch_later.browse` and `view.watch_later.category`. Selectors use the shared `selector.*` and `input.*` bindings. The live hints and `?` reference reflect overrides. Alt shortcuts require the terminal to transmit Alt/Meta; if macOS Option inserts symbols, enable Option-as-Meta or remap the actions.
## Currently watching
`C` opens one combined, offline list of started but unfinished videos. It uses the shared temporary tab and back/forward navigation; configure the shortcut with `view.watching.browse`.
A validated, acknowledged **positive playback position** adds the video—not merely launching Terebi or receiving position zero. Reaching the existing eligible **95% watched threshold** automatically removes it. Already-watched videos never enter. Unknown-duration or ineligible playback can appear but stays unfinished unless a later eligible report marks it watched or you remove it.
Entries and their title/creator/publication metadata survive restarts. Reports written while Tubular is closed are recovered on the next launch. Older sessions without video snapshots can populate the list when matching watch-later/saved metadata exists; otherwise they continue updating watched status without inventing missing metadata.
Use `x` for individual removal or `X` for bulk confirmation. Removal neither marks the video watched nor changes watch later/saved videos. Continued reports from existing sessions cannot re-add a removed entry, including after restart; deliberately starting a new session can re-add it if still unfinished.
## Collection removal ## Collection removal
Removal controls apply **only in watch-later and saved-video views**, including previews opened from those lists. Search, Feed, ordinary channel/category views, and their previews do not expose removal actions or hints. Text fields and selectors retain their normal controls. Removal controls apply **only in watch-later, saved-video, and currently-watching views**, including previews opened from those lists. Search, Feed, ordinary channel/category views, and their previews do not expose removal actions or hints. Text fields and selectors retain their normal controls.
`x` immediately removes only the selected video from the active collection. `X` snapshots the current list and opens bulk confirmation: watch later uses the current category and All/Unwatched/Watched filter; saved videos uses the current category's entire saved list. An empty list does not open confirmation. `x` immediately removes only the selected video from the active collection. `X` snapshots the current list and opens bulk confirmation: watch later uses the current category and All/Unwatched/Watched filter; saved videos uses the current category's entire saved list; Currently watching uses its entire combined list. An empty list does not open confirmation.
The prompt names the collection, category, count, and applicable filter. **`y` confirms; `n` or Esc cancels.** Enter never confirms. Only the snapshotted IDs are removed, even if membership or watched status changes while confirmation is open. The prompt names the collection, category, count, and applicable filter. **`y` or Enter confirms; `n` or Esc cancels.** Only the snapshotted IDs are removed, even if membership or watched status changes while confirmation is open.
Removal changes only the active collection. Matching category views update, while the other collection and global watched status remain untouched. Removal changes only the active collection. Matching views update, while other collections and global watched status remain untouched.
The old collection-specific removal settings are no longer accepted. Replace them with: The old collection-specific removal settings are no longer accepted. Replace them with:
+7 -2
View File
@@ -142,6 +142,7 @@ actions! {
SavedVideoSave, "video.saved.save", "Save the selected video (also in previews)", VIEW, ["v"]; SavedVideoSave, "video.saved.save", "Save the selected video (also in previews)", VIEW, ["v"];
SavedVideoBrowse, "view.saved.browse", "Choose saved videos across all categories or one category", VIEW, ["V"]; SavedVideoBrowse, "view.saved.browse", "Choose saved videos across all categories or one category", VIEW, ["V"];
SavedVideoBrowseCategory, "view.saved.category", "Browse the current category's saved videos", VIEW, ["alt+v"]; SavedVideoBrowseCategory, "view.saved.category", "Browse the current category's saved videos", VIEW, ["alt+v"];
WatchingBrowse, "view.watching.browse", "Browse started, unfinished videos", VIEW, ["C"];
SelectorNext, "selector.select.next", "Select the next category or channel", &[Context::SelectorNormal], ["j", "down"]; SelectorNext, "selector.select.next", "Select the next category or channel", &[Context::SelectorNormal], ["j", "down"];
SelectorPrevious, "selector.select.previous", "Select the previous category or channel", &[Context::SelectorNormal], ["k", "up"]; SelectorPrevious, "selector.select.previous", "Select the previous category or channel", &[Context::SelectorNormal], ["k", "up"];
SelectorInputNext, "selector.input.next", "Select the next match while editing the filter", &[Context::SelectorInput], ["down", "ctrl+j"]; SelectorInputNext, "selector.input.next", "Select the next match while editing the filter", &[Context::SelectorInput], ["down", "ctrl+j"];
@@ -538,16 +539,20 @@ impl Bindings {
), ),
], ],
), ),
(
"CURRENTLY WATCHING",
&[(&[WatchingBrowse], "Browse started, unfinished videos")],
),
( (
"COLLECTION LISTS", "COLLECTION LISTS",
&[ &[
( (
&[CollectionRemoveSelected], &[CollectionRemoveSelected],
"Remove selected from current watch-later / saved list only", "Remove selected from the current collection only",
), ),
( (
&[CollectionRemoveBulk], &[CollectionRemoveBulk],
"Confirm removal of current category / filtered subset", "Confirm removal of current list / category / filtered subset",
), ),
( (
&[], &[],
+194 -14
View File
File diff suppressed because one or more lines are too long
+637 -43
View File
@@ -59,28 +59,35 @@ impl StoredFile {
pub enum CollectionKind { pub enum CollectionKind {
WatchLater, WatchLater,
Saved, Saved,
Watching,
} }
struct Collection { struct Collection {
file: StoredFile, file: StoredFile,
videos: Vec<Video>, videos: Vec<Video>,
ids: BTreeSet<String>, ids: BTreeSet<String>,
require_channel: bool,
} }
impl Collection { impl Collection {
fn open(path: PathBuf) -> io::Result<Self> { fn open(path: PathBuf, require_channel: bool) -> io::Result<Self> {
let (file, document) = StoredFile::open(path)?; let (file, document) = StoredFile::open(path)?;
let (videos, ids) = let (videos, ids) = load_videos(&document, require_channel)
load_videos(&document).map_err(|e| invalid(format!("{}: {e}", file.path.display())))?; .map_err(|e| invalid(format!("{}: {e}", file.path.display())))?;
Ok(Self { file, videos, ids }) Ok(Self {
file,
videos,
ids,
require_channel,
})
} }
fn save(&mut self, video: &Video) -> io::Result<()> { fn save(&mut self, video: &Video) -> io::Result<()> {
if self.ids.contains(&video.id) { if self.ids.contains(&video.id) {
return Ok(()); return check_original(&self.file.path, self.file.original.as_deref());
} }
let mut video = video.clone(); let mut video = video.clone();
validate_saved(&mut video)?; validate_snapshot(&mut video, self.require_channel)?;
self.file.save(video_document( self.file.save(video_document(
self.videos.iter().chain(std::iter::once(&video)), self.videos.iter().chain(std::iter::once(&video)),
))?; ))?;
@@ -91,7 +98,7 @@ impl Collection {
fn remove(&mut self, ids: &[String]) -> io::Result<()> { fn remove(&mut self, ids: &[String]) -> io::Result<()> {
if !ids.iter().any(|id| self.ids.contains(id)) { if !ids.iter().any(|id| self.ids.contains(id)) {
return Ok(()); return check_original(&self.file.path, self.file.original.as_deref());
} }
self.file.save(video_document( self.file.save(video_document(
self.videos.iter().filter(|v| !ids.contains(&v.id)), self.videos.iter().filter(|v| !ids.contains(&v.id)),
@@ -105,6 +112,7 @@ impl Collection {
pub struct State { pub struct State {
watch_later: Collection, watch_later: Collection,
saved_videos: Collection, saved_videos: Collection,
watching: Collection,
watch_state: StoredFile, watch_state: StoredFile,
watched: BTreeSet<String>, watched: BTreeSet<String>,
pending: BTreeSet<String>, pending: BTreeSet<String>,
@@ -121,14 +129,20 @@ impl State {
.parent() .parent()
.filter(|p| !p.as_os_str().is_empty()) .filter(|p| !p.as_os_str().is_empty())
.unwrap_or(Path::new(".")); .unwrap_or(Path::new("."));
let watch_later = Collection::open(parent.join("watch-later.toml"))?; let watch_later = Collection::open(parent.join("watch-later.toml"), true)?;
let saved_videos = Collection::open(parent.join("saved-videos.toml"))?; let saved_videos = Collection::open(parent.join("saved-videos.toml"), true)?;
let mut watching = Collection::open(parent.join("currently-watching.toml"), false)?;
let (watch_state, document) = StoredFile::open(parent.join("watch-state.toml"))?; let (watch_state, document) = StoredFile::open(parent.join("watch-state.toml"))?;
let watched = load_watched(&document) let watched = load_watched(&document)
.map_err(|e| invalid(format!("{}: {e}", watch_state.path.display())))?; .map_err(|e| invalid(format!("{}: {e}", watch_state.path.display())))?;
let completed: Vec<_> = watching.ids.intersection(&watched).cloned().collect();
if !completed.is_empty() {
watching.remove(&completed)?;
}
Ok(Self { Ok(Self {
watch_later, watch_later,
saved_videos, saved_videos,
watching,
watch_state, watch_state,
watched, watched,
pending: BTreeSet::new(), pending: BTreeSet::new(),
@@ -144,6 +158,7 @@ impl State {
match kind { match kind {
CollectionKind::WatchLater => &self.watch_later, CollectionKind::WatchLater => &self.watch_later,
CollectionKind::Saved => &self.saved_videos, CollectionKind::Saved => &self.saved_videos,
CollectionKind::Watching => &self.watching,
} }
} }
@@ -151,6 +166,7 @@ impl State {
match kind { match kind {
CollectionKind::WatchLater => &mut self.watch_later, CollectionKind::WatchLater => &mut self.watch_later,
CollectionKind::Saved => &mut self.saved_videos, CollectionKind::Saved => &mut self.saved_videos,
CollectionKind::Watching => &mut self.watching,
} }
} }
@@ -165,19 +181,43 @@ impl State {
} }
pub fn save(&mut self, kind: CollectionKind, video: &Video) -> io::Result<()> { pub fn save(&mut self, kind: CollectionKind, video: &Video) -> io::Result<()> {
if matches!(kind, CollectionKind::Watching) && self.is_watched(&video.id) {
return Ok(());
}
self.collection_mut(kind).save(video) self.collection_mut(kind).save(video)
} }
pub fn remove(&mut self, kind: CollectionKind, ids: &[String]) -> io::Result<()> { pub fn remove(&mut self, kind: CollectionKind, ids: &[String]) -> io::Result<()> {
if matches!(kind, CollectionKind::Watching) {
self.discovery = None;
loop {
self.discover()?;
if self.discovery.is_none() {
break;
}
}
for stream in &mut self.streams {
match stream.prepare() {
Ok(()) => {}
Err(e) if e.kind() == io::ErrorKind::NotFound => continue,
Err(e) => return Err(e),
}
if stream.id.as_ref().is_some_and(|id| ids.contains(id)) {
stream.mark_seen()?;
}
}
}
self.collection_mut(kind).remove(ids) self.collection_mut(kind).remove(ids)
} }
pub fn progress_file(&mut self) -> io::Result<PathBuf> { pub fn progress_file(&mut self, video: &Video) -> io::Result<PathBuf> {
if self.streams.len() >= MAX_STREAMS { if self.streams.len() >= MAX_STREAMS {
return Err(invalid( return Err(invalid(
"Playback session limit reached; remove inactive playback-sessions directories and restart", "Playback session limit reached; remove inactive playback-sessions directories and restart",
)); ));
} }
let mut video = video.clone();
validate_snapshot(&mut video, false)?;
fs::create_dir_all(self.sessions.parent().unwrap_or(Path::new(".")))?; fs::create_dir_all(self.sessions.parent().unwrap_or(Path::new(".")))?;
private_directory(&self.sessions, true)?; private_directory(&self.sessions, true)?;
let directory = loop { let directory = loop {
@@ -192,6 +232,8 @@ impl State {
Err(e) => return Err(e), Err(e) => return Err(e),
} }
}; };
let (mut snapshot, _) = StoredFile::open(directory.join("video.toml"))?;
snapshot.save(video_document(std::iter::once(&video)))?;
let path = directory.join("progress.jsonl"); let path = directory.join("progress.jsonl");
self.streams.push(ProgressStream::new(path.clone())); self.streams.push(ProgressStream::new(path.clone()));
Ok(path) Ok(path)
@@ -220,6 +262,38 @@ impl State {
self.watched.append(&mut self.pending); self.watched.append(&mut self.pending);
self.notify_changed = true; self.notify_changed = true;
} }
let completed: Vec<_> = self
.watching
.ids
.intersection(&self.watched)
.cloned()
.collect();
if !completed.is_empty() {
self.watching.remove(&completed)?;
self.notify_changed = true;
}
for stream in &mut self.streams {
if !stream.started || stream.seen {
continue;
}
let id = stream.id.as_ref().unwrap();
if !self.watched.contains(id) {
let video = stream.video.as_ref().or_else(|| {
self.watch_later
.videos
.iter()
.chain(self.saved_videos.videos.iter())
.find(|video| &video.id == id)
});
if let Some(video) = video {
let added = !self.watching.ids.contains(id);
self.watching.save(video)?;
self.notify_changed |= added;
}
stream.mark_seen()?;
}
stream.started = false;
}
// Preserve notification after reporting a bad record; siblings were still processed. // Preserve notification after reporting a bad record; siblings were still processed.
if let Some(error) = error { if let Some(error) = error {
return Err(error); return Err(error);
@@ -258,7 +332,9 @@ impl State {
continue; continue;
} }
if self.streams.len() >= MAX_STREAMS { if self.streams.len() >= MAX_STREAMS {
break; return Err(invalid(
"Playback session limit reached; remove inactive playback-sessions directories and restart",
));
} }
self.streams.push(ProgressStream::new(path)); self.streams.push(ProgressStream::new(path));
} }
@@ -268,7 +344,7 @@ impl State {
} }
} }
fn validate_saved(video: &mut Video) -> io::Result<()> { fn validate_snapshot(video: &mut Video, require_channel: bool) -> io::Result<()> {
let date = video.publication_date.clone(); let date = video.publication_date.clone();
if let Some(date) = &date { if let Some(date) = &date {
let raw = date.strip_prefix('~').unwrap_or(date); let raw = date.strip_prefix('~').unwrap_or(date);
@@ -303,7 +379,7 @@ fn validate_saved(video: &mut Video) -> io::Result<()> {
{ {
return Err(invalid("Saved uploader_id must be a valid YouTube @handle")); return Err(invalid("Saved uploader_id must be a valid YouTube @handle"));
} }
if video.channel_id.is_none() && video.uploader_id.is_none() { if require_channel && video.channel_id.is_none() && video.uploader_id.is_none() {
return Err(invalid( return Err(invalid(
"Video has no channel identity; refresh its metadata before saving", "Video has no channel identity; refresh its metadata before saving",
)); ));
@@ -311,7 +387,10 @@ fn validate_saved(video: &mut Video) -> io::Result<()> {
Ok(()) Ok(())
} }
fn load_videos(document: &DocumentMut) -> io::Result<(Vec<Video>, BTreeSet<String>)> { fn load_videos(
document: &DocumentMut,
require_channel: bool,
) -> io::Result<(Vec<Video>, BTreeSet<String>)> {
if document if document
.iter() .iter()
.any(|(key, _)| !matches!(key, "version" | "videos")) .any(|(key, _)| !matches!(key, "version" | "videos"))
@@ -350,7 +429,7 @@ fn load_videos(document: &DocumentMut) -> io::Result<(Vec<Video>, BTreeSet<Strin
} }
let mut video: Video = serde_json::from_value(serde_json::Value::Object(fields)) let mut video: Video = serde_json::from_value(serde_json::Value::Object(fields))
.map_err(|e| invalid(format!("Invalid saved video: {e}")))?; .map_err(|e| invalid(format!("Invalid saved video: {e}")))?;
validate_saved(&mut video)?; validate_snapshot(&mut video, require_channel)?;
if ids.insert(video.id.clone()) { if ids.insert(video.id.clone()) {
videos.push(video); videos.push(video);
} }
@@ -472,6 +551,16 @@ fn read_bytes(path: &Path, limit: u64) -> io::Result<Option<Vec<u8>>> {
Ok(Some(bytes)) Ok(Some(bytes))
} }
fn check_original(path: &Path, original: Option<&[u8]>) -> io::Result<()> {
if read_bytes(path, MAX_STATE_BYTES)?.as_deref() != original {
return Err(io::Error::other(format!(
"{} changed on disk; restart Tubular before retrying (no changes written)",
path.display()
)));
}
Ok(())
}
fn atomic_write(path: &Path, original: Option<&[u8]>, contents: &[u8]) -> io::Result<()> { fn atomic_write(path: &Path, original: Option<&[u8]>, contents: &[u8]) -> io::Result<()> {
if contents.len() as u64 > MAX_STATE_BYTES { if contents.len() as u64 > MAX_STATE_BYTES {
return Err(invalid("Watch state exceeds the 8 MiB limit")); return Err(invalid("Watch state exceeds the 8 MiB limit"));
@@ -500,12 +589,7 @@ fn atomic_write(path: &Path, original: Option<&[u8]>, contents: &[u8]) -> io::Re
let result = (|| { let result = (|| {
file.write_all(contents)?; file.write_all(contents)?;
file.sync_all()?; file.sync_all()?;
if read_bytes(path, MAX_STATE_BYTES)?.as_deref() != original { check_original(path, original)?;
return Err(io::Error::other(format!(
"{} changed on disk; restart Tubular before retrying (no changes written)",
path.display()
)));
}
if original.is_none() { if original.is_none() {
fs::hard_link(&temporary, path)?; fs::hard_link(&temporary, path)?;
} else { } else {
@@ -602,6 +686,10 @@ struct ProgressRecord {
struct ProgressStream { struct ProgressStream {
path: PathBuf, path: PathBuf,
id: Option<String>, id: Option<String>,
video: Option<Video>,
seen_file: Option<StoredFile>,
seen: bool,
started: bool,
file: Option<File>, file: Option<File>,
partial: Vec<u8>, partial: Vec<u8>,
discarding: bool, discarding: bool,
@@ -614,6 +702,10 @@ impl ProgressStream {
Self { Self {
path, path,
id: None, id: None,
video: None,
seen_file: None,
seen: false,
started: false,
file: None, file: None,
partial: Vec::new(), partial: Vec::new(),
discarding: false, discarding: false,
@@ -622,6 +714,74 @@ impl ProgressStream {
} }
} }
fn prepare(&mut self) -> io::Result<()> {
if self.id.is_some() {
return Ok(());
}
private_directory(self.path.parent().unwrap(), false)?;
let marker = self.path.with_file_name("video-id");
let mut bytes = Vec::new();
regular_file(&marker, true)?
.take(65)
.read_to_end(&mut bytes)?;
let id = std::str::from_utf8(&bytes)
.ok()
.filter(|id| valid_video_id(id))
.ok_or_else(|| invalid("Invalid video-id session marker"))?;
let snapshot = self.path.with_file_name("video.toml");
let video = match regular_file(&snapshot, true) {
Ok(_) => {
let mut collection = Collection::open(snapshot, false)?;
if collection.videos.len() != 1 || collection.videos[0].id != id {
return Err(invalid(
"Session video metadata must match its video-id marker",
));
}
collection.videos.pop()
}
Err(e) if e.kind() == io::ErrorKind::NotFound => None,
Err(e) => return Err(e),
};
let seen_path = self.path.with_file_name("watching-seen.toml");
match regular_file(&seen_path, true) {
Ok(_) => {}
Err(e) if e.kind() == io::ErrorKind::NotFound => {}
Err(e) => return Err(e),
}
let (seen_file, document) = StoredFile::open(seen_path)?;
let seen = seen_file.original.is_some();
if seen
&& (document
.iter()
.any(|(key, _)| !matches!(key, "version" | "id"))
|| document.get("id").and_then(|item| item.as_str()) != Some(id))
{
return Err(invalid(
"Session watching admission must match its video-id marker",
));
}
self.id = Some(id.into());
self.video = video;
self.seen_file = Some(seen_file);
self.seen = seen;
Ok(())
}
fn mark_seen(&mut self) -> io::Result<()> {
if self.seen {
let file = self.seen_file.as_ref().unwrap();
return check_original(&file.path, file.original.as_deref());
}
private_directory(self.path.parent().unwrap(), false)?;
let mut document = DocumentMut::new();
document["version"] = value(1);
document["id"] = value(self.id.as_deref().unwrap());
self.seen_file.as_mut().unwrap().save(document)?;
self.seen = true;
self.started = false;
Ok(())
}
fn poll( fn poll(
&mut self, &mut self,
pending: &mut BTreeSet<String>, pending: &mut BTreeSet<String>,
@@ -649,17 +809,7 @@ impl ProgressStream {
watched: &BTreeSet<String>, watched: &BTreeSet<String>,
) -> io::Result<Option<io::Error>> { ) -> io::Result<Option<io::Error>> {
if self.file.is_none() { if self.file.is_none() {
private_directory(self.path.parent().unwrap(), false)?; self.prepare()?;
let marker = self.path.with_file_name("video-id");
let mut bytes = Vec::new();
regular_file(&marker, true)?
.take(65)
.read_to_end(&mut bytes)?;
let id = std::str::from_utf8(&bytes)
.ok()
.filter(|id| valid_video_id(id))
.ok_or_else(|| invalid("Invalid video-id session marker"))?;
self.id = Some(id.into());
self.file = Some(regular_file(&self.path, true)?); self.file = Some(regular_file(&self.path, true)?);
} }
let mut buffer = [0; STREAM_BYTES]; let mut buffer = [0; STREAM_BYTES];
@@ -699,7 +849,11 @@ impl ProgressStream {
)) ))
} }
fn record(&self, pending: &mut BTreeSet<String>, watched: &BTreeSet<String>) -> io::Result<()> { fn record(
&mut self,
pending: &mut BTreeSet<String>,
watched: &BTreeSet<String>,
) -> io::Result<()> {
let record: ProgressRecord = let record: ProgressRecord =
serde_json::from_slice(&self.partial).map_err(|e| self.record_error(e))?; serde_json::from_slice(&self.partial).map_err(|e| self.record_error(e))?;
if record.version != 1 || !matches!(record.event.as_str(), "progress" | "ended" | "closed") if record.version != 1 || !matches!(record.event.as_str(), "progress" | "ended" | "closed")
@@ -723,8 +877,13 @@ impl ProgressStream {
"position must be finite/nonnegative and duration positive/finite or null", "position must be finite/nonnegative and duration positive/finite or null",
)); ));
} }
if self.id.as_deref() == Some(id) if self.id.as_deref() != Some(id) {
&& record.completion_eligible return Ok(());
}
if record.position_seconds > 0.0 && !self.seen && !watched.contains(id) {
self.started = true;
}
if record.completion_eligible
&& record && record
.duration_seconds .duration_seconds
.is_some_and(|d| record.position_seconds / d >= 0.95) .is_some_and(|d| record.position_seconds / d >= 0.95)
@@ -807,7 +966,7 @@ mod tests {
assert_eq!(state.videos(kind).len(), 1); assert_eq!(state.videos(kind).len(), 1);
assert!(state.contains(kind, &video.id)); assert!(state.contains(kind, &video.id));
} }
let report = state.progress_file().unwrap(); let report = state.progress_file(&video).unwrap();
register_identity(&report, &video.id).unwrap(); register_identity(&report, &video.id).unwrap();
append( append(
&report, &report,
@@ -889,7 +1048,7 @@ mod tests {
fn watched_write_failure_keeps_memory_unchanged_and_retries_consumed_reports() { fn watched_write_failure_keeps_memory_unchanged_and_retries_consumed_reports() {
let directory = TestDirectory::new(); let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap(); let mut state = State::open(&directory.config()).unwrap();
let report = state.progress_file().unwrap(); let report = state.progress_file(&video()).unwrap();
register_identity(&report, &video().id).unwrap(); register_identity(&report, &video().id).unwrap();
append( append(
&report, &report,
@@ -899,6 +1058,7 @@ mod tests {
fs::write(&path, "malformed state").unwrap(); fs::write(&path, "malformed state").unwrap();
assert!(state.poll_progress().is_err()); assert!(state.poll_progress().is_err());
assert!(!state.is_watched(&video().id)); assert!(!state.is_watched(&video().id));
assert!(state.videos(CollectionKind::Watching).is_empty());
assert_eq!(fs::read_to_string(&path).unwrap(), "malformed state"); assert_eq!(fs::read_to_string(&path).unwrap(), "malformed state");
fs::remove_file(&path).unwrap(); fs::remove_file(&path).unwrap();
assert!(state.poll_progress().unwrap()); assert!(state.poll_progress().unwrap());
@@ -910,7 +1070,7 @@ mod tests {
fn partial_reports_survive_restart_and_bad_siblings_do_not_block_completion() { fn partial_reports_survive_restart_and_bad_siblings_do_not_block_completion() {
let directory = TestDirectory::new(); let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap(); let mut state = State::open(&directory.config()).unwrap();
let report = state.progress_file().unwrap(); let report = state.progress_file(&video()).unwrap();
register_identity(&report, &video().id).unwrap(); register_identity(&report, &video().id).unwrap();
let line = record(&video().id, "ended", true, 95.0); let line = record(&video().id, "ended", true, 95.0);
append(&report, &line.as_bytes()[..line.len() - 1]); append(&report, &line.as_bytes()[..line.len() - 1]);
@@ -918,8 +1078,10 @@ mod tests {
drop(state); drop(state);
let mut state = State::open(&directory.config()).unwrap(); let mut state = State::open(&directory.config()).unwrap();
assert!(!state.poll_progress().unwrap()); assert!(!state.poll_progress().unwrap());
let bad = state.progress_file().unwrap(); let mut other = video();
register_identity(&bad, "rQ_J9WH6CGk").unwrap(); other.id = "rQ_J9WH6CGk".into();
let bad = state.progress_file(&other).unwrap();
register_identity(&bad, &other.id).unwrap();
append(&bad, b"not json\n"); append(&bad, b"not json\n");
append(&report, b"\n"); append(&report, b"\n");
assert!(state.poll_progress().is_err()); assert!(state.poll_progress().is_err());
@@ -932,7 +1094,7 @@ mod tests {
fn wrong_identity_ineligible_positions_and_untrusted_reports_do_not_mark_watched() { fn wrong_identity_ineligible_positions_and_untrusted_reports_do_not_mark_watched() {
let directory = TestDirectory::new(); let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap(); let mut state = State::open(&directory.config()).unwrap();
let report = state.progress_file().unwrap(); let report = state.progress_file(&video()).unwrap();
register_identity(&report, &video().id).unwrap(); register_identity(&report, &video().id).unwrap();
for line in [ for line in [
record("rQ_J9WH6CGk", "progress", true, 100.0), record("rQ_J9WH6CGk", "progress", true, 100.0),
@@ -941,7 +1103,8 @@ mod tests {
] { ] {
append(&report, line.as_bytes()); append(&report, line.as_bytes());
} }
assert!(!state.poll_progress().unwrap()); assert!(state.poll_progress().unwrap());
assert!(state.contains(CollectionKind::Watching, &video().id));
assert!(!state.is_watched(&video().id)); assert!(!state.is_watched(&video().id));
assert!(!state.is_watched("rQ_J9WH6CGk")); assert!(!state.is_watched("rQ_J9WH6CGk"));
for line in [ for line in [
@@ -958,7 +1121,7 @@ mod tests {
} }
#[cfg(unix)] #[cfg(unix)]
{ {
let symlink = state.progress_file().unwrap(); let symlink = state.progress_file(&video()).unwrap();
register_identity(&symlink, &video().id).unwrap(); register_identity(&symlink, &video().id).unwrap();
let external = directory.0.join("external.jsonl"); let external = directory.0.join("external.jsonl");
append( append(
@@ -971,4 +1134,435 @@ mod tests {
assert!(!state.poll_progress().unwrap()); assert!(!state.poll_progress().unwrap());
} }
} }
#[test]
fn watching_requires_a_valid_positive_bound_acknowledgement() {
let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap();
let video = video();
let report = state.progress_file(&video).unwrap();
assert!(state.videos(CollectionKind::Watching).is_empty());
assert!(!state.poll_progress().unwrap());
register_identity(&report, &video.id).unwrap();
assert!(!state.poll_progress().unwrap());
for line in [
record("rQ_J9WH6CGk", "progress", true, 100.0),
record(&video.id, "progress", true, 0.0),
] {
append(&report, line.as_bytes());
assert!(!state.poll_progress().unwrap());
assert!(state.videos(CollectionKind::Watching).is_empty());
}
for line in [
record(&video.id, "unknown", true, 10.0),
record(&video.id, "progress", true, -1.0),
record(&video.id, "progress", true, 10.0)
.replace("\"duration_seconds\":100", "\"duration_seconds\":0"),
record(&video.id, "progress", true, 10.0).replace("\"version\":1", "\"version\":2"),
] {
append(&report, line.as_bytes());
assert!(state.poll_progress().is_err());
assert!(state.videos(CollectionKind::Watching).is_empty());
assert!(!state.is_watched(&video.id));
}
append(
&report,
record(&video.id, "progress", false, 10.0).as_bytes(),
);
assert!(state.poll_progress().unwrap());
assert!(state.contains(CollectionKind::Watching, &video.id));
assert!(!state.is_watched(&video.id));
}
#[test]
fn starts_arriving_while_closed_preserve_metadata_without_a_creator() {
let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap();
let mut video = video();
video.channel = None;
video.channel_id = None;
video.uploader_id = None;
assert!(state.save(CollectionKind::Saved, &video).is_err());
assert!(state.save(CollectionKind::WatchLater, &video).is_err());
let report = state.progress_file(&video).unwrap();
register_identity(&report, &video.id).unwrap();
drop(state);
append(
&report,
record(&video.id, "closed", true, 10.0)
.replace("\"duration_seconds\":100", "\"duration_seconds\":null")
.as_bytes(),
);
let mut state = State::open(&directory.config()).unwrap();
assert!(state.poll_progress().unwrap());
let stored = &state.videos(CollectionKind::Watching)[0];
assert_eq!(stored.id, video.id);
assert_eq!(stored.title, video.title);
assert_eq!(stored.channel, None);
assert_eq!(stored.channel_id, None);
assert_eq!(stored.uploader_id, None);
assert_eq!(stored.publication_date, video.publication_date);
assert_eq!(stored.timestamp, video.timestamp);
append(&report, record(&video.id, "ended", false, 100.0).as_bytes());
assert!(!state.poll_progress().unwrap());
assert!(!state.is_watched(&video.id));
drop(state);
let mut state = State::open(&directory.config()).unwrap();
assert!(state.contains(CollectionKind::Watching, &video.id));
assert!(!state.poll_progress().unwrap());
}
#[test]
fn completion_wins_over_parallel_sessions_late_reports_and_replays() {
let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap();
let video = video();
for kind in [CollectionKind::Saved, CollectionKind::WatchLater] {
state.save(kind, &video).unwrap();
}
let first = state.progress_file(&video).unwrap();
register_identity(&first, &video.id).unwrap();
append(&first, record(&video.id, "progress", true, 94.9).as_bytes());
assert!(state.poll_progress().unwrap());
assert!(state.contains(CollectionKind::Watching, &video.id));
assert!(!state.is_watched(&video.id));
let second = state.progress_file(&video).unwrap();
register_identity(&second, &video.id).unwrap();
append(
&second,
record(&video.id, "progress", true, 20.0).as_bytes(),
);
append(&first, record(&video.id, "ended", true, 95.0).as_bytes());
assert!(state.poll_progress().unwrap());
assert!(state.is_watched(&video.id));
assert!(!state.contains(CollectionKind::Watching, &video.id));
for report in [&first, &second] {
append(report, record(&video.id, "progress", true, 30.0).as_bytes());
}
let replay = state.progress_file(&video).unwrap();
register_identity(&replay, &video.id).unwrap();
append(
&replay,
record(&video.id, "progress", true, 10.0).as_bytes(),
);
assert!(!state.poll_progress().unwrap());
assert!(state.videos(CollectionKind::Watching).is_empty());
drop(state);
let mut state = State::open(&directory.config()).unwrap();
assert!(!state.poll_progress().unwrap());
assert!(state.is_watched(&video.id));
state
.remove(CollectionKind::Watching, std::slice::from_ref(&video.id))
.unwrap();
assert!(state.is_watched(&video.id));
assert!(state.contains(CollectionKind::Saved, &video.id));
assert!(state.contains(CollectionKind::WatchLater, &video.id));
assert!(state.videos(CollectionKind::Watching).is_empty());
}
#[test]
fn manual_removal_suppresses_existing_sessions_across_restart_but_not_new_sessions() {
let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap();
let video = video();
for kind in [CollectionKind::Saved, CollectionKind::WatchLater] {
state.save(kind, &video).unwrap();
}
let first = state.progress_file(&video).unwrap();
register_identity(&first, &video.id).unwrap();
append(&first, record(&video.id, "progress", true, 10.0).as_bytes());
assert!(state.poll_progress().unwrap());
// Simulate a crash after collection persistence but before its admission marker.
fs::remove_file(first.with_file_name("watching-seen.toml")).unwrap();
let second = state.progress_file(&video).unwrap();
register_identity(&second, &video.id).unwrap();
append(
&second,
record(&video.id, "progress", true, 20.0).as_bytes(),
);
drop(state);
let mut state = State::open(&directory.config()).unwrap();
state
.remove(CollectionKind::Watching, std::slice::from_ref(&video.id))
.unwrap();
for report in [&first, &second] {
append(report, record(&video.id, "progress", true, 50.0).as_bytes());
}
assert!(!state.poll_progress().unwrap());
assert!(state.videos(CollectionKind::Watching).is_empty());
assert!(!state.is_watched(&video.id));
assert!(state.contains(CollectionKind::Saved, &video.id));
assert!(state.contains(CollectionKind::WatchLater, &video.id));
drop(state);
let mut state = State::open(&directory.config()).unwrap();
assert!(!state.poll_progress().unwrap());
let replay = state.progress_file(&video).unwrap();
register_identity(&replay, &video.id).unwrap();
append(
&replay,
record(&video.id, "progress", true, 10.0).as_bytes(),
);
assert!(state.poll_progress().unwrap());
assert!(state.contains(CollectionKind::Watching, &video.id));
assert!(!state.is_watched(&video.id));
}
#[test]
fn watching_write_conflicts_retry_consumed_starts() {
let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap();
let video = video();
let report = state.progress_file(&video).unwrap();
register_identity(&report, &video.id).unwrap();
let path = directory.0.join("currently-watching.toml");
fs::write(&path, "external edit").unwrap();
append(
&report,
record(&video.id, "progress", true, 10.0).as_bytes(),
);
assert!(state.poll_progress().is_err());
assert!(state.videos(CollectionKind::Watching).is_empty());
assert_eq!(fs::read_to_string(&path).unwrap(), "external edit");
assert!(state.poll_progress().is_err());
fs::remove_file(&path).unwrap();
assert!(state.poll_progress().unwrap());
assert!(state.contains(CollectionKind::Watching, &video.id));
let original = fs::read(&path).unwrap();
append(&path, b"\n# external removal conflict\n");
let replay = state.progress_file(&video).unwrap();
register_identity(&replay, &video.id).unwrap();
append(
&replay,
record(&video.id, "progress", true, 20.0).as_bytes(),
);
assert!(state.poll_progress().is_err());
assert!(!replay.with_file_name("watching-seen.toml").exists());
fs::write(&path, original).unwrap();
assert!(!state.poll_progress().unwrap());
drop(state);
let mut state = State::open(&directory.config()).unwrap();
assert!(state.contains(CollectionKind::Watching, &video.id));
assert!(!state.poll_progress().unwrap());
}
#[test]
fn failed_session_admission_retries_without_losing_starts_or_allowing_readdition() {
let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap();
let video = video();
let report = state.progress_file(&video).unwrap();
register_identity(&report, &video.id).unwrap();
state.streams[0].prepare().unwrap();
let marker = report.with_file_name("watching-seen.toml");
append(&marker, b"external edit");
append(
&report,
record(&video.id, "progress", true, 10.0).as_bytes(),
);
assert!(state.poll_progress().is_err());
assert!(state.contains(CollectionKind::Watching, &video.id));
assert!(
state
.remove(CollectionKind::Watching, std::slice::from_ref(&video.id))
.is_err()
);
assert!(state.contains(CollectionKind::Watching, &video.id));
assert_eq!(fs::read_to_string(&marker).unwrap(), "external edit");
fs::remove_file(&marker).unwrap();
assert!(state.poll_progress().unwrap());
state
.remove(CollectionKind::Watching, std::slice::from_ref(&video.id))
.unwrap();
append(
&report,
record(&video.id, "progress", true, 50.0).as_bytes(),
);
assert!(!state.poll_progress().unwrap());
assert!(state.videos(CollectionKind::Watching).is_empty());
drop(state);
let mut state = State::open(&directory.config()).unwrap();
assert!(!state.poll_progress().unwrap());
assert!(state.videos(CollectionKind::Watching).is_empty());
}
#[test]
fn failed_completion_cleanup_retries_and_restart_prunes_stale_watching() {
let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap();
let video = video();
let report = state.progress_file(&video).unwrap();
register_identity(&report, &video.id).unwrap();
append(
&report,
record(&video.id, "progress", true, 10.0).as_bytes(),
);
assert!(state.poll_progress().unwrap());
let path = directory.0.join("currently-watching.toml");
let original = fs::read(&path).unwrap();
append(&path, b"\n# external edit\n");
let changed = fs::read(&path).unwrap();
append(&report, record(&video.id, "ended", true, 95.0).as_bytes());
assert!(state.poll_progress().is_err());
assert!(state.is_watched(&video.id));
assert!(state.contains(CollectionKind::Watching, &video.id));
assert_eq!(fs::read(&path).unwrap(), changed);
append(
&report,
record(&video.id, "progress", true, 20.0).as_bytes(),
);
assert!(state.poll_progress().is_err());
fs::write(&path, original).unwrap();
assert!(state.poll_progress().unwrap());
assert!(!state.contains(CollectionKind::Watching, &video.id));
drop(state);
// Crash recovery must also handle a watched write committed before collection cleanup.
fs::write(&path, video_document(std::iter::once(&video)).to_string()).unwrap();
let mut state = State::open(&directory.config()).unwrap();
assert!(state.is_watched(&video.id));
assert!(state.videos(CollectionKind::Watching).is_empty());
assert!(!state.poll_progress().unwrap());
}
#[test]
fn legacy_sessions_use_saved_metadata_when_available_and_still_mark_completion_without_it() {
let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap();
let video = video();
state.save(CollectionKind::Saved, &video).unwrap();
let report = state.progress_file(&video).unwrap();
fs::remove_file(report.with_file_name("video.toml")).unwrap();
register_identity(&report, &video.id).unwrap();
append(
&report,
record(&video.id, "progress", true, 10.0).as_bytes(),
);
assert!(state.poll_progress().unwrap());
assert!(state.contains(CollectionKind::Watching, &video.id));
append(&report, record(&video.id, "ended", true, 95.0).as_bytes());
assert!(state.poll_progress().unwrap());
assert!(state.is_watched(&video.id));
assert!(state.videos(CollectionKind::Watching).is_empty());
let mut other = video.clone();
other.id = "rQ_J9WH6CGk".into();
let report = state.progress_file(&other).unwrap();
fs::remove_file(report.with_file_name("video.toml")).unwrap();
register_identity(&report, &other.id).unwrap();
append(
&report,
record(&other.id, "progress", true, 10.0).as_bytes(),
);
assert!(!state.poll_progress().unwrap());
assert!(state.videos(CollectionKind::Watching).is_empty());
append(&report, record(&other.id, "closed", true, 95.0).as_bytes());
assert!(state.poll_progress().unwrap());
assert!(state.is_watched(&other.id));
}
#[test]
fn session_metadata_and_admission_markers_must_match_the_bound_identity() {
for mismatched_snapshot in [true, false] {
let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap();
let video = video();
let report = state.progress_file(&video).unwrap();
register_identity(&report, &video.id).unwrap();
if mismatched_snapshot {
let mut other = video.clone();
other.id = "rQ_J9WH6CGk".into();
fs::write(
report.with_file_name("video.toml"),
video_document(std::iter::once(&other)).to_string(),
)
.unwrap();
} else {
append(
&report.with_file_name("watching-seen.toml"),
b"version = 1\nid = \"rQ_J9WH6CGk\"\n",
);
}
append(&report, record(&video.id, "ended", true, 100.0).as_bytes());
assert!(state.poll_progress().is_err());
assert!(state.videos(CollectionKind::Watching).is_empty());
assert!(!state.is_watched(&video.id));
}
}
#[cfg(unix)]
#[test]
fn session_snapshots_reject_symlinks_and_shared_permissions() {
use std::os::unix::fs::PermissionsExt;
for symlink in [true, false] {
let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap();
let video = video();
let report = state.progress_file(&video).unwrap();
register_identity(&report, &video.id).unwrap();
let snapshot = report.with_file_name("video.toml");
if symlink {
let external = directory.0.join("external.toml");
fs::write(&external, fs::read(&snapshot).unwrap()).unwrap();
fs::remove_file(&snapshot).unwrap();
std::os::unix::fs::symlink(&external, &snapshot).unwrap();
} else {
fs::set_permissions(&snapshot, fs::Permissions::from_mode(0o644)).unwrap();
}
append(
&report,
record(&video.id, "progress", true, 100.0).as_bytes(),
);
assert!(state.poll_progress().is_err());
assert!(state.videos(CollectionKind::Watching).is_empty());
assert!(!state.is_watched(&video.id));
}
}
#[test]
fn watching_and_session_snapshots_keep_date_handle_channel_and_schema_validation() {
let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap();
for invalid_field in ["date", "handle", "channel"] {
let mut video = video();
match invalid_field {
"date" => video.publication_date = Some("2005-02-30".into()),
"handle" => video.uploader_id = Some("not-a-handle".into()),
"channel" => video.channel_id = Some("not-a-channel-id".into()),
_ => unreachable!(),
}
assert!(state.progress_file(&video).is_err());
assert!(state.save(CollectionKind::Watching, &video).is_err());
assert!(state.videos(CollectionKind::Watching).is_empty());
}
let path = directory.0.join("currently-watching.toml");
let contents = "version = 1\nvideos = []\nunexpected = true\n";
fs::write(&path, contents).unwrap();
assert!(State::open(&directory.config()).is_err());
assert_eq!(fs::read_to_string(&path).unwrap(), contents);
}
#[test]
fn watching_and_session_snapshots_reject_oversized_files() {
let directory = TestDirectory::new();
let mut state = State::open(&directory.config()).unwrap();
let video = video();
let report = state.progress_file(&video).unwrap();
register_identity(&report, &video.id).unwrap();
OpenOptions::new()
.write(true)
.open(report.with_file_name("video.toml"))
.unwrap()
.set_len(MAX_STATE_BYTES + 1)
.unwrap();
append(
&report,
record(&video.id, "progress", true, 100.0).as_bytes(),
);
assert!(state.poll_progress().is_err());
assert!(state.videos(CollectionKind::Watching).is_empty());
assert!(!state.is_watched(&video.id));
File::create(directory.0.join("currently-watching.toml"))
.unwrap()
.set_len(MAX_STATE_BYTES + 1)
.unwrap();
assert!(State::open(&directory.config()).is_err());
}
} }