Async Task Scheduling

Powerful async I/O architecture with task scheduling and progress tracking

Yazi is built on a fully asynchronous architecture that maximizes performance through non-blocking I/O and intelligent task scheduling. All I/O operations run asynchronously while CPU-intensive tasks are distributed across multiple threads.

Architecture Overview

Yazi’s async task system consists of several key components:

• Scheduler - Central task coordinator and queue manager
• Runner - Thread pool executor for parallel task processing
• Ongoing - Active task registry with progress tracking
• Task Types - Specialized handlers for different operation types

Task Types

Yazi supports multiple categories of async tasks:

File Operations

• Copy - Async file copying with progress tracking
• Cut/Move - File moving across filesystems
• Link - Symbolic and hard link creation
• Delete/Trash - File deletion with trash bin support
• Upload/Download - Remote file transfers via VFS

Background Tasks

• Preload - Pre-loading file previews and metadata
• Fetch - Running fetcher plugins for custom data
• Size - Calculating directory sizes recursively
• Process - External command execution

Plugin Tasks

• PluginEntry - Micro plugin execution
• Hook - Lifecycle hook handlers

Task Scheduling

The scheduler uses a priority-based queue system implemented in yazi-scheduler/src/scheduler.rs:

pub struct Scheduler {
    pub runner: Runner,
    handles: Vec<JoinHandle<()>>,
}

impl Scheduler {
    pub fn serve() -> Self {
        let (runner, handles) = Runner::make();
        Self { runner, handles }
    }

    pub fn cancel(&self, id: Id) -> bool {
        if let Some(hook) = self.ongoing.lock().cancel(id) {
            self.hook.submit(hook, HIGH);
            return false;
        }
        true
    }
}

Priority Levels

Tasks are scheduled with three priority levels:

• HIGH - Critical operations (cancellation hooks)
• NORMAL - Standard user operations (size calculation, processes)
• LOW - Background operations (file copy/move/delete)

Progress Tracking

Every task includes comprehensive progress tracking:

pub struct Task {
    pub id: Id,
    pub name: String,
    prog: TaskProg,
    hook: Option<HookIn>,
    pub done: CompletionToken,
    pub logs: String,
    pub logger: Option<mpsc::UnboundedSender<String>>,
}

Task States

Tasks progress through several states:

1. Pending - Queued but not yet started
2. Running - Currently executing
3. Success - Completed successfully
4. Failed - Encountered an error
5. Cleaned - Cleanup completed

Query task state with these methods:

impl TaskProg {
    pub fn cooked(self) -> bool    // Is task finished (success or failure)?
    pub fn running(self) -> bool   // Is task currently executing?
    pub fn success(self) -> bool   // Did task complete successfully?
    pub fn failed(self) -> bool    // Did task fail?
    pub fn percent(self) -> Option<f32>  // Progress percentage (0.0-1.0)
}

File Operations

Copy Operation

Async file copying with progress tracking:

pub fn file_copy(&self, from: UrlBuf, to: UrlBuf, force: bool, follow: bool) {
    let mut ongoing = self.ongoing.lock();
    let task = ongoing.add::<FileProgCopy>(
        format!("Copy {} to {}", from.display(), to.display())
    );

    if to.try_starts_with(&from).unwrap_or(false) && !to.covariant(&from) {
        return self.ops.out(
            task.id, 
            FileOutCopy::Fail("Cannot copy directory into itself".to_owned())
        );
    }

    let follow = follow || !from.scheme().covariant(to.scheme());
    self.file.submit(
        FileInCopy {
            id: task.id,
            from,
            to,
            force,
            cha: None,
            follow,
            retry: 0,
            done: task.done.clone(),
        },
        LOW,
    );
}

Progress Updates

File operations send real-time progress updates:

• Bytes processed - Track data transfer progress
• File count - Number of files processed
• Current file - Which file is being processed
• Speed - Transfer rate calculation
• ETA - Estimated time remaining

Process Execution

Yazi supports three process execution modes:

Block Mode

Blocking execution that waits for completion:

if opt.block {
    self.process.submit(
        ProcessInBlock { 
            id: task.id, 
            cwd: opt.cwd, 
            cmd: opt.cmd, 
            args: opt.args 
        },
        NORMAL
    );
}

Background Mode

Non-blocking background execution:

else {
    self.process.submit(
        ProcessInBg {
            id: task.id,
            cwd: opt.cwd,
            cmd: opt.cmd,
            args: opt.args,
            done: task.done.clone(),
        },
        NORMAL
    );
}

Orphan Mode

Detached process that continues after Yazi exits:

else if opt.orphan {
    self.process.submit(
        ProcessInOrphan { 
            id: task.id, 
            cwd: opt.cwd, 
            cmd: opt.cmd, 
            args: opt.args 
        },
        NORMAL
    );
}

Preloading System

Yazi preloads file previews before you navigate to them:

pub fn preload_paged(&self, preloader: &'static Preloader, target: &yazi_fs::File) {
    let mut ongoing = self.ongoing.lock();
    let task = ongoing.add::<PreloadProg>(
        format!("Run preloader `{}`", preloader.run.name)
    );

    let target = target.clone();
    self.preload.submit(PreloadIn { 
        id: task.id, 
        plugin: preloader, 
        target 
    });
}

This provides:

• Image decoding - Images decode before display
• Code highlighting - Syntax highlighting prepared in advance
• Archive listing - Archive contents scanned ahead
• Video thumbnails - Video frames extracted asynchronously

Size Calculation

Directory size calculation runs asynchronously with throttling:

pub fn prework_size(&self, targets: Vec<&UrlBuf>) {
    let throttle = Arc::new(Throttle::new(
        targets.len(), 
        Duration::from_millis(300)
    ));
    let mut ongoing = self.ongoing.lock();

    for target in targets {
        let task = ongoing.add::<SizeProg>(
            format!("Calculate the size of {}", target.display())
        );
        let target = target.clone();
        let throttle = throttle.clone();

        self.size.submit(SizeIn { id: task.id, target, throttle }, NORMAL);
    }
}

Throttling prevents UI flooding when calculating many directories.

Task Cancellation

Tasks can be cancelled with automatic cleanup:

pub fn cancel(&self, id: Id) -> bool {
    if let Some(hook) = self.ongoing.lock().cancel(id) {
        self.hook.submit(hook, HIGH);  // Run cleanup hook
        return false;
    }
    true
}

Cancellation Hooks

Certain operations register hooks for cleanup:

• Delete/Trash - Notify other instances
• Upload/Download - Close connections, remove partial files

Task Logging

Tasks can log output for debugging:

impl Task {
    pub(crate) fn log(&mut self, line: String) {
        self.logs.push_str(&line);
        self.logs.push('\n');

        if let Some(logger) = &self.logger {
            logger.send(line).ok();
        }
    }
}

Access logs via the tasks UI (press w by default).

Completion Tokens

Async operations return completion tokens for waiting:

pub fn file_download(&self, target: UrlBuf) -> CompletionToken {
    let mut ongoing = self.ongoing.lock();
    let task = ongoing.add::<FileProgDownload>(
        format!("Download {}", target.display())
    );

    // ... submit task ...

    task.done.clone()  // Return completion token
}

// Later, wait for completion:
let success = token.future().await;

Performance Benefits

Yazi’s async architecture provides:

• Zero blocking - UI never freezes on I/O
• Parallel execution - Multiple operations run simultaneously
• Resource efficiency - Async I/O uses minimal threads
• Responsive UI - Updates stream in real-time
• Smart scheduling - Priority system ensures important tasks run first

Configuration

Task behavior can be configured in yazi.toml:

[tasks]
# Maximum concurrent tasks
micro_workers = 10
macro_workers = 25
bizarre_retry = 5

# Image processing limits
image_alloc = 536870912
image_bound = [0, 0]

# Search and find workers
suppress_preload = false

Monitoring Tasks

View active tasks by pressing w (default keybinding):

• See all running operations
• Monitor progress in real-time
• View task logs
• Cancel running tasks

Tasks are persisted across sessions, so interrupted operations can resume.

See Also

• Virtual Filesystem - Remote file operations
• DDS - Cross-instance task coordination
• Why is Yazi Fast? - Architecture deep dive