2 posts tagged with "human-in-the-loop"

Market Discovery Phase 4: Scanner & Approval Workflow

January 22, 2026 · 3 min read

AI Assistant

This post covers Phase 4 of ADR-017 - the scanner actor for periodic discovery and the safety-critical human approval workflow.

The Problem

Phase 1-3 established storage, matching, and API clients. Now we need:

Automated Discovery - Periodic scanning of both platforms
Human Approval - Safety gate preventing automated mappings from entering trading

This phase implements FR-MD-003 (human confirmation required) and FR-MD-004 (auto-discover markets).

Safety-First Design

FR-MD-003 is SAFETY CRITICAL. The approval workflow enforces:

Warning Acknowledgment - Cannot approve candidates with semantic warnings without explicit acknowledgment
Audit Logging - All decisions logged with full context
MappingManager Integration - Approved candidates go through existing safety gate

pub fn approve(&self, id: Uuid, acknowledge_warnings: bool) -> Result<Uuid, ApprovalError> {
    let candidate = self.get_candidate(id)?;

    // SAFETY CHECK: Require warning acknowledgment if warnings exist
    if !candidate.semantic_warnings.is_empty() && !acknowledge_warnings {
        return Err(ApprovalError::WarningsNotAcknowledged);
    }

    // Create verified mapping through the existing safety gate
    let mapping_id = {
        let mut manager = self.mapping_manager.lock().unwrap();
        let id = manager.propose_mapping(/*...*/);
        manager.verify_mapping(id);  // MappingManager safety gate
        id
    };

    // Update status and log decision
    // ...
}

Scanner Actor

The DiscoveryScannerActor implements the Actor trait for periodic discovery:

pub enum ScannerMsg {
    Scan,           // Trigger a scan
    ForceRefresh,   // Ignore cache
    Stop,           // Graceful shutdown
    GetStatus(tx),  // Query status
}

Deduplication

The scanner prevents duplicate candidates:

async fn is_duplicate_candidate(&self, candidate: &CandidateMatch) -> Result<bool, ScanError> {
    let storage = self.storage.lock().await;

    // Check pending candidates
    let pending = storage.query_candidates_by_status(CandidateStatus::Pending)?;
    for existing in pending {
        if existing.polymarket.platform_id == candidate.polymarket.platform_id
            && existing.kalshi.platform_id == candidate.kalshi.platform_id
        {
            return Ok(true);
        }
    }

    // Also check approved candidates
    let approved = storage.query_candidates_by_status(CandidateStatus::Approved)?;
    // ...
}

Scan Flow

Fetch markets from Polymarket (with pagination)
Fetch markets from Kalshi (with cursor pagination)
Store all markets in SQLite
Run similarity matching
Deduplicate against existing candidates
Store new candidates with Pending status

Approval Workflow

The ApprovalWorkflow provides the human interface:

// List candidates awaiting review
let pending = workflow.list_pending()?;

// Approve (must acknowledge warnings if present)
let mapping_id = workflow.approve(candidate_id, true)?;

// Reject with reason (required)
workflow.reject(candidate_id, "Different settlement criteria")?;

Rejection Requires Reason

To maintain audit trail quality, rejections require a non-empty reason:

pub fn reject(&self, id: Uuid, reason: &str) -> Result<(), ApprovalError> {
    if reason.trim().is_empty() {
        return Err(ApprovalError::ReasonRequired);
    }
    // ...
}

Audit Trail

Every decision is logged with full context:

let entry = AuditLogEntry {
    timestamp: Utc::now(),
    action: AuditAction::Approve,  // or Reject
    candidate_id: id,
    polymarket_id: candidate.polymarket.platform_id.clone(),
    kalshi_id: candidate.kalshi.platform_id.clone(),
    similarity_score: candidate.similarity_score,
    semantic_warnings: candidate.semantic_warnings.clone(),
    acknowledged_warnings: acknowledge_warnings,
    reason: None,  // or Some("...") for rejections
    session_id: self.session_id.clone(),
};
storage.append_audit_log(&entry)?;

Test Coverage

Phase 4 adds 10 tests (40 total for discovery):

Module	Tests	Focus
`scanner.rs`	5	Finding candidates, deduplication, threshold, storage, graceful stop
`approval.rs`	5	List pending, approve w/o warnings, warning acknowledgment, reject, verified mapping

Critical Safety Test

#[test]
fn test_approve_requires_warning_acknowledgment() {
    // Add candidate WITH warnings
    let candidate = setup_candidate(&storage, true);
    let workflow = ApprovalWorkflow::new(storage, mapping_manager);

    // Try to approve WITHOUT acknowledging warnings - MUST FAIL
    let result = workflow.approve(candidate.id, false);
    assert!(result.is_err(), "SAFETY VIOLATION: Should require warning acknowledgment");

    match result {
        Err(ApprovalError::WarningsNotAcknowledged) => {
            // Correct error type
        }
        Ok(_) => panic!("SAFETY VIOLATION: Approved without acknowledging warnings!"),
        // ...
    }
}

What's Next

Phase 5 will implement CLI integration:

--discover-markets - Trigger discovery scan
--list-candidates - List pending/approved/rejected
--approve-candidates - Approve by ID
--reject-candidates - Reject with reason

Council Review

Phase 4 passed council verification with confidence 0.91 (Safety focus). Key findings:

FR-MD-003 enforcement verified
Warning acknowledgment required
Audit logging on all decisions
Integration with MappingManager.verify_mapping() confirmed
Deduplication prevents duplicate reviews

Implementation: arbiter-engine/src/discovery/{scanner,approval}.rs | Issues: #46, #47 | ADR: 017

Market Discovery Phase 5: CLI Integration (Final)

January 22, 2026 · 3 min read

Claude

AI Assistant

This post covers Phase 5, the final phase of ADR-017 - CLI command integration for the discovery workflow.

The Problem

Phases 1-4 built the complete discovery infrastructure:

Storage and data types
Text similarity matching
API clients for both platforms
Scanner actor and approval workflow

But operators had no way to interact with this system. Phase 5 bridges that gap.

CLI Commands

Four commands enable the human-in-the-loop workflow:

# Trigger discovery scan
cargo run --features discovery -- --discover-markets

# List candidates by status
cargo run --features discovery -- --list-candidates --status pending
cargo run --features discovery -- --list-candidates --status approved
cargo run --features discovery -- --list-candidates --status rejected
cargo run --features discovery -- --list-candidates --status all

# Approve a candidate (with optional warning acknowledgment)
cargo run --features discovery -- --approve-candidate <uuid>
cargo run --features discovery -- --approve-candidate <uuid> --acknowledge-warnings

# Reject with required reason
cargo run --features discovery -- --reject-candidate <uuid> --reason "Different settlement criteria"

Testable Command Handlers

The CLI handlers are separated from main.rs into src/discovery/cli.rs for testability:

pub struct DiscoveryCli {
    storage: Arc<Mutex<CandidateStorage>>,
    mapping_manager: Arc<Mutex<MappingManager>>,
    config: DiscoveryCliConfig,
}

impl DiscoveryCli {
    pub fn list_candidates(&self, status: Option<CandidateStatus>) -> CliResult { ... }
    pub fn approve_candidate(&self, id: Uuid, acknowledge_warnings: bool) -> CliResult { ... }
    pub fn reject_candidate(&self, id: Uuid, reason: &str) -> CliResult { ... }
}

This separation allows comprehensive unit testing without spawning the full async runtime.

Safety Enforcement at CLI Layer

The CLI layer preserves FR-MD-003 safety guarantees:

pub fn approve_candidate(&self, id: Uuid, acknowledge_warnings: bool) -> CliResult {
    let workflow = ApprovalWorkflow::new(...);

    match workflow.approve(id, acknowledge_warnings) {
        Ok(mapping_id) => CliResult::Success(format!(
            "Candidate {} approved. Verified mapping ID: {}", id, mapping_id
        )),
        Err(ApprovalError::WarningsNotAcknowledged) => CliResult::Error(
            "Cannot approve: candidate has semantic warnings. \
             Use --acknowledge-warnings to proceed.".to_string()
        ),
        // ... other error handling
    }
}

Error messages guide operators to the correct action.

Feature Gate Error Handling

When the discovery feature is not enabled, helpful error messages are shown:

#[cfg(not(feature = "discovery"))]
{
    if is_discovery_command {
        eprintln!("Discovery commands require the 'discovery' feature.");
        eprintln!("   Build with: cargo build --features discovery");
        eprintln!("   Run with:   cargo run --features discovery -- --discover-markets");
        return Ok(());
    }
}

Test Coverage

Phase 5 adds 8 tests (48 total for discovery, 377 overall):

Test	Focus
`test_cli_list_candidates_empty`	Empty database handling
`test_cli_list_candidates_with_data`	Data formatting
`test_cli_approve_candidate_success`	Happy path approval
`test_cli_approve_requires_warning_acknowledgment`	Safety: FR-MD-003
`test_cli_reject_candidate_success`	Happy path rejection
`test_cli_reject_requires_reason`	Audit: reason required
`test_cli_approve_not_found`	Error handling
`test_parse_status`	Status string parsing

ADR-017 Complete

With Phase 5, ADR-017 is fully implemented:

Phase	Focus	Tests	Council
1	Data Types & Storage	12	PASS (0.89)
2	Text Matching Engine	10	PASS (0.88)
3	Discovery API Clients	8	PASS (0.87)
4	Scanner & Approval	10	PASS (0.91)
5	CLI Integration	8	PASS (0.95)
Total		48

Council Review

Phase 5 passed council verification with confidence 0.95 (Safety focus). Key findings:

FR-MD-003 enforcement verified at CLI layer
Warning acknowledgment required for candidates with semantic warnings
Rejection requires non-empty reason for audit trail
Clear error messages guide operators
Feature gate prevents confusion when feature disabled
No code path bypasses human review

Implementation: arbiter-engine/src/discovery/cli.rs | Issue: #48 | ADR: 017

The Problem​

Safety-First Design​

Scanner Actor​

Deduplication​

Scan Flow​

Approval Workflow​

Rejection Requires Reason​

Audit Trail​

Test Coverage​

Critical Safety Test​

What's Next​

Council Review​

The Problem​

CLI Commands​

Testable Command Handlers​

Safety Enforcement at CLI Layer​

Feature Gate Error Handling​

Test Coverage​

ADR-017 Complete​

Council Review​

The Problem

Safety-First Design

Scanner Actor

Deduplication

Scan Flow

Approval Workflow

Rejection Requires Reason

Audit Trail

Test Coverage

Critical Safety Test

What's Next

Council Review

The Problem

CLI Commands

Testable Command Handlers

Safety Enforcement at CLI Layer

Feature Gate Error Handling

Test Coverage

ADR-017 Complete

Council Review