ArchitectureAdrADR-0005: Parallel Query Execution

ADR-0005: Parallel Query Execution

ADR-0005: Parallel Query Execution

Status

Implemented (v44)

Date

2026-01-01

Context

Complex questions benefit from multiple query strategies (e.g., direct SQL, aggregation-first SQL, and optional web fallback). The current graph executes a single path, which can reduce answer quality and increase retries on ambiguous questions.

Decision

Implement parallel execution for a limited set of query variants, with strict guards:

  • Add SplitterNode to generate up to 3 parallel GraphState variants (SQL variants and optional web-search path when complexity is high).
  • Add ConsensusNode to score and select the best result based on syntax validity, schema compliance, row quality, and execution time.
  • Add MergerNode to combine the selected result with alternates for traceability and synthesis.
  • Update GraphOrchestrator to execute variants in parallel with a timeout and to fall back to sequential execution if token budgets are exceeded or any guard fails.
  • Enforce a token budget guard and a circuit breaker to prevent overload and infinite loops.
  • Gate all parallel behavior behind JORVIS_PARALLEL_QUERIES with default false.

Guardrails

  • SqlGuard remains authoritative and must be applied to all SQL variants.
  • Token budget limits (max paths, max total tokens) are enforced before parallel execution.
  • Timeout for each parallel path is enforced; failures in one path must not kill others.
  • Feature flag default is false; parallel execution is opt-in and can be disabled at runtime.

Consequences

  • Increased token usage and complexity, mitigated by strict limits and fallback to sequential.
  • Better answer quality for complex queries due to variant exploration.
  • Requires additional tests for concurrency, timeouts, scoring, and fallback behavior.
  • Trace data must include path metadata for debugging and observability.

Alternatives Considered

  1. Keep sequential execution only.
  2. Retry-based sequential exploration (higher latency, less diversity).
  3. Web-search-only fallback (lower reliability for database-specific queries).

Out of Scope

  • Trace visualization UI (targeted for a later release, e.g., v46).
  • Changes to SqlGuard enforcement or SQLGlot sidecar behavior.

Rollout Plan

  1. Add ADR and get approval.
  2. Implement SplitterNode, ConsensusNode, MergerNode, and orchestrator changes.
  3. Add tests (12+), including parallel execution, timeout handling, and guardrail behavior.
  4. Keep JORVIS_PARALLEL_QUERIES=false by default; enable only after CI green and sign-off.