Solution Quality Metrics

Understanding solution quality goes beyond simple feasibility. This guide explains the key metrics for evaluating VRP solutions, how to interpret them, and strategies for improvement.

What Makes a Good Solution?

A high-quality VRP solution balances multiple objectives:

┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│  FEASIBLE   │ │  EFFICIENT  │ │   BALANCED  │ │  CUSTOMER   │ │   COST      │
│             │ │             │ │             │ │  FOCUSED    │ │ EFFECTIVE   │
├─────────────┤ ├─────────────┤ ├─────────────┤ ├─────────────┤ ├─────────────┤
│ • All jobs  │ │ • Min travel│ │ • Fair work │ │ • On time   │ │ • Min fleet │
│ • Capacity  │ │ • Min wait  │ │ • Occupancy │ │ • Preferences│ │ • Min hours │
│ • Time OK   │ │ • Short routes│ │ • Utilization│ │ • Priorities│ │ • Min overtime│
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘

The five pillars of solution quality

Core Quality Metrics

1. Feasibility Rate

The percentage of jobs successfully assigned:

Feasibility Rate = (Assigned Jobs / Total Jobs) × 100%

{
  "totalJobs": 50,
  "assignedJobs": 50,
  "unassigned": [],
  "feasibilityRate": 100
}

Ideal scenario - all jobs scheduled.

{
  "totalJobs": 50,
  "assignedJobs": 50,
  "unassigned": [],
  "feasibilityRate": 100
}

Ideal scenario - all jobs scheduled.

{
  "totalJobs": 50,
  "assignedJobs": 48,
  "unassigned": ["job-23", "job-47"],
  "feasibilityRate": 96
}

Acceptable with partial planning enabled.

{
  "totalJobs": 50,
  "assignedJobs": 42,
  "unassigned": [...],
  "feasibilityRate": 84
}

Indicates capacity or constraint issues.

2. Occupancy

How full the schedule is in terms of work time versus available time:

Occupancy = (Total Work Time / Total Available Time) × 100%

Work Time includes:

Travel time
Service time
Wait time (if any)

Available Time:

Shift duration minus breaks

{
  "occupancy": 0.78,  // 78% utilized
  "totalWorkTimeInSeconds": 28080,
  "totalAvailableTimeInSeconds": 36000,
  "breakdown": {
    "travelTime": 14400,  // 40%
    "serviceTime": 12600,  // 35%
    "waitTime": 1080       // 3%
  }
}

Interpreting Occupancy:

< 60%: Underutilized - consider reducing fleet
60-85%: Optimal range - good efficiency with buffer
> 85%: High utilization - limited flexibility
> 95%: Over-stretched - risk of delays

3. Workload Fairness

Measures how evenly work is distributed across resources:

Fairness = 1 - (Standard Deviation / Mean Work Time)

Unfair Distribution (0.45)          Fair Distribution (0.92)
Resource 1: ████████████████ 8h      Resource 1: ████████ 4h
Resource 2: ██ 1h                    Resource 2: ███████ 3.5h
Resource 3: ████ 2h                  Resource 3: ████████ 4h
Resource 4: ████████████████ 8h      Resource 4: ███████ 3.5h

Workload distribution visualization

{
  "workloadFairness": 0.85,
  "resourceWorkloads": [
    {"resource": "driver-1", "workTime": 420, "deviation": 15},
    {"resource": "driver-2", "workTime": 390, "deviation": -15},
    {"resource": "driver-3", "workTime": 405, "deviation": 0}
  ],
  "averageWorkTime": 405,
  "standardDeviation": 15
}

4. Travel Efficiency

Multiple metrics evaluate routing efficiency:

Total Distance & Time

{
  "totalTravelDistanceInMeters": 245000,
  "totalTravelTimeInSeconds": 28800,
  "averageSpeed": 30.6  // km/h
}

Distance per Job

Efficiency = Total Distance / Number of Jobs

Lower values indicate better clustering and routing.

Circuity Factor

Circuity = Actual Distance / Direct Distance

Values close to 1.0 indicate direct routes.

5. Time-Based Metrics

On-Time Performance

{
  "onTimeDeliveries": 47,
  "lateDeliveries": 3,
  "onTimeRate": 0.94,
  "averageDelay": 780  // seconds
}

Wait Time Analysis

{
  "totalWaitTimeInSeconds": 3600,
  "jobsWithWaitTime": 8,
  "averageWaitPerJob": 450,
  "waitTimePercentage": 0.05  // 5% of total time
}

High wait times indicate:

Poor time window alignment
Suboptimal route sequencing
Need for dynamic scheduling

Advanced Quality Indicators

Service Level Metrics

{
  "priorityLevels": {
    "high": {"total": 10, "assigned": 10, "rate": 1.0},
    "medium": {"total": 25, "assigned": 23, "rate": 0.92},
    "low": {"total": 15, "assigned": 12, "rate": 0.8}
  },
  "weightedFulfillment": 0.93
}

{
  "priorityLevels": {
    "high": {"total": 10, "assigned": 10, "rate": 1.0},
    "medium": {"total": 25, "assigned": 23, "rate": 0.92},
    "low": {"total": 15, "assigned": 12, "rate": 0.8}
  },
  "weightedFulfillment": 0.93
}

{
  "preferredResourceMatches": 35,
  "totalPreferences": 40,
  "preferenceRate": 0.875,
  "tagMatchRate": 0.95
}

{
  "hardWindowViolations": 0,
  "softWindowViolations": 5,
  "averageArrivalDeviation": -300,  // 5 min early on average
  "windowComplianceRate": 0.95
}

Cost Effectiveness

{
  "totalResources": 10,
  "activeResources": 7,
  "utilizationRate": 0.7,
  "costPerJob": 45.20,
  "costPerKm": 2.10
}

Comparing Solutions

When evaluating alternative solutions:

Multi-Criteria Comparison

Metric              Solution A    Solution B    Solution C
---------------------------------------------------------
Feasibility         100%          95%           100%
Travel Time         4.2h          3.8h          4.5h
Fleet Size          5             4             6
Workload Fairness   0.85          0.92          0.78
Cost                $1,250        $1,180        $1,320
Overall Score       8.2/10        8.7/10        7.5/10

Solution comparison matrix

Pareto Optimization

Often no single “best” solution exists. Consider Pareto-optimal solutions:

A solution is Pareto-optimal if improving one metric necessarily worsens another. Keep multiple Pareto-optimal solutions for different scenarios.

Weighted Scoring

Create a composite score based on business priorities:

const weights = {
  feasibility: 0.3,
  efficiency: 0.25,
  cost: 0.25,
  service: 0.2
};

const compositeScore = 
  (feasibilityRate * weights.feasibility) +
  (efficiencyScore * weights.efficiency) +
  (costScore * weights.cost) +
  (serviceScore * weights.service);

Improvement Strategies

For Low Feasibility

Enable Partial Planning

{"options": {"partialPlanning": true}}

Relax Constraints

Convert hard constraints to soft where possible

Add Resources

Increase fleet size or extend shifts

Review Requirements

Check if all constraints are necessary

For Poor Efficiency

{
  "weights": {
    "travelTimeWeight": 5,
    "regionMinimisationWeight": 10
  }
}

{
  "weights": {
    "travelTimeWeight": 5,
    "regionMinimisationWeight": 10
  }
}

{
  "weights": {
    "waitTimeWeight": 20
  },
  "options": {
    "snapUnit": 900  // 15-min slots
  }
}

{
  "options": {
    "distanceMatrix": {
      "enabled": true,
      "traffic": true
    }
  }
}

For Imbalanced Workload

{
  "options": {
    "fairWorkloadPerResource": true,
    "workloadSensitivity": 0.1,  // 10% tolerance
    "weights": {
      "workloadSpreadWeight": 100
    }
  }
}

Monitoring Quality Over Time

Track metrics to identify trends:

{
  "date": "2024-03-15",
  "metrics": {
    "feasibilityRate": 0.96,
    "occupancy": 0.78,
    "onTimeRate": 0.94,
    "costPerJob": 42.50
  }
}

Quality Benchmarks

Industry standards for different sectors:

Sector	Feasibility	Occupancy	On-Time	Fairness
Parcel Delivery	98%+	75-85%	95%+	0.80+
Field Service	95%+	65-75%	90%+	0.85+
Food Delivery	99%+	70-80%	98%+	0.75+
Medical Transport	100%	60-70%	99%+	0.90+

Benchmarks vary by region, business model, and service level agreements. Use these as starting points and adjust based on your specific requirements.

Best Practices

Quality Management Guidelines:

Define Success Metrics: Establish clear KPIs before optimization
Regular Monitoring: Track metrics daily/weekly
Iterative Improvement: Make small, measured changes
Balance Trade-offs: No solution is perfect in all dimensions
Document Decisions: Record why certain trade-offs were made
Benchmark Regularly: Compare against historical performance

Scoring System

Understand score components

Performance Guide

Optimize solution speed

Cost Optimization

Minimize operational costs

Unassigned Reasons

Fix quality issues

Platform

VRP API

Solution Quality Metrics

Solution Quality Metrics

What Makes a Good Solution?

Core Quality Metrics

1. Feasibility Rate

2. Occupancy

3. Workload Fairness

4. Travel Efficiency

Total Distance & Time

Distance per Job

Circuity Factor

5. Time-Based Metrics

On-Time Performance

Wait Time Analysis

Advanced Quality Indicators

Service Level Metrics

Cost Effectiveness

Comparing Solutions

Multi-Criteria Comparison

Pareto Optimization

Weighted Scoring

Improvement Strategies

For Low Feasibility

For Poor Efficiency

For Imbalanced Workload

Monitoring Quality Over Time

Quality Benchmarks

Best Practices

Scoring System

Performance Guide

Cost Optimization

Unassigned Reasons

Platform

VRP API

​Solution Quality Metrics

​What Makes a Good Solution?

​Core Quality Metrics

​1. Feasibility Rate

​2. Occupancy

​3. Workload Fairness

​4. Travel Efficiency

​Total Distance & Time

​Distance per Job

​Circuity Factor

​5. Time-Based Metrics

​On-Time Performance

​Wait Time Analysis

​Advanced Quality Indicators

​Service Level Metrics

​Cost Effectiveness

​Comparing Solutions

​Multi-Criteria Comparison

​Pareto Optimization

​Weighted Scoring

​Improvement Strategies

​For Low Feasibility

​For Poor Efficiency

​For Imbalanced Workload

​Monitoring Quality Over Time

​Quality Benchmarks

​Best Practices

​Related Topics

Scoring System

Performance Guide

Cost Optimization

Unassigned Reasons

Solution Quality Metrics

What Makes a Good Solution?

Core Quality Metrics

1. Feasibility Rate

2. Occupancy

3. Workload Fairness

4. Travel Efficiency

Total Distance & Time

Distance per Job

Circuity Factor

5. Time-Based Metrics

On-Time Performance

Wait Time Analysis

Advanced Quality Indicators

Service Level Metrics

Cost Effectiveness

Comparing Solutions

Multi-Criteria Comparison

Pareto Optimization

Weighted Scoring

Improvement Strategies

For Low Feasibility

For Poor Efficiency

For Imbalanced Workload

Monitoring Quality Over Time

Quality Benchmarks

Best Practices

Related Topics