Loadline
Testing & QualityloadlineLoadline is an Agent Skill for testing & quality published in the chromium/chromium repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Running and analyzing LoadLine 1 and 2 benchmarks on Android using Crossbench. Use when you need to measure page loading performance, evaluate performance-related changes in Chrome, or collect Perfetto traces with realistic Chrome workload.
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/chromium/chromium/blob/HEAD/agents/skills/loadline/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/loadline, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
LoadLine Benchmark
This skill guides the usage of LoadLine benchmarks to measure browser page loading performance by simulating real-world user journeys (loading a web page).
Quick Start
Read third_party/crossbench/config/benchmark/loadline2/README.md for basic
info on the LoadLine benchmark.
The entry point for all LoadLine benchmarks is the Crossbench script:
third_party/crossbench/cb.py
Common Command Pattern
third_party/crossbench/cb.py <variant> --browser=<browser> --stories=<stories> --repetitions=<N>
Benchmark Variants
| Variant | Description |
|---|---|
loadline2-phone | The latest LoadLine 2 benchmark optimized for mobile phones. |
loadline2-tablet | LoadLine 2 benchmark optimized for tablets. |
loadline-phone | Legacy LoadLine 1 benchmark for phones. |
loadline-tablet | Legacy LoadLine 1 benchmark for tablets. |
*-debug | Appends more tracing categories for easier debugging. |
Target Browsers (--browser)
- Android (Chrome): Use
--browser=adb:chrome(if one device) or--browser=$SERIAL:chrome. - Local Build: Use the path to the output directory, e.g.,
--browser=out/Release/chrome. - System Chrome: Use
--browser=chrome-stable,chrome-canary, etc.
Stories (--stories)
Default stories for LoadLine 2:
amazon_productcnn_articlewikipedia_articleglobo_homepagegoogle_search_result
Skip --stories flag to run everything or provide a comma-separated list for
specific stories.
Analyzing Results
LoadLine outputs two kinds of numbers: scores and breakdown.
Scores are in runs-per-minute, so higher is better. The main metric is called TOTAL_SCORE, this is the most stable and representative number.
Breakdown values are in milliseconds, so lower is better. Can be useful for analyzing which loading stage was affected in particular.
Statistical Significance
- Noise: Benchmark results are inherently noisy. A single repetition should only be used for smoke-testing.
- Reliability: To confidently determine changes of 1% or higher, run the benchmark with at least 50 repetitions.
- Interpretation: Changes below 1% are typically not considered statistically significant, even with 50 repetitions.
Identifying Tested Version
You can find the exact version of Chrome being tested in the benchmark logs by
looking for the 🏷️ STARTING BROWSER Version: line:
🏷️ STARTING BROWSER Version: 138.0.7204.168 stable
Common Workflows
Full Basic Run
Run the full benchmark on a connected Android device:
third_party/crossbench/cb.py loadline2-phone --browser=adb:chrome
Test a feature flag
See if a feature flag affects page loading performance:
third_party/crossbench/cb.py loadline2-phone --browser-config=feature_flag.hjson
where feature_flag.hjson looks like this:
{
flags: {
"experiment": {
"enabled": "--enable-features=YourFeature",
"disabled": "--disable-features=YourFeature",
},
},
browsers: {
"chrome": {
browser: "chrome",
driver: "adb",
flags: [ "experiment" ],
},
},
}
Compare past versions of Chrome
Check if clank/bin/install_chrome.py exists. If not (chromium-only checkouts),
see the next section for building Chromium locally.
Use clank/bin/install_chrome.py to install past versions on the connected
device. E.g.
clank/bin/install_chrome.py --channel dev --milestone 146 --signed
will install M146 with "Dev" channel branding. You can use multiple brandings to compare versions on the same device. E.g. install Canary:
clank/bin/install_chrome.py --channel canary --milestone 147 --signed
And then run LoadLine with two browsers:
third_party/crossbench/cb.py loadline2-phone --browser=adb:chrome-dev --browser=adb:chrome-canary
This will compare M146 and M147 on the same device.
Custom-built Chrome
- Build Chrome for Android with
android_channel="canary"gn arg. - Uninstall existing Chrome Canary from the device:
adb shell pm uninstall com.chrome.canary(this command fails if it's not installed; this is fine and can be ignored) - Install the custom built Chrome:
adb install out/$OUT_DIR/apks/Chrome.apk - Run LoadLine on chrome-canary:
third_party/crossbench/cb.py loadline2-phone --browser=adb:chrome-canary
Short run
Before running a full run on a custom-built Chrome, first try with a single repetition
third_party/crossbench/cb.py loadline2-phone --browser=adb:chrome-canary --repetitions=1
Don't trust the numbers, they are going to be very noisy. Do it just to verify that the benchmark runs without errors.