Response time, percentiles and jitter

An average response time can look fine while some of your visitors wait much longer. Pingara gives you three more views of the same data: percentiles (p50, p95 and p99) show how slow the slow checks are, and jitter shows how spread out your response times are within an hour.

Try this: open a monitor and find the jitter card beside Apdex Score. It is called Response Jitter on an HTTP monitor. Its percentage tells you how steady the monitor is. If it is amber or red, read on.

Percentiles: p50, p95 and p99

Pingara sorts the response times of a period's successful checks from fastest to slowest and reads off three points:

Only successful checks count, so a check that failed outright does not pull the percentiles around. Pingara works them out for every hourly and daily summary of a monitor. When there are no successful checks in a period, the percentiles read 0.

If p50 is 200 ms and p99 is 2,000 ms, most of your checks are fine, but about 1 in 100 takes ten times as long. An average would hide that.

Where you see them

• Latency Overview on the dashboard plots p50, p95 and p99 for the last 24 hours, one point per hour. Each point averages your monitors, weighted by how many successful checks each one ran.
• Latency by Region on a monitor's page shows a p95 beside each region's average on the 7-day and 30-day views. A ~ before a number means Pingara read a capped sample of checks instead of every one.
• The weekly report email lists p95 for each monitor.

Jitter

Jitter measures how steady your response time is. Pingara measures it one region at a time, using the population standard deviation of the response times of that region's successful checks in an hour. In plain terms, it is how far a typical check lands from that region's average for the hour, in milliseconds. If every check takes about the same time, jitter is close to 0. If response times are spread widely between fast and slow, jitter is large.

It measures spread around the region's average, not the change from one check to the next. A response time that climbs steadily through the hour has a high jitter even though each check is close to the one before it.

How the card gets its number:

1. Pingara takes the hour's successful checks and groups them by region. A check that failed outright is left out, so fast refusals and timeouts do not count.
2. A region is measured only if it has at least 3 successful checks in the hour. The section on when the card shows no jitter explains why.
3. For each measured region, Pingara divides its jitter by its own average response time. That is the region's jitter ratio.
4. The card shows the region with the highest ratio, the least steady one. When more than one region was measured, a second line names that region and says how many were measured, for example "Least steady of 4 regions measured: us-west-2". With one region measured, there is no region line.

Three more details:

• It is the population standard deviation, so Pingara divides by the number of checks, not by the number minus one.
• It uses the latest completed hour. The jitter card on a monitor shows that hour, so on an active monitor it can be a couple of hours behind. After a pause it can be much older.
• The percentage uses the same region's own average from the same hour. It is not the average on the Avg Response card, which covers the last 100 checks from all regions and counts failed ones.

The card's name follows the monitor type: Response Jitter for HTTP, RTT Jitter for ping, Connect Jitter for TCP, and so on. On a ping monitor, RTT Jitter is the variation between checks. Each ping check records the average round-trip time of 3 packets, and jitter is the spread of those averages. It is not the per-packet figure that the ping command prints.

Jitter on a monitor with several regions

Regions that are far apart answer at different speeds, but that distance is not jitter, because Pingara compares each region only with itself. Take a steady site checked from four regions that answer in about 20, 70, 80 and 220 ms, with each region's checks landing a little either side of its usual time. Over an hour of 12 checks per region, the least steady region is the 20 ms one. Its jitter is 2.5 ms on an average of 20 ms, a ratio of 0.125, so the card shows 2.50ms, "13% of avg latency", in green and names that region as the least steady of 4. If Pingara pooled all 48 checks instead, the gap between regions alone would give about 75 ms of jitter on an average of about 98 ms, a ratio of about 0.77, and the card would show red for a site where nothing is wrong.

Now suppose the 70 ms region starts alternating between 35 and 105 ms while the others stay put. Its jitter is 35 ms on an average of 70 ms, a ratio of 0.5, so the card turns red and names that region. You do not have to find it yourself.

To dig further:

• Open Latency by Region on the monitor's page and look at the named region's line. One line that jumps around while the others are smooth is a real problem on the path to that region.
• Watch the jitter over time. A card that was green and turned red is worth a look. One red hour can be a single slow check, see "Few checks, big swings" below.
• The card shows only the least steady region. If you want to see how steady the others are, use the chart.

When the card shows no jitter

The value shows - with a line of explanation in three cases:

• Needs 3 successful checks from one region in an hour. No region had enough successful checks in the latest completed hour. The floor is 3 because with only 2 checks the ratio is just the gap between them over their sum, so 100 ms and 154 ms already read amber from two points. Three keeps the figure from resting on a coin flip, and it still lets a 10-minute monitor (6 checks an hour in each region) be measured every hour.
• Available after the next hourly rollup. The monitor has no hourly rollup yet, which is normal in its first hour. Pingara writes one at the top of each hour for the hour just ended, so this line usually clears within an hour. If that first hour held too few checks, the card shows the first message until an hour that has enough. A monitor that is paused before its first rollup keeps this line until you resume it and the next hourly rollup runs.
• Unavailable for older rollups. The latest hourly rollup was written before Pingara measured jitter per region. It clears when the next hourly rollup is written, usually within a couple of hours. Pingara does not show the old combined figure in its place, because that figure is the one that counted distance between regions as jitter.

The first message is permanent on some monitors, because the number of checks an hour depends on the interval:

A 30-minute or 1-hour monitor can never reach 3 checks from one region in an hour, so it never shows a jitter figure. Use p95 and p99 to see its spread. The 3 checks must also be successful, so a 30-second monitor shows the same message during an hour in which every region fell short of 3 successful checks, for example a full outage. A region with fewer than 3 successful checks is left out of the count, and the card shows the regions that were measured.

Few checks, big swings

With few checks in the hour, one slow check moves the card a long way, because one point is a large share of the data. Say every check takes 200 ms except one:

So on a Free monitor, which checks every 5 minutes from one region, a single check at three times the usual time turns the card red for that hour, and one at twice the usual time turns it amber. The same slow check on a 30-second monitor reads 0.18. Treat one amber or red hour on a slow interval as a prompt to look at the chart, not as proof that the site is unsteady. The real signal is a region that stays amber or red hour after hour.

The same effect works across regions. The card shows the highest ratio, so a monitor checked from more regions has more chances to find an uneven one, and a four-region figure is not directly comparable with a one-region figure for the same site. Compare a monitor with itself over time, or with another monitor that checks from the same regions at the same interval.

How the jitter card is colored

A jitter of 30 ms means different things on a 40 ms monitor and a 600 ms one. So Pingara colors the card by the jitter ratio: jitter divided by the average response time. The card shows it as "% of avg latency".

For example, with an average of 200 ms:

• Jitter of 20 ms is 10%, green.
• Jitter of 50 ms is 25%, amber.
• Jitter of 100 ms is 50%, red.

The average used here is the named region's own average over the same hour, so it can differ from the Avg Response card (or Avg RTT and so on), which covers the last 100 checks from all regions. The card rounds the percentage to a whole number, so a card showing 15% in amber is just over the 0.15 line.

What to do about high jitter

• Check for load. Jitter that rises at the same hours every day usually follows traffic. Compare it with your Latency Overview.
• Compare the regions. On a monitor with several regions, the card names the least steady one. Open Latency by Region and compare that region's line with the others. If only that line is jumping, the cause is probably the network path to it, not your server. See Monitor Intervals and Regions.
• Look at the slow tail. A wide gap between p50 and p99 with high jitter points to a few very slow requests, such as a cold cache or a busy database.
• Check your Apdex threshold. Jitter does not trigger alerts. If you want to be told about slowdowns, set the Slow response threshold in the monitor's Advanced Settings, and see Apdex Scoring for how Pingara scores speed.

Next steps

• Apdex Scoring - Turn response times into a satisfaction score
• SLA reports and targets - Track uptime against a target
• Ping Monitoring - How ping checks measure round-trip time