It’s not very often that data aligns perfectly to illustrate a topic. Typically, there are very clear trends among the outlying data points that clearly support the topic in question, but it’s rarely absolute. So, while researching Sitecore performance topics this week, I spotted data points that almost perfectly align to support the premise of Geographical Performance Variance in User Experience, that is, in layman’s terms, the further away the user is from the application the slower the user experience of that application.
If you think about how you’ve test an application that resides locally on your machine, or within the same network, then you’re much less sensitive to the impact of distance on performance. But if your audience is physically located thousands of miles away, it will impact their experience of your application and if the impact is more than a few seconds it will cause user frustration, fewer page visits, shorter visit duration, and lower conversions.
Now, this may sound like conventional wisdom since we’re all we’re bound by physics when it comes to communication speeds. What makes this a potentially complex problem is:
- the global internet infrastructure is not equal in all parts of the world; and
- the lack of reliable data to support decisions as to how to address the problem of geographically disparate users in delivering exceptional online experiences
Thus, answering questions becomes guesswork. Should you invest in a CDN service? Should you deploy to a different Azure or Amazon region? Can you optimize the critical areas of the application without investing in additional services?
But with hard quantifiable data, decision making becomes much easier. And, as a bonus, non-technical people can access this data very quickly and without the need to raise service requests to IT.
As a Sitecore Technology Partner I was privy to the release of the Habitat demo application launched at Sitecore Symposium. Habitat is built using the best practice principles of Sitecore Helix, which was also released at the Symposium. And this week I found time to setup some Synthetic monitoring tests to run against habitat.sitecore.net to determine if there were any interesting lessons to be learned and shared with the Sitecore and Dynatrace communities around the topic of user experience and application tuning and optimization.
For those unfamiliar with the term, Synthetic monitoring enables customer online experiences and interactions to be simulated. These simulations can be individual pages visited along with mouse and keyboard actions, or multi-step transactions that includes steps like authentication. The simulations can then be replayed at intervals from either a Backbone perspective, a Last Mile perspective, or both. Each perspective provides its own use cases, insights, and benefits.
Synthetic Backbone monitoring differs from Real User Monitoring by measuring from a clean room environment on the internet backbone that removes the noise of local ISP performance, browser/device/OS types, local anti-virus software etc. Think of Synthetic monitoring from the backbone as the control group.
Using basic tests setup at various global node locations to monitor the homepage of habitat.sitecore.net we can clearly see the impact of geography on the average response time metric in the chart below.
The performance is good from most locations, but we can see that the further away the node location executing the test is from application, the slower the performance.
While the UK, Sweden and Copenhagen along with the US average sub 3 second response times, performance slows for Australia and New Zealand, with New Zealand also experiencing severe fluctuations in terms of consistency of response times.
Even just focusing the lens on the US market we can see examples of geographic performance variance. As distance from the habitat application (hosted in the UK) increases we see corresponding increases in response times resulting in over a 1 second performance degradation between the East and West coasts.
Not that I’m one to disagree with 1990’s pop sensations T.L.C, but Waterfalls should be embraced when appropriate, and diagnosing slow user experiences is certainly an appropriate use case for waterfall analysis. Focusing in on one of the New Zealand performance spikes (Date: 11.17 Time: 08:00) in the above chart allows us to quantify each and every object that was rendered on the homepage, and the contribution of that object to the overall page load time.
The slow response time is caused by objects on the homepage that are delivered from habitat.sitecore.net, where 1st Byte Time is around 95% of the total load time of the object. A quick investigation of tests from other locations, even the UK which is the most local node location to the Habitat application, show approximately 95% of the response time for objects is 1st Byte Time. This explains why we see such a strong correlation between distance from application and the associated response time, where the physical distance (and thus network latency) is compounding the 1st byte time issue.
1st byte time is the amount of time a browser waits to receive the first piece of information from a web server after requesting it. Delays in 1st byte time can cause noticeable delay in web page rendering. Typical causes of long 1st byte times include dynamic content creation, web server configuration, traffic volume, and network issues. The first two causes, dynamic content creation and web server configuration are much easier to control than the latter two. Given this is a demo application for a market-leading Web Content Management platform the issue is likely due to dynamic content creation and the lack of a global CDN and perhaps local cache optimization for habitat. The use of caching or a CDN will greatly improve the performance for each region, and using these same synthetic tests any performance improvement can be accurately quantified to justify investment.
So, through a few basic tests and some simple analysis, we’ve quantified performance for our application across different geographies, identified the cause of slow response times, and have determined optimization and tuning options.
The habitat application is hosted in the UK, and while it’s impressive that the response times from far away locations, such as West Coast USA, is sub 4 seconds, we should remember this is a demo application and there are some key points we need to take into consideration when thinking about your own applications:
- There is minimal load on the application and infrastructure
- There is no real complexity
- The pageweight is below average at just 75KB
- The number of objects on the homepage is low (around > 50)
- There are very limited 3rd party services
- There are no back-end integrations to other applications and data sources
- There are limited code releases to the application
Thus, real world applications will significantly compound the geographical performance variance we see with this habitat example, and are far more likely to require high levels of optimization, local application instances, and CDN’s to accelerate content delivery and deliver solid user experiences. But to quantify the challenge, determine where to optimize and invest, and validate improvements, you hard data to support decision making.
To understand the Geographic Performance Variance for your own Sitecore (or other) application, try out the free 14-day Synthetic Trial.