VPN servers   5200+    Countries    60

 

VPN Comparative Test

A test commissioned by NordVPN and performed by AV-TEST GmbH Date of the report: September 28th, 2020

 

 

Executive Summary

In September 2020, AV-TEST performed a test of Virtual Private Network (VPN) solutions and put them into contrast to the unencrypted connection. The VPNs have been reviewed for their download- and upload- performance at the three geographical locations the US west coast, the UK and Japan.

We perform this test in parallel for all products at one point in time, which provides an extract of the performance provided by the VPN products.

The presented evaluation assesses ExpressVPN, NordVPN, Private Internet Access, PrivateVPN, ProtonVPN, PureVPN and TorGuard.

NordVPN was the clear winner in this performance test for measured download and upload performance in all three major world regions. Getting the first spot in five tested categories and only missing the last top spot by a small margin. Outperforming the next ranked VPN by sometimes doubling the performance.

PrivateVPN is in second and TorGuard in third place with solid performances and no collapse in performance for any of the regions.

ExpressVPN, Private Internet Access and PureVPN share the next three ranks as they have different strength and weaknesses, but overall the performances have been very similar.

ProtonVPN was ranked last which is mostly due to very poor Asia performances but they did manage to get good average results in the US compared to some of their competitors.

Figure 1, the combined download and upload speed showing overall network performance for all tested locations connected to local and oversea VPN servers.

 

Testing Approach

Performance Test

For the performance test, we measure the speed of the download and the upload performance. Three geographical locations were chosen: the US location at the West-Coast, the European location in the UK and the East-Asian location in Japan. The VPN locations to be tested were in geographical proximity to the VM host location. Not all products allow such specifics but instead allow the overall country.

The third-party command-line application provided by speedtest.net is used to measure the performance. To our knowledge, there is no known affiliation between the site and its holding company Ookla, LLC and tested VPN products. The performance testing tool is set to independently choose the optimal server test location. All tests are run five times with a minute break in between.

 

For the test, the testing tool is run on all three geographical locations with no active VPN. The results are used as the benchmark to compare to the VPN products and subsequently evaluate them.

For the comparative purpose, the server IDs are independently chosen by the testing tool during the unencrypted reference test and used to test the performance on the non-encrypted connection to the oversea locations.

Each product is tested for all three VPN locations. When the VPN location is the same as the

geographical location the “default connection” is chosen, otherwise the VPN location is manually selected to connect to the oversea VPN server locations previously mentioned.

 

Windows

The tests were performed on Virtual Machines hosted on Microsoft Azure cloud computing service. All test devices were of identical hardware configurations. All patches available in September 2020 were previously installed.

There are a few general principles that were followed:

 

• Product cloud/Internet connection. The Internet was available to all tested
• Product configuration. All products were run with their default, out-of-the-box configuration. Including default protocol used

Limitations

The test is measured at a certain point in time and may not reflect the average performance over a longer period in time.

Also, the test was performed in parallel for all locations that means that the workload will have differed for each location because of the different times of day in each location.

 

Nevertheless, it does give a glimpse of the capabilities and service provided by the different VPN providers.

 

 

Test Results

Performance

Due to the nature of the technology, using a VPN connection will almost always have a performance reducing impact compared to an unsecured connection. The reduction is reflected in the following graph.

 Figure 2, the average download and upload performance in Mbps of unencrypted connections compared to the industry averages of VPN products for the three tested regions.

The differences for download and upload speed between secured and unsecured connections are not to be disregarded. The download speed is around 10 to 30 times higher and the upload speed three to nine times higher compared to the average connection secured by a VPN product.

Generally, the measured unprotected x performance for downloads is best in Asia followed by the EU and by the US and best upload performance was observed in the US followed by Asia and by the EU.

The overall best download performance for VPNs were observed in the US followed by the EU and then Asia. The performance drop from the US to the EU is around 20% and from the EU to Asia another 10%.

For the VPN upload performance, the EU and US switch places with the EU showing 10% improved performance over the US and the US is 20% faster than Asia.

 

 

Download

Following are the graphs representing the measured download performances for the three different world regions tested. Each is connected to a local server, either by choosing “best option” in the VPN client or manually connect to the local server.

 

 Figure 3, the average download speed in Mbps in Asia for local VPN connections compared to the unencrypted connection and industry average.

In Asia, the reference performs about 30 times better than the industry average for VPNs. NordVPN is significantly outperforming the other product with results almost twice as good as the next two products, which are PrivateVPN and TorGuard. These two still perform significantly better than the rest of the products which manage only two-digit, one digit download rates in the region.

 

Figure 4, the average download speed in Mbps in the EU for local VPN connections compared to the unencrypted connection and industry average.

 

In the EU the reference download value is not as high as observed In Asia. Also, the industry as a whole performs slightly better than in Asia. There is a thin lead for Private Internet Access over NordVPN. Both products share a 10% lead over the next group of products PrivateVPN, PureVPN and TorGuard which still perform reasonably well and in some cases significantly better than in the Asia location. ExpressVPN and again ProtonVPN are at the rear-end of the list.

Generally speaking, there is a much smaller spread of performances results than in the other two geo-locations.

 

 

 Figure 5, the average download speed in Mbps in the US for local VPN connections compared to the unencrypted connection and industry average.

In the US the industry average performance is the best of all three geo-locations. It is again NordVPN performing substantially better than the rest of the products, outperforming the second-placed PrivateVPN by more than a quarter of the download speed. PrivateVPN holds it second place in this location solidly over the next group of products ProtonVPN, TorGuard and PureVPN. There is another big drop in the performance from Express VPN and last Private Internet Access at the bottom of the pack.

 

 

Upload

Following are the graphs representing the measured Upload performances for the three different world regions tested. Each is connected to a local server, either by choosing “best option” in the VPN client or manually connect to the local server

 

 Figure 6, the average upload speed in Mbps in Asia for local VPN connections compared to the unencrypted connection and industry average.

Asia again is showing a solid performance when tested without an active VPN. The upload ranking is the same as seen for the download graph at this location. NordVPN outperforms the second and third product PrivateVPN and TorGuard by over 50%. Then there is a small drop to fourth place - Private Internet Access, and another significant drop for ExpressVPN, ProtonVPN and underwhelming PureVPN.

 

 

 

Figure 7, the average upload speed in Mbps in the EU for local VPN connections compared to the unencrypted connection and industry average.

 

The unsecured upload performance measured in the EU is the slowest of all tested regions, yet the VPN industry average outperforms Asia and the US. NordVPN almost doubles the performance of the second placed - PureVPN. Still, PureVPN leads the remaining vendors comfortably with Private Internet Access at third place. With more than 30% drop in performance PrivateVPN and TorGuard come in fourth and fifth place. ExpressVPN and ProtonVPN are placed last.

 

 Figure 8, the average upload speed in Mbps in the EU for local VPN connections compared to the unencrypted connection and industry average.

The US Upload reference speed is the best of all three geo-locations. The performance ranking of products is almost identical to the download ranking. Only ProtonVPN and TorGuard switch places. NordVPN is again leading by a significant margin followed by two almost equal - PrivateVPN and

 

 

TorGuard. A slight drop later is ProtonVPN which is almost equal to the industry average. The last three products, PureVPN, ExpressVPN and Private Internet Access, drop by an average of 30Mbps with each rank.

Overall Network performance

Showing the cumulative performance of the download and upload performance provides us with the overall network capabilities. This can be taken as an indicator of the network capabilities available by the VPN provider.

 

 Figure 9, the combined download and upload speed showing overall network performance for all tested user when connecting to local locations

There are some observations to be made when inspecting the combined results for upload and download performances for all locations connecting to the local VPN server. NordVPN outperforms the remaining VPN products by a margin of over 60%. PrivateVPN and TorGuard both show very similar performances in download, upload and overall. After another performance drop of 40% Private Internet Access is at fourth place, closely followed by PureVPN, which has overall better download performances but yields due to the lower download performance. The final two places in the list are held by ProtonVPN and a close last Express VPN.

 

 

 

Figure 10, the combined download and upload speed showing overall network performance for all tested user when connecting to oversea locations

 

Additionally, when reviewing the overall network performance, not to the local VPN server but to oversea connections such as Asia to the US, the picture is similar but not quite the same. The first three places are unchanged. NordVPN leads by an even bigger margin more than doubling the next place due to its very significant download results. ExpressVPN climbs to fourth place with solid download and upload performances. Closely followed by PureVPN which has the second-best overall oversea download results but didn’t achieve similar results with its upload speed. The last in the list are Private Internet Access and ProtonVPN.

 

 

Conclusion

The objective of the here-presented tests was to assess the download and upload performance by VPN products.

With its second to none showing in the majority of speed tests performed, NordVPN makes it hard for a user to get past its VPN solution. NordVPN provides a very fast VPN product which came first in the overall testing for all tested products. It loses out top place only once, for downloads in the EU and only by a small margin but topped all other test indicators, sometimes doubling the performance of the next competitor. Indeed, the NordVPN product delivered convincing results across all regions. It showed consistency in the network speed in both tested speed aspects in all regions connecting locally as well as connecting overseas.

The second and third-placed products are PrivateVPN and TorGuard. Both show consistent performance. PrivateVPN with a very small margin doing better in most regards than TorGuard. For oversea uploads TorGuard managed to slightly outperform Private VPN. Both products are stable across all locations and usually, one or both are found in the top three of the test.

ExpressVPN, PureVPN and Private Internet Access share rank four to six. On the one hand, PureVPN and Private Internet Access do very well in Europe. On the other hand, Private Internet Access is last in both categories for the US. PureVPN does very well for downloads overseas and generally better than the other two in downloads. All three products perform poorly in Asia.

ProtonVPN, like PureVPN, delivered a dissatisfied performance in Asia. It is ranked last in the EU. Only in the US does the product manage to achieve a solid middle field ranking.