With all these issues, it’s no surprise that many testing platforms default to using Chrome to emulate different browsers. On top of that, each phone you’ve placed in a datacenter for testing must be manually updated whenever a new version of the operating system is released.
Batteries tend to swell and become a fire hazard after a couple of years, SD cards wear out and USB connections become unstable. The phones themselves pose significant reliability challenges. There are a lot of different moving pieces, from the device itself to everything that needs to be in place for traffic shaping. WebPageTest tries to use real browsers and devices for testing whenever possible, but doing that at scale has some serious challenges, particularly when it comes to testing mobile browsers.
