How we test laptops at PCWorld
Since 1983, PCWorld has been testing PCs, and while we made the jump from paper to digital years ago, our mission remains the same: We’re here to help you make better choices about your PC hardware purchases.
Most importantly, this mission extends to our laptop coverage, which we handle with utmost care. If you’re wondering how we evaluate laptops and Chromebooks, buckle up for an in-depth explanation of the testing process.
To determine performance, we use benchmarking software like PCMark, 3DMark, and Cinebench. These tests, which we’ll cover in more depth below, are designed to give the CPU and GPU a sweat-inducing workout. We also evaluate more subjective matters like the laptop’s build quality, its typing experience, audio quality, available ports and more. Real-world testing is central to the PCWorld method.
After the testing process, we rate the laptop on a 0 to 5 scale, and determine whether the machine deserves an Editor’s Choice award.
Looking to pick up a brand new laptop? Check out PCWorld’s roundup of the best laptops available right now.
How we test Windows laptops
IDG / Matthew Smith
All of the Windows laptops we review are put under a series of stress tests. PCMark 10 measures CPU performance while the Handbrake video encoder test gauges CPU performance over time as well as indicating how well the cooling system handles throttling. Whether the Windows laptop is a budget machine or a more premium model, we run the same tests regardless of price.
That said, to keep things fair, we don’t dock the final score of budget laptops for having weaker performance than more premium machines. We evaluate budget laptops based on what they’re designed for: to provide value for more simple computing tasks. Now, here are more details on our benchmarks:
- PCMark 10: We run PCMark 10 to determine a laptop’s general performance. It runs workloads like web browsing, video chat, and light spreadsheet use. If the score is north of 5,000, then it’s a good laptop for productivity.
- Cinebench R20: This benchmark is designed to run all of the cores of a laptop’s CPU. It does this by rendering a complex 3D scene in a short amount of time. The end result reflects how well the laptop handles processor-intensive multi-threaded workloads.
- Handbrake 0.9.9 Encode: Handbrake is a multithreaded, long-duration test. We use it to encode a 32GB 1080p MKV file with the Android Tablet option switched on. The idea is to see how much performance drops when a laptop is under a lengthy computer load. The more CPU cores you throw at this benchmark, the faster it will run. This is a timed test, so lower scores are better.
- 3DMark Time Spy: We use the 3DMark Time Spy benchmark to determine a PC’s general graphics performance. It’s more demanding than 3DMark’s other benchmarks, making it a suitable test for more premium laptops. Time Spy’s graphics tests are rendered at 2560×1440 resolution and we use overall score. Higher numbers are better than lower ones.
- Battery life: To test the battery, we loop a 4K file of the short film Tears of Steel until the laptop dies. We run this test twice with the brightness level set at 250 to 260 nits. We also make sure to unplug any USB drives, as they consume power. Battery life can be impacted by a multitude of variables such as screen size, display type, battery size, and so on. A high resolution touch display will drain the battery much faster than a non-touch display, for example.
IDG / Matthew Smith
Gaming laptops run through the same series of tests as the Windows machines above. However, in addition to those benchmarks, we also run the built-in benchmarks in Shadow of the Tomb Raider and Metro Exodus: Enhanced Edition. While these titles are older, they’re still valuable because they offer stable performance. Neither the developers nor graphics card makers are providing performance updates for these titles anymore, making it easy to compare a newer laptop’s performance against older ones. In the future, we intend on adding more gaming benchmarks to our testing regimen.
- Shadow of the Tomb Raider: The Shadow of the Tomb Raider benchmark is built into the game and is super easy to run. We run this benchmark at the Highest graphics setting at 1080p. When the test wraps up, we record the average frame rate. The higher the frame rate, the better the gaming performance. This test can optionally show you how well the laptop handles ray tracing, which makes lighting effects look more realistic.
- Metro Exodus: Enhanced Edition: The Metro Exodus: Enhanced Edition benchmark is more demanding. We run this one at the Extreme graphics detail at 1080p. Again, the higher the frame rate, the better the performance. Like the Shadow of the Tomb Raider benchmark, we also record the average frame rate. This benchmark can also test ray-tracing effects as well, if desired.
IDG / Matthew Smith
Chromebooks aren’t designed to run the benchmarks covered above, which means we have to turn to web-based benchmarks to gauge performance. CrXPRT 2’s performance benchmark runs through six workloads total and is designed to see how the Chromebook handles day-to-day tasks. We also use CrXPRT 2’s battery benchmark, which loops videos and various tasks until the laptop dies. Below you’ll find the specifics of each web-based test.
- CrXPRT 2: The CrXPRT 2 performance benchmark measures the speed of the Chromebook using six different workloads. In other words, it determines how well the Chromebook handles day-to-day tasks like watching movies, editing images, and so on. It’s a good baseline for simpler tasks, which is what Chromebooks are specifically designed for. The higher the number, the better.
- Speedometer 2.0: Speedometer is a browser benchmark that gauges the responsiveness of web applications, which is pretty darn important for Chromebooks, as they’re entirely cloud-based. It evaluates the responsiveness by adding items to a to-do list over and over again.
- Basemark Web 3.0: Basemark Web 3.0 gauges how well the Chromebook can use web-based applications. This benchmark includes a variety of system and graphics tests to determine this. To get the best results possible, we first close all open tabs. The test itself then only takes a few minutes from start to completion. Similar to the CrXPRT 2 performance test, higher numbers are more desirable.
- Kraken 1.1: The Kraken 1.1 benchmark is a creation of Mozilla’s. Essentially, it’s a JavaScript performance benchmark that, according to Mozilla, measures the speed of several different test cases that have been extracted from real-world applications and libraries. The results are then recorded in milliseconds, so the lower number is the one to aim for in this specific case.
- Jetstream 2: The Jetstream 2 benchmark runs various advanced workloads as well as programming techniques, effectively combining both JavaScript and WebAssembly tests. Essentially, the Chromebook gets the “gold star” for booting up quickly and executing code quickly.
- CrXPRT 2 battery test: Not only is CrXPRT 2 a performance test, but it’s also a battery test! First, we make sure to disable the power settings, so the Chromebook doesn’t go to sleep on us in the middle of the battery test. We also lower the brightness to about 50 percent hoping to emulate realistic day-to-day use. As for the test itself, it continuously loops videos and various tasks until the laptop runs out of juice.
Subjective evaluations are just as important as the raw data we receive from the software benchmarks. It’s important to know how typing on a specific laptop keyboard feels, or what the audio sounds like when it’s cranked up all the way. That’s the real meat and potatoes, folks. We ask all our laptop testers to spend a good amount of time with each machine, and make observations about what the laptop is actually like to live with.
- Design: Sometimes design boils down to aesthetics and personal preference. Maybe you love the color scheme or the texture of a laptop’s lid. Some laptops are just boring gray slabs, which is entirely valid to call out. But more importantly, a laptop’s weight, shape, and overall “feel” can be evaluated against other machines in the same category and pricing class.
- Build quality: Laptops need to have a rugged build, especially if they’re lightweight and portable, as they need to be able to withstand the daily abuses of life. We look for any flex in the display or keyboard as well as the strength of the hinges connecting the display to the keyboard.
- Keyboard: Keyboards can really make or break a laptop, as nothing ruins useability quite like a poor typing experience. We evaluate travel distance and actuation points. At times, it comes down to just the feel of the keys.
binary comment
IDG / Matthew Smith
- Trackpad: Trackpad placement can be awkward, especially if it’s off-center. In addition to evaluating the positioning of the trackpad, we also observe the accuracy of it. Is there a responsive physical click? Does it ever misfire? These are just a couple of things we consider when evaluating the quality of the trackpad.
- Display/Screen Quality: The display is one of the most important aspects of a laptop. If you’ve got a crappy display, then what’s the point? We check out everything from the brightness level and richness of colors to the reflective glass touch layer (if there’s one) and whether the display has active pen support. All of these aspects can have a drastic impact on your experience.
- Audio: Historically speaking, laptop audio doesn’t have a great track record, especially when you crank the volume up. That said, the positioning of the speakers really matters here. Upwards firing speakers, for example, generally produce better audio. That’s why we ask the reviewer to listen to music, participate in video calls, and just use the laptop like they normally would.
- Webcam: Most laptops are outfitted with 1080p webcams these days, which is sharp enough for video conferencing. That said, we look for any graniness as well as the inclusion of a physical privacy shutter. Webcams can be an important factor when buying a laptop, especially for remote workers.
- Biometrics: Using a fingerprint sensor to log in to your laptop is always a plus, as security is really important to some buyers. That’s why we evaluate whether a laptop has a fingerprint sensor or Windows Hello support, and how well it works.
- Ports and connectivity: Limited connectivity options can be a real bummer, especially when it comes to gaming laptops, where you may want to hook up to an external mouse or monitor. There’s nothing more aggravating than lugging around an adapter. That’s why we ask the reviewer to carefully examine the variety of ports.
From budget laptops primed to handle everyday tasks to high-end gaming machines with loads of graphics firepower, we test just about every type of machine at PCWorld. It’s important to note that when we select a laptop to review, we write from the lens of who the laptop is designed for. Are you a college freshman on a tight budget, or are you a professional who travels often and needs a powerful yet portable machine? We take every aspect into consideration, especially when it comes to budget and hardware. Here’s a list of laptop review roundups we actively update:
- Budget laptops (under $500)
- Mainstream and premium laptops
- Chromebooks
- Entry-level gaming laptops (under $1,000)
- Midrange and high-end gaming laptops
PCWorld’s rating system is fairly straightforward. We rate the laptop on a scale of 0 to 5 stars, with 5 being the highest. Rarely do we rate a laptop lower than 3 stars. If we were to rate a laptop lower than 3 stars, one should assume the performance or battery life was miserable, or the machine has a serious design flaw. We also award an Editor’s Choice badge, which signifies the machine is truly exceptional. If a laptop gets 4.5 or 5 stars, the Editor’s Choice badge is a lock.
Laptops