PhoneArena redefines smartphone camera testing with the PhoneArena Camera Score benchmark

The camera is arguably the most important smartphone feature these days. It is the most contested field of battle for manufacturers: they spend enormous resources on R&D, and they often dedicate the largest chunk of time during launch events to showcase their new camera innovations. Meanwhile, emotions always run high among fans as they try to establish who has the best camera of all: the iPhone, the Galaxy, the Pixel, or some up-and-coming challenger like OnePlus or Huawei.In the race towards the most advanced imaging setup, phones started to come along with two, three, four, and sometimes even more snappers on board, each serving a different function. This made them extremely versatile, but the fact is that the smartphone camera became an increasingly complex monster, and while tech media did its best to provide fair and unbiased coverage, this increase in complexity, as well as the ever more heated competition in this field meant that surface-level, subjective evaluation no longer cut it. Don’t get us wrong, we believe expert subjective evaluation cannot be fully replaced when it comes to areas like photography and camera capability, but we realized this approach alone is not enough.What the market was needing was a reliable, holistic approach to smartphone camera testing: one that took into account the nearly countless major and minor performance aspects of all cameras involved in a single camera setup. If you wanted to get a full view of a phone’s camera performance, you had to make do with partial and fragmented coverage, often scattered across different sources.

So, we decided it was time to systemize the enormous camera testing know-how we had accumulated over the years, and create a whole new process for a more objective and unbiased testing that represents a holistic evaluation of a phone’s total camera setup.

We called this process: the PhoneArena Camera Score.

But we didn’t stop there. We knew it was extremely important for both enthusiasts and casual users to be able to compare and choose the best phone camera for their needs (or just for the hell of it), which is why we structured the testing process as a benchmark that yields reliable results that can be compared across brands and models.

Here’s how the PhoneArena Camera Score works

We evaluate each camera on a phone across a multitude of criteria: from essentials like exposure, color representation and detail resolution, to specialties like portrait mode separation quality and viewfinder reliability.

Depending on the camera (main, zoom, ultrawide or selfie), a bigger emphasis is put on those features seen as more foundational for the camera’s purpose. For example, the level of resolved detail has a higher relative weight when evaluating zoom cameras, as compared to when evaluating the ultrawide camera, for example.

Right now, our testing procedure covers more than 15 different criteria just for the main camera! To get a better idea of the type of things we look at, here are some examples:

• Exposure
• Subject exposure
• Color temperature
• Color tint
• Detail resolution level
• Oversharpening artifacts
• Portrait mode separation quality
• Preview reliability (HDR, resolution, portrait mode)
• Stabilization

We measure each camera’s still and video performance in a controlled environment, in order to keep results comparable. Our studio environment challenges phones with a number of different scenes, meant to provide a more holistic measurement of performance across different scenarios.

However, it’s important to note that we’ve checked these controlled environment tests against performance in real-world scenarios multiple times and made sure that our test results are representative of the kind of image quality you’re bound to get once you go out there and start taking photos. On top of that, we continue to double-check against real-world performance and use real-world results to evaluate certain camera aspects, such as portrait mode separation quality.

The widget will display an overall Camera Score rating, main sub-ratings for Photo Score and Video Score, as well as per-camera sub-ratings, so readers can get an idea of a particular camera’s showing.

The widget displays all these ratings, plus it provides indication of how a particular score compares relative to the best in each category. In the case of the examples above, you can see that the iPhone 15 Pro Max holds the current best score for Main camera in video, while its zoom photo score of 23 falls a bit short of the current best, which is 24 and belongs to the Pixel 8 Pro.

It’s important to note that we’ve assigned different weights to the different types of cameras (main, zoom, ultrawide and selfie). The ratios of the weights are as follows: 100 for the Main camera; 35 for Zoom; 30 for Ultrawide; and 35 for Selfie. These ratios are the same between the photo and video categories.

All the sub-ratings in the photo category make up the overall Photo Score, while the final Camera Score is the result of both Photo and Video scores, with the current weighting being 50:50 between Photo and Video.

The road ahead

This is just the beginning of the PhoneArena Camera Score, which ushers in a new era of sophisticated, objective, yet easy-to-understand camera rating, and we’re thrilled to offer this solution to every smartphone fan out there who wants to make the right decision when choosing their new product.Today, we’re launching v1 of the PhoneArena Camera Score, but we already have plans to add more testing criteria to our procedure, for an even deeper, more holistic and relevant smartphone camera rating. Our controlled testing setup is only bound to become more sophisticated with time.

You can already find the Camera Score widgets for some of the current top flagships (the whole iPhone 15 family, Pixel 8 and 8 Pro, iPhone 14 Pro Max, Galaxy S23 Ultra and Pixel 7 Pro) in the camera sections of their reviews, as well as in our dedicated camera review articles for each of these products. Over the coming weeks and months we’ll be adding the Camera Scores of all new phones that come in for review, as well as the big releases that hit the market over the last year.