As I wrote recently in article on how to assess the the best upgrades for Photoshop performance, it’s important to have some objective measures of performance in order to get good value for your money when upgrading your computer or otherwise optimizing performance. While there are a lot of great benchmarking tools out there, I’ve never felt there was tests designed to specifically assess Photoshop performance in a way that answered my questions. What performance should I expect from Photoshop with the latest hardware? Which features in Photoshop are most improved? etc.

So I decided to build my own benchmarking test for Photoshop: “G-Bench“, which I am making available for free. It will also be built into my upcoming Lumenzia v10+ and Web Sharp Pro v3.4+ updates (both should be available by Dec 1), where you can find it via the flyout (top-right three bars icon) under Utilities. And I have also created a standalone version of G-Bench in case you don’t have either or want to have a dedicated button for more frequent testing (which you can download for free here, as well as from the footer of my newsletters going forward). All of these panels are UXP panels, which means they can run natively on Apple Silicon (and are fully compatible with any Mac / Windows machine running PS 2023 or later).

Rational for a dedicated Photoshop benchmark

While it’s great to understand how fast your CPU or GPU is in general, it doesn’t tell you how fast a particular program will run. For example, with upgrade to the M1 Max (see my full review of the M1 for photographers), I found one video editor’s export time cut by 88% and another only cut by 35%. Software performance depends on the software as much as the hardware. If it isn’t optimized for multiple cores, GPUs, new instruction sets in the CPU, etc – then it may not show improvements in a generic bench test.

Furthermore, this applies not just to the program, but specific functions within it. For example, you might find that Surface Blur is suddenly 5x faster, while file saves only improved by 20% because they have different bottlenecks and may be optimized differently. So it’s important that your testing reflect the way you actually use the software. G-Bench is built to reflect the needs of photographers using Photoshop. So it focuses on testing where we tend to end up waiting: running filters, saving smart objects, etc.

Of course, you also use your computer for a lot of things other than Photoshop, so I recommend using G-Bench in addition to other benchmarks. They all compliment each other well.

 

How to use G-Bench

Just launch the “test and compare” dialog by clicking the button in G-Bench, or via flyout / utilities in Lumenzia and Web Sharp Pro. The first time you use it, you’ll need to set your preferred folder. This folder is used both to save the results of tests you run as well as allow you to review and compare tests you have already run. Once you’ve set that folder, you can click “run a test” to begin. The test may take roughly 5-60 minutes depending on the speed of your computer. Note that the display will update sporadically while the test is running (ie, the preview may appear lagging or frozen in the middle of the test). When it is done, you’ll see a popup dialog with results. If you leave the checkbox in that popup checked, a CSV file with your results will be saved to your folder when you click OK.

You may compare your tests  to each other, or to the tests which come with the panel. The “result A” and “result B” dropdowns each contain the same full list of all available tests. You can even import tests from other people by putting a copy of their CSV file into your designated folder. The top of the list reflects tests in your folder, and the bottom of the list reflects tests which come with the panel. Just select the two tests of interest (set A to your system and B to the reference data) and the click “compare results” to create a graph of the “weighted time”, which is the ultimate score for each test.

 

If you’d like to compare details of specific items in the test, just navigate in your file browser to the folder you selected to open the CSV files in Excel or Numbers. The CSV file contains test conditions (such as computer hardware) and results of specific tests (such as the time it took to enlarge the image 2.5x). If you’d like to see the CSV files for the included tests or use a formatted template to make the results more pretty, please click “export included tests” in the test and compare popup window. Note that the result A/B dropdowns will not list the duplicates of internal tests, they will only show once as included tests.

The same hardware can return different results with the exact same test. There are a variety of reasons for this, and it is important to try to test under realistic conditions – but also under conditions which would be consistent to help compare one test to another.

Tips for best results:

• Run the test a few times to get a sense of how much results vary on your machine. Typically, they will fall in a tight band. Every once in a while, you might see a big change – which probably indicates some background interruption or other unusual behavior in the operating system or PS. For example, when I ran the test shortly after a fresh install of Windows, the test times on a Surface Pro 9 were about 60% longer than I’ve otherwise seen. Most likely, there is some kind of optimization in the background when the computer is in this initial state. Try restarting the computer and testing again if you believe your result is an outlier.
• Avoid any heavy background processes (for example, importing images in LR while the PS test is running may cause slower results as PS competes for system resources).
• Avoid thermal and power limits. If your computer is in a very hot environment or running on minimal battery power, it may use a reduced performance mode.
• Avoid testing if free space on your scratch disk is low or available RAM is unusually low, the computer is likely to run extremely slowly in an inefficient scenario like this.
• Optimize PS as you normally would and/or try testing with different settings. Settings such as the amount of RAM you allow Photoshop may affect results. Be sure to restart PS after changing preferences, as some changes do not take immediate effect.

 

How to interpret results

I would also encourage you to run the entire test a few times. You’ll likely see the main result (weighted time) varies by 1-5%. This is normal, and you assume that a result in the middle is probably most indicative of performance. If you see a much larger variation, that likely indicators some other factor at work – such as a background program being very active during one run and not another. It could also be the result of changes you’d made in Photoshop (such as disabling OpenCL, which is one of the factor tracked in the test conditions recorded in the CSV). You might also see variation from one version of Photoshop to the next (such as speed improvements after code optimization, or perhaps a delay if there is a bug).

The line item tests should be relatively straight-forward. Each test is fairly specific. The reported time is an average of numerous loops (in most cases). This repeat testing helps provide better accuracy, as the test time varies a little each run – even under the same conditions. The fastest and shortest loop time is noted, and if you see huge variability there, you may wish to re-run the test. This might be caused by some transient issue that delayed Photoshop, but does not reflect the results you would typically see

The line items include a “weighting“. The purpose of this is so that each test reflect the real impact it has on a general workflow. You can think of this roughly as my estimation of the number of times you would use a given feature (on average) to edit a photo. So a weighting of 0.1 for motion blur indicates that I expect very few workflows will use this specific feature, whereas a weighting of say 10 for Adobe Camera RAW indicates I expect that feature would be used frequently. It’s much more predictive than just adding up all the times, but it’s far from perfect and certainly not a representation of exactly how you work. We are all a bit different. If you would like to try different weightings, please use the “export included tests” option and then open the exported XLSX (Excel template) to copy and paste your CSV data and then view the analysis tab to try your own weighting and look at the new result. You should then do the same with another data set, as the weighted time is only meaningful when compared to another test using the same weighting.

The total “weighted time” at the top is calculated by taking each tests average loop time and multiplying it by its weight, and then summing up all those numbers. So for example, if a test has a time of 7.2 seconds on average and a weight of 2, then it will contribute 14.4s to the total weighted time.

A few other notes on the data:

• The rule of thumb I’ve always applied as an engineer is that if there isn’t a 20% change in performance, you’re unlikely to notice it.
• The CSV file contains several important notes, be sure to look for them.
• There is some setup for some tests, which is un-tracked. For example, the auto-align test creates lighter and darker versions of the base image and rotates them before invoking the alignment. Only the alignment time is recorded in that case. So the total active test time reported in the CSV will be somewhat less than the actual time the test takes to complete.
• It’s important to keep these numbers in context. I deliberately kept the weighting for the brush tests lower than the expected use. These brush strokes take me about 1 second to do by hand, which means that some of the test time isn’t something I would perceive as any delay from the computer. You can certainly feel brush lag when it takes 2-3s for a large brush. Rather than trying to correct the values for this, I felt it best to just be simple and transparent with the data.
• These are tests based on the time it takes to process the output. I do not have a way to test the time it takes for the ACR interface to open, for example. I’m effectively just testing the time it takes from the moment you click OK in a dialog until you get the result.
• I do not plan to make many changes to the tests, as revising the tests would make previous test data useless. However, I anticipate I’ll eventually make a v2 and welcome your comments below on other tests you think would  be important to consider for your use of Photoshop for photography.

 

A few general observations from my own testing

I’ve tried running the test under various conditions on my computer to see what impacts performance and which aspects of Photoshop tend to be impacted. Your results will differ, but here’s what I see on an M1 Max and 2018 Macbook Pro:

• While Rosetta2 is amazing to run old software on your M1 Mac, it is also far slower than I expected. My G-Bench times are nearly twice as long when running under Rosetta. It’s impressive performance under emulation, but my old 2018 laptop runs slightly faster than the M1 under Rosetta2. If you need to run some old plugins occasionally, I would just launch PS under Rosetta selectively. Otherwise, native is clearly the way to go with Apple Silicon.
• As expected based on evaluating with Activity Monitor, GPU benefits are limited at this time. When I disable the GPU in Photoshop, the test times are nearly 50% longer – but almost the entire difference comes from the smart sharpen and enlargement test items. There is certainly a lot of untapped potential in a computer as powerful as the M1 Max, and I expect that software companies in general will increasingly take advantage of this more often.
• Background application activity generally doesn’t matter, until you really push things.
  • Allowing a web browser and anti-virus to be active in the background increased the weighted time by just under 3% (most of which came from the browser since there were no new files for the anti-virus to scan). You can safely leave a lot of things in the background without having any detectable impact on Photoshop.
  • If I leave Final Cut Pro X rendering a large video in the background, the weighted time grows by 34%. Put another way, it still takes 43% less time than my old 2018 computer for Photoshop tasks even after I throw this heavy load on it. So even though I may not be using my CPU and GPU to the max in Photoshop, the excess capacity is still a huge benefit because I can run heavy applications in the background without causing much detectable change in Photoshop’s performance.