Sony A9 vs Olympus E-M1x focus accuracy tests

All through the summer and autumn of 2020 I have been testing the Olympus E-M1x and E-M1iii with the 300mm and 40-150 plus TC against all the main Sony cameras (A9, A7r3, A7r4, A6400, plus 200-600 Sony 5.6-6.3). This has mostly been at the Hawk Conservancy Trust (HCT) in Hampshire, UK. The HTC birds are  vultures, falcons, kestrels, owls, eagles and kites. There is a huge range of shooting conditions, from large birds flying fast right at you (and sometimes hitting the camera) to aerial raptor combat, or super fast falcon  dives at very long range.  A Lanner falcon is about the size of wrapped cheese sandwich and flies at over 100mph, and it’s a challenge getting even a focus point on one, so some of these tests are quite extreme. This post covers a comparison of the Sony flagship wildlife camera, the A9, with the flagship Olympus equivalent, the EM1x.

Note: for Sony shooters and others I have done a much more detailed, super-nerdy post of the full methodology, data set, and sample images for the A9 by itself here.

AF Stress tests

The HCT provides a pretty good location for testing autofocus hit rate.  There are two displays daily, using more or less the same birds flying in the same sequence in the same way. This is about as close as you can get to a demanding but repeatable test setup for birds in-flight.

There is a morning session in which birds fly fast towards or across the camera, quite close in, using focal lengths between 150 mm and 450 mm. The starting position of the birds is almost always known.

The afternoon session has extremely fast and unpredictable bird movements where either the speeds are extreme as with the falcons, or the movement is incredibly sudden and you have maybe 1-5 seconds to focus, compose, and shoot.  The afternoon focal lengths are usually around 600 to 800 mm.

As a rough generalisation, the morning session tests the ability to lock focus over a sequence of shots, and the afternoon tests the ability to instantly acquire focus and capture very sudden movement.  The latter is a very tough test but quite representative of places like RSPB Bempton cliffs for example, where you have literally only seconds to grab shots of extremely fast moving Puffins or Gannets.

Camera setup

I tested the EM1x (firmware 1.3) and the A9 (firmware 6.0) each for two successive days. I then took the best session from each camera and used that details to generate a hit rate score.  For the Olympus E-M1x, I used 9 or 25 point continuous autofocus (CAF), with AF sensitivity set to +2 for oncoming birds and from 0 to -2 for panning shots, and electronic shutter L, at 15 fps. For the A9, I imported Mark Smith’s excellent settings for the camera (Mark is an astonishingly good pro bird photographer and A9 fan). The AF was usually zone, with 20 fps electronic shutter.  For both cameras, I shot in manual, wide open, at 1/2000 with auto ISO set to 6400 max, and used rear dial exposure compensation plus highlight/shadow indication to get the exposure right for the bird.

Results

I checked the focus accuracy of every image using FastRawViewer at 1:1 in the original raw format, with additional sharpness applied and using focus peaking tools, and exposure adjustments where necessary.

In the morning (focus lock) session the A9 got the highest score of all, at 72%. The EM1x was just behind at 61%. The EM1 iii followed behind both of these at 54%. 

But the tables were turned in the afternoon (focus acquisition) sessions. Here, the EM1x got the highest hit rate of 45%, with the A9 at 39% and the EM1 iii at 33%.  It is worth noting that the afternoon session is incredibly challenging for a focus system and these are all extremely good results. The EM1x had the highest average score over both sessions at 51%, and also produced some superb shots. The A9 was only just behind at 49%.  

Keepers and portfolio shots

The A9 frame rate is one-third faster than that of the EM1x. The number of keepers was more or less in proportion to this, with the EM1x generating around 60 over a complete day and the A9 generating 100. Out of those I got 14 portfolio shots from the A9, and 9 from the Olympus E-M1x. In my opinion, the EM1x produced shots of higher quality and captured more dramatic moments than the A9, but they both performed extremely well. Many of these shots are in the HTC gallery albums here for kites, and here for falcons, and you can judge for yourself. See below for why there are no A9 shots in the falcons album by the way.

That can’t be right!

There has been a lot of forum comment on the impossibility (aside from my implicit incompetence) of only getting a 50% hit rate with the A9, citing the test results from the excellent Mathieu Gasquet. 

I was also incredulous about this, so I checked 1600 A9 shots for the focus point. Was I just missing focus? Using AFV.exe I checked every individual photograph for where the actual focus point was in the image. This incredibly tedious process showed that for the 815 A9 shots when there was or should have been a focus point on the bird, the A9 only achieved an in-focus hit rate of 76% in the morning, and 53% in the afternoon. Or to put it another way, overall, 40% of the A9 images were not critically sharp even when there were one or more focus points on the target or the bird was in within the focus zone.   This is camera error, not user error.

Here are some examples. Each image shows the AFV.exe screen on the left, showing the location of the focus points, and a higher resolution view of the same image on the right, showing the level of focus.  

I have about 300 pictures where the A9 put a focus point directly on the bird, but the bird was out of focus. It would be tedious to show all of these, but the data exists.

How can these results be so different from Mirrorless Comparisons?

I am familiar with Bwlch Nant Yr Arian in Wales, UK, where Mattieu takes his test shots. The birds are large Kites, often in the hundreds, at a relatively constant distance of around 30-50 m. You choose your bird from many candidates, lock focus, fire your shots and voila, 90%!  It’s not a challenging situation. In similar (but tougher) circumstances at the HCT with large birds, the A9 had better focus accuracy than the E-M1x in my tests, by roughly the same margin as Mattieu obtained, but at a lower overall hit rate (71% in my tests vs 90% in his). But most of the shooting situations at the HCT and in my own shooting experience are much more challenging than at Bwlch Nant Yr Arian. In particular in the HCT afternoon session it is extremely difficult even to follow the birds, let alone lay a focus point on them.

It was in these afternoon sessions where the EM1x was better, in situations where you have to immediately focus and shoot a bird whose position is changing wildly and quickly and you have maybe 1-3 seconds to get the picture. In this situation, which is common when a single bird is being shot in the wild, the A9 was slower to gain focus and on many occasions did not get the shot (39% cf 45% on the EM1x), as shown above. 

I would also note that Matteiu also rates the Sony A6400 very highly, and higher than the EM1x. I took over 3400 shots over 7 sessions and found it had far worse focus accuracy than the EM1x, as I show in this post. So, although Mirrorless comparisons is a fabulous resource, Mattieu’s conclusions absolutely did not apply for me on any of the Sony cameras I tested (the A7R4, A9, A6400 and A7R3), and under the challenging conditions I shoot in.

Some have said that I had a bad copy of my A9. Possibly, but then my A9 and my A7R4, and my A6400 were all bad copies. I guess I am willing think this of Sony, but I have never had a bad copy of an Olympus, Nikon, Fuji or Canon camera, and I have owned a very large number of them.

Comments on the shooting experience

The A9 is excellent when you have achieved focus lock. It clings tenaciously to the subject, and once locked on, a high rate of images are in focus. The E-M1x is much less consistent after focus lock is achieved, and focus will sometimes drift after 5-10 shots. The answer to this, as used by expert photographers such as Scott Bourne is to ‘focus bump’ i.e. hit the focus button again occasionally, to reacquire the lock.

Where the A9 seems to be not as capable as the E-M1x, is in getting instant focus acquisition. The E-M1x will get focus immediately while the A9 takes a moment or two, and when that is all the time you have, the E-M1x seems better. That is the case for most of the ‘afternoon’ shots at the HCT.

My general impression was also that the A9 AF was less effective at long focal lengths where the image had to be cropped in. Under those circumstances there are fewer AF points available to put on the bird and the fantastic AF system struggles. Also, the combination of the A9 plus battery pack and the very big 200 to 600 zoom lens is pretty unwieldy and is far less quick to flip around than the super agile 300mm Olympus lens. There’s no question in my mind that the Olympus system is physically much more suited to capturing sudden movements just because of its lower inertia.

Using the 1.4 TC with the Sony 200-600

I also tested the Sony A9 and the 200-600 mm zoom with the Sony 1.4 x teleconverter which gives a focal length of 900 mm at f9 at the long end. This was only in the afternoon Falcon session as that is the one where the long focal length is needed. The focus hit rate was only 34% – far worse than the Olympus E-M1x.  In addition, the image quality was lower than the 300mm plus 1.4 tc, and the noise much worse. I didn’t get any portfolio quality results from this combination. These poor results and the disastrous performance of the A7R4 mean that for all practical purposes the Sony A9 is capped at a maximum of 600 mm focal length for me. 

Conclusion

The Sony A9 is undoubtedly a wonderful camera, and by all accounts improved by 5% or so by the A9ii. I was on the verge of buying one, until I assessed the results above. But overall, for me, given the types of BIF shooting I do, it has no better AF accuracy, noise or dynamic range when coupled with the 200-600 zoom (see these links for detailed anaysis), than the Olympus E-M1x, and critically, lacks that 240mm extra range at the long end.

Possibly the new A9iii could solve the problem if it has 50 Mpx as rumoured. However, in crop mode it will have no noise advantage over the E-M1x, and at a rumoured price of $5000, it is just a short hop to the new Olympus 150-400, for which 1000-2000mm is eminently achievable.