Apple "Retina Display" in iPhone 4: a Vision Scientist Perspective

Bookmark and Share
Our take on the controversy surrounding Apple "Retina Display" in the iPhone 4... Even Steve Jobs sent us an email about it:

Thanks! Well written and clearly explained.
Steve
Sent from my iPad
Pasted Graphic
I have closely followed the recent controversy around Apple's "Retina Display" following the announcement of the iPhone 4 and I would like to share my opinion as a vision scientist (I have to say that I have not seen it yet in person and I'm looking forward to). Hopefully this will clear things up a bit and provides a more balance and fairer evaluation of Apple new "Retina Display" to the public . This controversy started with the Wired article "Apple's Retina Display Claims Are False Marketing" with which I disagree on 2 points:

1) its title, that "Apple's Retina Display Claims Are False Marketing",

2) its expert's conclusion that the “Retina Display” is a misleading marketing term.

What are Apple's claims about its "Retina Display" and are they false marketing?

Apple's official marketing claim is that "the Retina display’s pixel density is so high, your eye is unable to distinguish individual pixels". In his keynote, Steve Jobs elaborated a bit more by referring to a magic number for the pixel density (around 300 pixels per inch or ppi) of a display hold about 10 to 12 inches away as the limit of the human retina to differentiate the pixels. Though Steve Jobs does not say explicitly, this magic number is in fact closely related to the standard visual acuity (20/20): a visual acuity of 20/20 means that a normal human eye can discriminate two points separated by 1 arc minute (1/60 deg) which is equivalent to an angular resolution of 30 cycles per degree (cpd). Seen from a distance of 1 foot (12 inches or 30 cm), a visual angle of 1 arc minute corresponds to a dot size of about 89 micrometers, that is a pixel density of 286.5 ppi (11.3 pixels per mm).



So, Apple "Retina Display" with its 326 ppi has a pixel density 14% better than the 286 ppi required to deliver a resolution compatible with a 20/20 visual acuity from a distance of 1 foot. More specifically, Apple "Retina Display" can deliver a visual resolution equivalent to a 20/17 acuity at a distance of 12", or for the sake of clarity to a 20/20 acuity at 10". In these conditions, refuting Apple's marketing claim would be unfair and misleading. In my opinion, Apple's claim is not just marketing, it is actually quite accurate based on a 20/20 visual acuity. However it is also important to note that the maximum acuity of a healthy human eye is approximately 20/16 to 20/12, so it would be inaccurate to refer to 20/20 visual acuity as "perfect" vision (despite the popular belief). The significance of the 20/20 standard can be best thought of as the lower limit of the normal visual acuity.

So what about Dr. Raymond Soneira's claim in the Wired article that Apple "Retina Display" is a misleading marketing term?

First let's look at Dr. Soneira's argument: he correctly states that the eye has an angular resolution of 50 cpd, which would require a pixel density of 477 ppi at a viewing distance of 1 foot. But what Dr. Soneira omits to say is that 50 cpd is the maximum resolution of the human eye, which is equivalent to a 20/12 acuity, the maximum acuity noted above (see note 1). On the contrary, a 20/20 acuity is equivalent to a resolution of 30 cpd and corresponds to the lower limit in normal acuity. Then, the question is whether this distinction between the lower and higher limits in normal visual acuity is critical enough to refute Apple's claims for its "Retina Display".

Now, Steve Jobs referred in his keynote to a viewing distance of 10 to 12" when claiming that the limit of the human retina is reached for a 300 ppi display. Why? Likely because 10" is the distance that corresponds to a 20/20 visual acuity when holding the "Retina Display". However I think that this viewing distance is too short from a practical point of view: personally, the most comfortable distance between my eyes and my iPhone 3G is about 18" (45 cm) and not 10" or 12", and guess what? At this comfort distance, the Apple "Retina Display" would provide exactly the higher limit in angular resolution argued by Dr. Soneira (50 cpd)! I believe that for most people a reasonable viewing distance when reading a book or using a mobile device is between 10" and 20" (see note 2). Under this normal range of viewing conditions, Apple "Retina Display" would have the capacity to span the full range of normal visual acuity, from 20/20 at 10" to 20/12 at 18", further justifying Apple's claims. That would be my first take-home message in this ongoing controversy.

What about the "Retina" term?

The maximum visual resolution is known to be limited by the photoreceptor spacing in the retina (in particular in its center, the fovea, where the cone photoreceptors are densely packed): cone spacing at the fovea is approximately 28 seconds of arc (about 0.5 arc minute), and a maximum theoretical resolution of about 60 cpd is possible based on this cone spacing (see note 3). This limitation partially results from the well-known Nyquist-Shannon sampling theorem that states that the sampling frequency of the retina by its individual photoreceptors (120 cpd) must be more than twice the frequency of the smallest discriminable input signal (explaining why 60 cpd would be the theoretical maximum resolution). How is this pertaining to the "Retina" qualification of the iPhone 4 display by Apple? Well, since this display can deliver a visual input to the retina with a spatial frequency up to 50 cpd when viewed from a distance of 18", it almost matches the retina resolution according to the Nyquist-Shannon sampling theorem (see note 4). As such, my second take-home message is that Apple new display can be called without dispute a "Retina" display. Could it get better? Sure, but so far Apple "Retina Display" is the closest thing ever done in display technology for the consumer market that matches the human retina resolution at these short distances (see notes 5, 6).

Sorry for this long and a bit technical blog entry, but as a scientist I think facts need to be explained, not just stated...
Bookmark and Share

William H.A. Beaudot, PhD
Founder & Chief Scientist
KYBERVISION Consulting, R&D

Finally, for those interested in playing with the resolution, pixel density, viewing distance, visual resolution, and acuity numbers we have an iPhone app "Field Of View" that has been available for more than a year on the AppStore (we will soon update it to include the new 960 x 640 resolution of the iPhone 4).

We have also updated our "Visual Acuity" app for iPhone and iPod touch to take advantage of the twice higher pixel density found in the Retina Display of the iPhone 4, which would be then the perfect device for testing "Near Visual Acuity".


NOTES:

1. Dr. Phil Plait, an astronomer, also points out this on his Discover blog ("Resolving the iPhone resolution").

2. This AppleInsider article ("Apple's iPhone 4 "Retina" display claims spark controversy") seems to disagree with me as it states that 18" is much farther than standard use for a mobile handset. I don't know what their sources are for such a claim, but common sense tells me that the most comfortable viewing distance for near reading should vary from person to person and may depend on several factors: visual acuity, display size (in terms of visual angle), and clarity of the content (which is limited by the pixel size) among others. My guess is that with the new iPhone 4 the latter will be much less a factor in the determination of this optimal viewing distance.

3. The chromatic organization of the "Retina Display" in terms of its red, green and blue sub-pixels has not been considered yet: Apple uses in its displays the RGB stripes layout commonly found in LCD panels (it is worth noting that alternative layouts have been recently proposed, see this AppleInsider article and the PenTile technology).


The human retina is also composed of 3 types of cones layered, however, in a much less regular fashion as illustrated by the human retina mosaic shown below. Moreover, the cone spacing reported above (28 arcsec) applies irrespective of the cones type which suggests that Apple's "Retina Display" has a higher resolution than the human retina in terms of its chromatic components. To demonstrate this, we can compare below "microscopic" screenshots of the chromatic mosaic of some Apple displays with the human retina mosaic when shown at the same scale (when viewed from a distance of 10"): note how small the sub-pixels already are compared to the cones in the human mosaic!

Human Retina Mosaic
Apple Displays


Chromatic Mosaic of the Human Retina: False color image showing the arrangement of the 'red', 'green' and 'blue' cones in the human retina at a location 1 deg nasal from the central fovea. The scale bar represents 5 arcmin of visual angle. Each cone can be as small as 28 arcsec. Adapted from Roorda Lab.

iPod touch 4G (326 ppi)
Pixel size: 78 x 78 µm
1 arcmin/pixel at 10"
iPhone 3G (165 ppi)
Pixel size: 155 x 155 µm
2 arcmin/pixel at 10"
iPod touch 2G (165 ppi)
Pixel size: 155 x 155 µm
2 arcmin/pixel at 10"
iPad (132 ppi)
Pixel size: 192 x 192 µm
2.6 arcmin/pixel at 10"

We have added a "microscopic" screenshot for the "Retina Display" of the iPod touch 4G and will add one for the iPhone 4 as soon we can get our hands on it (it came to my attention that some higher resolution screenshots have been since posted on Bryan Jones' blog, however the size reported for the scale bar on his pictures appears to be quite wrong... The pixel size should be those depicted in the pixel structure shown above).

4. The slight discrepancy between the maximum theoretical resolution of 60 cpd and the best visual acuity of 50 cpd appears to be due to the imperfect optics of the eye. This was demonstrated in particular by Drs. Rossi & Roorda in a recent publication, as emphasized by this Popular Mechanics article ("Does the iPhone 4's Retina Display Ignore the Science of Optics?").

5. As shown by the Wiki page "List of displays by pixel density", Apple "Retina Display" has the highest pixel density and the largest number of pixels (960 x 640) in its device category. Another phone with a slightly higher pixel density (333 ppi) appears to have been released for the Korean market only about 1 year ago: the LG-LU1400 with a WVGA resolution (800 x 480) and a 2.8" diagonal, however limited to 262K color (i.e. 2% better in pixel density, but 60% worse in pixel number).

6. Apple "Retina Display" is a high quality 3.5" TFT-LCD (Thin-Film-Transistor Liquid Crystal Display) with a resolution of 960×640 and a contrast ratio of 800:1. Apple stated that this display uses the IPS technology (In-Plane Switching) known to provide wide viewing angles and good color reproduction. It is also likely to use a LTPS TFT LCD technology (Low-Temperature Polycrystalline Silicon) often used in displays requiring higher TFT performance and low power consumption. Using this technology, much higher pixel densities have been already obtained: for example, LG Philips LCD has demonstrated in 2005 2-inch LTPS VGA LCD prototypes with IPS up to 403 ppi, Sanyo Epson Imaging Devices Corporation demonstrated LTPS LCD up to 500 ppi in 2006, and Casio announced in 2008 the industry's highest resolution with a 2.0-inch color TFT LCD screen with 546 ppi that is with a 960 x 540 resolution. What pixel density should have the perfect "Retina Display"? Assuming that 10" is the minimum distance at which a 20/12 visual acuity (50 cpd) should be delivered, a pixel density of 573 ppi would then be required, above which the human vision would not gain any benefit from the display resolution per see. However a higher resolution would be still desirable for autostereoscopic displays that use, for example, lenticular lens to provide glass-free 3D.

7. OLED microdisplays used in video glasses can achieve much higher pixel densities (several 1000s ppi, see for example MicroOLED's advances).