Category: The Front Line

Two Keys to Optimal HDR TVs: Dynamic HDR Metadata and Tone Mapping

The Society for Information Display LA Chapter held its 14th annual One-Day Conference on February 3, 2017 at the Costa Mesa Country Club in Costa Mesa, California. At the conference, Gerard Catapano gave a presentation entitled “HDR, Today into Tomorrow.”

Catapano, formerly Associate Director of Electronics Testing at Consumer Reports and now Director of Quality Assurance at Samsung’s QA Lab in Pine Brook, New Jersey, introduced high dynamic range (HDR) as “the latest and most innovative technology that helps film studios deliver a better expression of details in shadows and highlights to the consumer.” He presented the Consumer Technology Association’s definition of an HDR-compatible display as one that has at least these four attributes:

• Includes at least one interface that supports HDR signaling as defined in CEA-861-F, as extended by CEA-861.3.
• Receives and processes static HDR metadata compliant with CEA-861.3 for uncompressed video.
• Receives and processes the HDR10 Media Profile from IP, HDMI, or other video delivery sources. Other media profiles may be supported in addition.
• Applies an appropriate electro-optical transfer function (EOTF) before rendering the image.

The HDR standard has been endorsed by a variety of organizations, include the Blu-ray Disc Association, MPEG, the UHD Alliance, and the ITU. Although HDR is currently a premium feature, Catapano predicted that it will become a basic feature of TVs over all screen sizes and display technologies.

Samsung TV supports only the HDR10 media profile because it is an open standard that does not require licensing fees and, as a result, permits customization with

The CIE 1976 LAB Color Space clearly indicates the range of colors available for each level of luminance.

in the profile. Since use of at least HDR10 is required by the CTA definition of an HDR-compatible display, it will be supported by all major manufacturers.  Catapano noted that at NAB 2016, the major encoder manufacturers were offering 4K HDR as an option, and the major mastering and editing tool sets were implementing it.

The CTA definition only requires HDR sets to support static HDR metadata: metadata that is constant throughout the entire film or video. But much more can be done with dynamic HDR metadata, which changes scene by scene.  SMPTE ST.2094-40 provides for the use of dynamic metadata for tone mapping with HDR10.  In a subsequent conversation, Mindoo Chun, an engineer at the QA Lab, told me the dynamic metadata and tone-mapping technology codified in ST.2094-40 was developed by Samsung and a made available to SMPTE.

Tone mapping is a key technology in HDR TVs, Catapano said. It is a color-volume transform that renders incoming HDR content for a display having a dynamic range that is smaller than that for which the contents were coded. With static metadata, the only way to compress the scene with the greatest color volume so it fits into the set’s color volume is to over-compress the much larger number of less demanding scenes.

With dynamic metadata, each scene can be optimally compressed, with that result that many scenes will not require color-volume compression at all. Catapano observed that Samsung HDR TVs for the 2017 model year “are ready for ST.2094-40.”

An ICC color profile based on CIELAB. As with most real devices, this one can reproduce only a portion of the colors defined in the CIELAB color space.

Let’s say a little more about “color volume.”  The most common way of looking at color gamut is still with the 85-year-old CIE 1931 color diagram, which compresses the luminance (“brightness”) Z-axis so that the color space is pressed into a plane.  With the limited luminance capabilities displays have had until recently, that was a simple and (perhaps) adequate approach.  But with high dynamic range, you lose a lot of information that way.  Over the years many three-dimensional color spaces have been developed, with the CIELAB color space being the most common.  Now you can think of each value of luminance as having a two-dimensional color gamut associated with it, and the entire color volume is the stack of these two-dimensional gamuts running from black to white.  The gamut decreases at low and high luminance values, and one of the things you want to do in HDR  set is to have a relatively large gamut at high luminance levels so bright colors do not wash out.

Tone mapping maps the colors in the program material’s color volume to the smaller color volume of a less capable TV set while providing the best possible picture.  From Samsung’s point of view, it’s very convenient that the OLED TVs of arch-rival LG inherently have a smaller color volume than HDR LCD sets because they have a substantially smaller maximum luminance.  There is much more to say about that, but we’ll save it for another time.

Now, when you go into Costco or Fry’s to buy your next TV set, you can ask the sales associate whether the set supports SMPTE ST.2094-40. I look forward to hearing how that conversation goes.
Ken Werner is Principal of Nutmeg Consultants, specializing in the display industry, manufacturing, technology, and applications, including mobile devices and television. He consults for attorneys, investment analysts, and companies re-positioning themselves within the display industry or using displays in their products. He is the 2017 winner of the Society for Informational Display’s Lewis and Beatrice Winner Award.

You can reach him at kwerner@nutmegconsultants.com.

Turn Back The Clock?

A recent story in the Nikkei Asian Review states that Hon Hai Precision Industries – the new parent company of Sharp Electronics – is considering building an LCD panel facility in the United States. The finished panels would likely be intended for televisions.

According to the story, Hon Hai is responding to President Trump’s call to bring back jobs to the United States. When Hon Hai investment partner SoftBank Group’s chairman Masayoshi Son met with Trump last month, he said that both companies would make “significant investments” to create new jobs in the U.S.

In a related story on the CDRinfo.com Web site, Hon Hai chairman Terry Gou was quoted as saying that he’d consider going ahead with such an investment if “the U.S. is willing to offer land at a cost of US $1 for building the panel plants.” Apparently Apple is also part of the discussion and mentioned as a joint investor.

Gou was also quoted in the CDRinfo story as saying that “…U.S. President Donald Trump should love to see a vertically integrated industry such as panel manufacturing grow and develop in the country.”

Coincidentally, Hon Hai and Sharp are getting ready to break ground on a Gen 10 LCD fab in Guangzhou, China at a cost of $8.69B. That plant is scheduled to open in the fall of 2018. People familiar with the project said the proposed U.S. LCD fab would be of the same size and generation.

While this is an intriguing story, there are caveats. First off; LCD factories are mostly automated – they have to be, considering the manufacturing precision involved – so there wouldn’t be all that many permanent jobs created once construction is completed. (The same thing applies to Intel’s proposed semiconductor fab in Arizona.)

Second, most of the permanent jobs will likely require college degrees in the sciences (physics, engineering, and chemistry), aside from basic factory functions, shipping, and facilities maintenance.

But the biggest obstacle to building the plant will be the finished cost of the panels. There’s a reason why the LCD panel industry (and with it, television manufacturing) is migrating to China: Manufacturing costs there are much lower because labor rates are lower. That, in turn, will make Sharp-branded televisions much more expensive than those coming from Korea and China.

Consumers have been conditioned to expect ever-lower TV prices with ever-larger screens. Consider that you can already buy a 55-inch “smart” Ultra HDTV for $500 now: How will a US-made UHDTV compete against that price?

Consider also that in the 4K TV world, Samsung has over a 30% market share and LG has another 15%. Conversely, Sharp’s current TV market share is less than 1% and its brand doesn’t have anywhere near the cache it once had. So Hon Hai would have to find other customers for its panels to avoid underutilization of plant capacity.

Matters are further complicated by the fact that Hisense currently controls the marketing rights for the name “Sharp” in the United States and has no intention of giving them up. That little dust-up is why Hon Hai cut off supplies of VA LCD panels to Hisense last fall, forcing them to look elsewhere for a supplier.

Of course, there’s been plenty of talk in Washington about slapping 20% tariffs on foreign-made goods. That cost would be passed along to customers, and don’t you think Samsung and LG will adjust their prices as needed to maintain their dominant market shares? The net result would be that Sharp-branded LCD TVs would still languish on store shelves while Samsung, LG, Sony, Hisense, and TCL continued to dominate the market.

The recent election was filled with jingoistic slogans like “Bring Back America.” Well, then – which one? The America of the mid-1980s where the television manufacturing business involved lots of workers on assembly lines, hand-wiring CRT televisions and installing them into cabinets?

Sorry, that ain’t gonna happen. The US TV industry was pretty well decimated by 1986 when Zenith finally threw in the towel on TV production stateside. (Zenith was later acquired by LG Electronics.) The Japanese had our number. Then, the Koreans pulled the rug out from Japan, starting in the late 1990s. And now it’s the Chinese TV manufacturer’s turn to run with the ball.

The widespread availability of inexpensive LCD panels from China is a big reason why you can now afford to buy a 65-inch 4K TV for less than $800, or a 4K HDR model for about a grand. For that matter, you can now pick up a 50-inch Full HD (1080p) LCD TV for less than $300, and 42-inch sets have dropped below $200.

Question: Do you really want to pay 30 – 40% more for a given TV just because it’s made on this side of the Pacific Ocean? I didn’t think so. More expensive TVs will prompt people to delay their TV upgrades for a longer time period, which is exactly what Hon Hai doesn’t want to happen, and can’t afford to have happen if they’ve sunk a few billion dollars into an LCD fab.

Time marches on..

CES 2017: Afterthoughts and Second Thoughts

It’s been a few weeks since the annual extravaganza of consumer electronics in Las Vegas. As usual, it’s difficult to process everything one sees and produce a coherent show review within a few days. There are always products, trends, and demos that one winds up dwelling on for a few weeks. (Sometimes it’s better not to be the first to report on something!)

Overall, the show was busy and loaded with gadgets. Mind you; a good part of those gadgets were “shiny, sparkly” things, such as mobile phone cases with glitter and mirrors. Or must-have accessories, none of which really cost all that much. Numerous booths in the upper and lower South Hall were filled with exactly that, showcased by numerous Chinese/Korean/Taiwanese trading companies you’ve never heard of.

Add in a scattering of U.S. audio companies toward the front of the hall, plus the large areas reserved for AR/VR demos and the drone cages, and that pretty much sums up the South Hall experience. (A continuing puzzler is the presence of the United States Postal Service in the middle of all of these Asian manufacturers and wholesalers.)

In the Central Hall, the show continues to be dominated by the big CE brands – LG, Panasonic, Sony, Samsung, Intel, Qualcomm, Casio, Canon, Nikon, and relative newcomers TCL, Hisense, and Haier (who now owns the GE appliances business and made it a focal point of their booth). And the North Hall is basically divided between audio companies and automobile manufacturers, with the lines often blurring between them.

Much of the new tech appears in the Sands Expo Center, which due to the challenging logistics of travel, I don’t focus on much. There’s another crop of audio companies set up on the upper floors of the Venetian Hotel, and other venues host a variety of small, table top expos like Digital Experience and ShowStoppers.

So the first trend that jumped out at me is just how many of these Asian manufacturers and wholesalers have taken over the show. In the past, I’ve joked about large parts of the South Hall becoming the “Chinese Electronics Show,” but that’s a pretty good description of what you see there.

Shiny, sparkly stuff everywhere!

Another trend you couldn’t miss is just how important appliances have become to the product lines of companies like Panasonic, LG, and Samsung (not to mention Haier and Hisense). That shouldn’t come as a surprise – there’s much more profit in refrigerators, washers, dryers, and even things like the induction oven Panasonic showed this year when you compare appliances to the former kings of CES, televisions.

That’s not to say television isn’t important anymore. But when the amount of booth space devoted to TVs continues to shrink while the square footage given to appliances is growing, it doesn’t take long to connect the dots. In fact, more of the TV demos focused on the underlying technology than on specific lines or models. And right now (while this is being written), I can walk into Best Buy and pick up with a 55-inch LG Ultra HDTV with Web OS for all of $500 – or walk out with a 55-inch Hisense version with basic HDR support for the same price.  (Remember the good old days, when a 50-inch 1080p plasma TV cost $5,000?)

So it doesn’t make as much sense for manufacturers to invest a lot of time and money into promoting a category of products which has slim profit margins to begin with. But those ‘connected’ refrigerators? Dual-chamber washing machines? Cool kitchen gadgets? Now, there’s where a decent profit can be made, especially when you can sell a swath of these products in a bundle for consumers who are remodeling kitchens.

Never heard of Skyworth? Don’t worry, you will…

 

Appliances are where the action (and money) is these days.

One area that was disappointing was wireless connectivity – specifically, 60 GHz WiFi and wireless USB. Although I did mention some impressive demos from Peraso in my post-show coverage, I was surprised to see little space Qualcomm gave to 802.11ad products, particularly after the impressive demos shown last year. Despite the unique advantages of wireless operation in this band – limited, secure in-room connectivity with high bit rates over large channels – we’re still not seeing enough in the way of finished products.

Although other press accounts have talked about voice recognition being a big deal at the show (mostly with the autonomous car demonstrations), I didn’t see much that really wowed me. In past years, Conexant had excellent demos of voice recognition in noisy environments, but either they didn’t exhibit or didn’t reach out to me as they have in the past. The same observation applies to gesture recognition – there were some interesting products here and there that used a basic implementation, but nothing earth-shaking.

I mentioned augmented reality and virtual reality. From my view, the biggest problem with VR taking off in a big way is the size and weight of the headsets. Sure, we’ve all seen the Samsung Galaxy VR TV commercial where everyone is “thrilled” to get a VR (Oculus Rift) headset for Christmas, and they all “ooh!” and “ahhh!” at the VR experience.

Wearing VR headsets isn’t as comfortable as it looks…

What we don’t see is people taking these headsets off and putting them aside after the initial VR novelty wears off and sore necks start to manifest – not to mention possible problems with nausea due to a disconnect in the brain between perceived motion and actual motion. The latter is a real problem, similar to the issues with failed stereoscopic perception revealed by the roll-out of 3D seven years ago.

That’s not to say there isn’t a market for VR. There definitely is, but by my back-of-the-envelope calculations, we will need about 8K pixel resolution per eye to make it really work. (Some VR manufacturers and users are advocating for 11K per eye, refreshed as fast as 120 Hz to eliminate flicker.) With AR, on the other hand, things are much farther along, as Kopin demonstrated with its 2K x 2K near-to-eye OLED microdisplay fitted to a firefighter’s oxygen mask for search and rescue.

I may have said this before, but it’s worth repeating: LEDs are simply dominating the display sector. From the white LEDs with color filters used in conventional LCD TVs and the blue LEDs combined with quantum dots in HDR/WCG UHDTV models to organic white OLEDs with color filters in Ultra HDTVs, RGB OLEDs in smartphones and tablets, and the new super-small “micro” LEDs that make up the building blocks of super-bright, colorful videowalls with as much as 8K resolution…LEDs are basically taking over the world. (And I left out automotive displays and lights, appliances, indoor and outdoor lighting, and indicator lamps.)

How’s this for a cool keyboard design, which each key illuminated by a micro LED?

About the only area left to mention is the ever-growing Internet of Things trend. It was impossible to keep tabs on all of the IoT products at the show – remote pet food dispenser monitors, heart monitors, water quality monitors, connected TVs, massage chairs, doorbell cameras, connected appliances, home security systems, teenage driver monitors, control systems, and of course a slew of connected sensors in the most advanced car designs.

Memo to those readers in the commercial AV industry: If you haven’t figured out that room control systems for AV gear, lighting, shades, thermostats, audio, screens, and projectors are all entering the IoT world and leaving behind clunky, proprietary and expensive programming systems – well, that train is leaving the station, and you’d better not miss it.

As for interfacing all of this gear, we’re seeing a slow and steady move to the next-generation USB connector (version 3.0 Type-C) for new laptops and eventually, tablets and smartphones. Given that USB Type-C can also support display connections like HDMI and DisplayPort, that’s one or two less connectors to deal with. And given a move to AV-over-IT connectivity, we may be more concerned with USB-based switching and distribution equipment – or we’ll just encode all of our video and audio to JPEG2000, M-JPEG, H.264, or H.265 and use conventional fast network switches to do the job.

See you in Vegas next year?

HDMI 2.1: The Need For Speed Continues

Ever since HDMI version 2.0 was announced in September 2013, I’ve been pretty vocal about criticizing its “not quite fast enough” speed upgrade from 10.2 to 18 Gb/s, which turned out to be barely adequate for transporting 4K (3840×2160) video at full color resolution (RGB, or 4:4:4 in the world) at a frame rate of 60 Hz – and only with 8-bit color.

Given how quickly the display industry is shifting to 4K and even higher resolutions, it was inconceivable that this new interface would in effect create a “speed bump” in the 4K chain, particularly since high dynamic range (HDR) and wide color gamut (WCG) enhancements were becoming part of the UHD ecosystem. And both enhancements require at least 10-bit color rendering, something that would be impossible to pass through the HDMI 2.0 interface if using a full-resolution color format.

It didn’t help that HDMI’s competitor – DisplayPort – had already broken the 20 Gb/s barrier way back in 2007 with version 1.2 and could easily interface a 2160p/60 signal with 10-bit RGB color @ 60 Hz, and earlier in 2013 had announced version 1.3, which saw a speed boost to 32.4 Gb/s.

For a time there, I thought the superMHL format, which had its debut at CES 2015, might be the successor to HDMI. It was faster (36 Gb/s), had a large, reversible connector, was compatible with USB Type-C Alternate Mode, and most importantly, supported Display Stream Compression.

Alas; it appears superMHL turned out to be mostly a science experiment. The MHL Forum was conspicuous by its absence at CES 2017, but the HDMI Forum more than made up for it by unveiling version 2.1. And now, we’ve got a real horse race.

High dynamic range support will be much easier with version 2.1, especially deeper color from RGB sources.

THE DETAILS

The public press release on HDMI 2.1 is sketchy on details, except to say that the maximum speed of the interface has now reached a mind-boggling 48 Gb/s (that’s faster than most network switches!). Quite the leap from 18 Gb/s, wouldn’t you say?

The release goes on to talk about a new generation of 48G cables, a greatly improved eARC audio return channel with auto-detect, and finishes with a discussion of high dynamic range and higher video resolutions, both of which are possible with faster data rates that enable higher frame rats and deeper color. And of all of this happened while retaining the familiar 19-pin Type A connector. (Wha-a-a-t?)

But what’s really going on here? How did HDMI accelerate to 48 Gb/s? Hold on, and I’ll provide the details missing from the press release.

First off, the current version of HDMI uses three connections – well call them lanes, like DisplayPort does – to transport red, green, and blue display pixels. There’s a fourth lane for the clock to synchronize frames, and the balance of the connectors are used for ‘hot plug detect’ connections, the Data Display Channel (EDID). That doesn’t leave much room for expansion.

But HDMI 2.1 adds another lane for TMDS data (although it’s not really TMDS anymore) by taking over the clock lane and embedding clock data within the existing signal, much the same way it’s done with packet-based signaling systems.

Next, the physical data rate over each lane has been raised from 6 Gb/s to 12 Gb/s. I don’t know how that 100% increase was achieved, but that’s an impressive achievement considering that we are still waiting for 12G SDI cables to come to market.

The 12G number may also be a function of jiggering the acceptable signal-to-noise (SNR) ratio, something proposed a year ago by Steve Lampen of Belden – but then again, we’re not likely to see 12 Gb/s of data traveling down any display pipes in the immediate future. (For comparison, DisplayPort’s HBR3 cap is 8.1 Gb/s per lane.)

That’s not all. The standard ANSI coding format for HDMI, DVI, and DP (not to mention numerous other interfaces) is known as 8b/10b, coding 8-bit words into 10-bit symbols, resulting in about 20% overhead. Example: A 4K/60 signal encoded as an 8-bit RGB signal requires 17.28 Gb/s, and 20% of that is overhead from 8b/10b coding.

HDMI 2.1 has adopted a more obscure form of coding known as 16b/18b. You can find a IEEE PDF from 1999 describing how it works here, and it’s formally known as “partitioned DC-balanced 16b/18b transmission code.” The net effect of moving from 8b/10b to 16b/18b is reducing the overhead to about 12% from 20%. What’s interesting though is that the HDMI 2.1 signal isn’t really TMDS we’ve come to know and love when in this mode – it’s something else, possibly more of a packet structure.

HDMI is now compatible with USB Type-C Alternate Mode – a”must have” feature for any new display interface.

Last but not least, HDMI announced last fall that it was compatible with the USB Type-C Alternate Mode format. And now, it appears that HDMI 2.1 is also compatible with DisplayStream 1.2 compression, which is a much more efficient way to transport signals like 7680×4320/60 (8K, for those not paying attention). Although at 48 Gb/s, version 2.1 could theoretically transport that signal uncompressed using 4:2:0 color.)

Compatibility with DSC wouldn’t be that much of a shocker – superMHL also offered it and it’s another TMDS format. In fact, at second glance, it appears that much of the engineering that went into superMHL has now migrated over to HDMI 2.1 (about time) and the most significant breakthrough is doubling the interface speed.

Given that 40 Gb/s is definitely optical fiber territory, the only remaining question is why we still haven’t seen a detailed HDMI specification for direct optical interfaces. 48G cables will be expensive and difficult to engineer, but multimode optical fiber can already do the job and is cheap. To come up with 50-foot and longer manufactured optical cables for HDMI would be a piece of cake – and it’s already been done in the past for HDMI 1.3/1.4.

So there you have it: HDMI 2.1; a faster, smarter, and more appropriate display interface as we head into the era of 4K and beyond. How soon will we see HDMI 2.1 interfaces and cables? Well, considering it took almost 3 years for version 2.0 to achieve any significant presence in commercial AV, I’d say maybe a year from now at the earliest…and perhaps not until 2019 in any quantity.

By then, a good deal of the industry may have already shifted to AV-over-IP for the bulk of its signal switching and distribution, using simple format conversion at the display end. And we still have to see who is going to adopt on DisplayPort 1.3/1.4, still a “no-royalty” interface that can hit 32 Gb/s and supports all the forward-looking necessities (Type-C Alternate Mode, DSC, HDR).

Gentlemen, start your engines…