Category: The Front Line
EE LLC: A Million PCs and Nobody Knows Its Name
- Published on Friday, 27 February 2015 11:07
- Ken Werner
- 0 Comments
There is a company you’ve probably never heard of whose display technology sits inside a million notebook PCs. Entertainment Experience LLC has developed a total color management system whose mathematical and vision models are embedded in multidimensional look-up tables (LUTs). Current customers include Dell and Quanta, the world’s largest notebook PC ODM.
This week, I spoke separately with CEO John Parkinson and the technology’s inventor and developer Jim Sullivan.
When the image captured by a typical professional video camera is re-mapped to Rec.709, at least 70% of the color information is thrown away, Sullivan said. One example: grass green and laser green map to the same RGB values. All the energy that goes into maintaining color fidelity is applied only to that reduced gamut and does not address the loss of perceived fidelity that was inflicted early in the process. Part of what EE LLCs software product, eeColor, does is to compensate for that loss in perceived fidelity by intentionally breaking away from “hardware fidelity.” Parkinson noted that the software recomputes the values for each pixel in the frame individually in real time.
Most natural colors only occupy the central portion of the display’s color space. eeColor expands this portion to more completely fill the color space of the display. But memory colors, such as skin and sky, are displayed without modification. Doing this blindly would result in “memory colors,” such as skin and sky, becoming distorted. eeColor identifies the range of color coordinates that include these memory colors and preserves the color values of these pixels as the overal gamut is expanded. EE LLC calls these patches of preserved color coordinates “filters.”
eeColor incorporates models of skin tone developed at the Rochester Institute of Technology and uses them to maintain skin tone under brightness changes and color remapping. Different skin tones, from African to Scandinavian, are mostly a matter of brightness rather than color shift. But there are cultural preferences. Asians seem to like displays to show skin tones a bit more blue than do Americans and Europeans. This is addressed with slide bar in the UI.
Sullivan said that engineering the filters was the toughest part of the job. There are times when the color space is remapped that the skin-tone vector (for example) most move in the opposite director for the image vector for the overall scene. Strange things can happen at the boundaries of these filters with the overall display color space.
At EE LLC, they use “colorfulness” to describe the color content of a frame. This is the volume of colors contained in the 3D volume of the IPT color space. Percent of NTSC, which the hardware people have been trained to use, “is useless.”
They use the IPT color space because it preserves hue and brightness under gamut mapping. Moving the color portion (PT) of the vector does not change the perceived brightness (I). CIELAB doesn’t do this, nor does RGB, Sullivan said.
The vision model used by eeColor allows the viewer to observe rec.709 imagery in a relatively bright home or office environment and see the colors as they would appear in darkened cinema. The adaptive color boost is working with the behavior of the human visual system.
eeColor’s “management produces greater brightness with the same power, while keeping color quality,” Sullivan said. It is therefore very attractive for battery-operated devices.
The transformations performed by eeColor affect color only. For a complete solution, eeColor needed to handle sharpness and contrast, as well. For this they teamed up with Razzor Technologies, an RIT spin-off. Razzor’s approach also uses LUTs, so it was possible to combined the two sets of technologies into a single software product. Razzor’s sharpening technology avoids the white fringe that convolution filters impose on sharpened edges.
The company’s approach allows every input color to be mapped to an output color based on sound visual models, which has interesting applications. Among these is the ability to adjust for unit-to-unit variations in color rendition, and also to allow product manufacturers to compensate for the color differences between panels from different manufacturers.
The company has also worked with LG Display to minimize use of the blue phosphor and thus retard blue-phosphor aging.
These functions can be implemented in hardware, but Sullivan says EE LLC’s licensing fee per unit is less than the $3.00 unit cost (in volume) of a popular graphics co-processor. And with current personal devices having processing power to spare, EE LLC believes that software is the way to go. (The folks at Pixelworks may disagree.)
eeColor is currently available for the Microsoft and Android OS’s running on popular chips.
Has Sony Finally Seen The Light?
- Published on Wednesday, 25 February 2015 13:00
- Pete Putman
- 0 Comments
Sony’s ongoing financial woes have been well-documented by this writer over the past few years. Gone are the days when the Tokyo-based electronics giant could invent and own all parts of a media format, like the Walkman and Betacam.
It’s exceedingly difficult to make any money selling hardware to consumers these days, as fellow CE giants Panasonic, Toshiba, and Hitachi have all found out. And one of the biggest loss leaders is the Bravia television business, thanks to cutthroat competition from Samsung and LG, and now Chinese brands like Hisense and TCL.
Sony’s late entry into the LCD television marketplace a little over 10 years ago didn’t help. Back then, the company had OEM deals for LCD and plasma TVs with Pioneer and the aforementioned LG, along with a joint venture with Samsung to manufacture LCD televisions (S-LCD). But even with the Sony brand and decent market share, profits were nowhere to be found.
As losses piled up in the television unit, more red ink started flowing from Sony’s VAIO computer operations (since sold off to Lenovo). And in a real head-scratcher, Sony bought out its share of a mobile phone joint venture from Ericsson, only to see that miscalculation produce even more financial misery than the TV group ever did.
Now, chairman Kazuo Hirai has made it official: Sony will no longer chase higher sales in smartphones, where its Experia models just can’t compete with Samsung and Apple. And Sony won’t get any traction in the world’s largest mobile phone market, China, where home-grown brands like Huawei play a dominant role.
Significantly, Hirai also said that he would not rule out an exit strategy for both smartphones and televisions. (Sony’s TV operations were recently spun off as a separate operating unit so their losses can be clearly identified from the rest of the company.) Sony is on track to post a $1.5B loss for the current fiscal year that ends March 31, continuing a string of down years. Layoffs have continued company-wide and about 1200 more employees will be let go from the mobile division this year.
Despite the gloomy news, Sony’s ace in the hole is a burgeoning entertainment division. Sony Pictures, Sony Pictures Television, Sony Music, and PlayStation – taken together – are profitable operations. More than one institutional investor has called for Sony to exit the hardware business altogether and concentrate on content and software, which is where the money is nowadays.
But Sony has such a strong and rich legacy in consumer electronics that they can’t bring themselves to let go of the past, even after posting year upon year of record losses attributable to that same CE hardware. It’s gotten so bad that the company even announced last year that they would not pay a stock dividend for the first time in 50+ years. (Boy, did THAT news wake everyone up!)
In a recent Reuters story, Hirai stated that Sony would target a return on equity of more than 10% by 2018, aiming for an operating profit of $4.2 billion for fiscal 2017. That would be quite a turnaround, given Sony’s performance over the past five years. And it won’t be possible unless the company kisses the TV and phone businesses goodbye, once and for all.
Did Sony learn the lesson of Panasonic, who bit the bullet and shut down their plasma TV manufacturing business cold turkey in 2013, returning to profitability last year? (Panasonic is on track to make about a $2 billion profit for FY 2014.)
Panasonic also shut down other underperforming business units and shifted its focus to commercial products, and it would not surprise me to see them walk away from consumer TVs altogether in the next year or so as their market share is so small.
What about Sony’s Japanese competitors? Hitachi read the tea leaves several years ago and gave up on TVs altogether, while Toshiba is retrenching to the Japanese market. Sharp continues to struggle in the television business as its once-dominant 21% worldwide market share in TV shipments (2006) has dwindled to about 3% and a $250 million loss is staring them in the face for FY 2014.
It seems like everyone but Sony figured out the way back to profitability several years ago. Now, has Sony wised up? Have they finally seen the light?
Time will tell…
Pixelworks Improves Mobile Display Quality While Reducing System Cost
- Published on Thursday, 19 February 2015 15:29
- Ken Werner
- 0 Comments
We have just succeeded in wrapping our heads around the fact that quantum-dot enhancement can signficantly increase the color gamut and color saturation of LCDs with little — and eventually no — increase in system cost.
Now, with its “Iris” mobile display co-processor, Pixelworks is giving us another example of improved display performance with, in this case, reduced system cost.
In the Pixelworks suite at CES, Graham Loveridge (Senior Vice President of Strategic Marketing and Business Development), said Iris is the world’s first mobile display co-processor. Many of Iris’s functions have been performed by television video processing chips and cores for years. But incorporating those functions and others in a chip that takes up sufficiently little space and consumes sufficiently little power for a mobile device is new.
Pixelworks calls the display performance that results from Iris processing “True Clarity.”
One of the more obvious things Iris does is up-convert mobile-display video from 15 or 30 frames per second (fps) to 60 fps. In side-by-side demonstrations in the suite, this provided motion images with far less judder, much smoother scrolling, and motion that had much less blur. This shouldn’t be a surprise since we’ve seen the same evolution in large-screen television, and Iris uses motion estimation and motion compensation (MEMC) algorithms to do its work, which is also used for TV. Loveridge said that Iris is unique in that it does MEMC without producing a halo around moving images.
Pixelworks also claims enhanced colors and wider gamut through the use of a 3D look-up table, better contrast, better high-ambient visibility, and custom color tuning. The color tuning, Loveridge said, can be used to make sure that all displays in a production run look the same. But more than that, the OEM can buy displays from different manufacturers and tune them so they all look the same.
What is surprising is that all of this can be done with a power reduction of roughly 25%. Some of the saving comes from the Iris chip off-loading some functions from the GPU and CPU, and performing them more efficiently.
Because the Iris chip permits savings elsewhere in the display electronics, it can save $6 on panel cost, said Loveridge. The power savings permit a smaller battery, which can save another $2. Depending on order size, the Iris chip can had for less than $6. So, said Loveridge, “if a manufacturer is savvy he can improve system performance and simultaneously lower cost.”
TV-quality LCD cells are increasingly common in mobile devices. With the addition of TV-quality video processing, it will be even more appealing for viewers to do more of their entertainment viewing on mobile devices.
Ultra HD: Live From the 2015 HPA Tech Retreat
- Published on Thursday, 12 February 2015 15:12
- Pete Putman
- 0 Comments
Ultra HD: Live From the 2015 HPA Tech Retreat
As I write this, it is the morning of Day 2 at the annual Hollywood Post Alliance Tech Retreat. This annual conference brings together the top minds across a wide range of disciplines in the media production business. Cameras, lenses, codecs, displays, file formats and exchanges, content protection, archiving – they’re all here, as are representatives from the major studios, TV networks, software companies, colleges, universities, government agencies, and standards organizations.
Many of the sessions over the past few days have focused on next-generation television – specifically, capturing, editing, and finishing 4K images. Hand-in-hand with these additional pixels comes high dynamic range (HDR), which was prominently featured at CES in January. There’s also a new, wider color space (ITU BT.2020) to deal with, along with higher frame rates (how does 120 Hz grab you?).
I present a review of the Consumer Electronics Show every year at HPA (which now stands for the Hollywood Professional Alliance), and try my best to cram as much as I can in half an hour. Obviously, HDR was a big part of my presentation. And the overemphasis on HDR at CES provided a nice contrast to the presentations at HPA – at CES, it’s all about marketing hype, while at HPA, it’s all about engineering and making things work.
The average Joe may not understand much about “4K TV” or Ultra HD, but there is definitely more than meets the eye. At CES, an announcement was made about the new UHD Alliance, a partnership of TV manufacturers (Samsung, Sony, Panasonic), Hollywood studios (Disney, Warner Brothers, Fox), and other interested parties that include Netflix, Dolby, DirecTV, and Technicolor.
All well and good, but you need to understand the primary function of this Alliance is to promote the sale of Ultra HD televisions. And right now, television sales haven’t been as strong as they were five years ago. (The introduction of Ultra HD did boost sales a bit in 2014, which may have provided the impetus for the UHD Alliance.)
So here are a few of the problems with transitioning to Ultra HD. First off, not all of the pieces are in place for implementing add-ons like HDR, wider color gamuts, deeper color, and higher frame rates. It’s nice to talk about these features in conjunction with Ultra HD, but the mastering and delivery standards for HDR 4K movies and TV programs haven’t even been finalized yet.
Color is a particularly tricky issue, as LCD TVs with LED backlights render colors differently than LCD TVs equipped with quantum dot backlights. And OLED TVs require their own look-up tables as they are emissive displays, not transmissive. As far as frame rates go, consumer TVs generally can’t handle anything faster than 60 Hz and in fact prefer incoming signals to match up to one of four harmonically-related clock rates.
Next, there is a new version of copy protection coming to your television in the near future. It’s known as HDCP 2.2, and will ride along on an HDMI 2.0 connector. It is not backward-compatible with current versions, and you may be surprised to learn that early models of 4K TVs don’t support HDCP 2.2 yet. So there is a real compatibility problem lurking in the shadows if you are an early adopter.
You may be wondering where 4K Blu-ray content will come from. The first Ultra HD BD player was shown at CES, and you can expect those to show up late in the 4th quarter of this year. Suffice it to say that they will be running HDCP 2.2 on their HDMI outputs! Media players will also have to adopt version 2.2 if they are to access movies and other protected content.
Getting back to HDMI: Although version 2.0 was announced in September 2013, it’s pretty scarce on Ultra HDTVs. Most current-model sets I’ve seen have one or two HDMI 2.0 inputs, and as I just mentioned, many of those don’t support HDCP 2.2 yet. HDMI 2.0 is also speed challenged; with a maximum clock rate, it can’t support signals beyond 3840x2160p/60 with 8-bit RGB color.
Because of that, some UHDTV manufacturers are quietly adding DisplayPort 1.2 inputs to their products. Some of these interfaces are intended for connections to proprietary media players, but others are available for connections to set-top boxes, computers, and laptops. DP 1.2 can support 3840x2160p/60 with 10-bit RGB color as it has a much higher clock rate.
In summary, it’s all well and good that UHDTV is here, and initial sales are encouraging. But the plane isn’t finished yet, even though some of us want to fly it. The HPA presentations I’ve heard and seen the past two days clearly point out all of the back room details that have to be addressed before the media production, editing, mastering, and delivery ecosystem for UHDTV is ready to roll…
CES 2015: Upon Further Review…
- Published on Tuesday, 27 January 2015 15:44
- Pete Putman
- 0 Comments
There’s so much to take in at the International CES every year that you need a few weeks for it all to sink it. I posted my recap of the show earlier this month. Now, I want to expound on a couple of trends I saw in Las Vegas you need to watch. (Let’s move over to the referee’s TV to look at a reply.)
First and foremost is Ultra HDTV. No matter what you think of this next step in television viewing, it is coming, and nothing will stop it. The vast number of Ultra HD models shown at CES by Japanese, Korean, and Chinese manufacturers is all the proof you need. So is the aggressive discounting we’re now seeing on Ultra HDTVs, leading up to the Super Bowl.
Doubting Thomases like to drag out 3D TV as an example of a paradigm that never shifted anything. Not the same thing! 3D was beset by high buy-in costs, competing viewing systems, and a lack of compelling content. (Plus the fact that over 20% of the population can’t even see 3D correctly.) Eventually, Joe Six-Pack judged 3DTV to be an expensive, overpriced, and overhyped gimmick. And he was right.
Ultra HD is different. There are no competing standards for viewing 4K content. You can watch 4K on an LG or Vizio 65-inch TV just as easily as on a Sony or Samsung 65-inch TV. And so many manufacturers are in the 4K game that prices are falling like a stone. You can now buy 48-inch and 50-inch Ultra HD sets for less than $1,000, and 55-inch sets aren’t much more expensive.
Hand-in-hand with 4K came some new wrinkles. High dynamic range (HDR) was a topic much bandied about at CES, and now we are seeing multiple TV brands supporting it, usually by incorporating quantum dot (QD) backlight technology or modifying the pixel structure of LCD panels to add more white pixels.
QDs also bring with them expanded color gamut rendering, pushing way beyond the CRT-based ITU BT.709 color space in use today. (OLED TVs, like LG’s new lineup, can also display billions of colors.) Now, we can approximate what’s shown in movie theaters by covering the minimum DCI P3 color space – and more.
High frame rate (HFR) technology is also an integral part of UHDTV. It can refer to rates as low as 48 Hz and as high as 120 Hz. The higher rates would come into play with televised sports and concerts, not to mention virtual reality and gaming. At the lower end of things, 48 Hz could be used to master movies, a la “The Hobbit.”
Finally, the cost of making LCD panels has dropped so low for a myriad of reasons that the incremental difference between fabricating and cutting 2K (1080p) and 4K versions of a 65-inch panel is insignificant. Given the low profit margins – or zero profit margins – in making large 2K glass, it makes more sense to abandon 2K and focus on 4K entirely. And this is exactly what large Chinese panel manufacturers like TCL decided to do over a year ago.
When you see Ultra HD content displayed with full color sampling at high frame rates, you know this is a totally different experience than HDTV. The latter is limited by the BT.709 color space and a handful of frame rates, plus 4:2:0 color encoding. And many HDTVs use 8-bit LCD panels.
Not only that; a display equipped with just 1280 horizontal and 720 vertical pixels is still considered “HD.” Not so with Ultra HD. If it doesn’t have at least 3840 horizontal and 2160 vertical pixels, it ain’t Ultra HD. And that’s why we really should think of Ultra HD as “next-generation television,” and not just “4K.”
Make no mistake about it; Ultra HD will be in wide use very quickly as people begin to understand the benefits it brings to the table. The rapid decreases in retail prices brought along by slower TV sales and competition from the Chinese will only hasten this process.
Now, the other trend: At CES, the UHD Alliance was announced. This is a consortium of TV manufacturers (Panasonic, Sony, Samsung), content producers (Warner Brothers, Fox, Disney, Netflix, and DirecTV), and technology companies (Dolby and Technicolor).
Each has substantial skin in the 4K game: The first three obviously want to sell more televisions, while the studios are looking for more outlets for digital content. And Dolby wants more companies to adopt its high dynamic range Dolby Vision technology, while Technicolor is a player in mastering and distribution.
All well and good, except that not all of the pieces of the 4K “puzzle” are in place yet. SMPTE is still debating, discussing, and moving to adopt a wide range of standards for UHD-1 (up to 3840×2160) and UHD-2 (4096×2160 and above) televisions that touch on data rates, interfaces, frame rates, and color spaces. Similar work is also happening at the ITU.
Without standards in place, anyone can write their own rules for authoring and distributing 4K movies and TV shows, and incompatibility becomes a problem. We don’t even have a 4K Blu-ray player yet – the standards for that format were just adopted and announced at CES, but we won’t see the players and discs until much later this year.
Right now, the most logical path for 4K content distribution is through digital downloads and streaming, although you’d need sustained 15 Mb/s data rates on your Internet connection to make that happen. But Netflix is streaming several shows in 4K. Comcast is getting ready to launch a 4K channel. Other providers such as M-Go and UltraFlix are also providing movies and TV shows via streaming service to Sony, Samsung, and Vizio Ultra HDTVs, plus Nanotech’s Nuvola media player.
Ultra HD also brings along a new version of copy protection, HDCP 2.0. It’s ever more rigorous than 1.4, because it wants to see a secure key exchange in about 20 milliseconds – or it shuts down. (The pro AV dealers and installers are going to LOVE that!) By extension, HDCP 2.0 is incompatible with older versions of HDCP. So that may create a problem for consumers who buy a new 4K TV.
I’ve often said that CES wouldn’t be complete with a raft of announcements about associations, alliances, and consortiums. They’re all well and good, but many are motivated simply to kick up sales of a particular technology or product. The UHD Alliance would be wise to move slowly as standards bodies complete their work so that Ultra HD will be a success. No reason to rush here!
Okay, you can restart the game clock…