“Mixed-primary factorization for dual-frame computational displays” by Huang, Pająk, Kim, Kautz and Luebke
Conference:
Type:
Title:
- Mixed-primary factorization for dual-frame computational displays
Session/Category Title: Computational Cameras & Displays
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Increasing resolution and dynamic range of digital color displays is challenging with designs confined by cost and power specifications. This necessitates modern displays to trade-off spatial and temporal resolution for color reproduction capability. In this work we explore the idea of joint hardware and algorithm design to balance such trade-offs. We introduce a system that uses content-adaptive and compressive factorizations to reproduce colors. Each target frame is factorized into two products of high-resolution monochromatic and low-resolution color images, which then get integrated through temporal or spatial multiplexing. As our framework minimizes the error in colorimetric space, the perceived color rendition is high, and thanks to GPU acceleration, the results are generated in real-time. We evaluate our system with a LCD prototype that uses LED backlight array and temporal multiplexing to reproduce color images. Our approach enables high effective resolution and dynamic range without increasing power consumption. We also demonstrate low-cost extensions to hyperspectral and light-field imaging, which are possible due to compressive nature of our system.
References:
1. Michael Abrash. 2013. Why virtual isn’t real to your brain: judder. http://blogs.valvesoftware.com/abrash/why-virtual-isnt-real-to-your-brain-judder/. (2013).Google Scholar
2. Takeyuki Ajito, Takashi Obi, Masahiro Yamaguchi, and Nagaaki Ohyama. 2000. Expanded color gamut reproduced by six-primary projection display. Proc. SPIE 3954, 130–137. Google ScholarCross Ref
3. C Baumann. 1992. Ewald Hering’s Opponent Colors. History of an idea. Der Ophthalmologe : Zeitschrift der Deutschen Ophthalmologischen Gesellschaft 89, 3 (1992), 249–252.Google Scholar
4. Bryce E. Bayer. 1976. Color imaging array. U.S. Patent 3971065 A. (1976).Google Scholar
5. Johan Bergquist and Carl Wennstam. 2006. Field-Sequential-Colour Display with Adaptive Gamut. SID Symposium Digest of Technical Papers 37, 1 (2006), 1594–1597.Google Scholar
6. Michael W. Berry, Murray Browne, Amy N. Langville, V. Paul Pauca, and Robert J. Plemmons. 2006. Algorithms and Applications for Approximate Nonnegative Matrix Factorization. In Computational Statistics and Data Analysis. 155–173.Google Scholar
7. Floraine Berthouzoz and Raanan Fattal. 2012. Resolution Enhancement by Vibrating Displays. ACM Trans. Graph. 31, 2, Article 15 (2012). Google ScholarDigital Library
8. V. Blondel, N-D Ho, and P van Dooren. 2008. Weighted Nonnegative Matrix Factorization and Face Feature Extraction. In Image and Vision Computing. 1–17.Google Scholar
9. Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. 2011. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Found. Trends Mach. Learn. 3, 1 (2011), 1–122. Google ScholarDigital Library
10. Bruce Bridgeman, Derek Hendry, and Lawrence Stark. 1975. Failure to detect displacement of the visual world during saccadic eye movements. Vision Research 15, 6 (1975), 719 — 722. Google ScholarCross Ref
11. Haidong Chen, Ji Wang, Weifeng Chen, Huamin Qu, and Wei Chen. 2014. An image-space energy-saving visualization scheme for OLED displays. Computers & Graphics 38 (2014), 61 — 68. Google ScholarDigital Library
12. Weifeng Chen, Wei Chen, Haidong Chen, Zhengfang Zhang, and Huamin Qu. 2016. An Energy-saving Color Scheme for Direct Volume Rendering. Comput. Graph. 54, C (Feb. 2016), 57–64. Google ScholarDigital Library
13. Wei-Chung Cheng,Yu Hou, and Massoud Pedram. 2004. Power minimization in a backlit TFT-LCD display by concurrent brightness and contrast scaling. In Proceedings Design, Automation and Test in Europe Conference and Exhibition, Vol. 1. 252–257 Vol.1.Google ScholarCross Ref
14. Yu-Kuo Cheng, Yi-Pai Huang, Yi-Ru Cheng, and Han-Ping D. Shieh. 2009. Two-Field Scheme: Spatiotemporal Modulation for Field Sequential Color LCDs. Journal of Display Technology 5, 10 (2009).Google Scholar
15. Johnson Chuang, Daniel Weiskopf, and Torsten Möller. 2009. Energy Aware Color Sets. Computer Graphics Forum 28, 2 (2009), 203–211. Google ScholarCross Ref
16. Piotr Didyk, Elmar Eisemann, Tobias Ritschel, Karol Myszkowski, and Hans-Peter Seidel. 2010. Apparent Display Resolution Enhancement for Moving Images (SIGGRAPH 2010). ACM, Article 113, 8 pages.Google Scholar
17. Mark Fairchild. 2013. Color appearance models. John Wiley & Sons. Google ScholarCross Ref
18. Mark D. Fairchild and Ping-Hsu Chen. 2011. Brightness, lightness, and specifying color in high-dynamic-range scenes and images. Proc. SPIE 7867. Google ScholarCross Ref
19. Mark D. Fairchild and Garrett M. Johnson. 2004. The iCAM framework for image appearance, image differences, and image quality. Journal of Electronic Imaging 13 (2004), 126–138. Google ScholarCross Ref
20. Raanan Fattal. 2014. Dehazing using Color-Lines. ACM Transaction on Graphics 34, 1 (2014), 13:1–13:14.Google ScholarDigital Library
21. James A. Ferwerda, Peter Shirley, Sumanta N. Pattanaik, and Donald P. Greenberg. 1997. A Model of Visual Masking for Computer Graphics. In SIGGRAPH. 143–152. Google ScholarDigital Library
22. N. Fisekovic, T. Nauta, H. Cornelissen, and J. Bruinink. 2001. Improved motion-picture quality of AM-LCDs using scanning backlight. In Asia Display/IDW. 1637–1640.Google Scholar
23. Stephen R. Forrest. 2003. The road to high efficiency organic light emitting devices. Organic Electronics 4, 2–3 (2003), 45 — 48.Google ScholarCross Ref
24. A. Frankenstein and Werner von Jaworski. 1904. Verfahren und Vorrichtung zur Fernsichtbarmachung von Bildern und Gegenständen mittels Selenzellen, Dreifarbenfilter, und Zerlegung des Bildes in Punktgruppen durch Spiegelung. German Patent 172376. (1904).Google Scholar
25. Felix Heide, Douglas Lanman, Dikpal Reddy, Jan Kautz, Kari Pulli, and David Luebke. 2014. Cascaded Displays: Spatiotemporal Superresolution Using Offset Pixel Layers. ACM Trans. Graph. (SIGGRAPH) 33, 4 (2014), 60:1–60:11.Google ScholarDigital Library
26. K.I. Iourcha, K.S. Nayak, and Z. Hong. 1999. System and method for fixed-rate block-based image compression with inferred pixel values. US Patent 5,956,431. (1999).Google Scholar
27. Garrett M. Johnson, Xioyan Song, Ethan D. Montag, and Mark D. Fairchild. 2010. Derivation of a Color Space for Image Color Difference Measurement. Color Research and Application 35, 6 (2010). Google ScholarCross Ref
28. Kälil Käläntär, Tadashi Kishimoto, Kazuo Sekiya, Tetsuya Miyashita, and Tatsuo Uchida. 2006. Spatio-temporal scanning backlight mode for field-sequential-color optically-compensated-bend liquid-crystal display. Journal of SID 14, 2 (2006).Google Scholar
29. I. Kauvar, S. Yang, L. Shi, I. McDowall, and G. Wetzstein. 2015. Adaptive Color Display via Perceptually-driven Factored Spectral Projection. ACM Trans. Graph. (SIGGRAPH Asia) 34, 6 (2015). Google ScholarDigital Library
30. Gorham Kindem. 1981. The Demise of Kinemacolor: Technological, Legal, Economic, and Aesthetic Problems in Early Color Cinema History. Cinema Journal 20, 2 (1981), 3–14. Google ScholarCross Ref
31. Michiel A. Klompenhouwer. 2006. Comparison of LCD Motion Blur Reduction Methods using Temporal Impulse Response and MPRT. SID Symposium Digest of Technical Papers 37, 1 (2006).Google ScholarCross Ref
32. Edwin H. Land. 1959. Experiments in Color Vision. Scientific American 200, 5 (1959), 84–94. Google ScholarCross Ref
33. Erno H.A. Langendijk. 2007. A novel spectrum-sequential display design with a wide color gamut and reduced color breakup. Journal of the Society for Information Display 15, 4 (2007), 261–266. Google ScholarCross Ref
34. Y. H. Liu, Z. Z. Yang, and S. C. Wang. 2010. A novel sequential-color RGB-LED backlight driving system with local dimming control and dynamic bus voltage regulation. IEEE Transactions on Consumer Electronics 56, 4 (2010). Google ScholarDigital Library
35. Dhruv Mahajan, Ira Kemelmacher Shlizerman, Ravi Ramamoorthi, and Peter Belhumeur. 2007. A Theory of Locally Low Dimensional Light Transport. ACM Transaction on Graphics (SIGGRAPH) 26, 3 (2007), 62:1–62:10.Google Scholar
36. Rafal Mantiuk, Kil Joong Kim, Allan G. Rempel, and Wolfgang Heidrich. 2011. HDR-VDP-2: A Calibrated Visual Metric for Visibility and Quality Predictions in All Luminance Conditions. ACM Trans. Graph. 30, 4 (2011). Google ScholarDigital Library
37. Susana Martinez-Conde, Jorge Otero-Millan, and Stephen L Macknik. 2013. The impact of microsaccades on vision: towards a unified theory of saccadic function. Nature reviews. Neuroscience (2013). Google ScholarCross Ref
38. Belen Masia, Gordon Wetzstein, Piotr Didyk, and Diego Gutierrez. 2013. A survey on computational displays: Pushing the boundaries of optics, computation, and perception. Computers & Graphics 37, 8 (2013), 1012 — 1038. Google ScholarDigital Library
39. David Melcher. 2011. Visual stability. Philosophical Transactions of the Royal Society of London B: Biological Sciences 366, 1564 (2011), 468–475. Google ScholarCross Ref
40. Ankit Mohan, Ramesh Raskar, and Jack Tumblin. 2008. Agile Spectrum Imaging: Programmable Wavelength Modulation for Cameras and Projectors. Computer Graphics Forum 27, 2 (2008), 709–717. Google ScholarCross Ref
41. Mineo Mori, Toyohiko Hatada, Kazuo Ishikawa, Tosio Saishouji, Osamu Wada, Junichi Nakamura, and Nobuyoshi Terashima. 1999. Mechanism of color breakup in field-sequential-color projectors. Journal of the Society for Information Display 7, 4 (1999), 257–259. Google ScholarCross Ref
42. V. G. Moshnyaga and E. Morikawa. 2005. LCD display energy reduction by user monitoring. In 2005 International Conference on Computer Design. 94–97. Google ScholarDigital Library
43. K T Mullen. 1985. The contrast sensitivity of human colour vision to red-green and blue-yellow chromatic gratings. The Journal of Physiology 359, 1 (1985), 381–400. Google ScholarCross Ref
44. Prathyusha Narra and D. S. Zinger. 2004. An effective LED dimming approach. In Conference Record of the 2004 IEEE Industry Applications Conference, 2004. 39th IAS Annual Meeting., Vol. 3.Google Scholar
45. I. Omer and M. Werman. 2004. Color lines: image specific color representation. IEEE CVPR (2004), 946–953.Google Scholar
46. Mang Ou-Yang and Shih-Wei Huang. 2007. Design considerations between color gamut and brightness for multi-primary color displays. Journal of Display Technology 3, 1 (2007), 71–82. Google ScholarCross Ref
47. Hao Pan, Xiao-Fan Feng, and S. Daly. 2005. LCD motion blur modeling and analysis. In IEEE ICIP.Google Scholar
48. A. A. Polumordvinov. 1899. Russian Patent 10738. (1899).Google Scholar
49. Tania Pouli, Douglas W Cunningham, and Erik Reinhard. 2010. Image statistics and their applications in computer graphics. In European Computer Graphics Conference and Exhibition.Google Scholar
50. Joseph P. Rice, Steven W. Brown, David W. Allen, Howard W. Yoon, Maritoni Litorja, and Jeeseong C. Hwang. 2012. Hyperspectral image projector applications. Proc. SPIE 8254, 82540R–82540R-8. Google ScholarCross Ref
51. Joseph P. Rice, Steven W. Brown, Jorge E. Neira, and Robert R. Bousquet. 2007. A hyperspectral image projector for hyperspectral imagers. Proc. SPIE 6565, 65650C–65650C-12. Google ScholarCross Ref
52. Temkar N. Ruckmongathan. 2014. Addressing Techniques of Liquid Crystal Displays. Wiley. Google ScholarCross Ref
53. Alfred C. Schroeder. 1948. Color television tube. U.S. Patent 2446791A. (1948).Google Scholar
54. Helge Seetzen, Wolfgang Heidrich, Wolfgang Stuerzlinger, Greg Ward, Lorne White-head, Matthew Trentacoste, Abhijeet Ghosh, and Andrejs Vorozcovs. 2004. High Dynamic Range Display Systems. ACM Trans. Graph. (SIGGRAPH) 23, 3 (2004), 760–768. Google ScholarDigital Library
55. G. Sharma, W. Wu, and E. N. Dalal. 2005. The CIEDE2000 color-difference formula: implementation notes, supplementary test data, and mathematical observations. Color research and application 30, 1 (2005), 21–30. Google ScholarCross Ref
56. Louis D. Silverstein. 2005. STColor: Hybrid Spatial-Temporal Color Synthesis for Enhanced Display Image Quality. In SID Symposium Digest of Technical Papers, Vol. 36. 1112–1115.Google Scholar
57. Ernst Simonson and Josef Brozek. 1952. Flicker fusion frequency. Physiological Review 32, 3 (1952), 349–378.Google ScholarCross Ref
58. Wen-Chih Tai, Chi-Chung Tsai, Shian-Jun Chiou, Chih-Ping Su, Huang-Min Chen, Chia-Lin Liu, and Chi-Neng Mo. 2008. Field Sequential Color LCD-TV Using Multi-Area Control Algorithm. SID Symposium Digest of Technical Papers 39, 1 (2008), 1092–1095.Google Scholar
59. Masatsugu Teragawa, Akiko Yoshida, Kazuyoshi Yoshiyama, Shinji Nakagawa, Kazunari Tomizawa, and Yasuhiro Yoshida. 2012. Review Paper: Multi-primary-color displays: The latest technologies and their benefits. Journal of the Soc. for Info. Disp. 20, 1 (2012).Google Scholar
60. Takatoshi Tsujimura. 2012. OLED Display Fundamentals and Applications. Wiley. Google ScholarCross Ref
61. Rui Wang, Bowen Yu, Julio Marco, Tianlei Hu, Diego Gutierrez, and Hujun Bao. 2016. Real-time Rendering on a Power Budget. ACM Trans. Graph. 35, 4, Article 111 (July 2016), 11 pages.Google ScholarDigital Library
62. Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600–612. Google ScholarDigital Library
63. G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar. 2012. Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting. ACM Trans. Graph. (SIGGRAPH) 31, 4 (2012), 1–11. Google ScholarDigital Library
64. Lin Zhang, D. Zhang, Xuanqin Mou, and D. Zhang. 2011. FSIM: A Feature Similarity Index for Image Quality Assessment. IEEE Transactions on Image Processing 20, 8 (2011), 2378–2386. Google ScholarDigital Library
65. X. Zhang and B. A. Wandell. 1997. A spatial extension of CIELAB for digital color-image reproduction. Journal of the Society for Information Display 5, 1 (1997), 61–63. Google ScholarCross Ref
