“eyeSelfie: self directed eye alignment using reciprocal eye box imaging”

  • ©Tristan Swedish, Karin Roesch, Ik Hyun Lee, Krishna Rastogi, Shoshana Bernstein, and Ramesh Raskar



Session Title:

    VR, Display, and Interaction


    eyeSelfie: self directed eye alignment using reciprocal eye box imaging




    Eye alignment to the optical system is very critical in many modern devices, such as for biometrics, gaze tracking, head mounted displays, and health. We show alignment in the context of the most difficult challenge: retinal imaging. Alignment in retinal imaging, even conducted by a physician, is very challenging due to precise alignment requirements and lack of direct user eye gaze control. Self-imaging of the retina is nearly impossible.We frame this problem as a user-interface (UI) challenge. We can create a better UI by controlling the eye box of a projected cue. Our key concept is to exploit the reciprocity, “If you see me, I see you”, to develop near eye alignment displays. Two technical aspects are critical: a) tightness of the eye box and (b) the eye box discovery comfort. We demonstrate that previous pupil forming display architectures are not adequate to address alignment in depth. We then analyze two ray-based designs to determine efficacious fixation patterns. These ray based displays and a sequence of user steps allow lateral (x, y) and depth (z) wise alignment to deal with image centering and focus. We show a highly portable prototype and demonstrate the effectiveness through a user study.


    1. Cui, J., Wang, Y., Huang, J., Tan, T., and Sun, Z. 2004. An iris image synthesis method based on pca and super-resolution. In Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on, vol. 4, IEEE, 471–474. Google ScholarDigital Library
    2. Deering, M. F. 2005. A photon accurate model of the human eye. ACM Trans. Graph. 24, 3, 649–658. Google ScholarDigital Library
    3. Dodgson, N. A. 2002. Analysis of the viewing zone of multiview autostereoscopic displays. In Electronic Imaging 2002, vol. 4660, International Society for Optics and Photonics, 254–265.Google Scholar
    4. Filar, P., et al., 2011. Apparatus for photographing the anterior segment and retina of the eye through the use of a camera attachment. US Patent 7,883,210.Google Scholar
    5. Giardini, M., Livingstone, I., Jordan, S., Bolster, N., Peto, T., Burton, M., and Bastawrous, A. 2014. A smartphone based ophthalmoscope. In Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE, 2177–2180.Google Scholar
    6. Goldfain, E., Lagerway, W., Roberts, C., Slawson, S., and Krauter, A., 2005. Eye viewing device for large field viewing. US Patent 6,939,006.Google Scholar
    7. Huang, F., Wetzstein, G., Barsky, B., and Raskar, R. 2014. Eyeglasses-free Display: Towards Correcting Visual Aberrations with Computational Light Field Displays. ACM Trans. Graph. 33, 4, 1–12. Google ScholarDigital Library
    8. 2010. iExaminer for Panoptic. http://www.iexam.com/.Google Scholar
    9. Ishihara, M., and Kogawa, T., 2008. Fundus camera. US Patent 7,377,642.Google Scholar
    10. Itoh, Y., and Klinker, G. 2015. Light-field correction for spatial calibration of optical see-through head-mounted displays. Visualization and Computer Graphics, IEEE Transactions on 21, 4, 471–480.Google Scholar
    11. Kadambi, A., Whyte, R., Bhandari, A., Streeter, L., Barsi, C., Dorrington, A., and Raskar, R. 2013. Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles. ACM Trans. Graph. 32, 6, 167. Google ScholarDigital Library
    12. Keeler, C. 1997. 150 years since babbage’s ophthalmoscope. Archives of ophthalmology 115, 11, 1456.Google Scholar
    13. Kress, B., and Starner, T. 2013. A review of head-mounted displays (hmd) technologies and applications for consumer electronics. vol. 8720, SPIE, 87200A–87200A–13.Google Scholar
    14. Lanman, D., and Luebke, D. 2013. Near-eye light field displays. ACM Trans. Graph. 32, 6. Google ScholarDigital Library
    15. Lawson, E. M., and Raskar, R. 2014. Smart phone administered fundus imaging without additional imaging optics. Investigative Ophtalmology and Visual Science 55, 5, 1609.Google Scholar
    16. Lee, I.-H., and Choi, T.-S. 2013. Accurate registration using adaptive block processing for multispectral images. Circuits and Systems for Video Technology, IEEE Transactions on 23, 9, 1491–1501. Google ScholarDigital Library
    17. Lee, I.-H., Shim, S.-O., and Choi, T.-S. 2013. Improving focus measurement via variable window shape on surface radiance distribution for 3d shape reconstruction. Optics and Lasers in Engineering 51, 5, 520–526.Google ScholarCross Ref
    18. Lee, I., Tariq Mahmood, M., and Choi, T.-S. 2013. Adaptive window selection for 3d shape recovery from image focus. Optics & Laser Technology 45, 21–31.Google ScholarCross Ref
    19. Lefohn, A., Budge, B., Shirley, P., Caruso, R., and Reinhard, E. 2003. An ocularist’s approach to human iris synthesis. Computer Graphics and Applications, IEEE 23, 6, 70–75. Google ScholarDigital Library
    20. Maimone, A., Lanman, D., Rathinavel, K., Keller, K., Luebke, D., and Fuchs, H. 2014. Pinlight displays: Wide field of view augmented reality eyeglasses using defocused point light sources. ACM Trans. Graph. 33, 4, 89:1–89:11. Google ScholarDigital Library
    21. Makthal, S., and Ross, A. 2005. Synthesis of iris images using markov random fields. In Proc. 13th European Signal Processing Conf, Citeseer.Google Scholar
    22. Pamplona, V., Oliveira, M., and Baranoski, G. 2009. Photorealistic models for pupil light reflex and iridal pattern deformation. ACM Trans. Graph. 28, 4, 106. Google ScholarDigital Library
    23. Pamplona, V., Mohan, A., Oliveira, M., and Raskar, R. 2010. Netra: interactive display for estimating refractive errors and focal range. ACM Trans. Graph. 29, 4, 77. Google ScholarDigital Library
    24. Pamplona, V., Passos, E., Zizka, J., Oliveira, M., Lawson, E., Clua, E., and Raskar, R. 2011. Catra: interactive measuring and modeling of cataracts. ACM Trans. Graph. 30, 4, 47. Google ScholarDigital Library
    25. Pamplona, V. F., Oliveira, M. M., Aliaga, D. G., and Raskar, R. 2012. Tailored displays to compensate for visual aberrations. ACM Trans. Graph. 31, 4 (July), 81:1–81:12. Google ScholarDigital Library
    26. Pertuz, S., Puig, D., and Garcia, M. A. 2013. Analysis of focus measure operators for shape-from-focus. Pattern Recognition 46, 5, 1415–1432. Google ScholarDigital Library
    27. 2014. Volk Pictor Plus Portable Retinal Camera. http://www.volk.com/.Google Scholar
    28. Plopski, A., Itoh, Y., Nitschke, C., Kiyokawa, K., Klinker, G., and Takemura, H. 2015. Corneal-imaging calibration for optical see-through head-mounted displays. Visualization and Computer Graphics, IEEE Transactions on 21, 4, 481–490.Google Scholar
    29. Ritschel, T., Ihrke, M., Frisvad, J., Coppens, J., Myszkowski, K., and Seidel, H. 2009. Temporal glare: Real-time dynamic simulation of the scattering in the human eye. In Computer Graphics Forum, vol. 28, 183–192.Google ScholarCross Ref
    30. Sagar, M., Bullivant, D., Mallinson, G., and Hunter, P. 1994. A virtual environment and model of the eye for surgical simulation. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques, ACM, 205–212. Google ScholarDigital Library
    31. Samaniego, A., Boominathan, V., Sabharwal, A., and Veeraraghavan, A. 2014. mobilevision: A face-mounted, voice-activated, non-mydriatic lucky ophthalmoscope. In Proceedings of the Wireless Health 2014 on National Institutes of Health, ACM, 1–8. Google ScholarDigital Library
    32. Tran, K., Mendel, T. A., Holbrook, K. L., and Yates, P. A. 2012. Construction of an inexpensive, hand-held fundus camera through modification of a consumer “point-and-shoot” camerapoint-and-shoot hand-held fundus camera. IOVS 53, 12, 7600.Google Scholar
    33. Unar, J. A., Seng, W. C., and Abbasi, A. 2014. A review of biometric technology along with trends and prospects. Pattern Recognition 47, 8 (8), 2673–2688.Google Scholar
    34. Upatnieks, J., and Tai, A. M. 1997. Development of the holographic sight. vol. 2968, SPIE, 272–281.Google Scholar
    35. Wecker, L., Samavati, F., and Gavrilova, M. 2005. Iris synthesis: a reverse subdivision application. In Proceedings of the 3rd international conference on Computer graphics and interactive techniques in Australasia and South East Asia, ACM, 121–125. Google ScholarDigital Library
    36. Woittennek, F., Knobbe, J., Pugner, T., Dallmann, H.-G., Schelinski, U., and Gruger, H. 2015. Mems scanner mirror based system for retina scanning and in eye projection. In Proceedings of the SPIE, vol. 9375, 937506.Google Scholar
    37. W. Y. Lam, M., and V. G. Baranoski, G. 2006. A predictive light transport model for the human iris. Computer Graphics Forum 25, 3, 359–368.Google ScholarCross Ref

ACM Digital Library Publication: