“Cartographic Relief Shading with Neural Networks” by Jenny, Heitzler, Singh, Farmakis-Serebryakova, Liu, et al. …
Conference:
Type(s):
Interest Area:
- Research / Education and Scientific Visualization
Title:
- Cartographic Relief Shading with Neural Networks
Session/Category Title: TVCG Session on Data Visualization
Presenter(s)/Author(s):
Abstract:
Shaded relief is an effective method for visualising terrain on topographic maps, especially when the direction of illumination is adapted locally to emphasise individual terrain features. However, digital shading algorithms are unable to fully match the expressiveness of hand-crafted masterpieces, which are created through a laborious process by highly specialised cartographers. We replicate hand-drawn relief shading using U-Net neural networks. The deep neural networks are trained with manual shaded relief images of the Swiss topographic map series and terrain models of the same area. The networks generate shaded relief that closely resemble hand-drawn shaded relief art. The networks learn essential design principles from manual relief shading such as removing unnecessary terrain details, locally adjusting the illumination direction to accentuate individual terrain features, and varying brightness to emphasise larger landforms. Neural network shadings are generated from digital elevation models in a few seconds, and a study with 18 relief shading experts found that they are of high quality.
References:
1.
E. Imhof, Cartographic relief presentation, Berlin:De Gruyter, 1982.
2.
P. Collier, D. Forrest and A. Pearson, “The representation of topographic information on maps: The depiction of relief”, Cartographic Journal, vol. 40, no. 1, pp. 17-26, 2003.
3.
T. Patterson, “Producing manual small-scale shaded relief”, 10th ICA Mountain Cartography Workshop, pp. 1-11, 2016.
4.
E. R. Tufte, Envisioning information, Cheshire, CT:Graphics Press, 1990.
5.
P. Isola, J. Zhu, T. Zhou and A. A. Efros, “Image-to-image translation with conditional adversarial networks”, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967-5976, 2017.
6.
L. A. Gatys, A. S. Ecker and M. Bethge, “Image style transfer using convolutional neural networks”, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2414-2423, Decem 2016.
7.
S. Iizuka, E. Simo-Serra and H. Ishikawa, “Globally and locally consistent image completion”, ACM Transactions on Graphics, vol. 36, no. 4, Jul. 2017.
8.
S. Wan, Y. Xia, L. Qi, Y.-H. Yang and M. Atiquzzaman, “Automated colorization of a grayscale image with seed points propagation”, IEEE Transactions on Multimedia, 2020.
9.
R. Zhang, P. Isola and A. A. Efros, “Colorful image colorization”, Computer Vision ECCV, vol. 2016, pp. 649-666.
10.
O. Ronneberger, P. Fischer and T. Brox, “U-Net: convolutional networks for biomedical image segmentation”, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015: 18th International Conference Munich Germany October 5?92015 proceedings, vol. 9351, pp. 234-241, 2015.
11.
J. Long, E. Shelhamer and T. Darrell, “Fully convolutional networks for semantic segmentation”, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431-3440, 2015.
12.
A. W. Pearson, “Relief depiction: relief shading” in History of Cartography Cartography in the Twentieth Century, University of Chicago Press, vol. 6, pp. 1267-1272, 2015.
13.
M. Cavelti Hammer, H.-U. Feldmann and M. Oehrli, Farbe Licht und Schatten: Die Entwicklung der Reliefkartographie seit 1660, Murten:Cartographica Helvetica, 1997.
14.
A. Lobben and J. E. Meacham, “Education and cartography: cartographic textbooks” in History of Cartography Cartography in the Twentieth Century, University of Chicago Press, vol. 6, pp. 343-347, 2015.
15.
B. Jenny and L. Hurni, “Swiss-style colour relief shading modulated by elevation and by exposure to illumination”, The Cartographic Journal, vol. 43, no. 3, pp. 198-207, 2006.
16.
J. Y. Sun and P. Perona, “Where is the sun?”, Nature Neuroscience, vol. 1, pp. 183-184, 1998.
17.
I. C. McManus, J. Buckman and E. Woolley, “Is light in pictures presumed to come from the left side?”, Perception, vol. 33, no. 12, pp. 1421-1436, 2004.
18.
P. Gerardin, M. de Montalembert and P. Mamassian, “Shape from shading: New perspectives from the Polo Mint stimulus”, Journal of Vision, vol. 7, no. 11, pp. 1-11, 2007.
19.
J. Biland and A. Coltekin, “An empirical assessment of the impact of the light direction on the relief inversion effect in shaded relief maps: NNW is better than NW”, Cartography and Geographic Information Science, vol. 44, no. 4, pp. 358-372, 2017.
20.
A. Coltekin and J. Biland, “Comparing the terrain reversal effect in satellite images and in shaded relief maps: an examination of the effects of color and texture on 3D shape perception from shading”, International Journal of Digital Earth, vol. 12, no. 4, pp. 442-459, 2019.
21.
B. Jenny and T. Patterson, “Aerial perspective for shaded relief”, Cartography and Geographic Information Science, 2020.
22.
I. Viola and T. Isenberg, “Pondering the concept of abstraction in (illustrative) visualization”, IEEE Transactions on Visualization and Computer Graphics, vol. 24, no. 9, pp. 2573-2588, 2018.
23.
T. Patterson and B. Jenny, Shaded Relief Archive, 2010, [online] Available: http://shadedreliefarchive.com.
24.
A. M. Leonowicz, B. Jenny and L. Hurni, “Automated reduction of visual complexity in small-scale relief shading”, Cartographica: The International Journal for Geographic Information and Geovisualization, vol. 45, no. 1, pp. 64-74, 2010.
25.
S. Bell, “Drawing color hillshade by hand”, North American Cartographic Society NACIS 2019, 2019, [online] Available: https://youtu.be/A9rxd-FDx9E.
26.
T. T?th, “Accidental cARTographer”, Cartographic Perspectives, no. 67, pp. 19-28, 2011.
27.
P. Yoeli, “Analytical hill shading (a cartographic experiment)”, Surveying and Mapping, vol. 25, no. 4, pp. 573-579, 1965.
28.
P. YoeIi, “The mechanisation of analytical hill shading”, The Cartographic Journal, vol. 4, no. 2, pp. 82-88, 1966.
29.
B. K. P. Horn, “Hill shading and the reflectance map”, Proceedings of the Institute of Electrical and Electronics Engineers IEEE, vol. 69, no. 1, pp. 14-47, 1981.
30.
R. M. Batson, K. Edwards and E. M. Eliason, “Computer-generated shaded-relief maps”, Journal Research U.S. Geol. Survey, vol. 3, no. 4, pp. 401-408, 1975.
31.
K. Brassel, “A model for automatic hill-shading”, The American Cartographer, vol. 1, pp. 15-27, 1974.
32.
B. E. Marston and B. Jenny, “Improving the representation of major landforms in analytical relief shading”, International Journal of Geographical Information Science, vol. 29, no. 7, pp. 1144-1165, 2015.
33.
B. Jenny, “An interactive approach to analytical relief shading”, Cartographica: The International Journal for Geographic Information and Geovisualization, vol. 38, no. 12, pp. 67-75, 2001.
34.
R. Mark, Multidirectional oblique-weighted shaded-relief image of the Island of Hawaii, US Dept. of the Interior, US Geological Survey, 1992.
35.
I. V Florinsky and S. V Filippov, “Three-dimensional terrain modeling with multiple-source illumination”, Transactions in GIS, vol. 23, no. 5, pp. 937-959, 2019.
36.
P. J. Kennelly and A. J. Stewart, “General sky models for illuminating terrains”, International Journal of Geographical Information Science, vol. 28, no. 2, pp. 383-406, 2014.
37.
P. J. Kennelly and A. J. Stewart, “A uniform sky illumination model to enhance shading of terrain and urban areas”, Cartography and Geographic Information Science, vol. 33, no. 1, pp. 21-36, 2006.
38.
F. Veronesi and L. Hurni, “Changing the light azimuth in shaded relief representation by clustering aspect”, The Cartographic Journal, vol. 51, no. 4, pp. 291-300, 2014.
39.
P. J. Kennelly, “Geomorphology terrain maps displaying hill-shading with curvature”, Geomorphology, vol. 102, no. 3?4, pp. 567-577, 2008.
40.
F. Veronesi and L. Hurni, “A GIS tool to increase the visual quality of relief shading by automatically changing the light direction”, Computers Geosciences, vol. 74, pp. 121-127, 2015.
41.
M. Farmakis-Serebryakova and L. Hurni, “Comparison of relief shading techniques applied to landforms”, International Journal of Geo-Information, vol. 9, no. 4, pp. 1-22, 2020.
42.
R. Weibel, “Models and experiments for adaptive computer-assisted terrain generalization”, Cartography and Geographic Information Systems, vol. 19, no. 3, pp. 133-153, 1992.
43.
R. Geisthovel and L. Hurni, “Automated Swiss-style relief shading and rock hachuring”, The Cartographic Journal, vol. 55, no. 4, pp. 341-361, 2018.
44.
A. M. Leonowicz, B. Jenny and L. Hurni, “Terrain Sculptor: generalizing terrain models for relief shading”, Cartographic Perspectives, no. 67, pp. 51-60, 2011.
45.
E. Guilbert, J. Gaffuri and B. Jenny, “Terrain generalization” in Abstracting Geographic Information in a Data Rich World: Methodologies and Applications of Map Generalisation, Springer, pp. 227-258, 2014.
46.
K. Lawonn, I. Viola, B. Preim and T. Isenberg, “A survey of surface-based illustrative rendering for visualization”, Computer Graphics Forum, vol. 37, no. 6, pp. 205-234, 2018.
47.
S. Rusinkiewicz, M. Burns and D. Decarlo, “Exaggerated shading for depicting shape and detail”, ACM SIGGRAPH 2006 Papers SIGGRAPH 06, pp. 1199-1205, 2006.
48.
R. Vergne, R. Pacanowski, P. Barla, X. Granier and C. Schlick, “Light warping for enhanced surface depiction”, ACM Transactions on Graphics, vol. 28, no. 3, 2009.
49.
C. H. Lee, X. Hao and A. Varshney, “Geometry-dependent lighting”, IEEE Transactions on Visualization and Computer Graphics, vol. 12, no. 2, pp. 197-207, 2006.
50.
P. P. J. Sloan, W. Martin, A. Gooch and B. Gooch, “The lit sphere: A model for capturing NPR shading from art”, Proceedings – Graphics Interface, pp. 143-150, 2001.
51.
A. Tewari et al., “State of the art on neural rendering”, Computer Graphics Forum, vol. 39, no. 2, pp. 701-727, 2020.
52.
S. Bi, K. Sunkavalli, F. Perazzi, E. Shechtman, V. Kim and R. Ramamoorthi, “Deep CG2Real: Synthetic-to-real translation via image disentanglement”, Proceedings of the IEEE International Conference on Computer Vision, pp. 2730-2739, 2019.
53.
D. Zhang, C. Xian, G. Luo, Y. Xiong and C. Han, “DeepAO: Efficient screen space ambient occlusion generation via deep network”, IEEE Access, vol. 8, pp. 64434-64441, 2020.
54.
O. Nalbach, E. Arabadzhiyska, D. Mehta, H. P. Seidel and T. Ritschel, “Deep shading: convolutional neural networks for screen space shading”, Computer Graphics Forum, vol. 36, no. 4, pp. 65-78, 2017.
55.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition”, 3rd International Conference on Learning Representations ICLR 2015 – Conference Track Proceedings, pp. 1-14, 2015.
56.
Y. Jing, Y. Yang, Z. Feng, J. Ye, Y. Yu and M. Song, “Neural style transfer: A review”, IEEE Transactions on Visualization and Computer Graphics, 2019.
57.
A. Semmo, T. Isenberg and J. Dollner, “Neural style transfer: A paradigm shift for image-based artistic rendering?”, Proceedings – Non-Photorealistic Animation and Rendering NPAR 2017 – Part of Expressive 2017, 2017.
58.
T. Patterson, “Designing the Equal Earth physical map”, North American Cartographic Society NACIS 2019, 2019, [online] Available: https://youtu.be/UYQ6vhxc9Dw.
59.
X.-J. Mao, C. Shen and Y.-B. Yang, “Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections”, 30th Conference on Neural Information Processing Systems (NIPS 2016), vol. 29, 2016.
60.
K. He, X. Zhang, S. Ren and J. Sun, “Deep residual learning for image recognition”, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
61.
C. Szegedy et al., “Going deeper with convolutions”, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1-9, 2015.
62.
Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity”, IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, 2004.
63.
J. Johnson, A. Alahi and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution”, Computer Vision ECCV 2016, 2016.
64.
M. Heitzler and L. Hurni, “Cartographic reconstruction of building footprints from historical maps: A study on the Swiss Siegfried map”, Transactions in GIS, vol. 24, no. 2, pp. 442-461, Apr. 2020.
65.
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting”, Journal of Machine Learning Research, vol. 15, no. 56, pp. 1929-1958, 2014.
66.
K. He, X. Zhang, S. Ren and J. Sun, “Delving deep into rectifiers: surpassing human-level performance on ImageNet classification”, 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1026-1034, 2015.
67.
V. Nair and G. E. Hinton, “Rectified linear units improve restricted Boltzmann machines”, Proceedings of the 27th International Conference on International Conference on Machine Learning, pp. 807-814, 2010.
68.
B. Jenny and S. Raber, Relief cartographers at swisstopo, 2017, [online] Available: http://www.reliefshading.com/cartographers/swisstopo-cartographers/.
69.
Swiss Map Raster, [online] Available: https://shop.swisstopo.admin.ch/en/products/maps/digital_maps/digital.
70.
D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, 2014, [online] Available: .
71.
P. J. Kennelly et al., “Elevation models for reproducible evaluation of terrain representation”, Cartography and Geographic Information Science, 2020.
72.
Core ML: Integrate machine learning models into your app, 2020, [online] Available: https://developer.apple.com/documentation/corem/.
73.
T. Park, M. Y. Liu, T. C. Wang and J. Y. Zhu, “Semantic image synthesis with spatially-adaptive normalization”, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR, pp. 2332-2341, 2019.
74.
W. R. Tan, C. S. Chan, H. E. Aguirre and K. Tanaka, “ArtGAN: Artwork synthesis with conditional categorical GANs”, International Conference on Image Processing ICIP, pp. 3760-3764, 2017.
75.
C. Tietjen, T. Isenberg and B. Preim, “Combining silhouettes surface and volume rendering for surgery education and planning”, Seventh Joint Eurographics/IEEE VGTC conference on Visualization (EUROVIS 05), pp. 303-310, 2005.
76.
F. Ritter, C. Hansen, V. Dicken, O. Konrad, B. Preim and H. O. Peitgen, “Real-time illustration of vascular structures”, IEEE Transactions on Visualization and Computer Graphics, vol. 12, no. 5, pp. 877-884, 2006.
77.
A. Elgammal, B. Liu, M. Elhoseiny and M. Mazzone, CAN: Creative adversarial networks generating ?art? by learning about styles and deviating from style norms, 2017, [online] Available: http://arxiv.org/abs/1706.07068.
78.
P. J. Kennelly and A. J. Stewart, SkyLum, 2013, [online] Available: http://watkins.cs.queensu.ca/~jstewart/skyModels.zip.
79.
“Schweizer Weltatlas -Atlas Mondial Suisse -Atlante Mondiale Svizzero”, EDK Schweizerische Konferenz der kantonalen Erziehungsdirektoren, 2017.
80.
J. P. Theroux and E. A. Wingert, A Coastal zone management atlas of American Samoa, University of Hawaii Cartographic Laboratory, Department of Geography, 1981.
81.
A. Newell, K. Yang and J. Deng, “Stacked hourglass networks for human pose estimation”, Computer Vision – ECCV 2016, pp. 483-499, 2016.
82.
J. Wang et al., “Deep high-resolution representation learning for visual recognition”, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1, 2020.
