“3Doodle: Compact Abstraction of Objects With 3D Strokes” – ACM SIGGRAPH HISTORY ARCHIVES

“3Doodle: Compact Abstraction of Objects With 3D Strokes”

  • ©

Conference:


Type(s):


Title:

    3Doodle: Compact Abstraction of Objects With 3D Strokes

Presenter(s)/Author(s):



Abstract:


    We propose 3Doodle, a compact and efficient representation to convey characteristics of an object. Our approach generates 3D strokes from multi-view images. We express 3D sketch with contour of superquadrics (view-dependent component) and 3D cubic Bezier curves (view-independent component). 3Doodle can robustly obtain 3D sketch of various class of objects.

References:


    [1]
    Stephan Alaniz, Massimiliano Mancini, and Zeynep Akata. 2023. Iterative Superquadric Recomposition of 3D Objects from Multiple Views. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 18013–18023.

    [2]
    Hmrishav Bandyopadhyay, Subhadeep Koley, Ayan Das, Aneeshan Sain, Pinaki Nath Chowdhury, Tao Xiang, Ayan Kumar Bhunia, and Yi-Zhe Song. 2023. Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D Shapes. arXiv:2312.04043 [cs.CV]

    [3]
    Alan H Barr. 1981. Superquadrics and angle-preserving transformations. IEEE Computer graphics and Applications 1, 1 (1981), 11–23.

    [4]
    Jonathan T Barron. 2019. A general and adaptive robust loss function. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4331–4339.

    [5]
    Pierre B?nard, Aaron Hertzmann, et al. 2019. Line drawings from 3D models: A tutorial. Foundations and Trends? in Computer Graphics and Vision 11, 1–2 (2019), 1–159.

    [6]
    Itamar Berger, Ariel Shamir, Moshe Mahler, Elizabeth Carter, and Jessica Hodgins. 2013. Style and abstraction in portrait sketching. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1–12.

    [7]
    John Canny. 1986. A computational approach to edge detection. IEEE Transactions on pattern analysis and machine intelligence 6 (1986), 679–698.

    [8]
    Doug DeCarlo, Adam Finkelstein, Szymon Rusinkiewicz, and Anthony Santella. 2003. Suggestive contours for conveying shape. ACM Transactions on Graphics (TOG) 22, 3 (2003), 848–855.

    [9]
    Kangle Deng, Andrew Liu, Jun-Yan Zhu, and Deva Ramanan. 2022. Depth-supervised nerf: Fewer views and faster training for free. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12882–12891.

    [10]
    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16×16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).

    [11]
    Mathias Eitz, James Hays, and Marc Alexa. 2012. How Do Humans Sketch Objects? ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4 (2012), 44:1–44:10.

    [12]
    Ricardo Fabbri and Benjamin Kimia. 2010. 3D curve sketch: Flexible curve-based stereo reconstruction and calibration. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 1538–1545.

    [13]
    Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. 2022. Plenoxels: Radiance fields without neural networks. In Conference on Computer Vision and Pattern Recognition. 5501–5510.

    [14]
    Guangcong, Zhaoxi Chen, Chen Change Loy, and Ziwei Liu. 2023. SparseNeRF: Distilling Depth Ranking for Few-shot Novel View Synthesis. IEEE/CVF International Conference on Computer Vision (ICCV) (2023).

    [15]
    Benoit Guillard, Edoardo Remelli, Pierre Yvernay, and Pascal Fua. 2021. Sketch2mesh: Reconstructing and editing 3d shapes from sketches. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 13023–13032.

    [16]
    David Ha and Douglas Eck. 2017. A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477 (2017).

    [17]
    Xun Huang and Serge Belongie. 2017. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE international conference on computer vision. 1501–1510.

    [18]
    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-To-Image Translation With Conditional Adversarial Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

    [19]
    Tilke Judd, Fr?do Durand, and Edward Adelson. 2007. Apparent ridges for line drawing. ACM transactions on graphics (TOG) 26, 3 (2007), 19–es.

    [20]
    Moritz Kampelmuhler and Axel Pinz. 2020. Synthesizing human-like sketches from natural images using a conditional convolutional decoder. In Proceedings of the IEEE/CVF winter conference on applications of computer vision. 3203–3211.

    [21]
    Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

    [22]
    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012).

    [23]
    Changjian Li, Hao Pan, Adrien Bousseau, and Niloy J. Mitra. 2020b. Sketch2CAD: Sequential CAD Modeling by Sketching in Context. ACM Trans. Graph. (Proceedings of SIGGRAPH Asia 2020) 39, 6 (2020), 164:1–164:14.

    [24]
    Changjian Li, Hao Pan, Adrien Bousseau, and Niloy J Mitra. 2022. Free2CAD: Parsing freehand drawings into CAD commands. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1–16.

    [25]
    Tzu-Mao Li, Michal Luk??, Gharbi Micha?l, and Jonathan Ragan-Kelley. 2020a. Differentiable Vector Graphics Rasterization for Editing and Learning. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 39, 6 (2020), 193:1–193:15.

    [26]
    Difan Liu, Matthew Fisher, Aaron Hertzmann, and Evangelos Kalogerakis. 2021a. Neural strokes: Stylized line drawing of 3d shapes. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 14204–14213.

    [27]
    Difan Liu, Mohamed Nabail, Aaron Hertzmann, and Evangelos Kalogerakis. 2020. Neural contours: Learning to draw lines from 3d shapes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5428–5436.

    [28]
    Shaohui Liu, Yifan Yu, R?mi Pautrat, Marc Pollefeys, and Viktor Larsson. 2023. 3D Line Mapping Revisited. In Computer Vision and Pattern Recognition (CVPR).

    [29]
    Xueting Liu, Wenliang Wu, Huisi Wu, and Zhenkun Wen. 2021b. Deep style transfer for line drawings. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 353–361.

    [30]
    Ling Luo, Yulia Gryaditskaya, Yongxin Yang, Tao Xiang, and Yi-Zhe Song. 2021. Finegrained vr sketching: Dataset and insights. In 2021 International Conference on 3D Vision (3DV). IEEE, 1003–1013.

    [31]
    Nelson Max. 1995. Optical models for direct volume rendering. IEEE Transactions on Visualization and Computer Graphics 1, 2 (1995), 99–108.

    [32]
    Aryan Mikaeili, Or Perel, Mehdi Safaee, Daniel Cohen-Or, and Ali Mahdavi-Amiri. 2023. Sked: Sketch-guided text-based 3d editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 14607–14619.

    [33]
    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. 2021. Nerf: Representing scenes as neural radiance fields for view synthesis. Commun. ACM 65, 1 (2021), 99–106.

    [34]
    Yutaka Ohtake, Alexander Belyaev, and Hans-Peter Seidel. 2004. Ridge-valley lines on meshes via implicit surface fitting. In ACM SIGGRAPH 2004 Papers. 609–612.

    [35]
    Maxime Oquab, Timoth?e Darcet, Th?o Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. 2023. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023).

    [36]
    Despoina Paschalidou, Ali Osman Ulusoy, and Andreas Geiger. 2019. Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).

    [37]
    Despoina Paschalidou, Luc van Gool, and Andreas Geiger. 2020. Learning Unsupervised Hierarhical Part Decomposition of 3D Objects from a Single RGB Image. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).

    [38]
    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019).

    [39]
    Alex Pentland. 1986. Parts: Structured Descriptions of Shape. In AAAI. 695–701.

    [40]
    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 8748–8763.

    [41]
    Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. 2021. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 10901–10911.

    [42]
    Jifei Song, Kaiyue Pang, Yi-Zhe Song, Tao Xiang, and Timothy M Hospedales. 2018. Learning to sketch with shortcut cycle consistency. In Proceedings of the IEEE conference on computer vision and pattern recognition. 801–810.

    [43]
    Zhuo Su, Wenzhe Liu, Zitong Yu, Dewen Hu, Qing Liao, Qi Tian, Matti Pietik?inen, and Li Liu. 2021. Pixel difference networks for efficient edge detection. In Proceedings of the IEEE/CVF international conference on computer vision. 5117–5127.

    [44]
    Yael Vinker, Ehsan Pajouheshgar, Jessica Y Bo, Roman Christian Bachmann, Amit Haim Bermano, Daniel Cohen-Or, Amir Zamir, and Ariel Shamir. 2022. Clipasso: Semantically-aware object sketching. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1–11.

    [45]
    Markus Worchel and Marc Alexa. 2023. Differentiable Rendering of Parametric Geometry. ACM Trans. Graph. 42, 6, Article 232 (dec 2023), 18 pages.

    [46]
    Tong Wu, Jiaqi Wang, Xingang Pan, Xudong Xu, Christian Theobalt, Ziwei Liu, and Dahua Lin. 2023. Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstruction. In International Conference on Learning Representations (ICLR).

    [47]
    Yunfan Ye, Renjiao Yi, Zhirui Gao, Chenyang Zhu, Zhiping Cai, and Kai Xu. 2023. NEF: Neural Edge Fields for 3D Parametric Curve Reconstruction From Multi-View Images. In Conference on Computer Vision and Pattern Recognition (CVPR). 8486–8495.

    [48]
    Ran Yi, Yong-Jin Liu, Yu-Kun Lai, and Paul L Rosin. 2019. Apdrawinggan: Generating artistic portrait drawings from face photos with hierarchical gans. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10743–10752.

    [49]
    Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang. 2019. Free-form image inpainting with gated convolution. In Proceedings of the IEEE/CVF international conference on computer vision. 4471–4480.

    [50]
    Qian Yu, Yi-Zhe Song, Tao Xiang, and Timothy M Hospedales. 2017. SketchX!-Shoe/Chair fine-grained SBIR dataset. Shoe/Chair fine-grained SBIR dataset (2017).

    [51]
    Yu Zeng, Zhe Lin, and Vishal M Patel. 2022. Sketchedit: Mask-free local image manipulation with partial sketches. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5951–5961.

    [52]
    Kai Zhang, Nick Kolkin, Sai Bi, Fujun Luan, Zexiang Xu, Eli Shechtman, and Noah Snavely. 2022a. Arf: Artistic radiance fields. In European Conference on Computer Vision. Springer, 717–733.

    [53]
    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition. 586–595.

    [54]
    Song-Hai Zhang, Yuan-Chen Guo, and Qing-Wen Gu. 2021. Sketch2model: View-aware 3d modeling from single free-hand sketches. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6012–6021.

    [55]
    Yuechen Zhang, Zexin He, Jinbo Xing, Xufeng Yao, and Jiaya Jia. 2023. Ref-NPR: Reference-Based Non-Photorealistic Radiance Fields for Controllable Scene Stylization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4242–4251.

    [56]
    Yuanqing Zhang, Jiaming Sun, Xingyi He, Huan Fu, Rongfei Jia, and Xiaowei Zhou. 2022b. Modeling Indirect Illumination for Inverse Rendering. In CVPR.

    [57]
    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision. 2223–2232.


ACM Digital Library Publication:



Overview Page:



Submit a story:

If you would like to submit a story about this presentation, please contact us: historyarchives@siggraph.org