Skip to main content

Research Repository

Advanced Search

3D car shape reconstruction from a contour sketch using GAN and lazy learning

Nozawa, Naoki; Shum, Hubert P.H.; Feng, Qi; Ho, Edmond S.L.; Morishima, Shigeo

3D car shape reconstruction from a contour sketch using GAN and lazy learning Thumbnail


Authors

Naoki Nozawa

Qi Feng

Edmond S.L. Ho

Shigeo Morishima



Abstract

3D car models are heavily used in computer games, visual effects, and even automotive designs. As a result, producing such models with minimal labour costs is increasingly more important. To tackle the challenge, we propose a novel system to reconstruct a 3D car using a single sketch image. The system learns from a synthetic database of 3D car models and their corresponding 2D contour sketches and segmentation masks, allowing effective training with minimal data collection cost. The core of the system is a machine learning pipeline that combines the use of a generative adversarial network (GAN) and lazy learning. GAN, being a deep learning method, is capable of modelling complicated data distributions, enabling the effective modelling of a large variety of cars. Its major weakness is that as a global method, modelling the fine details in the local region is challenging. Lazy learning works well to preserve local features by generating a local subspace with relevant data samples. We demonstrate that the combined use of GAN and lazy learning produces is able to produce high-quality results, in which different types of cars with complicated local features can be generated effectively with a single sketch. Our method outperforms existing ones using other machine learning structures such as the variational autoencoder.

Citation

Nozawa, N., Shum, H. P., Feng, Q., Ho, E. S., & Morishima, S. (2022). 3D car shape reconstruction from a contour sketch using GAN and lazy learning. Visual Computer, 38(4), 1317-1330. https://doi.org/10.1007/s00371-020-02024-y

Journal Article Type Article
Acceptance Date Sep 5, 2020
Online Publication Date Apr 16, 2021
Publication Date 2022-04
Deposit Date Aug 11, 2021
Publicly Available Date Mar 28, 2024
Journal The Visual Computer
Print ISSN 0178-2789
Electronic ISSN 1432-2315
Publisher Springer
Peer Reviewed Peer Reviewed
Volume 38
Issue 4
Pages 1317-1330
DOI https://doi.org/10.1007/s00371-020-02024-y

Files

Published Journal Article (Advance online version) (3.6 Mb)
PDF

Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/

Copyright Statement
Advance online version This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.





You might also like



Downloadable Citations