eduzhai > Applied Sciences > Engineering >

Mask2CAD 3D Shape Prediction by Learning to Segment and Retrieve

  • king
  • (0) Download
  • 20210506
  • Save

... pages left unread,continue reading

Document pages: 22 pages

Abstract: Object recognition has seen significant progress in the image domain, withfocus primarily on 2D perception. We propose to leverage existing large-scaledatasets of 3D models to understand the underlying 3D structure of objects seenin an image by constructing a CAD-based representation of the objects and theirposes. We present Mask2CAD, which jointly detects objects in real-world imagesand for each detected object, optimizes for the most similar CAD model and itspose. We construct a joint embedding space between the detected regions of animage corresponding to an object and 3D CAD models, enabling retrieval of CADmodels for an input RGB image. This produces a clean, lightweightrepresentation of the objects in an image; this CAD-based representationensures a valid, efficient shape representation for applications such ascontent creation or interactive scenarios, and makes a step towardsunderstanding the transformation of real-world imagery to a synthetic domain.Experiments on real-world images from Pix3D demonstrate the advantage of ourapproach in comparison to state of the art. To facilitate future research, weadditionally propose a new image-to-3D baseline on ScanNet which featureslarger shape diversity, real-world occlusions, and challenging image views.

Please select stars to rate!


0 comments Sign in to leave a comment.

    Data loading, please wait...