eduzhai > Applied Sciences > Engineering >

Auto-Encoding for Shared Cross Domain Feature Representation and Image-to-Image Translation

  • Save

... pages left unread,continue reading

Document pages: 8 pages

Abstract: Image-to-image translation is a subset of computer vision and patternrecognition problems where our goal is to learn a mapping between input imagesof domain $ mathbf{X} 1$ and output images of domain $ mathbf{X} 2$. Currentmethods use neural networks with an encoder-decoder structure to learn amapping $G: mathbf{X} 1 to mathbf{X} 2$ such that the distribution of imagesfrom $ mathbf{X} 2$ and $G( mathbf{X} 1)$ are identical, where $G( mathbf{X} 1)= d G (f G ( mathbf{X} 1))$ and $f G ( cdot)$ is referred as the encoder and$d G( cdot)$ is referred to as the decoder. Currently, such methods which alsocompute an inverse mapping $F: mathbf{X} 2 to mathbf{X} 1$ use a separateencoder-decoder pair $d F (f F ( mathbf{X} 2))$ or at least a separate decoder$d F ( cdot)$ to do so. Here we introduce a method to perform cross domainimage-to-image translation across multiple domains using a singleencoder-decoder architecture. We use an auto-encoder network which given aninput image $ mathbf{X} 1$, first computes a latent domain encoding $Z d = f d( mathbf{X} 1)$ and a latent content encoding $Z c = f c ( mathbf{X} 1)$, wherethe domain encoding $Z d$ and content encoding $Z c$ are independent. And thena decoder network $g(Z d,Z c)$ creates a reconstruction of the original image$ mathbf{ widehat{X}} 1=g(Z d,Z c ) approx mathbf{X} 1$. Ideally, the domainencoding $Z d$ contains no information regarding the content of the image andthe content encoding $Z c$ contains no information regarding the domain of theimage. We use this property of the encodings to find the mapping across domains$G: X to Y$ by simply changing the domain encoding $Z d$ of the decoder sinput. $G( mathbf{X} 1 )=d(f d ( mathbf{x} 2^i ),f c ( mathbf{X} 1))$ where$ mathbf{x} 2^i$ is the $i^{th}$ observation of $ mathbf{X} 2$.

Please select stars to rate!


0 comments Sign in to leave a comment.

    Data loading, please wait...