![]() ![]() We provide a systematic framework to discover perceptually relevant image features from natural stimuli for perceptual inference tasks and therefore valuable for understanding both human and computer vision.Ĭitation: Liao C, Sawayama M, Xiao B (2023) Unsupervised learning reveals interpretable latent representations for translucency perception. Our results suggest that scale-specific representation of visual information might be crucial for humans to perceive materials. In contrast, the pixel-based embeddings from dimensionality reduction methods (e.g., t-SNE) do not correlate with perception. Particularly, we find the middle-layers of the latent space, which represent mid-to-low spatial scale features, can predict human perception. By manipulating specific layers of latent representation, we can independently modify certain visual attributes of the generated object, such as its shape, material, and color, without affecting the others. We train a deep image generation network to synthesize realistic translucent appearances from unlabeled data and learn a layer-wise latent representation that captures the statistical structure of images at multiple spatial scales. We present the first image-computable model that can predict human translucency judgments based on unsupervised learning from natural photographs of translucent objects. Perception of translucent materials (i.e., materials that transmit light) is challenging to study due to the high perceptual variability of their appearance across different scenes. Translucency is an essential visual phenomenon, facilitating our interactions with the environment. Together, our findings reveal that learning the scale-specific statistical structure of natural images might be crucial for humans to efficiently represent material properties across contexts. This layer-wise latent representation allows us to systematically discover perceptually relevant image features for human translucency perception. Moreover, we find the middle-layers of the latent space can successfully predict human translucency ratings, suggesting that translucent impressions are established in mid-to-low spatial scale features. The middle-layers of the latent space selectively encode translucency features and manipulation of such layers coherently modifies the translucency appearance, without changing the object’s shape or body color. Specifically, we find that manipulation on the early-layers (coarse spatial scale) transforms the object’s shape, while manipulation on the later-layers (fine spatial scale) modifies its body color. ![]() By embedding an image into the learned latent space, we can manipulate specific layers’ latent code to modify the appearance of the object in the image. Importantly, without supervision, human-understandable scene attributes, including the object’s shape, material, and body color, spontaneously emerge in the model’s layer-wise latent space in a scale-specific manner. We find our model, with its layer-wise latent representation, can synthesize images of diverse and realistic materials. Here, we develop an unsupervised style-based image generation model to identify perceptually relevant dimensions for translucent material appearances from photographs. Despite this, humans can still distinguish between different materials, and it remains unsolved how to systematically discover visual features pertinent to material inference from natural images. This problem especially stands out for translucent materials, whose appearance strongly depends on lighting, geometry, and viewpoint. ![]() Visual inference of materials is important but challenging because a given material can appear dramatically different in various scenes. Humans constantly assess the appearance of materials to plan actions, such as stepping on icy roads without slipping.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |