9
$\begingroup$

Display screens connected to cameras are significantly different from windows or mirrors. Looking through a window or mirror, we can focus on infinity, and we can move our vantage point to see a little different. With a display screen showing live camera footage, we see the field of view from a single point (the camera location) and our focus is on the screen.

Would it be possible to develop a screen + camera that is much closer in behaviour to a window or a mirror? I think the screen and camera would need to have the same surface area. Both would be sensitive to direction, and when camera pixel $(C_i, C_j)$ on the camera receives a photon with frequency $\nu$ from angles $(C_\phi, C_\theta)$, the screen would send a corresponding photon at frequency $\nu$ from position $(S_i, S_j)$ to direction $(S_\phi, S_\theta)$, where $(S_\phi, S_\theta)$ are calculated from $(C_\phi, C_\theta)$ mimicking either window- or mirror-like behaviour.

Is such a device theoretically possible? If yes, would such a device be technically feasible today? If yes, has there been any serious work on such devices? If it's theoretically possible but not feasible today, what would need to be developed before such devices may be on the horizon?

It should have a wide range of applications in telepresence, augmented reality, automotive engineering, and surely a lot of other fields.

Improve this question
$\endgroup$
2
  • $\begingroup$ The mere fact that it would be incredibly useful, yet you've never heard of it, should be a hint as to feasibility :-) $\endgroup$
    – Carl Witthoft
    Mar 18, 2016 at 11:43
  • $\begingroup$ @CarlWitthoft True, although there are plenty of gadgets we have heard of that aren't feasible at all, such as time machines (-: $\endgroup$
    – gerrit
    Mar 18, 2016 at 11:51

4 Answers 4

Reset to default
5
$\begingroup$

The technology to do what you want has existed for decades, and it is called holography. The problem with common photographic sensors and displays is that they only record/reproduce amplitude information about the light. In order to know e.g. what angle the ray came from, you need to also record the phase information of the light. This is precisely what holography does.

In the image shown at the bottom you can see that two images of a single hologram taken from different angles show the mouse as if it is viewed from different angles. There are parts of the scene visible from one angle that aren't even visible from the other angle such as the back of the mouse and the branch behind the mouse.

The technologies needed to make real-time holograms (akin to a camera with a screen) are still in the R&D phases and are very rudimentary right now. Spatial light modulators provide a way of producing 2D holograms in real time. This group was able to record the hologram using a standard 4K camera with a lens array and used spatial light modulators to reproduce the hologram in real-time (not particularly well though).

enter image description here

Improve this answer
$\endgroup$
1
$\begingroup$

Such a screen may be possible with similar technology to metamaterials which is known for the potential application as an 'invisibility cloak'. Some companies also claim to have achieved this for the military, but its effectiveness is questionable because all the PR around it uses still images and mock-ups.

The trick would be to pick up the light from all directions and re-produce that same scatter on the other side (or from the screen). There are ways of making things 'invisible' to some wavelengths using refraction to bend EM waves around a central object, but it's unlikely this would work for an arbitrarily placed 'screen', unless you could capture the input with a bundle of fibre optics and somehow reproduce it exactly at the other end (without 'twisting' of the outbound scatter).

All that seems quite hazy and far too under-developed for the practical application you are seeking here. Probably the best you could achieve would be a 3D lenticular screen with head/eye tracking so that it could manipulate the image according to the screen/head relative position.

This would only work for one person at a time with current technology, as far as I know. The input would then need to be processed into a 3D scene so that it could be re-displayed from other angles. This technology is reasonably mature and there are many technologies ranging from pure camera-based visible-light capturing with software processing, to active 3D scanning cameras that combine multiple active and passive inputs. Alternatively a tightly packed 2D array of cameras could be used and two suitable ones selected to match the relative head-screen orientation. Their field-of-view would still need to be manipulated according to the head-screen distance, this would most likely be easier to do digitally by cropping and scaling the image from a wide-angle lens.

Improve this answer
$\endgroup$
1
$\begingroup$

Transmitting each individual photon is infeasible, given the amount of computation that would be required, but technology for capturing some information about incoming light direction already exists and is used in the Lytro "light field" camera.

The corresponding light field display does not exist, as far as I'm aware. The Lytro system uses a conventional display with postprocessing that lets you adjust focal point, depth of field, etc. after the photo has been taken.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Well, you could transmit a bunch of N photons for each bunch of N photons that come in... I didn't mean literally processing each photon. $\endgroup$
    – gerrit
    Mar 17, 2016 at 15:40
  • $\begingroup$ Yes, although you've got angular quantisation issues to worry about there: for each point on the reciever you can only have a certain number N "buckets" of directions you're looking in. The more buckets, the more noise. $\endgroup$
    – pjc50
    Mar 17, 2016 at 15:52
  • $\begingroup$ No one said it was going to be easy ;-) $\endgroup$
    – gerrit
    Mar 17, 2016 at 15:54
0
$\begingroup$

3D Cameras

3D cameras that consist of two cameras apart in order to allow depth perception has been around for a long time. The only floor is that displaying it to a pair of human eyes in a way the brain can understand can be difficult. Most efforts today focus on only showing one image to each eye, and allow the brain to focus on synchronizing the images to one coherent narrative.

The problem with that is either you need a display very close to the eyes, or a pair of polarized glasses.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Although interesting, this is different from what I'm asking. To emulate the effect of a mirror or window goes much further than stereo-viewing. $\endgroup$
    – gerrit
    Mar 17, 2016 at 14:12
  • $\begingroup$ The problem with that is either you need a display very close to the eyes, or a pair of polarized glasses. Or a parallax barrier. $\endgroup$
    – Dan Henderson
    Mar 17, 2016 at 17:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.