Many of us have played with apps and enjoy games where the world around us comes alive with new ideas and fantasy scenarios. We can use filters on screens to amuse friends as we facetime, learn about constellations in the night sky, and play a whole new version of “the floor is made of lava”. But, have you ever wondered how augmented reality works?
Augmented reality is a tool that has grown in popularity and accessibility in recent years. We see examples of it pop up in mobile gaming, shopping apps, and a whole lot more. Essentially, what it means is augmenting, or enhancing our view of the world before us with additional images and tech. It is a digital addition to the scene created through clever applications and the use of the camera, sensors, processors, and other tech within your device. As smartphones get smarter, we are able to enjoy more seamless superimposed images and GPS-based information. With time, this can only expand further with greater applications as tech becomes faster, smarter, and more reliable. We may also see projections with holograms and other hands-free wearable tech.
There are essentially four different types of augmented reality technology. They are:
1) Marker-based AR
2) Markerless AR
3) Superimposition-based AR
4) Projection-based AR
Marker-based AR is one of the more simplistic forms of AR that has been part of our digital lives for longer than we realize. The marker triggers visual content and other responses when recognized by the device in question. All you need is a form of camera with optical imaging, the right software to process the image, and a compatible marker. In other words, it is image recognition. If you have ever scanned a QR code on the back of a promotional pamphlet or underneath a sculpture in a gallery, you already have some idea of how this works.
Marker-based content is great for providing additional content and context to items in exhibitions or in advertising. Markerless AR takes this in a different direction by doing away with all the physical codes and markers. This is more efficient and doesn’t rely on users being in precisely the right place to get the information they need. Instead, this approach uses information from the GPS system, gyroscope and accelerometer in the device to provide location-based data. This then overlays information onto the screen. This is the perfect way to turn a camera into a feature-rich map with recommendations and directions.
This next approach takes things further by creating a blend of our current reality and a virtual world. Applications can take advantage of your camera to superimpose items onto the image. There are lots of applications for this already in the world of retail and some medical fields. Retailers can now offer try-before-you-buy programs where consumers get to see the products within their reality – to a point. Shops like IKEA use this, so customers can place items in their room and get an idea of how everything fits together. There are also applications for trying on clothes and glasses from the comfort of your home.
Finally, there are projections. Superimposition is fun and provides an idea of an alternative reality. But, it does so on the flat screen of your device. Projection-based AR projects a 3D image out into the space beyond, which essentially gives us a hologram. Hologram-tech is already part of the entertainment industry, where festival curators can use holograms of artists to bring them back to life. There is even a tour called “An Evening with Whitney” taking advantage of this in 2020.
At first, the applications and tech above all sort really clever and futuristic. However, we then come to realize that three out of four are already in play on our smartphones and tablets. Apple has made use of AR in its systems for a long time now and the smart cameras, processors, gyroscopes, accelerometers, and other tech play their part. We can scan QR codes, barcodes, and other markers without thinking about it. Shop-and-scan apps are a breeze for those with iPhones. Markerless-AR is increasingly popular in educational apps and maps. Online shoppers with the benefit of mobile-friendly browsing on the iPhone also get to play around with those superimposition-based AR tools.
The odd one out right now is the projection-based AR. As yet, we can’t shoot holograms out of our phones as though they were R2D2. But this is a common trope in many modern sci-fi shows and scenes set in the not-to-distant future. Ads and video-call holograms float above the screens. With time, could this become a reality?
It all depends on the hardware installed in our future tech.
The most effective AR devices, with the potential for reliable imaging and realistic experiences all require the following:
1) A camera
2) A responsive sensor
3) A reliable processor
4) A projector
5) Some reflective mirrors
At the moment, the best smartphones and tablets do pretty well on the first three points. The cameras and lenses on offer create sharp images of the world around us, making it easier for AR systems to locate markers. Smart sensors then process the information about the location to aid with processing and modeling. As for the processors, your smartphone is a speedy, intelligent system with enough CPU, RAM, and flash memory to handle advanced systems. The inbuilt GPS and Bluetooth help too.
However, the projectors and reflective mirrors aren’t a necessary part of our experience right now. AR headsets are more likely to offer these projectors first because they have the space and hardware necessary. It will take a while to figure out how to make one small enough to fit into a smartphone or tablet. The same is true for the reflective mirrors to align the image with the viewer’s eyes. This isn’t a necessity on phones but could would in headsets with ease. If anyone is going to figure out how to get the projectors into their phones, it might be Apple.
Apple’s app store is full of opportunities to test out AR. You can redesign your garden with iScape, learn more about the stars and human anatomy, and also play tones of games. Minecraft Earth takes the original concept and gives it an AR twist. This is made possible through the high-end cameras, motion sensors, and graphics processors in all their devices. They have also created the ARKIT framework so that developers can create their own apps that are instantly compatible with iPhone and iPad devices. Not only does this ensure a great sense of quality in the applications and their processes, but it also makes the apps marketable to the countless Apple users.
This leads to an interesting question about the future of AR in Apple tech. Will everything remain within phones and tablets, just with improved technology, or could we see specialist devices and wearables? The most well-known AR wearable is perhaps the Google Glass – although not always for the right reasons. The idea of a pair of glasses with AR integration is appealing for those that want that hands-free approach. There is potential here for superimposition-AR without the barrier of a handheld device. Markerless-AR could place all those GPS-led images, directions, and recommendations right before your eyes.
Apple does have plans for AR Glasses. In fact, there have been patents floating around since the death of Google Glass in 2015. The rumored name is Apple Glass and we may see an announcement nearer the end of the year. Users will be able to see images displayed on the lenses and then control the image with gestures around the frames. The idea is that there won’t be a camera, but rather a LiDAR scanner for effective generation of content. It will then pair with the iPhone. It is also worth noting that they have filed a patent for a catadioptric optical system that projects images into the eyes. But, if we might not get glasses for a few years, this futuristic idea could be a long way off.
The fact that we use AR so much within our everyday lives, often without realizing it, shows how important it has become. It not only enhances our entertainment but also our experiences as gamers and consumers. At the moment, we seem to have hit a peak in what we can achieve with AR technology in smartphones. However, app developers can always tech that tech in exciting new directions. We will have to wait and see if Apple comes through with that Apple Glass concept in the next couple of years. Or, maybe we will get new projection-based AR tech through headsets first? Either way, augmented reality technology remains exciting and innovative.
You may also be interested in: