Augmented Image- A TCW project Case Study

TheCodeWork Team

6 October 2021 7 minutes read
content
Reading Time: 7 minutes

Hello again! I do hope that you are following our streak of blogs on Augmented reality and augmented image. What? Are you new here? Okay, have a look into these for better understanding and also to keep up with the flow of the blogs.

You can also check out our video on the same, on our Youtube channel at TheCodeWork.

Okay, now back to the context. So as you already know, we have been working on several segments of Augmented reality and these case studies are a detailed approach to sharing our ideas and work process with you. 

It is welcome, right?

So here, we are going to have a detailed look into the project of Augmented Image, developed by our very own Android Development team at TheCodeWork.

So here we go into the world of illusion blend in reality once again! Follow up.

Overview of the Augmented Image Project

arcore

The APIs of Augmented Images in ARCore allows you to develop AR apps that have the ability to augment and detect 2D images in the respective user’s environment. These images can be posters or simply any product packaging. 

This project deals with the detection of such 2D images and producing their augmented image to exhibit information with the help of ViewRenderable or produce a 3-D model. Sounds cool right?

So it works like this;

When ARCore starts tracking an image, it provides estimations for the position of the image and the orientation of every frame. This entire tracking activity takes place on the device itself. 

Bonus Point: You don’t require any active internet connection to detect or track these images. 

ARcore basically scans the image that you store in the database when the image is detected. It also creates a 3D image of that specific image.

Magic!

Augmented Images for Android and how it works?

augmented images for android

To begin with, you have to provide some reference images. After that, ARCore makes use of some machine vision algorithm to obtain characteristics from the grayscale data in every image. Consequently, it saves a representation of these characteristics in several Augmented Image databases. See the flow already?

During runtime, ARCore explores these saved characteristics on flat planes in the respective user’s surroundings. This further allows ARCore to detect the images in the real world and assess their orientation, position, and size even if they are not provided.

What all can Augmented Images do?

ARCore has the ability to track around 15-20 images together. However, ARCore does not track or detect recurring data of the same image. 

Also, guess what? The database of every Augmented Image has the ability to save information of around 1,000 reference images. How cool is that? It can store an endless number of databases. However, there is a catch. Out of these, only one particular database remains active at any particular given time.

augmented Image

In addition to all this, you can also add images to the database of Augmented Image during runtime. But do remember that, there is a limit of every database and that is around 1,000-image per database.  

Bonus: You can also download databases that are created in the past, by using a decent network connection. 

Okay, so now comes a bit of a tutorial for you. Are you ready? Here, we go;

When you add an image to the database, try and provide the physical size of that image. This will result in an enhanced image detection performance. Also,

Makes sense, right?

Check out the type of images that ARCore tracks or responds to;

augmented reality
augmented reality

When ARCore starts to track an image, it produces estimates for the orientation and position of the image in every frame. And through this process, ARCore consistently refines the estimates as it keeps collecting more data on the same.

Now here’s the flow of it;

On detecting an image, ARCore keeps “tracking” the orientation and position of the image even if the image is no more in the view of the camera angle. I call it “efficiency” though!

However, while doing so, ARCore takes the orientation and position of the image to be static in the given surroundings. 

With the help of TrackingMethod, you can check if the ARCore is tracking the image based on the camera view detection (FULL_TRACKING). Similarly, you can also check if the ARCore is tracking the image purely based on the last detected position of the same (LAST_KNOWN_POSE).

Note: All these tracking activities take place on the device itself without the need for an active internet connection.

Requirements to build Augmented Image

Now let’s take a look at what all you need to build an augmented image; 

Best practices

Check out the ways to select reference images and what we incorporated for our project;

 We suggest a score of 75 for reference images though

Here are two examples:

Installation steps

There are some following steps

Libraries/Dependencies with their versions :

Sceneform– 

sceneform

Sceneform is basically a 3D framework making it easier to build ARCore applications without using OpenGL. In our project,we used the 1.15.0 version of google.ar.sceneform.ux:sceneform-ux.

Permissions :

We are using Camera permission and hardware.camera.ar permission.

Output will be

After scanning the image/poster you will see the 3D model of the respective image.

augmented image

Challenges

In the beginning scanning, the image was quite difficult for us because we are scanning random images.

Overcoming challenges

After the research, we find that we can only scan images that we stored in our projects assets folder.

What we tried to do but did not succeed

In our project, we had to identify the pressure sensor. Unfortunately, the android ARCore SDK fails to support 3-D object recognition. So we failed to identify the sensor. 

So what we did instead is, we just attached an image on top of the sensors using the ‘Sensor Id’. This is exactly what we used for the augmented image too.

What are our future plans regarding the development of the project?

Our future plan for this project is to make the 3d model move, whenever we touch the screen. How cool would that be?

Conclusion

So that was all about this segment of our project on Augmented Image. I do hope that this was helpful in giving you a peek into our workflow and thought process too. 

The target industries for this project include gaming, e-commerce, health tech, and education. So you already have an idea of the vast impact that this field of research and work has on today’s world and business combined. 

Here’s to the aspiration of being a valuable cog in the tool.

Also, if you are an entrepreneur who wants to build something from scratch, then do reach out to us at TheCodeWork. We are a bunch of startup enthusiasts and it would be a pleasure to be of any assistance.

Till then, happy brainstorming.

TheCodeWork Team

Our Content Team at TheCodeWork believes in quality content. We write everything related to startups and products at large. We publish our blog every Wednesday. Subscribe to our newsletter to get notified of our awesome content.

Ready to get started?

Others also Read