Category Archives: Augmented Reality

Site Visit Feb 10 NFT Overlay Tests

These tests were based around the creation of an overlay which specifies to the user what to focus the camera on to generate the NFT content. The overlay disappears once the NFT has been tracked. IMG_0509

Once the tracking works a small square was programmed to appear this will be replaced with interactive content now that we know it works. This stingray NFT were successful except for the sizing of the NFT on the UI. This will be amended. IMG_0510

The Text NFT was unsuccessful however although the overlays sizing was better than the stingray.
IMG_0515

Below is the closest we could get to matching the Text NFT to the marker.

IMG_0516

 

SITE VIST 20 JAN ICON REFERENCE AND MANHIRE NFT

For this site visit we aimed to test the functionality of the geo location tracking as well as live testing the natural feature tracking at each of the three writers walk sculptures ( except Bill Manhire).

IMG_0368

At Vincent O’Sullivan’s Sculpture we tested the NFT, with a small cube appearing at the bottom of the NFT marker to show it was working. It did take a while for the tracking  to work however so we took another hi res image of the stingray to make into a better NFT marker image for tracking.

IMG_0369

We then proceeded to Bill Manhire’s Sculpture to test the location tracking, this worked without any problems (the 3D object spun faster depending on how close we were to the sculpture).

IMG_0372

Our next stop was Katherine Mansfield and the geo location tracking worked well with this sculpture too. The pop up that says augmented view is what the user will tap to initiate the AR views(360 and NFT).

IMG_0376

The NFT AR also worked once initialize but this still needs work. so as before we have re taken some NFT Images to revisit and refine this tracking.  Overall this Site Visit yielded some successful testing  and has given us information to build on from this point onwards.

Initial Augmented View UI Design

IMG_8301

360 View, onscreen interactive 360 AR instance.

  • Back top Map Icon in top Left
  • Information Screen Icon top right
  • Sawp to NFT Mode Icon bottom right

NFT View, Transparent Image overlay so the user can get in the right position to initiate the AR View.

  • Back top Map Icon in top Left (same as 360)
  • Information Screen Icon top right (same as 360)
  • Swap to 360 Mode Icon bottom right (same position as swap to NFT )

Third Meeting 20th December

Back End

Link to Slideshow included:

https://docs.google.com/presentation/d/18ZH8SSkTCBNYMoy3YmAe1uga2C0y_MWAuNIFAaeCqrE/edit?usp=sharing

Moving forwards with our CMS for the writer’s data we discussed refining our Data Fields/ Categories for writers.

  • The Short story field should be worded as “featured work/s” as short story implies a genre.

 

Relationship Network for CMS

Could we create a relationship network visualisation, to show the relationships between different pieces of information?  (for example known associates/influencers, linking the writers with each other or other prominent figures) this gives the user/researcher context.

 

Fields

As we are not completely familiar with classifying literature we should look at other models that categorise literature such as:

  • Book council
  • Wikipedia?
  • other sites?
  • NZ etc cross reference
  • Cultivating creative capital/ cultural legacy

 

NFT Tracking

  • Typeface for the etched Wellington Writer Walk sculptures (Kathrine Mansfield) Sydney will follow up on the Auckland Typographer so we can source the exact typeface

 

Sound for App

Start to source the sound effects, for the App sooner rather than later, so we can integrate them into the App

  • We should create a list of places we would need sound for feedback especially

 

Victoria Marketing Department

Victoria University’s Marketing Department will need to see the App around March for Branding and such. We need to keep this in mind and discuss it at each meeting.

 

Domain names URL

For our static website we need to think of a domain name / list of domain names to lock down, this needs to appeal to the teenager demographic and be memorable.

 

Send link to Sydney and Matt

The link to the online sandstorm blog need to be linked to Matt and Sydney for revision.

 

Next Steps 

  • Angle testing NFT onsite at the Waterfront to make sure the initialisation of NFT markers is fluid
  • Begin Research on Mansfield, Manhire and O’Sullivan
  • Paper Concepts for AR experiences completed by the 9th Jan 2017 ready for discussion and digital development
  • UI Concepts
  • General Bug fixes and refinement of App
  • Further integration of Back end and Front End
  • Map Development

NFT/APP Field Test 2

I was out for a walk on Friday 16th and I decided to test the application at the Katherine Mansfield Sculpture. The Location services worked well and mapped my location well (screenshot below).IMG_0265

The initialisation of an NFT on the sculpture itself worked but took too long and i needed to adjust my viewing angle to make it work. But once it did work the tracking was great and worked seamlessly.

For our next test we should work on the initialisation of the AR.

Unable to Dynamically Add Markers (Limitation of ARToolKit 5)

When developing for mobile applications its important to be aware of how intensive your code is and to try and optimise it at every point possible. One of the optimisations I have been looking into is dynamically loading NFT markers so as not to create a huge wait when the application first loads.

From testing I have deduced that each NFT marker added into the scene adds around 1.5-2.5 seconds of load time. This may not seem like much but when you consider that we are planning on having 3 NFT’s for each writer, and that we may end up having more than 3 writers in total those numbers quickly add up.

Unfortunately ARToolKit 5 doesn’t allow for dynamic NFT loading, so after some testing I have come up with a cleaver way to get around the problem. Instead of loading a new scene each time the player switches MapView to ARView (or vice versa) I am instancing an ARcontroller that is relevant to each writer and only stopping and starting this when the user needs those specific NFTs.  To put simply this allows for the software to only load the NFT’s relevant to the writer in the users location. It also then allows for very quick transitions between MapView and ARview by utilising camera culling masks

Precedent – Wallame

What is it

WallaMe is a free iOS and Android app that allows users to hide and share messages in the real world using augmented reality.

Users can take a picture of a surface around them and write, draw and add stickers and photos on them. Once the message (called Wall) is completed, it will be geolocalized and will remain visible through WallaMe’s AR viewer by everyone passing by. A Wall can also be made private, thus becoming visible only to specific people.

UNADJUSTEDNONRAW_thumb_ceAll the Walls created worldwide can be seen in a feed similar to those of social networks like Facebook and Instagram, and can be liked, commented on, and shared outside the app.

WallaMe is mostly used to create digital graffiti and for proximity messaging.

Source: https://en.wikipedia.org/wiki/WallaMe

 

How it works

Wallame allows you to create your own markers based o photos and map AR instances to them.

Content can be in the form of;

  • Images
  • Doodles(drawn using the app)
  • Text

An example video is below

Source: http://sites.gsu.edu/cetl/2016/07/28/cool-tools-wallame/

 

What will we use 

The ability to have a user created content is exciting as users could leave their own messages/ marks on the overall experience.

Other interactions such as the ability to doodle/ place content in space could be very powerful.

One limitation is that the app can only see one AR instance at a time, which is chosen by the user.

 

NFT Field Test 1

Jeff and I went for a walk to test out his fix NFT. To do this he adjusted the tracking of the feature points that the software tracks to only be the text (and not the texture of the stone).

To do this we developed a simple scene in which we could test the tracking and see if it worked on the actual typographic sculpture.

The result, seen below, worked well and tracked nicely using three lines of text on the sculpture.UNADJUSTEDNONRAW_thumb_a8

We also had a look for other natural features we could track. One of the points of interest we found were hidden QR Codes which we later scanned and found they were markers for Orienteering on the Waterfront.

https://wwfcourse.wordpress.com/

UNADJUSTEDNONRAW_thumb_a3

Furthermore we will need to check the typographic sculptures in different conditions if possible as below is the James K Baxter sculpture at high tide. This would not be ideal for NFT Tracking at all.

UNADJUSTEDNONRAW_thumb_9a

Second Meeting

The following were the points discussed in our second meeting

Choosing Writers

We chose the following three writers for development:

Katherine Mansfield, Bill Manhire and Vincent O’Sullivan

These writers were chosen for their locations on the walk and their ties to Victoria University.

 

Geo Location Augmented Reality

We created a basic 360 Ar Mobile Experience which can be triggered on a map using location services on a phone.

some of the points we discussed around this were:

  • giving the user a radar
  • creating a mini-map or using mini map mechanics to help design our way finding
  • Having a deep look at Pokemon go

 

Literary Atlas Map Design

We discussed how the map could look and we decided to create a few concepts to go over next meeting. Maps would be heavily text based and stylised to suit the Literary Atlas theme.

 

Testing the Natural Feature Tracking (NFT) 

With the images we took during our site visits we created NFT Markers to create AR experiences based on the sculptures.

Note, this will be explained fully in another blog post linked here:

In this meeting we discussed how we needed to test this further as the inclusion of this would be of great benefit to the Application as a whole as it is something tangible to interact with. I knew this from previous projects such as Standing Ovation.

Our main problem was the 3D sculpture faces messing with the 2D tracking. We will solve this by;

  • Doing more testing with the sculptures themselves
  • Potentially Adjusting the camera settings but this may not work on all phones

 

Audio Playlist

An Idea we had was a Literary playlist based off the audio files we have access to. We could create this playlist which would play while the user is walking so they could listen to these texts as well as visualise them. Some precedents for this are;

 

Users

Who are we targeting? We need to decide this. Some options are:

  • Tourists
  • Wellington Residents
  • Teenagers/HighSchools

 

Research Collections

The following sources should be used for writer research:

  • Digital NZ -create sets
  • NZ ETC
  • StQry Museum