Using VR Trackers – A Game Based on 3D Movement

In May-June of 2018, I developed a game which involved the usage of VR tracking technology to map the player’s movement in 3D space onto the 2D screen. The game encouraged the player to explore the space by presenting them with pickups which guided their movement. Aimed to be a relaxing game, players could collect the pickups at their own pace.

The initial idea revolved around placing a tracker on a real world object, such as a toy plane. Then, moving the object around the 3D space would move the object on the screen in the same way. It would be similar to a Kinect game, but fully tracking the 3D coordinates of the object in a way which included depth.

When considering the idea further, I planned to create a 3D environment which the player could explore by moving the object in the real space. However, it would have been time consuming and complex to develop such an environment in the limited I had, while ensuring it was interesting to explore, aesthetically pretty, and well polished.

Therefore, the final game was greatly simplified. It involved plotting a series of pickups which acted both as a goal and guidance for the player. It was no longer an exploration game, but it allowed me to focus on plotting deliberate movements.

Here is a gif of players interacting with the completed game:

Game Flow

To start the game, the player is prompted to collect a single blue pickup, which then starts the plotting of the movements. Each time, they have to start with the blue pickup, and when they collect it, another pickup will then turn blue, signifying it is the next in the sequence. This is what guides the player in a certain path. They can only see a number of pickups at any time (not the whole sequence), and collecting the pickups will reveal more that come after them.

When the last pickup (in the last phase) has been collected, the game will fade to black and restart. Text has been added to the first scene (the text over the first initial pickup telling the player to collect it). This has been added so it will be more suitable for an installation setting (when the player sees the text again, they know they are at the end, and they can pass it to someone else – making a clear differentiation between the start and end of the game). 

Routine of Movements

The routine of the movements the player has to make had to be designed. In preparation, some observations were made from real dance, and from taking recordings of example movements (using the full 3D space). These were then used as reference for some of the movements.

The first three movements are simple planar movements, which act as a warm up to get the player started. The first does not consist of much reaching and involves only linear movements. It was implemented by simple linear interpolation between points. There are four linear movements which are joined together.

The 2nd involves a circular action. The player has to kneel down to reach the pickups at the bottom. It consists of a repetitive motion of an arc, before transitioning into a full circle. It was implemented by moving a ‘tracker’ object around the space, which plotted the points as it moved. To get a circular motion, sin and cos functions were used with a changing angle.

The next (third planar movement) movement is also planar, but moves to the right-side plane, introducing some forward and backward movement (including depth), and guiding the player to reach upwards for the first time. The same linear interpolation and equation of a circle was used.

By this point, the player will have moved left, right, forward and backwards, as well as up and down.

There are then some transitional movements between this and the next ‘Helix’ phase. For this phase, the routine becomes a little more complex as the player has to move in a helix rotation, which can involve some turning. The player will need to pay attention to maintain their sense of depth as they move around.

The basis of the movement is from the equation of the circle used previously, but this time the tracker also moves up by a certain interval each time. Additionally, the radius of the Helix is adjusted over time using an animation curve (which means that the player starts from the middle and moves outwards).


The routine ends with a movement which moves from the top to the bottom as the player walks forwards, but also guides the player to make wavy movements with their arms. This phase is very challenging as the player must make full use of the depth cues to understand the positioning of the pickups. The phase was implemented by manipulation of a sine wave.

Besides designing the movements, it was important to ensure that they were connected together appropriately. The routine was designed with that in mind, but any required transitions were also added to keep the flow of movement (phases which start at the end of the last phase and move into the start of the next phase).

The Greatest Challenge: Depth

Presenting the sense of depth to the player was the largest challenge throughout the entire project. A number of measures were taken to make the understanding of depth easier.

  • A grid texture was added to the walls.
  • Shadows and projections were added so the player could see the position of their ‘hand’ in relation to the pickups on every axis.
  • A line renderer was added which emitted lines from the hand in every direction, hitting the walls.
  • The implementation was changed to only show a few pickups at a time (avoids clutter and means that the player is not confused by the position of many pickups at once).

Further methods which were explored or considered:

  • Changing the size or colour of the pickups depending on how far away they were from the camera or the player.
  • Testing various angles to see which aided the perception of depth.

These changes were definitely vital in letting the player gain that sense of depth.

Screen Shot 2018-06-21 at 13.37.40

Aesthetics and Polish

Because the game is simple in concept, the visuals and polish were important in creating a good experience. This was done through the use of textures, particle effects, lighting, and the models and appearance of the pickups and the ‘hand’. Additionally, a note will play every time the player collects a pickup. These notes join together to create a melody as the player completes the movements. Players that played the game before and after polish definitely noted a great improvement.

Further Considerations

There were some more considerations taken when designing and creating the game. The first was whether to include mechanics such as score (with the player gaining more score if they collect pickups in a chain) and timers. This could have added more complexity and challenge, but I decided not to include those features so the game would be more relaxing, and to let the players approach the movements without any stress.

At first, the player could collect the pickups in any order. In the final version, they are more guided. If the players could approach the plots in their own way, the movements would not be as deliberate or planned.

At one point, I considered adding instructional text to the game, but decided against it in the end. The game should mainly use other prompts to make it clear what the player must do. I thought that too much text would overcomplicate the game and take focus away from the movements. The only text in the game is the ‘UI’ text at the beginning which was explained earlier.


Initial testing was conducted at the beginning of development, when the routine of movements hadn’t been mapped out yet. There was a single helix-shaped movement which was presented to players. The goal of the initial test was to see how people responded to the general feel of the game.

It seemed that players enjoyed the concept, but the main problem was the issue of depth. This is the point at which I started to implement measures to make it easier for players to gain a sense of depth while playing the game (measures outlined in design and development section).

I also realised that it was a slightly difficult for players to collect the pickups, which meant that the movement was not very smooth. I made the collider size of the pickups larger to ease this problem.

I also tested various view points and camera positions to see which was more preferred. I tested a 1st person view, where the players saw the camera as if it came out of the controller they were holding, and a 3rd person view which mapped the room onto the screen. These two views were equally enjoyed, with some players preferring the 1st person view, and others preferring the 3rd person one.

The reason I decided to choose the 3rd person view was that I felt it placed more focus on the sweeping movements, giving a more dance-like feeling. With the 1st person view, the player tended to move their body around the room as if they were navigating the space (moving forward, turning etc), instead of using their whole body to make movements, and this was not the type of feel which was envisioned for this game.

After the routine was completed, the game was tested once more with players.

The first issue was that some players didn’t immediately realise they had to collect the coloured pickup first, so I made that indication more noticeable. I did this by making the pickup larger and animating it to pulsate. The game was also started by making the player collect a coloured pickup (so they make a connection that it is the one they must collect).

One player mentioned that they preferred to have more objective in the game in the form of scores and timers. I decided against this as it conflicted with the design choice to have a more relaxing experience (and some play testers certainly enjoyed not being rushed), but it’s clear that players looking for a more skill based game might not enjoy it.

How quickly the sense of depth was gained also varied greatly from player to player. Some could collect the pickups with ease, whereas others faced difficulty when navigating the space.

One player mentioned that they did not enjoy having to concentrate on the visual clues in order to gain the sense of depth, saying that it took away from the main concept of the game, which were the movements. Another player said that they enjoyed moving around the space and that trying to locate the where the pickups were was an interesting element of the game. Although this changes the nature of the game from one of relaxing movements to one of calculating the distances between objects on the screen using visual clues, which was not the core vision of the game.

As for overall feel and movements, people enjoyed the routine itself and did not mention any problems.

Conclusion and Potential Future Work

A potential improvement which could be made if the project were to be taken further, could be to calibrate the game to the user’s height/ arm span, etc. This means that the plotting of the pickups would be suitable for every player. The code for the plotting should be more generalised (using reference points, instead of plotting distinct coordinates) so that it could be adjusted as necessary.

A further development could be to design the game to involve using two controllers (which each have to collect different coloured pickups), allowing for even more complexity in movements. There could also be a multiplayer version where more than one person has to collect pickups at a time.

More complex and interesting movements could be plotted. There code be a mode allowing for user generated content, where the player moves their controller in the space and presses the trigger to plot a pickup at that point. Using this method of custom content, there will always be different movements for the players to try.

Additionally, the phasing of the plotted notes could have been improved to match better with the musical melody of the notes.

Overall, I am pleased with the final outcome of the game as it fulfils my initial goals.

The game still has its issues regarding the perception of depth, and every player has different thoughts regarding it. During testing, some players even suggested using the VR headset and turning the game into a full VR experience. I decided against this because it would have fundamentally changed the vision of what I wanted the game to be. I stuck with the initial idea and tried to find the best way to map the 3D space on the 2D screen.

Whether the experiment was fully successful or not, many players who played the game said that they enjoyed the experience.

Close Menu