Investigating the Dimensions of Spatial Language: A Preliminary Study

Abstract

In order to collect rich data on spatial prepositions, we have created a framework for gathering user annotations in 3D virtual environments. The framework is built using the Blender 3D modelling software and game engine. The framework initially provides two distinct tasks with which our preliminary study was conducted — a Selection Task and a Description Task. In both tasks, participants are given a first person view of a scene which they can navigate using the mouse and keyboard. In the Selection Task participants are given a preposition on screen and asked to select all figure-ground pairs in the scene which fit the preposition. Once they have selected all pairs they believe to be admissible they are shown another preposition and asked to repeat the process. In our preliminary study, we limited this task to ‘in’, ‘inside’, ‘on’, ‘on top of’, ‘against’, ‘over’, ‘under’, ‘above’ and ‘below’. In the Description Task objects are highlighted and participants are able to type in a spatial description of the object. In our preliminary study we asked participants to give descriptions of the object locations using a definite description, in the format figure + preposition + ground e.g. ‘the guitar by the bookshelf’. We also allowed the use of multiple prepositions if the participant deemed it necessary e.g. ‘the cup on the table near the lamp’. To make the annotations more relevant to our research we asked participants to only use the prepositions in the Selection Task plus ‘to the right of’, ‘to the left of’, ‘in front of’, ‘behind’, ‘near’, ‘next to’, ‘at’. The current dataset is from a session we hosted in one of our computing labs where we invited participants from across the university campus to take part. We asked participants to complete either the Selection Task or Description Task in one of our scenes. Participants were given a brief introduction to the study and instructions on how to complete the given task along with explanations of key terminology. All participants were asked to create annotations for ten minutes. 23 native English speakers participated, of which 13 performed the Selection Task and 10 performed the Description Tas