The initial step in most object-based classification methodologies is the application of a segmentation algorithm to define objects. Modelling the human visual process of object segmentation is a challenging task. Many theories in cognitive psychology propose that the human visual system (HVS) initially segments scenes into areas of uniform visual properties or primitive objects. If an accurate primitive-object segmentation algorithm is ever to be realized, a procedure must be in place to evaluate potential solutions. The most commonly used strategy to evaluate segmentation quality is a comparison against ground truth captured by human interpretation. A cognitive experiment reveals that ground truth captured in such a manner is at a larger scale than the desired primitive-object scale. To overcome this difficulty we consider the possibility of evaluating segmentation quality in an unsupervised manner without ground truth. Two requirements for any method which attempts to perform segmentation evaluation in such a manner are proposed, and the importance of these is illustrated by the poor performance of a metric which fails to meet them both. A novel metric, known as the spatial unsupervised (SU) metric, which meets both the requirements is proposed. Results demonstrate the SU metric to be a more reliable metric of segmentation quality compared to existing methods.