Assessing a Shape Descriptor for Analysis of Mesoamerican Hieroglyphics: A View Towards Practice in Digital Humanities

Abstract

We assess the Histogram of Orientation Shape Context (HOOSC) descriptor as a shape analysis tool, with the application domain of Maya hieroglyph analysis. Our aim is to introduce this descriptor to the wider Digital Humanities (DH) community, for possible DH-related shape analysis applications. We discuss some key issues for practitioners, namely the effect that certain parameters have on the performance of the descriptor. 

Maya hieroglyphs

Maya civilization is one of the major cultural development in ancient Mesoamerica. The ancient Maya language infused art with uniquely pictorial forms of hieroglyphic writing, which has left us an exceptionally rich legacy. Most Maya texts were written during the Classic period (AD 250-900) on various media types, including stone monuments. A special type of Maya scripts were written on bark cloths as folding books from the Post-Classic period (AD 1000-1519).

Maya texts were typically composed of glyph blocks. Blocks were typically composed of combinations of individual signs. Maya hieroglyphic analysis requires epigraphers to spend a significant amount of time browsing existing catalogs to identify individual glyphs.

Automatic Maya glyph analysis

Automatic Maya glyph analysis has been addressed as a shape matching problem, and HOOSC was developed as a robust shape descriptor [1]. We follow our previously proposed glyph retrieval system [2], which considers both shape and glyph co-occurrence information. The Retrieval pipeline is shown in the figure below.

Retrieval system pipeline

HOOSC

An input shape image (Maya glyph in our case) is first pre-process into thin lines as following:

pre-process steps

Then HOOSC descriptors are extracted from a set of evenly distributed pivot points along the lines as following:

  1. compute local orientation field;
  2. define a circular space centered at a given pivot point location;
  3. partition the circular space into rings and evenly distribution orientation angles;
  4. compute histogram of local orientation distribution within each split region;
  5. concatanate histograms into one vector representing the HOOSC shape feature of the given pivot point.

We follow the previously recommend settings to split the circular region into 8 angles (in step 3) and  quantize orientation into 8 bins (in step 4), and study the effect of the following parameter choices to the retrieval results:

The figure below shows spatial partition of a given pivot point with five different ring size (1, ½, ¼, 1/8, 1/16, all defined as a proportion to the mean of the pairwise distance between pivot points) on the local orientation field of the thinned edge image:
HOOSC

Results discussion

Our results on two datasets derived from ancient monuments and codices Maya hieroglyph data show the following:

The figures below show average of the groundtruth ranking in the returned retrieval results, using the codex dataset. Figure at the left shows results when only the shape similarity is considered; and figure at the right shows when the glyph context information (statistical language model) is incorporated:

result_shape result_LM

HOOSC code

HOOSC is a local orientation descriptor similar to HOG, and SIFT. It can be used in any thin line images, which are usually pre-processed from natural images to represent strong edges or object contours, or from hand-written or sketche images to extract the essense of the strokes.

Pre-processing

In the case of natural images, edge or contour detection algorithms can be used to extract thin lines; in the case of hand written or sketch images, a binarization process followed with mathematical morphology operations can be used to extract thin lines.

Main code

Requirement: matlab;
Input: pre-processed thin line image;
parameters: number of pivot points (N); size of spatial context (S); number of rings (R); orientation intervals (A); number of orientation bins in the histogram (B).
Output: a matrix with N rows and RxAxB columns, each row represents a feature vector for a pivot point.

Usage

The extracted local descriptor can be used in a Bag-of-Words framework, to represent each image as a frequency histogram of the appearance of visual words.

Download

HOOSC code download via Idiap resources page
Datasets used in this paper are available here (coming soon)

Paper:
Assessing a Shape Descriptor for Analysis of Mesoamerican Hieroglyphics: A View Towards Practice in Digital Humanities
Rui Hu, Jean-Marc Odobez, and Daniel Gatica-Perez, in: Digital Humanities, 2016.

Bibtex:
@INPROCEEDINGS{Hu_DH_2016,
author = {Hu, Rui and Odobez, Jean-Marc and Gatica-Perez, Daniel},
projects = {Idiap, MAAYA},
title = {Assessing a Shape Descriptor for Analysis of Mesoamerican Hieroglyphics: A View Towards Practice in Digital Humanities},
booktitle = {Digital Humanities},
year = {2016}
}

MAAYA Project Website