INTT: Identifier Name Tokenisation Tool

Butler, Simon (2011). INTT: Identifier Name Tokenisation Tool. The Open University.

Abstract

Identifier names are the main vehicle for semantic information during program comprehension. For tool-supported program comprehension tasks, including concept location and requirements traceability, identifier names need to be tokenised into their semantic constituents.
We present INTT, a Java library that implements an approach to the automated tokenisation of identifier names which improves on existing techniques in two ways. First, it improves the tokenisation accuracy for single-case identifier names and for identifier names containing digits, which existing techniques largely ignore. Second, performance gains over existing techniques are achieved using smaller oracles, making the approach easier to deploy.
Our tokenisation library and the datasets used for its evaluation are made available in this package. Also included is a database of unique identifier names extracted from the 60 Java projects, as a resource for further research on program comprehension.

Viewing alternatives

Item Actions

Export

You can export this page using these formats