Function in Engineering: Benchmarking Representations and Models

Summers, Joshua D.; Eckert, Claudia and Goel, Ashok (2017). Function in Engineering: Benchmarking Representations and Models. Artificial Intelligence for Engineering Design, Analysis and Manufacturing, 31(4) pp. 401–412.



This paper presents the requirements and needs for establishing a benchmarking protocol that considers representation characteristics, supported cognitive criteria, and enabled reasoning activities for the systematic comparison of function modeling representations. Problem types are defined as reverse engineering, familiar products, novel products, and single-component systems. As different modelling approaches share elements, a comparison of modelling approaches on multiple levels was also undertaken. It is recommended that researchers and developers of function modeling representations collaborate to define a canonically acceptable set of benchmark tests and evaluations so that clear benefits and weaknesses for the disparate collection of approaches can be compared. This paper is written as a call to action for the research community to begin establishing a benchmarking standard protocol for function modeling comparison purposes. This protocol should be refined with input from developers of the competing approaches in an academically open environment. At the same time the benchmarking criteria identified should also serve as a guide for validating a modelling approach or analyzing its failure.

Viewing alternatives

Download history


Public Attention

Altmetrics from Altmetric

Number of Citations

Citations from Dimensions

Item Actions