Abstract

A tactile record and playback system will progress _tactileology_—a new cross-disciplinary field related to tactile sensations—as it will enhance its use in the instruction, archiving, and analysis of human manipulation. In this paper, we describe two key devices for achieving tactileology: a tactile sensor capturing human tactile sense (fingernail color sensor) and a robotic tactile sensor, both of which can detect not only normal force but also tangential force. This is beneficial because people manipulate objects and tools in various ways, such as grasping, picking, and rubbing. The fingernail color sensor registers the three-dimensional (3D) force applied to a fingertip by detecting the fingernail color change caused by blood distribution under the fingernail, which can be observed with green illumination and a miniature camera. Since detecting this color change is more complicated than using a robotic sensor, the relationships between the image and 3D force are learned using a convolutional neural network (CNN). In the robotic sensor, the 3D force applied to a robotic finger transforms into a bright area using an illuminated acrylic core, a rubber robotic finger skin, and a miniature camera. We measure normal force and tangential force by the brightness and movement of the bright area, respectively. Using a force gauge or an electronic scale for measurement, we perform a series of evaluation experiments. The experimental results show that the precision of both the fingernail color sensor and the robotic tactile sensor are sufficient for our system.