Tactile-JEPATopology-Aware Self-Supervised Representation Learning
for Distributed Tactile Sensors
1 Sber AI2 Skoltech3 HSE University4 AIRI
Abstract
Distributed electronic skins have sparse, irregular sensor layouts that visual self-supervised methods do not naturally capture. We introduce Tactile-JEPA, which predicts masked sensor embeddings using the sensor connectivity graph. Its dual-scale masking combines connected local regions with globally distributed targets to learn both contact detail and the state of the sensing surface. Across three datasets, the learned representations transfer to force and pose estimation, classification and policy learning. Tactile-JEPA reduces in-hand orientation error by 20.8% and force estimation error by 6.3% relative to the strongest reported baseline for each task.
Sensor graphs and downstream results
Choose a dataset to update the sensor graph and the reported results below.
SPARSH-SKIN / downstream evaluation
Reported experiments from the paper. The mask controls above illustrate sampling and do not change these scores.
Illustrative masks: 14% per target, 70% initial context. Training varies these budgets. The graph and mask membership stay fixed when switching Flat / 3D.
Method
Each sensor’s short signal history is projected to a token. A context encoder processes visible tokens, and a predictor estimates target embeddings. These are matched with the corresponding outputs of an exponential-moving-average target encoder, which receives the complete input.
During downstream evaluation, the pre-trained tactile encoder is frozen and a task-specific head is trained on top. The graph guides mask sampling during pre-training; it is not required as a downstream input. The objective predicts embeddings rather than reconstructing sensor readings.
Datasets
We study magnetic and piezoresistive sensing across single-hand, bipedal and bimanual configurations. DECO-50 groups 2,124 raw taxels into 528 training nodes.
Target-mask ablation
Mixed targets give the lowest force error and highest position accuracy. Global-only targets give the lowest policy error in this ablation.
| Target masks | Force RMSE ↓Sparsh-skin · cN | Position accuracy ↑Sparsh-skin · x, % | Policy RMSE ↓DECO-50 · normalized |
|---|---|---|---|
| Image-style I-JEPA | 14.99 ± 1.03 | 93.50 ± 1.46 | 0.4959 ± 0.0083 |
| 4 local | 15.52 ± 1.03 | 94.32 ± 1.86 | 0.4705 ± 0.0145 |
| 4 global | 15.17 ± 0.96 | 95.32 ± 1.65 | 0.4600 ± 0.0019 |
| 2 local + 2 global | 14.05 ± 0.73 | 95.74 ± 1.05 | 0.4635 ± 0.0044 |
Training-run results; independent of the interactive sample. Graph variants above use global random context. The I-JEPA reference uses its image-style masking recipe. Ablation and main-table DECO values are reported separately as supplied.