Loopsense: low-scale, unobtrusive, and minimally invasive knitted force sensors for multi-modal input, enabled by selective loop-meshing

With growing interest in unobtrusive human-machine interfaces, garments and textiles in general bear great potential to become ubiquitous user interfaces of the future. Areas of application range from wearables in the consumer and sports field [33, 41, 51] and in healthcare [5, 21, 34, 50], to smart environments in automotive [9], office [17], and home interior [7], and interaction with robots [44].

Researchers have tackled the field of textile-based user interfaces for decades. Most prominently, Project Jacquard [41] showed capacitive sensors that are well integrated into a woven fabric in a way that made them virtually unrecognizable. Challenges with woven fabrics however include their lack of stretchabilty, and – when compared to weft knitting – inflexibility of the fabrication process. This commonly requires woven fabrics to be tailored later on, which may be challenging when electronics are integrated that must not be cut.

Due to their composition, knitted fabrics on the other hand are inherently elastic and therefore ideal for stretching and draping. In the past, knitted sensors have been used as input devices for HCI, e.g., for gesture recognition [27] or for implementing haptic textile button elements for versatile scenarios [1], but also in medicine and healthcare, e.g., for tracking bio-features [34], gait-monitoring scenarios [21], or grasp classification in rehabilitation [5]. Even for space suits, 3D knitting has been discovered as a powerful means of engineering functional and complex multi-component gear with integrated sensors [19, 36].

However, traditional approaches of knitting touch or strain sensors are usually realized as an areal insert of conductive yarn which can be relatively large and distinct [5, 6, 21, 31, 36]. While such solutions work well within laboratory use, they can only sense exerted force, hence they are unable to discern between actuation types, such as strain and pressure. This makes them vulnerable to accidental activation or false positives, e.g., when a strain sensor is accidentally pressed. Furthermore, they come with a deficit in appeal and with a considerable restriction to visual and haptic textile design. Even integration into widespread patterns such as Milano, Cardigan, or Cable Designs can be limited since they may interfere with the sensor’s performance. This has also implications beyond appeal, since a knitted fabric’s properties are often designed to serve functional purposes, e.g., to provide a required degree of stability, recovery, or anisotropic extensibility.

With Loopsense, we present a solution for knitting resistive sensor cells that unite multiple materials and therefore provide exceptional design latitude. Our knitted sensors are constructed from a special copper core litz wire enamelled with piezoresistive material [35, 38]. By using this material in our knitting process, we are able to scale down the actual sensor cell to a single intersection of two wires, which can be accomplished by a range of basic stitch types that common weft or v-bed knitting machines provide. In this paper, we present the results of our exploration into physical sensor cell construction, i.e., loop meshing, as well as integration into knit designs. By utilizing different types of stitches, we are able to fabricate sensor geometries exhibiting distinct responsiveness with respect to actuation (cf. Figure 1). As a result, our novel textile devices can be used to discern strain along two directions as well as orthogonal pressure. This is a fundamental advantage over state-of-the-art knitted sensors, as it not only extends the fabric device’s input space and expressiveness, but along its small scale and potential unobtrusiveness it also promises innovative utilization in a variety of scenarios of interaction with textiles. Moreover, our method does not require manual postprocessing or stacking of multiple fabric layers and can be applied to virtually any fully-fashioned or 3D knitted fabric design, since the sensor cells are established entirely throughout the knitting process.

Summarizing, the five major contributions of this paper are:

First advances into the field of sensing textiles and garments have been made more than two decades ago. An early example is the work of DeRossi et al. [8], which coated conventional fabrics with a conductive polymer to sense strain and temperature. Post at al. [40] used embroidery to augment existing fabrics with conductive yarns to embed electronic components and establish recognizable user interfaces operated by capacitive touch. More recently, Strohmeier et al. [46] demonstrated a multi-layer patch for hybrid resistive/capacitive input. Honnet et al. [14] showed a method of functionalizing a wider range of base material by polymerization, equipping them with piezoresistive properties. In contrast to these works, we target manufacturing-time integration, instead of relying on modification of existing material.

Related to this objective, there has been considerable progress in recent years: Poupyrev et al. [41] showed a conductive yarn that was fit for weaving processes which was later integrated into the Levi’s Trucker Jacket available for purchase. Olwal et al. [33] demonstrated conductive yarn integrated into braids, both used capacitive sensing techniques to detect finger hover and touch. Devendorf et al. [11] demonstrated tools for supporting design and fabrication of woven user interfaces. While some of these solutions are hard to customize, we focus on weft knitting, which enables to design fully-fashioned fabrics in high structural detail and a combination of numerous raw materials.

Driven also by the increasing accessibility of knitting technology in recent years [26, 32], there has been growing interest in this field. Wicaksono et al. [49] showed a multi-layered knitted musical keyboard that combined capacitive and resistive sensing methods. Vallett et al. [47] showed a flexible knitted keyboard based on a capacitive sensing and swept frequency Bode analysis. McDonald et al. [28] built on-top of this and integrated the material into the knit using a spacer knit structure for better haptic appeal and discoverability. In [27], they investigated user interface scenarios based on this technology, focusing on touch and gesture input. Aigner et al. [1] demonstrated a haptic spacer knit element based on resistive sensing, that could be used for continuous input of pressure. Yu et al. [52] showed a knitted scarf for gesture input using passive electrical impedance tomography. In contrast to these works, the purpose of our work is to enable detecting strain as well as pressure. Furthermore, we aim at maximum compatibility with a wide variety of knitting structures, while giving the designer the opportunity to completely hide away the sensing parts.

Leong et al. [20] presented a multi-layered stretchable sensor matrix for sensing pressure on prosthetic limbs. Similarly, Wicaksono et al. [49, 50] knitted custom pressure matrices and permanently fused layers together with melting yarn for additional stability. They demonstrated sensing mats and socks for posture tracking, gait analysis, and to drive reactive audio systems during dance performances. Similarly, Luo et al. [25] showed a tool for knitting building blocks for multi-layered knitted sensors, to realize custom knitted UI layouts. We add to this body of work by presenting a method for a fully-integrated manufacturing of potentially numerous sensors in a single fabric, which can be fabricated ready-made and therefore does not require stacking, aligning, or fusing of multiple layers, since we insert a multi-material litz wire into the knitting structure that combines conductive and piezoresistive properties.

Parzer et al. [35] showed a wire with similar properties, however they did not go beyond sewing and weaving. Luo et al. [24] presented a knitted multi-layer sensor matrix for sensing pressure distribution. While the functional fibre was of a conductive core and a piezoresistive sheath, it was comparable to ours in terms of composition, it was unsuitable for knitting with a reported diameter of 0.6 mm and therefore inlaid in a straight trace, supported by the surrounding loops. Pointner et al. [38] demonstrated a knitted sensor using similar material, however the integrated wire was clearly visible on both sides and structurally disruptive particularly on the back side of the fabric.

However, for those solutions to be attractive for most everyday scenarios, they have to provide appropriate integration methods that enable textile and UI designers to be in control of the visual and haptic appearance. This is important in particular in the field of textiles, where sociocultural aspects have a big significance. Devendorf et al. [10] performed artist residencies where HCI practitioner teamed up with engineers and craftspeople to bridge this gap in the domain of woven user interfaces. More recent projects that consider design aspects in textile based technology include Project Brookdale [43] and Skill-Sleeves [18], where researchers team up with designers and artists to explore collaborative design and development. Further notable references that investigate traditional textile crafting in the context of technology are Irene Posch’s works, e.g., "Knitted Radio" [39], where she explored alternative production procedures. Finally, the research of Mlakar et al. [29, 30] showed that design aspects are essential for designing affordances without being limited by the sensing components. Our solution provides considerable latitude, so knit structures and compositions can be designed with little constraints.

Since our sensor design requires merely a pair of intersecting wires, we were interested in the implications of different sensor cell structures that may show vastly different mechanical features. Given the potential geometric complexity of knitted fabrics, there are numerous possibilities to establish these intersections, or to embed them in the surrounding knit. We observed that in certain structures, threads tend to either compress or dodge, depending on which direction the fabric is strained. One common limitation of resistive knitted sensors is while the amount of stress can be inferred, the cause is unknown. We hypothesized that, especially for double-faced fabrics, the mesh geometry can be designed in a way that threads have little to no contact except if compressed by an external force, making them ideal for sensing pressure exclusively, and vice versa. Other structures may instead be most affected by strain forces. We see these features closely related to structural metamaterials [15, 16], and believe they can be purposefully engineered with the use of mechanics and rope statics. However, the physical complexity of a knit topology as well as non-trivial material properties render this a highly challenging task and we therefore argue at this point a reliable simulation by computational means may be impractical. Hence, to advance into this direction, we restricted to an empirical analysis of a selected number of sensor compositions that we manufactured and inspected. As in V-bed knitting, there are numerous options to intermesh yarn, we focused on the most basic ones that we judged least specific to particular pattern repeats and therefore most portable to other solutions.

As discussed, our evaluation’s results suggest that it is possible to combine multiple sensor types and utilize their response profiles’ diversity to infer high-level actuation and deformation states. Since the sensors are exceptionally low-scale, it should be viable to do so on a relatively small area of 2 to 3 cm², or even smaller, depending on knitting machine’s gauge. For sake of mere demonstration, we present a straightforward and prototypical approach of harnessing the sensors’ features: we implemented sensor fusion using simple data processing methods, such as arithmetic combination of sensor readings, as well as smoothing and thresholding. Our method is built to operate on live data, hence it only utilizes past sensor readings. Due to its low complexity, it provides output at an interactive rate with negligible processing load and can therefore be run on a small-scale embedded device with little computational power. A detailed description of the involved processing steps can be found in the supplementary material.

We fabricated three sensing fabric prototypes, using an Interlock substrate, with a combination of integrated knit, front-miss, and tuck sensors (cf. Figure 9).

For our first swatch, for reasons of visual communication of functionality, we chose to have our sensors visible on the A-side of the fabric. In the first step, our objective was to discern between strain along course and wale directions; we therefore integrated a knit D and a tuck B, since we found the response signals of those a good combination of mutual exclusiveness to operate activation functions for both actuation modes (cf. Figure 8). In a second variation, our objective was to additionally detect pressure, therefore we added a front-miss A as a third sensor to get an additional input feature. In order to have the sensors hidden from the A-side of the fabric, we combined it with a knit A and tuck A. In a third swatch we successfully united the upper two sensors (tuck and knit) to a single dual-mode sensor (cf. Figure 9), since the upper loop of the knit A can simultaneously serve as the lower loop for tuck A. This lowers complexity in programming, knitting, and wiring, and furthermore reduces the space required as well as interference with the knitting pattern. We refrained from merging all three sensors together at this point, since we expected the behavior of a combined front-miss and knit could deviate too much from the evaluated versions and as a result our reference data could be rendered invalid.

We fed the sensor features into signal processing pipelines that we tuned specifically for each of the swatches. The underlying principle is simple and relies on mutual masking and exclusion. Table 1 shows a simplified process: assuming ideal and normalized input for the sf 1 swatch, multiplying the signals of knit D (k_D) and tuck B (t_B) yields a measure representing wale-directional strain, since both sensors respond to this actuation. Conversely, multiplying with a flipped signal of tuck B (1 – t_B) yields the opposite, which must represent course-directional strain, since t_B does not respond to this modality and therefore stays at 0. Table 2 illustrates how this concept can be extended to a third mode, by combining three sensors with distinct response profiles. Since in practice, the sensor readings are not as distinct and ideal, we applied scaling, clamping, and thresholding to achieve or estimate the required masks. Most essential is however to chose sensor types that provide response features that are adequately distinct in order to minimize false-positives. Serving as an example implementation, processing details for the sensor fusion swatches presented in this paper can be found in the supplement. Figure 9 (right) shows a demonstration of all three actuation modes.

In this section, we presented swatches that joined two sensors to infer two modes, as well as three sensors to infer three modes. However, we expect that by join an ideally complementary pair of sensors, it should be possible to infer all three actuation types from just two sensors. We can see in Figure 8 that knit B and tuck B could be promising candidates for such a combination, since k_B responds to strain along wales, t_B responds to strain along courses, and both respond to pressure. This would further downscale the required sensing area and reduce manufacturing and wiring complexity. We did implement this prototypically, and already gained first promising results, however, they were not robust enough the be included in the paper.

One of our objectives was to increase textile design latitude by providing sensor systems that can be concealed within knitted structures and are therefore unobtrusive or even entirely unrecognizable. The focus is in increasing visual and haptic appeal while providing full functionality for preserving UI expressiveness. As discussed, we achieve this by reducing the component count, since all required materials are merged into one wire. In the following, we present a basic demonstration of integration, using sensor made merely of knit stitches (corresponding to Plain knit or Interlock knit A of Section 4), which is arguably the variant most portable among numerous knitting patterns.

We showed a novel method of fabricating low-scale knitted force sensors using a copper wire with special piezoresistive enamel. By harnessing knit mesh geometry, we are able to distinguish between different types of actuation and to infer continuous values representing wale- and course-directional strain, as well as orthogonal pressure. This ability to track multiple degrees of freedom on a fabric surface’s state makes them ideal for integration in fabric-based human interface devices, such as gloves, garments, seats, etc. Moreover, by merging all materials that are required for FSRs into one and therefore reducing complexity in knit structure and material composition, we are able to conceal our sensors in numerous appealing knitting patterns, since our solution requires only a single intersection of a pair of wires to produce functional force-sensing cells. Being fabricated on a computerized V-bed weft knitting machine, and in combination with our streamlined knit generation pipeline, our devices can be created quickly and ready-made, without requiring noteworthy manual postprocessing, finishing, or layering of multiple fabrics.

For next steps, we plan to closer investigate additional intersection types and approach the idea with an additional theoretical angle, considering loop-design based on rope mechanics and estimations based on computational physics simulation. Our goal is to provide a number of atomic and universal functional building-blocks, similar to those of [12] that could be easily placed as required. However, due to the versatility of V-bed knitting and the variety and individuality of knitted fabrics, we expect those will still have to be specially adapted for each scenario.

In terms of utilization, we plan to integrate our sensors into functional fully-fashioned 3D-knits, e.g., for specialized furnishing and garments, such as seats, gloves, etc. We will optimize the physical features of the substrate knits by complementing with additional material such as Spandex, Nylon, and melting yarn to achieve targeted improvement in elasticity and stability. For data processing, we plan to utilize ML techniques for more reliable tracking of the surfaces’ geometric state.

This research is part of the COMET project TextileUX (No. 865791, which is funded within the framework of COMET – Competence Centers for Excellent Technologies by BMVIT, BMDW, and the State of Upper Austria. The COMET program is handled by the FFG.

In yarn-based textiles, one distinction is to be made in particular between weaves and knits that is relevant to this work: in weaves, the warp and weft strands are going straight and the resulting fabric is usually cut and sewn. Knitted textiles on the other hand consist of yarn that is forming loops and traverses throughout the structure (cf. Figure 16 a). These loops provide slack and therefore inherent stretchability, a property that renders knits the ideal textile candidates for sensing strain, however they can also be used for sensing contact pressure.

Weft knitting, and in particular V-bed knitting, provides many opportunities for thoroughly designing and engineering geometries and yarn interaction locally and on loop-level. This can be a powerful feature for sensor design, especially since in V-bed knitting, two knit faces can be combined arbitrarily and loops can be transferred between needle beds, both of which are features we extensively utilize in our work. Note that this also implies that most of the knits presented in this paper cannot be easily reproduced using hand-knitting or single-bed machines.

In the following, we explain knitting-related terms that are required to comprehend the concepts discussed in this paper. However, due to space limitations we confine this list to the essentials. For more details, we refer to literature covering the field of knitting technology comprehensively, such as [45].