Human Factors and Applied Cognition
PsychologyThe College of Humanities and Social Sciences

Social Robotics & Embodied Cognition Lab


The Social Robotics and Embodied Cognition Lab focuses on research in social attention and embodied cognition and its application to Social Robotics and Design Thinking. With regard to Social Robotics, the goal is to unravel what sort of information humans use when judging the degree of intentionality underlying the actions of social agents (i.e., robots) and how attributing a mind to others influences attention, perception and performance. With regard to Design Thinking, the SREC lab is interested in the role of embodied cognition involved in designing and in particular how perception and action processes interact during design thinking.  In order to investigate these questions, we use behavioral measures, eye tracking and EEG.

Spatial attention and Social Robotics

The Effects of mind attribution on social attention processes in Human-Robot Interaction

Gaze direction is one of the most fundamental social cues used in everyday interactions

Importance of gaze direction

For engaging in interactions with other people we need to know who we are interacting with, and what others are going to do next. Based on this knowledge, which can be acquired directly through interacting with other people or indirectly by observing others interacting, we make inferences about their internal states, such as intentions, beliefs, and feelings. At the core of this mentalizing process is that our predictions about others’ behavior are not simply based on the current state of the world, but also on assumptions about their internal states. Thus, when interacting with others, perceptual processing of information in the environment has to be integrated with background knowledge we have about others or inferences we draw from observing their behavior.


Gaze direction can be used to communicate information about internal states, e.g. desires


One basic cognitive mechanism employed in social interactions is attending to eye gaze. Gaze direction is very informative, as it indicates another’s focus of interest and encourages the observer to shift attention to the gazed-at location. However, gaze direction is usually not processed in isolation, but relies on additional information in order to be interpreted in a meaningful way. In our research, we showed that attentional orienting to gaze direction is modulated by context information that informs about the social relevance of observed gaze behavior. In particular, we showed that information about the trustworthiness of an agent as well as information about its internal states (i.e., facial expression) influences how humans deploy their attentional resources when observing changes in gaze directions. Furthermore, humans tend to engage more attentional resources in social interactions if they have information available about the human-likeness and/or degree to which observed behavior can be interpreted in an intentional way: The more likely it is that observed behavior originates from an intentional agent, the more attentional resources are deployed to the agent and the objects the agent is currently looking at. Interestingly, this effect is independent of the physical appearance of the agent but only depends on whether or not a mind can be attributed to it.

Social Robot Head by Meka Robotics

The goal of the SREC lab is to identify observable correlates of intentionality by using a social robot (from Meka Robotics, see pictures and video). In particular, we are interested in under which conditions the gaze direction of robots is interpreted in an intentional compared to a mechanistic way. Another question is whether attributing a mind to the robot influences performance in joint action tasks with the robot.

Gaze following, mentalizing and autism

How fundamental interpretation of gaze direction together with context information is in understanding others becomes obvious in patients who have reduced abilities to select and parse socially relevant information in social environments, for instance in Autism Spectrum Disorder (ASD). People with ASD have reduced skills in interpreting others’ gaze direction in a meaningful way and further exhibit reduced mentalizing skills: that is, they have difficulties inferring mental states from observed behavior. As attending to gaze and being able to infer mental states have been shown to reciprocally impact each other in healthy individuals, the question arises of how difficulties in the social domain are actually caused in autism: Are both the abilities to follow gaze direction and to mentalize about internal states impaired or does the inability to mentalize cause problems in following gaze? Finding answers to this question is particularly important with regard to training programs for people with autism: if impairments were specific to mentalizing processes, then remaining gaze processing skills could be used to increase the functionality of the whole mentalizing network.

Stimuli used in research on the effects of attributing a mind on social attention


The goal of the SREC lab is to investigate the causal relationship between gaze following and mentalizing in autism – the hypothesis being that people with autism can follow the gaze of others when they have the possibility to explain gaze behavior in a meaningful way. In other words, we assume that people with ASD do not follow human gaze because they are unable to explain the observed behavior in a meaningful way. However, given that people with ASD have increased systemizing skills, we predict that they would be able to follow the gaze of a mechanistic system (i.e., robot), whose behavior can be explained by analytical reasoning based on systematic rules (i.e., in mechanistic ways). We assume that they would follow the gaze of a robot and that their orientation to robot gaze could be used to train them on linking observed changes in gaze direction to making predictions about behavior. The outcome of this research is of interest for understanding the mechanisms underlying the impaired social skills in ASD and can be used to establish guidelines for training programs for patients with ASD.

Embodied Cognition and Design Thinking

Supporting design thinking by optimizing the interplay between external and internal problem representations

To develop products, designers need to employ sophisticated problem solving strategies while processing complex information. However, these complex requirements on problem-solving abilities are confronted with cognitive limitations resulting from restricted human working-memory capacity. Critically, in order to solve complex design problems, knowledge and problem-solving operators stored in long-term memory must be simultaneously activated and executed in working-memory. However, both retrieving and processing information rely on the same cognitive resources, thereby causing a trade-off between storage and execution in working-memory: that is, while a high load for storing information allows only for a low processing quality, information that is relevant for problem solving will be stored only provisory in working memory if processing makes too great demands on memory capacities. Accordingly, working memory represents the bottleneck of creative thinking and results in considerably decreased problem solutions if it is overloaded. In that case, engineers adapt their processing to the limited cognitive resources by working with approximate representations or solution strategies. They confine themselves to use already known strategies and are not capable of performing complex processes in parallel. Thus, working-memory must be relieved during problem solving, in order to foster creative design thinking processes. This intent can be managed by externalizing internal problem representations thereby saving mental capacities for complex problem solving.

3D sketching in virtual environments (by J.H. Israel, Berlin) 


The research on design thinking is based on the theory of embodied cognition. Following this approach, cognitive actions occur both internally and externally, thereby stretching a distributed representation space for design thinking. However, not much is known about how internal and external representations are used for inferring solution strategies and what is processed internally and what externally. Furthermore, the question of how external representations facilitate problem solving is discussed contradictorily in literature. Minimal memory theorists assume that people minimize the reliance on internal memory and cognitive processes whenever possible. In contrast, soft constraint theorists claim that internal processes are offloaded on external processes only if cognitive costs for external processing are lower than for internal processing. In consequence, the degree to which cognitive processing is performed internally reliably sheds light on how suitable external representations are for solving a certain type of problem.

Based on these considerations, suitable support systems for design thinking are needed in order to effectively assist creative designing (e.g. virtual environments). Their development needs to be based on the understanding of cognitive mechanisms and perceptual processes underlying design thinking. The most challenging question is how external representations need to be configured and how interactions with these representations should be implemented in order to efficiently combine internal (cognitive) and external (motor) actions. Designers use external representations to clarify the problem, to identify spatial relations between parts of the draft and to release working-memory capacity. External representations further serve as means for storing drafts, supporting idea generation and reflection and for evaluating design concepts. By integrating external representations into design thinking processes and actively interacting with them, designers offload internal, cognitive processes on external, perceptive and motor actions. Optimal design support systems should assist this externalization process by providing appropriate means to create and store external problem representations, as well as intuitive and easy to use tools to manipulate the representations. This leads to the question of how environments supporting design thinking need to be set up in order to assist both creative and analytical aspects of design thinking in an optimal fashion.

3D sketching tools for virtual environments (developed by C. Zoellner und J.H. Israel, Berlin)

The SREC lab focuses on increasing the quality of design solutions by optimizing the way designers use different support systems in order to flexibly switch between internal and external representations. The goal is to derive guidelines for constructing support systems that account for the cognitive and perceptual limitations of human problem-solvers.

Print Friendly and PDF