In this paper we review some problems with traditional approaches for acquiring and representing knowledge in the context of developing user interfaces. Methodological implications for knowledge engineering and for human-computer interaction are studied. It turns out that in order to achieve the goal of developing human-oriented (in contrast to technology-oriented) human-computer interfaces developers have to develop sound knowledge of the structure and the representational dynamics of the cognitive system which is (...) interacting with the computer.We show that in a first step it is necessary to study and investigate the different levels and forms of representation that are involved in the interaction processes between computers and human cognitive systems. Only if designers have achieved some understanding about these representational mechanisms, user interfaces enabling individual experiences and skill development can be designed. In this paper we review mechanisms and processes for knowledge representation on a conceptual, epistemological, and methodologieal level, and sketch some ways out of the identified dilemmas for cognitive modeling in the domain of human-computer interaction. (shrink)

In response to the ever-increasing spread of online disinformation and misinformation, several human–computer interaction tools to enhance data literacy have been developed. Among them, many employ elements of gamification to increase user engagement and reach out to a broader audience. However, there are no systematic criteria to analyze their relevance and impact for building fake news resilience, partly due to the lack of a common understanding of data literacy. In this paper we put forward an operationalizable definition (...) of data literacy as a form of multidimensional critical thinking. We then survey 22 existing tools and classify them according to a framework of 10 criteria pointing to their gameful design and educational features. Through a comparative/contrastive analysis informed by a focus group, we provide a principled set of guidelines to develop more efficient human–computer interaction tools to teach how to critically think in the current media ecosystem. (shrink)

The first Young Researchers in Human–Robot Interaction Workshop, held on March 1, 2006 in Salt Lake City, Utah, provides insight into how to facilitate the establishment of the HRI community. Organized in conjunction with the first annual ACM/IEEE Human Robot Interaction Conference, the NSF-sponsored workshop assembled 15 graduate students from 5 different countries in computer science, psychology, engineering, and the arts to build the HRI community. This report highlights recommendations from discussion sessions, a synopsis (...) of the plenary address, and representative examples of the participants’ presentations. Participants emphasized that HRI is a unique field, requiring knowledge in computing, psychology, and communications despite the differences in the courses, methods, and philosophies across disciplines. The following are needed for future growth in HRI: stable, canonical robotics platforms for research purposes, a multidisciplinary community infrastructure to connect researchers, and a “Berlitz phrasebook” and collected reference materials for helping understand the “other” disciplines. (shrink)

To effectively improve the efficiency of international service design talent training and make it more in line with society's needs, we analyze the current status of international service design talent training and its professional training focus. Based on the above problems, from the perspective of cognitive psychology, artificial intelligence and human–computer interaction technology are used to construct the international service design talent training mode of the HCI intelligent service robot. This mode can be used to solve the (...) existing teaching problems by using novel means to ensure the quality of teaching. Finally, through the actual analysis of teaching cases, the effectiveness of the proposed talent training mode is verified. The HCI system is based on knowledge of cognitive psychology. According to the characteristics and functions of an educational robot, the robot is combined with traditional teaching activities, and the robot-assisted talent training mode is designed. Robot-assisted talent training is a feasible training method that can improve the efficiency of talent training. Students have confidence in their learning skills before the course, and the confidence is further strengthened after the end of the course. After the course, the students have a stronger sense of cooperation. This study can provide theoretical ideas for the research of international service talent training mode. (shrink)

In this article, human–computer interaction is explored as a design-oriented practice nurturing the becoming of what is not-yet in future-oriented and speculative manners. Such approaches have evolved over time and now the field seems ready to take leaps targeting social and culturally infused contexts, such as those suggested by critical design, design things, adversarial design, making futures, pluriversal design and critical fabulations. It is in this respect that feminist theories, methods and imaginaries are rendered important. Feminist theory (...) is in this article considered an important companion and part of the practical tool-kit necessary for generative, speculative and ethical approaches within the field of HCI. How to think with care is explored as a meta-design strategy directed and informed by feminist onto-epistemologies – a strategy intended to ‘seed’ speculative and social justice-oriented design endeavours through generative figurations and critical dilemmas to foster abilities and sensibilities for dealing with difference differently. What is advanced is the need for meta-design space in HCI, in this article referred to as a contact zone, a feminist figuration with the intention to open up for design explorations with ethical imperatives. Four other interrelated feminist figurations are also loosely explored in order to frame how thinking with care in HCI could be advanced further, i.e. diffractive thinking, intra-activism, becoming-with and response-ability. By considering serious feminist accounts of situated knowledges and touching visions, it is argued that feminist thinking is well on its way to offering real alternatives of great importance for HCI. (shrink)

We discuss here one of our projects, aimed at developing an automatic facial expression interpreter, mainly in terms of signaled emotions. We present some of the relevant findings on facial expressions from cognitive science and psychology that can be understood by and be useful to researchers in Human-Computer Interaction and Artificial Intelligence. We then give an overview of HCI applications involving automated facial expression recognition, we survey some of the latest progresses in this area reached by various (...) approaches in computer vision, and we describe the design of our facial expression recognizer. We also give some background knowledge about our motivation for understanding facial expressions and we propose an architecture for a multimodal intelligent interface capable of recognizing and adapting to computer users¿ affective states. Finally, we discuss current interdisciplinary issues and research questions which will need to be addressed for further progress to be made in the promising area of computational facial expression recognition. (shrink)

This paper examines Harold Garfinkel’s work with ELIZA and a related program LYRIC from 1967 to 1969. AI researchers have tended to treat successful human–machine interaction as if it relied primarily on non-human machine characteristics, and thus the often-reported attribution of human-like qualities to communication with computers has been criticized as a misperception—and humans who make such reports referred to as “deluded.” By contrast Garfinkel, building on two decades of prior research on information and communication, argued (...) that the ELIZA and the LYRIC “chatbots” were achieving interactions that felt human to many users by exploiting human sense-making practices. In keeping with his long-term practice of using “trouble” as a way of discovering the taken-for-granted practices of human sense-making, Garfinkel designed scripts for ELIZA and LYRIC that he could disrupt in order to reveal how their success depended on human social practices. Hence, the announcement “Machine Down” by the chatbot was a desired result of Garfinkel’s interactions with it. This early (but largely unknown) research has implications not only for understanding contemporary AI chatbots, but also opens possibilities for respecifying current information systems design and computational practices to provide for the design of more flexible information objects. (shrink)

Traditional human–computer interaction (HCI) allowed researchers and practitioners to share and rely on the ‘five E’s’ of usability, the principle that interactive systems should be designed to be effective, efficient, engaging, error tolerant, and easy to learn. A recent trend in HCI, however, is that academic researchers as well as practitioners are becoming increasingly interested in user experiences, i.e., understanding and designing for relationships between users and artifacts that are for instance affective, engaging, fun, playable, sociable, creative, (...) involving, meaningful, exciting, ambiguous, and curious. In this paper, it is argued that built into this shift in perspective there is a concurrent shift in accountability that is drawing attention to a number of ethical, moral, social, cultural, and political issues that have been traditionally de-emphasized in a field of research guided by usability concerns. Not surprisingly, this shift in accountability has also received scarce attention in HCI. To be able to find any answers to the question of what makes a good user experience, the field of HCI needs to develop a philosophy of technology. One building block for such a philosophy of technology in HCI is presented. Albert Borgmann argues that we need to be cautious and rethink the relationship as well as the often-assumed correspondence between what we consider useful and what we think of as good in technology. This junction—that some technologies may be both useful and good, while some technologies that are useful for some purposes might also be harmful, less good, in a broader context—is at the heart of Borgmann’s understanding of technology. Borgmann’s notion of the device paradigm is a valuable contribution to HCI as it points out that we are increasingly experiencing the world with, through, and by information technologies and that most of these technologies tend to be designed to provide commodities that effortlessly grant our wishes without demanding anything in return, such as patience, skills, or effort. This paper argues that Borgmann’s work is relevant and makes a valuable contribution to HCI in at least two ways: first, as a different way of seeing that raises important social, cultural, ethical, and moral issues from which contemporary HCI cannot escape; and second, as providing guidance as to how specific values might be incorporated into the design of interactive systems that foster engagement with reality. (shrink)

This critical analysis is to addresses the implications about emerging design approaches in Human Computing Interfaces (HCI), and Human Interface Devices (HID). A reflection about custom-built interfaces invigorates a wider discussion about the meaningful contexts in which their use is activated. The specific aim is to re-imagine, redefine and explore the potentiality and limitations of electronic performance tools, namely, how the choice of this tool and interface nearly always gives rise to new situations that must be tackled. Therefore, (...) addressing the material aesthetics of performance tools used in contemporary electronic performance, by artists who engage with such technologies, this article aims to critically analyse and historically place artistic engagement with tools and interfaces in contemporary performance settings and to indicate where there is room for new design approaches and hence new (and forgotten) modes of engagement to unfold. To amplify the relationship between performer and the spectator when using the emerging technologies of real-time performance tools, I refer to ‘Ode to Paraphernalia’, a set of self-crafted electronic performance tools and a performance. This project opens a pathway for a larger proposal that asks: what are the ways in which we can engineer interfaces that validate the circulation of subjugated knowledges in meaningful sets and settings? (shrink)

This reader collects and introduces important work in linguistics, computer science, artificial intelligence, and computational linguistics on the use of linguistic devices in natural languages to situate events in time: whether they are past, present, or future; whether they are real or hypothetical; when an event might have occurred, and how long it could have lasted. In focussing on the treatment and retrieval of time-based information it seeks to lay the foundation for temporally-aware natural language computer processing systems, (...) for example those that process documents on the worldwide web to answer questions or produce summaries. The development of such systems requires the application of technical knowledge from many different disciplines. The book is the first to bring these disciplines together, by means of classic and contemporary papers in four areas: tense, aspect, and event structure; temporal reasoning; the temporal structure of natural language discourse; and temporal annotation. Clear, self-contained editorial introductions to each area provide the necessary technical background for the non-specialist, explaining the underlying connections across disciplines.A wide range of students and professionals in academia and industry will value this book as an introduction and guide to a new and vital technology. The former include researchers, students, and teachers of natural language processing, linguistics, artificial intelligence, computational linguistics, computer science, information retrieval, library sciences, human-computer interaction, and cognitive science. Those in industry include corporate managers and researchers, software product developers, and engineers in information-intensive companies, such as on-line database and web-service providers. (shrink)

It has been argued that the Internet and social media increase the number of available viewpoints, perspectives, ideas and opinions available, leading to a very diverse pool of information. However, critics have argued that algorithms used by search engines, social networking platforms and other large online intermediaries actually decrease information diversity by forming so-called “filter bubbles”. This may form a serious threat to our democracies. In response to this threat others have developed algorithms and digital tools to combat filter bubbles. (...) This paper first provides examples of different software designs that try to break filter bubbles. Secondly, we show how norms required by two democracy models dominate the tools that are developed to fight the filter bubbles, while norms of other models are completely missing in the tools. The paper in conclusion argues that democracy itself is a contested concept and points to a variety of norms. Designers of diversity enhancing tools must thus be exposed to diverse conceptions of democracy. (shrink)

The appropriate methodology for psychological research depends on whether one is studying mental algorithms or their implementation. Mental algorithms are abstract specifications of the steps taken by procedures that run in the mind. Implementational issues concern the speed and reliability of these procedures. The algorithmic level can be explored only by studying across-task variation. This contrasts with psychology's dominant methodology of looking for within-task generalities, which is appropriate only for studying implementational issues.The implementation-algorithm distinction is related to a number of (...) other “levels” considered in cognitive science. Its realization in Anderson's ACT theory of cognition is discussed. Research at the algorithmic level is more promising because it is hard to make further fundamental scientific progress at the implementational level with the methodologies available. Protocol data, which are appropriate only for algorithm-level theories, provide a richer source than data at the implementational level. Research at the algorithmic level will also yield more insight into fundamental properties of human knowledge because it is the level at which significant learning transitions are defined.The best way to study the algorithmic level is to look for differential learning outcomes in pedagogical experiments that manipulate instructional experience. This provides control and prediction in realistically complex learning situations. The intelligent tutoring paradigm provides a particularly fruitful way to implement such experiments.The implications of this analysis for the issue of modularity of mind, the status of language, research on human/computer interaction, and connectionist models are also examined. (shrink)

The revolution of artificial intelligence (AI), particularly generative AI, and its implications for human–robot interaction (HRI) opened up the debate on crucial regulatory, business, societal, and ethical considerations. This paper explores essential issues from the anthropomorphic perspective, examining the complex interplay between humans and AI models in societal and corporate contexts. We provided a comprehensive review of existing literature on HRI, with a special emphasis on the impact of generative models such as ChatGPT. The scientometric study posits that (...) due to their advanced linguistic capabilities and ability to mimic human-like behavior, generative AIs like ChatGPT will continue to grow in popularity in pair with human rational empathy, tendency for personification and their advanced linguistic capabilities and ability to mimic human-like behavior. As they blur the boundaries between humans and robots, these models introduce fresh moral and philosophical dilemmas. Our research aims to extrapolate key trends and unique factors in HRI and to elucidate the technical aspects of generative AI that enhance its effectiveness in this field compared to traditional rule-based AI systems. We further discuss the challenges and limitations of applying generative AI in HRI, providing a future research agenda for AI optimization in diverse applications, including education, entertainment, and healthcare. (shrink)

Using anthropological theory, this paper examines human–android interactions (HAI) as an emerging aspect of android science. These interactions are described in terms of adaptive learning (which is largely subconscious). This article is based on the observations reported and supplementary data from two studies that took place in Japan with a teleoperated android robot called Telenoid in the socialization of school children and older adults. We argue that interacting with androids brings about a special context, an interval, and a space/time (...) for reflection and imagination that was not there before. During the interaction something happens. There is adaptive learning and as a result, both children and older adults accepted Telenoid, and the children and older adults accepted each other. Using frames of play and ritual, we make sense and ‘capture’ moments of adaptive learning, and the feedback that elicits a social response from all study participants that results in self-efficacy and socialization. While “ritual” refers to the application of what has been learned and “play” means that there are no obvious consequences of what has been learned. This analysis illuminates new understanding about the uncanny valley, cultural robotics and the therapeutic potential of HAI. This has implications for the acceptance of androids in ‘socialized roles’ and gives us insight into the subconscious adaptive learning processes that must take place within humans to accept androids into our society. This approach aims to provides a clearer conceptual basis and vocabulary for further research of android and humanoid development. (shrink)

Constraints and dependencies among the elements of embodied cognition form patterns or microstrategies of interactive behavior. Hard constraints determine which microstrategies are possible. Soft constraints determine which of the possible microstrategies are most likely to be selected. When selection is non‐deliberate or automatic the least effort microstrategy is chosen. In calculating the effort required to execute a microstrategy each of the three types of operations, memory retrieval, perception, and action, are given equal weight; that is, perceptual‐motor activity does not have (...) a privileged status with respect to memory. Soft constraints can work contrary to the designer's intentions by making the access of perfect knowledge in‐the‐world more effortful than the access of imperfect knowledge in‐the‐head. These implications of soft constraints are tested in two experiments. In experiment 1 we varied the perceptual‐motor effort of accessing knowledge in‐the‐world as well as the effort of retrieving items from memory. In experiment 2 we replicated one of the experiment 1 conditions to collect eye movement data. The results suggest that milliseconds matter. Soft constraints lead to a reliance on knowledge in‐the‐head even when the absolute difference in perceptual‐motor versus memory retrieval effort is small, and even when relying on memory leads to a higher error rate and lower performance. We discuss the implications of soft constraints for routine interactive behavior, accounts of embodied cognition, and tool and interface design. (shrink)

Artificial intelligence researchers interested in knowledge and in designing and implementing digitized artifacts for representing or sharing knowledge play a crucial role in the development of a knowledge-based economy. They help answer the question of how the computer devices they develop can be appropriated by the collectives that manage the flow of knowledge and the know-how underlying human organizations. A dialogical, constructivist view of interaction processes permits theorizing the role of digital tools, seen (...) as sociotechnical devices that serve both as resources and as sources of communication within our organizations. (shrink)

In this paper, we address the question of what current computers are from the point of view of human-computer interaction. In the early days of computing, the Turing machine (TM) has been the cornerstone of the understanding of computers. The TM defines what can be computed and how computation can be carried out. However, in the last decades, computers have evolved and increasingly become interactive systems, reacting in real-time to external events in an ongoing loop. We argue (...) that the TM does not provide a mechanistic explanation for interactive computing. The reason is that the fundamental phenomena relevant to interactive computing are out of the scope of classical computability theory. Part of the explanatory power of the TM relies on what we propose to call an execution model. An execution model belongs to a level of abstraction where it is possible to describe both the functional architecture and the execution in mechanistic terms. An updated execution model is warranted to provide the minimal mechanistic description for interactive computation as a counterpart of what the TM could explain regarding Church-Turing computation. It would support an explanation of the ubiquitous computing devices we know - those interacting with humans, e.g., through digital interfaces. We show that such a model is not available within interactive models of computation and that relevant abstractions and concerns are available in computer engineering but need to be identified and gathered. To fill this void, we propose to reflect on the level of abstraction required to support the mechanistic description of an interactive execution and propose some preliminary requirements. (shrink)

Synthetic biology, as an engineering approach to biological systems, has the potential to disruptively innovate the development of vaccines, therapeutics, and diagnostics. Data accessibility and differences in data-usage capabilities are important factors in shaping this innovation landscape. In this paper, the data that underpin synthetic biology responses to the COVID-19 pandemic are analyzed as positional information goods—goods whose value depends on exclusivity. The positionality of biological data impacts the ability to guide innovations toward societally preferred goals. From both an (...) ethical and economic point of view, positionality can lead to suboptimal as well as beneficial situations. When aiming for responsible innovation (i.e. embedding societal deliberation in the innovation process), it is important to consider hurdles and facilitators in data access and use. Central governance and knowledge commons provide routes to mitigate the negative effects of data positionality. (shrink)

People meeting a robot for the first time do not know what it is capable of and therefore how to interact with it—what actions to produce, and how to produce them. Despite social robotics’ long-standing interest in the effects of robots’ appearance and conduct on users, and efforts to identify factors likely to improve human–robot interaction, little attention has been paid to how participants evaluate their robotic partner in the unfolding of actual interactions. This paper draws from qualitative (...) analyses of video-recorded interactions between a robot and groups of participants, in the framework of ethnomethodology and conversation analysis. We analyse the particular ways in which participants shape their embodied actions, how they can reproduce a prior action that failed to obtain a response from the robot; and how they explore the robot’s embodied nature. We find a set of recurrent methods or practices, showing that robot-recipient design displays not only participants’ initial assumptions about the robot’s competences, but also more importantly perhaps their continuous assessment of the robot’s behaviour, and their attempts to adapt to it. Participants locally produce and constantly revise their understanding of the robot as a more or less competent co-participant, drawing from its past, current, and projected conduct and responsiveness. We discuss the implications of these findings for research in robotics and human–robot interactions, and the value of the approach to shed new light on old questions by paying attention to the quality of gesture and the sequential organisation of interaction. (shrink)

Interactions between an intelligent software agent and a human user are ubiquitous in everyday situations such as access to information, entertainment, and purchases. In such interactions, the ISA mediates the user’s access to the content, or controls some other aspect of the user experience, and is not designed to be neutral about outcomes of user choices. Like human users, ISAs are driven by goals, make autonomous decisions, and can learn from experience. Using ideas from bounded rationality, we frame (...) these interactions as instances of an ISA whose reward depends on actions performed by the user. Such agents benefit by steering the user’s behaviour towards outcomes that maximise the ISA’s utility, which may or may not be aligned with that of the user. Video games, news recommendation aggregation engines, and fitness trackers can all be instances of this general case. Our analysis facilitates distinguishing various subcases of interaction, as well as second-order effects that might include the possibility for adaptive interfaces to induce behavioural addiction, and/or change in user belief. We present these types of interaction within a conceptual framework, and review current examples of persuasive technologies and the issues that arise from their use. We argue that the nature of the feedback commonly used by learning agents to update their models and subsequent decisions could steer the behaviour of human users away from what benefits them, and in a direction that can undermine autonomy and cause further disparity between actions and goals as exemplified by addictive and compulsive behaviour. We discuss some of the ethical, social and legal implications of this technology and argue that it can sometimes exploit and reinforce weaknesses in human beings. (shrink)

The first Young Researchers in Human–Robot Interaction Workshop, held on March 1, 2006 in Salt Lake City, Utah, provides insight into how to facilitate the establishment of the HRI community. Organized in conjunction with the first annual ACM/IEEE Human Robot Interaction Conference, the NSF-sponsored workshop assembled 15 graduate students from 5 different countries in computer science, psychology, engineering, and the arts to build the HRI community. This report highlights recommendations from discussion sessions, a synopsis (...) of the plenary address, and representative examples of the participants’ presentations. Participants emphasized that HRI is a unique field, requiring knowledge in computing, psychology, and communications despite the differences in the courses, methods, and philosophies across disciplines. The following are needed for future growth in HRI: stable, canonical robotics platforms for research purposes, a multidisciplinary community infrastructure to connect researchers, and a “Berlitz phrasebook” and collected reference materials for helping understand the “other” disciplines. (shrink)

Robotization and artificial intelligence are expected to change societies profoundly. Trust is an important factor of human–technology interactions, as robots and AI increasingly contribute to tasks previously handled by humans. Currently, there is a need for studies investigating trust toward AI and robots, especially in first-encounter meetings. This article reports findings from a study investigating trust toward robots and AI in an online trust game experiment. The trust game manipulated the hypothetical opponents that were described as either AI or (...) robots. These were compared with control group opponents using only a human name or a nickname. Participants lived in the United States. Describing opponents with robots or AI did not impact participants’ trust toward them. The robot called jdrx894 was the most trusted opponent. Opponents named “jdrx894” were trusted more than opponents called “Michael.” Further analysis showed that having a degree in technology or engineering, exposure to robots online and robot use self-efficacy predicted higher trust toward robots and AI. Out of Big Five personality characteristics, openness to experience predicted higher trust, and conscientiousness predicted lower trust. Results suggest trust on robots and AI is contextual and it is also dependent on individual differences and knowledge on technology. (shrink)

The hypergrowth and hyperconnectivity of networks of artificial intelligence (AI) systems and algorithms increasingly cause our interactions with the world, socially and environmentally, more technologically mediated. AI systems start interfering with our choices or making decisions on our behalf: what we see, what we buy, which contents or foods we consume, where we travel to, who we hire, etc. It is imperative to understand the dynamics of human-computer interaction in the age of progressively more competent AI. This (...) essay presents five premises to serve as starting points for building our understanding of the “interactions’ in human-computer interactions. (shrink)

Bringing human–robot interaction (HRI) into conversation with scholarship from human geography, this paper considers how socially interactive robots become important agents in the production of social space and explores the utility of core geographic concepts of _scale_ and _place_ to critically examine evolving robotic spatialities. The paper grounds this discussion through reflections on a collaborative, interdisciplinary research project studying the development and deployment of interactive museum tour-guiding robots on a North American university campus. The project is a (...) collaboration among geographers, roboticists, a digital artist, and the directors/curators of two museums, and involves experimentation in the development of a tour-guiding robot with a “socially aware navigation system” alongside ongoing critical reflection into the socio-spatial context of human–robot interactions and their future possibilities. The paper reflects on the tensions between logics of control and contingency in robotic spatiality and argues that concepts of scale and place can help reflect on this tension in a productive way while calling attention to a broader range of stakeholders who should be included in robotic design and deployment. (shrink)

The Accreditation Board of Engineering and Technology (ABET) has declared that to achieve accredited status, “engineering programs must demonstrate that their graduates have an understanding of professional and ethical responsibility.” Many engineering professors struggle to integrate this required ethics instruction in technical classes and projects because of the lack of a formalized ethics-in-design approach. However, one methodology developed in human-computer interaction research, the Value-Sensitive Design approach, can serve as an engineering education tool which (...) bridges the gap between design and ethics for many engineering disciplines. The three major components of Value-Sensitive Design, conceptual, technical, and empirical, exemplified through a case study which focuses on the development of a command and control supervisory interface for a military cruise missile. (shrink)

Sometimes in the course of image-making, images are asked to represent unusual things. Around 1960, scientists and engineers working on the Computer-Aided Design Project at MIT began imagining that computers could be “active partners” to human designers. They began talking about a future of “human-computer symbiosis.” And they created a new type of image—the screenshot—that represented this new possibility. This paper describes early CAD research as a site for the emergence of the ideal of the interactive (...) computer and how this ideal was described and distributed through screenshots.Though we now routinely associate computers with interactivity, interactivity was beyond the average user’s experience in 1960.... (shrink)

Computational modeling of sketch understanding is interesting both scientifically and for creating systems that interact with people more naturally. Scientifically, understanding sketches requires modeling aspects of visual processing, spatial representations, and conceptual knowledge in an integrated way. Software that can understand sketches is starting to be used in classrooms, and it could have a potentially revolutionary impact as the models and technologies become more advanced. This paper looks at one such effort, Sketch Worksheets, which have been used in multiple (...) classroom experiments already, with students ranging from elementary school to college. Sketch Worksheets are a software equivalent of pencil and paper worksheets commonly found in classrooms, but they provide on-the-spot feedback based on what students draw. They are built on the CogSketch platform, which provides qualitative visual and spatial representations and analogical processing based on computational models of human cognition. This paper explores three issues. First, we examine how research from cognitive science and artificial intelligence, combined with the constraints of creating new kinds of educational software, led to the representations and processing in CogSketch. Second, we examine how these capabilities have been used in Sketch Worksheets, drawing upon experiments with fifth-grade students in biology and college students in engineering design and in geoscience. Finally, we examine some open issues in sketch understanding that need to be addressed to better model high-level aspects of vision, and for sketch understanding systems to reach their full potential for supporting education. (shrink)

RESUMO A série Black Mirror, transmitida entre 2011 e 2023 pela Netflix, tornou-se um fenômeno de mídia e seus episódios mostraram formas de interação homem-máquina (terminologia também referida como humano-computador). O nome da série se refere ao fato de que, quando uma tela é desligada, ela se torna um espelho negro que reflete a imagem do usuário. Este artigo1 tem como objetivo analisar os efeitos da interação homem-máquina nas organizações produtivas apresentadas em Black Mirror. Esta pesquisa utilizou a análise do (...) discurso, principalmente a partir das contribuições de Mikhail Bakhtin. O método consistiu em três etapas que contaram com a colaboração de estudantes de graduação e pós-graduação em Engenharia de Produção, integrando ensino e pesquisa e desenvolvendo o senso crítico dos pesquisadores-iniciantes. Os resultados indicam que, nas organizações apresentadas na série Black Mirror, há uma transição do conceito de fatores humanos para fatores ciborgues, bem como o uso da interação homem-máquina para promover o controle social. (shrink)

The World Wide Web has had a notable impact on a variety of epistemically-relevant activities, many of which lie at the heart of the discipline of knowledge engineering. Systems like Wikipedia, for example, have altered our views regarding the acquisition of knowledge, while citizen science systems such as Galaxy Zoo have arguably transformed our approach to knowledge discovery. Other Web-based systems have highlighted the ways in which the human social environment can be used to support (...) the development of intelligent systems, either by contributing to the provision of epistemic resources or by helping to shape the profile of machine learning. In the present paper, such systems are referred to as ‘knowledge machines’. In addition to providing an overview of the knowledge machine concept, the present paper reviews a number of issues that are associated with the scientific and philosophical study of knowledge machines. These include the potential impact of knowledge machines on the theory and practice of knowledge engineering, the role of social participation in the realization of intelligent systems, and the role of standardized, semantically-enriched data formats in supporting the ad hoc assembly of special-purpose knowledge systems and knowledge processing pipelines. (shrink)

African literature has played a major role in changing and shaping perceptions about African people and their way of life for the longest time. Unlike western cultures that are associated with advanced forms of writing, African literature is oral in nature, meaning it has to be recited and even performed. Although Africa has an old tribal culture, African philosophy is a new and strange idea among us. Although the problem of “universality” of African philosophy actually refers to the question of (...) whether Africa has heckling of philosophy in the Western sense, obviously, the philosophy bred by Africa’s native culture must be acknowledged. Therefore, the human–computer interaction-oriented method is proposed to appreciate African literature and African philosophy. To begin with, a physical object of tablet-aid is designed, and a depth camera is used to track the user’s hand and tablet-aid and then map them to the virtual scene, respectively. Then, a tactile redirection method is proposed to meet the user’s requirement of tactile consistency in head-mounted display virtual reality environment. Finally, electroencephalogram emotion recognition, based on multiscale convolution kernel convolutional neural networks, is proposed to appreciate the reflection of African philosophy in African literature. The experimental results show that the proposed method has a strong immersion and a good interactive experience in navigation, selection, and manipulation. The proposed HCI method is not only easy to use, but also improves the interaction efficiency and accuracy during appreciation. In addition, the simulation of EEG emotion recognition reveals that the accuracy of emotion classification in 33-channel is 90.63%, almost close to the accuracy of the whole channel, and the proposed algorithm outperforms three baselines with respect to classification accuracy. (shrink)

People may exhibit two kinds of modifications when demonstrating action for others: modifications to facilitate bottom‐up, or sensory‐based processing; and modifications to facilitate top‐down, or knowledge‐based processing. The current study examined actors' production of such modifications in action demonstrations for audiences that differed in their capacity for intentional reasoning. Actors' demonstrations of complex actions for a non‐anthropomorphic computer system and for people (adult and toddler) were compared. Evidence was found for greater highlighting of top‐down modifications in the demonstrations (...) for the human audiences versus the computer audience. Conversely, participants highlighted simple perceptual modifications for the computer audience, producing more punctuated and wider ranging motions. This study suggests that people consider differences in their audiences when demonstrating action. (shrink)

This ambitious book by one of the most original and provocative thinkers in science studies offers a sophisticated new understanding of the nature of scientific, mathematical, and engineering practice and the production of scientific knowledge. Andrew Pickering offers a new approach to the unpredictable nature of change in science, taking into account the extraordinary number of factors—social, technological, conceptual, and natural—that interact to affect the creation of scientific knowledge. In his view, machines, instruments, facts, theories, conceptual and (...) mathematical structures, disciplined practices, and human beings are in constantly shifting relationships with one another—"mangled" together in unforeseeable ways that are shaped by the contingencies of culture, time, and place. Situating material as well as human agency in their larger cultural context, Pickering uses case studies to show how this picture of the open, changeable nature of science advances a richer understanding of scientific work both past and present. Pickering examines in detail the building of the bubble chamber in particle physics, the search for the quark, the construction of the quarternion system in mathematics, and the introduction of computer-controlled machine tools in industry. He uses these examples to address the most basic elements of scientific practice—the development of experimental apparatus, the production of facts, the development of theory, and the interrelation of machines and social organization. (shrink)

What is ‘certainty’ in our everyday life and work? AI changing what it means to ‘be certain’. For example, when we engage with others and are doing our work, we make judgments in which we ‘trust our instinct’. Are we still able to do so if we engage with the ‘certainty’ of the machine? How is ‘being certain’ in ‘dialogue’ affected by our conceptions of what it means to be human in interaction with the ‘machine’, and consequent distinctions (...) between body and mind? Descartes first likened the human body to a machine (albeit a machine created by God) and endowed reason as being a transcendent universal instrument. La Mettrie proposed that language and culture separates humans from ‘animals’. However, in Diderot’s writing, ‘knowledge’ is seen as embodied, coming from our senses and language, and where gesture and rhythm evolve in practice. For D’Alembert, human ‘instinct’ is a skill, a learnt practice, and Rousseau believes that human certainty is something that cannot be measured. These questionings of the dichotomies of body and mind and human and animal, that relate practice and knowledge, also question the place and meaning of ‘rules’ of engaged practice. For Wittgenstein, one learns to make judgments through personal experience, ‘rules in practice’, whereby language is necessarily culturally situated and embodied, and calculation and Judgement are part of each other. (shrink)

By studying several cases of expert systems' use, a variety of difficulties were identified as directly depending on specific characteristics of experts and their tasks. This concerns more than the questions: “May experts be replaced by machines?” or “Is experts' knowledge explicable?”. The organisational structure of their work as well as the cyclic, non-plannable way of their task performing have further relevance. The paper introduces the concept of experts' systems to deal with diversities of their expertise and complexities of (...) their work. It draws a distinction between non-monotonic problem solving, exploration, medium and modification, and argues that these modes are not reducible to yet another improved input/output strategy or dialogue style but introduce additional functions supporting the human-computer interaction according to experts' needs. In the first few sections, the paper covers the theoretical and empirical results of our research, whereas Section 4 introduces our design suggestions for experts' systems. (shrink)

This article develops a critical theory of human–computer interaction intended to test some of the assumptions and omissions made in the field as it transitions from a cognitive theoretical frame to a phenomenological understanding of user experience described by Harrison et al. as a third research paradigm and similarly Bødker :24–31; Bødker, Interactions 22):24–31, 2015) as third-wave HCI. Although this particular focus on experience has provided some novel avenues of academic enquiry, this article draws attention to a (...) distinct bridge between the conventional HCI disciplinary concerns with predominantly task-based digital work and use context and a growing business interest in consumer experiences in digital environments. Critical HCI addresses the problem of experience in two interrelated ways. On one hand, it explores the role market logic plays in putting user experiences to work. On the other hand, it engages with ontological understandings of experience hitherto realized in HCI by way of a phenomenological matrix. The article concludes by bringing in an old thinker to consider experience in novel ways that relate ontological concerns to a broader political concept of experience capitalism. (shrink)

In this short paper, we introduce our Special Section in Feminist Theory titled ‘Feminist human-computer interaction: Struggles for past, contemporary and futuristic feminist theories in digital innovation’. Over the last years, we worked with the authors of the articles presented herein to bring together feminist theories with their practical application in the design, development, use and exploration of digital technologies. Our section follows three aspects: an overview of past feminist histories and discourse; the development of actionable, contemporary (...) theory; and speculative futures of what a feminist human-computer interaction could be. Together with the contributing authors, we are excited to explore these areas of post-disciplinary connection at the intersection of theory, practice and activism. (shrink)

A sociological revision of Aron Gurwitsch provides a helpful layered theory of conscious experience as a four-domain structure: _the theme_, _the thematic field_, _the halo_, and _the social horizon_. The social horizon—the totality of the social world that is unknown, vaguely known, taken for granted, or ignored by the subject despite objectively influencing the thoughts and actions of the subject—, helps conceptualize how everyday human–computer interaction (HCI) can obscure social structures. Two examples illustrate the usefulness of this (...) framework: (1) illuminating new forms of social control known as “surveillance capitalism” that influence the other domains of consciousness despite being invisible to the subject and (2) explaining how computer use tends to “hide” the ecological impacts of digital-technological systems from the attention of subjects. A sociological four-domain theory of consciousness supplements ethnomethodological HCI research by preserving the field’s long-standing focus on micro-level interactions and user experience while simultaneously drawing attention to a “commonsense ignorance of social structure”—in this case, the invisibility of the social structures behind personal computer use. Social structure can be reproduced even when actors are unaware of engaging in its reproduction. (shrink)

Responsibility has become a central concept in AI ethics; however, little research has been conducted into practitioners’ personal understandings of responsibility in the context of AI, including how responsibility should be defined and who is responsible when something goes wrong. In this article, we present findings from a 2020–2021 data set of interviews with AI practitioners and tech workers at a single multinational technology company and interpret them through the lens of feminist political thought. We reimagine responsibility in the context (...) of AI development and deployment as the product of work cultures that enable tech workers to be responsive and answerable for their products over the long and short term. From our interviews, we identify three key pain points in understanding the distribution of responsibility between actors and developing responsible design and deployment practices: (1) unstable business ecosystems and AI lifecycles, which require an approach to responsibility that accounts for the dynamic nature of these systems; (2) the issue of incentivizing engineers to take responsibility for the mundane maintenance practices essential to the functioning of AI systems and (3) the need to overcome individual and structural barriers to taking ownership over AI products and their effects. From these findings, we make three recommendations based on feminist theory: (1) organisations should move from a static model of responsibility to a dynamic and ethically motivated response-ability; (2) companies need to revalue care and maintenance practices; and (3) firms must move away from individualistic ideas of responsibility towards fostering wider cultures of responsibility. (shrink)

Under the wave of globalization, the ties between countries are getting closer and closer. Based on the differences in the languages of different countries, the importance of English as a universal language is becoming more and more prominent. In the past, English teaching was mainly taught by teachers and students. This mode of English learning is more of theoretical teaching, which has little effect on improving English ability. In the era of intelligence, with the upgrading of technology and the renewal (...) of ideas, interactive English teaching is more accepted by people and becomes a new solution for English teaching. Studying English teaching from the perspective of social cognition, this paper proposes to realize human-computer interaction with the help of advanced science and technology, create an interactive teaching environment, and realize interactive learning in an English environment. This paper designed a control experiment of the experimental group and the control group to verify the impact of human-computer interaction teaching on students’ English performance and investigated students’ performance and teachers’ feelings and their related opinions on human-computer interaction learning. The experimental results of this paper show that human-computer interactive English teaching can help to enhance students’ interest in English learning and improve English expression significantly. (shrink)

To make the interface design of computer application system better, meet the psychological and emotional needs of users, and be more humanized, the emotional factor is increasingly valued by interface designers. In the design of human-computer interaction graphical interfaces, the designer attaches great importance to the emotional design of the interface, and enhances the humanized design of the interface, which cannot only improve the comfort of the interface, but also improve the fun of the interface, to (...) ensure the psychological and emotional needs of users can be better satisfied. It may acquire information that is favorable to innovative design by utilizing cluster analysis algorithm to tackle the problem of complicated cultural information, and then utilize cellular genetic algorithm to carry out creative design of cultural items. It increases the availability of cultural and creative goods. The classic cluster analysis technique offers the maximum data clustering effect of 53.3%, according to the findings of this paper’s experiments. While the improved cluster analysis algorithm has the highest data clustering effect of 90%. It can be seen that the improved cluster analysis algorithm can effectively perform cluster analysis on a large amount of data in cultural and creative products. It thus finds out the most suitable designer’s creative information, which helps designers create better products. (shrink)

In the course of consumer behavior, it is necessary to study the relationship between the characteristics of psychological activities and the laws of behavior when consumers acquire and use products or services. With the development of the Internet and mobile terminals, electronic commerce has become an important form of consumption for people. In order to conduct experiential education in E-commerce combined with consumer behavior, courses to understand consumer satisfaction. From the perspective of E-commerce companies, this study proposes to use artificial (...) intelligence image recognition technology to recognize and analyze consumer facial expressions. First, it analyzes the way of human–computer interaction in the context of E-commerce and obtains consumer satisfaction with the product through HCI technology. Then, a deep neural network is used to predict the psychological behavior and consumer psychology of consumers to realize personalized product recommendations. In the course education of consumer behavior, it helps to understand consumer satisfaction and make a reasonable design. The experimental results show that consumers are highly satisfied with the products recommended by the system, and the degree of sanctification reaches 93.2%. It is found that the DNN model can learn consumer behavior rules during evaluation, and its prediction effect is increased by 10% compared with the traditional model, which confirms the effectiveness of the recommendation system under the DNN model. This study provides a reference for consumer psychological behavior analysis based on HCI in the context of AI, which is of great significance to help understand consumer satisfaction in consumer behavior education in the context of E-commerce. (shrink)

Given the recent call to strengthen collaboration between researchers and relevant practitioners, we consider participatory design as a way to advance Cognitive Science. Building on examples from the Learning Sciences and Human−Computer Interaction, we (a) explore what, why, who, when, and where researchers can collaborate with community members in Cognitive Science research; (b) examine the ways in which participatory‐design research can benefit the field; and (c) share ideas to incorporate participatory design into existing basic and applied research (...) programs. Through this article, we hope to spark deeper discussions on how cognitive scientists can collaborate with community members to benefit both research and practice. (shrink)

The unprecedented development of medical informatics is constantly transforming the concept of expertise in medical sciences in a way that has far-reaching consequences for both the theory of knowledge and the philosophy of informatics. Deep medicine is based on the assumption that medical diagnosis should take into account the wide array of possible health factors involved in the diagnostic process, such as not only genome analysis alone, but also the metabolome (analysis of all body metabolites important for e.g. drug-drug (...) interactions), microbiome (i.e. analysis of all bodily microorganisms interacting with host cells) or exposome (analysis of all environmental factors triggering potentially harmful cell mutations, such as UV radiation, heavy metals, various carcinogens, etc.). Deep data analysis is of tantamount importance for personalized diagnosis but, at the same time, hardly achievable by a regular human being. However, adequately designed artificial intelligence (e.g. a deep neural network) can undeniably be of great help for finding correlations between symptoms and underlying diseases. Nowadays AI applies to nearly every aspect of medicine, starting from the data analysis of scientific literature, through the diagnostic process, to the act of communication between physicians and their patients. Medical image processing algorithms greatly enhance the chances of successful recognition of melanoma or intestinal polyps. Communication tools designed for physicians use techniques known from social media to provide users with an opportunity to consult the case with colleagues from the same discipline. Natural language processing tools significantly improve doctor-patient communication by the automation of note-taking. Is this enough to support the claim that the non-epistemic competences in medicine are becoming more and more important? Can we attribute expertise only to a person? How is medical informatics changing the way most people usually understand human-computer interactions? (shrink)

Time and technology permeate the fabric of teamwork across a variety of settings to affect outcomes which have a wide range of consequences. However, there is a limited understanding about the interplay between these factors for teams, especially as applied to artificial intelligence (AI) technology. With the increasing integration of AI into human teams, we need to understand how environmental factors such as time scarcity interact with AI technology to affect team behaviors. To address this gap in the literature, (...) we investigated the interaction between the availability of intelligent technology and time scarcity in teams. Drawing from the theoretical perspective of computers are social actors and extant research on the use of heuristics and human–AI interaction, this study uses behavioral data from 56 teams who participated in a between-subjects 2 (intelligent assistant available × control/no intelligent assistant) × 2 (time scarcity × control/no time scarcity) lab experiment. Results show that teams working under time scarcity used the intelligent assistant more often and underperformed on a creative task compared to teams without the temporal constraints. Further, teams who had an intelligent assistant available to them had fewer interactions between members compared to teams who did not have the technology. Implications for research and applications are discussed. (shrink)

The import of computational learning theories and techniques on the ethics of human-robot interaction is explored in the context of recent developments of personal robotics. An epistemological reflection enables one to isolate a variety of background hypotheses that are needed to achieve successful learning from experience in autonomous personal robots. The conjectural character of these background hypotheses brings out theoretical and practical limitations in our ability to predict and control the behaviour of learning robots in their interactions with (...) humans. Responsibility ascription problems, which concern damages caused by learning robot actions, are analyzed in the light of these epistemic limitations. Finally, a broad framework is outlined for ethically motivated scientific inquiries, which aim at improving our capability to understand, anticipate, and selectively cope with harmful errors by learning robots. (shrink)