Visual-Textual Integration: Emoji as a Supplement in Health Information Design

3-5439

Tingyi S. Lin 1 and Yue Luo 2, *

1 Department of Design, National Taiwan University of Science and Technology, Taiwan Tech, Taipei City, Taiwan
2 College of Fine Arts, Guangdong Polytechnic Normal University (GPNU), Guangzhou City, China

The growing popularity of emoji has drawn attention to social media interactions, including within the context of health information. This study explored and analyzed the usage patterns of emoji in relation to health information by arranging a focus group and conducting a design experiment. To this end, we created a micro-database of 69 health-themed emoji representing various parts of speech; specifically, they could function as nouns, verbs, and adjectives simultaneously. An information design experiment was conducted to investigate how emoji could be used as a visual language to assist in the design and communication of health information. Our analysis yielded several important findings, including (1) recommendations regarding the quantity, frequency, and placement of emoji in health information; (2) an examination of the denotative aspects of emoji–word integration in visual language; and (3) the strategic positioning of emoji within appropriate sentence components, with an emphasis on the judicious use of repeated emoji to create or enhance visual hierarchies. This exploratory study constitutes a first step in understanding the design application paradigms of emoji in health communication and provides nuanced theoretical insights into visual health communication as well as practical implications for information design.

Keywords – Emoji, Health Information, Usage Patterns, Visual Language, Multimodal, Information Design

Relevance to Design Practice – This study synthesizes emoji usage patterns in health communication contexts from an information design perspective. It aims to establish a health-emoji database to enhance understanding regarding the denotation of emoji–word usage in visual language and to investigate emoji–word relationships through an information architecture analysis. The study also provides valuable insights for information design in the context of collaborative interdisciplinary health efforts.

Citation: Lin, T. S., & Luo, Y. (2024). Visual-textual integration: Emoji as a supplement in health information design. International Journal of Design, 18(2), 37-58. https://doi.org/10.57698/v18i2.03

Received February 6, 2024; Accepted June 18, 2024; Published August 31, 2024.

Copyright: © 2024 Lin & Luo. Copyright for this article is retained by the authors, with first publication rights granted to the International Journal of Design. All journal content is open-accessed and allowed to be shared and adapted in accordance with the Creative Commons Attribution 4.0 International (CC BY 4.0) License.

*Corresponding Author: yueluo@gpnu.edu.cn

Tingyi S. Lin (MA ’98/MFA ‘99/PhD ’06) is an information designer, visual storyteller, and strategic planner. She has held roles as a visiting scholar at the University of Chicago (’17-’18) and as a faculty member at the National Taiwan University of Science and Technology (NTUST), in addition to serving as adjunct faculty at National Taiwan University. Her career focuses on pioneering projects at the nexus of cutting-edge technology and health promotion through visual translation. Dr. Lin has advanced the integration of design research and creative practice, applying research-based design methodologies and validating their effectiveness through practical application.

Yue Luo (PhD, NTUST) is a faculty member (lecturer) at the College of Fine Arts, Guangdong Polytechnic Normal University. His research focuses on visual health communication, science communication, information design, and book design. His works have been published in international peer-reviewed journals such as Telematics and Informatics and SSM-Population Health.

Introduction

Emoji are colorful graphic symbols with predefined names and codes that are used to represent a wide range of expressions. These ideograms have evolved beyond mere facial expressions and are increasingly used on digital platforms to represent a broad range of objects, emotions, and concepts. Their designs now encompass several thematic categories, including animals, plants, body parts, clothing, family, food, occupations, sports, and vehicles (Rodrigues et al., 2018). According to a survey conducted by Unicode, 92% of global internet users reported using emoji in 2021 (Unicode, 2022), confirming their relevance as an important communication tool.

Over the past two decades, academic researchers have increasingly investigated the use of emoji—particularly facial emoji—in multiple fields, exploring the emotional impact, facial features, and meanings of emoji. Recent research on emoji has extended beyond social media to include advertising and brand marketing (Abell et al., 2022; Das et al., 2019), food safety (Pinto et al., 2020; Ray & Merle, 2021; Schouteten et al., 2023), doctor-patient relationship management (Jaeger et al., 2018; Sick et al., 2020), and psychological assessment (Marengo et al., 2019; Van Dam et al., 2019). Emoji facilitate communication among users from different regions, and their visual nature and emotional resonance make them particularly effective in overcoming language barriers among users with different native tongues. Given the widespread adoption of emoji and their effectiveness in human communication, the potential application of emoji in health communication and education settings warrants further investigation.

During the past five years, health and medical topics have garnered increasing attention, with studies appearing during this period that examine the role of emoji in health communication (Choi et al., 2020; Lin & Luo, 2023a, 2023b; Ray & Merle, 2021; Taylor et al., 2022). However, no study has adequately explored how emoji function as a visual language (Lotfinejad et al., 2020), and despite the growing use of emoji in the health sciences, there remains a lack of design research focused on their application in health communication and health education. In addition, the annually updated Emojipedia database does not include a specific healthcare category, which likely further limits the use of emoji in health-themed contexts. Moreover, users’ interpretations of emoji may not always align with default or intended meanings, leading to differences in understanding among individuals (Jaeger et al., 2018; Miller et al., 2016; Rodrigues et al., 2018). Although some degree of flexibility in interpreting emoji is generally considered advantageous for communication, different interpretations can result in ambiguity and misunderstandings if used incorrectly. To fully understand how emoji function as a visual language in the context of health information design, a detailed exploration of their usage patterns in health communication is essential.

To investigate the role of emoji as a visual supplement in health communication, this study (1) reviewed and summarized existing studies addressing the use of emoji in this field, (2) selected and analyzed health-related emoji from existing databases, and (3) developed benchmarks for the effective use of emoji as a visual language in health communication. To achieve these objectives, this study conducted a focus group and an experiment to analyze health-themed emoji in terms of their placement, frequency and quantity of use, part of speech attributes, and integration within text. The results were progressively synthesized to obtain recommendations for the optimal use of emoji in health communication contexts. The research findings offer both theoretical insights and practical suggestions for health information design.

Literature Analysis and Narrative Review

Emoji in Health Communication Design

Emoji are among the most extensively employed nonverbal cues in global online communication and have been extensively studied from a scientific perspective in the field of computer-mediated communication (CMC) (Bai et al., 2019). Emoji serve two core communication functions: first, they help convey emotional tone, and second, they help to reduce ambiguity when combined with textual language (Hand et al., 2023; Kaye et al., 2016). Although emoji more commonly appear in cheerful and informal contexts (Derks et al., 2007; Rosen et al., 2010), our previous studies have demonstrated the positive effects of emoji when they are incorporated into the design and dissemination of health information, positively impacting public perception and behavioral intention (Lin & Luo, 2022, 2023a, 2023b).

Drawing on visual narrative (Megehee & Woodside, 2010), visual persuasion (Seo et al., 2013), and dual coding theories (Paivio, 2014), researchers have investigated how emoji can be used as a visual language to effectively persuade and educate in health settings. Such an approach is crucial given the complexity of health-related and medical terminology and the communication gap that exists between medical professionals and the general public, which often leads to communication failures. Due to the lack of effective design interventions to address these failures, along with inadequate public attention to and engagement with health-related information, the use of emoji as a visual language may be the best option to effectively overcome or even eliminate these difficulties (Marengo et al., 2019). Existing visual narrative and persuasion theories offer valuable insights for improving representation and addressing communication challenges (Halverson et al., 2023; King, 2015; Troiano & Nante, 2018). From a psycholinguistic viewpoint, symbols like emoji may constitute a visual language that enhances the emotional expression of messages (Szabó, 2019). Notably, Willoughby and Liu (2018) observed that readers often focus on emoji within text strings, regardless of the sentence structure. Therefore, the inclusion of emoji in health information could effectively simplify content and help to clarify the meanings of unfamiliar or uncommon terms. Building upon these findings, we conducted a systematic review of the current research on the use of emoji in fields related to health communication and public health to develop strategies and recommendations for their application as a visual language in these domains.

The systematic literature review and analysis in this study were conducted in two stages. First, we collected information from 99 journals and conference proceedings in the fields of public health, health communication, and information management, as well as selected studies in the humanities (details provided in Appendix A). Keywords including “emoji,” “health,” “healthcare,” “medicine,” “public health,” and “health information,” as well as their combinations were used for the literature search, with the term “emoji” being a necessary search criterion. Second, we excluded comments, letters, and editorial materials from the collected literature to ensure that the remaining materials consisted only of original research articles, reviews or overviews, and conference papers. Although the terms “emoji” and “emoticon” are frequently confused or used interchangeably in academic literature, they are distinct forms of visual representation. Emoji are generally pictographs that correspond to faces and other objects, whereas emoticons consist of punctuation marks, numbers, and letters that form pictorial icons, often facial expressions (Rodrigues et al., 2018). To ensure authenticity and validity, research articles that focused on emoticons were manually excluded. Ultimately, we identified 34 relevant articles from 27 journals and conference proceedings. This collection included 20 empirical studies, seven content analysis and synthesis studies, and seven application studies examining the broad use of emoji. The collected papers were then categorized and coded (details are provided in Appendix B).

Research on the effects of emoji in health-related fields has grown significantly over the past five years (2018-2023), especially following the outbreak of the COVID-19 pandemic in 2020. While this research is still in its early stages, it has become increasingly clear that emoji can play an important role in health communication, with empirical studies exploring their use in a wide range of contexts, including those related to infectious diseases and vaccination (Boender et al., 2022; Lin & Luo, 2022, 2023a, 2023b; Lu & Sun, 2022), psychological conditions such as depression (Clough, Morrow, et al., 2023; Clough, Tanguay, et al., 2023; Hand et al., 2023; Lee et al., 2008; Taylor et al., 2022), healthcare chatbots and misinformation dissemination (Fadhil et al., 2018; Hou & Kankham, 2022; Temmesen et al., 2021; Yu & Zhao, 2022), and food safety (Benito-Ostolaza et al., 2021; Ray & Merle, 2021). Research by Walther and D’Addario (2001) indicated that pictorial representations can aid in reading comprehension, and Hand et al. (2023) demonstrated that the emotional valence of emoji can influence accompanying text to enhance overall perceptual persuasiveness. Similarly, this paper argues that emoji can affect arousal, making them a valuable tool in health communication that offers new possibilities for improving public health and health education.

However, the use of emoji may not be universally effective, and incorrect or inappropriate use may lead to neutral or even negative effects. For example, Willoughby and Liu (2018) determined that non-narrative health information without emoji tends to be perceived as more credible than information containing emoji. In addition, Fadhil et al. (2018) found that the use of emoji may result in lower evaluation scores due to users’ preference for text-based interactions with health chatbots. The integration of emoji is also affected by significant disparities that exist across different types of medical conditions. In one study, individuals diagnosed with autism spectrum disorder (ASD) and neurotypical individuals differed in their ability to identify representations of fear, surprise, and sadness in emoji but exhibited similarities in recognizing representations of happiness, disgust, and anger. Furthermore, emoji had a more pronounced effect on the emotions of the participants with ASD compared to the neurotypical participants when reading neutral text (Hand et al., 2023). Although these empirical studies support the general effectiveness of emoji in health communication and medical contexts, systematic evidence is needed to establish standard recommendations for the appropriate application of emoji as a visual language.

In addition to empirical research, topics related to emoji have also been frequently explored through content analysis and assessment studies in health-related fields. On various social media platforms, users employ emoji and their emotional valence to address current contingencies (e.g., for COVID-19: Das, 2021), discuss contentious issues (e.g., marijuana: Tran et al., 2018), and convey personal feelings (e.g., subjective well-being: Liu, 2023). Das (2021) reported that gender and regional differences affect emoji usage patterns when people are addressing health-related topics, and Al-Rawi et al. (2020) found that male users were more inclined to use positive emoji than female users in the context of COVID-19. However, that study also identified a consistent pattern in which both male and female users tended to use negative emoji when discussing gender-related health issues. Other studies have used emoji as assessment tools to evaluate food health (Jaeger et al., 2018; Pinto et al., 2020; Sick et al., 2022), social psychology domains (Davies et al., 2022; Thompson et al., 2018), and pain indices (Li et al., 2023; Liao et al., 2021). Furthermore, emoji provide a versatile and efficient method for assessing emotions and health across various scales, irrespective of participants’ literacy levels. These studies demonstrate the reliability and validity of emoji-based scales and recommend their use as measurement tools in various scenarios; therefore, the application of emoji as a visual language in health-related contexts requires systematic discussion.

Visual Semantics of Emoji

Emoji are characterized by frequent updates that result in several iterations. Both the Unicode database and Emojipedia regularly update the quantity and design styles of emoji each year, and as of the time of writing, the Unicode database contains 3,782 emoji (Unicode, 2023; https://www.unicode.org/emoji/charts/emoji-counts.html). Emoji are symbolic representations of abstract or concrete themes, and like words, they are polysemous, meaning that they serve multiple semantic and pragmatic purposes (Scheffler et al., 2022; Shardlow et al., 2022). Research by Kaye et al. (2016) and Hand et al. (2023) has confirmed that emoji can complement and enhance textual language by introducing attractive and elaborate visual communication elements.

However, the meaning of an emoji can vary based on context and individual interpretation. For instance, 💉 can be used as a noun to indicate a vaccine or a syringe, or as a verb to depict the act of injection or vaccination. Emoji are usually placed outside the syntactic structure of a sentence, typically appearing at the end of a sentence. The placement of emoji can enhance their ability to complement textual language and create interdependencies between symbols and text (i.e., they can be contextualized), which holds true across the diverse range of emoji types and styles (Grosz et al., 2021; Kaiser & Grosz, 2021). Additionally, Wicke and Bolognesi (2020) found that metaphors and metonymies are effective strategies for creating vivid imagery and explaining complex ideas. From this perspective, facial emoji such as 😣 are analogous to expressions that convey a user’s state of mind, whereas non-facial emoji (primarily those representing actions and objects) are interpreted in relation to the surrounding content, which often provides additional context or explanation.

Computer-meditated communication (CMC) technology has transformed human interaction, giving rise to a “quasi-language” comprised of emoji and similar visual elements. This composite mode of communication is both intriguing and effective and sometimes surpasses textual representation in its ability to express certain types of conceptual content via visual cues. Although these blended messages combining text and visual elements cannot always be clearly defined (Clough, Tanguay, et al., 2023; Grosz et al., 2023), the goal is to create a congruent and synchronized text–visual arrangement (Lin & Luo, 2023a; Tseng & Hsieh, 2019). This approach reveals semantic features in which perceptual attributes emerge from messages that are conveyed using emoji (Wicke & Bolognesi, 2020). Similar to how letters form words, which then become a means of communication and expression of thoughts (Willen & Strals, 2009), emoji can also evoke a wide range of emotions, cognitive responses, and associations through their distinctive forms of expression.

Emoji can also be creatively used to translate abstract concepts (Wicke & Bolognesi, 2020), convey metaphorical meanings, and repurpose content effectively (Wiseman & Gould, 2018) to complement textual language. According to schema congruity theory, proposed by Mandler (2014), greater perceived consistency between visual and textual languages leads to more positive evaluations of content. As predicted by schema-congruity effects and the fluency theory of aesthetics, the mutual alignment of textual and visual representation enhances reader satisfaction and facilitates smooth and efficient information processing (Lin & Luo, 2023a; Reber et al., 2004). Other research has shown that reading speed is considerably slower for emoji than for words (Scheffler et al., 2022). This finding suggests a trade-off between a slight delay in processing emoji and the richer, more nuanced expression they enable (Thompson et al., 2016), which ultimately enhances enjoyment and engagement (Lin & Luo, 2023b; Scheffler et al., 2022).

A thorough discussion of emoji would be incomplete without considering their application from a design perspective. Users do not necessarily use emoji singularly but often repeat the same emoji sequentially in a message. Such repetition is associated with the intensity of emotional expression or to provide additional emphasis. Repeated emoji can also establish a visual hierarchy, allowing for the representation of multiple levels of content, emotions, and attitudes. Visual hierarchy, a graphic design principle, involves the use of visual elements to direct attention in a predetermined order (Collins et al., 2015; Golombisky & Hagen, 2013). For instance, designers often construct a hierarchical layout to establish a reading order to cue their audience (Lupton & Phillips, 2008), or they may alter the size and quantity of visual elements to highlight specific aspects (Dabner et al., 2017; Golombisky & Hagen, 2013). Similarly, emoji have significant potential as a tool for creating an information hierarchy to highlight important content.

Methods

Based on the aforementioned research, this study hypothesized that emoji could be easily and accurately interpreted within textual health information messages, even though text can be read more quickly than corresponding emoji. To explore the combination, placement, and design of emoji as a visual language in health information, we conducted an experiment in two phases: first, a focus group was arranged to define and select a set of emoji from Emojipedia that could represent health-oriented perspectives. The selected emoji were then categorized according to their parts of speech using predetermined definitions. Subsequently, a laboratory experiment was conducted to investigate how people use emoji when designing health information. The participants’ demographic information (age, gender, degree, and college major) was collected immediately after the experiment. This approach enabled us to analyze and summarize the application paradigms of visual language. Ultimately, these normative design applications can deepen our understanding of the communicative functions, usage patterns, and processing features of emoji in health communication contexts.

Health-Themed Emoji, Sample Selection

In Phase I, we collected 120 items containing health information from 40 well-known Twitter (renamed to “X” in 2023) accounts focused on health news and public health education (the complete list is provided in Appendix B). The content from these items was translated into Chinese by two professional English translators who are also native Chinese speakers. The content covered vaccinations, cancer screenings, mental health, flu updates, and additional posts highlighting various health information. To identify relevant emoji, six experts (two health-care practitioners, two communication scholars, and two visual designers) were invited to participate in a focus group (Table 1). Each expert individually reviewed all the information and selected 69 emoji that were highly relevant to health information from the Emojipedia database (accessible at https://emojipedia.org/, last updated in November 2023). These emoji were then divided into seven categories: smileys, people, travel and places, animals and nature, objects, food and drink, and symbols (Figure 1). Although the health information collected included some time-related emoji (e.g., ⌚, 🕒), these were deemed to be insufficiently relevant to health or health care and were excluded from our selection.

Table 1. List of experts in the focus group.

Figure 1. Health-related emoji from Emojipedia (summarized by the focus group in this study).

Experimental Procedure and Participants

For the experiment, recruitment messages were posted on social media forums from November 20 to December 20, 2023, resulting in the recruitment of 60 participants. The ages of the participants ranged from 19 to 54 years (M = 27.73 years, SD = 8.04 years, 48.3% female participants). Among the participants, 93.4% held a university degree or higher, and 53.3% had an educational background in art and design. To ensure the validity of the experiment, all the participants confirmed having prior experience with using emoji.

After obtaining informed consent and collecting demographic information from all participants, we conducted an information design experiment focused on using emoji to enhance information communication. We randomly distributed the aforementioned 120 health information items among the 60 participants, with each participant responsible for designing six health information messages. To ensure that each information item had an equal probability of being assigned, we used the uniform distribution function in Microsoft Excel to make sure each item was repeated three times. Participants were provided with a pre-compiled list of health-themed emoji (Figure 1) and were instructed to use these emoji as visual language elements in each design task. The emoji could be used individually or as multiple units placed at multiple positions within a sentence. To prevent semantic confusion, the participants were instructed not to place emoji side-by-side in the same position. Data were collected and compiled after all the participants completed their design tasks.

Coding Scheme for Manual Annotation

To ensure data accuracy, four responses that did not use emoji were excluded, resulting in a total of 356 designed health information messages. We manually classified the data based on taxonomies because semantic features are components associated with lexical items. Two independent coders initially extracted the emoji used in each message, recorded their quantities, and noted their placement. The quantity of each emoji was counted as the actual number of times an emoji was used, regardless of the number of categories an emoji was included in. Emoji could be placed at the beginning, in the middle, at the end, or occupy two or more positions within a sentence. We used the following numerical coding system to classify the emoji based on their part(s) of speech in combination with the textual content: noun = 1; verb = 2; adjective = 3; noun + verb = 4; noun + adjective = 5; verb + adjective = 6; noun + verb + adjective = 7. The corresponding Krippendorff’s α value was 0.976.

To reveal the patterns in the application of emoji as a visual language in health information—including their positions, combinations, and hierarchical designs—we referred to Richard Saul Wurman’s “Five Hat Racks” principle, also known as “LATCH” (Wurman & Bradford, 1996). This concept helped us identify the location, category, and hierarchy of emoji within the designs. We coded seven sentence components (subject = 1, appositive = 2, object = 3, attributive = 4, adverbial = 5, complement = 6, and conjunction = 7) and used “#” to indicate the emoji’s placement in accordance with the semantic components of Chinese linguistics (Krippendorff’s α = 0.96). For example, when an emoji was placed at the beginning or the end of a sentence, it was recorded as “0” (e.g., 0#1 = start + emoji + subject and 5#0 = adverbial + emoji + end, respectively). When an emoji was placed between two sentence parts, it was recorded as “1#2” (subject + emoji + appositive). In addition, the sampled emoji were categorized according to their initial letter based on Emojipedia’s classification system (e.g., emoji under the people category were recorded as P, and those under the food and drink category were recorded as FD).

Visual hierarchy, an essential principle in graphic design, involves the arrangement of elements to indicate their importance. Visual language is commonly used to construct such hierarchies, and the number of visual elements often serves as the basis for such hierarchical design (Dabner et al., 2017; Golombisky & Hagen, 2013). A single emoji presents one visual level when used alone (e.g., 😱), whereas repeating elements can create a hierarchical structure to amplify a message (e.g., 😱😱😱). Therefore, a single emoji was coded as 0, and two or more emoji were coded as 1, to capture these hierarchical features.

Results

Phase I: Part of Speech Analysis

Since the default names and descriptions of emoji provided by Emojipedia do not fully capture the nuances of emoji usage patterns within specific contexts, a part-of-speech analysis is indispensable for a comprehensive understanding of how emoji function in terms of meaning and their patterns within sentences. This multicriteria analysis considers three aspects: (1) the predefined names and descriptions of each emoji in Emojipedia, (2) the syntactic associations between the textual language in the 120 health messages and emoji, and (3) the visual forms, meanings, and designs of the emoji themselves. From a linguistic perspective, an individual emoji can function as more than one part of speech depending on the context in which it is used. Therefore, understanding and analyzing these parts of speech can help us accurately define emoji and interpret how they are used.

Based on the three aspects described above, the focus group in this study identified nouns (including pronouns), verbs, and adjectives among the 69 selected emoji; adverbs, prepositions, conjunctions, interjections, and determiners were excluded (Figure 2). The results indicated that all selected health-themed emoji could represent nouns, 66.6% could represent verbs, and 44.9% could represent adjectives. This finding indicates that emoji can serve multiple grammatical functions, as previous studies have shown (Maier, 2023; Scheffler et al., 2022; Wicke & Bolognesi, 2020; Wiseman & Gould, 2018). Furthermore, 23.8% of all emoji were identified as representing a single part of speech; these were primarily emoji denoting professions, animals, and objects. Emoji that function as dual parts of speech accounted for 39.1%, with most representing nouns and verbs. For instance, 🤲 and 👁️ are commonly recognized as nouns representing hands and eyes, respectively, but are also categorized in Emojipedia as verbs (“hand washing/handling” and “watching/discovering,” respectively), which is of particular relevance in the context of health information. An additional 25 emoji were identified as representing three parts of speech, with most of them being facial expressions. For example, 😷 can represent a cold or flu (noun), the action of wearing a mask (verb), or the condition of having an illness (adjective). To gain a deeper understanding of this multimodal dialogue, we proceeded to the next phase of our study.

Figure 2. The part-of-speech classification of health-related emoji (summarized by the focus group in this study).

Phase II: Experiment Overview and Descriptive Statistics

A descriptive analysis was conducted to provide an overview of emoji usage in terms of their quantity, frequency, and placement. Regarding quantity, the analysis revealed that 68% of the health information assessed contained three or fewer emoji, with 30.1% using only one emoji as a visual language supplement (Table 2). This result suggests that most individuals did not heavily rely on emoji as visual supplements to communicate health information, with three emoji being the maximum number used. Comparing the frequency of emoji use, 🦠 emerged as the most frequently used emoji, appearing 81 times, followed by 😱 (50 occurrences), 💉 (45 occurrences), ☠ (38 occurrences), 🏥 (37 occurrences), and 😷 (35 occurrences) (Figure 3). The frequent use of these emoji reflects their ideographical popularity and effectiveness in conveying health information. For instance, 🦠 is mainly used to supplement nouns such as germ, virus, epidemic, and infection rate, all of which relate to disease. In contrast, 😱 is not used to supplement nouns but to evoke fear or describe something frightening. 💉 is used to represent the noun “vaccine” or to describe the action of “vaccinating.” These findings demonstrate that users can effectively extract visual meanings from emoji and integrate them into text. Furthermore, we observed that high-frequency emoji were often used repeatedly within a sentence to emphasize sentiments or describe scenarios. Further discussion regarding this topic is provided in the “Information Architecture Analysis” subsection later in this paper.

Table 2. The number of emoji used in health information.

Figure 3. The frequency of emoji usage among experimental participants. (Click on the image for larger version.)

A preliminary analysis of emoji placement revealed that the most common position for emoji was within a sentence (40.7%), followed by mixed placement within and at the end of a sentence (33.7%), and solely at the end of a sentence (19.9%). Other positions were relatively uncommon. These findings suggest that users are unlikely to initiate health information with emoji, even in instances where emoji are placed in multiple positions within a sentence. Moreover, a one-way analysis of variance found significant gender differences in emoji use (F = 6.88, p < 0.05). Specifically, female participants tended to use substantially more emoji in the health information they designed than did male participants (Mfemale = 3.0, SD = 1.04; Mmale = 2.35, SD = 0.88). Other variables, such as age, educational level, and having a background in art and design education, did not appear to result in significant variations in emoji usage.

Parts of Speech of the Emoji

In the present experiment, emoji were primarily categorized within the noun family, assuming singular roles as nouns, dual roles as nouns + verbs or nouns + adjectives, or triple roles as nouns + verbs + adjectives, for a total of 304 emoji (85.4%). Among these, most emoji used in the health information functioned as nouns only (52%), while another 22% represented both nouns and adjectives, and 19.7% depicted both nouns and verbs (Figure 4). Figure 4 further illustrates the number and percentage of different parts of speech the emoji represented. The results revealed that emoji were predominantly perceived as nouns, such as “hospital (Centers for Disease Control)” = 🏥, “outbreak (virus or cases)” = 🦠, and “vaccine” = 💉.

Figure 4. The part-of-speech analysis results of emoji usage among experimental participants.
(Click on the image for larger version.)

Studies by Grosz et al. (2021) and Kaiser and Grosz (2021) have emphasized the importance of coherence and discourse structure in the effective use of emoji, arguing that both emoji and text within a sentence should be consistent and synchronized. Our findings support this view and reveal that emoji often supplement nouns to highlight key content. For example, 🚬 and 🥃 were commonly recognized as representing nouns (“tobacco” and “alcohol,” respectively) and verbs (“smoking” and “drinking,” respectively). However, our experimental results revealed that these emoji could also function as adjectives to describe tobacco and alcohol products. Conversely, when emoji were used to represent verbs in the context of health information, they primarily focused on event causality. There were only a few instances in which emoji were used to supplement only verbs or adjectives. Moreover, among the 69 health-themed emoji, 22 were facial expressions. These were most frequently used as nouns to depict symptoms or emotional states rather than to describe the quality of events or convey emotional intensity.

Information Architecture Analysis

Based on the experimental results, we identified 53 distinct combinations of emoji and sentence component placement, totaling 763 emoji “location distributions.” Emoji placed after subjects, appositives, and objects were identified 92, 56, and 254 times, respectively, while emoji placed after attributives, adverbials, complements, and conjunctions were identified 134, 50, 147, and five times, respectively (Table 3). The data indicate a tendency among users to place emoji after objects; otherwise, users were most likely to place them after complements and attributives. Notably, the most common placement was after object components that were positioned at the ends of sentences, accounting for 19.8% (151 occurrences) of the total distribution. In addition, emoji were placed after complements and at the ends of sentences 100 times (13.1%) out of 763, making this the most frequent placement among the various combinations. This suggests that the highest levels of coherence and synchronization between textual representation and visual elements were achieved when emoji were placed after objects, after complements, or at the ends of sentences. Figure 5 presents the frequency distribution analysis of three information architectures involving emoji in health information.

Table 3. Emoji placed after various sentence parts (number of times).

Figure 5. The “LCH” analysis results of emoji usage among experimental participants. (Click on the image for larger version.)

Categorization is useful for identifying patterns by simplifying our perception and cognition. According to Emojipedia, there are seven types of emoji: smileys, people, travel and places, animals and nature, objects, food and drink, and symbols. Our results revealed that facial expression emoji (smileys) were the most frequently used, accounting for 19.9% of instances, followed by people emoji (14%). In several cases, two or more emoji were used together to form relationships and create complex meanings through the combination of multiple visual elements. The most common combinations consisted of smileys + people (10.1%) or smileys + objects (5.6%). The use of single emoji (44.9%) and pairs of emoji (38.5%) occurred consistently. These findings align with the part-of-speech analysis, demonstrating that users are proficient in using facial expression emoji and can effectively combine them with emoji in the people and objects categories. In addition, we determined that emoji used in health information generally do not exceed two categories.

This paper contends that using emoji in sequence can enhance clarity and emphasis in a text, potentially through the distinct tone that visual language provides. This inference is derived from the literary technique of repetition, where words are repeated multiple times to create rhythm, emphasize particular points, or convey deeper meanings. Unlike the Zero Width Joiner sequence used to create glyphs, this study focused on the repetition of sequenced emoji within a sentence. We also analyzed how single emoji and sequences of repeated emoji were used to emphasize and highlight meanings, revealing two layers of visual hierarchies: (1) the singular use of emoji (77.2%) and (2) the repetition of emoji two or more times in sequence (22.8%). For instance, the designed health information titled “Sickle Cell Disease 🦠 (SCD) affects approximately 100,000 Americans and millions worldwide 🤒🤒🤒” emphasizes the large number of infections by repeating 🤒 three times. Additionally, 83.9% of repeated emoji sequences were created by users with a background in art and design, suggesting that individuals with art and design expertise are particularly adept at utilizing expressive visual elements and employing visual language narratives to express emotions and feelings.

General Discussion

As an emerging form of visual language in health information communication, emoji do not strictly adhere to grammar or syntactic rules. The present study established a micro-database of 69 health-related emoji identified through focus group discussions and a part-of-speech attribution analysis. Subsequently, we conducted an information design experiment to uncover patterns and paradigms regarding the use of emoji in health information. The location–category–hierarchy framework used in this study was derived from Wurman’s LATCH method for the organization of information, which was specifically employed to examine the use of emoji in terms of their placement, parts of speech, and frequency of use.

The use of emoji in health information can positively influence public perception and behavioral intention; however, increased emoji use does not necessarily translate to better outcomes. Specifically, the use of too many emoji may disrupt the coherence between emoji and the text, leading to cognitive fatigue and reduced persuasiveness (Islam et al., 2018; Lin & Luo, 2023b; Mustapar et al., 2016). Although visual language tends to be more expressive than mere text through its use of form and color, which facilitates intuitive information processing, excessive repetition of visual elements may reduce its effectiveness (McBride & Dosher, 2002). Thus, the overuse of emoji is not recommended. This study found that using three or fewer emoji in health information appears to be generally effective when applied correctly, a finding that aligns with research by Madden and Langley (2003) which found that an excess of visual elements diverts attention, and that individuals can only fully process information when concentrating on four or fewer items.

Participants in this study tended to position emoji within sentences or used a combination of mid-sentence and end-sentence placements. This finding is consistent with psycholinguistic research on the effects of word positioning, which has shown that words placed at the beginning, in the middle, or at the end of a sentence have different effects on processing (Kuperman et al., 2010; Robus et al., 2020; Warren et al., 2009). Readers typically spend more time processing words at the end or in the middle of a sentence than words at the beginning of a sentence, a phenomenon known as the wrap-up effect. This occurs due to the higher-order cognitive processes required to understand and semantically integrate information accumulated from preceding words. Conversely, the launch effect, which is less pronounced, highlights how messages at the beginning of a sentence shape initial impressions and set expectations for the content that follows. Although emoji may not possess a linguistic structure as complete as that of textual information, they clearly carry semantic value. When emoji are placed in the middle or at the end of a sentence, the integration of semantic context is enhanced, making readers more likely to engage in higher-level processing (Robus et al., 2020). Thus, placing emoji in the middle of sentences or using a combination of middle and end placements can serve as an effective strategy for improving message clarity and impact.

Grosz et al. (2023) claim that emoji involve anaphoric dependencies and “can be linked to the preceding linguistic context.” Users often used emoji in their health information to supplement nouns (to signify threats) or gerunds (to indicate efficacy). These two prominent features align with concepts in the extended parallel process model (EPPM) developed by Witte (1992), which claims that threat descriptions are generally expressed using nouns, whereas efficacy tends to be conveyed through gerunds or combinations of nouns and verbs. Emoji exhibit the highest level of consistency in health information when they supplement nouns to describe threats, threat-related events, or affected populations, as well as when they combine nouns and verbs to express efficacy.

Participants in this study tended to place emoji after objects or complements, often positioning them at the ends of sentences. This is consistent with previous studies showing that users place emoji at the ends of sentences in approximately half of all instances (Garrison et al., 2011; Tauch & Kanjo, 2016) and for deliberate use (Amaghlobeli, 2012; Spina, 2019). In addition, facial expression emoji (smileys) and their combinations with character emoji (people) and object emoji (objects) were frequently used in the health information to express emotions and describe objective issues. Similar to how words can function as multiple parts of speech, facial expression emoji can convey both symptoms and emotions when combined with emoji in other categories. Furthermore, repeating a single emoji can create an effective high-level visual hierarchy, which, in turn, enhances perceived information quality and visual informativeness (Berlyne, 1958; Palmer, 1999). While users with a background in art and design may excel in this area, we believe that most users can benefit from the findings of this study, enabling them to create improved visual hierarchies through the use of emoji.

In summary, although it is effectively impossible to cover every type of emoji in a single study, we propose the following four guidelines for using emoji as visual language in health information:

Three or fewer emoji should be used to optimize clarity and effectiveness.
When using emoji to supplement text, position them in the middle of a sentence or use a combination of mid-sentence and end-sentence placements to maximize impact.
In health information, enhance coherence by selecting emoji that correspond to nouns to represent threats, and use emoji that correspond to gerunds to convey efficacy.
When placing emoji after objects, complements, or at the ends of sentences, select emoji from a single category. Appropriate use of facial emoji and repeating the same emoji can enhance the visual hierarchy to highlight important information or enhance emotional expression.

This paper provides valuable paradigms and principles for the use of emoji in visual health communication. To the best of our knowledge, this study constitutes the first exploration of its kind in this area. However, some current theories may not fully align with our research findings, suggesting certain limitations that necessitate additional research. Nevertheless, the present study is an initial step toward understanding emoji use in health communication. It offers both theoretical resources and practical insights related to visual communication in health contexts and health information design. Future research should include more comprehensive sets of emoji samples and databases to further enhance understanding regarding the use of emoji in this extensive quasi-linguistic system of textual language with greater precision.

Acknowledgments

This research was supported by the National Science and Technology Council [MoST 111-2410-H-011-033-MY2].

Disclosure Statement

No potential conflict of interest was reported by the author(s). All authors have made substantial contributions to the conceptualization, methodology, formal analysis, and writing of the paper.

References

Abell, A., Smith, L., & Biswas, D. (2022). What’s in a “Happy” meal? The effects of smiley faces in restaurant logos on price and healthfulness perceptions. Journal of Advertising, 53(1), 54-69. https://doi.org/10.1080/00913367.2022.2090465
Ally, B. A., Gold, C. A., & Budson, A. E. (2009). The picture superiority effect in patients with Alzheimer’s disease and mild cognitive impairment. Neuropsychologia, 47(2), 595-598. https://doi.org/10.1016/j.neuropsychologia.2008.10.010
Al-Rawi, A., Siddiqi, M., Morgan, R., Vandan, N., Smith, J., & Wenham, C. (2020). COVID-19 and the gendered use of emojis on Twitter: infodemiology study. Journal of Medical Internet Research, 22(11), e21646. https://doi.org/10.2196/21646
Amaghlobeli, N. (2012). Linguistic features of typographic emoticons in SMS discourse. Theory and Practice in Language Studies, 2(2), 348-354. https://doi.org/10.4304/tpls.2.2.348-354
Bai, Q., Dan, Q., Mu, Z., & Yang, M. (2019). A systematic review of emoji: Current research and future perspectives. Frontiers in Psychology, 10, 2221. https://doi.org/10.3389/fpsyg.2019.02221
Benito-Ostolaza, J. M., Echavarri, R., Garcia-Prado, A., & Oses-Eraso, N. (2021). Using visual stimuli to promote healthy snack choices among children. Social Science & Medicine, 270, 113587. https://doi.org/10.1016/j.socscimed.2020.113587
Berlyne, D. E. (1958). The influence of complexity and novelty in visual figures on orienting responses. Journal of Experimental Psychology, 55(3), 289-296. https://doi.org/10.1037/h0043555
Boender, T. S., Louis-Ferdinand, N., & Duschek, G. (2022). Digital visual communication for public health: Design proposal for a vaccinated emoji. Journal of Medical Internet Research, 24(4), e35786. https://doi.org/10.2196/35786
Choi, Y., Hyun, K. H., & Lee, J. H. (2020). Image-based tactile emojis: Improved interpretation of message intention and subtle nuance for visually impaired individuals. Human–Computer Interaction, 35(1), 40-69. https://doi.org/10.1080/07370024.2017.1324305
Clough, S., Morrow, E., Mutlu, B., Turkstra, L., & Duff, M. C. (2023). Emotion recognition of faces and emoji in individuals with moderate-severe traumatic brain injury. Brain Injury, 37(7), 596-610. https://doi.org/10.1080/02699052.2023.2181401
Clough, S., Tanguay, A. F., Mutlu, B., Turkstra, L. S., & Duff, M. C. (2023). How do individuals with and without traumatic brain injury interpret emoji? Similarities and differences in perceived valence, arousal, and emotion representation. Journal of Nonverbal Behavior, 47, 489-511. https://doi.org/10.1007/s10919-023-00433-w
Collins, W., Hass, A., Jeffery, K., Martin, A., Medeiros, R., & Tomljanovic, S. (2015). Graphic design and print production fundamentals. BCcampus.
Cunha, J. M., Lourenço, N., Correia, J., Martins, P., & Machado, P. (2019). Emojinating: Evolving emoji blends. In Proceedings of the international conference on computational intelligence in music, sound, art and design (pp. 110-126). Springer. https://doi.org/10.1007/978-3-030-16667-0_8
Dabner, D., Stewart, S., & Vickress, A. (2017). Graphic design school: The principles and practice of graphic design. John Wiley & Sons.
Das, A. (2021). How has the coronavirus (COVID-19) pandemic affected global emoji usage? Journal of Human Behavior in the Social Environment, 31(1-4), 425-434. https://doi.org/10.1080/10911359.2020.1838383
Das, G., Wiener, H. J., & Kareklas, I. (2019). To emoji or not to emoji? Examining the influence of emoji on consumer reactions to advertising. Journal of Business Research, 96, 147-156. https://doi.org/10.1016/j.jbusres.2018.11.007
Davies, J., McKenna, M., Denner, K., Bayley, J., & Morgan, M. (2022). The emoji current mood and experience scale: The development and initial validation of an ultra-brief, literacy independent measure of psychological health. Journal of Mental Health, 33(2), 218-226. https://doi.org/10.1080/09638237.2022.2069694
Derks, D., Bos, A. E., & Von Grumbkow, J. (2007). Emoticons and social interaction on the Internet: the importance of social context. Computers in Human Behavior, 23(1), 842-849. https://doi.org/10.1016/j.chb.2004.11.013
Eysenbach, G., & Köhler, C. (2002). How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ, 324(7337), 573-577. https://doi.org/10.1136/bmj.324.7337.573
Fadhil, A., Schiavo, G., Wang, Y., & Yilma, B. A. (2018). The effect of emojis when interacting with conversational interface assisted health coaching system. In Proceedings of the 12th EAI international conference on pervasive computing technologies for healthcare (pp. 378-383). ACM. https://doi.org/10.1145/3240925.3240965
Ferré, P., Haro, J., Pérez-Sánchez, M. Á., Moreno, I., & Hinojosa, J. A. (2023). Emoji-SP, the Spanish emoji database: Visual complexity, familiarity, frequency of use, clarity, and emotional valence and arousal norms for 1031 emojis. Behavior Research Methods, 55, 1715-1733. https://doi.org/10.3758/s13428-022-01893-6
Gallagher, J. R., & Avgoustopoulos, R. E. (2023). Emojination facilitates inclusive emoji design through technical writing: Fitting tactical technical communication inside institutional structures. Journal of Technical Writing and Communication, 54(1), 93-112. https://doi.org/10.1177/00472816231161062
Gallud, J. A., Fardoun, H. M., Andres, F., & Safa, N. (2018). A study on how older people use emojis. In Proceedings of the 19th international conference on human computer interaction (Article No. 20). ACM. https://doi.org/10.1145/3233824.3233861
Garrison, A., Remley, D., Thomas, P., & Wierszewski, E. (2011). Conventional faces: Emoticons in instant messaging discourse. Computers and Composition, 28(2), 112-125. https://doi.org/10.1016/j.compcom.2011.04.001
Golombisky, K., & Hagen, R. (2013). White space is not your enemy. Taylor & Francis. https://doi.org/10.4324/9780240812823
Grosz, P. G., Greenberg, G., De Leon, C., & Kaiser, E. (2023). A semantics of face emoji in discourse. Linguistics and Philosophy, 46, 9025-957. https://doi.org/10.1007/s10988-022-09369-8
Grosz, P., Kaiser, E., & Pierini, F. (2021). Discourse anaphoricity and first-person indexicality in emoji resolution. Proceedings of Sinn Und Bedeutung, 25, 340-357. https://doi.org/10.18148/sub/2021.v25i0.941
Halverson, C. M., Donnelly, C. E., Weiner, M., & Lee, J. L. (2023). Content analysis of emoji and emoticon use in clinical texting systems. JAMA Network Open, 6(6), e2318140. https://doi.org/10.1001/jamanetworkopen.2023.18140
Hand, C. J., Kennedy, A., Filik, R., Pitchford, M., & Robus, C. M. (2023). Emoji identification and emoji effects on sentence emotionality in ASD-diagnosed adults and neurotypical controls. Journal of Autism and Developmental Disorders, 53(6), 2514-2528. https://doi.org/10.1007/s10803-022-05557-4
Homann, L. A., Roberts, B. R., Ahmed, S., & Fernandes, M. A. (2022). Are emojis processed visuo-spatially or verbally? Evidence for dual codes. Visual Cognition, 30(4), 267-279. https://doi.org/10.1080/13506285.2022.2050871
Hönings, H., Knapp, D., Nguyễn, B. C., Richter, D., Williams, K., Dorsch, I., & Fietkiewicz, K. J. (2022). Health information diffusion on Twitter: The content and design of WHO tweets matter. Health Information & Libraries Journal, 39(1), 22-35. https://doi.org/10.1111/hir.12361
Hou, J. R., & Kankham, S. (2022). More than feelings? How Facebook reaction icons affect online users’ behavioral intentions toward online health rumor posts. Internet Research, 32(6), 1978-2002. https://doi.org/10.1108/INTR-04-2021-0236
Islam, A. K. M., Whelan, E., & Brooks, S. (2018). Social media overload and fatigue: The moderating role of multitasking computer self-efficacy. In Proceedings of the Americas conference on information systems. ALS eLibrary.
Jaeger, S. R., Roigard, C. M., & Ares, G. (2018). Measuring consumers’ product associations with emoji and emotion word questionnaires: Case studies with tasted foods and written stimuli. Food Research International, 111, 732-747. https://doi.org/10.1016/j.foodres.2018.04.010
Kaiser, E., & Grosz, P. G. (2021). Anaphoricity in emoji: An experimental investigation of face and non-face emoji. Proceedings of the Linguistic Society of America, 6(1), 1009-1023. https://doi.org/10.3765/plsa.v6i1.5067
Kaye, L. K., Wall, H. J., & Malone, S. A. (2016). “Turn that frown upside-down”: A contextual account of emoticon usage on different virtual platforms. Computers in Human Behavior, 60, 463-467. https://doi.org/10.1016/j.chb.2016.02.088
Kimura-Thollander, P., & Kumar, N. (2019). Examining the “global” language of emojis: Designing for cultural representation. In Proceedings of the conference on human factors in computing systems (Article No. 495). ACM. https://doi.org/10.1145/3290605.3300725
King, A. J. (2015). A content analysis of visual cancer information: prevalence and use of photographs and illustrations in printed health materials. Health Communication, 30(7), 722-731. https://doi.org/10.1080/10410236.2013.878778
Kuperman, V., Dambacher, M., Nuthmann, A., & Kliegl, R. (2010). The effect of word position on eye-movements in sentence and paragraph reading. Quarterly Journal of Experimental Psychology, 63(9), 1838-1857. https://doi.org/10.1080/17470211003602412
Lee, A. C. K., Tang, S. W., Yu, G. K. K., & Cheung, R. T. F. (2008). The smiley as a simple screening tool for depression after stroke: A preliminary study. International Journal of Nursing Studies, 45(7), 1081-1089. https://doi.org/10.1016/j.ijnurstu.2007.05.008
Li, L., Wu, S., Wang, J., Wang, C., Zuo, W., Yu, L., & Song, J. (2023). Development of the emoji faces pain scale and its validation on mobile devices in adult surgery patients: Longitudinal observational study. Journal of Medical Internet Research, 25, e41189. https://doi.org/10.2196/41189
Liao, W., Zhang, Y., Huang, X., Xu, X., & Peng, X. (2021). “Emoji, I can feel your pain”–Neural responses to facial and emoji expressions of pain. Biological Psychology, 163, 108134. https://doi.org/10.1016/j.biopsycho.2021.108134
Lin, T. S., & Luo, Y. (2023a). Health persuasion through emoji: How emoji interacted with information source to predict health behaviors in COVID-19 situation. SSM-Population Health, 21, 101343. https://doi.org/10.1016/j.ssmph.2023.101343
Lin, T. S., & Luo, Y. (2023b). Emoji and visual complexity in health information design: A moderated serial mediation model. Telematics and Informatics, 85, 102065. https://doi.org/10.1016/j.tele.2023.102065
Lin, T. S., & Luo, Y. (2022). Evaluating intertwined effects of emoji and message framing to vaccine communication strategy. In Proceedings of the international conference on human-computer interaction (pp. 35-48). Springer. https://doi.org/10.1007/978-3-031-05014-5_4
Liu, M. (2023). Are you really smiling? Display rules for emojis and the relationship between emotion management and psychological well-being. Frontiers in Psychology, 14, 1035742. https://doi.org/10.3389/fpsyg.2023.1035742
Lotfinejad, N., Assadi, R., Aelami, M. H., & Pittet, D. (2020). Emojis in public health and how they might be used for hand hygiene and infection prevention and control. Antimicrobial Resistance & Infection Control, 9, Article No. 27. https://doi.org/10.1186/s13756-020-0692-2
Lotfinejad, N., Tartari, E., Sauser, J., Fankhauser-Rodriguez, C., Pires, D., & Pittet, D. (2022). Are emojis ready to promote the WHO 5 moments for hand hygiene in healthcare? Antimicrobial Resistance & Infection Control, 11, Article No. 127. https://doi.org/10.1186/s13756-022-01164-1
Lu, F., & Sun, Y. (2022). COVID-19 vaccine hesitancy: The effects of combining direct and indirect online opinion cues on psychological reactance to health campaigns. Computers in Human Behavior, 127, 107057. https://doi.org/10.1016/j.chb.2021.107057
Lupton, E., & Phillips, J. C. (2008). Graphic design: The new basics. Princeton Architectural Press.
Madden, D. J., & Langley, L. K. (2003). Age-related changes in selective attention and perceptual load during visual search. Psychology and Aging, 18(1), 54-67. https://doi.org/10.1037/0882-7974.18.1.54
Maier, E. (2023). Emojis as pictures. Ergo, 10, 11. https://doi.org/10.3998/ergo.4641
Mandler, G. (2014). The structure of value: Accounting for taste. In M. S. Clark & S. T. Fiske (Eds.), Affect and cognition (pp. 3-36). Psychology Press.
Marengo, D., Giannotta, F., & Settanni, M. (2017). Assessing personality using emoji: An exploratory study. Personality and Individual Differences, 112, 74-78. https://doi.org/10.1016/j.paid.2017.02.037
Marengo, D., Settanni, M., & Giannotta, F. (2019). Development and preliminary validation of an image-based instrument to assess depressive symptoms. Psychiatry Research, 279, 180-185. https://doi.org/10.1016/j.psychres.2019.02.059
McBride, D. M., & Dosher, B. A. (2002). A comparison of conscious and automatic memory processes for picture and word stimuli: A process dissociation analysis. Consciousness and Cognition, 11(3), 423-460. https://doi.org/10.1016/S1053-8100(02)00007-7
Megehee, C. M., & Woodside, A. G. (2010). Creating visual narrative art for decoding stories that consumers and brands tell. Psychology & Marketing, 27(6), 603-622. https://doi.org/10.1002/mar.20347
Miller, H., Thebault-Spieker, J., Chang, S., Johnson, I., Terveen, L., & Hecht, B. (2016). “Blissfully happy” or “ready tofight”: Varying interpretations of emoji. Proceedings of the International AAAI Conference on Web and Social Media, 10(1), 259-268. https://doi.org/10.1609/icwsm.v10i1.14757
Miyake, E., & Martin, S. (2021). Long Covid: Online patient narratives, public health communication and vaccine hesitancy. Digital Health, 7. https://doi.org/10.1177/20552076211059649
Moussa, S. (2021). Measuring brand personality using emoji: Findings from Mokken scaling. Journal of Brand Management, 28(2), 116-132. https://doi.org/10.1057/s41262-020-00220-8
Mustapar, N. A., Abdullah, N., & Noor, N. L. M. (2016). A review towards developing a moment of information overload model. In Proceedings of the 4th international conference on user science and engineering (pp. 222-226). IEEE. https://doi.org/10.1109/IUSER.2016.7857964
Paivio, A. (2014). Mind and its evolution: A dual coding theoretical approach. Psychology Press. https://doi.org/10.4324/9781315785233
Palmer, S. E. (1999). Vision science: Photons to phenomenology. MIT press.
Pinto, V. R. A., Teixeira, C. G., Lima, T. S., Prata, E. R. B. D. A., Vidigal, M. C. T. R., Martins, E., Perrone, Í. T., & de Carvalho, A. F. (2020). Health beliefs towards kefir correlate with emotion and attitude: A study using an emoji scale in Brazil. Food Research International, 129, 108833. https://doi.org/10.1016/j.foodres.2019.108833
Ray, E. C., & Merle, P. F. (2021). Disgusting face, disease-ridden place?: Emoji influence on the interpretation of restaurant inspection reports. Health Communication, 36(14), 1867-1878. https://doi.org/10.1080/10410236.2020.1802867
Reber, R., Schwarz, N., & Winkielman, P. (2004). Processing fluency and aesthetic pleasure: Is beauty in the perceiver’s processing experience? Personality and Social Psychology Review, 8(4), 364-382. https://doi.org/10.1207/s15327957pspr0804_3
Robus, C. M., Hand, C. J., Filik, R., & Pitchford, M. (2020). Investigating effects of emoji on neutral narrative text: Evidence from eye movements and perceived emotional valence. Computers in Human Behavior, 109, 106361. https://doi.org/10.1016/j.chb.2020.106361
Rodrigues, D., Prada, M., Gaspar, R., Garrido, M. V., & Lopes, D. (2018). Lisbon emoji and emoticon database (LEED): Norms for emoji and emoticons in seven evaluative dimensions. Behavior Research Methods, 50, 392-405. https://doi.org/10.3758/s13428-017-0878-6
Rosen, L. D., Chang, J., Erwin, L., Carrier, L. M., & Cheever, N. A. (2010). The relationship between “textisms” and formal and informal writing among young adults. Communication Research, 37(3), 420-440. https://doi.org/10.1177/0093650210362465
Scheffler, T., Brandt, L., de la Fuente, M., & Nenchev, I. (2022). The processing of emoji-word substitutions: A self-paced-reading study. Computers in Human Behavior, 127, 107076. https://doi.org/10.1016/j.chb.2021.107076
Schouteten, J. J., Almli, V. L., Rini, L., & Berget, I. (2023). Valence, arousal and projective mapping of facial and non-facial emoji investigated using an incomplete block design approach. Food Quality and Preference, 105, 104762. https://doi.org/10.1016/j.foodqual.2022.104762
Seo, K., Dillard, J. P., & Shen, F. (2013). The effects of message framing and visual image on persuasion. Communication Quarterly, 61(5), 564-583. https://doi.org/10.1080/01463373.2013.822403
Shardlow, M., Gerber, L., & Nawaz, R. (2022). One emoji, many meanings: A corpus for the prediction and disambiguation of emoji sense. Expert Systems with Applications, 198, 116862. https://doi.org/10.1016/j.eswa.2022.116862
Sick, J., Monteleone, E., Dinnella, C., Pierguidi, L., & Spinelli, S. (2022). Development of an emoji-based self-report measurement tool to measure emotions elicited by foods in preadolescents. Food Quality and Preference, 100, 104585. https://doi.org/10.1016/j.foodqual.2022.104585
Sick, J., Monteleone, E., Pierguidi, L., Ares, G., & Spinelli, S. (2020). The meaning of emoji to describe food experiences in pre-adolescents. Foods, 9(9), 1307. https://doi.org/10.3390/foods9091307
Spina, S. (2019). Role of emoticons as structural markers in Twitter interactions. Discourse Processes, 56(4), 345-362. https://doi.org/10.1080/0163853X.2018.1510654
Szabó, Z. G. (2019). Semantic explanations. In E. Lepore & D. Sosa (Eds.), Oxford studies in philosophy of language (pp. 240-275). Oxford University Press.
Szeto, M. D., Barber, C., Ranpariya, V. K., Anderson, J., Hatch, J., Ward, J., Aguilera, M. N., Hassan, S., Hamp, A., Coolman, T., & Dellavalle, R. P. (2022). Emojis and emoticons in health care and dermatology communication: narrative review. JMIR Dermatology, 5(3), e33851. https://doi.org/10.2196/33851
Tauch, C., & Kanjo, E. (2016). The roles of emojis in mobile phone notifications. In Proceedings of the international joint conference on pervasive and ubiquitous computing (pp. 1560-1565). ACM. https://doi.org/10.1145/2968219.2968549
Taylor, H., Hand, C. J., Howman, H., & Filik, R. (2022). Autism, attachment, and alexithymia: Investigating emoji comprehension. International Journal of Human–Computer Interaction, 40(9), 2211-2224. https://doi.org/10.1080/10447318.2022.2154890
Temmesen, C. G., Nielsen, H. S., Andersen, H. L. M., Birch Petersen, K., & Clemensen, J. (2021). Using social media for qualitative Health research in Danish women of reproductive age: Online focus group study on Facebook. JMIR Formative Research, 5(5), e24108. https://doi.org/10.2196/24108
Thompson, C. A., Novotny, P. J., Bartz, A., Yost, K. J., & Sloan, J. A. (2018). Development of novel emoji scale to measure patient-reported outcomes in cancer patients. Journal of Clinical Oncology 36(7), Suppl. 174. https://doi.org/10.1200/JCO.2018.36.7_suppl.174
Thompson, D., Mackenzie, I. G., Leuthold, H., & Filik, R. (2016). Emotional responses to irony and emoticons in written language: Evidence from EDA and facial EMG. Psychophysiology, 53(7), 1054-1062. https://doi.org/10.1111/psyp.12642
Tigwell, G. W., Gorman, B. M., & Menzies, R. (2020). Emoji accessibility for visually impaired people. In Proceedings of the conference on human factors in computing systems (pp. 1-14). ACM. https://doi.org/10.1145/3313831.3376267
Tran, T., Nguyen, D., Nguyen, A., & Golen, E. (2018). Sentiment analysis of marijuana content via Facebook emoji-based reactions. In Proceedings of the international conference on communications (pp. 1-6). IEEE. https://doi.org/10.1109/ICC.2018.8422104
Troiano, G., & Nante, N. (2018). Emoji: What does the scientific literature say about them?-A new way to communicate in the 21th century. Journal of Human Behavior in the Social Environment, 28(4), 528-533. https://doi.org/10.1080/10911359.2018.1437103
Trope, Y., Liberman, N., & Wakslak, C. (2007). Construal levels and psychological distance: Effects on representation, prediction, evaluation, and behavior. Journal of Consumer Psychology, 17(2), 83-95. https://doi.org/10.1016/S1057-7408(07)70013-X
Tseng, T. H., & Hsieh, S. H. (2019). Determinants of emoticon usage in mobile instant messaging: A construal level theory perspective. Behaviour & Information Technology, 38(3), 289-301. https://doi.org/10.1080/0144929X.2018.1531927
Unicode. (2022). About emoji. Unicode. Retrieved May 30, 2024, from https://home.unicode.org/emoji/about-emoji/
Unicode. (2023). Emoji counts, v15.1. Retrieved May 30, 2024, from https://www.unicode.org/emoji/charts/emoji-counts.html
Van Dam, L., Rietstra, S., Van der Drift, E., Stams, G. J. J., Van der Mei, R., Mahfoud, M., Popma, A., Schlossberg, E., Pentland, A., & Reid, T. G. (2019). Can an emoji a day keep the doctor away? An explorative mixed-methods feasibility study to develop a self-help app for youth with mental health problems. Frontiers in Psychiatry, 10, 593. https://doi.org/10.3389/fpsyt.2019.00593
Walther, J. B., & D’addario, K. P. (2001). The impacts of emoticons on message interpretation in computer-mediated communication. Social Science Computer Review, 19(3), 324-347. https://doi.org/10.1177/089443930101900307
Warren, T., White, S. J., & Reichle, E. D. (2009). Investigating the causes of wrap-up effects: Evidence from eye movements and E-Z reader. Cognition, 111(1), 132-137. https://doi.org/10.1016/j.cognition.2008.12.011
Wicke, P., & Bolognesi, M. (2020). Emoji-based semantic representations for abstract and concrete concepts. Cognitive Processing, 21(4), 615-635. https://doi.org/10.1007/s10339-020-00971-x
Willen, B., & Strals, N. (2009). Lettering & type: Creating letters and designing typefaces. Princeton Architectural Press.
Willoughby, J. F., & Liu, S. (2018). Do pictures help tell the story? An experimental test of narrative and emojis in a health text message intervention. Computers in Human Behavior, 79, 75-82. https://doi.org/10.1016/j.chb.2017.10.031
Wiseman, S., & Gould, S. J. (2018). Repurposing emoji for personalised communication: Why 🍕 means “I love you”. In Proceedings of the conference on human factors in computing systems (Article No. 152). ACM. https://doi.org/10.1145/3173574.3173726
Witte, K. (1992). Putting the fear back into fear appeals: The extended parallel process model. Communication Monographs, 59(4), 329-349. https://doi.org/10.1080/03637759209376276
Wurman, R. S., & Bradford, P. (1996). Information architects. Graphis Press.
Xu, K., & Liao, T. (2020). Explicating cues: A typology for understanding emerging media technologies. Journal of Computer-Mediated Communication, 25(1), 32-43. https://doi.org/10.1093/jcmc/zmz023
Yu, S., & Zhao, L. (2022). Designing emotions for health care chatbots: Text-based or icon-based approach. Journal of Medical Internet Research, 24(12), e39573. https://doi.org/10.2196/39573

Appendix A. List of Journals Referencing Emoji in Health-Related Contexts

International Conference:

1. In 2018 12th EAI international conference on pervasive computing technologies for healthcare.

2. In International Conference on Human-Computer Interaction (2022).

3. In 2018 IEEE International Conference on Communications (ICC).

Appendix B. Summary of Health Communication and Public Health Research Exploring Emoji Use

No.	Article title (Year published)	Research scenarios	Emoji	Types of emoji	Analytical method	Results
Empirical studies on the application of emoji in health communication
1	Health persuasion through emoji: How emoji interacted with information source to predict health behaviors in COVID-19 situation. (2023)	COVID-19		Basic pattern	Web-based survey	The use of emoji in health communication led to increased perceived fear, heightened intention for preventive behavior, and reduced perceived controllability. Specifically, in the case of unofficial information sources, perceived fear served as a mediator for the impact of emoji on preventive behavioral intention.
2	Emoji and visual complexity in health information design: A moderated serial mediation model. (2023)	influenza A (H1N1)		Basic pattern	Automated content analysis and Web-based survey	The presence of emoji in health information leads to higher levels of perceived enjoyment, perceived interactivity, and social media engagement than their absence does. Emoji have a serial mediation effect on SME through PE and PI.
3	Digital visual communication for public health: design proposal for a vaccinated emoji. (2022)	Vaccinations		Redesign pattern	Design and feasibility of use	The utilization of vaccination emoji design could address the current public health necessity for enhanced digital communication regarding vaccine confidence.
4	Do pictures help tell the story? An experimental test of narrative and emojis in a health text message intervention. (2018)	Daily exercise and health interventions		Basic pattern	Web-based survey	In specific health contexts, health messages devoid of emoji might be preferable. However, for less impactful health messages or those not of particular interest to participants, narratives coupled with varying levels of emoji use (low or high) could effectively enhance perceptions of personalization and capture participant attention.
5	Are emojis ready to promote the WHO 5 moments for hand hygiene in healthcare? (2022)	Hand hygiene communication		Basic pattern	Web-based survey	Existing emoji might not effectively replace the text utilized in the WHO 5 Moments poster. However, emoji could serve to illustrate hand hygiene instructions in healthcare, potentially contributing to the promotion of this practice. Additional emoji to communicate health-related messages are required.
6	Disgusting Face, Disease-Ridden Place? : Emoji Influence on the Interpretation of Restaurant Inspection Reports. (2021)	Restaurant hygiene assessment and dining intentions		Basic pattern	Web-based survey	Disgusted face emoji, in contrast to text, heightened perceptions of risk and avoidance behavior. Meanwhile, smiling face emoji evoked stronger emotional responses related to sanitation among participants.
7	Using visual stimuli to promote healthy snack choices among children. (2021)	Child obesity		Redesign pattern	Randomized controlled trial	Positive visual stimuli (emoji) increase the probability of healthy snack choices among girls. Negative visual stimuli (emoji) do not seem to affect snack choices.
8	Designing Emotions for Health Care Chatbots: Text-Based or Icon-Based Approach. (2022)	Healthcare chatbot		Basic pattern	Web-based survey	When there was no text-based design, icon-based designs significantly increased the emotional intensity, and thus shortened the psychological distance and enhanced the behavioral intention.
9	The effect of emojis when interacting with conversational interface assisted health coaching system. (2018)	Healthcare chatbot		Basic pattern	Web-based survey	When the conversation is about private and personal aspects (e.g. mental health), a simple task may require the use of emoji to engage the user. Whereas when the conversation focuses on physical activity and information, plain text is preferred.
10	The smiley as a simple screening tool for depression after stroke: A preliminary study. (2008)	Depression after stroke		Basic pattern	A cross-sectional study and personal interview	The universally understood smiley emoji might be a feasible alternative to assess depression even for those with a low literacy level, a language barrier, or mild expressive aphasia.
11	Evaluating Intertwined Effects of Emoji and Message Framing to Vaccine Communication Strategy. (2022)	COVID-19 vaccination		Basic pattern	Web-based survey	Using negative emoji in health messages can help to establish the public’s self-efficacy, which is extremely significant in the gain framing. But when a health message is written in a loss framing, the emotional valence of emoji has no significant effect on self-efficacy.
12	Emotion recognition of faces and emoji in individuals with moderate-severe traumatic brain injury. (2023)	Moderate-severe traumatic brain injury		Basic pattern	Web-based survey	Because emotion representation is more ambiguous in emoji than in human faces, studying emoji use and perception in TBI is an important consideration for understanding functional communication and social participation after brain injury.
13	Development and preliminary validation of an image-based instrument to assess depressive symptoms. (2019)	Assessment of depressive symptoms		Basic pattern	Web-based survey	The use of the emoji-based measure made it possible to detect individuals experiencing clinically relevant depressive symptoms with good sensitivity and fair specificity.
14	Autism, attachment, and alexithymia: Investigating emoji comprehension. (2022)	Autism, attachment, and alexithymia		Basic pattern	Web-based survey	There are parallels between emoji classification and facial recognition in that the classification of simple emotions is usually more accurate than the classification of complex emotions (except for surprise).
15	Image-based tactile emojis: Improved interpretation of message intention and subtle nuance for visually impaired individuals. (2020)	Visually Impaired Individuals		Basic pattern	Laboratory experiments	The congenitally blind subjects were capable of recognizing the facial expressions of emoji. The image-based tactile emoji can enhance clarity in the computer-mediated communication environment for the visually impaired.
16	Using social media for qualitative Health research in Danish women of Reproductive Age: Online Focus Group Study on Facebook. (2021)	Danish Women of Reproductive Age		Basic pattern and Facebook’s reaction emoji	Online focus group study	When participants communicated with each other in the online focus groups, emoji or reaction symbols were used several times, typically when participants needed to denote a feeling, express humor or irony, or to emphasize a point.
17	COVID-19 vaccine hesitancy: The effects of combining direct and indirect online opinion cues on psychological reactance to health campaigns. (2022)	COVID-19 vaccination		The reaction emoji of Facebook	Web-based survey	Reaction emoji altered the comments' effects such that pro-vaccine comments triggered less reactance than anti-vaccine comments when the pro-vaccine comments were accompanied by agreement emoji.
18	More than feelings? How Facebook reaction icons affect online users’ behavioral intentions toward online health rumor posts. (2022)	Online health rumors		The reaction emoji of Facebook	Web-based survey	Negative reaction icons reduced Web users' behavioral intentions more than positive reaction icons. In addition, the inconsistency effect of the interaction (i.e., positive reaction icons with negative messages) had a greater negative impact on Web users' behavioral intentions than the consistency effect (i.e., positive reaction icons with positive messages).
19	Emoji Identification and Emoji Effects on Sentence Emotionality in ASD-Diagnosed Adults and Neurotypical Controls. (2023)	ASD-Diagnosed Adults and Neurotypical Controls		Basic pattern	Web-based survey	ASD-diagnosed participants rated otherwise neutral texts as more negative when presented with a sad emoji than NT participants. Although ASD-diagnosed and NT participants were similarly influenced by happy/positive emoji, ASD-diagnosed participants rated sentences + sad emoji more negatively than NT controls.
20	How do Individuals With and Without Traumatic Brain Injury Interpret Emoji? Similarities and Differences in Perceived Valence, Arousal, and Emotion Representation. (2023)	Recognizing facial emotions in persons with traumatic brain injury		Basic pattern	Web-based survey	There are many similarities in the way people with traumatic brain injury and non-brain injured peers perceive emoji in isolation, with small differences in arousal ratings for only a minority of emoji. Participants with traumatic brain injury rated their confidence in emoji labels as significantly lower than non-injured peers.

No.	Article title (Year published)	Research scenarios	Analytical method	Results
Synthesis study and content analysis of emoji in health research
1	Emojis and Emoticons in Health Care and Dermatology Communication: Narrative Review. (2022)	Healthcare and dermatology communication	Narrative review	Key subject areas that emerged from the investigation included the ability of emoji to improve communication within pediatric healthcare, enhance mood and psychological assessment or mental health screening in adults, develop interventions to improve patient medication adherence, complement novel means of public health and COVID-19 surveillance, and bolster dermatology-specific applications.
2	Are you really smiling? Display rules for emojis and the relationship between emotion management and psychological well-being. (2023)	Emotion management and psychological well-being	Web-based survey	Expressing emotions with emoji was associated with subjective well-being, whereas managing emotions with emoji was weakly associated with depressive symptoms.
3	COVID-19 and the gendered use of emojis on Twitter: Infodemiology study. (2020)	COVID-19 and the gendered use	A mixed method based on content analysis	There were many differences alongside discourses of men, women, and gender minorities when certain topics were discussed, such as death, financial and employment matters, gratitude, and health care, and several unique gendered emoji were used to express specific issues like community support.
4	Sentiment Analysis of Marijuana Content via Facebook Emoji-Based Reactions. (2018)	Reactions to Marijuana information	Content Analysis	Users responded to online information about marijuana in a way similar to the emotional valence of the emoji used.
5	Content Analysis of Emoji and Emoticon Use in Clinical Texting Systems. (2023)	Clinical texting communication	Content Analysis	Emoji are used primarily to convey new and interactionally salient information. The majority functioned emotively, that is, conveyed the internal state of the sender, and served to open, maintain, or close communication.
6	Assessing personality using emoji: An exploratory study. (2017)	Personality assessment and mental health	Web-based survey	Emoji might be useful in psychological assessment for the study of personality traits, especially those with known connections to emotional expression and affect. Emoji should be explored further as they have great potential to replace, at least in part, the traditional instruments to assess individual personality differences.
7	How has the coronavirus (COVID-19) pandemic affected global emoji usage? (2021)	COVID-19 and the emoji usage pattern	Content Analysis	The average usage of emoji by countries that are most affected by the pandemic dropped. People associate emoji with a (comparatively) lighthearted conversation, whereas the pandemic calls for more serious expressions, where emoji may be inadequate.
8	Long Covid: Online patient narratives, public health communication and vaccine hesitancy. (2021)	COVID-19	Content Analysis	The use of emoji in the COVID-19 wave at different time periods is generally associated with sadness, crying, sarcasm, incredulity, and anxiety. Emoji frequency varied and diversified from one Emoji to another.
Emoji scale application in health communication
1	Health beliefs towards kefir correlate with emotion and attitude: A study using an emoji scale in Brazil. (2020)	Probiotic beverage selection and evaluation	Laboratory experiments	Valence and arousal were moderated by the health benefits of kefir. Positive emotions increased when participants were re-exposed to milk beverages with information (0%, 15%, 30%, and 50% m/v), while negative emotions decreased. This conclusion is based on the emotional associations of the emoji scale.
2	Measuring consumers’ product associations with emoji and emotion word questionnaires: Case studies with tasted foods and written stimuli. (2018)	Tasted foods and emotional associations	Questionnaire of a central location test	Overall, emoji questionnaires were more discriminative, with tasted foods and written stimuli. For written stimuli describing negative consumption situations, emoji questionnaires performed better.
3	Development of the Emoji Faces Pain Scale and Its Validation on Mobile Devices in Adult Surgery Patients: Longitudinal Observational Study. (2023)	Adult patients who underwent surgery and mobile devices	A Delphi technique with web-based survey	A 6-level Emoji-FPS (Faces Pain Scale) was developed. Satisfactory validity and reliability of the Emoji-FPS were confirmed in patients who underwent perianal surgery.
4	The emoji current mood and experience scale: The development and initial validation of an ultra-brief, literacy independent measure of psychological health. (2022)	Mental health, wellbeing, resilience, and community connection	A cross-sectional online study and secondary validation	The studies reported provide robust initial evidence for the use of a short emoji-based tool with minimal literacy requirements for the measurement of a range of psychosocial domains including aspects of mental health, well-being, community connection, and resilience.
5	Development of an emoji-based self-report measurement tool to measure emotions elicited by foods in preadolescents. (2022)	Emotions in response to food products	Questionnaire development	An emoji-based self-report questionnaire with a food-specific emoji list was developed. 17 emoji pairs were associated with specific semantic and dimensional meanings. The questionnaire can be used to study preadolescents’ emotions elicited by foods.
6	“Emoji, I can feel your pain”–Neural responses to facial and Emoji expressions of pain. (2021)	Event-related brain potential	Laboratory experiments	Emoji enable similar neural responses as faces in pain perception, even though they may not be as potent as human faces, especially during the late-stage social cognitive processing.
7	Development of novel emoji scale to measure patient-reported outcomes in cancer patients. (2018)	Patient-reported outcomes	Laboratory experiments	Emoji responses were significantly associated with validated measures of patient-reported outcomes; the ordinal emoji scale is negatively related to fatigue and positively related to physical well-being and emotional well-being. 92% reported that they would use the Emoji Scale again, and 89% would recommend others to use the Emoji Scale.

Appendix C. List of X (Twitter) Accounts

IJDesign
Vol 18, No 2 (2024)

Visual-Textual In...

Reading Tools

Navigation Menu

Navigation Menu

IJDesignVol 18, No 2 (2024)

Visual-Textual In...

Reading Tools

Navigation Menu

Navigation Menu

IJDesign
Vol 18, No 2 (2024)