{"id":24905,"date":"2024-01-25T09:46:13","date_gmt":"2024-01-25T14:46:13","guid":{"rendered":"https:\/\/design.ncsu.edu\/graphic-design\/2024\/01\/25\/how-can-a-i-transform-voice-language-analysis\/"},"modified":"2025-04-01T03:32:29","modified_gmt":"2025-04-01T07:32:29","slug":"how-can-a-i-transform-voice-language-analysis","status":"publish","type":"post","link":"https:\/\/design.ncsu.edu\/graphic-design\/2024\/01\/25\/how-can-a-i-transform-voice-language-analysis\/","title":{"rendered":"How can A.I. Transform Voice Language Analysis?"},"content":{"rendered":"\n<p>Master of Graphic &amp; Experience Design (MGXD) students  collaborated with the&nbsp;<a href=\"https:\/\/ncsu-las.org\/\">Laboratory of Analytic Sciences (LAS)<\/a>&nbsp;here at NC State to explore the potential of machine learning to assist in voice language analysis tasks.<\/p>\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/MGXD_Landing_2024-1024x579.jpg\" alt=\"Graduate students working together in a studio space\" class=\"wp-image-35809\" \/><\/figure>\n\n\n<p>As AI\/ML technologies becomes more prevalent in modern workflows, new conveniences emerge even as some cognitive burdens increase. The advent of automatic speech recognition and other AI\/ML technologies, for example, have not only added more tasks to the language analysts\u2019 workflow, but unleashed new possibilities for how to go about information retrieval and exploration. In this project MGXD students used human-centered research methods to prototype a unified environment for analysts that enables them to take full advantage of novel technologies while seamlessly pivoting between the various stages of their workflow.<\/p>\n\n\n<p>Although the research team focused specifically on voice language analysis in the intelligence community for this project, there are many applications for this kind of interface across other domains: helping hearing impaired individuals understand the nuance of a conversation; facilitating better communication in a business or government policy meetings across cultures; helping academic researchers gather and analyze materials from a vast array of lectures and conversations; capturing and preserving the nuances of oral languages, etc. Machine learning is opening up powerful capabilities for engaging with human language in new ways to provide critical insight and enable action.<\/p>\n\n\n<p>The core research questions of this project:<\/p>\n\n\n<p><strong>How might the design of an interface use the affordances of ML to enable&nbsp;<\/strong><strong>voice language analysts to quickly produce reliable and robust intelligence that accurately conveys content, intent, and context?<\/strong><\/p>\n\n\n<p><em>Students were asked to:<\/em><\/p>\n\n\n<ul class=\"wp-block-list\">\n<li>Envision a unified interface environment<\/li>\n\n\n\n<li>Prototype new AI-powered ways of searching &amp; finding<\/li>\n\n\n\n<li>Visualize patterns in the data<\/li>\n\n\n\n<li>Focus on language analyst needs\/desires to create an enjoyable user experience<\/li>\n<\/ul>\n\n\n<p><em>This material is based upon work done, in whole or in part, in coordination with the Department of Defense (DoD). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the DoD and\/or any agency or entity of the United States Government.<\/em><\/p>\n\n\n<h2 class=\"wp-block-heading\">Final Prototypes<\/h2>\n\n\n<h2 class=\"wp-block-heading\">Human-Centered Design Process<\/h2>\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide5.jpeg\" alt=\"Three female students and a mentor from LAS looking at a laptop as they work through a language analysis exercise\" class=\"wp-image-35791\" \/><\/figure>\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide6.jpeg\" alt=\"Personas: Scanner\/Sloane: Identifies information or activity that answers all or part of an intelligence requirement and meets reporting thresholds. Translator\/Cameron: Translates foreign language material into coherent English QCer\/Ferris: conducts quality control review on summaries and translations, and the reports that reference them.\" class=\"wp-image-35792\" \/><\/figure>\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide8.jpeg\" alt=\"images of personas and scenarios written out and diagrammed\" class=\"wp-image-35793\" \/><\/figure>\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide9.jpeg\" alt=\"AS IS user journey mpas: Image of diagrammed user journey map\" class=\"wp-image-35794\" \/><\/figure>\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide10.jpeg\" alt=\"Ideation\/What IF: students sketching around a table using &quot;What If&quot; questions as prompts\" class=\"wp-image-35795\" \/><\/figure>\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide11.jpg\" alt=\"Benchmarking: images of lots of different types of data visualizations\" class=\"wp-image-35796\" \/><\/figure>\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide12.jpg\" alt=\"Sketches: many early sketches for the project\" class=\"wp-image-35797\" \/><\/figure>\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide13.jpg\" alt=\"Feature Sets: Image of post it notes in MIRO that cluster interface features together in logical groups\" class=\"wp-image-35799\" \/><\/figure>\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide15.jpeg\" alt=\"Critiques: Images of early work and students and analysts discussing\" class=\"wp-image-35800\" \/><\/figure>\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide17.jpeg\" alt=\"Hi-fi Prototypes &amp; Scenario Videos\" class=\"wp-image-35801\" \/><\/figure>\n\n\n<p><em>This story was originally published on the website of Professor of Graphic &amp; Experience Design&nbsp;<a href=\"https:\/\/www.helenarmstrongdesigner.com\/2021\/09\/09\/using-design-thinking-to-investigate-the-potential-of-knowledge-graphs\/\" target=\"_blank\" rel=\"noreferrer noopener\">Helen Armstrong<\/a>.<\/em><\/p>\n<p><em>This post was <a href=\"https:\/\/design.ncsu.edu\/blog\/2024\/01\/25\/how-can-a-i-transform-voice-language-analysis\/\">originally published<\/a> in College of Design Blog.<\/em><\/p>","protected":false,"raw":"<!-- wp:paragraph -->\n<p>Master of Graphic &amp; Experience Design (MGXD) students  collaborated with the&nbsp;<a href=\"https:\/\/ncsu-las.org\/\">Laboratory of Analytic Sciences (LAS)<\/a>&nbsp;here at NC State to explore the potential of machine learning to assist in voice language analysis tasks.<\/p>\n<!-- \/wp:paragraph -->\n<!-- wp:image {\"id\":35809,\"sizeSlug\":\"large\",\"linkDestination\":\"none\"} -->\n<figure class=\"wp-block-image size-large\"><img src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/MGXD_Landing_2024-1024x579.jpg\" alt=\"Graduate students working together in a studio space\" class=\"wp-image-35809\" \/><\/figure>\n<!-- \/wp:image -->\n<!-- wp:paragraph -->\n<p>As AI\/ML technologies becomes more prevalent in modern workflows, new conveniences emerge even as some cognitive burdens increase. The advent of automatic speech recognition and other AI\/ML technologies, for example, have not only added more tasks to the language analysts\u2019 workflow, but unleashed new possibilities for how to go about information retrieval and exploration. In this project MGXD students used human-centered research methods to prototype a unified environment for analysts that enables them to take full advantage of novel technologies while seamlessly pivoting between the various stages of their workflow.<\/p>\n<!-- \/wp:paragraph -->\n<!-- wp:paragraph -->\n<p>Although the research team focused specifically on voice language analysis in the intelligence community for this project, there are many applications for this kind of interface across other domains: helping hearing impaired individuals understand the nuance of a conversation; facilitating better communication in a business or government policy meetings across cultures; helping academic researchers gather and analyze materials from a vast array of lectures and conversations; capturing and preserving the nuances of oral languages, etc. Machine learning is opening up powerful capabilities for engaging with human language in new ways to provide critical insight and enable action.<\/p>\n<!-- \/wp:paragraph -->\n<!-- wp:paragraph -->\n<p>The core research questions of this project:<\/p>\n<!-- \/wp:paragraph -->\n<!-- wp:paragraph -->\n<p><strong>How might the design of an interface use the affordances of ML to enable&nbsp;<\/strong><strong>voice language analysts to quickly produce reliable and robust intelligence that accurately conveys content, intent, and context?<\/strong><\/p>\n<!-- \/wp:paragraph -->\n<!-- wp:paragraph -->\n<p><em>Students were asked to:<\/em><\/p>\n<!-- \/wp:paragraph -->\n<!-- wp:list -->\n<ul class=\"wp-block-list\"><!-- wp:list-item -->\n<li>Envision a unified interface environment<\/li>\n<!-- \/wp:list-item -->\n\n<!-- wp:list-item -->\n<li>Prototype new AI-powered ways of searching &amp; finding<\/li>\n<!-- \/wp:list-item -->\n\n<!-- wp:list-item -->\n<li>Visualize patterns in the data<\/li>\n<!-- \/wp:list-item -->\n\n<!-- wp:list-item -->\n<li>Focus on language analyst needs\/desires to create an enjoyable user experience<\/li>\n<!-- \/wp:list-item --><\/ul>\n<!-- \/wp:list -->\n<!-- wp:paragraph -->\n<p><em>This material is based upon work done, in whole or in part, in coordination with the Department of Defense (DoD). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the DoD and\/or any agency or entity of the United States Government.<\/em><\/p>\n<!-- \/wp:paragraph -->\n<!-- wp:heading -->\n<h2>Final Prototypes<\/h2>\n<!-- \/wp:heading -->\n<!-- wp:heading -->\n<h2>Human-Centered Design Process<\/h2>\n<!-- \/wp:heading -->\n<!-- wp:image {\"id\":35791,\"sizeSlug\":\"full\",\"linkDestination\":\"none\"} -->\n<figure class=\"wp-block-image size-full\"><img src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide5.jpeg\" alt=\"Three female students and a mentor from LAS looking at a laptop as they work through a language analysis exercise\" class=\"wp-image-35791\" \/><\/figure>\n<!-- \/wp:image -->\n<!-- wp:image {\"id\":35792,\"sizeSlug\":\"full\",\"linkDestination\":\"none\"} -->\n<figure class=\"wp-block-image size-full\"><img src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide6.jpeg\" alt=\"Personas: Scanner\/Sloane: Identifies information or activity that answers all or part of an intelligence requirement and meets reporting thresholds. Translator\/Cameron: Translates foreign language material into coherent English QCer\/Ferris: conducts quality control review on summaries and translations, and the reports that reference them.\" class=\"wp-image-35792\" \/><\/figure>\n<!-- \/wp:image -->\n<!-- wp:image {\"id\":35793,\"sizeSlug\":\"full\",\"linkDestination\":\"none\"} -->\n<figure class=\"wp-block-image size-full\"><img src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide8.jpeg\" alt=\"images of personas and scenarios written out and diagrammed\" class=\"wp-image-35793\" \/><\/figure>\n<!-- \/wp:image -->\n<!-- wp:image {\"id\":35794,\"sizeSlug\":\"full\",\"linkDestination\":\"none\"} -->\n<figure class=\"wp-block-image size-full\"><img src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide9.jpeg\" alt=\"AS IS user journey mpas: Image of diagrammed user journey map\" class=\"wp-image-35794\" \/><\/figure>\n<!-- \/wp:image -->\n<!-- wp:image {\"id\":35795,\"sizeSlug\":\"full\",\"linkDestination\":\"none\"} -->\n<figure class=\"wp-block-image size-full\"><img src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide10.jpeg\" alt=\"Ideation\/What IF: students sketching around a table using &quot;What If&quot; questions as prompts\" class=\"wp-image-35795\" \/><\/figure>\n<!-- \/wp:image -->\n<!-- wp:image {\"id\":35796,\"sizeSlug\":\"full\",\"linkDestination\":\"none\"} -->\n<figure class=\"wp-block-image size-full\"><img src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide11.jpg\" alt=\"Benchmarking: images of lots of different types of data visualizations\" class=\"wp-image-35796\" \/><\/figure>\n<!-- \/wp:image -->\n<!-- wp:image {\"id\":35797,\"sizeSlug\":\"full\",\"linkDestination\":\"none\"} -->\n<figure class=\"wp-block-image size-full\"><img src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide12.jpg\" alt=\"Sketches: many early sketches for the project\" class=\"wp-image-35797\" \/><\/figure>\n<!-- \/wp:image -->\n<!-- wp:image {\"id\":35799,\"sizeSlug\":\"full\",\"linkDestination\":\"none\"} -->\n<figure class=\"wp-block-image size-full\"><img src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide13.jpg\" alt=\"Feature Sets: Image of post it notes in MIRO that cluster interface features together in logical groups\" class=\"wp-image-35799\" \/><\/figure>\n<!-- \/wp:image -->\n<!-- wp:image {\"id\":35800,\"sizeSlug\":\"full\",\"linkDestination\":\"none\"} -->\n<figure class=\"wp-block-image size-full\"><img src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide15.jpeg\" alt=\"Critiques: Images of early work and students and analysts discussing\" class=\"wp-image-35800\" \/><\/figure>\n<!-- \/wp:image -->\n<!-- wp:image {\"id\":35801,\"sizeSlug\":\"full\",\"linkDestination\":\"none\"} -->\n<figure class=\"wp-block-image size-full\"><img src=\"https:\/\/design.ncsu.edu\/wp-content\/uploads\/2023\/12\/Slide17.jpeg\" alt=\"Hi-fi Prototypes &amp; Scenario Videos\" class=\"wp-image-35801\" \/><\/figure>\n<!-- \/wp:image -->\n<!-- wp:paragraph -->\n<p><em>This story was originally published on the website of Professor of Graphic &amp; Experience Design&nbsp;<a href=\"https:\/\/www.helenarmstrongdesigner.com\/2021\/09\/09\/using-design-thinking-to-investigate-the-potential-of-knowledge-graphs\/\" target=\"_blank\" rel=\"noreferrer noopener\">Helen Armstrong<\/a>.<\/em><\/p>\n<!-- \/wp:paragraph -->"},"excerpt":{"rendered":"<p>Master of Graphic &#038; Experience Design (MGXD) students  collaborated with the\u00a0Laboratory of Analytic Sciences (LAS)\u00a0here at NC State to explore the potential of machine learning to assist in voice language analysis tasks.<\/p>\n","protected":false},"author":270,"featured_media":24906,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"source":"ncstate_wire","ncst_custom_author":"","ncst_show_custom_author":false,"ncst_dynamicHeaderBlockName":"","ncst_dynamicHeaderData":"","ncst_content_audit_freq":"","ncst_content_audit_date":"","ncst_content_audit_display":false,"ncst_backToTopFlag":"","footnotes":""},"categories":[1],"tags":[5],"class_list":["post-24905","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","tag-_from-newswire-collection-271"],"displayCategory":null,"acf":{"ncst_posts_meta_modified_date":null},"_links":{"self":[{"href":"https:\/\/design.ncsu.edu\/graphic-design\/wp-json\/wp\/v2\/posts\/24905","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/design.ncsu.edu\/graphic-design\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/design.ncsu.edu\/graphic-design\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/design.ncsu.edu\/graphic-design\/wp-json\/wp\/v2\/users\/270"}],"replies":[{"embeddable":true,"href":"https:\/\/design.ncsu.edu\/graphic-design\/wp-json\/wp\/v2\/comments?post=24905"}],"version-history":[{"count":3,"href":"https:\/\/design.ncsu.edu\/graphic-design\/wp-json\/wp\/v2\/posts\/24905\/revisions"}],"predecessor-version":[{"id":25124,"href":"https:\/\/design.ncsu.edu\/graphic-design\/wp-json\/wp\/v2\/posts\/24905\/revisions\/25124"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/design.ncsu.edu\/graphic-design\/wp-json\/wp\/v2\/media\/24906"}],"wp:attachment":[{"href":"https:\/\/design.ncsu.edu\/graphic-design\/wp-json\/wp\/v2\/media?parent=24905"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/design.ncsu.edu\/graphic-design\/wp-json\/wp\/v2\/categories?post=24905"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/design.ncsu.edu\/graphic-design\/wp-json\/wp\/v2\/tags?post=24905"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}