WEBTOON Took the First Step towards Barrier-Free Webtoon for Visually Impaired
WEBTOON Took the First Step towards Barrier-Free Webtoon for Visually Impaired
WEBTOON Took the First Step towards Barrier-Free Webtoon for Visually Impaired
- WEBTOON lowers the barrier of image-based content by introducing “AI-generated alternative (alt) text technology”
- The beta service will launch in January 2023 with 180,000 episodes in the Korean language
- The goal is to develop speaker-inferring feature and explanatory text for wordless scenes
2022-12-05
WEBTOON, the global No. 1 storytech platform, disclosed newly developed AI technology to help people with visual impairments experience webtoons. Webtoon has been inaccessible to the visually impaired, but WEBTOON’s endeavor is going to lower the barrier of image-based content in general.
In the webinar dedicated to sharing technology and know-how of open information accessibility, titled “Beyond Now, Access to the New World” held on December 5th, 2022, WEBTOON disclosed “AI-generated alternative (alt) text technology” based on AI technology for the first time.
WEBTOON’s barrier-free webtoon service is the first of its kind in the webtoon environment and is expected to contribute to expansion of barrier-free content. The company plans to launch a beta service in January 2023 with 180,000 completed and ongoing episodes, and further improve the service by looping in user feedback. The beta service will first be provided in the Korean language and will later expand to other languages.
Recently, scenes and speeches from webtoons have become ever so common and often turned into a “meme”, and increasingly more video adaptations from webtoon are introduced on various platforms. Such a phenomenon gave rise to a growing demand for original webtoons. As webtoons successfully positioned itself as a genre of popular culture, WEBTOON began to develop AI-generated alternative (alt) text technology from the second half of 2021 with the aim of making webtoons accessible for all users and having no one left out.
In the webinar, project manager Seungseob Yoon of WEBTOON gave a speech on “A small yet big progress in webtoon service for the visually impaired” and explained the intention of the service saying, “After giving a lot of thought, keeping in mind that we have users as a whole, and we should not differentiate them into the two groups of the disabled and non-disabled, we concluded that we should give it a try, even if it would be quite challenging.”
To make webtoons accessible to visually impaired people, image content should be converted to text, and then the text should be converted to sound. WEBTOON focused on finding ways to convert the image content to text in a speedy manner as there are thousands of completed titles and hundreds of ongoing episodes newly published every week. After one year of research, the team developed AI-generated alternative (alt) text technology. Text-to-speech can be provided by “VoiceOver” and “TalkBack” services pre-installed in smartphone operating systems.
AI-generated alternative (alt) text technology starts by △ distinguishing one scene from another, △ distinguishing which text is part of the speech bubble and which is part of background illustrations, △ extracting the speech, and △ figuring out which order they come in the story in relation to other speech and scenes. The team leveraged two main technologies, which are Optical Character Reader (OCR) and Webtoon Object Detection.
The majority of webtoons use a vertical scrolling view, making the image size per episode extremely large. That is why distinguishing one scene from another should come first before identifying speech area and extracting text. This process requires much attention and care because the speech, speech bubbles, and cuts need to be preserved intact so that the order of speech can be arranged properly. WEBTOON developed Webtoon Object Detection, a technology to infer cut separations, speech bubbles, and speech areas.
Once the cuts are distinguished, the Webtoon Object Detection attempts to identify speech areas. To deliver narratives in webtoons, the order of speech bubbles needs to be perceived accurately, and speeches in the background and speech bubbles need to be distinguishable. On the other hand, signboards and other background texts can disrupt the narrative when included in the perceived speech. The general-purpose OCR tends to perceive all texts in one sentence when they are closely next to each other regardless of whether they were in a speech bubble or not, resulting in confusion in the speech order. Webtoon Object Detection identifies speech areas first, before going over other texts, to reduce such errors.
The speech produced by OCR with Webtoon Object Detection is extracted to text, and then the order of speech is arranged using the location of cuts. And the script for webtoon-to-text is ready to be read. The long-term goal of WEBTOON is to add the speaker detection feature and explanatory text for wordless scenes.
While giving the presentation titled “AI-generated alternative (alt) text technology”, AI researcher Junhyun Park of WEBTOON said, “The current version is in its early stage. We will continue to develop and improve the technology until we can offer a comprehensive webtoon-to-text that delivers the narrative flawlessly.” Following Researcher Park, AI engineer Jihoon Lee gave a talk on “From AI to service”. Lee said, “Using AI technology, we expect to support accessibility to various information in a sustainable way. There is a growing social demand for better access, and this is a direction for all of us.”
“Webtoon-to-speech is part of the technological endeavor to protect diversity, a cherished value of WEBTOON,” said Daesik Kim, Executive Director of WEBTOON AI. He continued, “as the global No. 1 storytech platform, WEBTOON will continue to build an environment where all users can create and enjoy webtoons free from barriers, such as disabilities and language.”
-END-