TALL – Text Analysis for ALL, a new R Shiny app for NLP and Text Mining workflows

Interested in publishing a one-time post on R-bloggers.com? Press here to learn how.

TALL – Text Analysis for ALL is an R Shiny app that includes a wide set of methodologies specifically tailored for various text analysis tasks. It aims to address the needs of researchers without extensive programming skills, providing a versatile and general-purpose tool for analyzing textual data. With TALL, researchers can leverage a wide range of text analysis techniques without the burden of extensive programming knowledge, enabling them to extract valuable insights from textual data in a more efficient and accessible manner.

Setup

TALL can be installed in two ways, depending on whether you want the stable version or the latest development version.

Official release

You can install the official release of TALL from the Comprehensive R Archive Network CRAN and updated monthly.

if (!require("pak", quietly=TRUE)) install.packages("pak")
pak::pkg_install("tall")

Development release

If you want access to the most recent features and updates not yet available on CRAN, you can install the development version directly from our GitHub repository with:

if (!require("pak", quietly=TRUE)) install.packages("pak")
pak::pkg_install("massimoaria/tall")

Run Tall

Load the library with:

library("tall")

and then run TALL shiny app with:

tall()

Introduction

In the age of information abundance, researchers across diverse disciplines are confronted with the formidable task of analyzing voluminous textual data. Textual data, encompassing research articles, social media posts, customer reviews, and survey responses, harbors invaluable insights that can propel knowledge advancement in various fields, ranging from social sciences to healthcare and beyond. Researchers endeavor to analyze textual data to unveil patterns, discern trends, extract meaningful information, and gain deeper understandings of diverse phenomena. By leveraging sophisticated natural language processing (NLP) techniques and machine learning algorithms, researchers can delve into the semantic and syntactic structures of texts, perform topic detection, polarity detection, and text summarization, among other analyses. Additionally, the advent of digital platforms and the exponential growth of online content have generated unprecedented volumes of textual data that were previously inaccessible or challenging to acquire.

Researchers can harness the power of these textual resources to delve into novel research questions, corroborate existing theories, and generate groundbreaking insights. Through the utilization of computational tools and methodologies, researchers can efficiently process and analyze expansive volumes of text, substantially reducing the time and effort expended compared to manual analysis. Furthermore, there is a burgeoning recognition of the need for text analysis tools tailored to individuals who may not possess in-depth programming expertise. While programming languages like R and Python offer powerful capabilities for data analysis, not all researchers have the time or resources to acquire proficiency in these languages. To address this challenge, a growing number of user-friendly text analysis tools have emerged, providing researchers with a viable alternative to traditional programming-based approaches. These tools empower researchers from diverse backgrounds to effectively process and analyze textual data, fostering a more inclusive research environment and democratizing access to the transformative power of text analysis.

For researchers who lack programming skills, TALL offers a viable solution, providing an intuitive interface that allow researchers to interact with data and perform analyses without the need for extensive programming knowledge.

TALL offers a comprehensive workflow for data cleaning, pre-processing, statistical analysis, and visualization of textual data, by combining state-of-the-art text analysis techniques into an R Shiny app.

TALL workflow

First TALL seamlessly integrates the functionalities of a suite of R packages designed for NLP tasks with the user-friendly interface of web applications through the Shiny package environment.

The TALL workflow streamlines the discovery and analysis of textual data by systematically processing and exploring its content. This comprehensive framework empowers researchers with a versatile toolkit for text analysis, enabling them to efficiently navigate and extract meaningful insights from large volumes of textual data.

By leveraging the strengths of both R packages and Shiny’s interactive web interface, TALL provides a powerful and accessible platform for researchers to conduct thorough the following workflow:

  1. Import and Manipulation

  2. Pre-processing and Cleaning

  3. Statistical Text Analysis and Dynamic Visualization

Some screenshot from TALL

Import text from multiple file formats

Edit, split, and add external information

Automatic Lemmatization and PoS-Tagging through LLM

Language, Model, and Analysis Term Selection

Tagging Special Entities through multiple regex

 

Semantic Tagging

Automatic Multi-word creation

Multi-word creation by a list and Custom Term List

OVERVIEW – Descriptive statistics, concordance analysis and word frequency distributions

WORDS – Multiple methods for Topic Detection

DOCUMENTS – Main approaches for entire texts

Published by

Massimo Aria

I’m Massimo Aria, full professor of Statistics for Social Sciences at the Department of Economics and Statistics of the University of Naples Federico II. My research focuses on bibliometrics, science mapping, text mining, statistical modelling, and machine learning. I am the author of over 150 scientific articles, and creator of several R packages including the widely used Bibliometrix and Tall. I am also the co-founder of K-Synth, an academic spin-off dedicated to transforming data into actionable knowledge through innovative statistical and computational methods. Over the years, I’ve been involved in numerous national and international research projects, and I actively support open science and reproducible research.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.