Minister's Innovation Award - Merit
Sentiment Analysis for Public Transport (SAPT)
Quick-sensing of Commuter Sentiments through Data Science

Impetus

For Public Transport Council (PTC) to understand ground sentiments effectively, there is a need to monitor citizens’ voices within digital space. In this regard, local citizenry and online journalism through social media are increasing rapidly. The number of online comments can grow within a short time when the content goes viral. This presents a need to seed technology capability to enable quick sensing of social media comments across various channels.

To distil social media comments across various channels manually is highly laborious. Unlike structured numerical data, which can be processed relatively quickly with extremely high precision, the analysis of unstructured text data, on the other hand, is significantly more complex since accurate interpretation depends on numerous factors such as linguistic context. Notably, substantial effort is required to identify the relevant posts using keywords and harvest their comments. Subsequently, each of those comments, which could be in the thousands, had to be understood to develop context and establish key topics and sentiments.

Most commercial platforms available in the market typically are not readily equipped to cater for local colloquial language (e.g. Singlish and sarcasm), which affects the accuracy and reliability of the analysis. This aspect is particularly crucial as its use is widespread in the local social media. Therefore, in order to handle such complexities, AI and natural language processing (NLP) techniques using local datasets are required.

Statement of Need

The Sentiment Analysis for Public Transport (SAPT) project aims to establish an accurate digital tool that is capable of enabling quick-sensing of commuter sentiments to support PTC’s policy-making process. This is achieved through the analysis of unstructured social media content with cognisance to colloquial language (e.g. Singlish) and sarcasm.

Innovation

The SAPT project began tackling the challenge of establishing the comment sentiments and topics within a short amount of time. The first phase was addressed through a procurement of a basic desktop-based sentiment analysis tool that uses A*STAR’s technology which can produce fine-grained classification of the sentiment and emotions in unstructured text-based content. It does so by using built-in libraries of colloquial language and a sarcasm detection model incorporating local language lexicons.

To achieve better efficiency and accuracy, the team further partnered with A*STAR to develop the second-generation, advanced social media listening platform (“Resonance Social”) where PTC contributed lead use cases, pilot testing of features, and suggested new features.

Developed by engineers and scientists from A*STAR’s Institute of High Performance Computing, Resonance Social is powered by state-of-the-art multidimensional sentiment and emotion analysis engine known as “CrystalFeel”, rooted in their affective computing research. As an integrative platform, Resonance Social presents accurate, real-time, and interactive emotion analytic dashboards from multiple digital and social media data sources. Analysis results are then displayed on a configurable dashboard with various visualisations.

Resonance Social Dashboard

End-result Statement

A web-based sentiment analysis tool that is equipped for local Singlish content, is used to identify relevant social media content based on topics of interest and enable quick sensing through fine-grained sentiment and emotions analysis. In parallel, the PTC analytics team has also been trained and equipped with the necessary digital skillsets (i.e. topic modelling) to effectively handle the full spectrum of tasks required in sentiment analysis.

Impact

Between manual human interpretation and tools-based sentiment analysis, the time taken to interpret a dataset of about 100 comments (or total of 5,000 words) can be reduced from about 120 minutes to about 5 minutes. Using a basic desktop-based sentiment analysis tool, internal tests using out-of-sample local content show that the classification accuracy is up to 64%, which is sufficient for quick sensing and building context.

There is still significant time to gather and compile social media data and visualise the analysis results. Through collaboration with A*STAR on Resonance Social, the total data analysis time is further reduced by up to 83%. Furthermore, the classification accuracy, tested against transport-related local social media data, is improved by up to 72% due to A*STAR’s new multidimensional emotion and sentiment analysis engine, “CrystalFeel”, which is comparable with leading tools in the market.

Beyond the technical delivery and performance of the system, the project also serves as a springboard to enhance the digital skillsets of PTC officers. In particular, sentiment analysis will henceforth form part of the functional training for relevant divisions within PTC, which will be equipped with the necessary training and software. Moving forward, where needed, PTC also intends to share the technology and basic skillsets with the rest of the Ministry of Transport (MOT) family.

Outcome Statement

Through this project, the team has partnered A*STAR to develop an accurate and customisable sentiment analysis tool that enables quick sensing of social media content which is cognisant of colloquial language. With the significant reduction in data collection and analysis time, the team can focus on curating more potential data sources and use cases. The end-to-end, one-stop, web-based sentiment analysis platform also enables the team to learn basic sentiment, emotions and big social data analytics concepts and skillsets and leverage such technology to support PTC’s policy-making process.

^ source: Statistica, July 2021, statistica.com, Retrieved from https://www.statista.com/statistics/489234/number-of-social-network-users-in-singapore/

* based on DOS Singapore population of 5.69m as at June 2020