Automated Topic Detection & Clustering: Identify What Matters In A Glance

We’ve just added Topic Clustering to our ever and fast-growing list of powerful features in 20/Twenty – our comprehensive social listening and monitoring solution.

With this all-new Top Topics and clustering feature, you can now identify the key conversation topics across huge datasets in an instant. Simply define your filters and hit ‘Apply’ to see the top topics of the day (or a custom duration) unravel!

Discover top topics in an instant, with the power of clustering with Circus Social's 20/Twenty social listening and monitoring tool

Identify Top Topics With Auto-Categorization

Powered by 20/Twenty’s intelligent clustering algorithm built using powerful AI, you can now view conversations auto-grouped into relevant categories as part of our robust social listening.

Eliminating the need for you to trawl manually through thousands of posts, you can reach data insights much faster and cut down your research time in half. With intelligently surfaced topics, you can quickly identify the top areas of online conversation that need your attention.

Deep-dive into each auto-surfaced topic cluster to discover the top keywords behind each cluster and right-click to view the actual conversations behind these topic groups.

Access powerful topic clustering with social listening via 20/Twenty

Identify Topic Sentiments In A Glance

Detect top negative or positive sentiment topics quickly and deep dive to discover more topic groups or keywords behind these. Right-click to view raw posts.

Identify top sentiments from topic clustering powered by 20/Twenty AI and machine learning
Identify top sentiments from topic clustering powered by 20/Twenty AI and machine learning








Using this nifty and uber-powerful feature, you can quickly isolate problem, concern or issue areas associated with your brands or products and address these points via optimized communication or communications and product strategy to your customers. Perfect for those PR or crisis management moments.

How Does This Work?

20/Twenty utilizes huge and continuously growing datasets comprising of posts, conversations and data points across all supported social media and data sources to train it’s machine learning algorithms and power our AI.

All the data that streams into our platform is intelligently classified by our Natural Language Processing (NLP) Engine, into more than 180 clusters across multiple industries, based on the content in the post. This map of clusters can be visualized at multiple levels in order to get a quick overview of the different themes of conversations with their overall sentiment, impact and traction. Each of these clusters can also be explored in detail, to see the individual conversations that fall under it.

Access powerful topic clustering with social listening via 20/Twenty

Topic Clustering Hierarchy

More Filters = More Power

These topics are automatically generated based on your filter settings. This is based on 20/Twenty’s advanced clustering algorithm. Top 10 topics along with sentiments are displayed. You can deep-dive into each parent topic to identify the top 10 keywords and phrases driving these topics. Right-click on any topic to view the actual conversations behind these topics.

You can further sort these via built-in performance metrics on 20/Twenty such as impact, traction, and post volume to get to the most impactful conversation clusters in an instant.

Topic Clustering In Action

Here are some auto-topic clusters identified across various industries –

Singapore Government

FMCG

Automotive

There’s more where that came from! Stay tuned and keep an eye out for more cool updates soon to come.

If you’ve got any questions or would like to find out how Topic Clustering and how 20/Twenty’s machine learning and AI prowess can help you and your business, reach out to us at insights@circussocial.com or contact us here and we’ll be in touch!

If you’re a 20/Twenty user, visit our Help & Support Hub to find out more about topic clustering.

Check out how the 20/Twenty Social Monitoring platform has grown!

Growth of 20/Twenty Social Monitoring & Intelligence Platform – Under the Hood

Growth of 20/Twenty Social Media Monitoring & Intelligence Platform – Under the Hood

We have often been asked about the challenges we faced in scaling up our technology stack to manage big data. I have attempted to address this in this post which is the first of a series of blog posts on this and similar topics.

20/Twenty was created ground-up as the most intuitive and easy to use cloud based (SaaS) Social Media Monitoring & Intelligence platform in the world.   Based on our deep understanding of what marketers needed and the awesome designs we created, we signed up our first client even before the product was officially launched. The pressure to quickly deliver the first version of the product was intense 🙂

From an engineering point of view, there’s a huge amount of data that we pull (Think Big Data!), process, augment and then visualize in the platform all on a near real-time basis. Imagine someone tweeting and it appears on our platform within a few seconds along with augmented information including Gender, Sentiment, Engagement, Spam score etc.

The evolution of 20/Twenty has already seen a few stages of growth. The graph below shows how 20/Twenty data has grown over the last 2 years since our product launch. This is a really cool growth for a startup like Circus Social both from a business perspective as well as from an engineering standpoint. We used several tricks from the books as well as a few practical hacks to ensure our ability to fetch, process, augment and visualize high volumes of data continued to become better, though this journey was not without pain!

social-intelligence-20twenty-big-data-growth

Stage 1

We created over 200 custom marketing applications in our previous avatar at Circus Social working with some of the biggest brands in the world. We used the same open source technologies (PHP / MySQL) to create the first version of 20/Twenty. This worked well and as our data grew in the first few months, we continued to grow vertically by adding more capacity (CPU/RAM).

Most of the queries from the application were read queries whereas a bulk of “write operations” were being performed by our data crawlers. We therefore created an efficient master-slave architecture where the application would read from the slaves and the crawler scripts would write into the master. This worked well in general but the exponential increase in the volume of data meant that certain queries were running extremely slow and impacting the user experience.

Stage 2

Since our data volume was growing exponentially and the relational aspects of the database were not the core of our application, we realized that sooner or later, we would have to move to a NoSQL database. However, the performance issues that were cropping up had to be sorted quickly and without a downtime. We quickly realized that we needed a dedicated search engine and MySQL was not good enough for this purpose.

We explored several options and Elasticsearch came to our rescue here. Elasticsearch is a distributed, RESTful search and analytics engine that centrally stores your data in a manner which can be retrieved / read really fast by your applications. Our awesome tech team deployed this in a matter of days. The improvement in performance was remarkable. The plan worked and we cheered!

Stage 3

Word spread in Singapore and Asia about how good our platform was (and our sales team did a good job too!) and we continued to sign up new clients. The volume of data continued to grow for existing clients as well as new clients. The tech stack of MySQL and ElasticSearch did not let us down but we wanted to create an architecture that would scale infinitely, if there’s a thing like that.

In Stage 3, we moved the core of our database from MySQL to Cassandra (Elasticsearch was now interacting with Cassandra) and the backend code from PHP to Node.js. We also migrated most of our front end code to Angular.js for better performance. This was a major architectural change on a live application being used by several clients so we created a parallel production like environment and ran it parallelly for several weeks to ensure everything was working as desired before switching over.

While we did the above, we continued to work on cool new features on the product and opened up our data API’s to a few clients who wanted a deeper integration with their own applications. Other tools we used during this and other stages were Postman, Github and JIRA.

As we scale further from here, we will probably have newer and more exciting technology challenges and we will keep posting about them. If you are excited to work on some of these, do write to us at joinus@circussocial.com