The Time Detective’s™ Game uses an immense and growing timeline to provide an interactive learning experience for it’s players. As the timeline continues to grow, managing the events and generating insights becomes a challenging task. To overcome this challenge, I’ve developed an intelligent event categorizer and similarity analyzer using advanced natural language processing techniques. This project not only helps in categorizing events automatically but also provides the maintainer of the database with valuable insights by analyzing the similarity between different events and offering a comprehensive view of the timeline through a NetworkX graph.
- Event Categorization: To automatically categorize events, we leveraged the power of the zero-shot classification model from Hugging Face, specifically the bart-large-mnli model. The model analyzes event descriptions and predicts their categories based on the context. The implementation involved fine-tuning the model and using it to categorize events in the dataset.
- Similarity Matrix Generation: After categorizing the events, we used the SentenceTransformer library, which employs pre-trained models to generate embeddings (numeric representations) for each event. These embeddings were used to calculate similarity scores between events. The process also took into account event categories and sort values, which were incorporated into the embeddings.
- NetworkX Graph Generation: With the similarity matrix in hand, we created a NetworkX graph to visualize the relationships between events. This powerful tool enables the creator to explore events based on their similarities, providing an intuitive way to discover connections between historical events. The graph has revealed incredible insights into isolated events that require more context, categories that need more events, and how events connect to each other in history. These insights have proven invaluable in maintaining the game and timeline, ensuring that the events generated have historical substance and offer a rich learning experience for players.
- Flask Web Application: To make the entire process user-friendly, a Flask web application was constructed that allows users to manage the DynamoDB database and run the event categorization, similarity matrix generation, and NetworkX graph creation tasks. The web application includes buttons to execute each task, with a confirmation message displayed upon completion.
The Time Detective’s intelligent event categorizer and similarity analyzer provide an efficient solution for the creator to manage and explore historical events. By leveraging cutting-edge NLP models and similarity analysis techniques, the platform offers a rich toolkit to maintain the game and timeline. The NetworkX graph, in particular, has proven to be an indispensable resource, enabling the creator to identify gaps, improve event connections, and ensure that the platform continues to deliver a high-quality, engaging experience for players. Want to visualize the final results in your browser? The latest graph is available for visualization here (It takes a while to load, click “wait” when prompted to wait or kill).