Visual Instruction Tuning using Richly Decorated Traffic Volume Heatmaps

Abstract

In addition to advancements in Large Language Models (LLM), there is a growing body of literature highlighting the incremental enhancement of zero-shot and few-shot performance achieved through Instruction Tuning within the domain of Large Multimodal Models (LMM). While existing research predominantly emphasizes the broad applicability of these models across diverse benchmarks, our focus is distinctly directed towards a task-specific context: predicting future traffic volume. Specifically, our study contributes findings on the effective application of Instruction Tuning to improve predictions of traffic volumes during specific temporal intervals, such as rush hours or weekends. This improvement is facilitated by transforming traffic volume heatmaps overlaid on maps into more intricate images that integrate additional information, including date and time details, latitude and longitude coordinates, and a comprehensive color scale.