Accelerating Sequential Pattern Mining on Spark: A SPADE-Based Approach

Abstract

This letter presents a vertical-based adaptation of SPADE on Spark that significantly minimizes inter-worker communication. We achieve up to 6.2 × speedup over Spark MLlib’s PrefixSpan, enabling more efficient sequential pattern mining with minimal data movement and strong performance in distributed environments.