I think in a framework like Storm your data would be "flowing" through your cluster with little persistence on disk.