apache spark - Proper save/load of MatrixFactorizationModel -


i have matrixfactorizationmodel object. if i'm trying recommend products single user right after constructing model through als.train(...) takes 300ms (for data , hardware). if save model disk , load recommendation takes 2000ms. spark warns:

15/07/17 11:05:47 warn matrixfactorizationmodel: user factor not have partitioner. prediction on individual records slow. 15/07/17 11:05:47 warn matrixfactorizationmodel: user factor not cached. prediction slow. 15/07/17 11:05:47 warn matrixfactorizationmodel: product factor not have partitioner. prediction on individual records slow. 15/07/17 11:05:47 warn matrixfactorizationmodel: product factor not cached. prediction slow. 

how can create/set partitioner , cache user , product factors after loading model? following approach didn't help:

model.userfeatures().cache(); model.productfeatures().cache(); 

also trying repartition rdds , create new model repartitioned versions didn't help.

you don't have use parenthesis, userfeatures rdd of (int, array[double]) not take parameters.

this you:

model.userfeatures.cache model.productfeatures.cache 

Comments