Typically, a user-focused approach of evaluation of recommender systems requires the users to recollect their experiences, exposing study results to memory biases. In this paper, we describe a study conducted to test a framework, that allows recommender systems to be used and evaluated simultaneously. In this study, we asked 140 participants about their expected, perceived, and actual quality of the recommendations. We compare the performance of two recommender systems. The singular value decomposition recommendation system was able to correctly predict more than half of all evaluations and performed better than participants expected. However, users were more satisfied with the suggestions of the user-based collaborative filtering recommendation system. Our approach allows to compare actual item ratings, expected quality, and perceived quality of recommendations. Serendipity was …