A system, method and program product for processing user interests. A system is provided that includes: a gesture management system that receives gesture data from a collection device for an inputted interest of a user; a pattern detection system that receives and analyzes behavior data associated with the inputted interest; an interest affinity scoring system that calculates an affinity score for the inputted interest based on the gesture data and an analysis of the behavior data; a dynamic classification system that assigns a dynamically generated tag to the inputted interest based on an inputted context associated with the inputted interest; and a user interest database that stores structured interest information for the user, including a unique record for the inputted interest that includes the affinity score and dynamically generated tag.