A big data database managed by a procurement service aggregates purchase data received from federal, state and local government agencies through Freedom of Information Act requests, state public records requests and private sector business entities. An automated system processes a vast amount of purchase data files acquired from numerous different agencies through a number of different transports, on a variety of different media, included within several different file formats. A best match is selected for each acquired file with one of a multitude of configuration files available to process the purchase files. The file is then processed with the selected configuration file and aggregated into the database. The database is then made available to customers of the procurement service for search, reports and analysis purposes.