Processing Billions of RDF Triples on a Single Machine using Streaming and Sorting

We consider the feasibility of processing billions of RDF triples on a single commodity machine using streaming and sorting techniques and focusing on RDF processing tasks relevant for Linked Data consumption: data filtering and transformation, RDFS inference, owl:sameAs smushing and statistics extraction. To investigate this research question we built rdfpro (RDF Processor), an open source tool that provides streaming and sorting-based processors for the considered tasks and allows their sequential and parallel composition in complex pipelines. An empirical evaluation of rdfpro in four application scenario—dataset analysis, filtering, merging and massaging—shows the effectiveness of the tool and allows to positively answer our research question.