Combining Machine Learning and Crowdsourcing for Better Understanding Commodity Reviews

In e-commerce systems, customer reviews are important in-formation for understanding market feedbacks on certain commodities. However, accurate analyzing reviews is chal-lenging due to the complexity of natural language process-ing and informal descriptions in reviews. Existing methods mainly focus on studying efficient algorithms that cannot guarantee the accuracy for review analysis. Crowdsourcing can improve the accuracy of review analysis while it is sub-ject to extra costs and low response time. In this work, we combine machine learning and crowdsourcing together for better understanding customer reviews. First, we collectively use multiple machine learning algorithms to pre-process re-view classification. Second, we select the reviews on which all machine learning algorithms cannot agree and assign them to humans to process. Third, the results from machine learn-ing and crowdsourcing are aggregated to be the final analysis results. Finally, we perform real experiments with practical review data to confirm the effectiveness of our method