Design of iMacros-based Data Crawler and the Behavioral Analysis of Facebook Users
This work addresses the problem for researchers needing unrestricted data from Facebook, but it is incremental as it builds on existing web crawling technologies.
The researchers tackled the challenge of obtaining datasets from Online Social Networks by designing IMcrawler, an iMacros-based data crawler that collects publicly accessible information from Facebook, extracting personal information and wall activities from user profiles.
Obtaining the desired dataset is still a prime challenge faced by researchers while analyzing Online Social Network (OSN) sites. Application Programming Interfaces (APIs) provided by OSN service providers for retrieving data impose several unavoidable restrictions which make it difficult to get a desirable dataset. In this paper, we present an iMacros technology-based data crawler called IMcrawler, capable of collecting every piece of information which is accessible through a browser from the Facebook website within the legal framework which permits access to publicly shared user content on OSNs. The proposed crawler addresses most of the challenges allied with web data extraction approaches and most of the APIs provided by OSN service providers. Two broad sections have been extracted from Facebook user profiles, namely, Personal Information and Wall Activities. The present work is the first attempt towards providing the detailed description of crawler design for the Facebook website.