A crawler is a bot that extract data from a webpage .

When we talk about SE crawlers like google, those are bots that explore your website and extract data to add it to their result database ....