Main reasons we can't crawl

There are a few reasons why crawling your website might be an issue and therefore we would not be able to list your stock on our search engine. 

The main reasons why we would not be able to crawl are listed below:

All stock information is listed on one page 

This means there is no specific advert pages for each car and therefore it will make it difficult for when a buyer clicks on your Carsnip advert as they will not be directed to that specific car.

All stock is hosted by a different domain

If your car adverts are not hosted by your own domain, then this might become a problem depending which domain they are redirected too. For example, if your adverts link to the official car make site (e.g. Volvo website or Audi website) then this would mean we would have to grab all the stock from that site and cannot select your own cars. Also, if your stock is redirected to other usd car sites, this means we would be funnelling traffic elsewhere and our purpose is to send users to your own website.

All information is loaded via JavaScript

There can be multiple reasons why JavaScript can prevent us from crawling a site:

• Page Listings are loaded with JavaScript
• Advert Page Listings are loaded with JavaScript
• Whole websites are delivered by JavaScript
• Infinite Scroll with JavaScript on page listings 

Other reasons:

• Dynamic pagination via PHP/AJAX.
• Having a high amount of sold cars mixed in with available cars; this can be resolved if there was some form of indication that the car is sold on the car advert itself, making it an obvious indication that the car is no longer available (i.e. adding the word 'sold' anywhere on the text of the advert). However, it would not be ideal to fill our search engine with multiple sold items. 
• Inconsistent layout of adverts. 
• Make sure your robots.txt is not blocking access to crawl - if you have given permission for us to crawl your website then you will need to make sure your robots.txt will allow this to happen, otherwise this will cause an issue. 
• These reasons do not mean that we want to advise people to not use JavaScript, however we just want to explain the reasons why it may be problematic.

There are ways to resolve the problems listed above and if you needed to help to discuss, please don't hesitate to send us an email on support@carsnip.com