To perform most of its duties, 4SEO needs to know the content and characteristics of your website pages. It gets this information by crawling your site in the background and analyzing all pages it can discover, in very much the same way search engines will do so.
Being an extension running inside your site, it does have advantages over Google and Bing in that it can better understand and identify content and metadata.
There are 2 ways to run the analysis, and you can use them together:
- background site analysis
- from the admin
This is the main and normal way of analyzing your site content, and it's happening automatically as visitors are viewing your pages.
When a visitor loads any page on your site, a small invisible image link is added to the page and will later, in the background, trigger 4SEO's crawler. The visitor will not see or notice anything, and the page performance is not affected.
The crawler will then pick one of the pages pending analysis, load it and analyze its content. As all this process is triggered by that small image, it does not slow down the site page rendering or performance and allows for a thorough analysis.
If the page is deemed important, information about it is saved to the database and will be re-used later in various way.
Once that page has been analyzed, the crawler will stop and will only be triggered again when the same or another visitor loads another page. The same process will then repeat.
Only Public pages will be crawled and analyzed. Any page that requires logging-in will not be processed.
As the base analysis process relies on having visitors on the site, it won't be enough when you don't have any visitors just yet. For instance before launch, or if there's little traffic on the site and analyzing all pages takes too much time.
In that case, use the backend crawler controls. Click on the
Analyze toolbar button of the Pages page. This opens up the crawler controls:
Run analysis now button starts looking for pages and analyze them. The counters will update as it progresses.
While the analysis is running, you can:
- slow down or speed up the crawler by sliding the
Slower/Fasterslider. This lets you control the workload on your server.
- use 4SEO menu to navigate to any other page of 4SEO. The analysis will keep going, as long as you do not navigate to another Joomla page or reload the page
- Click the
Run in backgroundbutton: this will stop live crawling and return to the normal analysis mode, triggered by visitors on the front end.
You can entirely disable site analysis from the Settings toolbar button of the Pages page. This can be useful if you do not add any content to your site anymore for instance.