Last post 5 hours 11 min ago

Posts last week 81

Average response time last week 44 min

All time posts 70348

All time tickets 10857

All time avg. posts per day 20

Helpdesk is open from Monday through Friday CET

Please create an (free) account to post any question in the support area.

Please check the development versions area. Look at the changelog, maybe your specific problem has been resolved already!

All tickets are private and they cannot be viewed by anyone. We have made public only a few tickets that we found helpful, after removing private information from them.

#9122 – 4seo does not crawl my site

Posted in ‘4SEO’

This is a public ticket. Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Thursday, 13 October 2022 00:03 UTC

Jordan

I installed 4SEO on 2 of my sites

www.example.de ->OK

but www.example.de the 4 seo crawler shows always 0 pages.

PHP 7.4

SettingValue PHP Built On Linux frankjoomlaklon 5.10.0-17-amd64 #1 SMP Debian 5.10.136-1 (2022-08-13) x86_64 Database Type mysql Database Version 5.5.5-10.3.36-MariaDB-0+deb10u2 Database Collation latin1_swedish_ci Database Connection Collation utf8mb4_general_ci PHP Version 7.4.32 Web Server Apache/2.4.38 (Debian) WebServer to PHP Interface apache2handler Joomla! Version Joomla! 3.10.11 Stable [ Daraja ] 14-August-2022 14:18 GMT Joomla! Platform Version Joomla Platform 13.1.0 Stable [ Curiosity ] 24-Apr-2013 00:00 GMT User Agent Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:105.0) Gecko/20100101 Firefox/105.0

Thank You

Frank Jordan

Thursday, 13 October 2022 02:41 UTC

Jordan

solved.

It was a curl problem. But basically a wrong routing table for the container.

you can close it

But i suggest to include some useful error message if curl fails for the homepage

Thursday, 13 October 2022 08:44 UTC

wb_weeblr

It was a curl problem. But basically a wrong routing table for the container.

Glad you sorted it out. Unfortunately, we see this more often than not. Also DNS issues when there's a load balancer and the webservers cannot resolve their own domain names.

But i suggest to include some useful error message if curl fails for the homepage

It should actually, you should have the home page listed as "Broken link". I'll try and see why this is not happening.

Best regards

Yannick Gaultier

weeblr.com / @weeblr

Friday, 14 October 2022 10:11 UTC

Jordan

this issue might occur for many customers. I took me many hours to analyze it. Once identified via logfile and debugged via console it was easy to fix it via /etc/hosts

since my website was running stable for many years i did not expect configuration errors on my server.

Maybe you add some small sentices how 4SEO crawls the site: e.g. curl https://www.example.de from the console

I debugged 4seo logfile joomla logfiles apache logfile and finally the source code->curl ;-)

First I suspected ssl redirect issues (connection refused). but is was simple dns host lookup

Best greeting from germany

Frank

Friday, 14 October 2022 10:47 UTC

wb_weeblr

this issue might occur for many customers

Well, very few people know what a container is and I would say only a very small percentage of our users self-host, so it's not so common. The DNS issue I mentioned, we had only once in 18 months.

Most people will use shared hosting or managed hosting if they have higher needs. In such cases, our crawling issues are more with spam and bots filters.

Maybe you add some small sentices how 4SEO crawls the site: e.g. curl https://www.example.de from the console

We actually do not use curl specifically. We use the Joomla HTTP client, which uses whatever is present on the server. curl is preferred if present I think, but we do not chose that, although as it's the most common, we set curl options in some cases (to bypass certificates checks for instance).

Maybe you add some small sentices how 4SEO crawls the site: e.g. curl https://www.example.de from the console

When you see that 4SEO requests for pages timeout after 15000ms, it's the sign that either:

- DNS is wrong as in your case

- in 99% of cases, the server is blocking the requests at some point.

Glad your sorted it out anyway!

Closing this ticket now, feel free to open a new one as needed. If you do so, please mention this ticket number in the new one.

If you created any superadmin account for us, be sure to delete or block it now to avoid unnecessary risk in the future.

Best regards

Yannick Gaultier

weeblr.com / @weeblr

This ticket is closed, therefore read-only. You can no longer reply to it. If you need to provide more information, please open a new ticket and mention this ticket's number.