Several questions after migration

Tuesday, 24 November 2015 20:46 UTC

testeuse

Hi,

how are you? i would need your help please.

I am working on a site www.xxxx.fr, it has been hacked and i had to upgrade it from joomla 1.5 to joomla 3.4 as fast as i can.

I don't want to loose the seo i had so i have to be really fast. I already lost 35 000 links in google... i don't really know how much it will penalized me but i know this is not good :(

Anyway, all that to say, i really need your help because i am trying to have same urls just like the old one but there is always a little something...

My problems are :

1/ I used to have something called "avis" in them but no way to find what it was. I have no menu called avis.
i have reprodcuced the exact menu than before with exact categories...

2/ I used to have that url construction :

menu name/main category/ subcategory :

http://www.xxxx.fr/avis/les-produits-de-beaute/soins-du-visage/cremes-de-jour_c9_m220/

But now, i have : main category / menu name / subcategory

http://www.xxxx.fr/soins-du-visage/les-produits-de-beaute/cremes-de-jour_c9/

And anyway lol they are not the same.

3/ Do i really need to have the exact same urls than my other site? Can't the 301 redirection help me a little bit?
If yes, is it automaticall?

Please, help me lol

Regards,
Nadine

Wednesday, 25 November 2015 08:26 UTC

wb_weeblr

Hi

1 - Was sh404SEF running on that site before?

2 -

1/ I used to have something called "avis" in them but no way to find what it was. I have no menu called avis.

i have reprodcuced the exact menu than before with exact categories...

sh404SEF stricly do NOT use menu structure to build URLs. Menus just don't matter.

3/ Do i really need to have the exact same urls than my other site? Can't the 301 redirection help me a little bit?

Same URLs are ideal, 301 will have a small loss of ranking associated with them.

3 -

But now, i have : main category / menu name / subcategory

What type of content are we talking about? joomla articles?

Rgds

Wednesday, 25 November 2015 10:01 UTC

testeuse

1 - Was sh404SEF running on that site before?
Yes, site was on joomla 1.5 and i was using sh404sef 2.2.3.945 on it.

You can see old site here :
http://www.xxxx.fr/old/administrator
and new one :
http://www.xxxx.fr/administrator

Access codes for both : xxxx/xxxx

2 - But now, i have : main category / menu name / subcategory

What type of content are we talking about? joomla articles?

I am using jreviews on that site, so i don't know if it is jreviews or joomla but it is articles.

Before :

http://www.xxxx.fr/old/avis/les-produits-de-beaute/soins-du-visage/cremes-de-jour_c9_m220/

Now :

http://www.xxxx.fr/soins-du-visage/les-produits-de-beaute/cremes-de-jour_c9/

I don't know if i answered but i hope, it will help to find a solution.

Regards,
Nadine

Wednesday, 25 November 2015 11:34 UTC

wb_weeblr

Hi

I am using jreviews on that site, so i don't know if it is jreviews or joomla but it is articles.

Nope, it's JReviews content, not Joomla standard content.

To be honest, I don't really know how to handle this properly. I mean how to achieve exactly the same URLs (why do you have /old/ in the URLs by the way??)

the "_m220" at the end looks like the menu item id, and that's totally up to JReviews. In addition, I doubt the menu item id is going to be the same on the new site. Likewise, I'm not sure the category id (c=9) is going to be the same id on the new site, at least for all categories.

So even if we find a way to build the same URLs as you can always have a programmer build a custom plugin for sh404SEF/Jreviews to achieve the same URLs, that's totally not going to be quick.

So the only solution is redirects. For 35000 URLs, this means either:

1 - you pick only the top 100 URLs and you create manual redirects in your .htaccess
or
2 - you create complex rules in your .htaccess that will compute the new URL based on the old URLs (tricky)

1 is your best option, but maybe there's a #3 you can try: enable "Try to autoredirect 404s" in sh404SEF configuration, under the "Error page" tab.
As new URLs seems to look like the old ones mostly, maybe sh404SEF can find the right redirect on most cases. You can surely do that together with #1, and that can be a quick solution.

Rgds

Wednesday, 25 November 2015 19:32 UTC

testeuse

Hi thank you for your answer and sorry for late one but i have been hacked just after your answer.

So i had to find a solution that i still don't really have.

Thank you for your advices and to answer to your question, i have old in my old links because the old site is in a folder called (old) in my new one.

If i only use automatic redirect with sh404sef would it be so bad?
I would like to do 1rst option and 3 but i don't know how to make redicrection manually.
Could you please give me an example?

About 3rd option, what cobfiguration should i use? the one already in it?

I am really sorry for those questions, i hope you can help me.

Regards,
Nadine

Thursday, 26 November 2015 14:12 UTC

wb_weeblr

Hi

I would say the first thing to do is establish a list of your top 100 or 200 most important URLs on your old site (by traffic, or sale, what you think make them important).

Then you can enable autoredirect in sh404SEF, and try the old URLs, starting with #1, #2,... and see if the results look good. If URLs are similar, it should find the correct one, but it's not guaranteed.

The config is just one setting:

enable "Try to autoredirect 404s" in sh404SEF configuration, under the "Error page" tab.

Note that for this to work, the new URLs must be in the database. Meaning that your new site pages must have been crawled or visited enough so that all or most URLs have been created.

To do solution #1, you can simply use sh404SEF and add redirects - called aliases in sh404SEF (see documentation). Pretty tedious, but it'll work.

You can also do it in .htaccess; it's faster, but more technical. There are good tutorials on the web for that.

Rgds

Wednesday, 02 December 2015 19:51 UTC

testeuse

Hi,

Thank you for your complete answer and sorry for late answer but i had to deal with the hacked problems and a new host.

I will try to do the htaccess solution with the help of a friend, i hope we will make it.

I am sorry but i still have questions, i hope fast ones :

1/ On my new host they had to change htaccess to block google bots because they were too many at same time so they crash the server :
Investigating this further it seems that the issue is caused by the fact that the server uses the allocated CPU and Memory resources due to the many simultaneous connections at the moment. Which is why it returns the given error message.

Checking this further it seems that most of those processes are actually coming from the Google bots which are making a new constantly and currently there 110 request towards the domain xxxx.fr coming from the Google IP address xxxx

2/ I also had that mail from google :

Increased pages "soft 404" on http://www.xxxx.fr/

To: http://www.xxxx.fr/ Webmaster,

Googlebot has identified a significant increase in the number of URLs that should display on http://www.xxxx.fr/ a 404 (page not found), but does not currently do. This can interfere with the navigation of Internet users. In seeking to access a particular page, they may indeed happen elsewhere on your website.

Fix the problem

Check the settings of your CMS and your server to make sure these URLs display an error code 404 (Page Not Found) or 410 (Page deleted) in response to requests existent pages.

Could you please tell me what it means and what i can do fo that.

3/ Last thing is about url construction, because before redirection, i have to be sure of my new urls.
i would like to know if i could have that kind of url construction :
http://www.xxxx.fr/Directory/Category/Subcategroy_c156/
But i have that :
http://www.xxxx.fr/Category/Directory/Subcategroy_c156/

I mean, is it possible to change the order of urls content directly in sh404sef admin? If not, it will be ok.

I am sorry for those questions, i hope you will be able to advice me, if not, don't worry, it will be ok.

Thank you for your help and sorry for all those questions.

Regards,
Nadine

Thursday, 03 December 2015 12:56 UTC

wb_weeblr

Hi

Checking this further it seems that most of those processes are actually coming from the Google bots which are making a new constantly and currently there 110 request towards the domain xxxx.fr coming from the Google IP address xxxx

In how much time did this bot did 110 requests? if they do it in a day, 110 requests is very small, if they do it 1 minute, it's very high.

There's no reason GoogleBot would crawl your site very fast. They pay a lot of attention to that and always crawl the site carefully. I have never heard of anything like this to be honest.

What hosting company are you using? is it well known and a serious one?

Increased pages "soft 404" on http://www.xxxx.fr/

That's more worrying. It means they try to load some pages on your site, and the page returns a success (code 200), but when they look at your page content, it looks like it's a 404 (for instance it has almost no content, or there's 404 written on it, etc).
This may happen if you have a plugin to redirect all errors to the home page for instance (a very, very bad thing to do).
Do they give an example of one of those URLs?

http://www.xxxx.fr/Directory/Category/Subcategroy_c156/

But i have that :

http://www.xxxx.fr/Category/Directory/Subcategroy_c156/

If this is still the JReviews URLs, we have little control on them, so no, I don't think so.

At least automatically. You can always change them manually, in the URL manager of course.

Rgds

Friday, 04 December 2015 10:43 UTC

testeuse

Hi,

Thank you for your answer.

1/ In how much time did this bot did 110 requests? if they do it in a day, 110 requests is very small, if they do it 1 minute, it's very high.

What hosting company are you using? is it well known and a serious one?

They said, i had those request by minutes that's why the server crashed.
The hosting is siteground.

2/ Increased pages "soft 404" on http://www.xxxx.fr/

That's more worrying. It means they try to load some pages on your site, and the page returns a success (code 200), but when they look at your page content, it looks like it's a 404 (for instance it has almost no content, or there's 404 written on it, etc).

This may happen if you have a plugin to redirect all errors to the home page for instance (a very, very bad thing to do).

Do they give an example of one of those URLs?

I am not using anything else than sh404sef for seo or redirection...
I have added a screencapture of my configuration for sh404 redirections maybe i did something bad.

Regards,
Nadine

Friday, 04 December 2015 11:27 UTC

wb_weeblr

Hi

1 - Siteground is good, there's no problem here. I don't see why google would request that.

2 - I don't see either that sh404SEF would do anything like what I explained.

Do they give an example of one of those URLs?

That's a key point.

3 - However at the moment, http://www.xxxx.fr/ is offline and replies with a 503 Under maintenance code. How long have you been offline? That could certainly have an impact.

Rgds

Friday, 04 December 2015 16:23 UTC

testeuse

Hi,

I am offline since a month right now i have already lost 100 000 links.

i have attach to the mail the soft 404 urls that google show me.

Regards,
Nadine

Friday, 04 December 2015 16:29 UTC

wb_weeblr

Hi

But if you've been offline for a month, that's probably the reason of those message. Offline is ok for a few days, but not for weeks. After sometimes, they start thinking all those URLs are gone.

What happens if you try to load one of those URL yourself (for instance: pinceau-blush-les-merveilleuses-laduree.html).

Does it render correctly?
Does it have content?

But to be honest, I think the problem is being offline.

Rgds

Friday, 04 December 2015 17:42 UTC

testeuse

Hi,

Thank you for fast answer.

1/ pinceau-blush-les-merveilleuses-laduree.html

Did not work, it redirects me to another page. I tried several times still redirect me to another page.
Then i use the search and found the page and now it works.

The url that works : http://www.xxxx.fr/pinceau-blush-les-merveilleuses-laduree.html

I already had this problem for pages that exists but when i search urls directly it redirect me automatically.

2/ About offline, do you think i should go online even if i still did not do all my redirections in htaccess file?
Is it better to be online than not even without .
Maybe i could do redirections later or while online?

Regards,
Nadine

Friday, 04 December 2015 17:45 UTC

wb_weeblr

Hi

1 -

I tried several times still redirect me to another page.

It redirects you to which page? exactly?

2 -

2/ About offline, do you think i should go online even if i still did not do all my redirections in htaccess file?

Is it better to be online than not even without .

Maybe i could do redirections later or while online?

I have no authoritative opinion on that,but I think a site that's been offline for a month is a little bit like a disappeared business for Google.

Rgds

Friday, 04 December 2015 18:00 UTC

testeuse

1/ It redirects me to a similar page that i did not keep.

I tried again with those urls :

rouge-a-levres-finishing-lipstick-fashion-fair.html
redirect me to that page :
http://www.xxxx.fr/rouge-a-levres-bio-ainhoa.html

For this one, the url has changed cause there is no more (re) for (regossip) in kunena url :
forum/series/718-regossip-girl-sur-t1.html
and it redirect me to that page :
http://www.xxxx.fr/forum/series.html

2/ Ok, thank you for telling me that anyway, i am going to do my best to make it online tonight or tomorrow.
How could i ask robots to slow the crawl please so my server not crash?

Regards,
Nadine

Friday, 04 December 2015 18:06 UTC

wb_weeblr

Hi

1 - OK, the problem is not that you are "redirected". The problem is that this URL doesn't exists in the database yet (because no link to that page has ever been displayed). When you do a search, Joomla creates the link and pass it on to sh404SEF, and so the URL is stored in the database. Which is why the auto-redirect stops.
Before that, a 404 is triggered, but because you have enabled the "Try to auto-redirect 404s" feature, well, you get redirected to a page with a similar URL.

I think those URL don't exist because you don't have any traffic on the site, nobody display pages and so no or few URLs are created.

2 -The usual problem is the opposite: Google bot not coming often enough (and there's little you can do about it).

I would look through Search Console (previously Google Webmaster tools). See this page: https://support.google.com/webmasters/answer/48620?hl=en

Rgds

Friday, 04 December 2015 18:16 UTC

testeuse

1/ Ok, so do i have to change something or should i click on every menu link before making it online?

Before that, a 404 is triggered, but because you have enabled the "Try to auto-redirect 404s" feature, well, you get redirected to a page with a similar URL.

Is it bad?

2/ I already tried, but it said i can't change it because it is already set, no way to make a change.
I wanted to make it lower because i had the too much bot at same time problem.

Regards,
Nadine

Friday, 04 December 2015 18:19 UTC

wb_weeblr

Hi

1 - No, it's by having visitors that links are created

It's not bad or good, it's a choice you make. If you leave that on, I would advise that you enable the "Display message" option, so that visitors are informed they have been redirected to another page

2 - That's strange, but there's little advice I can give on top of that. I'm surprised about it, but it's not something I have used really.

Rgds

Friday, 04 December 2015 19:05 UTC

testeuse

Ok, thank you for your time and advices.

I will try to put it online and see what will happen lol

Thank you again.

Regards,
Nadine

Saturday, 05 December 2015 00:45 UTC

testeuse

Hi,

I a sorry, i have another question, please.

I was watching 404 urls from sh404sef, new ones for today as i empty the folder several times today.

I found in it a lot of strange urls, from ebay, apple and wordpress.
When i was hacked i had false folders of ebay and apple in my ftp, does it mean it is still hacked or those files doesn't exist anymore so they goes in 404 errors.

How could i see the non sef urls for those links please, if it exists.

Here is some examples :

Strange urls

///////////////////////
Old hack urls
/////////////////////////////

apple-touch-icon-precomposed.png

apple-touch-icon.png

ebay

ebay/

ebay/51505e52a3a251b170a3d457ae2f96ac

ebay/6a0d2bd96e436f69757c21172db61526/eBayISAPI.php

ebay/cb1e6cc34f9c830653577e1b012f88e3/

ebay/cb1e6cc34f9c830653577e1b012f88e3/eBayISAPI.php

ebay/eaa585b2f2d083ba3a6e564622337376/eBayISAPI.php

///////////////////////
Wordpress urls??
//////////////////////////////

blog/wp-admin/

old/wp-admin/

test/wp-admin/

web/ui/page/error.html

wordpress/wp-admin/

wp-admin/

wp-login.php

wp/wp-admin/

////////////////////////////////////////
My old template
////////////////////////////////////////////////////

templates/stylecouncil/js/xtcMenu.js

templates/stylecouncil/XTC/cssproc.php

test/wp-admin/

web/ui/page/error.html

wordpress/wp-admin/
wp-admin/
wp-login.php
wp/wp-admin/

/////////////////////////////
Wrong site name but real articles names, it is xxxxS not xxxx...
////////////////////////////////////////////////////////////////////////////

~xxxx/ds-creation-la-maison-du-tissage-paris-10eme.html

~xxxx/forum/films/640-votre-vos-derniers-films-vus-au-cinema.html

~xxxx/forum/soins-des-cheveux/1466-croissance-des-cheveux.html

~xxxx/forum/soins-des-cheveux/15236-les-colorations-pour-cheveux-naturels-ou-

defrises.html

Thank you for your help.

Regards,
Nadine

Monday, 07 December 2015 14:20 UTC

wb_weeblr

Hi

Those are requests to your site made by others, usually hacking attempts. If you see them as 404, then all is good: the requests have failed and a 404 was returned to the attackers.
You would start having a problem if those requests got a 200 response from your web server, meaning somehow they had been executed.

#1436 – Several questions after migration