Hi
Upon further inspection of the table, we noticed that it has 800,000+ records.
That is totally fine and not excessive at all. This table contains both valid URLs and 404s requests.
Why is the software generating so many records?
sh404SEF does not generate any single records. It only records those generated by Joomla and your site.
It's causing sh404 queries to run slow, the website loading performance slow
sh404SEF runs very fast, even with multiple times the number of records you have in the urls table.
SELECT `oldurl`, `newurl`, `id`, `rank`, `dateadd`, soundex(`oldurl`) AS soundex_value FROM `sh404sef_urls`
This query is a very special one, that does not happen on normal pages. It happens on 404s. When a 404 happens, the "Similar URLs" plugin search the database for similar URLs and offers them to your visitors faced with a 404 to try keep them on the site.
Now this request is very heavy because it does not just search for exact matches but also for words that sound similar. If your server cannot cope with this workoad, then best is to simply disable the "Similar URL" plugin. You'll likely have to change also the text of your 404 error page (under "Error page" in sh404SEF configuration) to your preferred message.
Best regards
Yannick Gaultier
weeblr.com / @weeblr