Fix bug in robots rule with .git suffix

What does this MR do?

In this MR we change the Robots.txt to forbid routes ending in .git but not namespace nor projects with the word .git on them.

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Separation of EE specific content

Related to #36837 (closed)