.Google confines robots.txt assistance to four industries, clarifying its posture on unsupported instructions.Google merely sustains four details robots.txt fields.In need of support instructions in robots.txt will be actually disregarded.Look at bookkeeping your robots.txt data because of this improve.