Lighthouse CLI tool version and user agent on Chrome can vary on version 2.92
The Lighthouse tool and a user agent on Chrome, Chrome canary & the Command line interface (CLI) will differ, and the outcome may differ somewhat. Until Lighthouse version 2.9.2, robots.txt files weren’t reviewed.
A local web developer from Stoneycroft was having some issue with Google Light House CLI
When they were using the lighthouse command-line interface and chrome lighthouse tool version Whitehouse 2.9.40.
The site was being blocked on the CLI but not on the Chrome extension
It said the page is blocked from indexing robots.txt but the chrome lighthouse tool which is 2.8 using an emulated Nexus 5x user account says it is fine and can be indexed.
After doing some research on the Google Lighthouse CLI it appears that version 2.8 of the crime extension did not check the robots.txt file. They weren’t checked until version 2.9.2 which is why they were seeing it pass in 2.8
